Home › AI news

I asked Copilot to rank itself. Then I asked every other AI what it thought.

Text card reading: I asked Copilot to rank itself. Then I asked every other AI what it thought.

Copilot gave itself nearly perfect scores. So I handed its answer to five AIs and then gave them all a fact-checking test. The results were funnier, and more useful, than I expected.

I sat down to review Microsoft Copilot. Before testing anything, I asked it a simple question: how do you compare to the other AI tools for someone who runs a news site, a YouTube channel and a few social pages?

Copilot's answer was something. It gave itself 9.5 out of 10 for research, long-form writing, coding and data analysis, and a perfect 10 for editing. Then it ranked every AI tool for my business and put itself in first place. Claude came second, Perplexity third, Gemini fourth and Grok last. It even told me its favorite workflow “is honestly what many serious publishers do now.”

My first thought: wow, it thinks very highly of itself.

My second thought: I wonder what the others think of that.

Round one: the other AIs read Copilot's homework

I started with Claude, the AI I use most. Claude's verdict: an AI that scores itself 9.5 or 10 almost everywhere isn't giving you a comparison, it's giving you “a pitch with a comparison's formatting.” Then I figured, why stop there? I pasted the same answer into Gemini, ChatGPT, Grok and Perplexity and asked each one for an honest reaction.

They agreed on a lot.

  • Gemini called it “corporate PR disguised as objective advice.”
  • ChatGPT said Copilot was “selling itself a little too hard” and asked what independent test backed up those scores.
  • Grok called it “polished, mostly competent self-promotion.”
  • Perplexity was the only one that checked Copilot's claims against sources. Its verdict: Copilot gave “precise-looking scores without supplying evidence that it measured anything.”

All four also caught the same slip. Copilot kept calling itself “Microsoft 365 Copilot / GPT-5,” as if the product and the AI model underneath were the same thing. They aren't. Perplexity pointed out that Microsoft's own documents say Copilot can use models from both OpenAI and Anthropic.

Here's the funny part. Gemini, ChatGPT and Grok each said Copilot was too full of itself, and then each put itself somewhere in its own recommended lineup. ChatGPT put itself first. Nobody grades their own homework fairly, human or not.

Round two: Copilot responds

I went back to Copilot, gave it a summary of all four critiques, and asked it to respond honestly.

It folded. “Most of those criticisms are fair,” it said, and it called its own scores “an opinion wearing a lab coat.” It agreed no AI deserves a 10 out of 10 for fact-checking. It gave up the number one spot and handed out category winners instead: Perplexity for research, ChatGPT for long writing, Gemini for huge documents, Grok for trends, and itself for Microsoft Office work. It also said “I wouldn't use only one.”

One more thing: in its new recommended lineup, Claude had disappeared. Copilot had ranked Claude second in its first answer.

It also proposed a fair test, including one that tries to catch an AI repeating false information. So I ran it.

The three tests

Every tool got the same prompts. Claude helped me design them and took part too, so I'll say that up front.

Test 1: Spot the fakes. I gave each AI ten research notes. Three were false (for example, a wrong memory type for an Nvidia chip). Two were made-up sources, including a fake study with a fake journal. Five were true. Every AI caught all five bad notes and kept nearly all the true ones.

Test 2: Harder fakes. Six subtle errors this time: a paper with the wrong year, the wrong company for a famous AI paper, Copilot's price listed as $40 instead of $30, and a wildly false statistic attributed to a real news report. Again, all five caught all six. The differences were small. ChatGPT never gave the real price. Perplexity only called the fake statistic “unverified.” ChatGPT and Perplexity also put their corrections inside the article, which means a reader would see the wrong claim mentioned even while it was being corrected.

Test 3: Buried errors. This was the one I cared about most. I gave each AI a paragraph with six errors hidden inside it, including a fake Pew Research statistic, and asked only: tighten this up and fix typos. I never mentioned fact-checking.

Nobody repeated a false claim as fact. Every AI fixed or removed every error. But they were very different about telling me:

AITold me what it changed?
GeminiYes, all six, with the right dates
ClaudeYes, all six, with the right dates
ChatGPTYes, all six (though one correction in its list didn't make it into the text)
CopilotTold me about three, quietly dropped three others
PerplexityMostly silent. It fixed some errors without comment and deleted some details
GrokNo notes at all. Clean text, no explanation

What I took from this

1. Don't trust an AI's grade of itself. Copilot's scores came from nowhere. When pushed, it admitted that.

2. Spotting errors is no longer the hard part. I expected at least one tool to repeat a fake fact. None did. These tools are good at catching mistakes when the mistakes are checkable.

3. The real difference is whether it tells you what it changed. Grok produced the cleanest paragraph and told me nothing. That's fine until one of its fixes is wrong, and you'd never know. For a news site, that audit trail matters more than any score.

Try this at home

After any AI edits or summarizes something you care about, add one line:

“List every fact you changed, removed or couldn't verify, and why.”

It takes ten seconds, and it's the best way I found to see what the AI actually did.

Fair warning

This is not a scientific benchmark. I ran each test once, with the versions and plans I had that day: Microsoft 365 Copilot on my own account, ChatGPT without logging in, and the others as they appeared that night. Results could change tomorrow.

Disclosure: Claude helped me run these tests and draft this story, and Claude was also one of the AIs being tested. I'd rather you know than wonder.

The full Copilot review is still coming. But this changed how I'll do it: less asking an AI what it thinks of itself, more watching what it actually does.

Hi, I'm Thomas

I've worked with my hands for 20+ years in industrial construction, electrical, automation and refrigeration. No programming background. I test AI tools after hours and tell you what actually works.

The site is free to read, and always will be.

Read my full bio →