Rob Wiblin

Final witness of this report is that while the Fortune article that launched this study into the mainstream claimed that the research was based on 150 interviews with leaders and a survey of 350 employees, the paper itself reveals that it's really based on 52 interviews and a paper survey of another 153 people.

Why did Fortune inflate the sample size by 2-3 fold, prompt dozens of other news outlets to do the same, and never correct the record?

I'll discuss my theory for that next.

But the small sample size really matters because it means the uncertainty intervals on their results are actually huge.

The finding that 5% of companies have successful custom AI projects, that appears to be based on just those 52 interviews.

That means that that 5% number is probably based on something like two or three actual successful companies out of 52.

Flip one or two interviews the other way and you'd get a completely different headline.

The true rate could easily be three times higher or three times lower.

You just can't tell from a sample this tiny.

Not that you'd learn that by reading the report or anything journalists wrote when covering it for that matter.

But things get worse.

In this case, not only was this paper not peer reviewed or scrutinized by external experts, the article wasn't even available to journalists around the world trying to cover the story so they could describe its claims precisely and think through whether they actually made sense.

In fact, when that fortune story went mega viral, the report describing the method and results wasn't publicly available anywhere.

The link to the underlying report, I kid you not, took you to a Google form where you could specifically request the PDF by giving them your personal details and explaining why you were interested in their organization and how you plan to use the agentic web.

It still takes you to that form even now.

The bottom line is that this survey went viral around the world, moved markets, and was already on track to hugely shape AI discourse, telling many people something they desperately wanted to believe before the media or the public or policymakers could read it and see that its headline result might be based on not even a handful of custom AI deployments.

That's media incentives for you.

So if it's not a great or even a good piece of research, why did it become one of the most widely cited things written about AI in 2025?

Surely a big part of the explanation has to be that it has the MIT label on it.

Like the Fortune article that made it famous, everyone, including me, came to refer to it just as the MIT study.