Marco Arment
π€ SpeakerAppearances Over Time
Podcast Appearances
And then one or more anonymous people wrote in with regard to AI and productivity and the promise of, oh, all of these people will 10x their productivity versus scientific studies about it.
So again, I'm just going to kind of hit the high level real fast.
The scientific evidence doesn't actually back the idea that AI provides a huge productivity boost.
A recent study from Anthropic found that for junior developers on a set task, the time taken to complete a task was not statistically significantly faster, just two minutes, than those who did not use AI tools, and their understanding of the output and ability to answer questions on it was actually much worse.
I'll put a link in the show notes.
Separately, a randomized controlled trial from mid-2025 found that experienced developers were slower with AI.
Separately, a Microsoft study from 2024 found that for trivial tasks, the users were able to easily describe AI could save some time.
But for complex tasks, it was of minimal benefit.
So trivial tasks that you can describe to AI quickly, yeah, you save time.
But anything kind of complex, not so great.
Then a survey from Atlassian found that only 4% of companies are seeing return on investment on their integration of AI tools.
Separately, a study from Berkeley at the California Review Management found that there is no statistically significant relationship between AI adoption and productivity gains.
At Harvard Business Review, there's a study that describes how AI-generated slop is actually harming productivity, even when it seems to be producing work.
Think an increase in the number of PRs to an open-source project that in actuality just wastes the maintainer's time.
Then finally, MIT's state of AI in business finds that 95% of companies are seeing zero returns.
Cool, cool, cool, cool, cool, cool, cool, cool.
John, I am happy to go through these links that you compiled, or would you prefer to?
How would you like to proceed?
It's just some more in the same vein.