Menu
Sign In Search Podcasts Libraries Charts People & Topics Add Podcast API Blog Pricing

Marco Arment

πŸ‘€ Speaker
8819 total appearances

Appearances Over Time

Podcast Appearances

Accidental Tech Podcast
678: Mentoring a Box of Numbers

And then one or more anonymous people wrote in with regard to AI and productivity and the promise of, oh, all of these people will 10x their productivity versus scientific studies about it.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

So again, I'm just going to kind of hit the high level real fast.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

The scientific evidence doesn't actually back the idea that AI provides a huge productivity boost.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

A recent study from Anthropic found that for junior developers on a set task, the time taken to complete a task was not statistically significantly faster, just two minutes, than those who did not use AI tools, and their understanding of the output and ability to answer questions on it was actually much worse.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

I'll put a link in the show notes.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

Separately, a randomized controlled trial from mid-2025 found that experienced developers were slower with AI.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

Separately, a Microsoft study from 2024 found that for trivial tasks, the users were able to easily describe AI could save some time.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

But for complex tasks, it was of minimal benefit.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

So trivial tasks that you can describe to AI quickly, yeah, you save time.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

But anything kind of complex, not so great.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

Then a survey from Atlassian found that only 4% of companies are seeing return on investment on their integration of AI tools.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

Separately, a study from Berkeley at the California Review Management found that there is no statistically significant relationship between AI adoption and productivity gains.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

At Harvard Business Review, there's a study that describes how AI-generated slop is actually harming productivity, even when it seems to be producing work.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

Think an increase in the number of PRs to an open-source project that in actuality just wastes the maintainer's time.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

Then finally, MIT's state of AI in business finds that 95% of companies are seeing zero returns.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

Cool, cool, cool, cool, cool, cool, cool, cool.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

John, I am happy to go through these links that you compiled, or would you prefer to?

Accidental Tech Podcast
678: Mentoring a Box of Numbers

How would you like to proceed?

Accidental Tech Podcast
678: Mentoring a Box of Numbers

You can.

Accidental Tech Podcast
678: Mentoring a Box of Numbers

It's just some more in the same vein.