Nathaniel Whittemore

They say this would revolutionize knowledge work in government services, but could also be termed to harmful ends, from authoritarian surveillance of whole populations to influence operations that tailor manipulation to each individual and run at a scale no human team could match.

Now, interestingly, and this is where I wish there was a bit more of a discussion, they write that while this is the scenario that is most likely based on the evidence that they've seen, they also note, speeding up one part of a process often just shifts the bottleneck elsewhere.

Overall, pace is capped by the parts that haven't sped up.

In computing, they write, this is known as Amdahl's Law, and the same logic can apply to organization.

Anthropic has already encountered one signature of Amdahl's Law.

As we've begun to push more code around the organization, human code review has become a new bottleneck.

We've also encountered this friction outside engineering.

There's been an explosion of new ideas, initiatives, tools, and simulations as a result of anthropic employees working with highly capable models.

Far more than we have the capacity to pursue.

The rate at which organizations can spot and fix these bottlenecks may be a skill that improves over time, and it may become the most important skill for any organization.

This gets into a lot of the ideas that I've explored around the infinite backlog, and why all of a sudden I don't think we're not going to have jobs.

Aaron Levy from Box commented on this part, saying that it points to the key element of the optimistic scenario for AI.

AI, he writes, lowers the barrier dramatically to allowing us to do more.

As a result of that, we have far more ideas than we can pursue, and the ones that we want to pursue were ultimately limited by our ability to go take on the surrounding work to execute those ideas.

There's almost no amount of AI progress that can happen where that goes away.

AI is going to let us build much more software, launch more marketing campaigns, research more drugs, and so on.

All of this work, even when augmented by agents, still ultimately requires people to manage.

Now back to Anthropic, the third scenario they point to is the full recursive self-improvement scenario, where AI systems start to build their own successors.

Now this scenario is where you see the most hand waviness from Anthropic, with them just not really knowing how to guess at all the implications of this.

The final section is the one that has been jumped all over, especially by AI safety advocates.