Nathaniel Whittemore

Alternatively, and this is my editorialization but I think we're seeing lots of evidence of right now, the binding constraint to AI progress could be in the supply chain, not the model.

Advancing and defusing the frontier may require more energy and compute than presently exists.

The pace of chip fabrication, grid expansion, or interconnect bandwidth may be the constraint rather than the intelligence itself.

Now still they say, even if model capabilities were frozen at today's level, we would expect major changes to occur in the world.

They point to the example of Mythos Preview finding more than 10,000 high and critical severity software vulnerabilities across many of the world's most important systems.

Still, like I said, they don't believe that this scenario is particularly likely.

Every capability we can measure, they write, has so far followed the same curve.

We've not yet seen that curve bend.

Of the three futures we consider, this one would give governments and societies the most time to adapt.

We are more worried they continue about the next two, which would move faster and leave far less room for preparation.

Scenario two, then, is the AI labs continuing to see compounding efficiency gains.

In this scenario, they say, AI development becomes substantially automated, but humans continue to set research directions and judge results.

In this scenario, 100-person companies could do the work of 10,000 or 100,000-person organizations.

They say this would revolutionize knowledge work in government services, but could also be termed to harmful ends, from authoritarian surveillance of whole populations to influence operations that tailor manipulation to each individual and run at a scale no human team could match.

Now, interestingly, and this is where I wish there was a bit more of a discussion, they write that while this is the scenario that is most likely based on the evidence that they've seen, they also note, speeding up one part of a process often just shifts the bottleneck elsewhere.

Overall, pace is capped by the parts that haven't sped up.

In computing, they write, this is known as Amdahl's Law, and the same logic can apply to organization.

Anthropic has already encountered one signature of Amdahl's Law.

As we've begun to push more code around the organization, human code review has become a new bottleneck.

We've also encountered this friction outside engineering.