Nathaniel Whittemore

But even if we suppose that Claude never achieves good research taste, a conservative reading of our evidence still implies compounding acceleration.

Now the meat of the piece, and the part that's generating the most discussion, is the last section on three possible futures.

The first possible future, which they say they include for completeness but don't believe it's likely, is one in which the trend stalls but today's AI capabilities are widely diffused.

They write, This article features many exponential trajectories, but these trajectories may actually turn out to be S-curves.

We may be approaching the bend in the curve, where returns to scale diminish and the line straightens then flattens.

The judgment that separates a competent researcher from a great one might be a capability that cannot come from scaling up training inputs like compute and data.

If so, getting past this bottleneck would require a new idea, like an architectural approach that supplants the transformer architecture that all current frontier models use.

Alternatively, and this is my editorialization but I think we're seeing lots of evidence of right now, the binding constraint to AI progress could be in the supply chain, not the model.

Advancing and defusing the frontier may require more energy and compute than presently exists.

The pace of chip fabrication, grid expansion, or interconnect bandwidth may be the constraint rather than the intelligence itself.

Now still they say, even if model capabilities were frozen at today's level, we would expect major changes to occur in the world.

They point to the example of Mythos Preview finding more than 10,000 high and critical severity software vulnerabilities across many of the world's most important systems.

Still, like I said, they don't believe that this scenario is particularly likely.

Every capability we can measure, they write, has so far followed the same curve.

We've not yet seen that curve bend.

Of the three futures we consider, this one would give governments and societies the most time to adapt.

We are more worried they continue about the next two, which would move faster and leave far less room for preparation.

Scenario two, then, is the AI labs continuing to see compounding efficiency gains.

In this scenario, they say, AI development becomes substantially automated, but humans continue to set research directions and judge results.

In this scenario, 100-person companies could do the work of 10,000 or 100,000-person organizations.