Cal Newport

Because LLMs are very good at understanding and producing code.

The type of tools that a coding harness has to access in order to, you know, execute things are simple and text-based.

Like this is like a perfect case market.

to build the first killer apps on top of LLMs.

And starting last year and then really picking up speed last fall, the major players really got involved in these harnesses.

And so I think that's what these charts are all showing is like, yeah, these harnesses make us, they're getting really good at the, we can suddenly do a lot of programming tasks we couldn't before.

But does that mean that recursive self-improvement is imminent?

It does not.

And there's two reasons why.

And I want to look at these one by one.

Point number one, faster software development doesn't equal smarter AI, right?

All right, so these tools help computer programmers produce code faster or find mistakes or issues in existing code or systems faster as well.

These capabilities, though useful and perhaps a good source of revenue for these companies, doesn't add up to AI being able to improve themselves, to create AI systems that are much smarter than what humans would have otherwise been able to produce.

Now, why is this?

Because

The bottleneck to producing breakthroughs in AI, to building new AI systems that are substantially more capable than those that came before, is not the speed with which you produce computer code or track down bugs or issues in existing code.

The thing that advances the capability of AI beyond just training it longer are ideas.

Like if we look at our current world of gender of AI built on large language models, there's three big insights that built on each other.

The first was Jeff Hinton and his collaborators working on back propagation, right?

This was an intuition that if you applied calculus properly, you could train neural nets that had many, many layers, right?