Dwarkesh Patel

What are sort of like surprising takeaways from the experience?

Yeah.

What is the best way for somebody to learn from it?

Is it just like delete all the code and try to re-implement from scratch, try to add modifications to it?

Yeah, I think that's a great question.

Interesting.

You tweeted out that coding models were actually of very little help to you in assembling this repository.

And I'm curious why that was.

And they just couldn't internalize that you had your own?

The reason I think this question is so interesting is because the main story people have about AI exploding and getting to superintelligence pretty rapidly is AI automating, AI engineering, and AI research.

And so they'll look at the fact that you can have Cloud Code make entire application, CRUD applications from scratch and be like, if you had this same capability inside of OpenAI and DeepMind and everything, well, just imagine the level of like just, you know, a thousand of you or a million of you in parallel finding little architectural tweaks.

And so it's quite interesting to hear you say that this is the thing they're sort of asymmetrically worse at.

And it's like quite relevant to forecasting whether the AI 2027 type explosion is likely to happen anytime soon.

I think that's a good way of putting it.

Very naive question, but the architectural tweaks that you're adding to NanoChat, they're in a paper somewhere, right?

They might even be in a repo somewhere.

So is it surprising that they aren't able to integrate that into whenever you're like add rope embeddings or something, they do that in the wrong way?

Yeah.

Actually, here's another reason why this is really interesting.

Through the history of programming, there's been many productivity improvements, compilers, linting, better programming languages, etc.,