Dwarkesh Patel

When would you expect that kind of thing to start happening?

And more general question about like multi-agent systems and a sort of like independent AI civilization and culture.

And can you identify the key bottleneck that's preventing this kind of collaboration between LLMs?

Maybe like the way I would put it is...

Yeah.

So you've talked about how you were at Tesla leading self-driving from 2017 to 2022.

And then you firsthand saw this progress from, we went from cool demos to now thousands of cars out there actually autonomously doing drives.

Why did that take a decade?

Like what was happening through that time?

That's very interesting to hear you say that the sort of safety guarantees you need from software are actually not dissimilar to self-driving because what people will often say is that self-driving took so long because the cost of failure is so high.

Like a human makes a mistake on average every 400,000 miles or every seven years.

And if you had to release a coding agent that couldn't make a mistake for at least seven years, it would be much harder to deploy.

But I guess your point is that if you made a catastrophic coding mistake, like breaking some important system every seven years.

And in fact, in terms of sort of wall clock time, it would be much less than seven years because you're like constantly outputting code like that, right?

So it's like per tokens, or in terms of tokens, it would be seven years.

But in terms of wall clock time, it would be pretty close.

There's another objection people make to that analogy, which is that with self-driving, what took a big fraction of that time was solving the problem of having basic perception that's robust and building representations and having a model that has some common sense so it can generalize to when it sees something that's slightly out of distribution.

If somebody's waving down the road this way, you don't need to train for it.

The thing will...

have some understanding of how to respond to something like that.