Dwarkesh Patel

That's very interesting to hear you say that the sort of safety guarantees you need from software are actually not dissimilar to self-driving because what people will often say is that self-driving took so long because the cost of failure is so high.

Like a human makes a mistake on average every 400,000 miles or every seven years.

And if you had to release a coding agent that couldn't make a mistake for at least seven years, it would be much harder to deploy.

But I guess your point is that if you made a catastrophic coding mistake, like breaking some important system every seven years.

And in fact, in terms of sort of wall clock time, it would be much less than seven years because you're like constantly outputting code like that, right?

So it's like per tokens, or in terms of tokens, it would be seven years.

But in terms of wall clock time, it would be pretty close.

There's another objection people make to that analogy, which is that with self-driving, what took a big fraction of that time was solving the problem of having basic perception that's robust and building representations and having a model that has some common sense so it can generalize to when it sees something that's slightly out of distribution.

If somebody's waving down the road this way, you don't need to train for it.

The thing will...

have some understanding of how to respond to something like that.

And these are things we're getting for free with LLMs or VLMs today.

So we don't have to solve these very basic representation problems.

And so now deploying AIs across different domains will sort of be like deploying a self-driving car with current models to a different city, which is hard, but not like a 10-year-long task.

You let self-driving for five years at Tesla.

Because one, the start is at 1980, not 10 years ago.

And then two, the end is not here yet.

I'm curious to bounce two other ways in which the analogy might be different.

And the reason I'm especially curious about this is because I think the question of how fast AI is deployed, how valuable it is when it's early on is like potentially the most important question in the world right now, right?

Like if you're trying to model what the year 2030 looks like, this is the question you want to have some understanding of.