Stuart Russell

So to give you a simple example, right?

What color do we want the sky to be?

It's not sure.

So it shouldn't mess with the sky.

unless it knows for sure that we really want purple with green stripes.

No, in some sense, I'm thinking more like the ideal butler.

To the extent that the butler can anticipate your wishes, they should help you bring them about.

But in areas where there's uncertainty, it can ask questions.

I agree with you.

I mean, I think that's... So you're putting your finger on, in some sense, version 2.0, right?

So let's get version 1.0 clear, right?

This form of AI where...

It has to further our interests, but it doesn't know what those interests are, right?

It then puts an obligation on it to learn more and to be helpful where it understands well enough and to be cautious where it doesn't understand well and so on.

So that actually we can formulate as a mathematical problem, and at least under idealized circumstances, we can literally solve that problem.

So we can make AI systems that know how to solve this problem and help the entities that they are interacting with.

So, yes.

I mean, that's sort of what happens in The Matrix, right?

The AI systems in The Matrix, they tried to give us a utopia, right?

But it failed miserably, and fields and fields of humans had to be destroyed.