Stuart Russell

Right, so keyed to humans.

And the difficulty that I mentioned earlier, the King Midas problem, how do we specify what we want the future to be like so that it can do it for us?

How do we specify the objectives?

Actually, we have to give up on that idea.

Because it's not possible, right?

We've seen this over and over again in human history.

We don't know how to specify the future properly.

We don't know how to say what we want.

And, you know, I always use the example of the genie, right?

What's the third wish that you give to the genie who's granted you three wishes, right?

Undo the first two wishes because I've made a mess of the universe.

So...

So in fact, what we're going to do is we're going to make it the machine's job to figure out.

So it has to bring about the future that we want, but it has to figure out what that is.

And it's going to start out not knowing.

And, uh,

Over time, through interacting with us and observing the choices we make, it will learn more about what we want the future to be like, but probably it will forever have residual uncertainty

about what we really want the future to be like.

It'll be fairly sure about some things and it can help us with those.

And it'll be uncertain about other things and it'll be, in those cases, it will not take action that might upset humans with that aspect of the world.