Stuart Russell

This is, I think, many of the 20th century technologies, right?

So in statistics, you minimize a loss function.

The loss function is exogenously specified.

In control theory, you minimize a cost function.

In operations research, you maximize a reward function, and so on.

So in all these disciplines, this is how we conceive of the problem, right?

and it's the wrong problem because we cannot specify with certainty the correct objective right we need uncertainty we need the machine to be uncertain about that's objective what it is that it's supposed to be maximizing favorite idea of yours uh i've heard you say somewhere uh well i shouldn't pick favorites but it just sounds beautiful we need to teach machines humility uh

That they're humble in that they know that they don't know what it is they're supposed to be doing.

And that those objectives, I mean, they exist.

They are within us, but we may not be able to explicate them.

We may not even know exactly.

you know, how we want our future to go.

Exactly.

And the machine, you know, a machine that's uncertain is going to be deferential to us.

So if we say don't do that, well, now the machines learn something a bit more about our true objectives, because

something that it thought was reasonable in pursuit of our objective turns out not to be so now it's learned something so it's going to defer because it wants to be doing what we really want and um you know that that point i think is absolutely central to solving the control problem and it's a different kind of ai when you when you

take away this idea that the objective is known, then in fact, a lot of the theoretical frameworks that we're so familiar with, you know, Markov decision processes, goal-based planning, you know, standard games research, all of these techniques actually become inapplicable.

And you get a more complicated problem because, because now,

The interaction with the human becomes part of the problem because the human by making choices is giving you more information about the true objective and that information helps you achieve the objective better.

And so that really means that you're mostly dealing with game theoretic problems where you've got the machine and the human and they're coupled together rather than a machine going off by itself with a fixed objective.