David Duvenaud

Yeah.

I mean, the other thing I want to say is I think people have stronger preferences about the future than they sort of think they do at first blush.

So like if you have discussions, I've had a lot of discussions with people who've thought a lot about the future and they sort of say something like a thousand years from now, if it's like aliens or robots or humans running the earth, like, do I really care?

And I think...

there's this exercise you can do.

I think natively, yeah, we don't actually have strong preferences about the future because it doesn't matter that much, especially the distant future for our actions.

And we only end up having preferences about things that we spend a bunch of time sort of practicing taking actions over.

And maybe a good example is if you try to ask a dog, what house do you want to live in?

Or how do you want to be treated a year from now?

You might say there's no coherent sense in which the dog has preferences about this.

But it does have a whole bunch of short-term preferences that could be chained together.

And you could try to elicit those and run them forward.

And same thing with humans.

It's like my initial reaction if I think about the world 1,000 years from now.

And it's like some future race has taken over Earth.

Maybe I'm not bothered by that.

But then I think, wait, wait, wait.

So like right now I have kids and they're going to be having kids.

And if a thousand years from now, some other race is taking over, where's the day where my kids get like killed or starved or like replaced or uploaded or whatever?

Right.