Stuart Russell

And the best they could come up with was late 20th century regular human life with all of its problems.

And I think this is a really interesting point.

And absolutely central because, you know, there's a lot of science fiction where super intelligent robots, you know, they just want to help humans.

And the humans who don't like that, you know, they just give them a little brain operation and then they do like it.

And it takes away human motivation.

By taking away failure, taking away disease, you actually lose important parts of human life.

And it becomes in some sense pointless.

So if it turns out that there simply isn't any way that humans can really flourish,

in coexistence with super intelligent machines, even if they're perfectly designed to solve this problem of figuring out what futures humans want and bringing about those futures.

If that's not possible, then those machines will actually disappear.

Why would they disappear?

Because that's the best thing for us.

Maybe they would stay available for real existential emergencies, like if there is a giant asteroid about to hit the Earth, maybe they'll help us because they at least want the human species to continue.

But to some extent...

It's not a perfect analogy, but it's sort of the way that human parents have to at some point step back from their kids' lives and say, okay, no, you have to tie your own shoelaces today.

Yep, there are science fiction stories like that too.

Yeah, I hope there is some happy medium where...

The AI systems can be there and we can take advantage of those capabilities to have a civilization that's much better than the one we have now.

But I think you're right.

A civilization with no challenges is not conducive to human flourishing.