Max Tegmark

And you could totally imagine, yeah, maybe this oxygen is kind of annoying because it caused more corrosion.

So let's get rid of the oxygen.

And good luck surviving after that.

I'm not particularly concerned that they would want to kill us just because that would be a goal in itself.

We've driven a number of the elephant species extinct.

It wasn't because we didn't like elephants.

The basic problem is you just don't want to give, you don't want to cede control over your planet to some other more intelligent entity that doesn't share your goals.

It's that simple.

Which brings us to another key challenge which AI safety researchers have been grappling with for a long time.

How do you make AI, first of all, understand our goals and then adopt our goals and then retain them as they get smarter, right?

All three of those are really hard, right?

Like a human child, first, they're just not smart enough to understand our goals.

They can't even talk.

And then eventually they're teenagers and understand our goals just fine, but they don't share.

But there is, fortunately, a magic phase in the middle where they're smart enough to understand our goals and malleable enough that we can hopefully, with good parenting, teach them right from wrong and instill good goals in them, right?

So those are all...

tough challenges with computers.

And then, you know, even if you teach your kids good goals when they're little, they might outgrow them too.

And that's a challenge for machines.

They keep improving.