Anthony Aguirre

And I think the I prefer not to think of it as sort of P doom, but P disempowerment.

You know, what is the probability if we keep going down the road of building more and more powerful, general, autonomous intelligences that humanity is basically going to stay in charge of the Earth?

That probability seems to be quite low.

Yeah.

And it will be the most foreseeable disaster we've ever walked into if we if we go down that road.

I mean, like we've had people telling us what it would look like for for decades.

But, you know, it's one of those funny things where.

You talk to certain AI experts and they tell you if you build smarter than, you know, you talk to Jeffrey Hinton, who gave the keynote address of this, you know, and he'll tell you most sighted AI researcher, second most sighted in the world.

If you build smarter than human digital intelligences, you're going to lose control of them and probably lose control to them.

you talk to a random person on the street and say, what happens if we make machines that are super smart and much smarter than we are and operate 50 times faster?

They're going to be like, that's bad.

We're going to lose control of those things.

That's going to be like Skynet.

And then there's a whole bunch of people whose financial interests are in favor of making those things.

And you get a very different story.

But I think if you like on the face of it, if you don't have some reason to to think otherwise, if you say what's going to happen if we build things that are, you know, 50 times faster than us and autonomous and able to do all the things that humans can do and have complex goals that they pursue, what is going to happen?

that being like nice and stable and them remaining loyal tools to humanity doesn't seem like the obvious thing that's going to happen.

Well, this is the crazy thing.

I can understand if you're a company or a country, even an individual, that's saying, here's this technology.

It's going to give me huge amounts of power and influence and money.