Anthony Aguirre

I want it.

Totally get it.

It doesn't mean we should let them have it, but I understand the motivation.

But that's not what super intelligence is going to be like.

Super intelligence is not going to be something that grants you power.

Super intelligence is going to be something that absorbs power.

Like, people are not going to build these things and it's not going to be the genie that's at their command that does all the stuff that they want.

That is not what's going to happen.

These things are going to, like, inevitably be out of control.

And so I think it's just a fundamental misunderstanding of what the real situation is.

And I think if people, countries, companies really came to terms with the fact that superintelligence is not going to be something they control, then the motivations suddenly change, right?

seeking something that you're not going to control and it's just something you're going to lose on the world doesn't make any sense.

And so I'm a little bit hopeful that...

If the people who are in this race can really do the hard thinking and really understand what the nature of the technology is, that at least some of them will shift their view on it and might be more open to like, let's figure out how we can not race down this road that's going to disempower everybody.

A little bit.

Yeah.

I mean, it will be too late soon, but I think we do have some time right now.

I think something I struggle with is, you know, when exactly is it going to be too late?

Like if we build AGI by some definition.

Yeah.