Anthony Aguirre

really good at math is not the way superintelligence has been envisioned since Nick Bostrom's book back in 2014 or 15 or something, Superintelligence.

The way people use it in the field is something that is very broadly capable more than like

Either the very best humans or all of humans together, like any individual humans, whichever way you put it, it's something that is incredibly competent across an incredibly wide range of tasks.

And there's sort of nothing that humans can do that it can't do.

So that is where AI companies are going at the moment.

And some of them are quieter about this and some of them are just saying it out loud and sort of leaning into it.

But why everyone else is just saying, OK, that's where we're going.

What can we do?

You know, that's not a great situation.

Hundreds of millions.

I mean, these are being, you know, OpenAI now is serving, I think it was 700 million weekly active users.

Yeah.

So the scale is just tremendous.

And, you know, Google, of course, has has hundreds of millions or billions of customers, not necessarily for Gemini, its AI model, but in general.

So and Facebook, of course, has infinite amounts of data on everybody.

So that certainly the ability for if these AI systems are given access to the information that the companies have on Google,

all the humans that are using them, they're going to have incredibly detailed dossiers on everybody.

I mean, the avenues of malfeasance are enormous, right?

Like if you imagine an AI system that has the sort of understanding of the humans and all of their secrets, could we see like AI just doing large scale blackmail to get what it wants?

Sure.