Max Tegmark

building an API where code can control this super powerful thing, right?

That's very unfortunate because one thing that systems like GPT-4 have going for them is that they are an oracle in the sense that they just answer questions.

There is no robot connected to GPT-4.

GPT-4 can't go and do stock trading based on its thinking.

It is not an agent.

An intelligent agent is something that takes in information from the world, processes it to figure out what action to take based on its goals that it has, and then does something back on the world.

But once you have an API, for example, GPT-4, nothing stops Joe Schmo and a lot of other people from building real agents, which just keep making calls somewhere in some inner loop somewhere to these powerful Oracle systems.

Which makes them themselves much more powerful.

That's another kind of unfortunate development, which I think we would have been better off delaying.

I don't want to pick on any particular companies.

I think they're all under a lot of pressure to make money.

And again, the reason we're calling for this pause is to give them all cover to do what they know is the right thing.

Slow down a little bit at this point, but

Everything we've talked about, I hope, will make it clear to people watching this why these sort of human-level tools can cause a gradual acceleration.

You keep using yesterday's technology to build tomorrow's technology.

And when you do...

that over and over again, you naturally get an explosion.

You know, that's the definition of an explosion in science, right?

Like if you have two people and they fall in love, now you have four people and then they can make more babies and now you have eight people and then you have 16, 32, 64, et cetera.

We call that a population explosion, where it's just that each, if it's instead free neutrons,