Dario Amodei

Yeah. Yeah. I mean, you, you might imagine from outside the field that like, there's only one of these, right. That like you made it, you've only made one, but the truth is that like the scale up is very quick. Like we, we do this today. We make a model and then we deploy thousands, maybe tens of thousands of instances of it. I think by the time, uh,

you know, certainly within two to three years, whether we have these super powerful AIs or not, clusters are going to get to the size where you'll be able to deploy millions of these and they'll be, you know, faster than humans. And so if your picture is, oh, we'll have one and it'll take a while to make them, my point there was, no, actually you have millions of them right away.

Yeah. So yeah, let's, let's describe the extreme. So like one, one extreme would be, well, look, um, you know, uh, if we look at kind of evolutionary history, like there was this big acceleration where, you know, for hundreds of thousands of years, we just had like. single-celled organisms, and then we had mammals, and then we had apes, and then that quickly turned to humans.

Humans quickly built industrial civilization. And so this is going to keep speeding up. And there's no ceiling at the human level. Once models get much, much smarter than humans, they'll get really good at building the next models. And if you write down a simple differential equation, this is an exponential. And so what's going to happen is that

uh models will build faster models models will build faster models and those models will build you know nanobots that can like take over the world and produce much more energy than you could produce otherwise and and so if you just kind of like solve this abstract differential equation then like five days after we you know we build the first ai that's more powerful than humans then then uh you know like the world will be filled with these ais and every possible technology that could be invented like will be invented um i'm caricaturing this a little bit um

But I, you know, I think that's one extreme. And the reason that I think that's not the case is that one, I think they just neglect like the laws of physics. Like it's only possible to do things so fast in the physical world. Like some of those loops go through, you know, producing faster hardware. It takes a long time to produce faster hardware. Things take a long time.

There's this issue of complexity. Like I think no matter how smart you are, like, People talk about, oh, we can make models of biological systems that'll do everything in biological systems. Look, I think computational modeling can do a lot. I did a lot of computational modeling when I worked in biology. But there are a lot of things that you can't predict.

They're complex enough that just iterating, just running the experiment is going to beat any modeling, no matter how smart the system doing the modeling is.

Yeah, I think, well, the modeling's going to be hard, and getting the model to match the physical world is going to be hard. All right, so he does have to interrupt the physical world to verify. Yeah, yeah, yeah. But it's just, you know, you just look at even the simplest problems. Like, you know, I think I talk about, like, you know, the three-body problem or simple chaotic prediction, like...

you know, or, or like predicting the economy, it's really hard to predict the economy two years out. Like maybe the case is like, you know, normal, you know, humans can predict what's going to happen in the economy next quarter. Or they can't really do that.

Maybe, maybe a AI system that's, you know, a zillion times smarter can only predict it out a year or something instead of, instead of, you know, you have these kinds of exponential increase in computer intelligence for linear increase in, in, in ability to predict. Same with, again, like, you know, biological molecules interacting.

You don't know what's going to happen when you perturb a complex system. You can find simple parts in it. If you're smarter, you're better at finding these simple parts. And then I think human institutions. Human institutions are just... are really difficult. It's been hard to get people.

I won't give specific examples, but it's been hard to get people to adopt even the technologies that we've developed, even ones where the case for their efficacy is very, very strong. People have concerns. They think things are conspiracy theories. It's been very difficult. It's also been very difficult to get you know, very simple things through the regulatory system, right?

I think, you know, and, you know, I don't want to disparage anyone who, you know, works in regulatory systems of any technology. There are hard trade-offs they have to deal with. They have to save lives. But the system as a whole, I think, makes some obvious trade-offs that are very far from maximizing human welfare. And so if we bring AI systems into this, you know,

into these human systems, often the level of intelligence may just not be the limiting factor, right? It just may be that it takes a long time to do something. Now, if the AI system circumvented all governments, if it just said, I'm dictator of the world and I'm going to do whatever, some of these things it could do. Again, the things have to do with complexity.

I still think a lot of things would take a while. I don't think it helps that the AI systems can produce a lot of energy or go to the moon. Like, Some people in comments responded to the essay saying the AI system can produce a lot of energy and smarter AI systems. That's missing the point. That kind of cycle doesn't solve the key problems that I'm talking about here.

So I think a bunch of people missed the point there. But even if it were completely unaligned and, you know, could get around all these human obstacles, it would have trouble. But again, if you want this to be an AI system that doesn't take over the world, that doesn't destroy humanity, then, then basically, you know, it's, it's, it's going to need to follow basic human laws, right?

You know, if we want to have an actually good world, like we're going to have to have an AI system that, that interacts with humans, not one that kind of creates its own legal system or disregards all the laws or all of that. So as inefficient as these processes are, you know,

we're going to have to deal with them because there needs to be some popular and democratic legitimacy in how these systems are rolled out. We can't have a small group of people who are developing these systems say, this is what's best for everyone, right? I think it's wrong. And I think in practice, it's not going to work anyway. So you put all those things together and we're not going to

We're not going to change the world and upload everyone in five minutes. A, I don't think it's going to happen. And B, to the extent that it could happen, it's not the way to lead to a good world. So that's on one side. On the other side, there's another set of perspectives, which I have actually in some ways more sympathy for, which is, look, We've seen big productivity increases before, right?