Dario Amodei

Alternatively, it's possible that taking over countries is feasible with only AI surveillance and AI propaganda, and never actually presents a clear moment where it's obvious what is going on and where a nuclear response would be appropriate.

Maybe these things aren't feasible and the nuclear deterrent will still be effective, but it seems too high stakes to take a risk.

A second possible objection is that there might be countermeasures we can take against these tools of autocracy.

We can counter drones with our own drones, cyber defense will improve along with cyber attack, there may be ways to immunize people against propaganda, etc.

My response is that these defenses will only be possible with comparably powerful AI.

If there isn't some counterforce with a comparably smart and numerous country of geniuses in a data center, it won't be possible to match the quality or quantity of drones, for cyber defense to outsmart cyber offense, etc.

So the question of countermeasures reduces to the question of a balance of power in powerful AI.

Here, I am concerned about the recursive or self-reinforcing property of powerful AI which I discussed at the beginning of this essay.

That each generation of AI can be used to design and train the next generation of AI.

This leads to a risk of a runaway advantage, where the current leader in powerful AI may be able to increase their lead and may be difficult to catch up with.

We need to make sure it is not an authoritarian country that gets to this loop first.

Furthermore, even if a balance of power can be achieved, there is still risk that the world could be split up into autocratic spheres, as in 1984.

Even if several competing powers each have their powerful AI models, and none can overpower the others, each power could still internally repress their own population, and would be very difficult to overthrow, since the populations don't have powerful AI to defend themselves.

It is thus important to prevent AI-enabled autocracy even if it doesn't lead to a single country taking over the world.

How do we defend against this wide range of autocratic tools and potential threat actors?

As in the previous sections, there are several things I think we can do.

First, we should absolutely not be selling chips, chip-making tools, or data centers to the CCP.

Chips and chip-making tools are the single greatest bottleneck to powerful AI, and blocking them is a simple but extremely effective measure, perhaps the most important single action we can take.

It makes no sense to sell the CCP the tools with which to build an AI totalitarian state and possibly conquer us militarily.

A number of complicated arguments are made to justify such sales, such as the idea that spreading our tech stack around the world allows America to win, in some general, unspecified economic battle.