Dario Amodei

Although biology is currently the most serious vector of attack, there are many other vectors and it is possible that a more dangerous one may emerge.

The general principle is that without countermeasures, AI is likely to continuously lower the barrier to destructive activity on a larger and larger scale, and humanity needs a serious response to this threat.

The previous section discussed the risk of individuals and small organizations co-opting a small subset of the country of geniuses in a data center to cause large-scale destruction.

But we should also worry, likely substantially more so, about misuse of AI for the purpose of wielding or seizing power, likely by larger and more established actors.

In Machines of Loving Grace, I discuss the possibility that authoritarian governments might use powerful AI to surveil or repress their citizens in ways that would be extremely difficult to reform or overthrow.

Current autocracies are limited in how repressive they can be by the need to have humans carry out their orders, and humans often have limits in how inhumane they are willing to be.

But AI-enabled autocracies would not have such limits.

Worse yet, countries could also use their advantage in AI to gain power over other countries.

If the country of geniuses as a whole was simply owned and controlled by a single, human, country's military apparatus, and other countries did not have equivalent capabilities, it is hard to see how they could defend themselves.

They would be outsmarted at every turn, similar to a war between humans and mice.

Putting these two concerns together leads to the alarming possibility of a global totalitarian dictatorship.

Obviously, it should be one of our highest priorities to prevent this outcome.

There are many ways in which AI could enable, entrench, or expand autocracy, but I'll list a few that I'm most worried about.

Note that some of these applications have legitimate defensive uses, and I am not necessarily arguing against them in absolute terms.

I am nevertheless worried that they structurally tend to favor autocracies.

Fully Autonomous Weapons A swarm of millions or billions of fully automated armed drones, locally controlled by powerful AI and strategically coordinated across the world by an even more powerful AI, could be an unbeatable army, capable of both defeating any military in the world and suppressing dissent within a country by following around every citizen.

Developments in the Russia-Ukraine war should alert us to the fact that drone warfare is already with us, though not fully autonomous yet, and a tiny fraction of what might be possible with powerful AI.

R&D from powerful AI could make the drones of one country far superior to those of others, speed up their manufacture, make them more resistant to electronic attacks, improve their maneuvering, and so on.

Of course, these weapons also have legitimate uses in the defense of democracy.

They have been key to defending Ukraine and would likely be key to defending Taiwan.