Dario Amodei

In my view, this is like selling nuclear weapons to North Korea and then bragging that the missile casings are made by Boeing and so the US is winning.

China is several years behind the US in their ability to produce frontier chips in quantity, and the critical period for building the country of geniuses in a data center is very likely to be within those next several years.

There is no reason to give a giant boost to their AI industry during this critical period.

Second, it makes sense to use AI to empower democracies to resist autocracies.

This is the reason Anthropic considers it important to provide AI to the intelligence and defense communities in the US and its democratic allies.

Defending democracies that are under attack, such as Ukraine and, via cyber attacks, Taiwan, seems especially high priority, as does empowering democracies to use their intelligence services to disrupt and degrade autocracies from the inside.

At some level the only way to respond to autocratic threats is to match and outclass them militarily.

A coalition of the US and its democratic allies, if it achieved predominance in powerful AI, would be in a position to not only defend itself against autocracies, but contain them and limit their AI totalitarian abuses.

Third, we need to draw a hard line against AI abuses within democracies.

There need to be limits to what we allow our governments to do with AI, so that they don't seize power or repress their own people.

The formulation I have come up with is that we should use AI for national defence in all ways except those which would make us more like our autocratic adversaries.

Where should the line be drawn?

In the list at the beginning of this section, two items, using AI for domestic mass surveillance and mass propaganda, seem to me like bright red lines and entirely illegitimate.

Some might argue that there's no need to do anything, at least in the US, since domestic mass surveillance is already illegal under the Fourth Amendment.

But the rapid progress of AI may create situations that our existing legal frameworks are not well designed to deal with.

For example, it would likely not be unconstitutional for the US government to conduct massively scaled recordings of all public conversations, for example, things people say to each other on a street corner, and previously it would have been difficult to sort through this volume of information, but with AI it could all be transcribed, interpreted, and triangulated to create a picture of the attitude and loyalties of many or most citizens.

I would support civil liberties-focused legislation or maybe even a constitutional amendment that imposes stronger guardrails against AI-powered abuses.

The other two items, fully autonomous weapons and AI for strategic decision-making, are harder lines to draw since they have legitimate uses in defending democracy while also being prone to abuse.

Here I think what is warranted is extreme care and scrutiny combined with guardrails to prevent abuses.

My main fear is having too small a number of fingers on the button, such that one or a handful of people could essentially operate a drone army without needing any other humans to cooperate to carry out their orders.