Dario Amodei

As AI systems get more powerful, we may need to have more direct and immediate oversight mechanisms to ensure they are not misused, perhaps involving branches of government other than the executive.

I think we should approach fully autonomous weapons in particular with great caution and not rush into their use without proper safeguards.

Fourth, after drawing a hard line against AI abuses in democracies, we should use that precedent to create an international taboo against the worst abuses of powerful AI.

I recognise that the current political winds have turned against international cooperation and international norms, but this is a case where we sorely need them.

The world needs to understand the dark potential of powerful AI in the hands of autocrats, and to recognize that certain uses of AI amount to an attempt to permanently steal their freedom and impose a totalitarian state from which they can't escape.

I would even argue that in some cases, large-scale surveillance with powerful AI, mass propaganda with powerful AI, and certain types of offensive uses of fully autonomous weapons should be considered crimes against humanity.

More generally, a robust norm against AI-enabled totalitarianism and all its tools and instruments is sorely needed.

It is possible to have an even stronger version of this position, which is that because the possibilities of AI-enabled totalitarianism are so dark, autocracy is simply not a form of government that people can accept in the post-powerful AI age.

Just as feudalism became unworkable with the Industrial Revolution,

the AI age could lead inevitably and logically to the conclusion that democracy, and, hopefully, democracy improved and reinvigorated by AI, as I discuss in Machines of Loving Grace, is the only viable form of government if humanity is to have a good future.

Fifth and finally, AI companies should be carefully watched, as should their connection to the government, which is necessary, but must have limits and boundaries.

The sheer amount of capability embodied in powerful AI is such that ordinary corporate governance, which is designed to protect shareholders and prevent ordinary abuses such as fraud, is unlikely to be up to the task of governing AI companies.

There may also be value in companies publicly committing to, perhaps even as part of corporate governance, not take certain actions, such as privately building or stockpiling military hardware.

using large amounts of computing resources by single individuals in unaccountable ways, or using their AI products as propaganda to manipulate public opinion in their favor.

The danger here comes from many directions, and some directions are in tension with others.

The only constant is that we must seek accountability, norms, and guardrails for everyone, even as we empower good actors to keep bad actors in check.

The previous three sections were essentially about security risks posed by powerful AI.

Risks from the AI itself, risks from misuse by individuals and small organizations and risks of misuse by states and large organizations.

If we put aside security risks or assume they have been solved, the next question is economic.

What will be the effect of this infusion of incredible human capital on the economy?