Dario Amodei

Our hope is that transparency legislation will give a better sense over time of how likely or severe autonomy risks are shaping up to be, as well as the nature of these risks and how best to prevent them.

As more specific and actionable evidence of risks emerges, if it does, future legislation over the coming years can be surgically focused on the precise and well-substantiated direction of risks, minimising collateral damage.

To be clear, if truly strong evidence of risks emerges, then rules should be proportionately strong.

Overall, I am optimistic that a mixture of alignment training, mechanistic interpretability, efforts to find and publicly disclose concerning behaviors, safeguards, and societal-level rules can address AI autonomy risks, although I am most worried about societal-level rules and the behavior of the least responsible players, and it's the least responsible players who advocate most strongly against regulation.

I believe the remedy is what it always is in a democracy.

Those of us who believe in this cause should make our case that these risks are real and that our fellow citizens need to band together to protect themselves.

Heading.

2.

A surprising and terrible empowerment.

Subheading.

Misuse for destruction.

Let's suppose that the problems of AI autonomy have been solved.

We are no longer worried that the country of AI geniuses will go rogue and overpower humanity.

The AI geniuses do what humans want them to do, and because they have enormous commercial value, individuals and organizations throughout the world can rent one or more AI geniuses to do various tasks for them.

Everyone having a superintelligent genius in their pocket is an amazing advance and will lead to an incredible creation of economic value and improvement in the quality of human life.

I talk about these benefits in great detail in Machines of Loving Grace.

But not every effect of making everyone superhumanly capable will be positive.

As Bill Joy wrote 25 years ago in Why the Future Doesn't Need Us, Quote,

building nuclear weapons required, at least for a time, access to both rare, indeed, effectively unavailable, raw materials and protected information.

Biological and chemical weapons programs also tended to require large-scale activities.