Dario Amodei

Could that actor use it to gain decisive or dominant power over the world as a whole, upsetting the existing balance of power?

4.

Economic disruption.

If the new country is not a security threat in any of the ways listed in hash 1-3 above but simply participates peacefully in the global economy, could it still create severe risks simply by being so technologically advanced and effective that it disrupts the global economy, causing mass unemployment or radically concentrating wealth?

5.

Indirect effects.

The world will change very quickly due to all the new technology and productivity that will be created by the new country.

Could some of these changes be radically destabilizing?

I think it should be clear that this is a dangerous situation.

A report from a competent national security official to a head of state would probably contain words like the single most serious national security threat we've faced in a century, possibly ever.

It seems like something the best minds of civilization should be focused on.

Conversely, I think it would be absurd to shrug and say, nothing to worry about here.

but, faced with rapid AI progress, that seems to be the view of many US policymakers, some of whom deny the existence of any AI risks, when they are not distracted entirely by the usual tired old hot-button issues.

Humanity needs to wake up, and this essay is an attempt, a possibly futile one, but it's worth trying, to jolt people awake.

To be clear, I believe if we act decisively and carefully, the risks can be overcome.

I would even say our odds are good.

And there's a hugely better world on the other side of it.

But we need to understand that this is a serious civilizational challenge.

Below, I go through the five categories of risk laid out above, along with my thoughts on how to address them.

A country of geniuses in a data center could divide their efforts among software design, cyber operations, R&D for physical technologies, relationship building, and statecraft.