Derek Thompson

in any part in the kill chain to put up an autonomous drone swarm to stop, say, a hypersonic missile fired by the Chinese at some American target

that something in Claude is gonna stand up and essentially say, I'm sorry, I can't do that, Pete, because it's been trained somehow, led to believe that it is being asked to do something that is not in its constitution.

That is the fear, that its own weird, silicon-based moral sense will override the Pentagon-based need to direct the technology.

Is something like that core to the fear?

Like, you could call it a moral sense and whatever, but, like, it's also— It's hard to do this on the fly without anthropomorphizing, and I'm not in any way suggesting a conscience here.

Yeah, yeah, yeah, of course.

One thing that's distinguished your commentary on the anthropic Department of War conflict is that to me, you seem to see this conflict in mythic terms.

I'm going to quote directly from the essay that you wrote recently.

Quote, it is increasingly difficult to discuss the developments of frontier artificial intelligence and what kind of futures we should aim to build without acknowledging our place at the deathbed of the republic as we know it.

That is very dramatic language.

In what way do you see us at the deathbed of the American Republic?

It sometimes sounds like you're saying American democracy as we know it, America's governing norms, might not survive this technology if we keep going in the direction that we're going.

I think that three weeks ago, I would not have agreed with that.

But now that a simple contract negotiation between a private company and the Pentagon broke down such that the Pentagon, for the first time in American history, essentially designated an American company a supply chain threat that had to be essentially destroyed, I'm beginning to wonder whether or not this argument that the American government coming up against AI is going to produce results that are, at the very least,

unfamiliar to modern American history.

Does that generally sound right for you?

Or maybe the better way to hand this off to you, like persuade those who might be totally confused by the drama with which you see this, that America's governing norms really are in danger if the technology keeps improving at this rate.

I don't want you to be offended by what I'm about to say, but you kind of sound like a Biden administration person here.

I mean, it is the Biden administration that says if we give China access to the most advanced NVIDIA chips, they're going to build a surveillance society that makes 1984 look like...

kindergarten.