Dwarkesh Patel

And make no mistake about it, mass surveillance is nowhere near the top of the highest stakes thing that one could do with AGI.

This is just an example that has come up early in the development of this technology and is giving us a sneak peek at the power dynamics that will be at play.

Now, the military insists that the law already prohibits mass surveillance.

And so Anthropic should let its models be used for, quote, all lawful purposes, end quote.

But of course, as we saw with the Snowden revelations in 2013, even for this very specific example of mass surveillance, the government is very willing to use secret and deceptive interpretations of the law to justify its actions.

Remember, what we learned from Snowden was that the NSA, which, by the way, is a part of the Department of War,

was using the 2001 Patriot Act to justify collecting every single phone record in America because the argument was that some subset of them might be relevant for a future investigation.

And they ran this program for years under a secret court order.

So when the Pentagon today says, we will never use your models for mass surveillance because it's already illegal, so your red lines are unnecessary, it would be incredibly naive to take that at face value.

No government is going to call what they are doing mass surveillance.

For them, it will always have a different euphemism.

So Anthropic comes back and says, no, we don't trust you.

We want the right to draw these red lines and to refuse you service.

if we determine that you're breaking the contract and you're breaking the terms of service.

But now think about it from the military's perspective.

In the future, every single soldier in the field, every single bureaucrat and analyst in the Pentagon, even the generals, are going to be AIs.

And on current track, those AIs are going to be provided by a private company.

I'm guessing that Pete Hegseth is not thinking about Gen AI in those terms.

But sooner or later, the stakes will become obvious, just as after 1945, the stakes of nuclear weapons became obvious to everybody in the world.

And now a private company insists that it reserves the right to say to you, hey, you're breaking the values and the terms of service that we have embedded in our contract with you, and so we're cutting you off.