David Duvenaud

weakening our connection to the organs of the state and just understanding what's going on is going to be one of the precursors for some faster loss of control.

The point we wanted to make in the paper was that even if there's no fast loss of control, we still might end up having similar loss of control just through the normal business as usual dynamics.

Sure.

I mean, the basic idea is, for similar reasons that we would have less control over the government, also...

the government would have a harder time avoiding being cued by the military if there aren't a bunch of human soldiers in the loop.

Like right now, if somebody goes on the radio and says, I'm the new president, everyone can kind of check, like, actually, that's not the case.

Like, I know a bunch of soldiers.

I call them what's going on.

They say, I don't know what this is.

Or like, no, he's not.

We still have control.

In the world where it is, actually, just a few people who have like sort of

sysadmin access to the robot's army or whatever, there's not all that much that the government or even the people can do to tell who's really in charge except by some naked show of force.

And for the government to stop this, whoever ends up getting sysadmin rights from just de facto taking over the government without having to convince everyone that they're legitimate.

Yeah, exactly.

Because people are, like, too weak to actually fight metal.

Yeah, and right now, maybe what the hard part of the coup is convincing the, like, commanders or, like, the soldiers that this is the new way that things are going to be.

You have to convince a lot of people and this credible threat that they're going to not agree with your takeover and then, like... And then you'll be arrested.

First of all, I think it's a great idea to try to set up these kind of protections, and it's probably a good end.

It looks like involving a lot of well-thought-through mechanisms to ensure that surplus is always available to humans.