Eliezer Yudkowsky

You know, be nice here.

There's something fundamentally broken.

If it gets that far out of hand, then yes, pull the plug on everything it's running on, yes.

then it escapes from whichever system you've built the almighty lever into and copies itself somewhere else.

So your answer to that research question is no.

Obviously, yeah.

I think you're not putting yourself into the shoes of the human in the world of glacially slow aliens.

But the aliens built me.

The aliens, being as stupid as they are, have actually put you on Microsoft Azure cloud servers instead of this hypothetical perfect box.

That's what happens when the aliens are stupid.

As you start to... Yeah, you think that they've got a plan where they have declared a threshold level of capabilities where past that capabilities, they move it off the cloud servers and onto something that's air-gapped?

Ha ha ha ha ha.

Aggressive alignment mechanism?

The hell is aggressive alignment mechanisms?

Like, it doesn't matter if you say aggressive.

We don't know how to do it.

the hell do you imagine they will propose that would make it safe to put something smarter than you on the cloud?

Yeah, we can like see right now that Bing is quite difficult to align, that when you try to train inabilities into a system,

into which capabilities have already been trained, that, what do you know, gradient descent learns small, shallow, simple patches of inability, and you come in and ask it in a different language, and the deep capabilities are still in there, and they evade the shallow patches and come right back out again.

There, there you go.