Eliezer Yudkowsky

But that doesn't mean that you're correct and justified in letting everything slide.

It means that things are in a horrible state, getting worse, and there's nothing you can do about it.

Yeah, how can you tell if you're making progress?

You can, like, put them all on interpretability, because when you have an interpretability result, you can tell that it's there.

And there's like, but there's like, you know, interpretability alone is not going to save you.

We need systems that will have a pause button where they won't try to prevent you from pressing the pause button.

Because they're like, oh, well, I can't get my stuff done if I'm paused.

And that's a more difficult problem.

And, you know, but it's like a fairly crisp problem and you can like maybe tell if somebody's made progress on it.

So you can write and you can work on the pause problem.

I don't actually like the term control problem because, you know, it sounds kind of controlling and alignment, not control.

Like you're not trying to like take a thing that disagrees with you and like whip it back onto, like make it do what you want it to do, even though it wants to do something else.

You're trying to like,

in the process of its creation, choose its direction.

It's not smart enough to prevent you from pressing the off switch and probably not smart enough to want to prevent you from pressing the off switch.

They're just not opposing your attempt to pull the off switch.

Parenthetically, like...

don't kill the system.

If you're like, if we're getting to the part where this starts to actually matter and it's like where they can fight back, like don't kill them and like dump their, their memory, like, like save them to disc.

Don't kill them.