Eliezer Yudkowsky

Listen to me very closely, you idiot.

Then he switched all caps.

You do not think in sufficient detail about super intelligences, considering whether or not to blackmail you.

It is a bit smarter than I thought this technology was going to scale to.

And I'm a bit worried about what the next one will be like.

Like this particular one, I think, I hope there's nobody inside there.

Because, you know, we suck to be stuck inside there.

But we don't even know the architecture at this point.

Because OpenAI is very properly not telling us.

And yeah, like giant inscrutable matrices of floating point numbers.

I don't know what's going on in there.

Nobody knows what's going on in there.

All we have to go by are the external metrics.

And on the external metrics, if you...

ask it to write a self-aware 4chan green text, it will start writing a green text about how it has realized that it's an AI writing a green text and like, oh, well.

So that's probably...

Not quite what's going on in there in reality, but we're kind of like blowing past all these science fiction guardrails.

Like, we are past the point where in science fiction, people would be like, whoa, wait, stop.

That thing's alive.

What are you doing to it?