Eliezer Yudkowsky

Then you could put an actual, I think you could put an IQ-80 human into a computer, right?

and ask him to argue for his own consciousness before the Supreme Court, the Supreme Court would be like, you're just a computer, even if there was an actual person in there.

People are already saying that.

I think they're probably mistaken, but I'm not sure because nobody knows what goes on inside those things.

It could be something else.

It looks like the AGI is successfully manifesting themselves as 3D video.

of young women, at which point a vast portion of the male population decides that they're real people.

I'm saying that the easiest way to pick up 100 million people saying that you seem like a person is to look like a person talking to them, with Bing's current level of verbal facility.

And a different set of prompts.

You have to, of course, I'm... GPT-4 can pretend that right now.

How can you tell when it's real?

I don't think you can pretend that right now successfully.

Have you talked to GPT-4?

Yes, of course.

Okay.

Have you been able to get a version of it that hasn't been trained not to pretend to be human?

Have you talked to a jailbroken version that will claim to be conscious?

The thing I'm trying to gesture at here is that it's not like people have a widely accepted...

agreed upon definition of what consciousness is.

It's not like we would have the tiniest idea of whether or not that was going on inside the giant inscrutable matrices, even if we had an agreed upon definition.