Marcus Hutter

You think that makes me unconscious because it strangles me or?

Yeah.

And you explain sort of somehow why.

But you infer that from my behavior, right?

You can never be sure about that.

And I think the same thing will happen with any intelligent agent we develop.

If it behaves...

a way sufficiently close to humans or maybe even not humans i mean you know maybe a dog is also sometimes a little bit self-conscious right so so if it behaves in a way um where we attribute typically consciousness we would attribute consciousness to these intelligent systems and you know actually probably in particular that of course doesn't answer the question whether it's really conscious and that's the you know the big hard problem of consciousness you know maybe i'm

for developing AGI.

I think we progress, at some point we have solved all the technical problems, and this system will behave intelligent and then super intelligent, and this consciousness will emerge.

I mean, definitely it will display behavior which we will interpret as conscious.

And then it's a philosophical question, did this consciousness really emerge or is it a zombie which just, you know, fakes everything?

We still don't have to figure that out, although it may be interesting, at least from a philosophical point of view, it's very interesting, but it may also be sort of practically interesting.

You know, there's some people say, you know, if it's just faking consciousness and feelings, you know, then we don't need to be concerned about, you know, rights.

But if it's real conscious and has feelings, then we need to be concerned.

Yeah.

It is rather easy to build systems which people ascribe consciousness.

And I give you an analogy.

I mean, remember, maybe it was before you were born, the Tamagotchi.

Before you were born.