David Kyle Johnson

And the thing is, he's asking questions that a smart person should ask.

The thing is, they were asked decades ago and answered, and he just doesn't appear to know that.

So he writes, so my position is, if these machines are not conscious, what more could it possibly take to convince you that they are?

So that's, again, that's an old question.

I answered that question many times on my own blog over the last 20 years.

You need to know something about the process.

You can't know by the output alone.

Because a sophisticated enough magician can fool you with the, right, you have to know that magic's not real to know that the magician's not using magic.

In this case, I think where the analogy breaks down is I think that a conscious machine is possible.

I'm not saying it's impossible.

It's just that LLMs are not programmed to be conscious.

They are programmed to mimic language.

So I would need to know something about the process.

And then he goes on to—I'll finish up with this.

He asks, again, another really interesting philosophical question, but one that he seems to be unaware of.

There's a long conversation about this that's already happened.

And that is, but now, as an evolutionary biologist, I say the following.

If these creatures are not conscious, then what the hell is consciousness for?

So what's he talking about?

I don't know.