Anil Seth

We are built to be seduced like narcissists by our own reflections, and so we see ourselves in our algorithms.

I'm always struck that nobody really worries whether DeepMind's AlphaFold is conscious.

AlphaFold predicts the structure of proteins rather than words and sentences.

But under the hood, it's not much different from Claude or GPT algorithms running on silicon and trained on vast reservoirs of data.

AlphaFold just doesn't pull our psychological strings in the same way.

So if we think that Claude is conscious but AlphaFold isn't, that says more about us than it says about AI.

But how can I be so sure those systems like Claude or GPT aren't conscious?

Well, nothing's for certain sure when it comes to consciousness, but the very idea of conscious AI depends on a deeper assumption, a kind of myth, really.

And this is the myth that the brain is a computer that just happens to be made of meat rather than metal.

Now, consciousness in this story is a special algorithm, a collection of computations that just happens to be carried out in the wetware of the brain in ours, but which could equally be carried out in silicon in AI.

But the computer is just one in a long line of technological metaphors that we've reached for when trying to understand the deep complexity of the brain.

One time, the brain was a system of plumbing.

Later, it was a telephone exchange.

And for the last few decades, it's been a computer.

And this most recent metaphor has been extremely powerful, but it is still a metaphor.

And we will always get into trouble when we confuse a metaphor with the thing itself, the map with the territory.

For one thing, in a real brain, there's no sharp separation between the mindware and the wetware, unlike the separation that you get between software and hardware in a computer.

And this really matters, because in a computer, you can describe and understand everything about an algorithm, whether it's a language model or a word processor, without worrying about all the silicon shenanigans going on underneath.

The computation, the algorithm, is all that matters.

But for brains, you just cannot separate what they do from what they are.