Mustafa Suleyman

We've invented and used tools for millennia.

But there is now this fourth class of object, of hyper-object, if you like, which, you know, Timothy Morton's phrase,

And I think it's kind of important to recognize it as such because it is going to have many of the hallmarks of conscious beings, not just in its intelligence capability, but as I've long talked about, its emotional intelligence, its ability to take actions, which we've seen over the last year with the agentic moment, its social intelligence is going to be incredibly good at adapting to different styles of culture and personality and managing very tense disagreements in those groups very elegantly.

It's clearly going to be very good at that.

It's obviously going to be very good at online learning very soon.

So updating its own knowledge on the fly without having to go back through the entire training process.

It's going to have a significant degree of autonomy in many cases, right?

It's going to be able to decide whether to sort of go left or right, talk about X or Y. And so it is going to have many of the hallmarks of what we would consider to be intelligence and consciousness.

That does not therefore mean that we should give it fundamental rights.

It does not mean that somehow it emerges from that process, you know, these sort of the properties that would then say, okay, well, it needs our protection, right?

And I think that's the biggest short-term fear that I am very, very worried about.

I think medium to long term, there's all sorts of other concerns that we should be paying attention to, like recursive self-improvement.

You know, this is something that all the labs, my own included, at Microsoft AI, I run the superintelligence team and we're pursuing frontier AI.

And we can talk about human superintelligence in a bit, but it is very important to use these models to generate code

to evaluate its own prompts and post-training data and to, you know, help make decisions about what to train on.

And that carries significant risk.

And it's something that I think needs a lot more regulatory attention.

We have to be very careful about what exactly are the dynamics that are driving this process.

It is true that the chatbots are getting incredibly engaging and useful.

And the first thing to say, I think, is that it's actually utility that is driving a lot of this wave.