Mustafa Suleyman

But there is a significant difference between providing access to information and actually the persuasive campaigning, electioneering, organizing part of it.

This is where the precautionary principle really matters because what I identified in autonomy, self-improvement, and goal-setting

are areas of increased risk, not, you know, sort of total red lines.

I mean, nothing is completely black and white.

And so clearly if you give a system, you know, the ability to constantly self-improve and to act completely independently of a human, it is going to raise the stakes and be much more dangerous.

And I think that we have to have a regime where the human user is liable for the use of these things.

And he can't just claim that like,

You know, I set off this process and came back on, you know, Monday morning and suddenly it's done all sorts of crazy things in, you know, my house or in my community.

It's a great point.

And I sort of made that case in the essay that actually medical superintelligence is something that's just around the corner, is extremely likely to be very safe from a broader AI safety perspective.

And in general, to the extent that we can narrow these capabilities, limit their ability to act

you know sort of autonomously and focus on containment i mean this was the subject of my previous book the coming wave it was all about how proliferation was inevitable and the hard task for us collectively is containment both technical and socio-political because we have to make sure that we remain in control as a species i mean we have this rising set of capabilities which you know

completely unchecked over the next decade or two is likely to surpass human capabilities on all fronts including the ability to self-improve over time which means that it could exponentially improve itself way beyond all of you know sort of humanity's expertise combined and so narrowing

to focus on subdomains especially when those subdomains are massively valuable for impacting qualities i mean you know we will save you know literally decades of people's lives over the next few years through the advances that are coming in in medicine and i just mean the application of known best practice not like you know the sort of speculative invention of the solution for all medicine thing which i think is also happening in the background but just near term

These things are going to be brilliant at giving nurses and doctors timely advice and helping them adhere to known best practice, which saves money and lives.

So clearly that's a much safer route to go.

However, the tricky thing is that it's been generality that has driven a lot of the progress so far.

Jointly training these models on all the text on the web, getting them to be the best that they can possibly be, then distilling that knowledge back into smaller and more inference-efficient models

you know, has been the kind of method that has been most successful.

And so I think there's skepticism in the community that we can give up on the general purpose training regime.