Mustafa Suleyman

Or if it was able to set its own goals.

If it just had, if it was by design, it was capable of saying, you know, today I'm going to wake up and I'm going to go learn about physics or I'm going to go build a, you know, wind turbine.

You know, this is clearly going to infringe on sort of human will.

So, you know, autonomy, goal setting, self-improvement, you know, these are big foundational pieces.

Those capabilities aren't quite there in the models yet.

But you can certainly see them coming in the next three to five years.

And so I don't necessarily think we need to sort of snap to regulation, but I think it's part of the discussion that we need to start having as a species because these are very, very powerful systems that we're bringing into the world.

And most importantly, they're going to be very cheap.

And many of them are going to be available in open source.

So we have to contend with the fact that this isn't just about managing and governing the frontier labs.

It's about managing the entire ecosystem and how everybody plays with these things.

You know, I think that there's risks up and down the stack.

The risks of open source is that anybody can copy it, reproduce it, experiment with it with basically zero sort of oversight.

And that's a good thing today.

You know, people can be creative and productive on their own laptop.

That's awesome.

But in 10 years time, when if the models have continued to get as good by 2035 as they did between 2015 and 2025, you know, people struggle to intuit the impact of an exponential.

You know, that isn't just linear.

That is basically a doubling of compute every 18 months for the next 10 years.

So you're talking, you know, six, seven doublings over that time period.