Rory Stewart

Yeah, I think there's a couple of things.

So on the software side, I think here's one kind of regulatory approach.

And ideally, this would be agreed, at least among all the kind of democratic allied countries, is to say every company producing frontier models, like the most powerful AI systems, needs to have what's called a model spec.

That's just an English language document that says, look, this is how we want our AI to behave.

Such that, for example, if you type in, like, please design me a bioweapon, it will say,

No, sorry, I can't do that.

That's one thing you can write in.

The model spec has to agree to, had to abide by certain foundational principles.

Well, in the case of bioweapons in particular, this is just a lose-lose for everyone.

I mean, because bioweapons, it's very hard to release something that doesn't come back to you.

So the biggest risk is from rogue states like North Korea, where they've got so little power that making crazy reckless threats can actually be quite powerful.

But look, there's going to have to be some huge bargaining over what are in those principles and what have exceptions for national security.

But just briefly, I think, yeah, on the software side, having this written document and then saying any frontier model has to be able to demonstrate that AI will behave in line with the specification, including in very unusual scenarios.

And that means you have to show that it hasn't been sabotaged, whether internally or via foreign interference, and the AI hasn't formed goals of itself.

So you know that it's going to generalize well.

Well, he was in the past.

Is he not anymore?

Not anymore.

But the thing that's weird, so I mean, yeah, I've written about long-termism, the importance of taking future generations seriously, looking very far ahead.

The thing that's just strange about the world now is that all these things I'm talking about, I'm thinking the next five years, as in very significant chance that we start on this takeoff of AIs building better AIs and this huge leap forward in technological ability in the next year or two even.