Matt Clifford

One, do you think that's true?

And two, given what you said about attackers only have to get lucky once, defenders have to get lucky every time, what do you expect the real world consequences of that to look like, say, a year out?

Well, let's up the ante a little bit on that.

Oh, we're going to up the ante.

Well, so you recently wrote what I think is a brilliant essay that everyone should read on this idea of recursive self-improvement.

So you'll finesse it better than this, but effectively, AI either can improve itself and therefore you could get some sort of intelligence explosion.

We can get into whether that's the right way to

Why would it explode?

So effectively right now, AI progress is bottlenecked on Jack's very smart colleagues and their counterparts.

And they need to keep coming up with good ideas and they need to keep improving the machine.

Jack's written a great essay about the timelines to a machine improving itself, which would remove at least one bottleneck in that process.

Yes.

And so this, to Jack's, I think, really productive analogy of we're building power plants that spit out nuclear bombs, I think you can up the ante and say that is the next level of that.

I suppose some people would say, well, Rory said, we want to make a choice.

There's regulation.

You say, yeah, we should make a choice.

I suppose one more profound worry would be, do we have a choice?

Not in the sense that, you know, in some technical sense, of course, we have a choice.

But there is this sort of, you know, some of my, you know, much more, you know, sort of left-leaning friends would talk about, you know, the sort of techno-capitalism, you know, that actually there is no choice here.

We kind of have to do this at some level.