Rob Wiblin

Yeah.

If you had a dedicated group of semi-experts.

Do I understand right from your notes that you think the probability of something going catastrophically wrong increases a great deal at the point that we have, I guess, like AI or laboratories that can work very autonomously or AI agents that can go away and do biological research without having to direct humans or having to have humans in the loop a great deal?

And so why do you think that is such a central issue?

What's the biggest misconception or misunderstanding that AI people in particular have about AI biorisk?

One, I think, is just...

I am just deeply terrified.

You don't shy away from the fact that one of the motivations that you have for working on this is that you worry that...

our vulnerability to pandemics, humanity's vulnerability to biological catastrophes, is one of the ways that AI might gain leverage, that it might basically exploit us to take more power, seize control over things.

What are the ways that you could imagine, or the most likely ways that you can imagine an AI, a misaligned AI doing that in future?

So, yeah, I think Carl Schulman once had this quote that at the point that an AI can develop a single biological weapon, a single pandemic that would kill a large fraction of humanity if it were released, it's basically operating at the level of the nuclear powers in terms of its ability to deter any action to interfere with it.

And just as we really can't do all that much to stop Russia from misbehaving in all kinds of different ways because just the threat of nuclear retaliation is too great.

At the point that an AI could demonstrate to us that it had developed a single pathogen that has a very high rate of spread and it would kill a huge fraction of people infected,

it gets very difficult to know exactly how would you combat this?

I mean, maybe you feel like you have to reach some kind of accommodation or you have to go at it and just hope that you survive.

It's a very grim situation.

I think one of the reasons that, I guess, AI misalignment-focused people

sometimes are not so bought into bio as being such a great focus, trying to improve resilience to bio as being such a great focus, is they think that any AI in this situation is going to be overdetermined that it could do this.

It's going to have such an easy time making

just a whole, many, many different pandemic viruses.