Dario Amodei

Most prominently, the gene synthesis industry makes biological specimens on demand, and there is no federal requirement that providers screen orders to make sure they do not contain pathogens.

An MIT study found that 36 out of 38 providers fulfilled an order containing the sequence of the 1918 flu.

I am supportive of mandated gene synthesis screening that would make it harder for individuals to weaponize pathogens in order to reduce both AI-driven biological risks and also biological risks in general.

But this is not something we have today.

It would also be only one tool in reducing risk.

It is a complement to guardrails on AI systems, not a substitute.

The best objection is one that I've rarely seen raised.

That there is a gap between the models being useful in principle and the actual propensity of bad actors to use them.

Most individual bad actors are disturbed individuals, so almost by definition their behavior is unpredictable and irrational.

And it's these bad actors, the unskilled ones, who might have stood to benefit the most from AI making it much easier to kill many people.

Just because a type of violent attack is possible, doesn't mean someone will decide to do it.

Perhaps biological attacks will be unappealing because they are reasonably likely to infect the perpetrator, they don't cater to the militarist-style fantasies that many violent individuals or groups have, and it is hard to selectively target specific people.

It could also be that going through a process that takes months, even if an AI walks you through it, involves an amount of patience that most disturbed individuals simply don't have.

We may simply get lucky and motive and ability don't combine, in practice, in quite the right way.

but this seems like very flimsy protection to rely on.

The motives of disturbed loners can change for any reason or no reason, and in fact there are already instances of LLMs being used in attacks, just not with biology.

The focus on disturbed loners also ignores ideologically motivated terrorists, who are often willing to expend large amounts of time and effort, for example, the 9-11 hijackers.

Wanting to kill as many people as possible is a motive that will probably arise sooner or later, and it unfortunately suggests bioweapons as the method.

Even if this motive is extremely rare, it only has to materialize once.

And as biology advances, increasingly driven by AI itself, it may also become possible to carry out more selective attacks, for example, targeted against people with specific ancestries, which adds yet another, very chilling, possible motive.