Dario Amodei

preventing the elicitation of this kind of information in the face of serious adversarial pressure.

So-called jailbreaks likely demands layers of defenses beyond those ordinarily baked into training.

Crucially, this will break the correlation between ability and motive.

The disturbed loner who wants to kill people but lacks the discipline or skill to do so will now be elevated to the capability level of the PhD virologist, who is unlikely to have this motivation.

This concern generalizes beyond biology, although I think biology is the scariest area, to any area where great destruction is possible but currently requires a high level of skill and discipline.

To put it another way, renting a powerful AI gives intelligence to malicious, but otherwise average, people.

I am worried there are potentially a large number of such people out there, and that if they have access to an easy way to kill millions of people, sooner or later one of them will do it.

Additionally, those who do have expertise may be enabled to commit even larger-scale destruction than they could before.

Biology is by far the area I'm most worried about, because of its very large potential for destruction and the difficulty of defending against it, so I'll focus on biology in particular.

But much of what I say here applies to other risks, like cyber attacks, chemical weapons, or nuclear technology.

I am not going to go into detail about how to make biological weapons, for reasons that should be obvious.

But at a high level, I am concerned that LLMs are approaching, or may already have reached, the knowledge needed to create and release them end-to-end, and that their potential for destruction is very high.

Some biological agents could cause millions of deaths if a determined effort was made to release them for maximum spread.

However, this would still take a very high level of skill, including a number of very specific steps and procedures that are not widely known.

My concern is not merely fixed or static knowledge.

I am concerned that LLMs will be able to take someone of average knowledge and ability and walk them through a complex process that might otherwise go wrong or require debugging in an interactive way, similar to how tech support might help a non-technical person debug and fix complicated computer-related problems, although this would be a more extended process, probably lasting over weeks or months.

More capable LLMs, substantially beyond the power of today's, might be capable of enabling even more frightening acts.

In 2024, a group of prominent scientists wrote a letter warning about the risks of researching, and potentially creating, a dangerous new type of organism, mirror life.

The DNA, RNA, ribosomes, and proteins that make up biological organisms all have the same chirality, also called handedness, that causes them to be not equivalent to a version of themselves reflected in the mirror, just as your right hand cannot be rotated in such a way as to be identical to your left.

But the whole system of proteins binding to each other, the machinery of DNA synthesis and RNA translation and the construction and breakdown of proteins, all depends on this handedness.