80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
Sure.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
But nevertheless, they created lots of designs for putative ricin.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
Now, unlike the Evo case, they did not, in fact, make these in a lab because this would deeply contravene international law.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
It would contravene the Biological Weapons Convention.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
It would contravene an awful lot of national laws.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
I think they were based in the US.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
But what they did do is they can use other tools to estimate what's in silico, other tools to predict, would this thing probably function?
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
And through a lot of sort of careful design, they got to putative sequences that are different than current ricin, so modified ricin, however, that were coming out as very likely still, in fact, to work.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
And then the ones that they guessed that were likely to be functional, they sent off to gene synthesis companies.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
including ones that do industry best practice screening.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
So screening that is meant to detect if a customer is wishing to order ricin or part of the smallpox genome, the company is meant to refuse to do so, in fact, flag this order and potentially even report it.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
And they got them through.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
because they'd modified it enough that the existing screening systems didn't spot the change.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
They had what's called obfuscated the design, but they kept the design true enough to the underlying biology that they are pretty sure that this would in fact work.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
This is never going to be as good as an experiment where you could actually prove that rice and wood function, but that would be deeply unethical.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
So that can't happen.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
This is really the best sort of proxy experiment we can do.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
There's a reason it was written up in Science, you know, one of the top journals.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
That deeply worried me because this is something that I and others in the community, you know, sort of worried about these sorts of risks.
80,000 Hours Podcast
AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR
have been thinking about for a number of years.