Zershaaneh Qureshi

speaker

728 appearances 5 recordings 1 series first heard May 2026 last heard 27 Aug

Zershaaneh Qureshi’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 5 in all, peaking in Aug 2026 with 2.

Appearances

newest first · ▶ plays the moment

80,000 Hours Podcast · #252 – Owain Evans on accidentally training AI models to be evil · 20 Aug 2026 podcast

You can find more details at 80thousandours.org, so just head over to the site, click on work with us, and just bear in mind that applications close on the 30th of August.

All right, back to the conversation.

Let's push on here.

I know that you've explored quite a few different strategies for mitigating emergent misalignment.

One of those is dilution, where you mix a small batch of misaligned data in with a much larger batch of normal data.

Another one is post hoc fine-tuning, so after the model's already been made misaligned, you do uh

Another phase of training on um a batch of good data to sort of steer its behavior.

Um, and then another one is inoculation prompting, where during training you use a prompt that reframes the bad behaviour as expected or acceptable in a certain context.

So things like writing in secure code is fine here so that the model learns that it can do those individual bad actions without having to change its broader personality.

So I don't want to get too in the weeds here, um, but I do just want to ask, like, what is the headline for what went wrong with all of these strategies?

In what sense did they fail to remove the emergent misalignment?

Mm-hmm mm.

Yeah, I think it would be useful to get an example here.

So one example I found in one of your papers that was just super wacky but also quite illustrative, um, was this attempt to dilute them misaligned data in a case where the bad data mixed in was poisonous fish recipes.

Uh yeah, how did that go?

Yeah, that's so bizarre.

Um, but I see how it's concerning, right?

Because, you know

The safety tests we give these AIs, the behavioral evaluations that we give them, um, are not really likely to pick this kind of thing up, right?

Like it's not the trigger being something sort of fish or maritime related is not necessarily the thing that we're we're definitely gonna test for.

Showing 101–120 of 728 · page 6 of 37