“Practical Learnings from Synthetic Document Finetuning” by Axel Højmark, Jérémy Scheurer

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded27 May 2026

Publisherfeeds.type3.audio ↗

Handlerec:18596766

Show notes

We've been using Synthetic Document Finetuning (SDF) quite a bit at Apollo Research lately. This post covers a few tweaks to the standard SDF recipe specific to our use cases, plus some general tips and tricks for getting good results. We’re sharing these notes in case they’re useful to others doing research with SDF. 1. What Is SDF? Synthetic Document Finetuning (SDF) is a knowledge editing technique where models are finetuned on LLM-generated documents consistent with a target fact or belief. As described in Slocum et al. (2025), SDF "often succeeds at implanting beliefs that behave similarly to genuine knowledge." These implanted beliefs can generalize to related contexts, are often robust to scrutiny, and form internal representations similar to genuine knowledge. We mostly followed the pipeline described in Slocum et al. (2025) and the safety-research/false-facts repository. The pipeline has several stages: Universe Context: Define a "universe" description where the target belief is true.Fact Extraction: Extract discrete claims from that universe that the synthetic documents will revolve around.Generation: Use an LLM to generate a large, diverse corpus of synthetic documents. This is done by having the LLM first brainstorm document types (blogs, papers, memos), then come up with specific ideas [...] ---Outline:(00:32) 1. What Is SDF?(02:03) Iterating on Universes and Generation Prompts(03:42) 2. Getting Models to Surface the Information(04:14) Dropping the DOCTAG(04:56) Dropping Webtext to Increase Saliency(06:13) Matching the Test Distribution(07:11) Prepending Eval Prompts(08:20) 3. Training Details(08:24) Document Length and Token Counts(09:05) Training for Multiple Epochs(09:37) Running Experiments with LoRA & Tinker(10:33) 4. Dealing with Gibberish(12:14) 5. Evaluating Effects(13:58) 6. Other Explorations(17:06) Acknowledgements --- First published: May 26th, 2026 Source: https://www.lesswrong.com/posts/7zGgFPLaTXJwCJccB/practical-learnings-from-synthetic-document-finetuning --- Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.