“An analysis of AI-generated content at the Mechanistic Interpretability Workshop” by Andy Arditi, Ivan Arcuschin

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded14 Jul 2026

Publisherfeeds.type3.audio ↗

Handlerec:19288175

Show notes

Introduction Over the past few years, AI tools have become useful for conducting technical AI research. In the early ChatGPT era (~2023–2024), chat assistants were maybe useful as sounding boards for research ideas, or as editors for polishing a paper draft. In the more recent Claude Code era, competent coding agents can code up and run experiments; with enough direction, they can do much of the technical heavy lifting on a PhD-level research project (Schwartz, 2026); and in well-defined settings, they can autonomously try new things and iterate without a human in the loop (Karpathy, 2026). While these tools can unlock researchers to be more productive and widen their ambitions, they can also be abused – anyone can now hand an agent a research prompt, tell it to run the experiments and write up the results in a LaTeX document, and get back an artifact that roughly resembles a conference paper in form. With this steep of a change in the research process, it seems important to study how it is affecting technical research and peer review. The Mechanistic Interpretability Workshop has run three times in the past two years – at ICML 2024, NeurIPS 2025, and ICML 2026. [...] ---Outline:(00:13) Introduction(04:18) Submissions are growing rapidly(05:45) Individual authors are submitting more papers(06:50) How we measure "AI-generatedness"(08:46) AI-generated writing is becoming more prevalent(09:01) The best papers are still mostly human-written(09:51) Solo-authored and repeat-first-author papers tend to be more AI-generated(10:19) Reviews are increasingly AI-generated too(11:02) AI-generated papers receive higher scores from AI-generated reviews than from human-written reviews(12:14) What do AI-generated papers look like?(14:37) Musings on research in the age of AI(14:41) AI-assisted research is here to stay(16:14) Authors should take responsibility for their work(17:15) How peer review might adapt(18:34) Acknowledgements(18:53) Citation information The original text contained 4 footnotes which were omitted from this narration. --- First published: July 14th, 2026 Source: https://www.lesswrong.com/posts/r7FBQ8XDs6qBYc4K4/an-analysis-of-ai-generated-content-at-the-mechanistic --- Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.