LessWrong (Curated & Popular)

“Surprising LLM reasoning failures make me think we still need qualitative breakthroughs for AGI” by Kaj_Sotala

17 Apr 2025

Audio

Description

Introduction Writing this post puts me in a weird epistemic position. I simultaneously believe that: The reasoning failures that I'll discuss are strong evidence that current LLM- or, more generally, transformer-based approaches won't get us AGI As soon as major AI labs read about the specific reasoning failures described here, they might fix them But future versions of GPT, Claude etc. succeeding at the tasks I've described here will provide zero evidence of their ability to reach AGI. If someone makes a future post where they report that they tested an LLM on all the specific things I described here it aced all of them, that will not update my position at all. That is because all of the reasoning failures that I describe here are surprising in the sense that given everything else that they can do, you’d expect LLMs to succeed at all of these tasks. The [...] ---Outline:(00:13) Introduction(02:13) Reasoning failures(02:17) Sliding puzzle problem(07:17) Simple coaching instructions(09:22) Repeatedly failing at tic-tac-toe(10:48) Repeatedly offering an incorrect fix(13:48) Various people's simple tests(15:06) Various failures at logic and consistency while writing fiction(15:21) Inability to write young characters when first prompted(17:12) Paranormal posers(19:12) Global details replacing local ones(20:19) Stereotyped behaviors replacing character-specific ones(21:21) Top secret marine databases(23:32) Wandering items(23:53) Sycophancy(24:49) What's going on here?(32:18) How about scaling? Or reasoning models?--- First published: April 15th, 2025 Source: https://www.lesswrong.com/posts/sgpCuokhMb8JmkoSn/untitled-draft-7shu --- Narrated by TYPE III AUDIO. ---Images from the article:

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes

🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Other episodes from LessWrong (Curated & Popular)

Transcribed and ready to explore now

"Ada Palmer: Inventing the Renaissance" by Martin Sustrik

28 Jan 2026

LessWrong (Curated & Popular)

"The inaugural Redwood Research podcast" by Buck, ryan_greenblatt

27 Jan 2026

LessWrong (Curated & Popular)

"Canada Lost Its Measles Elimination Status Because We Don’t Have Enough Nurses Who Speak Low German" by jenn

26 Jan 2026

LessWrong (Curated & Popular)

[Linkpost] "“The first two weeks are the hardest”: my first digital declutter" by mingyuan

20 Jan 2026

LessWrong (Curated & Popular)

"Why we are excited about confession!" by boazbarak, Gabriel Wu, Manas Joglekar

19 Jan 2026

LessWrong (Curated & Popular)

"Backyard cat fight shows Schelling points preexist language" by jchan

16 Jan 2026

LessWrong (Curated & Popular)

View all episodes from LessWrong (Curated & Popular)

Comments

There are no comments yet.

Please log in to write the first comment.

Report any issue

LessWrong (Curated & Popular)

“Surprising LLM reasoning failures make me think we still need qualitative breakthroughs for AGI” by Kaj_Sotala

This episode hasn't been transcribed yet

Other episodes from LessWrong (Curated & Popular)

"Ada Palmer: Inventing the Renaissance" by Martin Sustrik

"The inaugural Redwood Research podcast" by Buck, ryan_greenblatt

"Canada Lost Its Measles Elimination Status Because We Don’t Have Enough Nurses Who Speak Low German" by jenn

[Linkpost] "“The first two weeks are the hardest”: my first digital declutter" by mingyuan

"Why we are excited about confession!" by boazbarak, Gabriel Wu, Manas Joglekar

"Backyard cat fight shows Schelling points preexist language" by jchan

Sign in to Audioscrape

Share this moment