Module 2: The MLP Layer - Where Transformers Store Knowledge

episode

The AI Concepts Podcast not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

Related

recent transcribed audio

Your partner wants profit. You want freedom. Now what? 22 min · 16 Sep 2026 · The Optometry Success Podcast

Board of County Supervisors Meeting - Work Session - Sep 15, 2026 3h 17m · 15 Sep 2026 · Prince William County Board of Supervisors

Finance and Budget Committee - Sep 15, 2026 1h 43m · 15 Sep 2026 · Prince William County Finance and Budget Committee

Sept. 15, 2026 Board of Supervisors Meeting 8h 46m · 15 Sep 2026 · Fairfax County Board of Supervisors

The Art of True Happiness | Dr. Arthur Brooks 2h 49m · 14 Sep 2026 · Huberman Lab

The Chris Wolfe Podcast: The Power of Quality Supplements 48 min · 14 Sep 2026 · EyeCode Media

Report any issue

Source

Recorded06 Jan 2026

Publisherfeed.podbean.com ↗

Handlerec:15137386

Show notes

Shay explains where a transformer actually stores knowledge: not in attention, but in the MLP (feed-forward) layer. The episode frames the transformer block as a two-step loop: attention moves information between tokens, then the MLP transforms each token’s representation independently to inject learned knowledge.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Sign in