Module 2: Attention Is All You Need (The Concept)

episode

The AI Concepts Podcast not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

Related

recent transcribed audio

Essentials: How to Assess & Improve All Aspects of Your Fitness | Dr. Andy Galpin 35 min · 17 Sep 2026 · Huberman Lab

#502 – Psychiatry, Insane Asylums, Mental Illness, ECT, Lobotomies, Freud & Jung 3h 43m · 17 Sep 2026 · Lex Fridman Podcast

Your partner wants profit. You want freedom. Now what? 22 min · 16 Sep 2026 · The Optometry Success Podcast

Board of County Supervisors Meeting - Work Session - Sep 15, 2026 3h 17m · 15 Sep 2026 · Prince William County Board of Supervisors

Finance and Budget Committee - Sep 15, 2026 1h 43m · 15 Sep 2026 · Prince William County Finance and Budget Committee

Sept. 15, 2026 Board of Supervisors Meeting 8h 46m · 15 Sep 2026 · Fairfax County Board of Supervisors

Report any issue

Source

Recorded03 Jan 2026

Publisherfeed.podbean.com ↗

Handlerec:15078901

Show notes

Shay breaks down the 2017 paper "Attention Is All You Need" and introduces the transformer: a non-recurrent architecture that uses self-attention to process entire sequences in parallel. The episode explains positional encoding, how self-attention creates context-aware token representations, the three key advantages over RNNs (parallelization, global receptive field, and precise signal mixing), the quadratic computational trade-off, and teases a follow-up episode that will dive into the math behind attention.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Sign in