Rethinking Model Size: Train Large, Then Compress with Joseph Gonzalez - #378

episode

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

Related

recent transcribed audio

Reacting To People’s Diabolical Revenge Stories 18 min · 22 Sep 2026 · The Joe Santagato Show

5 On It & Tim Jenkins breaks down Bears tape (Hour 3) 39 min · 22 Sep 2026 · Rahimi & Grote Show

Patrick Mannelly talks Bears' loss & Ian Happ talks Cubs' outlook (Hour 2) 44 min · 22 Sep 2026 · Rahimi & Grote Show

'Lanterns' Episode 6 Deep Dive | House of R 1h 41m · 22 Sep 2026 · House of R

Trump's 29% disaster grows as he's in love with Mamdani 1h 12m · 22 Sep 2026 · The David Pakman Show

5 On It: Is the idea of Caleb Williams starting against the Eagles conceivable to you? 16 min · 22 Sep 2026 · Rahimi & Grote Show

Report any issue

Source

Recorded25 May 2020

Publisherfeeds.megaphone.fm ↗

Handlerec:12496160

Show notes

Today we’re joined by Joseph Gonzalez, Assistant Professor in the EECS department at UC Berkeley. In our conversation, we explore Joseph’s paper “Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers,” which looks at compute-efficient training strategies for models. We discuss the two main problems being solved; 1) How can we rapidly iterate on variations in architecture? And 2) If we make models bigger, is it really improving any efficiency?

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Sign in