Rohin Shah

speaker

1,071 appearances 1 recordings 1 series first heard Jun 2026 last heard 2 Jun

Rohin Shah’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Jun 2026 with 1.

Appearances

newest first · ▶ plays the moment

80,000 Hours Podcast · #245 – Rohin Shah on what it's really like to run AGI safety at Google DeepMind (and where I disagree with 'doomers') · 2 Jun 2026 podcast

And so some information about the reasoning that they're doing has to be present in that chain of thought.

That's part number one.

And then part two is that given how we train language models today, the chain of thought is actually legible and understandable to humans.

And so I'll maybe address these two parts separately.

So the first part is,

Actually, the AI really does have to put some information into the chain of thought because there's no other way for it to do the hard reasoning needed to solve difficult tasks.

The basic argument here is what I'll call it opaque serial depth.

The idea is that you can just look at the architecture of an AI system and then say, suppose this AI system has to do its reasoning only via the billions of floating point numbers that exist inside of it.

It can't use the tokens that it's outputting.

Then how many steps of cognition can it do using only these billions of floating point numbers?

And the answer for transformers is actually, actually not very much.

The idea is like basically sequential steps of computation

have to go deeper and deeper in the model.

So the model is made up of a number of layers, and there's just no way for information from a later layer to flow back to an earlier layer, except via going through the tokens in the chain of thought.

And so overall, the opaque serial depth of a transformer is very low.

Why do I expect this to continue?

This is actually a crucial...

It's a crucial aspect of why pre-training can be so efficient.

So GPUs and TPUs are very powerful computationally, but the way that they get this power is by doing a ton of computations in parallel.

So if you need to compute A plus B to get C, and then you need C, the intermediate result, to feed into a further computation that would be serial computation, GPUs and TPUs, not very good at that.

Showing 361–380 of 1,071 · page 19 of 54