Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Ajeya Cotra

๐Ÿ‘ค Speaker
626 total appearances

Appearances Over Time

Podcast Appearances

Astral Codex Ten Podcast
What Happened With Bio Anchors?

If this is true, then all three anchors, seconds, hours, and years, were off by at least an order of magnitude.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

But it turns out that none of this matters very much.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

The highest and lowest anchors cancel out, so that the most plausible anchor, human brain with time horizon of hours to days, is around the average.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

If you remove all of the other anchors and just keep that one, the model's estimates barely change.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

But also, we're talking about crossing 12 orders of magnitude here.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

The difference between the different time horizon anchors doesn't register much on that level, compared to things like algorithmic progress, which have exponential effects.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Maybe this is the model basically working as intended.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

You try lots of different anchors, put more weight on the more plausible ones, take a weighted average of each of them, and hopefully get something close to the real value.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Bio anchors did.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Or maybe it was just good luck.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Still hard to tell.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Eliezer Yudkowsky argued that the whole methodology was fundamentally flawed, partly because of the argument above, he didn't trust the anchors, but also partly because he expected the calculations to be obviated by some sort of paradigm shift that couldn't be shoehorned into algorithmic progress, like how you couldn't build an aeroplane in 1900 but you could in 1920.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

As of 2026, still before AGI has been invented and we get a good historical perspective, no such shift has occurred.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

The scaling laws have mostly held.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Whatever artificial space you try to measure models in, the measurement has mostly worked in a predictable way.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

There have really only been two kinks in the history of AI so far.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

First, a kink in training run size around 2010.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Here's a graph showing different notable AI models in terms of the training compute used over time, running all the way from 1955 to 2025, and we see that the gradient changes sharply upwards on a line of best fit around all these points at around 2010.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Second, a kink in time horizons around 2024 and the invention of test time compute.

Astral Codex Ten Podcast
What Happened With Bio Anchors?

Here's a graph from Meta.