Jeremiah

Welcome to the Astral Codex X podcast for the 1st of March, 2026.

Title, All Lawful Use, Much More Than You Wanted to Know, a guest post by anonymous readers.

This is an audio version of Astral Codex X, Scott Alexander's Substack.

If you like it, you can subscribe at astralcodex10.substack.com.

If you like it, you can subscribe at astralcodex10.substack.com.

Welcome to the Astral Codex X podcast for the 26th of February, 2026.

Title, Next Token Predictor is an AI's job, not its species.

This is an audio version of Astral Codex X, Scott Alexander's Substack.

If you like it, you can subscribe at astralcodex10.substack.com.

1.

In The Argument, link in post, Kelsey Piper gives a good description of the ways that AIs are more than just next token predictors or stochastic parrots.

For example, they also use fine-tuning and RLHF.

But commenters, while appreciating the subtleties she introduces, object that they're still just extra layers on top of a machine that basically runs on next-token prediction.

Here's a comment.

Quote, No, it's just next token prediction on a biased set.

If you fine-tune on a set of recipes, it will be more likely to predict recipes.

If you fine-tune on answers that have been selected by humans to be typical of a helpful assistant, it will be more likely to predict text characteristics of a helpful assistant.

Next token prediction is a structural property, the structural property, of what these models are.

It can't be changed by fine-tuning.

Scott writes, I want to approach this from a different direction.