Casey Newton

What was happening in the year 1930?

So this is a new language model, a vintage LLM called Talkie, and it is trained exclusively on data from before 1931.

This is a research project built by three guys, David Duveno, Nick Levine, and Alec Radford, the lead author of the GPT-1 paper, former OpenAI researcher over there.

And this is a fascinating project that has been burning up my timeline this week because this is an experiment in what happens if you only feed a large language model data from before a certain cutoff.

And it said, gay, you're happy?

Yeah, I love this experiment.

I love weird niche language models.

One of my favorite language models of all time was Golden Gate Claude, which was the special version of Claude that was pathologically obsessed with the Golden Gate Bridge.

I would put Taki in that category of an experimental research model that is maybe not all that useful on its own, but helps illuminate something interesting and important about these language models and what happens when you train them in specific ways.

So today we wanted to talk with one of the creators of Taki.

We are going to bring in David Duvenaux.

David is an associate professor at the University of Toronto who researches AGI governance and catastrophic risk mitigation.

And he is one of the co-creators of Taki.

That's true.

David Duvenaux, welcome to HardFork.

Thank you very much, Kevin.

So this project, Taki, is fascinating.

It is a vintage LLM.

Explain why you and Nick and Alec made this thing.

I assume that