Regina Barber

Special thanks to Valentina Rodriguez-Sanchez for her beautiful voiceover work and to Daniela Amico for interpreting. This episode was produced by Hannah Chin and edited by our showrunner, Rebecca Ramirez. Tyler Jones checked the facts, and Jimmy Keeley was the audio engineer. Beth Donovan is our senior director, and Colin Campbell is our senior vice president of podcasting strategy.

Special thanks to Valentina Rodriguez-Sanchez for her beautiful voiceover work and to Daniela Amico for interpreting. This episode was produced by Hannah Chin and edited by our showrunner, Rebecca Ramirez. Tyler Jones checked the facts, and Jimmy Keeley was the audio engineer. Beth Donovan is our senior director, and Colin Campbell is our senior vice president of podcasting strategy.

Special thanks to Valentina Rodriguez-Sanchez for her beautiful voiceover work and to Daniela Amico for interpreting. This episode was produced by Hannah Chin and edited by our showrunner, Rebecca Ramirez. Tyler Jones checked the facts, and Jimmy Keeley was the audio engineer. Beth Donovan is our senior director, and Colin Campbell is our senior vice president of podcasting strategy.

I'm Regina Barber. Thank you for listening to Shortwave, the science podcast from NPR.

I'm Regina Barber. Thank you for listening to Shortwave, the science podcast from NPR.

I'm Regina Barber. Thank you for listening to Shortwave, the science podcast from NPR.

You're listening to Shortwave from NPR. It seems like these days generative AI is everywhere. It's in my Google searches. It's suggested as a tool on TikTok. It's running customer service chats. And there's a lot of forms that generative AI can take, like it can create images or video.

You're listening to Shortwave from NPR. It seems like these days generative AI is everywhere. It's in my Google searches. It's suggested as a tool on TikTok. It's running customer service chats. And there's a lot of forms that generative AI can take, like it can create images or video.

You're listening to Shortwave from NPR. It seems like these days generative AI is everywhere. It's in my Google searches. It's suggested as a tool on TikTok. It's running customer service chats. And there's a lot of forms that generative AI can take, like it can create images or video.

But the ones that have been in the news recently, DeepSeek R1, OpenAI's ChatGBT, Google Gemini, Apple Intelligence, all of those are large language models. A large language model is kind of like the predictive text feature in your phone, but on steroids.

But the ones that have been in the news recently, DeepSeek R1, OpenAI's ChatGBT, Google Gemini, Apple Intelligence, all of those are large language models. A large language model is kind of like the predictive text feature in your phone, but on steroids.

But the ones that have been in the news recently, DeepSeek R1, OpenAI's ChatGBT, Google Gemini, Apple Intelligence, all of those are large language models. A large language model is kind of like the predictive text feature in your phone, but on steroids.

That's Ilya Shumailov. He's a computer scientist and he says in order to teach these models, scientists have to train them on a lot of human written examples. Like, they basically make the models read the entire internet. which works for a while. But nowadays, thanks in part to these same large language models, a lot of the content on our internet is written by generative AI.

That's Ilya Shumailov. He's a computer scientist and he says in order to teach these models, scientists have to train them on a lot of human written examples. Like, they basically make the models read the entire internet. which works for a while. But nowadays, thanks in part to these same large language models, a lot of the content on our internet is written by generative AI.

That's Ilya Shumailov. He's a computer scientist and he says in order to teach these models, scientists have to train them on a lot of human written examples. Like, they basically make the models read the entire internet. which works for a while. But nowadays, thanks in part to these same large language models, a lot of the content on our internet is written by generative AI.

In the spring of 2023, Elio was a research fellow at the University of Oxford. And he and his brother were talking over lunch. They were like, OK, if the Internet is full of machine-generated content and that machine-generated content goes into future machines, what's going to happen?

In the spring of 2023, Elio was a research fellow at the University of Oxford. And he and his brother were talking over lunch. They were like, OK, if the Internet is full of machine-generated content and that machine-generated content goes into future machines, what's going to happen?

In the spring of 2023, Elio was a research fellow at the University of Oxford. And he and his brother were talking over lunch. They were like, OK, if the Internet is full of machine-generated content and that machine-generated content goes into future machines, what's going to happen?

Ilya and his team did this research study indicating that eventually, any large language model that learns from its own synthetic data would start to degrade over time, producing results that got worse and worse and worse.

Ilya and his team did this research study indicating that eventually, any large language model that learns from its own synthetic data would start to degrade over time, producing results that got worse and worse and worse.