8 Predictions for the Era of Continual Learning

episode

Dwarkesh Podcast 8 min 1 speaker 7 chapters transcribed 1 month ago

▲ 0

Transcript

jump: chapters · speakers · find in transcript

Transcript

Transcript generated automatically by AI and may contain errors.

Why is true continual learning essential for AI performance?

Dwarkesh Patel 0:00

So I've explained elsewhere why I think actual continual learning is needed. I don't think you can have AIs that perform whole jobs as competently as humans if they are forced to just write markdown files from session to session. Just to give an illustrative example, imagine if this is the way that students learn to play the saxophone. So you have one student, he's never played the saxophone before, he goes into the music hall, he tries to play it, of course, this is his first time, so he fails. And he writes down a bunch of notes about what went wrong. The next student who's waiting outside the music hall, he comes in, he reads all these notes. He's also never played, so of course he messes up, and he continues to add on to these notes.

Dwarkesh Patel 0:33

And you have an infinity of students who are outside the music hall who keep writing notes to the next person. I don't think there's any sequence of text they could write to each other that would allow the subsequent student to just nail the saxophone from the first try. At some point, you actually have to accumulate the relevant experience into your Brain. I think the same thing will be true for a lot of skills that we want AIs to actually accumulate from all the different workplaces in which they're deployed. Okay, so what changes once we have actual continual learning? One, I think that a lot of the proposals that have been put forward about regulating AI assume that you train a model and then you deploy it.

Dwarkesh Patel 1:09

And therefore, if you run a bunch of checks on the model before it is deployed, we can make sure that it's not gonna aid in cyber

How will continual learning reshape AI safety regulations and model inspections?

Dwarkesh Patel 1:15

Attacks or do something crazy. I don't think this assumption necessarily makes sense in the future. And this is one of the many reasons I'm actually kind of worried about locking in some kind of safety regulatory regime right now, because we don't know what kind of technology we're going to be dealing with even within a year, let alone within five years or 10 years. What if the model is improving every single day based on the millions of sessions of work it does in that day? If that happens, we could potentially be locked. In an archaic and potentially counterproductive approach to dealing with the threats from AI. To the extent the government wants some way to do some kind of safety evaluation on model providers, I think it would make more sense to do monthly or quarterly wrist inspections rather than trying to single out some special moment that occurs after training is done, but before deployment begins, because that will not be a meaningfully distinct category in the future.

Dwarkesh Patel 2:02

Two, how the labs do technical alignment. Would probably totally need to change. Right now, a lot of research is focused on the question of how we make sure that a frozen set of weights behaves well during deployment. But I'm not aware of much research on the question of how we make it so that even with constant wait updates, the AI system never falls prey to jailbreaks or changes into a deceptive or evil persona. And if AIs are consolidating learnings between users as well, how do we prevent Users from injecting backdoors or some kind of malicious inclination into the base model. In some sense, this is actually kind of what the human alignment problem is, right? Humans improve in a self-directed way.

What new challenges does continual learning create for technical alignment and jailbreak resistance?

Dwarkesh Patel 2:40

If you have kids, I don't have kids, but I imagine this is what happens. If you have kids, they go out, they learn new things. Sometimes they go crazy. They get one-shotted by crazy ideologies, they take the wrong drug, they become super weird. But you hope that you've given them enough common sense. Sense and basic values that they improve as people in a self-directed way without ending up with some super weird beliefs or some misanthropic ideas. Three, the diversity of AI minds will increase. Right now, there are less than five prominent AI minds, by which I mean the base models, which are served to millions or hundreds of millions or billions of users at once. And they're all quite similar to each other, by the way, because they've also been.