Rob Wiblin

Face masks, yeah.

Oh God, okay, yeah.

So to sum up the picture as I vaguely see it, there's a whole lot of things that we can do to try to improve refusal behavior that I imagine with a big push, we could maybe become, the closed source models like Claude could become quite robust against jailbreaks, quite unwilling to help with obvious production of bioweapons.

There we've got a challenge that it might be difficult to get all of the frontier models to have it because we're currently seeing like some companies compete on safety, some companies compete on speed and not having safety.

That's like what they almost view as their comparative advantage.

And so you could just have like, if you have like one model that is incredibly capable that has almost no refusal behavior, then well, it doesn't, you haven't helped all that much.

But setting aside the closed source models where maybe we could pull that off,

With the open source models, it's going to be possible always basically to fine tune them to get over any of this like reluctance that they have to help.

So then the question is like, you have to make them incapable of helping.

And what can we do there?

I suppose we could try to take the knowledge out of the training data so that it's not that they know how to do virology, but they have been told not to do it, is that they simply couldn't help you even if they wanted to.

But there you've got to challenge that the data that you would use to teach them virology probably is public, probably could be harvested off of the internet to a great extent.

And so someone could try to add that knowledge back in to an open source model just before they used it.

Do I understand the broad picture right?

Well, like more hopefully distillation.

So you train a new model that's like a smaller version of the other one, but you make sure that none of the information that goes from the original one to the second one includes anything about biology or virology.

Okay, so access controls have their place.

They're quite useful, potentially.

They can buy us some time.

They can buy us some risk reduction guardrails.