Illia Polosukhin

And there's this concept of sleeper agents where you can actually train in a specific way the model where it will not show up in benchmarking, it will not show up in normal usage, but in a specific context under specific conditions, it will change its output.

Right.

So you can actually literally train like very specific activations.

There's some research on that.

And so like, again, we have no idea if that was trained like that or not.

I mean, I'm assuming not, but we don't know.

And so it is especially like if you're using it for financial, for medical, for legal, for any of these reasons, like you actually have no confidence in the outcome.

So ideally, we should have a model where we know what went in, all the inputs and how it was trained.

Now, the challenge was that, even the challenge was open source in general, is that there's no monetization, right?

There's no way, if I build really cool model, I spend hundreds of thousands of millions of dollars to train it, there's no way for me to make money of this.

Because it's open source, now everybody's using it, you can not pay me.

So we kind of need a new model

Yeah, exactly.

And it's kind of like... It's a strange model because...

what it sends up is actually kind of like the startups don't want to take on that risk because like, you know, as soon as you target you, like your costs are exploding.

And so you're going to take the open source models that are, don't have this license and the individuals, right.

Who, you know, would want to use this, would just use it locally or whatever through services directly.

Yeah.

And then every country outside of U.S.

will just ignore those rules.