Rob Wiblin

Do you worry that the scientist AI that you might train using the kind of data that you're imagining

would be kind of incompetent at a practical level because it wouldn't have had that kind of, I guess, unless you did a lot of this sort of work where the experiments were running were like, do you click this button on a particular webpage?

It wouldn't actually learn to do the things that people want the models to be able to do.

Yeah, I feel like a lot of the critiques of this idea, and my question somewhat reflect that, is that people, including me, I guess, have had in mind a vision of an AI that's extremely different in how it's trained, maybe the data that's being used and the structure and the affordances that it has.

And you want to say, actually, we can make it remarkably similar.

We can take almost all of the data that we're using to train current LLMs and just reformat it a bit and then use it again.

We'll use all of the different efficiencies, all of the algorithmic improvements.

We're just going to

We're just going to give it a somewhat different set of inputs and outputs, but it's more or less the same in almost every other respect.

Yes.

And that's why it's so practical.

So the last I saw online, the organization that you're leading, Law Zero, has raised something like $100 million.

And I think most nonprofits year-end would be pretty happy to have raised $100 million or so.

But I guess you're up against organizations that have $100 billion.

So that's the theory of change.

What sort of experiments do you want to run and how much money would you need for them?

I see, okay.

So they've really taught them to the test, is that the idea?

So comparing like for like, if you train a model of the type that you're envisaging versus a standard model using the same amount of data, same amount of compute, what are the considerations as to whether, I guess, we think that the scientist AI would be more honest and safer in terms of capability, both in terms of prediction and agency, would we expect it to be better or worse and how much better or worse?

It's one way of phrasing this that current models as we train them, they're designed to predict what people would say, and they learn to understand something about the truth as a side effect, as an instrumental part of predicting what would be said.