Rob Wiblin

So is the approach that you would take there is take a current frontier model and then do reinforcement learning to get it to speak as if it were a scientist AI?

No.

Okay.

So yeah, the reason I was asking is if we're going to go from a current state-of-the-art agentic model and try to make it more like a scientist AI to make it more honest, how do we do that if not reinforcement learning?

Are you saying we're going to do something more like we get it to predict past events based only on having data from before that time?

Yes.

Oh, okay.

That's how we do it.

Okay.

Yes.

Yes.

So I think I have a decent picture of the predictor model that's taking in statements, throwing out probabilities of them being true.

Is there more that it would be useful for me and other people to have in their minds to picture how this entire system would work where it's not only the predictor, you're building scaffolding around it to give it like partial agency and so on?

There's a longstanding worry that Oracle AIs are structurally disadvantaged, that they're going to be less intelligent, all else equal, because they don't have the option of basically running experiments, of taking actions in order to discover how things work most effectively.

And I think there's other worries along these lines that basically it's the things that make AIs intelligent that make them dangerous and vice versa.

What do you think are the chances that that is true?

I guess if we wind back a year or two ago, we had AI models that were, in a sense, extremely knowledgeable, extremely smart.

But if you just tried to get them to navigate a web page, they would struggle to do it.

It seemed like there's a very big difference potentially between scientific intelligence or ability to predict things versus ability to navigate the world in practical terms.

And it took a lot of extra training, a lot of extra effort in order to get them to be able to take useful actions.