Rob Wiblin

And if always the recommendation is the same or always, in practical terms, the output is the same thing from our point of view, then that strongly suggests that the tail isn't containing any important information.

It's not containing a second set of reasoning that could affect the ultimate outcome.

Would this be the process of basically asking it to come up with its own non-human readable language?

Okay, and your theory for that is...

pre-training just packs an enormous punch.

It's an enormous amount to shape them.

So they're really good at English.

They're really good at human language.

And if you ask them to come up with another, you know, their own internal different language, in theory, surely there is a better language for reasoning, but they're not able to bring along everything that they've learned from pre-training in the same way.

They're having to start from scratch.

And so at least at this point, that comes out substantially behind where they just are now using English or whatever other human language.

Yep, that's exactly right.

If you think that this opaque serial depth or the fact that they don't have a very great serial depth without us being able to look at it, if that is so key to our ability to monitor them and ensure that they're basically aligned or not doing anything too harmful, is that a potential kind of governance target for GDM that you could have some internal policy saying, I mean, it sounds like there's not huge incentives yet to violate that anyway, but let's say in future you could get better performance at some point or at some point there'll probably be a crossover.

Yeah.

you could still have an internal governance standard saying, well, they can't think for more than like this amount or they can't have this many thoughts one after another before someone would in principle be able to scrutinize it because it actually just would be dangerous to exceed that.

So Gemini 3 Pro came out not that long ago.

The AI safety blogger, Javi Masvic, who was on the show a couple of years ago, he had a bunch of fairly critical things to say on his blog about the frontier safety report that came out, I think, simultaneously with the launch.

He, I guess...

Broadly speaking, he was worried that GDM was basically hiding a bunch of information that he thought would be inconvenient or create PR problems or regulatory problems for DeepMind if it was more salient and more easy to read.

There's a whole lot of different things.