Rob Wiblin

People can Google and read the blog post if they want.

But a few that stood out to me was...

He was troubled that on persuasion evaluations, you only gave odds ratios rather than absolute levels, so you couldn't tell exactly how persuasive Gemini 3 or Gemini 2.5 was.

On the cybersecurity evals, on his reading, he thought that you treated the model hacking the test rather than solving the test in the natural way as kind of a green light, a reason not to be concerned, rather than a red light, a reason to be more worried.

And in terms of helping people to acquire WMDs, it fell to him, and I think...

I think a lot of people have had this impression, not just about GDM, but about companies in general, that when it seems like the models are starting to approach the line, maybe it's a bit ambiguous whether they're above or below the line that they had six months or a year ago.

It feels like the goalposts kind of shift and the standards rise over time so that always the model is basically acceptable to put out.

And that's what he was worried about was happening here with Gemini 3 as well.

How do you respond to these kinds of objections?

Thanks.

I guess you don't know whether it could have done the harder thing because it might have done it because it couldn't do the harder thing or it might have just done it because it was easier.

I mean, you can understand, to a cynic like Zhu Yi, doesn't it seem overly convenient that the test is good enough to determine that the model is safe, but not good enough for you to include any specific details in the report itself?

I mean, people don't read papers, Rohan.

They read the model card because they're psyched about a new model launch.

I guess is it just impractical on that kind of timeline to write the equivalent of a paper or provide the level of detail and rigor that you would in a paper in the model card?

Okay, and the third concern was that there's this general phenomenon, it seems, that as the models get more capable with each iteration, it feels like the bar for what would be troubling, the bar for what would seem to be unsafe, rises approximately the same as the capabilities have gone up.

What do you make of that?

Yeah, I mean, that speaks to the fact that the purpose of these model cards in your mind is very different than the purpose that they serve in the mind of someone like Shvi, or I guess like commentators in general.

I think Shvi and many people want them to be an accountability mechanism, a mechanism by which

the alarm could be sounded if the models were dangerous or were becoming more dangerous, where GDM would have to reveal that that was the case in the model card because they have to put out these results.