Rob Wiblin

And then the ultimate real world measure is actually just observed productivity, right?

Like if they are seeing internally that they're discovering insights like faster than they were before, then that's a very like late but also very clear signal.

And that's the point at which they should definitely sound the alarm and like we should sort of

know what's happening.

So yeah.

Yeah, I think that...

The response just tends to differ based on the actual information that's being asked for.

So benchmark scores, they already release.

Like I said, they release it at the point of releasing a product, which I think is fine for now.

But I would like to move it to a regime where they release benchmark scores at some sort of fixed cadence, even if they don't have a product release.

Benchmark scores are not considered sensitive information.

But this other stuff that I think is a lot more informative on the margin is much more fraught, right?

They don't necessarily want to share with the world the rate at which they're gaining algorithmic insights because you want to maintain some mystery about that for competitive reasons.

It's risky for you if it's a little bit too fast because then, I don't know, competitors will start...

paying more attention to you and trying to copy you and trying to find out what's going on.

It's also risky for you if it's too slow, because then that's kind of embarrassing.

Yeah, investors lose heart.

And another thing I didn't mention earlier is that I would really like them to be reporting their

most concerning misalignment related safety incidents.

So like, has it ever been the case that in real life use within the company, the model lied about something important and covered up the logs?