Rohin Shah
speaker
1,071 appearances
1 recordings
1 series
first heard Jun 2026
last heard 2 Jun
Rohin Shah’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jun 2026 with 1.
Appearances
And if you restrict them to natural language, that's the part they're smart in, and that leads to better performance.
Yeah, I think that's definitely doable.
I think it would be even better for it to be a broader industry standard.
As a general rule, I tend to favor things that don't require individual companies to unilaterally do stuff.
It's much more stable if you make it an industry standard.
But yes, I think that would totally work.
In fact, I think by the time this podcast comes out, we will have published a paper that does describe how you could calculate this opaque serial depth for any given architecture and has some code for how you could do this for models that are at least the ones that are implemented in JAXX.
which is just a kind of framework that people use to implement models.
Yeah, I...
have so many thoughts on this one.
I guess for most of these, I would say my primary response is that we care about the Frontier Safety Report as a way of saying we have made a formal determination that this model is safe and we're ready to release it from a safety perspective.
I think that aspect of the Frontier Safety Report, great.
For the persuasion one, I'm not that familiar with it.
It's done by a separate team, so I can't say too much about it.
But I expect that the answer is they thought that this was the most important graph, and so they put that one in, and they weren't particularly thinking about how it would be red-teamed by external observers.
And I expect that they will publish a paper at some point, probably in 2026,
that will go into more detail about this.
But what I am confident about is that, you know, they weren't particularly trying to hide any results here.
On the cyber side, I think I'm a little confused by that particular criticism, just because, so you said that, you know, it was something about treating the model hacking the test as a green light instead of a red light.
I'm a little confused by this because we did count those as successes.
Showing 441–460 of 1,071 · page 23 of 54
← Previous
Next →