Rohin Shah
speaker
1,071 appearances
1 recordings
1 series
first heard Jun 2026
last heard 2 Jun
Rohin Shah’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jun 2026 with 1.
Appearances
There probably won't be.
I mean, I do think that to what extent are the AIs proposing new ideas?
Those are the things that I think most influence the hyperbolic growth prediction.
That's one thing you could be looking at.
And I would say that the superhuman coder probably doesn't really hit that bar.
But really what I want to do is just measure AI progress and then notice when it starts accelerating.
And so actually we worked with Epoch recently to develop exactly this kind of a measure.
The paper I think was just released.
It's called Rosetta Stone for AI Benchmarks, probably.
And so essentially, we just take the benchmark scores that a model gets for a wide variety of models, and then we sort of stitch the benchmarks together to get a sort of general capability score for models that applies over the entire range from back in 2020 all the way to models that were released now, even though any individual benchmark would have saturated over a much smaller period.
And then you'd take the capability scores that this measure spits out.
You plot them against the release time for the model.
And these capability scores, we never told when producing them, the statistical model that produces them.
We don't include any information about the release date or any time-based information into it, just benchmark performance.
And nonetheless, when you start plotting this against release time, it's just a nice line.
It's great.
And you see that AI progress mostly looks pretty linear on this particular graph.
And the method is really very simple.
The main thing that it depends on is that we have benchmarks that can capture AI systems' performance.
Yeah, exactly.
Showing 801–820 of 1,071 · page 41 of 54
← Previous
Next →