Daniel Whitenack
speaker
2,536 appearances
14 recordings
1 series
first heard May 2026
last heard 10 Sep
Daniel Whitenack’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 14 in all, peaking in Jul 2026 with 4.
Appearances
Where there's an example input, there's some expected output, right?
podcast-host
And you run your model with the input, see what the model produces as the output, compare the two, and then there is some scoring metric to uh tell you how how well you did.
podcast-host
And and so these are these are benchmarks, MMLU, uh
podcast-host
SWE, uh uh for the coding side, there's scoring arenas that are that are interesting if you if you look up graduate reasoning, math benchmarks, you know, all all of these sorts of things.
podcast-host
Um and generally if we look at kind of the more recent models that are closed, you know, Opus GPT five point five, blah blah blah, and the and the open or the uh the closed models, those um and then the open models that are more recent, so things like Kimi models and Quinn 3.5 and these models
podcast-host
That are that are more recent on the open side.
podcast-host
Generally, there is still a gap in terms of like those models being a few percentage points to maybe a little bit more scoring lower on these benchmarks than the state of the art closed source models.
podcast-host
I guess my
podcast-host
I don't know if it's a I don't know if it's a hot take, Chris, but maybe my divisive question would be who cares about benchmarks and why does that why does this even matter?
podcast-host
Or or maybe another way to put it is that's that's fun to think about all of these benchmarks, but it has nothing to do with the real world.
podcast-host
True.
podcast-host
That's it's a good point.
podcast-host
think
podcast-host
I love where you're going with this because you mentioned some things like claud quad design and we talked about clod code and the in a previous episode, which people can listen to, which was super interesting to get into the kind of innards of that.
podcast-host
But I think I think where I'm at on this whole question is probably a year ago, I would have really been rooting
podcast-host
for, you know, the open model to to win and we're gonna get there and it's gonna be all open models forever.
podcast-host
I think now I'm at a point where my mind isn't really thinking about models.
podcast-host
And the question of what model you're using is
podcast-host
You're a point.
podcast-host
Kind of irrelevant.
podcast-host
Showing 2421–2440 of 2,536 · page 122 of 127
← Previous
Next →