Dwarkesh
speaker
1,722 appearances
5 recordings
1 series
first heard Apr 2025
last heard 25 Nov
Dwarkesh’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 2 in all, peaking in Nov 2025 with 1.
Appearances
Thank you so much.
All right.
Appreciate it.
That was great.
Yeah, I enjoyed it.
Yes, me too.
Hey, everybody.
I hope you enjoyed that episode.
If you did, the most helpful thing you can do is just share it with other people who you think might enjoy it.
It's also helpful if you leave a rating or a comment on whatever platform you're listening on.
If you're interested in sponsoring the podcast, you can reach out at dwarkesh.com slash advertise.
Otherwise, I'll see you in the next one.
Public benchmarks can be useful, but they don't always measure what you think they do.
Take AMI or HMMT, which both contain extremely challenging, competition-level math problems.
Now, LibVox researchers found that some of the larger models were brute-forcing their answers through repeated trial and error, and they were sidestepping the very kind of mathematical reasoning that these benchmarks were designed to test in the first place.
And on top of this, some of the questions had actually leaked into the model's training data.
So when LabelBox researchers swapped in fresh, equally challenging problems, the scores plummeted, and in many cases by more than 40%.
So when a LabelBox customer wanted to improve their model's math capabilities, LabelBox brought in a team of previous Math Olympiad winners to develop brand new AMI and HMMT-style problems that the models couldn't have possibly memorized, along with clever variations that made brute force approaches computationally infeasible for the model.
Labelbox also mapped out the model's strengths and weaknesses.
For example, was it better at domains like combinatorics or algebra?
Showing 341–360 of 1,722 · page 18 of 87
← Previous
Next →