Micah-Hill Smith
speaker
601 appearances
1 recordings
1 series
first heard Jan 2026
last heard 8 Jan
Micah-Hill Smith’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jan 2026 with 1.
Appearances
We're very aware of that.
We have what we call a mystery shopper policy.
And so and we're totally transparent with all the labs we work with about this, that we will register accounts not on our own domain and run both intelligence evals and performance benchmarks.
That's the job without
That's the job.
them being unidentified it and no one's ever had a problem with that.
Because like a thing that turns out to actually be quite a
Good factor in the industry is that
They all want to believe that none of their competitors could manipulate what we're doing either.
That's true.
I mean okay, the the the b the biggest one like that I'll bring off like is more of a conceptual one actually than like direct shenanigans.
It's that the things that get measured become things that get targeted by landscape they're trying to build, right?
Exactly.
So
That
doesn't mean anything that we should really caution out against.
Like I'm not talking about training on test set, but if you know that you're going to be great at not a particular thing.
If you're a researcher, there are a whole bunch of things that you can do to try to get better at that thing that preferably are going to be helpful for a wider range of how actual users want to use the thing that you're building, but will not necessarily do that.
So
For instance, the models are exceptional now at answering competition maths problems.
Showing 121–140 of 601 · page 7 of 31
← Previous
Next →