Micah-Hill Smith

speaker

601 appearances 1 recordings 1 series first heard Jan 2026 last heard 8 Jan

Micah-Hill Smith’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Jan 2026 with 1.

Appearances

newest first · ▶ plays the moment

Latent Space: The AI Engineer Podcast · Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith · 8 Jan 2026 podcast

Which shouldn't be that crazy.

But it turns out there are an enormous number of things that you've got to control for.

And I mean, back when we started the website, like one of the reasons why we were we realized that we had to run the evals ourselves and couldn't just take um results from the labs was just that they would all prompt the models differently.

And when

You're competing over a few points then.

You can um answer into the model.

Yeah, that in the extreme.

And like you get crazy cases like back when I'm Googled a Gemini one point oh Ultra and needed a number that would say it was better than GPT four and um like constructed um I think never published like chain of thought examples, thirty two of them in every topic in MMLU to run it to get the score.

Like there are so many things that you've got to do.

They never shipped Ultra, right?

Um this one had never been widely popular.

Yeah.

I mean it I'm sure it existed.

But yeah, so we were

pretty sure that we needed to run them ourselves and just run them in the same way across all the models.

And we were we were also did certain from the start that

You couldn't look at those in isolation.

You needed to look at them alongside the cost and performance stuff.

So like um I mean we're paying for it personally at the start.

So well the numbers weren't nearly as bad a couple of years ago.

Showing 61–80 of 601 · page 4 of 31