Benedict Evans

speaker

941 appearances 1 recordings 1 series first heard May 2025 last heard May 2025

Benedict Evans’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

No recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.

Appearances

newest first · ▶ plays the moment

The MAD Podcast with Matt Turck · AI Eats the World: Benedict Evans on What Really Matters Now · 22 May 2025 podcast

And there's

So I think there's a really important

kind of conceptual point around error rates.

Which is well there's we could talk about this all this there's many important conceptual points.

But one I think important conceptual point is that there's an enormous difference between saying that was correct eighty nine percent of the time and now it's correct ninety one percent of the time, on the one hand.

And on the other hand, saying that was wrong and now it's right.

Those are completely different things.

Yep.

And you can draw all the lines on charts you want saying the error rate is going down.

But there's a very broad c and there's a very broad class of use case where you don't care if it's wrong sometimes.

You want something that's roughly right or kind of looks like what the right answer would probably look like.

And maybe there isn't a wrong answer, or maybe you can fix it, or maybe you're not gonna give it to a client and you just you're just brainstorming.

So the broadcast a problem where there isn't necessarily a wrong answer, where this doesn't kind of matter that much at a lower error rate is just better.

And then there's a it's like a faster chip.

The chip's faster every year, the error rate's lower every year.

There's another broad class of problem where no, there is a right answer and a wrong answer.

And if you cannot depend on this to be right all the time, as opposed to slightly more of the time, then you either can't use it or you have to use it in very different ways to the ways you could use it if it was always right.

And I think an awful lot of those what those SaaS companies are doing is thinking about A, the difference between a prompt and a product, but B, how do you manage the error rate?

So do you where do you put the probabilistic system and where do you put the deterministic system?

So very crudely, like do you use the LLM to go talk to Oracle?

Showing 81–100 of 941 · page 5 of 48