Dianne Penn

speaker

606 appearances 1 recordings 1 series first heard Jul 2026 last heard 26 Jul

Dianne Penn’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Jul 2026 with 1.

Appearances

newest first · ▶ plays the moment

Lenny's Podcast: Product | Career | Growth · Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future | Dianne Penn · 26 Jul 2026 podcast

Give me what situations this was happening.

Like, what's the exact situation?

Like paragraph, what did you ask?

What was Claude's response?

Going to like that level of detail.

And what I saw was something like 80% of what people meant in the early days for this failure was Claude would not write the right JSON.

And so then, okay, let's generate maybe to start just 30 to 40 examples of when Claude was not doing this thing correctly.

And then that actually is your eval set.

And you can have essentially a prompt and a response.

And if that is not working in the right golden answer that you might have, then that means that the eval essentially is beneficial because it's identifying a pain point consistently.

And so then we added that to our response.

repositories for evals.

And when we have versions of Cloud, we actually run that eval and just check.

I think at this point, it's always 100% or like 99.9.

And so it's no longer a pain point.

But in the early days, it was taking the user feedback, figuring out actually what they mean.

Can we reproduce it?

Is it consistent?

Is it a big issue?

And then figuring out how to

Showing 281–300 of 606 · page 15 of 31