Brian Fioca

speaker

409 appearances 1 recordings 1 series first heard Dec 2025 last heard 26 Dec

Brian Fioca’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Dec 2025 with 1.

Appearances

newest first · ▶ plays the moment

Latent Space: The AI Engineer Podcast · ⚡️GPT5-Codex-Max: Training Agents with Personality, Tools & Trust — Brian Fioca + Bill Chen, OpenAI · 26 Dec 2025 podcast

And well, I'm sorry, that was a little bit so true.

It was repeated way too many times.

But I I think um there are a lot of academic evals, all right.

There's like sweep ends, there's um other like you name it.

Um but I think there's a slightly lack of evals off the real world, um, on sort of what people care about the most.

And we want to make sure that whatever we're developed.

Developing model wise as well as product wise are aligned and are actually making the most amount of useful impact on this world.

And applied evals is really uh in that direction, uh capturing uh all of those sorts of uh real world use cases and things for us to hill climb uh on together.

I like to think of it as like we have I mean, people say it's a PhD in in an API, right?

But you if you you you know, you hire a PhD student, they don't know how to do the job.

You have to give them a job description.

Okay, that's a prompt, right?

So now you have your policy, uh, and then you have them do the job and they're gonna kind of like flail around, right?

So they need mentorship, they need guardrails, they need evals, performance reviews on how to do their job, the best practices.

And so what we're doing is we're trying to put our models out there and see.

what they're good at, what they're not good at talking to our customers are like, oh, we could really use your model for more things if it could do this one thing.

Here's our eval for, or help us build those evals with you so that we can see where we're deficient and go back and train the model to be able to do that job in the way that we wouldn't normally get to see it form.

Yeah.

How do you do multi turn evals?

So I think that's the really hard thing that I mean, sometimes you need multi turn if it doesn't get around the first go, but it could just get around the first go, then it's no longer multi turn.

Showing 261–280 of 409 · page 14 of 21