Ali Ghodsi

speaker

877 appearances 1 recordings 1 series first heard Sep 2026 last heard 18 Sep

Ali Ghodsi’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Sep 2026 with 1.

newest first · ▶ plays the moment

But you know, the whole token Max and thing was happening around you know February, March period already it was happening.

Uh so yeah, we just had the luck of being maybe a few quarters ahead uh of folks to see what what was happening here and it was getting out of hand.

Uh so we already had a gateway, so it's called Unity Gateway, where we were already this gateway was being used to provide token capacity.

So you can get OpenAI, Anthropic, Gemini, Grok capacity.

a any customer can come to us and we'll just provide them that capacity because we have a relationship with those and any open source model.

Uh so we started putting in budget constraints in place and giving people warnings like, okay, you have this much of your budget left.

You're getting close to your kind of ceiling.

So we started doing that per person and for group.

And then we started doing great analytics so we could predict exactly where the costs were going.

Uh and then yeah, and then we added smart routers that could actually pick

cheaper models if you're getting close to your budget or if you know you have simple questions.

We started doing that.

We also built a harness called OmniGent, which can multiplex between the different harnesses.

Turns out actually the harness itself matters.

Like you can use the same model but different harnesses, there's almost 2x different cost difference.

Mm.

Even exactly same model.

You know, same version, but different harness, you get two X difference in actual cost.

So if you can change harness, uh you you you can get a lot of leverage in the costs.

So we started using all of this.

Showing 721–740 of 877 · page 37 of 44