Andrew Lee
speaker
2,833 appearances
3 recordings
1 series
first heard Mar 2025
last heard 15 May
Andrew Lee’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 2 in all, peaking in May 2026 with 1.
Appearances
We start stripping the tool call responses, or at least truncating the tool call responses, and then stripping them.
We start truncating and then stripping the tool call arguments.
Then we start collapsing tool calls.
And then we start shrinking down the assistant messages.
And then finally we get to some LLM-based summary.
And we do this in buckets, moving back so that we can have sort of a minimal impact on caching.
Basically, you want to like avoid messing with prefixes as much as you can.
So you you as you go back, basically you get into these buckets where we have like different levels of compression.
And those buckets, as they get older, tend to like get added to very slowly.
And then once they hit a certain threshold, like we shrink them down.
And this system has actually worked basically like pretty well.
And the core thesis basically is like you you generally care a lot more about recent stuff and you trust the agent to go and like look things up when it needs to.
And I would say it's not perfect.
We do definitely do have people say that agents forget things.
It definitely does still cost us a lot of money to run.
But I think it's generally worked.
And our plan is to
double down on this type of architecture.
And we have lots of ideas how to improve this, but the the basic approach of these sort of like like decreasing fidelity as you go back and these like bucketed cash aware chunks, I I think is the right approach.
Well there there's two pieces.
Showing 81–100 of 2,833 · page 5 of 142
← Previous
Next →