Andrew Lee
speaker
2,833 appearances
3 recordings
1 series
first heard Mar 2025
last heard 15 May
Andrew Lee’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 2 in all, peaking in May 2026 with 1.
Appearances
So it's
It's all new.
Yeah.
So caching has actually become a much bigger deal because now that like the real context is in the file system, there's just a lot more tool calls that need to be done to do the basic operations of the agent because you're loading in a bunch of files and stuff.
And so we really have to make that caching work if we don't want this to be like crazy expensive.
So that's that's been very much at the forefront.
We we came up with a new approach to context management.
that we shipped in December that basically works like this.
You take your whole chat history.
And you put it in the file system.
So it's all accessible in the file system.
And then you find a way to summarize that whole history in kind of a fixed length number of tokens by having recent stuff be included in like sort of high granularity, like the the last thing you sent, you'll probably have most of it or all of it there.
And then older things basically have like decreasing fidelity as you go back.
So if you have a very long
chat, the the stuff in your current turn, like the current thing that's running, it's probably all going to be there, including like all the thinking blocks and all the tool call responses and and you know all the files and things are probably going to be sent to the LLM for bit, depending on how long the run is.
But for most kind of short runs that'll be the case.
The previous turn will probably mostly be there.
You're going to have the full user message, you'll probably have the y the assistant response, you'll probably have
The tool call arguments, you'll probably have the tool call responses, you'll probably have the thinking blocks.
But as you go farther back, we start stripping the thinking blocks.
Showing 61–80 of 2,833 · page 4 of 142
← Previous
Next →