Andrew Lee
speaker
2,833 appearances
3 recordings
1 series
first heard Mar 2025
last heard 15 May
Andrew Lee’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 2 in all, peaking in May 2026 with 1.
Appearances
One is
It fairly quickly got large enough that like it started to break down and not be useful.
So we had a lot of agents that were like working for a while and then after a week or two or whatever, they kinda stopped working because the JSON object got really, really large.
The other problem is it is terrible for caching to put mutable state into your system prompt.
Um and caching is very important for cost.
Um and so uh but the benefit of
Of having it in the system prop at all times is you know the the LM doesn't have to reason about how to retrieve it, it's just there, right?
Um and so we looked other ways to do this and we settled up using just a SQL database because it turns out the models are just super good at writing and using SQL.
Like they're they're way better than I am at at this.
Um and if you just tell it, look, you've got a SQL database and you can store stuff in there and can you retrieve it, and then you can give yourself instructions on how to manage
Manage that thing.
Um, it works, it works really well.
Um I I agree this is not the end of the story, right?
So you totally can't have instructions that say, like, you know, every time you get an email, store the list, and then every time you start, load the whole list, and that list could get bigger and bigger and bigger and eventually cause problems.
And so uh we're not done.
We're gonna have to keep working on this.
Um I think there's gonna be, in addition to being you know, even smarter about how to manage.
That SQL data, there's gonna be lots of other tools that we use.
So, you know, as an example, one of the things I'm looking at now is in a world where you've compacted old history, but that old history becomes relevant, um, you could potentially give the agent tools for uncompacting portions of the history, right?
Be like, okay, you know, in this particular case, I really need do need to know what the what the earlier messages were.
Showing 1161–1180 of 2,833 · page 59 of 142
← Previous
Next →