Lance Martin
speaker
624 appearances
1 recordings
1 series
first heard Sep 2025
last heard Sep 2025
Lance Martin’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
I'd used Anthropic's explicit caching header in the past.
I do think an important and subtle point here is that caching doesn't solve the long context problem.
So it, of course, solves the problem of latency and cost.
But if you still have 100,000 tokens in context, whether it's cached or not, the LM is utilizing that context.
This came up, I actually asked Anton this in their context fraud meetup or in their context fraud webinar, and they kind of had mentioned that the characterization of context fraud that they made, they think they would expect to apply whether or not using caching.
Caching shouldn't actually help you with all the context problems.
It absolutely helps you with
and
You know, that's a great point.
It actually echoes some of Walt and Dan's points from cognition.
Also that the summarization compaction step is just non-trivial.
You have to be very careful with it.
Devin uses a fine-tuned model for doing summarization within the context of coding.
So they obviously spend a lot of time and effort on that particular step.
And Manus kind of calls out that they are very careful about information loss.
whenever they do pruning, compaction, summarization, they always use a file system to offload things so they can retrieve it.
So it's a good call out that compaction is risky when you're building agents and very tricky.
So that's a good thing to hit, actually.
I maybe think about this on two dimensions, writing memories, reading memories, and then the degree of automation on both of those.
So take the simplest case, which actually I quite like, cloud code.
Showing 381–400 of 624 · page 20 of 32
← Previous
Next →