Harry Stebbings

speaker

53,185 appearances 215 recordings 3 series first heard Aug 2024 last heard yesterday

Harry Stebbings’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 166 in all, peaking in Jun 2026 with 26.

newest first · ▶ plays the moment

And I was like, why did I not know this before? podcast-host

And so I don't think many will know this. podcast-host

What should we know about KV caching? podcast-host

Why is this important? podcast-host

Can you explain it to me a little bit? podcast-host

What does it mean then when we hear about compression and uncompression of KV caching and potential entropy within the system? podcast-host

Is KV caching the hardest element of building that inference infrastructure? podcast-host

Or is it you name it uh latency SLOs or load spikes or anything else that we could come up with? podcast-host

Is that the hardest? podcast-host

Like what do we not see that we should see? podcast-host

Totally get that. podcast-host

It saves us money because we don't have to use as much compute, but then it's harder from a memory challenge perspective. podcast-host

How do you think about the right logical next step then? podcast-host

If you appreciate the importance of saving on compute, but the challenge of memory with KB caching, what's the answer then? podcast-host

That we just have bigger and bigger memory stacks on chip? podcast-host

What does that look like? podcast-host

Data it needs to make smart decisions. podcast-host

I just want to break some of the things you said up there. podcast-host

You said that you think model sizes will increase. podcast-host

I thought we were all moving to owning our own intelligence, every enterprise having their own smaller model with proprietary data. podcast-host

Showing 1001–1020 of 53,185 · page 51 of 2660