Harry Stebbings

speaker

52,669 appearances 213 recordings 3 series first heard Aug 2024 last heard 2d ago

Harry Stebbings’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 164 in all, peaking in Jun 2026 with 26.

newest first · ▶ plays the moment

Value downstream. podcast-host

I'm jumping around, but fuck it. podcast-host

When I was doing the research, I was reading about KV caching and compression as part of this. podcast-host

And I was honestly getting lost, but I was intrigued and digging deeper and deeper. podcast-host

And I was like, why did I not know this before? podcast-host

And so I don't think many will know this. podcast-host

What should we know about KV caching? podcast-host

Why is this important? podcast-host

Can you explain it to me a little bit? podcast-host

What does it mean then when we hear about compression and uncompression of KV caching and potential entropy within the system? podcast-host

Is KV caching the hardest element of building that inference infrastructure? podcast-host

Or is it you name it uh latency SLOs or load spikes or anything else that we could come up with? podcast-host

Is that the hardest? podcast-host

Like what do we not see that we should see? podcast-host

Totally get that. podcast-host

It saves us money because we don't have to use as much compute, but then it's harder from a memory challenge perspective. podcast-host

How do you think about the right logical next step then? podcast-host

If you appreciate the importance of saving on compute, but the challenge of memory with KB caching, what's the answer then? podcast-host

That we just have bigger and bigger memory stacks on chip? podcast-host

What does that look like? podcast-host

Showing 481–500 of 52,669 · page 25 of 2634