KV cache

concept

15 mentions 3 recordings first heard Mar 2026 last heard 3 Aug ↑13 vs the 6 months before

Key‑value cache mechanism used by large language models

Trend

mentions per week · public audio 7d 30d 6M 12M

Peak in the week of 30 Jul — 13 mentions across 1 series.

newest first · ▶ plays the moment

Ali TahaLatent Space: The AI Engineer Podcast · The Inference Engineering Masterclass — Philip Kiely & Ali... · 1:39:49 · 3 Aug

…You're just going to update the KV cache.…

· Open transcript →

Ali TahaLatent Space: The AI Engineer Podcast · The Inference Engineering Masterclass — Philip Kiely & Ali... · 1:39:44 · 3 Aug

…And in that case, I don't think inference is going to change that much because we still use KV cache and inference.…

· Open transcript →

Ali TahaLatent Space: The AI Engineer Podcast · The Inference Engineering Masterclass — Philip Kiely & Ali... · 1:39:40 · 3 Aug

…And I do see that KV cache is the way forward.…

· Open transcript →

Ali TahaLatent Space: The AI Engineer Podcast · The Inference Engineering Masterclass — Philip Kiely & Ali... · 1:39:15 · 3 Aug

…With KV cache, or rather not KV cache compaction, but……

Ali TahaLatent Space: The AI Engineer Podcast · The Inference Engineering Masterclass — Philip Kiely & Ali... · 1:39:13 · 3 Aug

…KV cache compaction fixes that.…

Ali TahaLatent Space: The AI Engineer Podcast · The Inference Engineering Masterclass — Philip Kiely & Ali... · 1:38:22 · 3 Aug

…Or the other path, which is you do KB cache……

Philip KielyLatent Space: The AI Engineer Podcast · The Inference Engineering Masterclass — Philip Kiely & Ali... · 1:37:46 · 3 Aug

…I think maybe one relevant topic there is your absolute……

15 moments · sign in to read them all