Tag: #vram
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
LLM Inference VRAM Math — the KV Cache Blows Up Before the Weights Do
How to answer "will this model fit on our GPU" with a few multiplications. Weight memory is just parameter count times bytes per parameter, but the thing that actually blocks a deployment is the KV cache, which grows wit
2026-07-26 · 12 min read #llm#inference#vram#kv-cache#quantization