Tag: #sparsity
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
Making Inference Fast — Quantization, Sparsity, and Dataflow from a Hardware Lens
We break down the cost structure of inference through the memory-wall lens, then connect quantization (INT8/FP8/FP4), structured sparsity (2:4), dataflow architectures, operator fusion, batching and KV caching into one p
2026-06-16 · 18 min read #inference#quantization#sparsity#dataflow#gpu