Tag: #continuous-batching
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 3 posts
Inside vLLM (3) — How Continuous Batching Keeps the GPU Busy
Explains why static batching leaves the GPU idle, and how vLLM iteration-level scheduling removes that idle time. Covers the difference in character between prefill and decode, V1 unified scheduler and its token budget,
2026-08-12 · 7 min read #vllm#continuous-batching#chunked-prefill#llm#inferenceComplete LLM Serving Optimization Guide: KV Cache, PagedAttention, and Quantization
A complete engineering deep dive into LLM serving optimization. From the KV cache memory crisis to the virtual-memory revolution of PagedAttention, continuous batching, speculative decoding, quantization techniques, and
2026-03-18 · 30 min read #model-serving#kv-cache#paged-attention#vllm#quantizationComplete Guide to vLLM Production Serving Optimization: From PagedAttention to Kubernetes Deployment
A comprehensive production-focused guide covering vLLM core architecture including PagedAttention, optimization techniques such as Continuous Batching, Tensor Parallelism, Speculative Decoding, and Prefix Caching, detail
2026-03-07 · 18 min read #llm#vllm#paged-attention#continuous-batching#tensor-parallelism