Tag: #llm-serving

한국어English日本語中文

← All posts

• LLM Inference Serving 2026 — Comparing vLLM, SGLang, and TensorRT-LLM

  A clear overview of LLM inference serving in 2026. From core principles such as the difference in nature between prefill and decode, continuous batching, and paged KV cache, to a strengths-and-weaknesses comparison of vL

  2026-06-26 · 15 min read #llm-serving#vllm#sglang#tensorrt-llm#inference

• Making Inference Fast — Speculative Decoding and Throughput Optimization

  From the fundamental reason LLM decode is slow, to how speculative decoding boosts speed, variants such as Medusa and EAGLE, chunked prefill and prefill/decode disaggregation, the latency versus throughput trade-off, and

  2026-06-26 · 13 min read #speculative-decoding#throughput#inference#mlops#latency

• Serving Multimodal LLMs — The New Challenges Image Input Creates

  From how multimodal LLM serving differs from text-only serving, to the added vision-encoder stage, variable visual token counts, prefill cost spikes, the difficulty of multimodal KV cache and batching, latency decomposit

  2026-06-26 · 14 min read #mlops#multimodal#llm-serving#vllm#kv-cache