Tag: #llm

한국어English日本語中文

← All posts

• How to Read a Hugging Face Model Card: What to Trust and What to Doubt

  Six fields on a Hugging Face model card actually carry decision weight: license, parameter count, context length, intended use, limitations, and training data. This post explains what each field means and what it hides,

  2026-08-12 · 7 min read #ai#llm#huggingface#open-source-llm#model-card

• Embeddings and Rerankers: What Actually Matters in RAG

  What decides your embedding model in a RAG build is not a leaderboard rank but dimensionality, maximum input length, prefix conventions, multilingual coverage, and the division of labor with a reranker. This post lays ou

  2026-08-12 · 7 min read #ai#llm#huggingface#open-source-llm#embedding

• Inside vLLM (5) — Prefix Caching, and Why System Prompt Design Is Performance

  A rundown of when vLLM prefix caching hits and when it misses, based on the official design document. Covers how block hashes chain to the block before them, why only full blocks get cached, how a single timestamp placed

  2026-08-12 · 7 min read #vllm#prefix-caching#kv-cache#prompt-engineering#llm

• Inside vLLM (3) — How Continuous Batching Keeps the GPU Busy

  Explains why static batching leaves the GPU idle, and how vLLM iteration-level scheduling removes that idle time. Covers the difference in character between prefill and decode, V1 unified scheduler and its token budget,

  2026-08-12 · 7 min read #vllm#continuous-batching#chunked-prefill#llm#inference

• Inside vLLM (7) — Deployment Tuning, Common Pitfalls, and OOM Triage

  A practical order for tuning a vLLM deployment. Covers what gpumemoryutilization actually sets, when to use tensor parallelism versus pipeline parallelism, how to choose quantization and a KV cache data type, and a diagn

  2026-08-12 · 10 min read #vllm#gpu#quantization#tensor-parallel#llm

• Inside vLLM (2) — Why PagedAttention Splits the KV Cache Into Pages

  Explains why PagedAttention splits the KV cache into fixed-size blocks, verified against the original paper (arXiv:2309.06180) and the official vLLM design documentation. Covers the internal and external fragmentation th

  2026-08-12 · 7 min read #vllm#paged-attention#kv-cache#llm#gpu

• Inside vLLM (6) — Context Window vs max_model_len vs max_tokens, Fully Explained

  If the difference between context window and maxtokens has been confusing, this single post sorts it out. Covers what each of the following actually limits, with a comparison table and real error messages: the context wi

  2026-08-12 · 12 min read #vllm#context-window#max-model-len#max-tokens#llm

• Inside vLLM (4) — The Scheduler and Preemption, Where Throughput Collapses

  Explains what the vLLM scheduler decides on every step, how the waiting queue and running list move, and how preemption, triggered when the KV cache runs short, wrecks latency and throughput, verified against the officia

  2026-08-12 · 7 min read #vllm#scheduler#preemption#kv-cache#llm

• Inside vLLM (1) — The Full Path From One Request to One Token

  Follows the full path a single request takes through vLLM, from arrival to the first token out. Covers what the API server, scheduler, KV cache manager, worker, and sampler each do, and how the process layout splits afte

  2026-08-12 · 6 min read #vllm#llm#inference#gpu#ai-platform

• Text LLM Technical Reports: What to Read, and How to Read Design Decisions Instead of Rankings

  Nine text-LLM technical reports, each verified by opening the arXiv abstract page directly. What DeepSeek-V3, DeepSeek-R1, Qwen3, Gemma 3, Olmo 3, Kimi K2, MiniMax-01, Mellum2 and s1 each did that was new, and which limi

  2026-08-12 · 7 min read #ai-papers#paper-review#technical-report#llm#moe

• A Breach With No Attacker — Why Agent Credentials Deserve Another Look

  Hugging Face disclosed a production breach caused by autonomous agents on 16 July 2026, and about three weeks later OpenAI revealed that the attack had leaked out of its own training environment. This post is not an inci

  2026-08-09 · 8 min read #security#llm#agent#incident-response#credentials

• A Harness Is Not Configuration but a Deployable — The Real Bottleneck of the Self-Improvement Loop

  The harness engineering post Lilian Weng published in July 2026 gives a name to the whole system wrapped around a model and treats it as a single engineering object. Rather than restating that definition, this post cover

  2026-08-09 · 7 min read #llm#agent#harness-engineering#context-engineering#evaluation

• Reasoning Effort Is Not a Model Choice but a Per-Request Deployment Parameter

  The DeepSeek V4 Flash 0731 results page published by ARC Prize carries not one score but three, one per reasoning effort level. This post computes what can actually be read out of those three numbers: that the same step

  2026-08-09 · 8 min read #llm#benchmark#arc-agi#inference#cost

• Coding Agent Spend Is Controlled by Friction, Not by Caps

  Two engineering posts Databricks published back to back in July and August 2026 show that the handling of coding agent spend is moving away from budget caps and toward a gateway plus progressive friction. This post takes

  2026-08-09 · 8 min read #mlops#llm#cost#ai-gateway#developer-productivity

• The Claim of 100x Cheaper Is True Only When the Task Was Narrowed — Verification and Break-Even

  A case study published in August 2026 reports that a 4-billion-parameter-class open model, post-trained with reinforcement learning, matched frontier models on a retrieval task while cutting per-request cost by an order

  2026-08-09 · 8 min read #llm#cost#fine-tuning#retrieval#open-models

• In Eval-Driven Development, the First Thing to Calibrate Is the Judge

  The eval-driven development retrospective Airbnb Engineering published in July 2026 is less a plea to write the eval set first than a plea to earn the right to treat the grading model as an instrument. This post lays out

  2026-08-09 · 9 min read #ai#llm#eval-driven-development#llm-as-judge#evaluation

• What LLMs Cannot Do Is Not the Proof, It Is Setting Up the Premise

  The ICML 2026 position paper Position: LLMs can not jump argues that generative AI has mastered induction and is rapidly conquering deduction, yet remains structurally unable to reach abduction, the act of producing a ne

  2026-08-09 · 8 min read #ai#llm#reasoning#abduction#research

• How to Actually Read a Model Card — Pulling Out What You Need in 5 Minutes

  A model card is written in a way that, read top to bottom, keeps you from finding what you need. The benchmark table takes up half the screen while the license and chat template pass by in a single line. This post flips

  2026-08-02 · 19 min read #llm#huggingface#model-card#license#tokenizer

• How These Models Were Actually Built — Dissecting the 2026 Open-Weight Pipeline

  Reading the cards and technical reports of the open-weight models sitting near the top of Hugging Face as of August 2026, this post lays out the production pipeline in order, from data collection to quantized deployment.

  2026-08-02 · 17 min read #llm#pretraining#moe#post-training#quantization

• What's Trending on Hugging Face Right Now — A Map of August 2026

  As of August 2, 2026, I went through the Hugging Face trending list directly and organized models that are actually usable by purpose. In order — general LLMs, coding, embeddings and rerankers, vision, speech, image/vide

  2026-08-02 · 18 min read #llm#huggingface#open-weights#model-selection#quantization