Tag: #llm

한국어English日本語中文

← All posts

• The Conditions Under Which a 9B Fine-Tuned for 500 Dollars Beat the Frontier — And How Narrow They Are

  On July 28, 2026, a post scored 336 points on Hacker News. It reports that Fermisense trained Qwen3.5-9B with GRPO on roughly 500 dollars worth of GPU time and beat five frontier configurations — using the same tools and

  2026-07-31 · 11 min read #ai#llm#fine-tuning#reinforcement-learning#inference-cost

• How a 26B Model Runs in 2GB of RAM — Resident Memory and Working Set Are Not the Same Number

  TurboFieldfare, posted to Show HN on July 29, 2026, claims to run Gemma 4 26B-A4B on M-series Macs in about 2GB of RAM. 14.3GB is installed on disk, the only thing resident in RAM is a 1.35GB shared core, and the expert

  2026-07-31 · 12 min read #ai#llm#quantization#apple-silicon#moe

• GPT-5.6 and the Limits of Price-Performance — How to Find Your Workload's Place on the Curve

  On July 30, 2026, OpenAI cut GPT-5.6 Luna by 80% and Terra by 20%. That is three weeks after the July 9 general availability, and the top-end Sol price is unchanged. Luna went from 1 dollar input and 6 dollars output per

  2026-07-31 · 12 min read #ai#llm#openai#inference-cost#prompt-caching

• A Paper With Fake Authors Got Accepted for an Oral — a Systems Analysis of Peer Review's Failure in the AI Era

  On July 30, 2026, two reviewers disclosed that of the 22 ML conference submissions they reviewed this summer, 15 contained fabricated citations, nonexistent authors, or unmistakable signs of LLM generation. Two of them c

  2026-07-31 · 13 min read #ai#peer-review#research-integrity#llm#academia

• What Transfers Through Distillation and What Does Not — Distilling DeepSeek into GPT-OSS Did Not Bring the Censorship Along

  CTGT's experiment, posted to Show HN on July 30, 2026, reports that distilling financial reasoning from DeepSeek V4 Flash as the teacher into GPT-OSS as the student carried the capability over but not the political censo

  2026-07-31 · 12 min read #ai#llm#distillation#alignment#open-weights

• What DeepSeek V4 Flash Actually Changes — A Cost-per-Capability Question, Not a Leaderboard One

  On July 31, 2026, DeepSeek moved its V4-Flash API into public beta. The architecture is identical to April's preview — 284B total parameters, 13B active MoE, 1M context — and the only thing that changed is post-training.

  2026-07-31 · 11 min read #ai#llm#deepseek#inference-cost#moe

• Building an Internal Knowledge Base on an LLM — Permission-Aware Retrieval, Freshness, and the Eval Set You Need Before Launch

  The account Cerebras published on July 15, 2026 of how they built their internal knowledge base exposes the internals of a system fielding more than 15,000 queries a day from humans, automation, and agents alike. But the

  2026-07-31 · 14 min read #ai#rag#enterprise-search#llm#platform-engineering

• What It Means to Generate 3D CAD From a Prompt — Mesh vs. B-rep, and the Constraint Bottleneck

  A tool called CAID that generates 3D CAD models from a prompt showed up on GeekNews Show GN. The limitations its own creator disclosed summarize the state of this entire field with precision — no automatic dimension veri

  2026-07-31 · 12 min read #ai#cad#llm#manufacturing#geometry

• LLM Inference VRAM Math — the KV Cache Blows Up Before the Weights Do

  How to answer "will this model fit on our GPU" with a few multiplications. Weight memory is just parameter count times bytes per parameter, but the thing that actually blocks a deployment is the KV cache, which grows wit

  2026-07-26 · 12 min read #llm#inference#vram#kv-cache#quantization

• LLM API Cost Optimization — Break-Even Math for Output Tokens, Prompt Caching and Routing

  Cutting an LLM API bill in half is arithmetic, not intuition. Because output tokens cost several times what input tokens cost, the biggest lever is almost always controlling output length, with prompt caching next. This

  2026-07-26 · 12 min read #llm#cost-optimization#prompt-caching#rag#model-routing

• Stabilizing LLM Structured Output — Four Layers of Defense Against Parse Failures

  A layered treatment for the problem where you ask for JSON and get it wrapped in a markdown fence, followed by an explanatory sentence, or cut off at the token ceiling. Classify the failure types first, then stack the de

  2026-07-26 · 12 min read #llm#structured-output#json-schema#constrained-decoding#validation

• When RAG Answers the Wrong Thing — a Debugging Procedure That Separates Retrieval Failure from Generation Failure

  When RAG returns a wrong answer, most teams start by fixing the prompt, but the majority of real causes sit in the retrieval stage. This post starts with the single experiment — hand the model the gold chunk directly — t

  2026-07-26 · 13 min read #llm#rag#retrieval#chunking#evaluation

• How to Stop Evaluating LLMs on Vibes — Sample Size, Judge Bias, and CI Regression Tests

  A team that edits a prompt and ships on "it seems better now" has no way to see the regressions quietly piling up. This post splits evaluation into four layers — assertions, golden datasets, LLM-as-judge, and human revie

  2026-07-26 · 12 min read #llm#evaluation#llm-as-judge#statistics#regression-testing

• RAG · Fine-Tuning · Long-Context — Which One for My Problem: What the Papers Actually Measured, and What No One Measured

  It's the most common question in LLM architecture, yet most answers are decision trees with no sources. This post answers only with what has been measured. That fine-tuning fails at injecting new knowledge has been measu

  2026-07-17 · 23 min read #rag#llm#fine-tuning#long-context#ai

• How to Actually Cut Your LLM API Bill — Why "90% Off From Caching" Is 25% on the Invoice

  A prompt cache read costs one-tenth of the input price. But that does not take 90% off your invoice. Follow the worked example Anthropic publishes in its own docs and, even with the cache fully warm, the total drops only

  2026-07-17 · 22 min read #llm#cost-optimization#prompt-caching#api#ai

• How Much VRAM to Run an LLM Locally — Compute It From Formulas, Not Tables

  The answer to "how many GB does an 8B model need?" is not a table but two formulas. Weights are parameter count times bpw divided by 8; the KV cache is 2 times layers times KV heads times headdim times bytes times tokens

  2026-07-17 · 24 min read #llm#quantization#kv-cache#local-llm#gpu

• Did Context Engineering Replace Prompt Engineering — What's Measured and What Isn't

  "Prompt engineering is dead" appears in none of the primary sources that coined the term. Karpathy listed few-shot examples and task descriptions as components of context engineering, and Anthropic called it "a natural p

  2026-07-17 · 20 min read #llm#context-engineering#prompt-engineering#long-context#ai-agent

• How AI Agent Memory Is Actually Built — Four Designs, and What the Benchmarks Really Prove

  "Agent memory" is not a single technique but a blur of at least four distinct designs — a file scratchpad, summarization/compaction, vector retrieval, and knowledge graphs. This post pins down what each actually does fro

  2026-07-17 · 21 min read #ai#ai-agent#agent-memory#llm#benchmark

• Simulated Customers Never Walk Away — Where LLM User Simulators Inflate Agent Scores

  In τ-bench-style conversational agent benchmarks, the 'user' role is played by another LLM. But this simulator isn't the thing being measured — it's the measuring instrument, and instruments need calibration. Three verif

  2026-07-16 · 21 min read #ai#llm#evaluation#agents#simulation

• PD Disaggregation Doesn't Increase Throughput — What Prefill/Decode Disaggregation Actually Buys You

  PD disaggregation — splitting prefill and decode across separate GPUs — is a design that landed in vLLM, SGLang, and TensorRT-LLM alike in 2026, but wherever you look, all you find are numbers like "2x to 7x." Yet the of

  2026-07-16 · 16 min read #llm#ai#inference#kv-cache#vllm