Tag: #llm

한국어English日本語中文

← All posts

• The Complete Guide to LLM Training Data Preprocessing — From Web Crawls to Token Packing, with the Latest Papers

  Good models come from good data, and good data comes from a preprocessing pipeline. This post walks through the entire pretraining data process step by step — web crawl collection → text extraction → language identificat

  2026-07-09 · 7 min read #ai#llm#data-engineering#preprocessing#training

• LLM Caching, Explained — Why Prompt Caching and Prefix Caches Save You Money

  Why does a matching prompt prefix cut costs to a tenth? The answer lives in the transformer's KV cache. Because attention is causal, the Key/Value vectors of earlier tokens never change no matter what comes after — so th

  2026-07-08 · 7 min read #ai#llm#caching#inference#performance

• The State of LLM Quantization — From GPTQ and AWQ to FP8, MXFP4, and KV-Cache Quantization

  Quantization represents a model's numbers in fewer bits to cut memory and cost. Why quality survives fewer bits (outliers and scaling), how GPTQ and AWQ differ in approach, llama.cpp GGUF k-quants, QLoRA's NF4, the 2026

  2026-07-08 · 7 min read #ai#llm#quantization#inference#optimization

• AI Model Development, Start to Finish — a Realistic Lifecycle from Data to Deployment

  Model development starts not with pretraining but with a decision ladder — does prompting suffice, does RAG suffice, do you need fine-tuning? The eval-first principle of building your test set before your model, data qua

  2026-07-07 · 8 min read #ai#ml#llm#mlops#training

• How to Use OpenRouter — 300+ LLMs Through One API

  How to call 300+ models from 60+ providers through a single OpenAI-compatible endpoint with OpenRouter — from getting a key and your first curl request to SDK drop-in, routing, fallbacks, and streaming.

  2026-07-06 · 8 min read #openrouter#llm#ai-gateway#api#openai-compatible

• LiteLLM in Practice — 100+ LLMs Behind One Interface

  A hands-on quickstart for calling OpenAI, Anthropic, Gemini, Bedrock, OpenRouter, and local models through one OpenAI-shaped interface with LiteLLM — from install to streaming, Router, and the Proxy Server.

  2026-07-06 · 9 min read #litellm#llm#ai-gateway#python#openai-compatible

• Observability Deep Dive: Logs, Tracing, and LLM Monitoring

  From the three pillars — logs, metrics, and traces — and how they correlate through traceid, to the difference between Loki and OpenSearch for logs, OpenTelemetry-centered distributed tracing (Jaeger and Tempo), and the

  2026-07-03 · 14 min read #observability#opentelemetry#tracing#logging#llm

• How to Become an AI Engineer in 2026 — LLMs, RAG, Agents, Evals, and a Career Roadmap

  The complete 2026 guide to becoming an AI Engineer. From how the role differs from ML Engineer and Data Scientist to LLM APIs, prompt engineering, RAG design, agents, LoRA fine-tuning, vLLM serving, evals-driven developm

  2026-07-02 · 26 min read #ai-engineer#career#llm#rag#ai-agents

• Understanding Positional Encoding — From Sine Waves to RoPE

  Starting from why Transformers need positional information, this article explains sinusoidal, learned, and relative positional encodings step by step, then RoPE and ALiBi. It connects length extrapolation and context ext

  2026-06-26 · 17 min read #llm#positional-encoding#rope#alibi#long-context

• Dissecting the Transformer — From Attention to KV Cache

  A from-scratch breakdown of the Transformer: self-attention, multi-head, positional encoding, the FFN, and residual connections with normalization. It connects tensor shapes and parameter counts, causal masking, encoder/

  2026-06-26 · 16 min read #llm#transformer#attention#positional-encoding#kv-cache

• The Evolution of Attention — MQA, GQA, FlashAttention, and Long Context

  We analyze the memory and compute cost of standard attention, then explain how MQA and GQA shrink the KV cache and how FlashAttention optimizes IO. We compare sliding-window and long-context techniques and trace how all

  2026-06-26 · 18 min read #llm#attention#flashattention#gqa#mqa

• Vision LLM Architecture — How an Image Becomes Language

  A vision-language model processes an image with a vision encoder, then passes it through a projector to produce tokens an LLM can read. From patch embedding to arbitrary-resolution handling, we trace the full path by whi

  2026-06-26 · 20 min read #llm#vision-language-model#multimodal#vit#qwen2-vl

• Multimodal Tokenization and Fusion — Turning Images and Audio Into Tokens

  A deep look at how images, audio, and video become tokens and get woven into one sequence with text. We cover patch and VQ image tokenization, discrete-codec audio tokenization, frame sampling, interleaving and separator

  2026-06-26 · 14 min read #llm#multimodal#tokenization#vision-language#q-former

• Doing Investment Research with AI — Smartly, but Carefully

  How to use AI tools to speed up investment research like summarizing filings, screening stocks, and translating, plus the traps of hallucination, staleness, source verification, backtest pitfalls, and data bias. From the

  2026-06-18 · 16 min read #finance#ai#investment-research#llm#due-diligence

• On-Device and Edge AI — When AI Moves Inside the Device

  AI inference, once confined to the cloud, is moving into smartphones, PCs, and embedded devices. This post examines the three drivers (latency, privacy, cost), the rise of NPUs and on-device LLMs, and the investment and

  2026-06-18 · 19 min read #ai#edge-ai#on-device#npu#llm

• Local LLM Inference Optimization — From Quantization to Breaking the VRAM Ceiling

  Privacy concerns, cost pressure, and big-tech fatigue are driving a local LLM revival. We map the entire landscape of local inference optimization: VRAM-first hardware thinking, GGUF and AWQ quantization, llama.cpp vs vL

  2026-06-12 · 16 min read #llm#inference#quantization#llama-cpp#vllm

• Building an LLM from Scratch — A Stanford CS336 Style Learning Roadmap

  Stanford CS336 (Language Modeling from Scratch) keeps hitting the Hacker News front page, and from-scratch LLM building is hot again. This post dissects the full curriculum — tokenizers, attention, distributed training,

  2026-06-12 · 15 min read #llm#transformer#cs336#deep-learning#tokenizer

• The Rise of Diffusion LMs — Can They Become an Alternative to Autoregression

  In June 2026 Google released DiffusionGemma, and text diffusion models are lighting up GeekNews and Hacker News. We critically analyze the structural limits of autoregressive generation, the principles of mask-based deno

  2026-06-12 · 17 min read #llm#diffusion#text-generation#gemma#inference

• Artisans and Builders — Where Developers Should Stand When LLMs Erode Their Careers

  Built around two essays — Artisans and builders from the Stack Overflow blog and the HN-viral LLMs are eroding my software engineering career — this post redefines developer identity in the AI era. It covers the democrat

  2026-06-12 · 22 min read #career#ai#engineering-culture#llm#senior-engineer

• Can an Imitation Machine Discover — The Sutton Provocation and the RL Counterattack

  Rich Sutton, the godfather of reinforcement learning, set the community ablaze in June 2026 by arguing that generative AI trained with supervised learning is an imitation model and therefore limited for genuinely new sci

  2026-06-12 · 22 min read #ai#reinforcement-learning#llm#research#ai-agent