Tag: #quantization

한국어English日本語中文

← All posts

• Edge AI Complete Guide 2025: On-Device Inference, Model Optimization, TensorRT/ONNX/CoreML

  Everything about Edge AI! On-device inference (TensorRT/ONNX Runtime/CoreML/TFLite), model optimization (quantization/pruning/knowledge distillation), hardware (NVIDIA Jetson/Apple Neural Engine/Qualcomm NPU), Federated

  2026-04-13 · 21 min read #edge-ai#on-device#inference#tensorrt#onnx

• Complete Guide to LLM Fine-tuning with Unsloth 2025: QLoRA, 4-bit Quantization, 2x Faster Training

  Everything about LLM fine-tuning with Unsloth! QLoRA/LoRA theory, 4-bit quantization (bitsandbytes), Unsloth 2x speed secret, Llama 3/Mistral/Qwen fine-tuning, data preparation, training config, VRAM optimization, GGUF/G

  2026-03-25 · 17 min read #unsloth#llm#fine-tuning#qlora#lora

• Complete LLM Serving Optimization Guide: KV Cache, PagedAttention, and Quantization

  A complete engineering deep dive into LLM serving optimization. From the KV cache memory crisis to the virtual-memory revolution of PagedAttention, continuous batching, speculative decoding, quantization techniques, and

  2026-03-18 · 30 min read #model-serving#kv-cache#paged-attention#vllm#quantization

• NPU Deep Dive: How Transformer Architecture Runs Directly on Silicon

  A complete technical breakdown of how NPUs differ from CPUs and GPUs, how every transformer operation maps to hardware, and why LLM inference is memory-bound — not compute-bound. From Apple ANE to Qualcomm Hexagon, Groq

  2026-03-18 · 22 min read #npu#transformer#ai-hardware#quantization#kv-cache

• Deep Learning Model Quantization Complete Guide: Master INT8, INT4, GPTQ, AWQ, GGUF

  A comprehensive guide to mastering deep learning model quantization. Understand the principles from FP32 to INT8 and INT4 quantization, and master GPTQ, AWQ, GGUF, bitsandbytes, AutoGPTQ, and llama.cpp through practical

  2026-03-17 · 23 min read #quantization#model-compression#gptq#awq#gguf

• On-Device AI 2026: Your Smartphone Becomes a Personal AI Server

  On-device AI is the centerpiece of mobile innovation in 2026. With Apple Neural Engines, Snapdragon AI, and Google Tensor optimizations, smartphones now achieve both privacy protection and ultra-low latency while dramati

  2026-03-16 · 9 min read #on-device-ai#edge-inference#apple-intelligence#privacy#mobile-ai

• LLM Quantization Practical Guide: GPTQ, AWQ, GGUF Format Comparison and Precision-Performance Trade-offs

  A practical guide to LLM quantization covering GPTQ, AWQ, and GGUF format comparison with precision-performance trade-off analysis.

  2026-03-14 · 21 min read #llm#quantization#gptq#awq#gguf

• LLM Quantization Comparison Guide — GPTQ, AWQ, GGUF, bitsandbytes Practical Application

  A comparative guide to LLM quantization techniques including GPTQ, AWQ, GGUF, and bitsandbytes with practical application tips.

  2026-03-09 · 24 min read #llm#quantization#gptq#awq#gguf

• Complete LLM Quantization Comparison: GPTQ, AWQ, GGUF Practical Application Guide

  From the core principles of LLM quantization to comparative analysis of GPTQ, AWQ, GGUF, and BitsAndBytes techniques, covering practical application in vLLM and llama.cpp environments and quality-performance trade-offs.

  2026-03-06 · 14 min read #llm#quantization#gptq#awq#gguf

• BitNet Paper Analysis: The Era of 1-Bit LLMs — From Ternary Weights to CPU Inference

  A comprehensive guide analyzing Microsoft Research's BitNet series (v1, b1.58, a4.8, 2B4T), covering ternary weight training principles, the bitnet.cpp inference framework, and real-world benchmarks.

  2026-03-06 · 23 min read #ai-papers#bitnet#1-bit-llm#quantization#model-efficiency

• Complete LLM Quantization Comparison — GPTQ vs AWQ vs GGUF

  A comprehensive guide to LLM Quantization — from quantization fundamentals to comparing GPTQ, AWQ, and GGUF methods, vLLM/llama.cpp integration, and practical benchmarks.

  2026-03-03 · 6 min read #llm#quantization#gptq#awq#gguf

• The Complete Guide to vLLM & Ollama: LLM Serving Engine Setup, Parameters, and Environment Variables

  A comprehensive deep dive comparing vLLM PagedAttention architecture and Ollama local LLM runtime environments. Covers everything from installation, server startup, API calls, key CLI arguments, sampling parameters, envi

  2026-03-01 · 46 min read #vllm#ollama#model-serving#inference#gpu