Tag: #gptq
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 6 posts
GPU Memory Management & LLM Inference Optimization: vLLM, PagedAttention, GPTQ, TensorRT-LLM
A complete guide to LLM inference optimization covering HBM memory hierarchy, KV cache calculations, PagedAttention, GPTQ/AWQ quantization, continuous batching, and vLLM vs TensorRT-LLM comparison.
2026-03-17 · 16 min read #gpumemory#llminference#vllm#paged-attention#gptqDeep Learning Model Quantization Complete Guide: Master INT8, INT4, GPTQ, AWQ, GGUF
A comprehensive guide to mastering deep learning model quantization. Understand the principles from FP32 to INT8 and INT4 quantization, and master GPTQ, AWQ, GGUF, bitsandbytes, AutoGPTQ, and llama.cpp through practical
2026-03-17 · 23 min read #quantization#model-compression#gptq#awq#ggufLLM Quantization Practical Guide: GPTQ, AWQ, GGUF Format Comparison and Precision-Performance Trade-offs
A practical guide to LLM quantization covering GPTQ, AWQ, and GGUF format comparison with precision-performance trade-off analysis.
2026-03-14 · 21 min read #llm#quantization#gptq#awq#ggufLLM Quantization Comparison Guide — GPTQ, AWQ, GGUF, bitsandbytes Practical Application
A comparative guide to LLM quantization techniques including GPTQ, AWQ, GGUF, and bitsandbytes with practical application tips.
2026-03-09 · 24 min read #llm#quantization#gptq#awq#ggufComplete LLM Quantization Comparison: GPTQ, AWQ, GGUF Practical Application Guide
From the core principles of LLM quantization to comparative analysis of GPTQ, AWQ, GGUF, and BitsAndBytes techniques, covering practical application in vLLM and llama.cpp environments and quality-performance trade-offs.
2026-03-06 · 14 min read #llm#quantization#gptq#awq#ggufComplete LLM Quantization Comparison — GPTQ vs AWQ vs GGUF
A comprehensive guide to LLM Quantization — from quantization fundamentals to comparing GPTQ, AWQ, and GGUF methods, vLLM/llama.cpp integration, and practical benchmarks.
2026-03-03 · 6 min read #llm#quantization#gptq#awq#gguf