Tag: #gguf
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 7 posts
Local AI & On-Device LLMs 2026 — Ollama · LM Studio · Jan · Msty · Open WebUI · GPT4All · AnythingLLM · Faraday Deep Dive
By May 2026, local AI is no longer a hobby. An M4 Max MacBook Pro runs Llama 4 Scout 109B MoE at 24 tokens per second. Desktop runtimes like Ollama, LM Studio, Jan, and Msty unify GUI and CLI, while Open WebUI, AnythingL
2026-05-16 · 23 min read #local-ai#on-device-llm#ollama#lm-studio#janEdge AI & TinyML 2026 — LiteRT / ExecuTorch / Edge Impulse / Jetson / Coral / Hailo / Sipeed K230 / llama.cpp / Phi-4 Deep-Dive Guide
A full-stack map of the 2026 Edge AI / TinyML ecosystem — the dual standard formed after TFLite Micro was rebranded as LiteRT and ExecuTorch reached GA, the TinyML cloud workflow created by Edge Impulse, the accelerator
2026-05-16 · 32 min read #edge-ai#tinyml#tflite-micro#litert#executorchDeep Learning Model Quantization Complete Guide: Master INT8, INT4, GPTQ, AWQ, GGUF
A comprehensive guide to mastering deep learning model quantization. Understand the principles from FP32 to INT8 and INT4 quantization, and master GPTQ, AWQ, GGUF, bitsandbytes, AutoGPTQ, and llama.cpp through practical
2026-03-17 · 23 min read #quantization#model-compression#gptq#awq#ggufLLM Quantization Practical Guide: GPTQ, AWQ, GGUF Format Comparison and Precision-Performance Trade-offs
A practical guide to LLM quantization covering GPTQ, AWQ, and GGUF format comparison with precision-performance trade-off analysis.
2026-03-14 · 21 min read #llm#quantization#gptq#awq#ggufLLM Quantization Comparison Guide — GPTQ, AWQ, GGUF, bitsandbytes Practical Application
A comparative guide to LLM quantization techniques including GPTQ, AWQ, GGUF, and bitsandbytes with practical application tips.
2026-03-09 · 24 min read #llm#quantization#gptq#awq#ggufComplete LLM Quantization Comparison: GPTQ, AWQ, GGUF Practical Application Guide
From the core principles of LLM quantization to comparative analysis of GPTQ, AWQ, GGUF, and BitsAndBytes techniques, covering practical application in vLLM and llama.cpp environments and quality-performance trade-offs.
2026-03-06 · 14 min read #llm#quantization#gptq#awq#ggufComplete LLM Quantization Comparison — GPTQ vs AWQ vs GGUF
A comprehensive guide to LLM Quantization — from quantization fundamentals to comparing GPTQ, AWQ, and GGUF methods, vLLM/llama.cpp integration, and practical benchmarks.
2026-03-03 · 6 min read #llm#quantization#gptq#awq#gguf