Tag: #local-inference
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
Running GLM-5.2 on a slow computer — how colibrì streams a 744B model from disk
colibrì is a ~1,300-line pure-C inference engine that runs GLM-5.2, a 744B-parameter MoE model, on a consumer PC with 25GB of RAM. The trick is MoE sparsity plus disk streaming: only ~9.9GB of dense layers stay resident,
2026-07-11 · 5 min read #ai#llm#local-inference#moe#quantization