Tag: #mlx

한국어English日本語中文

← All posts

• LLM Serving & Local Inference in 2026 — vLLM / llama.cpp / MLX / Ollama / LM Studio / SGLang / TGI Deep Dive

  A map of the 2026 LLM serving and inference landscape. Datacenter camp (vLLM, SGLang, TGI, Triton, TensorRT-LLM), local camp (llama.cpp, MLX, llamafile, Ollama, LM Studio, GPT4All), emerging camp (KTransformers, MLC LLM,

  2026-05-16 · 25 min read #llm#model-serving#inference#vllm#llama-cpp

• Local AI & On-Device LLMs 2026 — Ollama · LM Studio · Jan · Msty · Open WebUI · GPT4All · AnythingLLM · Faraday Deep Dive

  By May 2026, local AI is no longer a hobby. An M4 Max MacBook Pro runs Llama 4 Scout 109B MoE at 24 tokens per second. Desktop runtimes like Ollama, LM Studio, Jan, and Msty unify GUI and CLI, while Open WebUI, AnythingL

  2026-05-16 · 23 min read #local-ai#on-device-llm#ollama#lm-studio#jan

• LLM Fine-Tuning 2026 Deep Dive — LoRA · QLoRA · DoRA · GaLore · Unsloth · Axolotl · TRL · PEFT · MLX-LM Complete Guide

  In 2026, the LLM fine-tuning ecosystem has exploded from the simple adapters of LoRA into a sprawling tree — QLoRA at 4-bit, DoRA with weight decomposition, GaLore with gradient projection. Hugging Face PEFT 0.14 and TRL

  2026-05-16 · 24 min read #llm-fine-tuning#lora#qlora#dora#galore

• AI Inference Engines 2026 - vLLM · SGLang · llama.cpp · TGI · TensorRT-LLM · MLX · mistral.rs · DeepSpeed-MII · Aphrodite Deep Dive

  In 2026, LLM engineering is no longer about which model — it is about which inference engine. We dissect vLLM V1, SGLang 0.4, TensorRT-LLM, TGI 3.x, llama.cpp, MLX-LM, mistral.rs, DeepSpeed-MII, Aphrodite, CTranslate2, E

  2026-05-16 · 20 min read #llm-inference#vllm#sglang#llama-cpp#tgi

• MLX Deep Dive — Apple's ML Framework for Apple Silicon: Unified Memory, Lazy Graphs, and the Mac-Native Flow (2026 Hands-On)

  MLX is the array framework that Apple's ML team — the same people who previously built PyTorch and JAX — rewrote from scratch for Apple Silicon. The core thesis is one sentence: the M-series GPU shares RAM with the CPU,

  2026-05-14 · 19 min read #mlx#apple-silicon#ml-framework#unified-memory#metal

• Running LLMs on Apple Silicon: Inside M4/M5 Architecture for AI Inference

  A deep technical dive into Apple M4/M5 Unified Memory Architecture and its implications for LLM inference. Covers the Neural Engine, MLX framework, llama.cpp Metal backend, real benchmarks, and an honest comparison with

  2026-03-18 · 14 min read #apple-silicon#m5#model-serving#unified-memory#mlx