Tag: #adamw
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
Mathematical Optimization for Machine Learning: From Adam to Convex Optimization and ZeRO
A comprehensive guide to ML training optimization covering SGD, Adam, AdamW, Lion optimizer, cosine annealing scheduling, Focal Loss, InfoNCE, and ZeRO optimizer.
2026-03-17 · 12 min read #optimization#adamw#learning-rate-scheduling#loss-functions#convex-optimization