Tag: #pytorch
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 23 posts
Complete Guide to Multi-GPU Distributed Training: DDP, FSDP, DeepSpeed
Systematically analyze the core components of multi-GPU distributed training including DDP, FSDP, and DeepSpeed ZeRO based on PyTorch official documentation, with practical setup instructions.
2026-03-01 · 21 min read #gpu#cuda#distributed-training#deep-learning#pytorchMastering Slurm: A Practical Guide to the HPC/AI Cluster Workload Manager
A comprehensive, hands-on guide to the Slurm workload manager. Covers architecture (slurmctld/slurmd/slurmdbd), core concepts (Partitions/QoS/Fairshare), essential commands (sbatch/srun/salloc), GPU scheduling (GRES/MIG/
2026-03-01 · 15 min read #slurm#hpc#gpu#distributed-training#clusterCUDA Hands-on Complete Guide: Everything About GPU Computing
A comprehensive guide covering NVIDIA CUDA from fundamental concepts to GPU architecture, C/C++ kernel programming, memory optimization, Python integration (PyTorch, Numba, CuPy), Multi-GPU training, profiling, and troub
2026-03-01 · 51 min read #cuda#gpu#nvidia#deep-learning#machine-learning