태그: #torchtitan
GPU·LLM·MLOps·쿠버네티스, 그리고 마음가짐에 관한 글 · 1 편
분산 학습 & GPU 인프라 2026 딥다이브 — DeepSpeed, FSDP2, Megatron-LM, Ray Train, JAX, TorchTitan, Blackwell GB200, MI325X, TPU v5p 총정리
DeepSpeed/FSDP2/Megatron-LM/Ray Train/JAX/TorchTitan/Composer를 비교하고, NVIDIA Blackwell GB200 NVL72, AMD MI325X, Intel Gaudi 3, AWS Trainium 2, Google TPU v5p/v6e Trillium까지. 3D 병렬화, ZeRO-FSDP 등가성, MoE All-to-All, fp8/mxfp
2026-05-16 · 23 분 읽기 #distributed-training#deepspeed#fsdp#megatron-lm#ray