Tag: #cuda
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 23 posts
AMD vs. NVIDIA: What Actually Differs — Why the Stack Is the Problem, Not the Hardware
Breaks down the difference between AMD and NVIDIA GPUs across four layers — hardware architecture, software stack, porting path, and ecosystem maturity — without taking sides. Covers the mapping from SM to CU and from Te
2026-08-02 · 16 min read #amd#rocm#hip#nvidia#cudaWhat It Really Means to Hand-Tune a GPU Kernel — Making One Transpose Kernel 5x Faster
Starting from threads, warps, and the memory hierarchy, this post covers what it actually means to hand-modify a GPU kernel. It explains why occupancy is a symptom rather than a goal, and why most kernels are bound by me
2026-08-02 · 19 min read #cuda#gpu-kernel#nsight-compute#memory-bandwidth#performanceThree Layers of Writing a Kernel — Comparing CUDA C++, Triton, and CUTLASS on the Same Problem
Compares what changes when you approach the same GPU kernel by hand in CUDA C++, tile-by-tile in Python with Triton, or assembled from templates in CUTLASS and CuTe. We actually write a row-wise softmax in both CUDA C++
2026-08-02 · 18 min read #triton#cuda#cutlass#gpu-kernel#compilerRust 1.97 Cut Off Pre-Volta GPUs — Inside the nvptx64 Baseline Bump
Rust 1.97 (July 9, 2026) raised the minimum requirements for the nvptx64-nvidia-cuda target to PTX ISA 7.0 (CUDA 11 driver or newer) and SM 7.0 (Volta or newer). Maxwell- and Pascal-generation GPUs, and CUDA 10 and older
2026-07-16 · 13 min read #rust#cuda#gpu#compiler#nvidiaDiffusion LLMs That Write CUDA Kernels — DICE and Why Parallel Generation Might Help
DICE, a February 2026 preprint, claims that diffusion large language models beat autoregressive models of the same size at generating CUDA kernels, setting a new state of the art. The core idea: instead of writing one to
2026-07-11 · 6 min read #ai#llm#diffusion#cuda#gpuCUDA Architecture Visualized — From Threads to Tensor Cores
A diagram-driven tour of the CUDA execution model and GPU hardware. We walk through the grid-block-warp-thread hierarchy, SM internals, the memory hierarchy, warp scheduling and occupancy, tensor cores, and streams, all
2026-06-27 · 17 min read #cuda#gpu#tensor-core#warp#memory-hierarchyGPU vs TPU vs ASIC — The 2026 Inference War
A comparison of the GPU, TPU, and ASIC competition over 2026 inference workloads. We cover Google TPU v6 Trillium and Ironwood, the fast-growing cloud in-house inference ASICs, the throughput/latency/cost/power trade-off
2026-06-16 · 23 min read #gpu#tpu#asic#inference#ai-hardwareDistributed Training & GPU Infrastructure 2026 Deep-Dive — DeepSpeed, FSDP2, Megatron-LM, Ray Train, JAX, TorchTitan, Blackwell GB200, MI325X, TPU v5p
A comparison of DeepSpeed, FSDP2, Megatron-LM, Ray Train, JAX, TorchTitan, and Composer — plus NVIDIA Blackwell GB200 NVL72, AMD MI325X, Intel Gaudi 3, AWS Trainium 2, and Google TPU v5p/v6e Trillium. 3D parallelism, ZeR
2026-05-16 · 14 min read #distributed-training#deepspeed#fsdp#megatron-lm#rayCUDA GPU Programming Model Deep Dive — SIMT, Memory Hierarchy, Tensor Core, Kernel Optimization (2025)
The engine behind ChatGPT, Stable Diffusion, and Sora — NVIDIA GPUs and CUDA. This post dissects the CUDA programming model from scratch: GPU hardware architecture (SM, Warp, CUDA Core), SIMT vs SIMD, thread hierarchy (G
2026-04-15 · 15 min read #cuda#gpu#nvidia#ai#machine-learningSemiconductor Deep Dive -- Complete Guide to CPU, GPU, RAM, ASIC, and CUDA Architecture
How does a CPU execute instructions, how does RAM store data, and why are GPU/CUDA essential for AI? A deep dive from semiconductor fundamentals to ASIC custom chips.
2026-04-10 · 22 min read #ai#semiconductor#cpu#gpu#cudaGPU Software Engineer Complete Guide: From CUDA Architecture to vGPU/MIG, InfiniBand, and K8s GPU Scheduling — System Optimization Mastery
A complete analysis of the LG Uplus GPU Software Engineer JD. From GPU compute architecture, memory hierarchy, CUDA programming, vGPU/MIG virtualization, InfiniBand/RDMA networking, K8s GPU Device Plugin, to large-scale
2026-03-23 · 50 min read #gpu#cuda#system-software#virtualization#vgpuNVIDIA GPU and CUDA Architecture Deep Dive: Why GPUs Dominate AI
From H100 hardware specs to Tensor Core WMMA API. A complete breakdown of the SIMT execution model, shared memory tiling, warp divergence, and why GPUs are purpose-built for AI workloads.
2026-03-18 · 14 min read #cuda#gpu#nvidia#matrix-multiplication#parallel-computingGPU Hardware Complete Guide for AI: From Architecture to Selection Criteria
A comprehensive guide to GPU hardware for AI research and training. Covers NVIDIA GPU architectures (Hopper, Blackwell), Tensor Core, NVLink, HBM memory, A100/H100/H200/B200 comparisons, and cloud GPU options in detail.
2026-03-17 · 23 min read #gpu#hardware#nvidia#cuda#gpu-cudaAdvanced CUDA GPU Programming: Warp Optimization, Tensor Cores, and Triton Kernels
A comprehensive deep dive into CUDA memory hierarchy, Warp optimization, Tensor Core WMMA API, Flash Attention implementation, and Triton custom kernel authoring for accelerating AI model training.
2026-03-17 · 19 min read #cuda#gpuprogramming#tensorcore#triton#flash-attentionCUDA Programming Complete Guide: GPU Parallel Computing Zero to Hero
A comprehensive guide to mastering CUDA programming from fundamentals to advanced optimization. Covers GPU architecture, kernel writing, memory optimization, mixed precision training, and cuDNN/cuBLAS usage with practica
2026-03-17 · 30 min read #cuda#gpu#gpu-cuda#parallel-computing#nvidiaComplete Guide to Building a Linux GPU Server for Deep Learning
A step-by-step guide to building a Linux GPU server for deep learning development, covering everything from NVIDIA driver installation to Docker GPU environments, based on official NVIDIA documentation.
2026-03-01 · 20 min read #linux#gpu#cuda#nvidia-driver#deep-learningNVIDIA GPU Operator Complete Guide: Components, Installation, and KubeVirt GPU Passthrough
A detailed analysis of NVIDIA GPU Operator architecture and the roles of its 7 core components (Driver, Container Toolkit, Device Plugin, DCGM, MIG Manager, Node Feature Discovery, GFD), covering Helm-based installation,
2026-03-01 · 18 min read #gpu-operator#nvidia#kubernetes#kubevirt#gpuComplete Guide to Multi-GPU Distributed Training: DDP, FSDP, DeepSpeed
Systematically analyze the core components of multi-GPU distributed training including DDP, FSDP, and DeepSpeed ZeRO based on PyTorch official documentation, with practical setup instructions.
2026-03-01 · 21 min read #gpu#cuda#distributed-training#deep-learning#pytorchThe Complete Autonomous Driving & Robotics Tech Stack: From C++, ROS2, CUDA, TensorRT to VLM/VLA, Simulation, and Beyond
A comprehensive guide to the core technology stack behind autonomous driving and robotics. Covering Modern C++, ROS/ROS2, CUDA parallel programming, TensorRT optimization, model compression (quantization/pruning), sensor
2026-03-01 · 22 min read #autonomous-driving#robotics#ros2#cuda#tensorrtMastering Slurm: A Practical Guide to the HPC/AI Cluster Workload Manager
A comprehensive, hands-on guide to the Slurm workload manager. Covers architecture (slurmctld/slurmd/slurmdbd), core concepts (Partitions/QoS/Fairshare), essential commands (sbatch/srun/salloc), GPU scheduling (GRES/MIG/
2026-03-01 · 15 min read #slurm#hpc#gpu#distributed-training#cluster