Tag: #pytorch

한국어English日本語中文

← All posts

• AI for Everyone, Part 6 — Drawing Digits From Words With a 1.11M Diffusion Model

  We built a conditional diffusion model with 1.11 million parameters that draws a 0 when you type "zero". The forward process that adds noise is a single formula; the reverse process that restores the image is that same l

  2026-08-24 · 8 min read #ai#diffusion#ddpm#generative#pytorch

• AI for Everyone, Part 5 — Colourising Photos With a 0.47M U-Net, and Why the Colours Came Out Washed Out

  The smallest model in this series — a 472K-parameter U-Net — restored colour to greyscale CIFAR-10 images. Shapes survived intact, but the colours came out noticeably washed out. That is not a capacity problem; it is a c

  2026-08-23 · 7 min read #ai#computer-vision#unet#colorization#pytorch

• AI for Everyone, Part 4 — Captioning Images With 1.37M Parameters, and Why Part 3 Bug Was Absent Here

  We joined a CNN encoder to a transformer decoder and captioned Fashion-MNIST images. With 1.37 million parameters and ten minutes of training, the label hit rate reached 91%. This part looks at what cross-attention does

  2026-08-22 · 7 min read #ai#captioning#multimodal#transformer#pytorch

• AI for Everyone, Part 3 — Loss of 0.0017, Accuracy of 7.5%: The Culprit Was One Padding Slot

  We built a VQA model — one that answers questions about an image — with 1.48 million parameters. Training loss fell to 0.0017 while accuracy sat at 7.5%, worse than guessing. The cause was not the model but a single line

  2026-08-21 · 8 min read #ai#vqa#multimodal#debugging#pytorch

• AI for Everyone, Part 1 — Training a 16M-Parameter Language Model From Scratch in 15 Minutes

  Training a language model from scratch on a single GPU. With the TinyStories dataset and a 16-million-parameter decoder-only transformer, we produced readable English fairy tales in 15 minutes. We look at why the causal

  2026-08-19 · 9 min read #ai#llm#transformer#pytorch#hands-on

• A Map of the LLM Training Stack in 2026 — What Each Layer Does For You, and What It Hides

  Sorts LLM training frameworks into three layers and maps out their lineage. The bottom layer is the execution engines — PyTorch distributed, DeepSpeed, Megatron-Core. The middle layer is training loops like torchtitan an

  2026-08-02 · 13 min read #mlops#llm-training#pytorch#trl#framework

• GPU Compiler and Framework Landscape — One Problem, Turning a Graph into a Kernel, a Different Answer at Every Layer

  This post puts NVCC and PTX, LLVM, MLIR, Triton, torch.compile, XLA, IREE, and TVM on one map. Different names, different owners, but they all solve the same problem: turning a computation graph into an executable kernel

  2026-08-02 · 21 min read #gpu#compiler#mlir#triton#pytorch

• torchcomms in PyTorch 2.13 — The New Distributed Communication Backend Coming for c10d

  Among the highlights of PyTorch 2.13, released on July 8, 2026, the most structural change is torchcomms — PyTorch Distributed's new communication backend has begun landing in core's CI and DeviceMesh paths. torchcomms i

  2026-07-17 · 10 min read #pytorch#distributed-training#nccl#deep-learning

• Running Small Models Hands-On with a Single RTX 5090 — microGPT, OCR, Music Generation

  I SSHed into a single RTX 5090 (Blackwell, 32GB) and ran a trio of small models by hand. I trained a char-level GPT from scratch in 28 seconds (10.75M parameters, 1.17M tokens/s), pitted a dedicated OCR model (TrOCR) aga

  2026-07-11 · 8 min read #pytorch#gpu#llm#ocr#hands-on

• The 2026 Vision Model Development & Fine-Tuning Guide — CNN, ViT, DETR, SAM 2, VLMs and a Real Decision Tree

  Vision model development in 2026 is no longer 'grab a ResNet and call it a day.' Between CNNs, ViTs, DETR variants, SAM 2, and VLMs like LLaVA, Qwen-VL, Gemini Vision, and Claude Vision, your choice for the same photo ca

  2026-05-14 · 20 min read #computer-vision#vision-model#cnn#vit#detr

• [Deep RL] 03. PyTorch Deep Learning Basics: From Tensors to Neural Networks

  Learn PyTorch tensor operations, automatic differentiation, and neural network components, then implement TensorBoard monitoring and a GAN example using Atari images.

  2026-03-19 · 20 min read #reinforcement-learning#deep-learning#ai#pytorch

• Building LLM from Scratch: Complete Guide to Understanding GPT through Code

  A complete guide to building and understanding Large Language Models from scratch. Implement a full GPT architecture in PyTorch — from tokenizer to Transformer, pretraining, and fine-tuning.

  2026-03-17 · 21 min read #llm#gpt#transformer#from-scratch#deep-learning

• Torch-Titan Complete Guide: Everything About Large-Scale Distributed Training with PyTorch

  A complete guide to mastering large-scale LLM distributed training with PyTorch Titan (torchtitan). Covers FSDP2, pipeline parallelism, tensor parallelism, 4D parallelism, Flash Attention, and mixed precision with hands-

  2026-03-17 · 19 min read #torch-titan#distributed-training#pytorch#fsdp#deep-learning

• PyTorch Internals & Advanced Optimization: autograd, torch.compile, FSDP, and Triton

  A complete guide to PyTorch internals: autograd engine, torch.compile() with TorchInductor, FSDP distributed training, gradient checkpointing, and custom CUDA operations.

  2026-03-17 · 10 min read #pytorch#torch-compile#fsdp#triton#mixedprecision

• PyTorch Advanced Techniques Complete Guide: torch.compile, Custom Ops, Memory Optimization

  A complete guide to mastering PyTorch advanced techniques. Covers torch.compile, custom operators, memory optimization, Gradient Checkpointing, torch.vmap, functorch, and PyTorch Profiler with hands-on examples.

  2026-03-17 · 17 min read #pytorch#advanced#torch-compile#memory-optimization#custom-operators

• DeepSpeed Complete Guide: ZeRO Optimization and Large-Scale Model Training

  A complete guide to mastering Microsoft DeepSpeed. Learn ZeRO-1/2/3 optimization, Offload, pipeline parallelism, mixed precision, MoE, and DeepSpeed Inference with practical configurations and code examples.

  2026-03-17 · 15 min read #deepspeed#zero-optimization#distributed-training#llm#pytorch

• PyTorch Complete Guide: Zero to Hero — From Tensors to Distributed Training

  A comprehensive guide to mastering PyTorch from the basics to advanced techniques. Learn tensor operations, automatic differentiation, CNN/RNN/Transformer implementation, and distributed training with practical examples

  2026-03-17 · 21 min read #pytorch#deep-learning#ai#python#neural-network

• Deep Learning Debugging Complete Guide: From Diagnosing Training Failures to Performance Optimization

  A complete guide to systematically diagnosing and resolving deep learning training failures. Covers Loss NaN, vanishing/exploding gradients, overfitting, slow convergence, and out-of-memory errors with real-world code ex

  2026-03-17 · 19 min read #deep-learning#debugging#pytorch#training#optimization

• The Complete torchaudio Guide — From Audio Processing to Speech Recognition, TTS, and Music Analysis

  From audio loading and spectrogram transforms to Mel filter banks, MFCC, speech recognition (Wav2Vec2/Whisper), TTS, speaker diarization, and noise reduction — everything about audio AI with PyTorch.

  2026-03-02 · 9 min read #ai-platform#pytorch#torchaudio#audio#speech-recognition

• The Complete torchvision Guide — From Image Classification to Object Detection and Segmentation

  From torchvision transforms v2 and pretrained models (ResNet to ViT) to datasets, Object Detection (Faster R-CNN, YOLO), Segmentation, and hands-on fine-tuning — master computer vision in practice with PyTorch.

  2026-03-02 · 8 min read #ai-platform#pytorch#torchvision#computer-vision#cnn