Blog
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 3517 posts
#2026-03 765#english 592#culture 264#deep-dive 254#kubernetes 247#career 229#ai 216#llm 208#devops 193#2026-04 146#security 141#database 114#observability 113#communication 109#history 107#architecture 100#productivity 96#finance 88#economy 84#mindset 81#psychology 80#ai-papers 79#food 78#it 78#travel 78#deep-learning 77#japanese 77#networking 77#performance 72#business-travel 70#linux 70#gpu 69#ai-agent 66#cs-fundamentals 63#postgresql 60#rag 58#self-improvement 55#learning 53#mlops 53#ai-platform 51
[Deep RL] 15. Trust Region Methods: TRPO, PPO, ACKTR
Trust region techniques for stable policy updates: PPO clipping objective, TRPO constrained optimization, and ACKTR
2026-03-19 · 8 min read #reinforcement-learning#deep-learning#ai[Deep RL] 12. Training Chatbots with Reinforcement Learning
Combining Seq2Seq models with reinforcement learning for chatbot training: from RNN basics to Self-Critical Sequence Training (SCST)
2026-03-19 · 7 min read #reinforcement-learning#deep-learning#ai#nlp[Deep RL] 09. Policy Gradient: Policy-Based Reinforcement Learning
Understand the difference between value-based and policy-based methods, and apply the mathematical derivation of Policy Gradient and the REINFORCE algorithm to CartPole and Pong.
2026-03-19 · 6 min read #reinforcement-learning#deep-learning#ai#policy-gradient[Deep RL] 06. Deep Q-Network: Principles and Implementation of DQN
Understand the core ideas of DQN (experience replay, target network) that overcome the limitations of table-based Q-learning, and implement it in the Pong environment.
2026-03-19 · 10 min read #reinforcement-learning#deep-learning#ai#dqn[Deep RL] 03. PyTorch Deep Learning Basics: From Tensors to Neural Networks
Learn PyTorch tensor operations, automatic differentiation, and neural network components, then implement TensorBoard monitoring and a GAN example using Atari images.
2026-03-19 · 20 min read #reinforcement-learning#deep-learning#ai#pytorch[Deep RL] 19. Practical Applications of Deep Reinforcement Learning
Diverse real-world applications of deep RL: robot control, autonomous driving, resource management, recommendation systems, NLP, and game AI
2026-03-19 · 7 min read #reinforcement-learning#deep-learning#ai[Deep RL] 16. Black-Box Optimization: Evolution Strategies and Genetic Algorithms
Optimizing policies without gradients: principles and implementation of Evolution Strategies (ES) and Genetic Algorithms (GA)
2026-03-19 · 8 min read #reinforcement-learning#deep-learning#ai[Deep RL] 13. Web Navigation and Reinforcement Learning
Combining browser automation with reinforcement learning: implementing web agents using the Mini World of Bits benchmark and OpenAI Universe
2026-03-19 · 8 min read #reinforcement-learning#deep-learning#ai[Deep RL] 10. Actor-Critic Methods: A2C and Hyperparameter Tuning
Understand the Actor-Critic architecture that solves the variance problem of REINFORCE, apply the A2C algorithm to Pong, and analyze the impact of key hyperparameters such as learning rate and entropy coefficient.
2026-03-19 · 8 min read #reinforcement-learning#deep-learning#ai#actor-critic[Deep RL] 07. DQN Extensions: Double DQN, Dueling DQN, Rainbow
Implement six core extension techniques that improve DQN performance (N-step, Double DQN, Noisy Networks, Prioritized Replay, Dueling DQN, Categorical DQN) and Rainbow which combines them all.
2026-03-19 · 11 min read #reinforcement-learning#deep-learning#ai#dqn[Deep RL] 04. Solving CartPole with the Cross-Entropy Method
Explore the taxonomy of reinforcement learning methods and solve CartPole and FrozenLake environments using the Cross-Entropy method.
2026-03-19 · 9 min read #reinforcement-learning#deep-learning#ai[Deep RL] 01. What is Reinforcement Learning: MDP and Fundamental Concepts
We compare three paradigms of machine learning, and explore the core components of reinforcement learning (agent, environment, reward) along with the mathematical foundations of Markov Decision Processes (MDP).
2026-03-19 · 9 min read #reinforcement-learning#deep-learning#ai[OS Concepts] 18. Virtual Machines: From Hypervisors to Containers
Covers VM benefits, Type 1/Type 2 hypervisors, hardware-assisted virtualization (VT-x), paravirtualization, containers (Docker, Kubernetes), and VM migration.
2026-03-19 · 10 min read #operating-system#cs-fundamentals[OS Concepts] 15. File-System Internals
Covers file system mounting, partitions, block groups, ext4 internals, APFS, performance optimization, and NFS.
2026-03-19 · 9 min read #operating-system#cs-fundamentals[OS Concepts] 12. I/O Systems
Covers I/O hardware (ports, buses, controllers), polling, interrupts, DMA, application I/O interfaces, and the kernel I/O subsystem.
2026-03-19 · 8 min read #operating-system#cs-fundamentals[OS Concepts] 09. Main Memory Management
Covers memory management background (address binding, logical/physical addresses, MMU), contiguous memory allocation, paging (basic concepts, TLB, page table structures), swapping, and memory protection.
2026-03-19 · 22 min read #operating-system#cs-fundamentals[OS Concepts] 06. Process Synchronization Tools
Covers the critical section problem, Peterson's solution, hardware support (memory barriers, CAS, atomic variables), mutex locks, semaphores, monitors, and liveness problems (deadlock, starvation, priority inversion).
2026-03-19 · 9 min read #operating-system#cs-fundamentals[OS Concepts] 03. Processes: Concepts, Creation, Communication
Covers process concepts (states, PCB), process scheduling, process operations (fork, exec), inter-process communication (shared memory, message passing), pipes, and client-server communication (sockets, RPC).
2026-03-19 · 8 min read #operating-system#cs-fundamentalsUzbekistan Complete Guide: Food, Tourism, Culture, History, Economy, IT & Business Travel
A comprehensive guide to Uzbekistan covering food, tourist attractions, culture, history, notable figures, economy, IT ecosystem, and business travel tips. From plov and Registan Square to IT Park.
2026-03-19 · 6 min read #uzbekistan#central-asia#travel#food#cultureUnited Kingdom Complete Guide: Food, Tourism, Culture, History, Economy, IT, Business Travel
A comprehensive guide to the United Kingdom covering food, tourism, culture, history, famous figures, economy, IT ecosystem, and business travel tips.
2026-03-19 · 8 min read #united-kingdom#travel#food#culture#history