A Modern Understanding of the Operating System — io_uring, cgroups/namespaces, eBPF, NUMA, GPU UVM, EEVDF, Zero-Copy Complete Guide (2025)

Why Relearn the OS Now

If you ask "Did we not learn the OS back in undergrad?", here is the answer:

• io_uring (2019) — the successor to epoll/kqueue. Node.js, Rust tokio, and PostgreSQL 17 are adopting it.
• cgroups v2 (2016+, mainstream in 2022) — the foundation of resource control in Docker and K8s.
• eBPF — that very technology which showed up in the earlier posts (Observability, Network). The kernel revolution of the 2020s.
• EEVDF (2024 Linux 6.6) — replaces the CFS scheduler.
• NUMA — the moment you go past 32 vCPUs, performance starts taking a serious hit.
• GPU UVM (Unified Virtual Memory) — the foundation of LLM inference and training.
• io_uring-based networking — competing with AF_XDP and DPDK.

If an engineer in 2025 does not know the OS, they cannot explain why their own app is slow, why the container gets hit by OOM, or why throughput will not rise even at 100% CPU.

Part 1 — Processes, Threads, Coroutines — a Modern Comparison

Processes

• A separate address space. Strong isolation.
• Expensive to create (fork + exec).
• IPC required.
• Examples: Unix shell pipes, Chrome tabs.

Threads

• Share the same address space.
• Cheap to create (pthread_create).
• Easy to communicate through shared memory + dangerous (data races).
• Kernel scheduling → context switch overhead.

Coroutines / Fibers / Green Threads

• User-space scheduling.
• Creation costs almost nothing (tens of millions are possible).
• Cooperative scheduling — they yield at await points.
• Examples: Go goroutines, Rust async, Python asyncio, Java Virtual Threads (2023).

Java Virtual Threads — Project Loom (2023)

JDK 21 LTS. "Leave your existing thread code almost as it is and handle millions of concurrent requests."

Traditional threads: 1MB+ of OS stack per thread → tens of thousands is the ceiling. Virtual threads: scheduled inside the JVM, with the stack allocated only when needed → millions are possible.

You get the concurrency of non-blocking while still writing blocking I/O. With Spring Boot adoption in 2024-2025, the terrain of Java backends is shifting.

Go goroutines

• M:N scheduling (M kernel threads : N goroutines).
• Communication over channels.
• The stack starts at 2KB and expands dynamically.
• GOMAXPROCS decides the number of P (Processors).

Which One, When?

Part 2 — io_uring — Going Beyond epoll

A History of I/O Evolution

1. Blocking I/O — read() blocks until the data arrives.
2. select/poll — scans the entire FD set. O(n).
3. epoll (Linux, 2002) / kqueue (BSD) — event-driven, O(1).
4. aio_read(POSIX AIO) — plenty of limitations. Barely used.
5. io_uring (2019, Jens Axboe) — truly asynchronous.

The Structure of io_uring

Submission Queue (SQ) + Completion Queue (CQ). Both are mmap-ed shared memory. Work is submitted and completions are checked without a syscall.

io_uring_prep_read(sqe, fd, buf, len, offset);
io_uring_submit(&ring);  // one syscall submits many requests
// ... later
io_uring_wait_cqe(&ring, &cqe);

Benefits:

• Syscall overhead drops sharply — zero in SQPOLL mode.
• Batching — submit several I/Os at once.
• Files + network + timers + accept unified.
• sendmsg, recvmsg, splice, and so on are mostly supported.

Adoption:

• Rust tokio (partial), glommio, monoio.
• Node.js — experimental.
• PostgreSQL 17 (2024) — evaluating io_uring on an async I/O foundation.
• QEMU, RocksDB, ScyllaDB.

Security issue: in 2023 Google blocked io_uring inside the Chrome sandbox. It widens the kernel attack surface. Hence the recommendation to use it only in trusted environments.

Part 3 — The Modern Era of Virtual Memory

Four-Level Page Tables (x86_64)

CR3 → PML4 → PDPT → PD → PT → Physical Page

A 48-bit virtual address = 9+9+9+9+12 bits.

Five Levels — a 57-bit Address Space (Ice Lake 2021+)

Needed on large servers. Moving toward being the Linux default in 2024.

TLB (Translation Lookaside Buffer)

The cache for virtual→physical translation. A few hundred entries in size. A TLB miss is the hidden killer of performance.

Solutions:

• Huge Pages (2MB, 1GB) — one TLB entry covers a large region.
• Transparent Huge Pages (THP) — Linux merges them automatically. That said, it can also become a source of latency spikes.

Memory Overcommit

The Linux default: "pretend to hand over all the memory that was requested" → pages get allocated when they are actually used. Run short on memory later → the OOM Killer fires.

echo 2 > /proc/sys/vm/overcommit_memory  # strict accounting

Part 4 — cgroups + namespaces = Containers

cgroups v2

A hierarchy of resource groups. Limits CPU, memory, I/O, and the PID count.

/sys/fs/cgroup/
  my-app/
    cpu.max         # "50000 100000" → 50% CPU
    memory.max      # "1G"
    io.max          # "8:0 rbps=10485760"  → 10MB/s read

namespaces

A "view of its own" for a process.

• mnt: the filesystem.
• pid: process IDs.
• net: network interfaces and routing.
• ipc: IPC.
• uts: hostname.
• user: UID/GID mapping.
• cgroup: the cgroup view.
• time: added in 2020, virtualizes boot time.

What Docker Really Is

container = chroot + namespaces + cgroups + capabilities + seccomp + AppArmor/SELinux

It is not a virtual machine. It shares one kernel, and only the "visible region" differs.

Rootless Containers (2020+)

Running containers without root by way of the user namespace. Podman and Buildah led the way.

Part 5 — eBPF — Injecting Code Into the Kernel

What eBPF Is

Extended BPF. A small VM that runs in kernel space. Safety is guaranteed by a sandbox and a verifier.

Why it is a revolution:

• Functionality is added without modifying the kernel.
• Traditionally that meant developing a module and rebooting.
• eBPF loads and unloads in real time.

Where It Is Used

• Observability: bpftrace, Parca, Pixie.
• Security: Tetragon, Falco.
• Networking: Cilium, Katran, ultra-fast L4 LB with XDP.
• Profiling: continuous profiling (Parca, Polar Signals).

XDP (eXpress Data Path)

Runs eBPF at the NIC driver level on the network card. Packets are handled before they enter the kernel stack → used for things like DDoS defense. Tens of millions of packets per second can be processed.

The Development Stack

• bpftrace — one-line scripts (awk style).
• libbpf + CO-RE(Compile Once Run Everywhere) — C/Rust.
• Aya (Rust) — maturity rose in 2024.

// bpftrace example: tracing the open() syscall
bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s -> %s\n", comm, str(args->filename)); }'

Part 6 — NUMA — the Hidden Cost Beyond 32 Cores

What NUMA Is

Non-Uniform Memory Access. A large server has several sockets (physical CPUs), and each socket carries its own memory bank. Accessing the memory of another socket is 1.5-3x slower.

Checking It

numactl --hardware
# node 0 cpus: 0-23
# node 0 size: 96GB
# node 1 cpus: 24-47
# node 1 size: 96GB
# node distances: 10 (local), 21 (remote)

NUMA Binding

# Run the process only on NUMA node 0
numactl --cpunodebind=0 --membind=0 ./my-app

Essential on DB servers, LLM inference, and high-performance proxies. Leave it to the defaults and the scheduler makes decisions that are not optimal.

Kubernetes + NUMA

• Topology Manager (alpha→beta) — places Pods along NUMA boundaries.
• CPU Manager static policy — allocates dedicated cores.

On an LLM inference K8s cluster, this setting makes a 30%+ difference in throughput.

Part 7 — The Evolution of the Linux Scheduler

CFS (Completely Fair Scheduler, 2007-2024)

• Pursues fairness through virtual runtime (vruntime).
• Priority through the nice value.
• Managed with a red-black tree.

EEVDF (Earliest Eligible Virtual Deadline First, 2024)

The default since Linux 6.6. Led by Peter Zijlstra.

• Fixes the weakness of CFS (unfavorable to latency-sensitive tasks).
• Introduces a latency tolerance (slice) parameter.
• Friendlier to media, gaming, and interactive workloads.

CPU Isolation Techniques

• isolcpus — excludes the named cores from ordinary scheduling.
• nohz_full — removes the timer interrupt.
• RCU callback offload — keeps the named cores undisturbed.

Standard in low-latency trading and HFT infrastructure.

Part 8 — How GPU Drivers Relate to LLMs

The Complexity of GPU Containers

To use a GPU through Docker:

• NVIDIA Container Toolkit — mounts the host driver into the container.
• NVIDIA MIG (Multi-Instance GPU) — splits an H100 into 7 instances.
• MPS (Multi-Process Service) — GPU sharing.

UVM (Unified Virtual Memory, CUDA 6+)

The CPU and the GPU share the same address space. On a page fault, only the page that is needed gets transferred. Essential when handling a model larger than GPU memory during LLM training.

CUDA Graph

Captures a repeating kernel-call pattern as a graph and removes the overhead. There are cases of a 20%+ speedup in the decoding loop of LLM inference.

GPU OS Issues in 2024-2025

• Fractional GPU sharing (Run.ai, Aptakube) — production issues are frequent.
• NVIDIA DCGM + K8s metrics — the standard for GPU observability.
• AMD ROCm, Intel oneAPI — maturity as a CUDA replacement is climbing slowly.

Part 9 — The Final Boss of I/O Performance

Zero-Copy

Traditional: read() → buffer → write(), a copy at every step. Zero-copy: sendfile(), splice(), io_uring, MSG_ZEROCOPY — the kernel DMAs directly.

A real case: Kafka uses sendfile() when sending data to a consumer → CPU utilization halved.

DMA (Direct Memory Access)

The device accesses memory directly with no CPU involvement. Network cards, SSDs, and GPUs all carry a built-in DMA engine.

RDMA (Remote DMA)

Direct access to the memory of another machine with no CPU involvement. Infiniband, RoCE (RDMA over Converged Ethernet).

• Latency on the order of 1μs.
• HFT, HPC, and large data warehouses.
• The default for AI training clusters in 2024 — NVIDIA GPUDirect RDMA.

DPDK / AF_XDP

Bypasses the kernel network stack and drives the NIC directly from user space. Cloud virtual switches, 5G UPF, high-performance proxies (Cloudflare, Fastly).

Part 10 — Where WSL2, Containers, and Virtualization Intersect

WSL2 (Windows Subsystem for Linux 2)

• It actually runs a Linux kernel inside a lightweight Hyper-V VM.
• The Windows kernel and the Linux kernel coexist.
• File performance: the WSL native FS is fast, Windows /mnt/c is slow.
• Default systemd support was added in 2024.

Firecracker (the base of AWS Lambda)

• Open-sourced in 2018. A microVM.
• Boots in under 125ms.
• KVM-based, a VMM of fewer than 120 lines.
• Fly.io and Kata Containers use it as well.

gVisor (Google)

• A user-space reimplementation of the kernel.
• Syscalls are handled by the user-space Sentry → the attack surface of the host kernel shrinks.
• There is a performance overhead, but the isolation is strong.

Kata Containers

• An OCI-compatible container runtime + a microVM.
• The convenience of a container + the isolation of a VM.
• Useful on multi-tenant K8s.

Part 11 — The Tools of Observation

Continuous Profiling

• Parca, Pyroscope, Polar Signals.
• eBPF-based always-on profiling with under 1% overhead.
• It solves riddles like "P99 is slow but the CPU is idle".

Part 12 — OS Checklist (12 Items)

1. Check ulimit — the FD limit and the nproc limit on production servers.
2. Swap policy — swappiness=1 (DB) or 0 (latency sensitive).
3. Transparent Huge Pages — for a DB, turning it off is usually better.
4. NUMA binding — on systems with two or more sockets.
5. Check io_uring support — adjust the fd ceiling on recent kernels.
6. Use cgroups v2 — v1 is feature-limited.
7. seccomp profiles — restrict container syscalls.
8. Tune the basic TCP parameters — somaxconn, tcp_max_syn_backlog.
9. Check the kernel version — the latest LTS (6.6+) has mature EEVDF and io_uring.
10. eBPF observability infrastructure — keep at least one profiling tool always on.
11. Monitor OOM Killer logs — the clues live in dmesg.
12. CPU governor — set performance mode (on servers).

Part 13 — Ten Anti-Patterns

1. Treating a container as a VM — forget the "shared kernel" and misunderstandings about security and performance follow.
2. Tens of thousands of threads in one process — context switch hell. Coroutines are the answer.
3. Mass concurrency with blocking I/O — event loop / coroutines / Virtual Threads.
4. Keeping swappiness=60 (the default) — a DB needs it lowered.
5. Ignoring THP on a large-RAM machine — it can become the source of latency spikes.
6. Ignoring NUMA — run a single process on a big machine and you get half the performance.
7. Leaving strace running on a syscall-heavy app — 100x slower. Use eBPF.
8. Running a container as root — go rootless or drop capabilities.
9. Treating Firecracker like an "ordinary K8s container" — storage and networking differ.
10. Attaching a GPU to docker run just like that — check the Toolkit + driver version matrix.

Closing — the OS Is Still "the Boundary of Performance"

In 2025 the performance ceiling of an app is often set by the OS. Knowing io_uring or not makes a 2x difference in the throughput of a network server, the cgroups v2 settings decide whether a container gets OOM-killed, and NUMA binding governs the cost of LLM inference.

The OS is not "below my app" but "part of my app". A good engineer can cross that boundary at the moment it is needed. They explore /proc/, run perf top, and get their answer from one line of bpftrace.

What you learned in an undergraduate OS class (page tables, schedulers, semaphores) is still alive. Only the form has evolved. The OS of 2025 is a world where a single kernel, thousands of containers, GPUs, and RDMA coexist. Understanding that world is the new foundation for an engineer.

Next Post Preview — "Compilers and Modern Language Runtimes" — LLVM, JIT, GC, Inline Caching, Escape Analysis, and WASM Runtimes

Below the OS is hardware; above the OS is the runtime. Next comes how a language actually runs.

• The dominance of LLVM — the skeleton shared by Rust, Swift, Julia, Zig, and Crystal
• Inside the JIT compiler — TurboFan in V8, C2 in the JVM, LuaJIT
• Hidden Classes and Inline Caching — the secret behind V8 making JS fast
• Escape Analysis — leave it on the stack or put it on the heap
• The lineage of garbage collectors — from Mark & Sweep to ZGC, Shenandoah, G1, and the tri-color concurrent GC of Go
• Tiered Compilation — V8 Ignition/Sparkplug/Maglev/TurboFan
• Monomorphization in Rust — why generics are fast
• Work-stealing in Go — inside the goroutine scheduler
• The Specializing Adaptive Interpreter in Python — the leap of 3.13
• The WASM runtimes — Wasmtime, wasmer, WasmEdge compared

"What happens between here and my code running?" In the next post.