containerd CRI Implementation: Kubernetes Runtime Integration
containerd implements the Kubernetes CRI (Container Runtime Interface) as a built-in plugin. This post analyzes the CRI gRPC service implementation details, Pod Sandbox management, container spec translation, streaming API, RuntimeClass, and NRI.
1. CRI gRPC Service
1.1 Service Structure
CRI consists of two gRPC services:
CRI gRPC services:
RuntimeService:
+-- PodSandbox management
| RunPodSandbox
| StopPodSandbox
| RemovePodSandbox
| PodSandboxStatus
| ListPodSandbox
|
+-- Container management
| CreateContainer
| StartContainer
| StopContainer
| RemoveContainer
| ListContainers
| ContainerStatus
| UpdateContainerResources
|
+-- Streaming
| ExecSync
| Exec
| Attach
| PortForward
|
+-- Runtime info
Status
Version
ImageService:
+-- PullImage
+-- ListImages
+-- ImageStatus
+-- RemoveImage
+-- ImageFsInfo
1.2 Socket Configuration
CRI socket:
containerd serves CRI on the same gRPC socket:
/run/containerd/containerd.sock
kubelet configuration:
--container-runtime-endpoint=unix:///run/containerd/containerd.sock
CRI plugin registers CRI services on the containerd server:
Plugin ID: io.containerd.grpc.v1.cri
2. Pod Sandbox
2.1 Pod Sandbox Concept
A Pod Sandbox represents the isolation environment for a Pod:
Pod Sandbox composition:
Pod Sandbox = Pause container + shared namespaces
Shared resources:
- Network namespace (same IP, port space)
- IPC namespace (inter-process communication)
- UTS namespace (hostname)
- PID namespace (optional)
Isolated resources:
- Mount namespace (per container)
- cgroup (per container resource limits)
2.2 RunPodSandbox Flow
RunPodSandbox processing:
1. Create Sandbox metadata
- Generate ID
- Create log directory
|
v
2. Pull Pause image
- Determine image from sandbox_image config
- Default: version dependent (registry.k8s.io/pause:3.10.2 per current docs)
|
v
3. Prepare Pause container snapshot
|
v
4. Generate OCI spec
- Minimal spec for Pause container
- Include hostname, DNS configuration
|
v
5. Create network namespace
- Create namespace file at /var/run/netns/
|
v
6. Call CNI plugin
- Create network interface
- Allocate IP
|
v
7. Create and start Pause container Task
|
v
8. Set Sandbox state to SANDBOX_READY
2.3 Pause Container
Pause container role:
1. Namespace holder:
- First process in network namespace
- Namespace persists even if App containers exit
- Binds namespace lifecycle to Pod
2. PID 1 role:
- Init process of Pod PID namespace
- Reaps zombie processes
- Minimal resource usage (approx. 1MB)
3. Behavior:
- Waits indefinitely via pause() system call
- Exits on SIGTERM
3. Container Spec Translation
3.1 CRI Request to OCI Spec
Spec translation process:
CRI ContainerConfig:
- Image
- Command, Args
- Envs
- Mounts
- Devices
- SecurityContext
- Resources
|
v
containerd CRI plugin translates
|
v
OCI Runtime Spec:
- root (image snapshot path)
- process (command, env, capabilities)
- mounts (volumes, special filesystems)
- linux.resources (cgroup settings)
- linux.namespaces (shared with Sandbox)
- hooks (OCI hooks)
3.2 Resource Translation
Kubernetes resources -> OCI resource translation:
CPU:
requests.cpu: 250m
-> linux.resources.cpu.shares = 256
(based on 1000m = 1024 shares)
limits.cpu: 500m
-> linux.resources.cpu.quota = 50000
linux.resources.cpu.period = 100000
(500m/1000m * 100000us)
Memory:
limits.memory: 512Mi
-> linux.resources.memory.limit = 536870912
(in bytes)
requests.memory:
-> Used for scheduling only, not reflected in OCI spec
Hugepages:
limits.hugepages-2Mi: 100Mi
-> linux.resources.hugepageLimits:
pageSize: "2MB"
limit: 104857600
3.3 Security Context Translation
SecurityContext -> OCI spec translation:
runAsUser: 1000
-> process.user.uid = 1000
runAsGroup: 1000
-> process.user.gid = 1000
readOnlyRootFilesystem: true
-> root.readonly = true
privileged: true
-> Grant all capabilities
-> Allow all device access
-> Disable AppArmor/SELinux/Seccomp
capabilities:
add: ["NET_ADMIN"]
drop: ["ALL"]
-> process.capabilities configuration
seccompProfile:
type: RuntimeDefault
-> Apply linux.seccomp profile
4. Streaming API
4.1 ExecSync
ExecSync operation:
Synchronously execute command in container:
1. kubelet calls ExecSync(containerID, cmd, timeout)
|
v
2. containerd sends Exec request to shim
|
v
3. Shim executes runc exec
- Create new process in container namespaces
|
v
4. Capture stdout/stderr
|
v
5. Wait for process exit
|
v
6. Return exit code + stdout + stderr
Use cases: liveness/readiness probes, kubectl exec (sync)
4.2 Exec (Async Streaming)
Exec streaming operation:
1. kubelet calls Exec(containerID, cmd, stdin, stdout, stderr)
|
v
2. containerd returns streaming URL
- Streaming server address: https://node:10250/exec/...
|
v
3. kubelet passes URL to client
|
v
4. Client connects to streaming server via WebSocket/SPDY
|
v
5. Streaming server performs actual Exec via containerd
|
v
6. Bidirectional stdin/stdout/stderr streaming
Streaming protocols:
- SPDY (legacy)
- WebSocket (modern)
4.3 Attach
Attach operation:
Connect to a running container's main process:
1. Generate streaming URL (similar to Exec)
|
v
2. Connect to container's stdin/stdout/stderr
- Does not create new process
- Directly connects to existing process I/O
|
v
3. Bidirectional streaming
Use case: kubectl attach
4.4 PortForward
PortForward operation:
Forward local traffic to Pod port:
1. Generate streaming URL
|
v
2. Execute socat/nsenter in Pod's network namespace
- Create TCP connection to specified port
|
v
3. Bidirectional data transfer between local port and Pod port
Implementation:
containerd enters the Pod's network namespace and
creates a TCP connection to the target port.
Use case: kubectl port-forward
5. RuntimeClass
5.1 RuntimeClass Mapping
RuntimeClass processing:
1. Kubernetes RuntimeClass resource:
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: kata
handler: kata
2. kubelet passes runtime_handler = "kata"
when calling CRI RunPodSandbox
3. containerd maps handler to runtime config:
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]
runtime_type = "io.containerd.kata.v2"
4. Create Task with corresponding shim binary:
containerd-shim-kata-v2
5.2 Default Runtime
Default runtime configuration:
[plugins."io.containerd.grpc.v1.cri".containerd]
default_runtime_name = "runc"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true
If Pod has no runtimeClassName, default runtime (runc) is used
5.3 RuntimeClass Overhead
RuntimeClass resource overhead:
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: kata
handler: kata
overhead:
podFixed:
memory: "160Mi"
cpu: "250m"
Overhead processing:
- kubelet adds overhead to Pod resources
- Scheduler includes overhead when selecting nodes
- Reflects fixed costs of VM-based runtimes
6. NRI (Node Resource Interface)
6.1 NRI Overview
NRI is a plugin extension mechanism for containerd that allows registering hooks on container lifecycle events:
NRI architecture:
kubelet -> containerd
|
+-- NRI Plugin 1 (resource allocation)
+-- NRI Plugin 2 (topology awareness)
+-- NRI Plugin 3 (monitoring)
NRI plugins receive container lifecycle events and
can modify the OCI spec.
6.2 NRI Hook Points
NRI hook points:
1. RunPodSandbox:
- Called on Pod creation
- Pod-level resource allocation
2. CreateContainer:
- Called on container creation
- Can modify OCI spec
- CPU pinning, memory NUMA allocation, etc.
3. StartContainer:
- Called on container start
4. UpdateContainer:
- Called on resource update
5. StopContainer:
- Called on container stop
- Resource release
6. RemoveContainer:
- Called on container removal
6.3 NRI Use Cases
NRI use cases:
1. CPU/memory topology-aware allocation:
- NUMA-aware CPU pinning
- Allocate memory to specific NUMA nodes
- Integration with topology manager
2. Device resource management:
- GPU allocation optimization
- RDMA resource management
- Device plugin complementation
3. Security policy enforcement:
- Dynamic Seccomp profiles
- Runtime security rule injection
4. Monitoring/auditing:
- Container start/stop event logging
- Resource usage tracking
7. Image Service
7.1 Image Pull
CRI PullImage processing:
1. kubelet calls PullImage(imageSpec, authConfig)
|
v
2. containerd resolves image reference
- Tag or digest
- Apply registry auth credentials
|
v
3. Download image
- Manifest, Config, Layers
- Store in k8s.io namespace
|
v
4. Unpack layers
- Create snapshot chain via Snapshotter
|
v
5. Return image reference (imageRef)
7.2 Image Caching
Image caching:
containerd image caching:
- Skip download if layer already exists in Content Store
- Skip unpacking if snapshot already exists in Snapshotter
- Accurate deduplication based on digests
kubelet image policy:
imagePullPolicy: Always
-> Always check registry manifest (layers leverage cache)
imagePullPolicy: IfNotPresent
-> Pull only if not available locally
imagePullPolicy: Never
-> Use local images only
8. Monitoring and Debugging
8.1 CRI Metrics
containerd CRI-related metrics:
container_runtime_cri_operations_total: CRI operation count
container_runtime_cri_operations_errors_total: CRI operation error count
container_runtime_cri_operations_latency_seconds: CRI operation latency
containerd internal metrics:
containerd_task_count: Running Task count
containerd_container_count: Container count
containerd_image_pull_duration_seconds: Image pull duration
8.2 Debugging Tools
Debugging tools:
1. crictl (CRI CLI):
crictl ps # List containers
crictl pods # List pods
crictl images # List images
crictl inspect CONTAINER_ID # Container details
crictl logs CONTAINER_ID # Container logs
crictl exec -it CONTAINER_ID /bin/sh # exec
2. ctr (containerd CLI):
ctr -n k8s.io containers list
ctr -n k8s.io tasks list
ctr -n k8s.io images list
3. containerd logs:
journalctl -u containerd -f
9. What You Actually Do With crictl
The eight sections above describe what CRI is. On a node, the one channel through which you actually touch CRI is crictl. And crictl confuses people from the very first invocation. The cri-tools docs state that the default endpoints are now deprecated and that the runtime endpoint should always be set instead. If you do not set it, crictl tries the known socket candidates in order and burns several seconds on each failed candidate before the connection times out. When a crictl command that does nothing takes ten-odd seconds, that probing is usually the reason. The config file is /etc/crictl.yaml, and these are the keys the docs show.
# /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 2
debug: true
pull-image-on-create: false
max-retries: 3
The same values can be passed as flags. -r, --runtime-endpoint selects the runtime service and -i, --image-endpoint the image service, the latter defaulting to the runtime-endpoint setting. -t, --timeout defaults to 2 seconds and --max-retries defaults to 3, retrying an explicitly set endpoint with exponential backoff. When a response looks wrong, adding -D, --debug to see the request and response verbatim is faster than digging through logs.
crictl info # runtime information
crictl pods # ID / Created / State / Name / Namespace / Attempt / Runtime
crictl ps -a # CONTAINER / IMAGE / CREATED / STATE / NAME / ATTEMPT / POD ID / POD / NAMESPACE
crictl ps -p POD_ID # only containers in one pod
crictl inspectp POD_ID # pod sandbox status
crictl inspect CONTAINER_ID # container status
crictl logs -f --tail 100 CONTAINER_ID
crictl logs -p CONTAINER_ID # logs of the previous instance
crictl stats # container resource usage
crictl statsp # per-pod statistics
crictl imagefsinfo # image filesystem usage
Knowing what to read in the output is where the real difference lies. The Runtime column of crictl pods shows which runtime handler the pod went down to, so you can confirm that a RuntimeClass was applied as intended from the result rather than from the manifest. crictl ps without -a shows only running containers, which loses an entire class of containers that just died and are waiting to restart. When chasing a CrashLoop, -a is effectively mandatory. The ATTEMPT column tells you how many times kubelet has recreated the same container, and the POD ID column is the only link between a container and its sandbox. What crictl inspect returns contains the result of the translation from section 3, the spec as finally applied, so you can see with your own eyes whether the SecurityContext you asked for actually took effect. Field names vary by release, so check the exact field in the docs for the version you run.
10. The Config Keys That Govern Behaviour
Like the image side, the CRI side starts with a version split. Under version 2 in containerd 1.x everything sat below [plugins."io.containerd.grpc.v1.cri"]; version 3 in containerd 2.x divides it into the runtime side [plugins.'io.containerd.cri.v1.runtime'] and the image side [plugins.'io.containerd.cri.v1.images']. Get the section name wrong and containerd raises no error: it ignores the block and comes up on defaults. That is why, when a config change produces no change at all, the section name deserves suspicion before the value does.
# containerd 1.x
version = 2
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.k8s.io/pause:3.10.2"
max_container_log_line_size = 16384
enable_unprivileged_ports = true
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = false
# containerd 2.x
version = 3
[plugins.'io.containerd.cri.v1.images'.pinned_images]
sandbox = 'registry.k8s.io/pause:3.10.2'
[plugins.'io.containerd.cri.v1.runtime']
max_container_log_line_size = 16384
enable_unprivileged_ports = true
device_ownership_from_security_context = false
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc.options]
SystemdCgroup = true
The relocation of the pause image between versions is what trips people most often in practice. In 1.x it was a single sandbox_image line in the CRI section; in 2.x it moved under the image plugin's pinned_images as the sandbox key. When an air-gapped cluster that repointed the pause image at an internal registry upgrades to containerd 2.x, the old key is ignored and the default public registry address comes back, and the result is that not a single Pod starts anywhere on the node.
SystemdCgroup defaults to false in both versions. This value must match kubelet's cgroup driver setting. If kubelet runs with the systemd driver while the runtime runs cgroupfs, the two compute different cgroup paths for the same Pod, so containers do start but resource accounting drifts and the node becomes unstable. The symptom does not look like one Pod's problem; it looks like node-wide restarts, which is why it takes so long to pin down. kubeadm-installed clusters default to systemd, so forgetting to flip this to true on a node whose config was written by hand is the classic mistake.
max_container_log_line_size defaults to 16384 bytes, and the documented description is that a log line longer than the limit will be split into multiple lines. If an application emits single-line JSON logs and one of those lines exceeds 16KB, the log shipper receives two fragments of broken JSON. When parse failures appear only intermittently and only reproduce on certain requests, this value is worth suspecting first. Services that dump whole payloads into logs are the usual victims.
enable_unprivileged_ports defaults to true, and the documented description is that it configures net.ipv4.ip_unprivileged_port_start=0 for all containers not using host network. In other words, a non-root process inside the container can bind ports below 1024. device_ownership_from_security_context appears in the version 3 sample with a default of false.
11. NRI From an Operational Angle
Section 6 described what NRI is; here we only look at what it takes to actually turn it on. This is the config block the containerd docs show.
[plugins."io.containerd.nri.v1.nri"]
disable = true
disable_connections = false
plugin_config_path = "/etc/nri/conf.d"
plugin_path = "/opt/nri/plugins"
plugin_registration_timeout = "5s"
plugin_request_timeout = "2s"
socket_path = "/var/run/nri/nri.sock"
The first thing to notice is disable = true, that is, it is off by default. If you deployed an NRI plugin and nothing happens, check this one line before you check the plugin. socket_path defaults to /var/run/nri/nri.sock, and externally running plugins connect over that socket. If the plugin runs as a DaemonSet, that path has to be mounted as a hostPath, and a mismatched path simply fails to connect without raising an error. It is also worth remembering that plugin_request_timeout defaults to 2 seconds. NRI hooks sit on the container creation path, so a slow plugin slows Pod startup by exactly that much, and what happens on timeout depends on the plugin implementation and configuration. Budget for that latency before putting an NRI plugin into a production cluster.
One thing deserves an honest note. containerd's NRI document covers the config block and a behavioural overview but does not enumerate the event list. Which hooks fire at which point, and what each hook may change, has to come from the NRI repository itself: https://github.com/containerd/nri
12. A Worked Example: Tracking a Pod That Will Not Start
When a Pod is stuck in Pending or ContainerCreating, the calls kubelet issues over CRI come in a fixed order. RunPodSandbox creates the sandbox, PullImage fetches the image, CreateContainer creates the container, StartContainer starts it. Knowing which stage it stopped at narrows where to look down to one place, so walking this order is the fastest diagnostic path.
# 1) was the sandbox created
crictl pods --namespace my-ns
# 2) is the image on the node
crictl images | grep my-app
# 3) does a container record exist (including dead ones)
crictl ps -a -p POD_ID
# 4) why the start failed
crictl inspect CONTAINER_ID
crictl logs -p CONTAINER_ID
# 5) the runtime's own view
journalctl -u containerd -f
If the Pod is not in crictl pods at all, it failed at RunPodSandbox. Failures at this stage are usually either a pause image pull failure or a CNI plugin failure, and the containerd log is what separates the two. If the sandbox exists but crictl ps -a shows no container, it stopped at PullImage, and the image being absent from crictl images corroborates that. If the container record exists but its state never advances past Created, CreateContainer succeeded while StartContainer failed, and that is the moment to read what crictl inspect returns. A missing mount path, or a requested user that does not exist in the image, surfaces here. If the state is Exited, the container did start and then died, so from here it is an application problem rather than a runtime problem, and crictl logs -p gives you the previous instance's output.
13. Failure Cases and the Order to Diagnose Them
Not being able to fetch the pause image in an air-gapped cluster is the most destructive failure. No matter how carefully the application images were mirrored to the internal registry, if the sandbox cannot be created then no Pod starts on that node at all. Because the symptom spans the whole node rather than one workload, it is easy to mistake for a network outage. Check, in order: that no new sandbox appears in crictl pods, the image pull error in the containerd log, and the pause image address in the config. As noted above, that key lives in different places in 1.x and 2.x.
A cgroup driver mismatch drags on because the symptom is vague. If only one of kubelet and containerd uses systemd, Pods start but restart unstably, and replacing the node reproduces it. Reading the runtime side with crictl info and comparing it against the kubelet config is the fastest route.
A RuntimeClass whose handler name does not match a runtimes entry in the containerd config is also common. The Kubernetes RuntimeClass passes validation, but the moment kubelet hands that handler down over CRI, containerd cannot find a matching runtime and sandbox creation fails. The RuntimeClass handler and the key under containerd.runtimes must match character for character, and the corresponding shim binary must be on PATH. Verify by checking that the Runtime column of crictl pods shows the handler you expected.
crictl pointed at the wrong endpoint makes the whole diagnosis meaningless. If another runtime socket is left over on the node, crictl attaches to that one and shows an empty list, which leads to the wrong conclusion that there are no containers. Confirm what it is currently attached to with crictl info, then check that runtime-endpoint in /etc/crictl.yaml matches kubelet's --container-runtime-endpoint, as the first step.
Last is the case where only exec and attach fail. As section 4 showed, both require the API server to connect back to the streaming server at the URL kubelet returned. So even when ordinary Pod operation is perfectly healthy, a blocked path from the API server to the node's streaming address makes only exec time out. Firewalls, the node's advertised address and proxy settings are the candidates, and checking whether crictl exec works directly on the node immediately separates a runtime problem from a network path problem.
14. When to Use kubectl Instead of crictl
crictl bypasses the API server. That makes it powerful for diagnosis and dangerous for changing state. kubelet continuously watches the containers and sandboxes it created and reports their state to the API server. If a human deletes an object kubelet owns with crictl rm or crictl rmp, kubelet's view and the actual state diverge, and the result is either a recreation or a strange intermediate state. If you want a container back, delete the Pod or roll out the workload, which means doing it with kubectl.
Put simply, the boundary is this: use kubectl to learn what Kubernetes is trying to do, and crictl to learn what the runtime actually did. Events, scheduling decisions, Pod specs and controller state exist only on the kubectl side. Conversely, whether the sandbox was created, whether the image is really on the node, and how many times a container has been retried are more accurate on the crictl side. The point where the two views disagree is the location of the problem. And deleting anything with crictl is safest restricted to situations where kubelet is already dead and kubectl cannot reach.
15. References
- containerd CRI plugin config: https://github.com/containerd/containerd/blob/main/docs/cri/config.md (checked 2026-08-16)
- containerd NRI support: https://github.com/containerd/containerd/blob/main/docs/NRI.md (checked 2026-08-16)
- NRI repository: https://github.com/containerd/nri (checked 2026-08-16)
- crictl user guide: https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md (checked 2026-08-16)
- CRI API definition: https://github.com/kubernetes/cri-api/blob/master/pkg/apis/runtime/v1/api.proto (checked 2026-08-16)
16. Summary
containerd's CRI implementation is the core interface between Kubernetes and container runtimes. Pod-level isolation via Pod Sandbox, accurate CRI-to-OCI spec translation, WebSocket/SPDY-based streaming, multi-runtime support via RuntimeClass, and flexible extension via NRI are its key features. This layered design establishes containerd as a reliable container runtime for Kubernetes.