- Introduction: Why Kubernetes Cost Management Matters
- Core FinOps Principles Applied to Kubernetes
- The Main Causes of Wasted Kubernetes Resources
- Establishing Cost Visibility: Kubecost and OpenCost
- Resource Optimization Strategies
- Cutting Cost with Autoscaling
- Image Optimization and Storage Cost Reduction
- Monitoring Cost Metrics with Prometheus
- Cost Optimization Strategies per Cloud
- Failure Cases: Outages Caused by Over-Aggressive Cost Cutting
- FinOps Operations Checklist
- FinOps Team Culture and Cost Awareness
- The FinOps Maturity Model
- Summary and Recommendations
- References

Introduction: Why Kubernetes Cost Management Matters
As Kubernetes adoption accelerates, cloud cost is becoming a serious problem. According to the CNCF 2025 FinOps report, an average of 30-35% of enterprise cloud spending related to Kubernetes is wasted. This goes beyond simply over-allocating resources: it comes from structural problems such as a lack of cost visibility, unclear ownership between teams, and the absence of an optimization process.
The FinOps Foundation defines FinOps as "an operational framework in which engineering, finance, and business teams collaborate on data to maximize the business value of the cloud". Traditional IT infrastructure ran on a CapEx (capital expenditure) model where a single purchase was the end of it, but the cloud runs on an OpEx (operating expenditure) model that incurs cost every hour and every minute. In that environment FinOps is not optional but essential.
This article covers FinOps strategies specific to Kubernetes environments. From establishing cost visibility to resource optimization, autoscaling, and team culture, it gives you a guide you can apply directly in practice.
Core FinOps Principles Applied to Kubernetes
The Three FinOps Principles
The core principles put forward by the FinOps Foundation are as follows.
- Teams take ownership of their own cloud usage - engineering teams have to be aware of cost and make the optimization decisions
- Decisions are driven by the business value of the cloud - the goal is cost efficiency relative to business value, not raw cost reduction
- A centralized team drives FinOps - tools, processes, and best practices are managed centrally, while each team does the execution
The Inform, Optimize, Operate Cycle
FinOps runs as a repeating three-stage cycle.
| Stage | Goal | Applied to Kubernetes |
|---|---|---|
| Inform | Establish cost visibility | Adopt Kubecost/OpenCost, per-namespace cost dashboards |
| Optimize | Execute cost optimization | Request/Limit tuning, spot instances, cleaning up idle resources |
| Operate | Continuous governance | Cost alerts, regular reviews, per-team budget management |
The Main Causes of Wasted Kubernetes Resources
Before you start optimizing cost, you need to understand exactly where the waste comes from.
1. Excessive Request/Limit Settings
This is the most common cause. Developers tend to set high values to be "safe".
# Problem: resource requests far above actual usage
apiVersion: v1
kind: Pod
metadata:
name: over-provisioned-app
spec:
containers:
- name: app
image: my-app:latest
resources:
requests:
cpu: '2' # actual usage: 200m
memory: '4Gi' # actual usage: 512Mi
limits:
cpu: '4'
memory: '8Gi'
In the example above, CPU is requested at 10 times actual usage and memory at 8 times. A single Pod like this is not a big problem, but if 100 Pods look this way, the waste runs to thousands of dollars a month.
2. No Scaling Policy
This is when the Pod count does not shrink as traffic falls, or the same resources stay up at night and on weekends.
3. Idle Resources Left Behind
This is when PersistentVolumes, LoadBalancer Services, and test namespaces that are no longer used are never cleaned up.
4. Node Fragmentation
This is when small Pods are spread across many nodes and the utilization of each node drops.
# Check per-node resource utilization
kubectl top nodes
# Example output
# NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
# node-1 800m 20% 2Gi 25%
# node-2 600m 15% 1.5Gi 18%
# node-3 400m 10% 1Gi 12%
# => all three nodes below 25% utilization - can be consolidated onto one node
Establishing Cost Visibility: Kubecost and OpenCost
The first step in cost optimization is knowing "how much you are spending right now".
Installing and Configuring OpenCost
OpenCost is an official CNCF project and the open source standard for Kubernetes cost monitoring. It integrates with Prometheus and collects cost data in real time.
# Install OpenCost with Helm
helm repo add opencost https://opencost.github.io/opencost-helm-chart
helm repo update
helm install opencost opencost/opencost \
--namespace opencost-system \
--create-namespace \
--set opencost.prometheus.internal.enabled=true \
--set opencost.ui.enabled=true
OpenCost's custom pricing configuration lets you reflect your actual cloud rates.
# opencost-custom-pricing.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: opencost-custom-pricing
namespace: opencost-system
data:
default.json: |
{
"provider": "custom",
"description": "Custom pricing for on-prem + cloud hybrid",
"CPU": "0.031611",
"spotCPU": "0.012644",
"RAM": "0.004237",
"spotRAM": "0.001694",
"storage": "0.000138888",
"GPU": "0.95"
}
kubectl apply -f opencost-custom-pricing.yaml
Installing and Configuring Kubecost
Kubecost is a commercial/open source hybrid tool built on OpenCost that offers richer features (alerts, recommendations, governance).
# Install Kubecost (Free Tier)
helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm repo update
helm install kubecost kubecost/cost-analyzer \
--namespace kubecost \
--create-namespace \
--set kubecostToken="YOUR_TOKEN" \
--set prometheus.server.persistentVolume.enabled=true \
--set prometheus.server.persistentVolume.size=32Gi
The Kubecost API lets you query cost data programmatically.
# Query cost by namespace (last 7 days)
curl -s "http://kubecost.example.com/model/allocation?window=7d&aggregate=namespace" | \
python3 -m json.tool
# Example output (simplified)
# {
# "data": [{
# "production": {
# "cpuCost": 245.67,
# "ramCost": 123.45,
# "pvCost": 34.56,
# "totalCost": 403.68
# },
# "staging": {
# "cpuCost": 89.12,
# "ramCost": 45.23,
# "pvCost": 12.34,
# "totalCost": 146.69
# }
# }]
# }
Cost Monitoring Tool Comparison
| Feature | OpenCost | Kubecost Free | Kubecost Enterprise | CloudHealth | Spot.io (NetApp) |
|---|---|---|---|---|---|
| License | Open source (Apache 2.0) | Free (15 days of data) | Commercial | Commercial | Commercial |
| CNCF project | O | X (built on it) | X | X | X |
| Real-time cost monitoring | O | O | O | O | O |
| Per-namespace cost | O | O | O | O | O |
| Cost saving recommendations | X | Basic | Advanced | Advanced | Advanced |
| Multi-cluster support | O (manual) | X | O | O | O |
| Alerts/alarms | X | Basic | Advanced | Advanced | Advanced |
| Spot instance management | X | X | X | X | O |
| Data retention | Depends on Prometheus | 15 days | Unlimited | Unlimited | Unlimited |
| Cost allocation accuracy | High | High | Very high | High | High |
| Installation difficulty | Low | Low | Medium | High | Medium |
| Monthly cost | Free | Free | Per cluster | Negotiated | % of savings |
Recommendation: a realistic path is for a small team to start with OpenCost and switch to Kubecost Enterprise or Spot.io once spending grows. 68% of FinOps Foundation member companies followed this path.
Resource Optimization Strategies
Request/Limit Tuning: Automatic Right-Sizing with VPA
The Vertical Pod Autoscaler (VPA) analyzes actual resource usage patterns and recommends appropriate Request/Limit values.
# vpa-recommendation.yaml
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: app-vpa
namespace: production
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: my-app
updatePolicy:
updateMode: 'Off' # recommendations only, never applied automatically (safe)
resourcePolicy:
containerPolicies:
- containerName: app
minAllowed:
cpu: '100m'
memory: '128Mi'
maxAllowed:
cpu: '2'
memory: '4Gi'
controlledResources: ['cpu', 'memory']
# Check the VPA recommendations
kubectl describe vpa app-vpa -n production
# Example output
# Recommendation:
# Container Recommendations:
# Container Name: app
# Lower Bound:
# Cpu: 150m
# Memory: 256Mi
# Target:
# Cpu: 250m
# Memory: 512Mi
# Uncapped Target:
# Cpu: 250m
# Memory: 512Mi
# Upper Bound:
# Cpu: 800m
# Memory: 1Gi
Caution: VPA's updateMode: "Auto" restarts Pods. In-Place Resource Resize (KEP-1287), supported from Kubernetes 1.33, lets you adjust resources without a restart. In production you must start in "Off" mode, review the recommended values, and then apply them gradually.
Per-Namespace ResourceQuota
Capping resource usage per team prevents cost from running away.
# resourcequota-team-backend.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: team-backend-quota
namespace: team-backend
spec:
hard:
requests.cpu: '20'
requests.memory: '40Gi'
limits.cpu: '40'
limits.memory: '80Gi'
persistentvolumeclaims: '10'
services.loadbalancers: '2'
pods: '50'
# Cost angle: cap the number of LoadBalancer services (about $18 per month each on AWS)
# Check ResourceQuota usage
kubectl describe resourcequota team-backend-quota -n team-backend
# Example output
# Name: team-backend-quota
# Namespace: team-backend
# Resource Used Hard
# -------- ---- ----
# limits.cpu 12 40
# limits.memory 24Gi 80Gi
# persistentvolumeclaims 3 10
# pods 15 50
# requests.cpu 6 20
# requests.memory 12Gi 40Gi
# services.loadbalancers 1 2
LimitRange Configuration
This enforces default resource values and a valid range at the individual Pod/Container level.
# limitrange-default.yaml
apiVersion: v1
kind: LimitRange
metadata:
name: default-limits
namespace: team-backend
spec:
limits:
- type: Container
default: # default Limit (applied automatically when Request is unset)
cpu: '500m'
memory: '512Mi'
defaultRequest: # default Request
cpu: '100m'
memory: '128Mi'
min: # minimum (nothing below this)
cpu: '50m'
memory: '64Mi'
max: # maximum (nothing above this)
cpu: '4'
memory: '8Gi'
- type: PersistentVolumeClaim
min:
storage: '1Gi'
max:
storage: '100Gi' # cap on PVC size
kubectl apply -f limitrange-default.yaml
# Defaults are applied automatically to a Pod created without Request/Limit
kubectl run test-pod --image=nginx -n team-backend
kubectl describe pod test-pod -n team-backend | grep -A 5 "Limits\|Requests"
Node Pool Optimization: Using Spot/Preemptible Instances
Spot instances are 60-90% cheaper than On-Demand, but the cloud provider can reclaim them at any time. Used correctly, they can cut Kubernetes cost dramatically.
# spot-tolerant-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: batch-processor
namespace: production
spec:
replicas: 5
selector:
matchLabels:
app: batch-processor
template:
metadata:
labels:
app: batch-processor
spec:
# Schedule onto spot nodes
nodeSelector:
node.kubernetes.io/capacity-type: spot
tolerations:
- key: 'spot'
operator: 'Equal'
value: 'true'
effect: 'NoSchedule'
# Graceful shutdown - terminate cleanly when the spot instance is reclaimed
terminationGracePeriodSeconds: 120
containers:
- name: processor
image: batch-processor:latest
resources:
requests:
cpu: '500m'
memory: '1Gi'
limits:
cpu: '1'
memory: '2Gi'
# Handle the spot instance reclamation signal
lifecycle:
preStop:
exec:
command: ['/bin/sh', '-c', 'kill -SIGTERM 1 && sleep 90']
# Pod Disruption Budget configuration
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: batch-processor
Workloads that suit spot instances versus workloads that do not
| Suitable workloads | Unsuitable workloads |
|---|---|
| Batch jobs / data processing | Single-instance databases |
| CI/CD pipelines | Stateful services (Kafka, Redis) |
| Stateless web servers (many replicas) | Long-running transactions |
| Dev/test environments | Real-time stream processing |
| ML training (with checkpointing) | Leader-election based services |
Cutting Cost with Autoscaling
Karpenter: Next-Generation Node Provisioning
Karpenter is a node provisioner that started at AWS and is now expanding to multi-cloud. It offers faster and more flexible node management than Cluster Autoscaler.
# karpenter-nodepool.yaml
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: cost-optimized
spec:
template:
spec:
requirements:
- key: kubernetes.io/arch
operator: In
values: ['amd64']
- key: karpenter.sh/capacity-type
operator: In
values: ['spot', 'on-demand'] # spot first, On-Demand on failure
- key: karpenter.k8s.aws/instance-category
operator: In
values: ['c', 'm', 'r'] # compute / general purpose / memory optimized
- key: karpenter.k8s.aws/instance-generation
operator: Gt
values: ['5'] # 6th generation and newer only (better price-performance)
nodeClassRef:
group: karpenter.k8s.aws
kind: EC2NodeClass
name: default
limits:
cpu: '100'
memory: '400Gi'
disruption:
consolidationPolicy: WhenEmptyOrUnderutilized
consolidateAfter: 30s
# underutilized nodes are consolidated automatically after 30 seconds
weight: 10 # preferred over other NodePools
# karpenter-ec2nodeclass.yaml
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
name: default
spec:
amiSelectorTerms:
- alias: 'al2023@latest'
subnetSelectorTerms:
- tags:
karpenter.sh/discovery: 'my-cluster'
securityGroupSelectorTerms:
- tags:
karpenter.sh/discovery: 'my-cluster'
blockDeviceMappings:
- deviceName: /dev/xvda
ebs:
volumeSize: 50Gi
volumeType: gp3
encrypted: true
Karpenter vs Cluster Autoscaler
| Item | Karpenter | Cluster Autoscaler |
|---|---|---|
| Scale-up speed | Seconds to 1 minute | 2-5 minutes |
| Instance type selection | Automatic best fit (many types) | Fixed per node group |
| Scale-down | Aggressive consolidation | Conservative (waits 10+ minutes) |
| Spot handling | Native support, automatic switchover | Needs a separate node group |
| Multi-AZ spread | Automatic | Configured per node group |
| Bin packing efficiency | High (chosen from Pod size) | Low (fixed node size) |
| Node fragmentation | Resolved automatically (consolidation) | Manual |
| Cloud support | AWS (GA), Azure (Preview) | AWS, GCP, and Azure |
Practical tip: Karpenter's
consolidationPolicy: WhenEmptyOrUnderutilizedautomatically moves Pods off nodes with low resource utilization onto other nodes and terminates those nodes. That alone can cut node cost by 20-30% (see the best practices in the official Karpenter documentation).
Automating Night and Weekend Scale-Down
Automatically shrinking non-production workloads outside business hours can save a considerable amount.
# cronjob-scaledown.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: nighttime-scaledown
namespace: kube-system
spec:
schedule: '0 22 * * 1-5' # 22:00 on weekdays (KST)
jobTemplate:
spec:
template:
spec:
serviceAccountName: scaler-sa
containers:
- name: scaler
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- |
# Scale every Deployment in the staging namespace down to 0
for deploy in $(kubectl get deploy -n staging -o name); do
kubectl scale $deploy --replicas=0 -n staging
done
echo "Scaled down staging at $(date)"
restartPolicy: OnFailure
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: morning-scaleup
namespace: kube-system
spec:
schedule: '0 8 * * 1-5' # 08:00 on weekdays (KST)
jobTemplate:
spec:
template:
spec:
serviceAccountName: scaler-sa
containers:
- name: scaler
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- |
# Restore the staging namespace Deployments to their original state
kubectl scale deploy/api-server --replicas=3 -n staging
kubectl scale deploy/web-frontend --replicas=2 -n staging
kubectl scale deploy/worker --replicas=2 -n staging
echo "Scaled up staging at $(date)"
restartPolicy: OnFailure
Image Optimization and Storage Cost Reduction
Container Image Optimization
A large container image increases registry storage cost, image pull time, and network transfer cost alike.
# Bad: enormous image (1.2GB+)
FROM node:20
WORKDIR /app
COPY . .
RUN npm install
CMD ["node", "server.js"]
# Good: optimized with a multi-stage build (under 150MB)
FROM node:20-slim AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
FROM gcr.io/distroless/nodejs20-debian12
WORKDIR /app
COPY /app/dist ./dist
COPY /app/node_modules ./node_modules
CMD ["dist/server.js"]
Cleaning Up Unused PersistentVolumes
Abandoned PVs/PVCs keep generating cloud storage cost.
# Check PVs in the Released state (unbound but not deleted)
kubectl get pv --field-selector=status.phase=Released
# Find PVCs that are not in use (not mounted by any Pod)
kubectl get pvc --all-namespaces -o json | \
python3 -c "
import json, sys
data = json.load(sys.stdin)
for pvc in data['items']:
ns = pvc['metadata']['namespace']
name = pvc['metadata']['name']
phase = pvc['status'].get('phase', 'Unknown')
if phase == 'Bound':
print(f'{ns}/{name} - Bound but check if any pod uses it')
"
# Check whether any Pod uses a given PVC
kubectl get pods --all-namespaces -o json | \
python3 -c "
import json, sys
data = json.load(sys.stdin)
used_pvcs = set()
for pod in data['items']:
volumes = pod['spec'].get('volumes', [])
for vol in volumes:
if 'persistentVolumeClaim' in vol:
ns = pod['metadata']['namespace']
pvc_name = vol['persistentVolumeClaim']['claimName']
used_pvcs.add(f'{ns}/{pvc_name}')
for pvc in sorted(used_pvcs):
print(f'IN USE: {pvc}')
"
Idle Resource Detection Script
#!/bin/bash
# idle-resource-detector.sh
# Script that detects idle resources to find cost saving opportunities
echo "=== Idle resource detection report ==="
echo "Date: $(date)"
echo ""
# 1. Deployments left at 0 replicas instead of being deleted
echo "--- Deployments with 0 replicas ---"
kubectl get deploy --all-namespaces -o json | \
python3 -c "
import json, sys
data = json.load(sys.stdin)
for d in data['items']:
if d['spec'].get('replicas', 1) == 0:
print(f\" {d['metadata']['namespace']}/{d['metadata']['name']}\")
"
# 2. Jobs that have been Completed for more than 7 days
echo ""
echo "--- Jobs completed more than 7 days ago ---"
kubectl get jobs --all-namespaces --field-selector=status.successful=1 \
-o custom-columns="NAMESPACE:.metadata.namespace,NAME:.metadata.name,COMPLETED:.status.completionTime"
# 3. LoadBalancer Services with no external traffic
echo ""
echo "--- LoadBalancer type Services (currently billed) ---"
kubectl get svc --all-namespaces --field-selector=spec.type=LoadBalancer \
-o custom-columns="NAMESPACE:.metadata.namespace,NAME:.metadata.name,EXTERNAL-IP:.status.loadBalancer.ingress[0].hostname"
Monitoring Cost Metrics with Prometheus
Wiring OpenCost into Prometheus lets you monitor cost trends in real time from a Grafana dashboard.
# prometheus-cost-alerts.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: cost-alerts
namespace: monitoring
spec:
groups:
- name: cost-optimization
rules:
# Detect containers using 20% or less of their CPU request
- alert: LowCPUUtilization
expr: |
(
sum(rate(container_cpu_usage_seconds_total[5m])) by (namespace, pod, container)
/
sum(kube_pod_container_resource_requests{resource="cpu"}) by (namespace, pod, container)
) < 0.2
for: 24h
labels:
severity: warning
category: cost
annotations:
summary: 'CPU utilization below 20% of Request'
description: '{{ `{{ $labels.namespace }}` }}/{{ `{{ $labels.pod }}` }} has stayed below 20% of its CPU Request for 24 hours. Lowering the Request value is recommended.'
# Detect containers using 30% or less of their memory request
- alert: LowMemoryUtilization
expr: |
(
sum(container_memory_working_set_bytes) by (namespace, pod, container)
/
sum(kube_pod_container_resource_requests{resource="memory"}) by (namespace, pod, container)
) < 0.3
for: 24h
labels:
severity: warning
category: cost
annotations:
summary: 'Memory utilization below 30% of Request'
description: '{{ `{{ $labels.namespace }}` }}/{{ `{{ $labels.pod }}` }} has stayed below 30% of its memory Request for 24 hours.'
# Daily per-namespace cost over the threshold
- alert: NamespaceCostThresholdExceeded
expr: |
sum(
sum_over_time(opencost_container_cost_cpu_hourly[24h]) +
sum_over_time(opencost_container_cost_memory_hourly[24h])
) by (namespace) > 100
labels:
severity: critical
category: cost
annotations:
summary: 'Namespace daily cost threshold exceeded'
description: 'The daily cost of the {{ `{{ $labels.namespace }}` }} namespace exceeded 100 USD.'
Cost Optimization Strategies per Cloud
AWS EKS Cost Optimization
# Query the AWS Savings Plans recommendations
aws ce get-savings-plans-purchase-recommendation \
--savings-plans-type COMPUTE_SP \
--term-in-years ONE_YEAR \
--payment-option NO_UPFRONT \
--lookback-period-in-days SIXTY_DAYS
# Reserved Instance recommendations for EKS nodes
aws ce get-reservation-purchase-recommendation \
--service "Amazon Elastic Compute Cloud - Compute" \
--lookback-period-in-days SIXTY_DAYS
GCP GKE Cost Optimization
# Check the GKE cost recommendations
gcloud recommender recommendations list \
--recommender=google.compute.instance.MachineTypeRecommender \
--project=my-project \
--location=asia-northeast3-a \
--format="table(content.overview.resourceName, content.overview.recommendedMachineType.name, primaryImpact.costProjection.cost.units)"
# Check the Committed Use Discounts
gcloud compute commitments list --project=my-project
Multi-Cloud Cost Comparison
| Item | AWS EKS | GCP GKE | Azure AKS |
|---|---|---|---|
| Control plane cost | $73 per month (per cluster) | Free (Standard) / $73 per month (Enterprise) | Free (Standard) / $73 per month (Premium) |
| Spot discount | 60-90% | 60-91% | 60-90% |
| Minimum spot warning | 2-minute warning | 30-second warning | 30-second warning |
| Savings Plan/CUD | Compute Savings Plans | Committed Use Discounts | Azure Reservations |
| Maximum commitment discount | 72% (3 years, all upfront) | 70% (3-year commitment) | 72% (3-year reservation) |
| Autopilot/serverless | Fargate | GKE Autopilot | AKS Virtual Nodes |
Failure Cases: Outages Caused by Over-Aggressive Cost Cutting
Case 1: A Large-Scale Outage from Running 100% Spot Instances
One startup ran every node of its production cluster on spot instances to maximize savings.
What happened:
- AWS reclaimed spot capacity on a large scale in one region
- 70% of all nodes terminated at the same time
- With no node left to schedule Pods on, the service went down completely
- Provisioning On-Demand nodes took 8 minutes
Lessons:
- Core production services must be placed on On-Demand nodes
- Keep the spot share below 70% of the total
- Guarantee a minimum number of available Pods with a PodDisruptionBudget
# Required: PodDisruptionBudget configuration
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: api-server-pdb
namespace: production
spec:
minAvailable: 2 # always keep at least 2 Pods
selector:
matchLabels:
app: api-server
Case 2: Frequent OOMs from Setting Resource Requests Too Low
During cost optimization the memory Request of every Pod was set tightly against real usage, and OOMKilled became frequent at peak traffic hours.
Lessons:
- Set Request from P95 usage plus a 20% buffer rather than from P99 usage
- Set Limit to 1.5-2x the Request to leave burst headroom
- Do not follow the VPA recommendations blindly; adjust them for the traffic pattern
Case 3: Developer Productivity Loss from Deleting Test Environments
A policy of deleting every dev/test environment at night was introduced to save cost, but the time difference with overseas teams caused serious collaboration problems.
Lessons:
- Cost saving has to be balanced against developer experience (DX)
- Scaling down (replicas=0) is safer than deleting outright
- For a global team, set the schedule with each time zone in mind
FinOps Operations Checklist
Weekly Cost Review Checklist
| Check item | Owner | Tool |
|---|---|---|
| Review per-namespace cost trends | FinOps team | Kubecost/OpenCost |
| List Pods below 20% CPU/memory utilization | SRE | Prometheus/Grafana |
| Check the spot instance ratio (target: 50-70%) | Infra | Cloud console |
| Clean up unused PV/PVC | Dev team | kubectl scripts |
| Check the number of LoadBalancer Services | Infra | kubectl |
| Clean up old tags in the image registry | DevOps | Registry API |
Monthly Cost Review Checklist
| Check item | Owner | Tool |
|---|---|---|
| Analyze the cost change against the previous month | FinOps team | Cloud billing |
| Check Savings Plan/CUD coverage | FinOps team | Cloud console |
| Update Request/Limit from the VPA recommendations | Dev team | VPA Recommender |
| Review node instance type optimization | Infra | Karpenter logs |
| Share the per-team cost allocation report | FinOps team | Kubecost |
| Analyze the cause of cost anomalies | SRE | Cloud Cost Explorer |
Quarterly Cost Review Checklist
| Check item | Owner | Tool |
|---|---|---|
| Review Reserved Instance/CUD renewals | FinOps team | Cloud console |
| Architecture-level cost optimization (consolidating microservices, etc.) | Architect | Design review |
| Update the cost forecasting model | FinOps team | Spreadsheet/BI |
| FinOps maturity self-assessment | FinOps team | FinOps Foundation framework |
FinOps Team Culture and Cost Awareness
Cost Tagging Strategy
Every Kubernetes resource needs a consistent set of tags (labels) so that cost can be tracked accurately.
# Standard label definitions for cost tracking
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
namespace: production
labels:
app.kubernetes.io/name: api-server
app.kubernetes.io/part-of: payment-platform
# FinOps cost tags
cost-center: 'engineering'
team: 'backend'
env: 'production'
project: 'payment-v2'
spec:
replicas: 3
selector:
matchLabels:
app.kubernetes.io/name: api-server
template:
metadata:
labels:
app.kubernetes.io/name: api-server
cost-center: 'engineering'
team: 'backend'
env: 'production'
project: 'payment-v2'
spec:
containers:
- name: api-server
image: api-server:v2.1.0
resources:
requests:
cpu: '500m'
memory: '1Gi'
limits:
cpu: '1'
memory: '2Gi'
Enforcing Cost Tags with OPA/Gatekeeper
# cost-label-constraint.yaml
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: k8srequiredcostlabels
spec:
crd:
spec:
names:
kind: K8sRequiredCostLabels
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8srequiredcostlabels
required_labels := {"cost-center", "team", "env", "project"}
violation[{"msg": msg}] {
provided := {label | input.review.object.metadata.labels[label]}
missing := required_labels - provided
count(missing) > 0
msg := sprintf("required cost tags are missing: %v", [missing])
}
---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredCostLabels
metadata:
name: require-cost-labels
spec:
match:
kinds:
- apiGroups: ['apps']
kinds: ['Deployment', 'StatefulSet', 'DaemonSet']
namespaces: ['production', 'staging']
Building a Cost-Aware Culture
FinOps cannot succeed on tools alone. It needs a culture where the whole team is aware of cost and takes part in optimization.
Key elements of a cost-aware culture:
- Cost visibility: share the per-team cost dashboard every week
- Cost ownership: each team is responsible for the cost of its own namespace
- Incentives: build a culture that shares and recognizes cost saving wins
- Education: developers have to understand what resource Request/Limit mean and what they affect
- Automation: reduce manual work and pursue policy-driven automatic optimization
The FinOps Maturity Model
The FinOps Foundation defines organizational FinOps maturity in three stages.
| Stage | Characteristics | Kubernetes indicators |
|---|---|---|
| Crawl (starting) | Basic cost visibility, manual optimization | Adopt OpenCost, start monthly cost reviews |
| Walk (developing) | Per-team cost allocation, automation begins | Kubecost alerts, VPA recommendations applied, spot 50%+ |
| Run (mature) | Real-time optimization, cost forecasting, culture settled | Karpenter auto-consolidation, cost forecasting model, a FinOps team |
Summary and Recommendations
Quick Wins You Can Run Immediately
- Install OpenCost (1 hour): cost visibility straight away
- Deploy VPA in recommendation mode (30 minutes): start collecting right-sizing data
- Clean up unused resources (2 hours): delete Released PVs, empty namespaces, and old Jobs
- Apply LimitRange (1 hour): force defaults onto Pods with no Request set
Mid-Term Optimization (1-3 Months)
- Adopt Kubecost or a commercial tool and start per-team cost allocation
- Introduce spot instances (start with non-production and expand to production gradually)
- Adopt Karpenter and turn on automatic node consolidation
- Deploy the night/weekend scale-down CronJob
Long-Term Strategy (3-12 Months)
- Form a dedicated FinOps team or role
- Optimize Savings Plans/CUDs
- Build a cost forecasting model (based on historical data)
- Embed a cost-aware culture (education, dashboards, incentives)
Kubernetes cost optimization is not a one-off project but a continuous process. The key is to repeat the FinOps Inform-Optimize-Operate cycle and raise your maturity step by step. The most important first step is "knowing how much you are spending right now". Install OpenCost and start your first cost review by following the checklists in this article.
References
- FinOps Foundation - Kubernetes Best Practices - guidelines for applying FinOps to Kubernetes
- Kubecost Documentation - Kubecost installation, configuration, and API usage
- OpenCost - CNCF Project - OpenCost architecture and installation guide
- AWS EKS Best Practices - Cost Optimization - the official AWS EKS cost optimization guide
- Karpenter Documentation - Best Practices - Karpenter node provisioning and consolidation strategy
- GCP GKE Cost Optimization - the official GKE cost optimization documentation
- Azure AKS Cost Optimization - AKS cost management best practices