Context
EKS clusters serving 50+ microservices across development and production, scaled by Cluster Autoscaler on On-Demand instances.
The problem
Cluster Autoscaler scales node groups, so capacity arrives in the shapes you defined in advance rather than the shapes the pending pods actually need. That mismatch shows up as paid-for headroom. The obvious saving, Spot instances, carries an obvious risk, and applying it uniformly would trade cost for reliability on services that cannot absorb an interruption.
- step: Pending podsreal requirements
- step: Karpenterprovisions to fit
- control: Interruption toleranceper workload
- result: Spoteligible non-prod
- result: On-Demandcritical services
Approach
- 01
Replace Cluster Autoscaler with Karpenter
Karpenter provisions nodes against the actual requirements of pending pods rather than scaling predefined groups, which removes most of the shape mismatch.
- 02
Separate workloads by interruption tolerance
The saving comes from knowing which workloads can be interrupted. Eligible non-production workloads moved to Spot; critical services stayed on On-Demand deliberately.
- 03
Keep the split explicit
Capacity type is a property of the workload, not a global cluster setting, so the reliability trade is made per service and stays visible.