Managing over 1,400 Amazon EKS clusters at scale, Salesforce replaced static workload sizing with a combined Vertical and Horizontal Pod Autoscaling approach. This innovative strategy dynamically adjusts resource allocations and pod replicas, yielding more than $5 million in cloud infrastructure savings while boosting reliability and developer self-service.
- Combines VPA and HPA for precise, dynamic workload resource tuning and replica scaling
- Migrated cluster autoscaling to Karpenter for more efficient compute utilization
- Achieved over $5 million in annual infrastructure cost savings globally
Infrastructure signal
Salesforce operates one of the world’s largest multi-tenant Kubernetes platforms, spanning more than 1,400 Amazon EKS clusters with thousands of nodes and workloads. This scale exposed inefficiencies inherent in static resource configurations, where over-provisioning led to substantial, recurring cloud spend without corresponding utilization.
To remedy this, Salesforce’s engineering team implemented a multi-dimensional autoscaling strategy combining the Vertical Pod Autoscaler (VPA) and Horizontal Pod Autoscaler (HPA). This integration dynamically adjusts pod CPU and memory allocations based on metrics, and independently scales replica counts to meet real-time demand. Additionally, they transitioned from traditional Cluster Autoscaler to Karpenter, an open-source node autoscaler optimized for rapid, cost-effective provisioning in Amazon EKS.
Developer impact
The move to dynamic autoscaling not only cuts costs but also simplifies developer workflows. By embracing Karpenter, internal teams now enjoy self-serve infrastructure capabilities, reducing dependency on operations for provisioning and scaling compute resources. Developers can focus on deploying applications confident that resource sizing will adapt automatically.
Moreover, the combined use of VPA and HPA supports various workload types effectively. For example, stateful or leader-follower patterns benefit from VPA-only scaling, which right-sizes resources without changing pod counts, while stateless workloads leverage HPA to scale replicas horizontally. This nuanced approach improves application reliability and performance in production environments, including high-demand Istio control plane components.
What teams should watch
Engineering and cloud infrastructure teams managing Kubernetes at scale should consider adopting multi-dimensional autoscaling patterns. Implementing VPA alongside HPA enables finely tuned resource management that handles diverse workload characteristics, reducing waste and operational overhead. Teams should also evaluate node autoscaling solutions like Karpenter for more agile, cost-conscious infrastructure scaling on Amazon EKS.
It is critical to carefully select autoscaling metrics and ensure that vertical and horizontal autoscalers do not conflict by scaling the same dimension simultaneously. Salesforce’s approach of independently tuning resource allocation and replica counts serves as a best practice blueprint. Observability enhancements are also vital to monitor autoscaler behavior and prevent performance regressions as cluster size grows.