The newest Kubernetes 1.36 release introduces an alpha feature that significantly improves the scalability of controllers handling large resource sets by moving event filtering upstream to the API server. This server-side sharding reduces redundant event processing in horizontally scaled controller replicas and paves the way for more efficient resource use and lower cloud costs.
- API server filters watch events per replica, reducing duplicate processing.
- Significant decrease in CPU, memory, and network use for large-scale controllers.
- Alpha feature requires explicit enablement; feedback from large-cluster operators sought.
Infrastructure signal
Kubernetes 1.36 introduces server-side sharded list and watch to tackle scalability issues of controllers when clusters grow beyond tens of thousands of nodes. By shifting event filtering from each controller replica to the API server, this feature lowers the volume of data transmitted and processed per replica, effectively reducing infrastructure resource consumption.
The mechanism relies on hash-range selectors sent with each API request, enabling the API server to return only the subset of resource events each replica owns. This reduces duplicated CPU cycles spent on deserializing and discarding irrelevant events and cuts network traffic scaling with replicas rather than shard size, improving overall cluster reliability and cost efficiency.
Developer impact
Controller and operator teams need to adapt their informer configurations to inject shardSelector filters in ListOptions. This involves specifying hash ranges for each replica's shard, requiring changes in deployment manifests or orchestration pipelines managing controller replicas.
The feature provides shardInfo feedback within API responses to confirm filtering, but fallbacks to client-side filtering remain necessary when the server does not support the selector. Being alpha, users must explicitly enable the ShardedListAndWatch feature gate in their API server, testing workflows carefully to avoid disruptions in event handling and resource synchronization.
What teams should watch
Cloud platform teams running large Kubernetes deployments should evaluate leveraging server-side sharded list and watch to optimize API server load and controller efficiency. Monitoring observability pipelines and metrics related to informers and API call patterns will show impact over time.
Controller authors and operators should track upstream Kubernetes updates for this feature's progression beyond alpha and gather feedback from real-world usage, particularly concerning compatibility with custom controllers and APIs that reference metadata.uid or metadata.namespace fields.
Teams must be prepared for scenarios where shardSelector is ignored and fallback to client-side sharding or filtering is required, maintaining robustness in event processing. Collaborating in community channels like SIG API Machinery is recommended to share implementation experiences and request enhancements.