Between late August and early September 2026, the GLM-5.3 Worker AI model experienced an uncommon flaw exposing KV cache pages from prior requests under very long prompt conditions. Cloudflare swiftly identified the cause linked to a token pool size mismatch and an underlying inference engine bug, applying multiple remediation steps to ensure tenant isolation and reliability.
- KV cache pool resized to support full GLM-5.3 context window
- SGLang inference engine patched to safely abort over-length requests
- Model offline less than 24 hours with no deliberate exploit found
Infrastructure signal
The root cause of the cross-request KV cache exposure was a configuration inconsistency: GLM-5.3 was deployed with a token context window of 1,048,576 tokens, but the GPU prefill KV cache pool could only hold 681,280 tokens. This disparity meant that large prompts caused the inference engine to trigger a fallback path that did not clear previous request caches fully, resulting in data from prior sessions leaking into new responses.
To remediate, Cloudflare doubled the GPU pool size to 2.04 million tokens, exceeding the configured context window for GLM-5.3, allowing all valid requests to fit without invoking cache fallback logic. Additionally, the open-source SGLang engine was updated to properly reject and abort incomplete or oversized requests, preventing stale cache usage regardless of pool size. These combined improvements strengthen isolation guarantees and reduce fragility in token pool sizing against future model or deployment changes.
Developer impact
Developers relying on GLM-5.3 Workers AI should note that long prompt handling behavior has been hardened, eliminating previously silent fallback paths that could leak data across requests under rare conditions. Prompt input lengths up to the maximum configured context window can now be safely processed without unexpected cache contamination.
Deployment-wise, teams using or upgrading SGLang-based inference engines need to incorporate rejection-path safety checks to better handle resource constraints or configuration mismatches. Observability and monitoring around cache pool sizes and prompt context lengths should be enhanced to proactively identify any misalignments before they impact response isolation or reliability.
What teams should watch
Infrastructure and security teams should continue monitoring for cache isolation issues, focusing on token pool sizing and inference engine version consistency. Any future model upgrades or configuration changes must be validated against GPU memory pool capacity to prevent similar bugs from triggering stale cache leakage.
Developer teams should update CI testing to include long-context workflows and simulate over-length prompts to verify proper request rejection behavior. Logging and observability improvements in inference engines, particularly around cache state changes and request aborts, will provide early warning signals if the system encounters unexpected input sizes or memory conditions.
Platform teams managing multi-tenant AI services must be vigilant about disaggregated inference architectures and how state like KV cache is shared or cleared between requests. This incident underscores the need for fail-safe mechanisms within inference pipelines to preserve strict tenant data isolation while optimizing GPU resource usage.