Saturday, 31 October 2026Global tech news & guides — what changed, and what it means for you.
AuTechWire
Original editorial illustration of a dark server hall at night: two adjacent glowing memory-grid panels, warm amber light from the left panel spilling across the boundary into its neighbour, and a long ribbon of amber light stretching past the edge of its own panel

Cloudflare’s AI Cache Leak: How 155 Responses Got Other Users’ Data

by

Cloudflare’s AI Cache Leak: How 155 Responses Got Other Users’ Data

In brief

  • Cloudflare says a sizing mismatch and an SGLang bug let its GLM-5.3 model return other customers’ data in 155 responses, all to very long prompts.
  • The leak ran from August 27 to September 9, 2026, and only affected prompts longer than 681,280 tokens — the size of the prefill KV cache pool, against an advertised context window of 1,048,576 tokens.
  • The responses looked normal. They arrived with HTTP 200, coherent text, and a token count matching the long prompt — which is why Cloudflare’s own pre-launch tests, which checked status codes and performance but not output quality, missed the problem.
  • Cloudflare says it found no evidence of deliberate exploitation, and no other Workers AI model was affected. The model was taken offline within 40 minutes of the incident being declared on September 9, and restored the same day with a larger pool and a patched engine.
  • The wider lesson is for anyone running hosted inference: in disaggregated serving, the KV cache pool is a tenant-isolation boundary, and nothing in the default stack forces it to match the advertised context length.

Last verified: October 29, 2026.

What leaked, and to whom

On October 26, 2026, Cloudflare published an incident report that is unusually candid about a failure most providers would prefer to describe in vaguer terms. Between August 27 and September 9, 2026, a small number of customers using its GLM-5.3 model on Workers AI received responses that contained data from other customers’ requests to the same model, made shortly before theirs. In total, 155 responses were affected.

GLM-5.3 is a text-generation model from Z.ai, built for agentic coding workflows, and served on Workers AI as @cf/zai-org/glm-5.3. The exposure began the day the model launched on the platform. It was reported on September 8 by security researcher Gadi Evron of Knostic, through Cloudflare’s HackerOne program. Cloudflare reproduced the issue the next morning, declared an incident, and took the model offline within 40 minutes. It was restored to public access at 19:40 UTC on September 9, after a corrected configuration and a patched inference engine were deployed and validated against the researcher’s reproduction script.

Two limits on the damage are worth stating precisely, because they are easy to over- or under-read. First, every affected response went to a prompt longer than 681,280 tokens — an extremely long input, far beyond what most users ever send. Ordinary short prompts could not trigger this bug. Second, Cloudflare says that after excluding its own testing and the researcher’s activity, the affected requests matched the timing and volume of ordinary long-context use, and it found no evidence of deliberate exploitation. That is a statement about what its logs show, not a guarantee that no one noticed anything unusual. It also says no other model hosted on Workers AI was affected, because every other disaggregated deployment had KV pools larger than its context length on both prefill and decode.

The two faults that had to line up

This was not a single bug. It took two faults working together, and understanding both is the whole story.

The first fault was a configuration mismatch that Cloudflare knew about before launch. Workers AI serves GLM-5.3 with SGLang, an open-source inference engine, in a disaggregated setup: one pool of GPUs processes the prompt (prefill) and produces the model’s KV cache, then hands that cache over remote direct memory access to a second pool of GPUs that generate the response (decode). The KV cache is GPU memory organized as fixed-size pages, shared across requests. Pages are not zeroed between requests; the next request assigned a page simply overwrites it. That design is common and fast, but it makes the engine responsible for tracking exactly which pages belong to which request.

SGLang sizes its KV pool at startup from whatever GPU memory remains after loading the model weights. The advertised context length is a separate configuration value, and nothing forces the two to agree. On Cloudflare’s GLM-5.3 prefill nodes, the pool came to 681,280 tokens while the model’s context window was 1,048,576 tokens. Cloudflare says it expected the engine to return an error for any prompt it could not process, and its long-context tests returned successful responses, so it did not believe the mismatch was an issue. That assumption was the critical testing gap: the tests validated HTTP status codes and engine performance at long contexts, without validating model output and coherence. A reply that returns 200 and streams text passes a status-code test. It can still contain someone else’s data.

The second fault was in how SGLang handled the over-length requests that resulted. When a prompt arrived that was longer than the KV pool could hold, the scheduler rejected it, recorded the error to be returned to the client, and swapped the prompt for a one-token placeholder so the request could drain through the normal batch machinery without allocating real memory. On a single-node deployment, that worked: the placeholder ran, the batch finished, and the client received an error.

In disaggregated mode, it did not. The admission queues on both prefill and decode checked whether a request was finished, not whether it was rejected — and a rejected request stored its rejection in a way that still reported it as live. So prefill admitted it, ran a trivial forward pass on the one-token placeholder, and sent one page of KV to decode with a “transfer complete” signal. Decode had reserved pages for the full prompt length — eight pages in Cloudflare’s illustrative diagram — and assumed prefill had overwritten all of them. Only the first page had actually been written by this request. The rest still held what earlier requests left in them: other users’ prompts and partial generations. Decode then generated a response while attending over those stale pages, and the client received HTTP 200 instead of HTTP 400, with a token count matching the long prompt and a reply that read as coherent. It even arrived faster than a genuine long-prompt prefill should have taken — a clue visible only in hindsight.

Cloudflare notes that this bug had been reported to SGLang by another user as issue #30233 on July 6, 2026, and that it had been flagging this class of issue with SGLang since version 0.5.10. The incident accelerated work already under way with the SGLang team.

How it was fixed, and what remains

Cloudflare fixed both faults before restoring the model, and the order matters. First, it fixed its own configuration to match its GLM-5.2 setup, freeing enough GPU memory that the prefill KV pool now holds about 2.04 million tokens — roughly twice the 1,048,576-token context length. A prompt that fits the context window now always fits the pool, so the rejection path is never reached for a valid request. Cloudflare is explicit that this alone prevents the bug from being exercised, but that relying on pool size alone is fragile: any change to quantization, tensor parallelism, CUDA graph sizes, or a future model version can change the pool size in a way that is easy to miss.

So it also fixed the engine. The patched engine retires a request carrying a pending rejection at the disaggregation admission queue, with its original error, before any bootstrap, allocation, or KV transfer happens. The decode node now verifies that the number of tokens transferred matches the number it reserved. The upstream SGLang fixes shipped in versions 0.5.20 and 0.5.21, with the complete fix in v0.5.22 on October 19, 2026. Cloudflare recommends all operators using SGLang update to v0.5.22 or later, and says it has applied the engine fix and further hardening to every model it serves with SGLang, with deployment checks that require both fixes for every new deployment.

The timeline also shows a response that was fast once the report landed: report received September 8 at 23:11 UTC, verification beginning September 9 at 08:00, incident declared 13:22, model set to private at 14:00, root cause identified at 14:35, all prefill nodes rolled to the corrected configuration by 14:52, full deployment rebuilt with the patch by 19:04, model restored at 19:40. The seven-week gap between the fix and the October 26 coordinated publication, alongside SGLang’s post, reflects the disclosure process with the researcher and the upstream maintainers — not, on the published account, a delay in remediation.

What this means if you use hosted AI

For most readers, the direct risk from this incident is narrow: it required prompts over 681,280 tokens to one specific model on one platform, it is fixed, and Cloudflare says no other Workers AI model was affected. If you sent very long prompts to GLM-5.3 on Workers AI between August 27 and September 9, 2026, Cloudflare’s disclosure is the prompt to check whether you were notified as an affected customer — the company says a small number of customers were involved but has not given an exact count.

The broader lesson travels further, and it connects to the other AI-infrastructure stories of this month. AI agents are increasingly wired to tools that act on real systems — CrowdStrike’s October report describes agents running real intrusions with limited human involvement — and the infrastructure underneath those agents is shared, fast, and optimized for throughput. In that world, a memory pool that is not zeroed between requests, a context length that is configured separately from the pool that must hold it, and a test suite that checks status codes instead of output are not obscure engineering details. They are the isolation boundary, and this incident shows what happens when two small gaps in it line up: a reply that looks perfect and contains someone else’s data.

Three questions are worth asking any hosted-inference provider after this disclosure. First, is your KV pool provably larger than your advertised context window, checked against the running engine rather than inferred from configuration? Cloudflare now checks exactly that before any disaggregated deployment change ships. Second, do your long-context tests inspect response content near the maximum input size, not just the status code and the stream? Third, what SGLang version are you running, and does it include the v0.5.22 rejection-path fixes? A provider that can answer all three concretely has learned this incident’s lesson. One that answers with assurances about “enterprise-grade isolation” has not.

What to watch next

Two things deserve follow-up. The first is whether any affected customer speaks publicly about what their responses contained, which would turn a count of 155 responses into a clearer picture of the data types involved. Cloudflare’s report does not characterize the leaked content beyond “data from other customers’ requests,” and the prompts in question were long agentic-coding inputs, so the contents could range from code to credentials pasted into a prompt. The second is whether other SGLang operators audit their own deployments and report what they find. The upstream fix is public, and Cloudflare’s recommendation to update to v0.5.22 is explicit — silence from other operators is not evidence of safety, only of not having looked.

This incident will not be the last of its kind. As context windows grow and disaggregated serving spreads, the gap between what a model advertises and what its memory pool can actually hold will keep reappearing in new configurations. The providers that catch it will be the ones that test what their models actually output at the limits, not just whether they answer.

Sources

Related reading

Shoaib Hassan

Shoaib Hassan is the founder and editor of AuTechWire, an independent technology publication covering broadband, mobile networks, privacy, startups, and consumer tech.

Related Stories

Leave a Comment