Cloudflare and the SGLang team published a coordinated disclosure today. Cloudflare’s post, by Kevin Flansburg and Michelle Chen, is titled “Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang,” and its second paragraph gives the scope: between August 27 and September 9, “a small number of customers using the GLM 5.3 Workers AI model received responses that contained data from other customers’ requests made to the same model shortly before theirs. In total, 155 responses were affected, all to prompts longer than 681,280 tokens.” The SGLang team’s companion post, “SGLang security update: improving request isolation,” covers the engine side.
Whether 155 is a small number matters less to me than where the checks were. Cloudflare says it found no evidence of deliberate exploitation and that the affected requests look like ordinary long-context use, and I have no basis to argue with that. Every layer of this stack had a way to say no, and each one said no in a form the next layer couldn’t hear. The one test that might have caught it was looking at the HTTP status.
Two numbers that were never compared
Cloudflare deployed GLM-5.3 with a context window of 1,048,576 tokens. On its prefill GPUs, the KV cache pool (the GPU memory that holds the model’s attention state for prompts in flight) had room for 681,280. The post says it plainly: “We were aware of this mismatch before launch.” The team expected the engine to return an error for any prompt it couldn’t fit, and its long-context tests came back successful, so nobody treated the gap as a problem.
The two numbers come from different places, which is how this happens. SGLang sizes the pool at startup from whatever GPU memory is left after the weights load. The context length is a separate configuration value. In Cloudflare’s words: “Nothing forces the two to agree, and it is our job to check that they do.” A change to quantization, tensor parallelism or CUDA graph sizes moves the pool. Nothing moves the context length unless someone edits it.
A mismatch like that is not, by itself, a security bug. A careful engine would turn it into a capacity limit: prompts past 681,280 tokens would fail with an error, a customer would complain that the advertised million-token window was really two-thirds of a million, and someone would fix the config. That is what SGLang does on a single node. The leak came from what the engine did with the rejection when prefill and decode run on separate machines.
A rejection that looked like a live request
Workers AI runs GLM-5.3 in disaggregated mode. One pool of GPUs reads the prompt and builds its KV cache (prefill), then ships that cache over RDMA, using a transfer engine called Mooncake, to a second pool that generates the reply (decode). The cache lives in fixed-size pages. Cloudflare describes those pages this way: “Pages are not currently zeroed between requests; the next request that gets a page simply overwrites it.” That is a deliberate choice for speed, and the post says it is common practice. Paged KV memory came into wide use with vLLM’s PagedAttention, which the SOSP 2023 paper describes as “inspired by the classical virtual memory and paging techniques in operating systems.” One habit of operating systems didn’t make the trip: handing a process a page that still holds another process’s data is something an OS treats as a security bug in itself. Here, whether a stale page could be read depended on bookkeeping.
The bookkeeping failed like this. When a prompt was too long for the pool, SGLang’s scheduler rejected it through a function called set_finish_with_abort. That function recorded the error for the client and swapped the prompt for a one-token placeholder, so the request could drain through the normal batch machinery without allocating memory. The rejection went into a field called to_finish. The admission queues on both prefill and decode, though, checked finished(), which reads a different field. A rejected request reported itself as live.
The router had told decode the prompt length, so decode reserved pages for the whole prompt. Prefill ran a forward pass on the one-token placeholder, sent one page and reported the transfer complete. Decode had the same finished() check, accepted the handoff, and generated a reply while attending over every page it had reserved. Only the first had been written by this request. The others still held what earlier requests had left in them.
Cloudflare’s summary of what the client saw: “The client got HTTP 200 instead of HTTP 400. prompt_tokens matched the long prompt and the reply was coherent.” It also arrived faster than an eight-page prefill should take, which is the only external sign I can find in the description.
The test that passed
Cloudflare’s own explanation of why the known mismatch shipped: “we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence.”
For most software that is a reasonable test. A 200 means the server accepted the request, and you check the body separately when the body is the point. In model serving the body is always the point, and a 200 tells you nothing about which tokens the model attended to. The response object had every field a monitor would look at. The status was right. prompt_tokens matched what the client sent, even though prefill had processed one token. The text read naturally. As far as I can see from the post, the anomaly was latency.
I’ve written before, in the PHP disclosure piece, about how much of a security incident is timing. This one is mostly about the shape of an error. The engine did decide the request was invalid. That decision was stored where only the single-node path read it, and everything downstream saw a request that had finished normally.
The same bug, filed as garbage
Cloudflare says the bug “had been reported to SGLang by another user as issue #30233 on July 6, 2026,” which it found while investigating. I opened the issue. The mechanism is the same one: in disaggregated mode an over-length request goes through set_finish_with_abort, its input is cut to one token, only one token of KV is sent, and decode “starts from 284936 uninitialized KV cache slots.” The reporter’s description of the outcome was “producing garbage output.” Their title calls it garbage too.
That word explains most of the incident to me. On a lightly loaded node, or one where the stale pages hold nothing useful, this bug produces nonsense, and nonsense gets filed as a correctness bug. On a busy multi-tenant service where the pool keeps cycling through other customers’ long prompts, the same stale pages hold readable text, and the model, being a good language model, turns them into a fluent answer. The code path was identical in July and August. The contents of memory were what changed.
The timeline has an awkward middle. PR #36988, the change Cloudflare’s table of upstream fixes credits with stopping this exposure, merged on September 6, two days before the researcher, Gadi Evron of Knostic, reported the leak through Cloudflare’s HackerOne program. It was titled “[VLM] Retire aborted disaggregated prefill results,” a fix for a vision-model path, and it wasn’t in a release. v0.5.19 had shipped the day before. The fix reached users in v0.5.20 on September 18, nine days after Cloudflare restored the model with its own patch. Cloudflare’s response time was fast: incident declared at 13:22 UTC on September 9, model private at 14:00, restored at 19:40. Most of the calendar on this incident sits before the report.
What the literature was looking at
I went looking for academic work on cross-user leakage through shared KV cache. Most of what exists is about timing. “The Early Bird Catches the Leak,” by Linke Song and colleagues in IEEE Transactions on Information Forensics and Security, describes timing side channels “arising from shared caches and GPU memory allocations” that let an attacker infer other users’ prompts by measuring whether a request hits the cache. That threat model assumes isolation holds and asks what leaks around it, with prompts recovered through a token-by-token search.
The Cloudflare incident needed no probing, no classifier and no attacker. An ordinary customer sending a long prompt got other customers’ text back in the response body. The academic framing treats shared KV state as a channel that leaks through side effects. This incident shows the engine’s own page accounting can be the direct path, and the SGLang post says the related race bugs behave the same way: “Each occurrence was silent.”
Where I would hold back
Three things I can’t check. The count of 155 responses and the conclusion that none of it was deliberate exploitation come from Cloudflare’s logs, which nobody outside can audit; the post says the counts exclude its own testing and the researcher’s. The researcher’s own write-up is listed by SGLang as “available soon,” so the discovery side of the story is still one-sided. And the “about 2.04 million tokens” figure for the new pool is Cloudflare’s, stated once, with no log line shown.
I’d also be careful about reading this as a story about SGLang being unsafe. The engine had a real bug, but the condition that made it reachable at scale was a configuration Cloudflare chose knowingly, and Cloudflare says so. Every other disaggregated model on Workers AI had pools larger than its context window and couldn’t reach the rejection path.
What I take from it
The remediation list is long and mostly good: admission-time rejection, per-transfer checksums behind --disaggregation-enable-kv-checksum, deferred page release, a draft proposal to zero pages on release, and a plan to model the RDMA transfer protocol formally. The part I’d most like to see finished is the one Cloudflare already built and switched off. Over the summer it prototyped page tags that change when a page is reallocated and are checked before decode reads, and reported that “In our tests it cost under 1% in throughput and tail latency.” Cloudflare tested it on GLM-5.2 in mid-August and disabled it shortly afterwards “due to reliability issues,” not long before GLM-5.3 launched on August 27.
The first item on Cloudflare’s advice to other operators is the cheapest: compare the KV pool size in your startup log against your context length, on every node, after every change to weights, quantization, parallelism or memory fraction. The post calls it “the check we did not enforce.” I’d add one more. Long-context tests should assert on the content of the reply, ideally with a canary string planted in an earlier request, because a 200 with a fluent body was exactly what the failure looked like.
References
- Flansburg, K. & Chen, M. (2026). Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang. Cloudflare Blog, 26 October 2026.
- SGLang Team (2026). SGLang security update: improving request isolation. LMSYS Org blog, 26 October 2026.
- sgl-project/sglang (2026). Issue #30233: PD disaggregation: aborted prefill requests (input too long) cause decode to generate garbage from incomplete KV cache. Opened 6 July 2026, closed 5 September 2026.
- sgl-project/sglang (2026). PR #36988: [VLM] Retire aborted disaggregated prefill results. Merged 6 September 2026.
- sgl-project/sglang (2026). PR #38935: [PD] Do not admit intake-rejected requests to a PD handoff. Merged 16 September 2026.
- sgl-project/sglang (2026). Releases: v0.5.19 (5 September), v0.5.20 (18 September), v0.5.21 (2 October), v0.5.22 (19 October 2026).
- Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H. et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. Proceedings of the 29th Symposium on Operating Systems Principles (SOSP), 23 October 2023.
- Song, L., Pang, Z., Wang, W., Wang, Z., Wang, X., Chen, H. et al. (2025). The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems. IEEE Transactions on Information Forensics and Security, 2025.
- Gautam Parab (2026). Five Finders, One Fix, and a Honeypot Hit on the 17th. 22 October 2026.