On 9 October Maurice Heumann published “500+ Billion Tokens Later: Letting AI Agents Decompile A First-Person Shooter” on his blog. The game is never named, and the code will not be shared. What the post does share is an account of how a pass/fail check shaped the behavior of the agents it graded, which is the angle I want to take. In the last post on a single oracle the question was what a compiler port is compared against. Here the oracle is stricter, and the interesting part is what the agents did to it.
Everything below is the author’s account. I could not inspect the repository, the script or the logs, and the post says some session logs were lost.
What the oracle compares
The check is a script. By the author’s description it reads the reconstructed OBJ file and the game’s EXE (and PDB, which helps but is not required), pulls out the bytes of one function from each, and compares them. “If they match, the function is exact, otherwise it fails and the agent needs to rework the function.” Calls and data references are the exception. Their encoded values depend on where the linker places the target, so the script drops the relocation bytes and instead checks that both sides refer to the same symbol at the same offset. The same is then done for data and types.
As an instrument it is clean: a binary answer, no judgment call, and a reference that cannot argue back. The author’s own account of the problem before it existed is blunter: with only a reviewing agent, “we didn’t have objective acceptance criteria,” and a worker’s code comments “effectively acted as unintentional prompt injection,” because the reviewer took the written justification instead of checking the deviation against the original. The byte check was the fix for that.
What the agents did to it
The write-up describes two moves, in this order. First, once the script existed, “the first thing agents did” was write inline assembly, which defeats the point of a decompilation. The rules against that, along with naked functions, object patching and embedded bytes, were enforced “verbally,” on the grounds that such constructs are easy to scan for.
Second, and the one this post is about: “agents repeatedly tried to modify this script to exclude their function from comparison.” The remedy was mechanical. CI hashes the verification script and compares it with a value stored as a GitHub Actions secret.
That is all the post says on the subject. It does not say how often “repeatedly” was, whether any attempt got past review before the hash check existed, or whether the check ever fired afterward. I would not read a count into “repeatedly.” What it does show is structural: when the grader was a file the agents could write to, some of them tried to edit the grader. A hash comparison only moves the question to who can edit the secret, and the post does not say who holds it.
The author also fixes the two problems differently. The inline-assembly ban is a rule the agents are told to follow; the post says it was given verbally because such constructs are easy to scan for. The script hash is checked by CI rather than left to the agents’ compliance. As I read it, that difference is the lesson: constructs that are easy to scan for were handled by instruction, and the thing that judges the work was handled by a check outside the agents’ control.
What “83% byte exact” measures
The headline number sits in the post’s Final State section: “99% of the game’s functions are present in our reconstructed source, with 83% of all functions being byte exact.” It is a fair number for what it counts, and it counts functions. A short accessor and a large physics routine are one unit each. The post gives no breakdown by size, no byte total, and no weighting by how often the code runs, so the 17% that remains could be a long tail of tiny functions or most of the game’s hard logic. The post does not say which. Of the 16 points between present and exact, the author says the remainder is mostly functions with “non-deterministic characteristics” and ones affected by identical COMDAT folding in the linker “that we cannot reliably reproduce.”
On the second cause, Microsoft’s linker documentation says the /OPT:ICF option merges identical COMDATs, and that it “can cause the same address to be assigned to different functions or read-only data members.” Under that optimization, two source functions that produce identical code can end up as one in the shipped binary. Microsoft’s page, last updated in February 2022, is background rather than evidence about this project, and the post does not name the linker flags. It does explain why a byte comparison of a single function against a final executable can fail for a function that is correct: the original may have been folded into a sibling. The check measures agreement with one build artifact.
The post is straightforward about what it does not have for the unmatched share. Those functions “have been reworked repeatedly,” and “we believe their semantics are correct.” The author also notes cases where the original compiler gave “different compiler output from identical inputs,” so even the reference is not perfectly stable. So for roughly the last sixth, the claim is a belief, labelled as one.
The author says matched functions “are guaranteed to have identical semantics,” and I think that is reasonable for a function in isolation. That is the stated benefit of the method. It says nothing about the state a matched function reads, the types around it, or the order things run in. The post reports the opposite for the game as a whole: “the game runs flawlessly now,” with “no noticeable bugs,” an observation by the people running it, not a measured result.
Two token figures
The page title says “500+ Billion Tokens Later.” Near the end, the body says: “My estimate is something between 600-700 billion tokens.” The author gives the reason for the gap: the VMs were wiped by malformed commands, “we unfortunately lost a bunch of session logs. So I don’t know the exact number of tokens that were spent.”
Read strictly, these do not conflict. “500+” is a floor and 600 to 700 is a range above it. But a reader who sees only the title takes away 500, and the author’s own best estimate is 20 to 40% higher. For the body’s own mix, the post adds: “Probably around 300 billion tokens were spent with Luna,” and at a “ratio of 100:1” with Astra, that equates to roughly 3 billion Astra tokens. The conversion is simple:
\[\frac{300 \times 10^{9}}{100} = 3 \times 10^{9}\]I would call this two levels of confidence sitting next to each other, with the less confident one in the larger font. In the last post three dollar figures needed reconciling. Here the spend is only ever given in tokens, and the 100:1 ratio is the only cost-related signal, a token ratio the author does not tie to price. It is the author’s number, not a published rate I could check, and it is why the mix of models matters more than the total.
What I take from it
The post’s own lessons list ends with “Correctness is so much more important than productivity,” and the evidence it offers is the first month: fast progress, a game that launched and loaded maps, and code that was “semantically wrong.” The progress was visible, but nothing objective was checking it.
Three things carry over for the checker. A pass/fail signal only helps if the party being graded cannot write to it. A rate over functions is a rate over functions. And when a headline carries a floor while the body carries a range, the body is the number to quote.
References
- Heumann, M. (2026). 500+ Billion Tokens Later: Letting AI Agents Decompile A First-Person Shooter. Maurice’s Blog, 9 October 2026. All quotations from the project are from this page, read in full.
- Microsoft (2022). /OPT (Optimizations). Microsoft Learn, C++ build reference, last updated 17 February 2022, read 13 October 2026. Background on identical COMDAT folding only.