Ninety-Six Percent of Half the Cases: The Arithmetic Under Sophos's Daybreak Number

On 9 October OpenAI published a customer story headlined “Sophos cuts threat investigation time by 96% with OpenAI Daybreak” (OpenAI, 2026). Sophos runs a managed detection and response service, which means a security operations centre staffed by other people watches your network for you, and the story is about agents built on OpenAI models doing part of that work. Three numbers sit in large type at the top of the page: 89 seconds, 96%, 52%.

I wanted to know where the 96% comes from, because a reduction is a ratio and a ratio has two ends. The page gives both. “Sophos’s existing process averaged around 38 minutes,” it says, and then, quoting Sophos’s chief technology officer John Peterson, “the average response time for cases using those agents has fallen to about 89 seconds.” Thirty-eight minutes is 2,280 seconds. 89 over 2,280 is 3.9%, so the reduction is 96.1%. That is the headline, and the arithmetic is right.

The trouble is what the two ends describe. The page calls the 38 minutes the “existing process” average and never says which cases it covers. The natural reading is an old average over everything Sophos handled. The results list, though, words it as reducing “the average response time for cases using agents from approximately 38 minutes to 89 seconds,” so the page is ambiguous about its own denominator. The 89 seconds is the new average for “cases using AI agents,” which the page suggests is about half of them. Peterson again: “About half of the cases we handle are now being automated by agents.” The results list is a little more precise, saying the agents resolve “52% of MDR cases end-to-end,” though it never says the 89 seconds covers exactly those cases. So, on the natural reading, the 96% compares a whole population before with a selected half of it after. On either reading it is a statement about the agent-handled cases. How long a Sophos case takes now is a different question, and the 96% doesn’t answer it.

What the whole caseload looks like

The page does not report the new average across all cases, so here is the version you can build from its own numbers. Give 52% of cases 89 seconds. Give the other 48% the old 38 minutes, on the assumption that nothing about the human-handled cases changed. This also treats the cases behind the 89-second average and the 52% closed end-to-end as the same cases, which the page implies but does not say. The blended average is 0.52 × 89 + 0.48 × 2,280 = 1,141 seconds, or about 19 minutes. Against 38 minutes, that is a reduction of 50%.

The same 38-minute baseline, two different afters Slope chart in minutes. On the left, the old 38-minute average, which the page calls its existing process and does not scope; the natural reading is all cases. On the right, the agent-handled cases average 89 seconds, about 1.5 minutes, which is the 96% reduction in the headline. Also on the right, the average across all cases is about 19 minutes if 52% of cases take 89 seconds and the other 48% still take 38 minutes, a 50% reduction. The 19-minute line is the writer's own arithmetic, not a figure Sophos reports. AVERAGE TIME PER CASE · MINUTES · OPENAI / SOPHOS CASE STUDY, 9 OCT 2026 Before After 38 minutes, the existing-process average before agents (scope not stated). OpenAI case study, 9 Oct 2026 89 seconds, cases using agents. OpenAI case study, 9 Oct 2026 About 19 minutes, all cases, writer's arithmetic: 52% at 89 s, 48% at 38 min 38 min old process ≈19 min (−50%) all cases, if the rest still take 38 89 s (−96%) only the cases the agents took Dashed line: my arithmetic from the page's own numbers. The page does not report an all-case average after agents.
Read naturally, the headline compares the old average for every case with the new average for about half of them. Average the new figure over all cases and the cut halves.

Fifty percent is still a large number. If a security operations centre halved its average case time in a year, that would be worth a customer story on its own. The trouble is only that the headline number answers a narrower question than it appears to, and the page does not give the figure that answers the broad one.

The assumption doing the work

My 19 minutes rests on one guess: that the cases humans still handle take as long as the average case used to. That is probably generous. If the agents finish the cases they finish in a minute and a half, they are most likely the routine ones, and the page does not say which kinds of case go to agents and which go to people. Automation that takes the easy half leaves a harder half, and a harder half takes longer than the old average did.

The blended figure is sensitive to that. If the human-handled cases now average 50 minutes, the all-case reduction falls to about 35%. At 60 minutes it is 22%. At about 78 minutes the gain is gone entirely, and the average case takes as long as it did before the agents arrived, even though half the cases now close in 89 seconds.

How the all-case reduction depends on the human-handled half Line chart. The horizontal axis is the average minutes per case for the 48% of cases still handled by people, from 35 to 80. The vertical axis is the reduction in average time across all cases, against the old 38 minutes, assuming the other 52% take 89 seconds. At 38 minutes the reduction is 50%. At 50 minutes it is about 35%. At 60 minutes it is about 22%. At about 78 minutes it reaches zero. These are the writer's calculations; the case study reports none of them. ALL-CASE REDUCTION VS TIME LEFT ON THE HUMAN HALF · WRITER'S ARITHMETIC 0% 25% 50% 40 50 60 70 80 Average minutes per case for the 48% still handled by people Human half at 38 min: all-case reduction 50.0% Human half at 50 min: all-case reduction 34.8% Human half at 60 min: all-case reduction 22.2% Human half at about 77.6 min: all-case reduction 0% 50% if the human half is unchanged 35% at 50 min 22% at 60 min 0% at about 78 min
The case study does not say how long the human-handled cases take now. That one missing number decides whether the whole operation got 50% faster or barely faster at all.

I am not claiming the human half got slower. I don’t know, and neither does anyone reading the page. That is the problem with a headline built on the selected half: the number that would settle it is the one left out.

Response time, investigation time, and when

Three smaller things on the page make the 96% harder to pin down.

The first is the noun. The headline says investigation time. The quote and the results list say response time. In security operations those are often different clocks. One measures how long until someone has worked out what happened, the other how long until something is done about it, and either can be measured from the alert firing or from the moment an analyst picks it up. The page defines neither. I’ve written before about three honest discount figures on one model that disagreed because each measured a different thing; an undefined “time” is the same hazard with one number instead of three.

The second is the date. The 52% and the 89 seconds are not new in October. Sophos’s own press release of 22 June 2026, more than three and a half months earlier and the day it announced it had joined OpenAI’s Daybreak Cyber Partner Program, already said its managed detection and response service was “resolving 52% of cases end to end with AI, with an average response time of 89 seconds” (Sophos, 2026). OpenAI’s own Daybreak announcement is dated the same day (OpenAI, 2026). The October story credits the 89 seconds to “agents we’ve been able to build through the Daybreak programme.” Both can be true if Sophos had Daybreak access before the announcement. The June release, though, describes an agentic security operations centre already running and its Daybreak adoption as “phased” and just beginning, so the figures may predate the Daybreak agents. Identical figures in June and October suggest the case study is not reporting a fresh measurement, and neither document says over what period the averages were taken. The June release also gives no 38-minute baseline. That figure, and with it the 96%, first appears, in the sources I found, in the October story.

The third is the second 96%. The page says the old 38 minutes was “performance Peterson says was better than 96% of professional security operations centres.” OpenAI, to its credit, attributes that to Peterson rather than stating it. I could find no published benchmark it comes from. It is a vendor’s view of its own baseline, and it should be read that way.

Speed is the only thing measured

The other missing number is accuracy. A case that closes in 89 seconds is a good outcome if it was closed correctly and a bad one if an attack was waved through. The page reports no miss rate, no reopen rate, and nothing on how often a human later disagreed with an agent’s call.

Reporting both is possible. A short conference paper from February describes an agent run for 30 days in a production security operations centre and audited with the same quality scorecard used for its human analysts (Chichurov et al., 2026). It gives speed the way the Sophos page does, 3.5 minutes to a verdict against 30 minutes for people, and says plainly that this covers only the 78% of tickets the agent closed itself; the other 22% went to humans as inconclusive. It also gives the number the Sophos page leaves out. On 500 randomly sampled closed incidents, the agent’s dispositions were correct 90.9% of the time, against 99.8% for the human control group.

Missing accuracy matters more in this field than most, because research published this month finds that dismissal gets too little scrutiny in triage agents. Tsegai and colleagues built a benchmark that replays an enterprise’s telemetry through a live security information system and asks agents to investigate 1,247 alerts from a multi-stage attack. Every one of the five representative approaches they tested missed at least 40.4% of the attack-related alerts. Attack alerts were more likely to be dismissed when a search came back empty, and dismissals got no more scrutiny than escalations (Tsegai et al., 2026). Their own design, which requires an independent challenge to every proposed decision and stronger evidence before dismissal, cut the false-negative rate to 3.1% while sending 18.4% of alerts to human analysts. That is a lab benchmark, not Sophos’s production system, and it says nothing about how Sophos’s agents perform. It does show that “resolved by AI” and “resolved correctly” are separate measurements, and a speed figure tells you nothing about the second.

Practitioners say much the same. In interviews with 20 people who work in security operations, published in August and covering language models in general rather than autonomous agents, the time savings they perceived were for low-stakes tasks whose output is quick to check, and they described limited trust in model output for high-stakes decisions (Rabieinejad et al., 2026). The Sophos page describes a design with that caution built in: three customer-chosen modes from notify-only to full authority, and Peterson saying “anything we don’t feel comfortable with an agent handling gets passed off for human judgement.” That is a sensible design. It is also another way of saying the agents get the cases Sophos is comfortable with, which is the selection effect again.

Reading the cases that weren’t counted

There is an old lesson about measuring from the cases that happen to be in front of you. In 1943 Abraham Wald, working for the Statistical Research Group at Columbia, wrote a set of memoranda on how to estimate the vulnerability of different parts of an aircraft when the only planes you can inspect are the ones that came back. The information about what brings a plane down sits with the aircraft that did not return. Marc Mangel and Francisco Samaniego set out his method in the Journal of the American Statistical Association in 1984 (Mangel and Samaniego, 1984). The Sophos page has the same blind spot. It shows the cases that were counted, and the cases that would settle how the whole operation changed are the ones it leaves out.

None of this makes the case study false. Every number on it can be correct. Ninety-six percent is a fair description of how much faster the agent-handled half closes than the old process averaged. A buyer comparing security vendors needs something else: the average time and the miss rate across all cases, before and after, with the same definition of time and a stated period. For Sophos that is three numbers it almost certainly has. Daybreak last came up here in a piece about Google’s gated Gemini 4 Argon release, which also touched OpenAI’s vetted cyber programs and the question of who decides who gets in. Partner case studies like this one raise a second question: what the partners count.

References

  1. OpenAI (2026). Sophos cuts threat investigation time by 96% with OpenAI Daybreak. Customer story, 9 October 2026.
  2. Sophos (2026). Sophos Joins the OpenAI Daybreak Cyber Partner Program to Strengthen Customer Defense with Frontier AI. Press release, 22 June 2026.
  3. OpenAI (2026). Daybreak: Tools for securing every organization in the world. 22 June 2026.
  4. Chichurov, V., Ibragimova, A. and Tararykov, A. (2026). The Adaptive AI SOC Agent – Moving Beyond Linear Playbooks with Cognitive Reasoning. 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC), pp. 1–3, 18 February 2026.
  5. Tsegai, S. A., Kantchelian, A., Yao, D. and Gao, P. (2026). From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage. arXiv:2610.10608, 7 October 2026.
  6. Rabieinejad, E., Dehghantanha, A., Zarrinkalam, F. and Dastgerdy, S. (2026). Beyond the Hype: Evaluating LLM Integration and Practical Limitations in Security Operation Centers. arXiv:2608.17154, 17 August 2026.
  7. Mangel, M. and Samaniego, F. J. (1984). Abraham Wald’s Work on Aircraft Survivability. Journal of the American Statistical Association 79(386): 259–267, June 1984.