DeepSeek-V4.1-Flash went up on Hugging Face at 02:17 UTC this morning under an MIT licence. The first sentence of the model card introduces “a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters.” Three paragraphs down, in a list of “additional architectural components,” sits a second figure: “Engram conditional memory (196B parameters, sparsely accessed via token-based lookup).”
So the argument that started within the hour, 552B or 748B, has an easy answer. It is both. The card discloses both in plain type, so nobody is hiding anything. The harder problem is that neither number is the one you need, and that the same card carries two more that are also correct.
The same architecture paragraph says the model “activate[s] only 8B parameters per token during prefill and 16B during decode.” And the checkpoint itself, whose per-dtype tensor counts Hugging Face publishes as metadata on the repo, holds 763,205,315,794 parameters across 48 sharded safetensors files, about 14.6 billion more than the 552B and 196.6B it does disclose. Neither the card nor the tech report names that total or accounts for the remainder; the report repeats the same pair, “552B backbone parameters and 196B Engram parameters.”
Five numbers, one artifact, spanning a factor of about ninety-five. Each one is the correct answer to a different question. The questions have drifted far enough apart that a single column labelled “parameters” now answers none of them.
8B and 16B say what one token costs. They are the weights actually multiplied during a forward pass. The config sets 384 routed experts per MoE layer with six active per token, 1.56% of the routed pool, across 40 layers arranged as a 20-layer causal encoder feeding a 20-layer decoder. Compute per token, and therefore serving cost, tracks this number and nothing else.
The transformer itself is the 552B. That is the part which scales the way the models people have intuitions about scale, and it is the figure every headline used today.
196.6B says how much the model memorised. I recomputed it from config.json rather than
trusting the round number: two Engram layers, at positions 1 and 14, with 384,006,168 and
384,016,682 embeddings and a head dimension of 256. That comes to 196,613,849,600
parameters, which the card and the tech report both give as 196B. Just over a quarter of
everything in the checkpoint is a lookup table.
What has to be on the machine is 763.2B parameters and 510.3 GB, the only pair here that survives contact with a purchase order. The released weights occupy 510,296,708,312 bytes, an average of 5.35 bits per parameter, because the checkpoint ships mixed-precision. No comparison table prints that figure, and it decides how many accelerators you buy.
The gap between the headline and the memory disclosure is where the whole problem lives. Engram weights are parameters in the only sense that matters to a disk and in almost no sense that matters to a FLOP counter.
Engram is not a DeepSeek-V4.1 improvisation. It was described in January in arXiv:2601.07372, “Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models” (Cheng, Tian, Zeng, Dai et al., 12 January 2026). Its framing states the problem: “While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation.”
The experimental design concedes the accounting problem outright. Having scaled Engram to 27B parameters, the authors report that it beats “a strictly iso-parameter and iso-FLOPs MoE baseline.” They had to hold both constant, because neither alone is a fair control any more. Match parameters and the memory model wins on compute; match compute and it wins on parameters. The paper also reports a U-shaped scaling law governing how to split a budget between neural computation and static memory, which makes the ratio of stored to active parameters a design knob rather than a property of the architecture family.
No published reporting standard for sparse models extends model cards or datasheets to cover the total, active and backbone distinction. What exists is a convention, visible in the survey literature since 2025 (Cai et al., IEEE TKDE, doi:10.1109/tkde.2025.3554028), of printing total and active side by side. That convention was built for MoE, where the inactive weights are experts. It has no column for a lookup table.
The divergence is not a quirk of this one release. It shows up as soon as you put V4.1-Flash beside the model it replaces.
DeepSeek’s previous Flash model carried a 284B backbone and activated 13B per token. The new one nearly doubles the backbone and cuts activation to 8B in prefill. A reader tracking “parameters” saw the model get 94% bigger. A finance team tracking cost per token saw it get 38% cheaper. Both were reading the correct number for their question, in the way that a depreciation schedule and an asset’s actual life are two correct answers about the same servers.
The release’s most consequential claim is not a parameter count at all, and it got almost no pickup. The card states that Compressed Sparse Attention 2 plus “FP4 main KV caching (E2M1 format, one E4M3 scale per 16 channels)” reduces the global KV cache to 890 bytes per token, “roughly 1/4 of DeepSeek-V4-Flash,” with a figure caption claiming approximately 437-fold reduction against DeepSeek-V1. The other half of that story sits one paragraph earlier on the same card, where SWA Bounded Replay cuts the persistent footprint to roughly 1/8 of V4-Flash’s by avoiding SSD writes. The tech report states both in one place and supplies the labels the card never uses: the 890 bytes is the cache that is “always in HBM”, the 1/8 figure the one “always on SSD or in host memory”. All of it is DeepSeek measuring DeepSeek, unaudited. If it holds, it changes the economics of a million-token context far more than any parameter figure does.
I read five 2026 model cards directly rather than through coverage.
Kimi K3 is the one doing this properly. Its spec table lists total parameters (2.8T), activated parameters (104B), and then, on its own row, “Parameters of Vision Encoder: 401M.” That third row is the habit the field needs, generalised. Qwen3.5-397B-A17B gives total and active and stops, which is the 2025 convention working as designed on the architecture it was designed for.
Then there are the two cards that say nothing. GLM-5.2’s, as published, states no active-parameter figure (the string “activ” does not appear in the README at all) and no total in the prose either; the 753B you see on the repo is computed by Hugging Face’s file inspector from the weights. MiniMax-M2.7’s is the same story, and I had it wrong until I grepped the file: no parameter count of any kind appears in that README, and the ~229B on its repo page is again the file inspector talking. Two labs shipped frontier open-weight MoEs and disclosed their size only by accident of tooling.
None of this is new, which is the consoling part. In July 2019, Lample, Sablayrolles, Ranzato, Denoyer and Jégou published Large Memory Layers with Product Keys, promising to increase capacity “by up to a billion parameters with a negligible computational overhead,” and demonstrated a 12-layer memory-augmented model beating a 24-layer baseline while running twice as fast. Eighteen months later the Switch Transformer paper (arXiv:2101.03961, January 2021) put the contradiction in its own abstract, “a sparsely-activated model — with outrageous numbers of parameters — but a constant computational cost,” and then let the press quote its 1.6 trillion alongside GPT-3’s 175 billion as if the two numbers measured the same thing. The disclosure was there in 2021. The single column survived it.
What has changed is the scale of the divergence. In 2021 it was a footnote about routing. Today one released artifact spans 8B to 763B, a quarter of the weights are a hash table, and the lab’s own researchers had to run iso-parameter and iso-FLOPs baselines to say anything meaningful about it.
The remedy is four fields on every card: stored parameters, active parameters per token, a component breakdown for anything that isn’t the backbone, and the on-disk byte count of the released weights. That it is this easy is what makes the omission irritating. Kimi K3 already ships three of the four in its spec table. DeepSeek states three of the four in prose today, spread across three separate paragraphs, and leaves the fourth to whoever is willing to sum a metadata blob.
Until then, a useful reflex when a parameter count crosses your desk: ask which of the four it is. Every headline I read about this release used the backbone figure, which is the one that answers none of them cleanly. I would rather have the 510 gigabytes. That one you can weigh.
References