The EmbeddingGemma 2 Model Card Has One Baseline. The Launch Charts Have Twenty-One More.

On 6 October Google DeepMind released EmbeddingGemma 2, a 740-million-parameter model that maps text, images, video and audio into one 768-dimensional vector space. The launch post, by Sahil Dua and Henrique Schechter Vera, calls it “best-in-class for its size.” I wanted to see what that sentence rests on, so I read the post, the model card on ai.google.dev (last updated 6 October, per its footer) and the Hugging Face card, and then opened the three chart images in the post, which carry most of the comparison.

In short, the claim is a size-versus-score claim, it is drawn in charts rather than tables, and the numbers behind the charts are not in the card as of 14 October 2026. The numbers that are in the card are mostly about the previous model, and they are less exciting than the headline.

What the card compares it with

The card’s main results table has two columns of scores: EmbeddingGemma 2 and EmbeddingGemma 1. As of 14 October 2026, that is the whole comparison set. For the text rows, plus English from the truncation table, here is what the two columns say.

EmbeddingGemma 1 versus EmbeddingGemma 2 on three MTEB subsets A dumbbell chart. MTEB multilingual v2 mean over tasks: 61.15 for EmbeddingGemma 1, 61.36 for EmbeddingGemma 2, a change of plus 0.21. MTEB English v2: 69.67 for version 1, 68.46 for version 2, a change of minus 1.21. MTEB code v1: 68.76 for version 1, 78.68 for version 2, a change of plus 9.92. The version 2 values and the version 1 code and multilingual values are from the version 2 model card; the version 1 English value is from the version 1 model card. The code gain is the only large movement. MTEB MEAN(TASK), 768D, FULL PRECISION · GOOGLE MODEL CARDS, 2025-2026 60 65 70 75 80 MTEB multilingual v2 61.15 — EmbeddingGemma 1 model card 61.36 — EmbeddingGemma 2 model card, 2026-10-06 +0.21 MTEB English v2 69.67 — EmbeddingGemma 1 model card 68.46 — EmbeddingGemma 2 model card, 2026-10-06 −1.21 MTEB code v1 68.76 — EmbeddingGemma 1 model card 78.68 — EmbeddingGemma 2 model card, 2026-10-06 +9.92 EmbeddingGemma 1EmbeddingGemma 2Right column: change, points Both versions evaluated by Google; the card does not name who ran the runs. English v1 figure from the v1 card.
The one large movement is code. On multilingual text the two models are 0.21 points apart, and on English the new model is 1.21 lower.

On code retrieval, the gain is real: 68.76 to 78.68 on the MTEB code subset, 9.92 points, or 14.4 percent in relative terms, which is where the card’s “~14% improvement on code tasks” comes from. On the multilingual subset the two models are 0.21 points apart, which I would not call a difference. And on the English subset, which the new card lists only for the new model (68.46), the first model’s card gives 69.67, so the new model is 1.21 points lower. The launch post says only that it “matches” the old one on multilingual text. It does not mention English.

I would not read much into 1.21 points; I have no run-to-run spread to compare it with. But a model that is flat or slightly down on two of three text subsets and up on code is a different story from “most capable,” and the card does not tell the first story either.

Every number in the table carries three conditions. The scores are for the 768-dimension output, they use “the full-precision checkpoint” (the card’s phrase), and they are mostly reported as a mean over tasks (the image benchmark is a mean over task types). The card says the model “was evaluated” on these benchmarks, in the passive voice. It does not say who ran them, on what hardware, or with which version of the evaluation harness. One row, the audio benchmark MAEB, carries a stray “Hugging Face” label on both cards, and I cannot tell what it is meant to signal. These are all Google’s numbers, as vendor claims on a vendor’s card.

Where the comparison lives

The competitors are in the launch post, as three charts: MTEB Code, the lite image benchmark MIEB, and the audio benchmark MAEB. Each plots score against model size on a log axis, with a dotted line joining the points that no smaller model beats. No values are printed, so what follows is my reading of the images, hedged accordingly. By my count the charts name 21 other models, among them Qwen3-Embedding at 0.6B and 8B, granite-embedding-311m, EmbeddingGemma 1 and several omni-modal models from Jina, BidirLM and LCO.

Read as a frontier chart, the claim holds up better than “best under 1B” would suggest, and also worse. On the code chart the new model sits above every model of similar or larger size except two at 4B and 8B (pplx-embed-v1-4b and Qwen3-Embedding-8B), and above Qwen3-Embedding-0.6B, which is bigger (the new model’s dot reads a little below the card’s 78.68, so treat the chart as approximate). On the audio chart it is not the top-scoring small model: jina-embeddings-v5-omni-nano scores higher, and its point sits just left of the 1B tick, close enough that I would not swear to which side of the line it falls on. The new model is on the frontier because it is smaller than the models that beat it. On the image chart a 3B model, LCO-Embedding-Omni-3B, is slightly ahead.

So “leading scores among sub-1B multimodal embedders for its size” is best read as “on the size-versus-score frontier,” which is a narrower and more defensible claim than “highest score under a billion.” The post’s other phrases, “matching or outperforming many larger models” and “the most capable model for on-device multimodal embeddings,” name no models and no benchmark in the sentence itself.

Size is elastic here. The card says 740M parameters in total, built from a 270M text model with optional 170M vision and 300M audio encoders, so the effective size is 270M, 440M, 570M or 740M depending on what you load. By my reading, the code chart plots the model at roughly 0.3B and the audio chart nearer 0.7B, which is legitimate, since text-only work needs only the text model. It also means “under 1B” is true in every configuration.

The 6x that costs different amounts

The storage claim is “up to 6x.” The Matryoshka truncation table shows what 6x costs, because it is the 128-dimension row: 768 divided by 128. The card says quality holds “close to lossless down to 256 dimensions” and that 128 “degrades multimodal quality substantially.”

Share of full-size score kept when EmbeddingGemma 2 vectors are truncated Four small panels, one per benchmark, each showing the share of the 768-dimension score retained at 768, 512, 256 and 128 dimensions (full scores 61.36, 78.68, 59.01 and 69.54). MTEB multilingual: 99.7 percent at 512 dimensions, 98.5 percent at 256 dimensions, 94.3 percent at 128 dimensions; MTEB code: 98.2 percent at 512 dimensions, 96.8 percent at 256 dimensions, 90.8 percent at 128 dimensions; MMEB v2 overall: 98.9 percent at 512 dimensions, 95.3 percent at 256 dimensions, 77.4 percent at 128 dimensions; MSEB retrieval: 99.5 percent at 512 dimensions, 96.0 percent at 256 dimensions, 81.6 percent at 128 dimensions. At 128 dimensions, the six-fold storage saving, the multimodal and audio benchmarks lose more than a sixth of their score while the text benchmarks lose under a tenth. SCORE RETAINED VS 768D, BY OUTPUT DIMENSION · GOOGLE MODEL CARD, OCT 2026 MTEB multilingual 61.36 of 61.36 = 100.0% kept at 768d — EmbeddingGemma 2 model card 768 61.17 of 61.36 = 99.7% kept at 512d — EmbeddingGemma 2 model card 512 60.41 of 61.36 = 98.5% kept at 256d — EmbeddingGemma 2 model card 256 57.89 of 61.36 = 94.3% kept at 128d — EmbeddingGemma 2 model card 128 94.3% 98.5% MTEB code 78.68 of 78.68 = 100.0% kept at 768d — EmbeddingGemma 2 model card 768 77.24 of 78.68 = 98.2% kept at 512d — EmbeddingGemma 2 model card 512 76.18 of 78.68 = 96.8% kept at 256d — EmbeddingGemma 2 model card 256 71.41 of 78.68 = 90.8% kept at 128d — EmbeddingGemma 2 model card 128 90.8% 96.8% MMEB v2 overall 59.01 of 59.01 = 100.0% kept at 768d — EmbeddingGemma 2 model card 768 58.38 of 59.01 = 98.9% kept at 512d — EmbeddingGemma 2 model card 512 56.24 of 59.01 = 95.3% kept at 256d — EmbeddingGemma 2 model card 256 45.65 of 59.01 = 77.4% kept at 128d — EmbeddingGemma 2 model card 128 77.4% 95.3% MSEB retrieval 69.54 of 69.54 = 100.0% kept at 768d — EmbeddingGemma 2 model card 768 69.18 of 69.54 = 99.5% kept at 512d — EmbeddingGemma 2 model card 512 66.76 of 69.54 = 96.0% kept at 256d — EmbeddingGemma 2 model card 256 56.71 of 69.54 = 81.6% kept at 128d — EmbeddingGemma 2 model card 128 81.6% 96.0% Each panel runs 75 to 100 percent of that benchmark's own 768d score. Percentages are my arithmetic on the card's table.
Truncating to 256 dimensions costs the multimodal and audio scores about 4 to 5 percent. The 6x saving, at 128 dimensions, costs them 18 to 23 percent.

At 256 dimensions, a 3x saving, the three text scores lose 1.0 to 3.2 percent and the multimodal score 4.7 percent. At 128 dimensions the card’s own table has the multimodal overall score falling from 59.01 to 45.65, 13.36 points, and audio retrieval from 69.54 to 56.71. The three text benchmarks fall 2.8 to 7.3 points. “Up to 6x storage reduction” is true, and “minimal impact on quality” is true to 512 and arguable at 256. For a vector index of a million documents in 32-bit floats, my arithmetic gives about 3.1 GB at 768 dimensions and 0.5 GB at 128, which is the 6x, with the card’s caveat attached.

One more condition. The card says all results use the full-precision checkpoint. The launch post’s on-device memory figures, about 191 MB for text-only and 567 MB for the full model on a Pixel 11 Pro, are described as “with quantization.” I found no quality numbers for a quantized EmbeddingGemma 2 in either document, as of 14 October. The first model’s card did publish them: its Q8_0 and Q4_0 rows land within about 0.5 points of full precision on multilingual MTEB (60.93 and 60.62 against 61.15). That is a precedent for the new model’s quantized score, not a measurement of it.

A budget that is the whole window

Back to the card’s input limits, which hide a small piece of arithmetic. A single shared 8,192-token window is spent at 1 token per text subword, 280 tokens per image, 140 per video frame and 25 per second of audio, and the card lists maximum inputs of about 29 images, 58 frames and 327 seconds.

How much of the 8,192-token window each stated maximum input uses Four bars against the 8,192-token window. Text at 1 token per subword: 8,192 tokens, none left. Images at the default 280 tokens each, 29 of them: 8,120 tokens, 72 left. Video frames at 140 tokens each, 58 of them: 8,120 tokens, 72 left. Audio at 25 tokens per second, 327 seconds: 8,175 tokens, 17 left. Each stated maximum is the whole window rounded down. TOKENS USED BY THE STATED MAXIMUM INPUT · EMBEDDINGGEMMA 2 MODEL CARD, OCT 2026 Text8,192 subwords 8,192 of 8,192 tokens — EmbeddingGemma 2 model card, 2026-10-06 8,192 used, 0 left Images29 × 280 tokens 8,120 of 8,192 tokens — EmbeddingGemma 2 model card, 2026-10-06 8,120 used, 72 left Video frames58 × 140 tokens 8,120 of 8,192 tokens — EmbeddingGemma 2 model card, 2026-10-06 8,120 used, 72 left Audio327 s × 25 tokens 8,175 of 8,192 tokens — EmbeddingGemma 2 model card, 2026-10-06 8,175 used, 17 left Outline is the 8,192-token window. Defaults only; a lower vision budget (70 to 1,120 tokens per image) changes the counts.
The card's maximum inputs are the 8,192-token window divided by the per-item cost, rounded down. Nothing is left over for a caption.

Each of those is the window divided by the cost and rounded down, so “up to 29 images” means 8,120 tokens of images and nothing else. Any caption, prefix or second modality takes from the same pot, which the card notes. The launch post’s “5.5 minutes of audio” is the 327-second figure. The card also warns that the model should not be run in float16, because its activations exceed that format’s range and it “returns NaN or silently degraded embeddings” rather than raising an error. Silent failure is common in numerical code. A card that says so in one sentence is less common.

The license: Apache 2.0, plus a policy

The first EmbeddingGemma was under the Gemma Terms of Use, and its Hugging Face repository was gated with manual approval. I read both on 14 October: the Hub metadata lists the license as “gemma” with manual gating. The second model’s Hub metadata lists “apache-2.0,” and the repository has no gate. The launch post says it is “released under a commercially permissive Apache 2.0 license,” and the card’s license link goes to a page that reproduces the Apache 2.0 text, last updated 1 April 2026, at a URL the card labels as the Gemma 4 license.

That change matters. The Gemma terms, as of their 1 April 2026 revision, make the Prohibited Use Policy part of the agreement, require anyone distributing the model or derivatives to include its restrictions as “an enforceable provision,” and let Google restrict use “remotely or otherwise” when it reasonably believes the terms are being violated. Apache 2.0 has none of that. It is what I wrote about in Open Is a Promise Until the File Ships: the file is the promise, and here the file changed.

The wrinkle is a sentence in the card’s safety section: “Deployments must adhere to the Gemma Prohibited Use Policy.” The policy’s last modification date is 21 February 2024, and it says Google reserves the right to update it. Apache 2.0 does not incorporate a use policy, and the card does not say that this sentence is a condition of the license rather than a request. I found no text, as of 14 October 2026, that says which one governs. It may be a leftover from the template of earlier cards; the same card refers to “Vision-Language Model (VLM) technology” in its ethics section, which reads like inherited wording for an embedding model. But “may be a leftover” is my guess, and a deployer who needs to be sure has to ask Google or a lawyer, not the card. The practical position: the license file is Apache 2.0, and the card contains one sentence that points the other way.

What I would want from the card

A table with the competitors in it, since the charts evidently have the numbers. The evaluator named, and the harness version. The quantized checkpoint’s scores next to the figures the post uses for memory. And the license sentence reconciled with the license. On the numbers that are published, I read EmbeddingGemma 2 as a better code-retrieval model than its predecessor, level on multilingual and English text, and a credible small multimodal model on a size-versus-score frontier. That is a good release. It is a smaller claim than “best-in-class,” and the card’s own tables never make the larger one.

For the sub-billion-parameter models that make up most downloads, see the earlier post on model size and download counts.

References

  1. Dua, S. and Schechter Vera, H., Google DeepMind (2026). EmbeddingGemma 2: an open, lightweight multimodal embedding model. Google blog, 6 October 2026.
  2. Google DeepMind (2026). EmbeddingGemma 2 model card. Google AI for Developers, last updated 6 October 2026.
  3. Google DeepMind (2026). google/embeddinggemma-2. Hugging Face model card; accessed 14 October 2026.
  4. Google DeepMind (2025). EmbeddingGemma model card. Google AI for Developers; accessed 14 October 2026.
  5. Google DeepMind (2025). google/embeddinggemma-300m. Hugging Face; gated repository, license tag “gemma”.
  6. Google (2026). Gemma Terms of Use. Last modified 1 April 2026.
  7. Google (2026). Gemma 4 license (Apache License 2.0). Last updated 1 April 2026.
  8. Google (2024). Gemma Prohibited Use Policy. Last modified 21 February 2024.
  9. Schechter Vera, H., Dua, S., et al. (2025). EmbeddingGemma: Powerful and Lightweight Text Representations. arXiv:2509.20354, 24 September 2025 (v3, 1 November 2025).