Gemma 4Open-Weights LLM

Every AI platform we architect eventually hits the same question: what do we run when the model cannot be an API call? Data that must stay in the building, apps that must work offline, per-request economics that a hosted frontier model cannot survive. For 2026, Gemma 4 is our default first answer to that question, and this brief is about why, what it actually costs to run, and where that answer stops being right.

What it actually is

Gemma 4 (Google, April 2, 2026) is Google's open-weights family built from the same research line as Gemini 3, and the first Gemma generation released under a plain Apache 2.0 license. Four models launched:

• E2B and E4B: edge models that activate an effective 2 billion and 4 billion parameter footprint, built for phones, browsers, and embedded hardware. 128K context window, and the only launch variants with native audio input.
• 26B A4B: a Mixture-of-Experts model, roughly 26B total parameters with only 3.8B active per token. Context up to 256K (262,144 tokens).
• 31B dense: the flagship. All parameters active, 256K context, and the size Google says fits on a single 80GB NVIDIA H100.

All four process text and images at variable resolutions, and Google states all models handle video input as well. The family is trained on over 140 languages, ships with configurable thinking modes, and has native support for function calling, structured JSON output, and system instructions per the model card.

The family has kept moving since launch, per the official release log: Multi-Token Prediction (MTP) variants of all four launch models landed on April 16, 2026, and a fifth model, Gemma 4 12B Unified, arrived June 3, 2026 with text, image, and audio input at 256K context, closing the audio gap in the mid-range.

The license is the headline

Previous Gemma generations shipped under Google's bespoke Gemma Terms of Use. Gemma 4 ships under Apache 2.0, and for client work that is not a footnote, it is the differentiator:

• No user-count gates, no attribution badges, no derivative naming rules. Compare the Llama 4 Community License, where "Built with Llama" attribution and the derivative naming clause are product-shaping obligations. Apache 2.0 has none of that: fine-tune it, white-label it, ship it under your own brand.
• An explicit patent grant. Standard Apache 2.0 machinery, and exactly the clause enterprise counsel asks about first.
• Legal review shrinks from a memo to a checkbox. Our healthcare and fintech clients run procurement processes where "bespoke AI license" triggers weeks of review. Apache 2.0 is already on every approved-license list on earth.

The practical effect: in evaluations where Gemma 4 and a conditionally licensed model are within noise of each other on task quality, the license decides it, before benchmarks are discussed.

Real deployment cost

Google publishes an official memory table in the Gemma 4 docs, and it is the real sizing document. Weights only, before KV cache:

| Model | BF16 | SFP8 | Q4_0 | |---|---|---|---| | E2B | 11.4GB | 5.7GB | 2.9GB | | E4B | 17.9GB | 8.9GB | 4.5GB | | 12B | 26.7GB | 13.4GB | 6.7GB | | 26B A4B | 57.7GB | 28.8GB | 14.4GB | | 31B | 69.9GB | 34.9GB | 17.5GB |

Read that table carefully, because the model names mislead in both directions:

• The "effective" models are bigger on disk than their names suggest. E2B needs 11.4GB at BF16. The 2B refers to active compute, not weights. On-device deployments get it far smaller: Google's edge stack runs E2B in under 1.5GB on supported devices using LiteRT's 2-bit and 4-bit weights.
• The MoE saves compute, not memory. The 26B A4B holds nearly as much memory as the 31B dense while activating 3.8B parameters per token. You provision for 26B and pay per-token compute like a 4B. If your bottleneck is VRAM rather than throughput, the MoE buys you nothing.
• Quantization-aware training is the production default. Google ships QAT checkpoints (q4_0 GGUFs for the llama.cpp ecosystem, plus unquantized QAT weights for custom pipelines) trained to hold quality at 4-bit, which is a materially better starting point than post-hoc quantization. Still eval the delta on your own tasks.

What this maps to in hardware: the 31B at Q4 fits a 24GB consumer GPU or a 24GB Apple Silicon Mac; community measurements on Apple hardware put the Q4 31B at roughly 40 to 50 tokens/second on an M4 Max and 15 to 25 on a 24GB M2/M3 Pro. At the bottom of the ladder, Google's own edge numbers: E2B on a Raspberry Pi 5 CPU does 133 prefill / 7.6 decode tokens per second, and 3,700 prefill / 31 decode on a Qualcomm Dragonwing IQ8 NPU. That spread, one family from a Pi to an H100, is the operational argument for Gemma 4: one license, one chat template, one eval harness across every deployment tier.

For managed deployment, Google Cloud runs the family on Vertex AI Model Garden, Cloud Run GPUs (RTX PRO 6000 Blackwell, 96GB, scale-to-zero), and GKE with vLLM, with TPU serving via vLLM TPU announced alongside. The option that matters for regulated clients: Google has committed Gemma 4 across its sovereign offerings, up to air-gapped on-premises Google Distributed Cloud.

The Arena story, read honestly

Google's launch claim is that the 31B dense ranks #3 among all open models on the Arena text leaderboard, with the 26B MoE at #6, both ahead of models with many times their parameter count; the DeepMind model page lists the 31B thinking variant at an Arena score of 1452. The rank is real and the achievement is real: a 31B dense model in that neighborhood changes what "small" means.

Three caveats before you repeat it in a business case. Arena measures human preference on chat, not your workload. The margins over the adjacent Qwen models are thin enough that community reads of the same data saw a lead a single leaderboard update could erase. And leaderboards move monthly; the ranks quoted here are the April 2026 launch snapshot. Our standing advice is unchanged: vendor benchmarks shortlist models, your own task evals pick them.

Launch-week reality, and behavior that matters

Gemma 4's first weeks were rougher than the launch post, and the failure modes are instructive (community postmortem):

• Day-one ecosystem lag. The new architecture broke Hugging Face Transformers and PEFT support at release, and teams with fine-tuning plans waited on upstream fixes. By July 2026 the ecosystem has caught up (official QAT GGUFs, llama.cpp, Ollama, vLLM, Unsloth), but the lesson generalizes: for any new open-weights family, pin runtime versions and budget slack between release day and production day.
• MoE throughput lagged in early local runtimes. One early community test measured the 26B MoE at 11 tokens/second where a comparable Qwen model did 60+ on identical hardware. Runtime-level, not architectural, and improving, but it is why we benchmark the actual runtime you will ship, not the architecture diagram.
• Audio is edge-only at the top. The 26B and 31B do not take audio input; E2B, E4B, and the June 12B Unified do. A voice product on the flagship needs a separate transcription stage.
• Video input has practical limits. Local runtimes process short clips at low frame rates (roughly a minute at about 1 frame/second per Unsloth's deployment docs). Treat it as sampled-frames understanding, not long-form video analysis.
• Mind the sampling defaults and thinking hygiene. Recommended defaults are temperature 1.0, top_p 0.95, top_k 64, and in multi-turn use you keep only final answers in history, never prior thought blocks. Small details, real quality deltas.

When to use it, and when not

Use Gemma 4 when:

• Data cannot leave the device or the VPC and you want one Apache 2.0 family covering every tier, phone-class intake to server-side reasoning, under a license procurement will not fight.
• The product is genuinely edge: field apps that work offline, kiosks, NPU-equipped hardware, on-device inference where E2B/E4B latency and footprint are the feature.
• You are replacing a mid-tier hosted model on cost: a Q4 31B on a 24GB GPU serves a large class of extraction, summarization, and agent workloads with zero per-token vendor spend.
• White-label or fine-tune-and-rebrand plans make conditionally licensed alternatives legally awkward.

Do not use it when:

• The workload needs frontier-grade reasoning. The family tops out at 31B dense; the hardest problems still belong to a hosted frontier model or a much larger open model, ideally behind a router so only those requests pay for it.
• You need audio input at the large sizes, or long-form video understanding, today.
• Serving throughput is the whole game and your stack's MoE support is immature; measure your runtime before committing to the 26B A4B.
• Your context requirements genuinely exceed 256K.

How we would architect it for a client

The fit we see most clearly is the compliance-heavy field-workforce pattern, the same shape as the NDIS provider-management platform we build as a long-term development partner (Vertex360): mobile workers capturing forms, photos, and notes in environments with unreliable connectivity, feeding a regulated back office.

1. E4B on the device for offline intake: classification, structured extraction to JSON via the native constrained output support, running in the app's own process so nothing leaves the handset unprocessed.
2. 31B (QAT q4_0) in the client's sovereign cloud for the heavy work: cross-document summarization, compliance narrative drafting, agentic workflows over the case record. One GPU class, predictable capex, no per-token bill.
3. One eval harness across both tiers. Same task suite, run against E4B, 31B, and the incumbent hosted model, so every routing decision is a measured tradeoff. This is standard model engineering discipline, not Gemma-specific.
4. A swappable model interface anyway. Apache 2.0 removes the license reasons to leave, not the technical ones. The open-weights field re-ranks every quarter, and the gateway keeps the exit cost at a config change.

Gemma 4 is not the best model in the world. It is something more useful: the best-licensed, best-laddered family for the deployments where the model has to live where the data lives. For that slot, in mid-2026, it is the first name on our shortlist.