Agnes AI has published Agnes-3.0-Flash Preview on HuggingFace: a 33B-parameter open-weights checkpoint under Apache 2.0. The most interesting part of this release is not the size but the attention structure itself.
KV cache grows in only 18 layers
Agnes-3.0-Flash Preview is a hybrid-attention decoder. For every four layers, three run a gated delta rule (recurrent, with per-layer state independent of sequence length) and the fourth runs standard global attention, alternating 3:1. Of the 72 decoder layers, 54 are delta-rule recurrent layers and 18 are global attention layers.
This directly changes the memory curve at inference time: the recurrent state is fixed per layer regardless of sequence length, so only the 18 attention layers hold a KV cache that grows with context. The model card lists every spec: recurrent state in fp32, a causal conv (kernel 4) in front of each delta-rule layer, 16 key heads / 48 value heads, hidden size 5,120, vocabulary of 248,320, and 3-axis rotary (text / height / width) with interleaved mrope sections 11:11:10.
The context window is 262,144 tokens. Recommended hardware from the model card: a single NVIDIA H200 141GB or H100 80GB, roughly 66GB for the bf16 checkpoint, 128GB+ host memory. A single-GPU workstation can run the full 262k-context multimodal model — no cluster required.
Benchmarks, vendor-reported
The model card includes a comparison table but explicitly notes the figures were "compiled from different sources, harnesses, and model snapshots and do not constitute a controlled head-to-head comparison." Reference values for Agnes-3.0-Flash Preview: IFBench 74.20, GPQA Diamond 85.05, SciCode 38.08. In the same table, Qwen3.8-27B posts GPQA 90.5 and MiniMax M3 92.9 — Agnes clearly trails the leaders on GPQA; on IFBench, 74.20 beats Qwen3.6-35B-A3B's 64.4 and Kimi K2.5's 43.7.
In other words, this is not a leaderboard flagship but a single-GPU long-context multimodal base. It supports text, image and video understanding, tool calling via
A rare honest label
The model card carries a dedicated "Model version clarification" section: these open weights are the Preview checkpoint (33B, 262k context), while the Agnes 3.0 Flash listed on Artificial Analysis is a different production/API checkpoint with a 1M-token context — "its benchmark results should not be attributed to the Preview weights released here." The repository was initially published without the Preview suffix and was renamed to make the distinction explicit.
Open-weight and API versions sharing a name with different capabilities is a trap users usually discover the hard way; writing the difference into the first section of the model card and renaming the repo deserves a note.
So what
As of this writing, 7 community quantizations already exist on HF (llama.cpp / LM Studio / Ollama / Jan ecosystems), with 474 downloads last month — not hot, but the combination of hybrid architecture + Apache 2.0 + single-GPU entry makes it a cheap experiment target for local-deployment and long-context teams. Whether the 3:1 delta-rule/attention mix holds at other scales is worth watching.
Sources: model card https://huggingface.co/Agnes-AI/Agnes-3.0-Flash ; The Open Weights https://www.theopenweights.com/news/agnes-3-0-flash-z86d