In mid-April, Meta officially released the Llama 4 series, bringing the Mixture-of-Experts (MoE) architecture to the Llama family for the first time. The lightest-positioned model in the series, Scout (17B activated / 109B total), is now open-source, sparking wide developer community discussion — not because it's the largest, but because it makes for the first time billion-scale context + vision understanding + single-card deployment possible.
Scout's most striking technical spec is the 10M token context window. Traditional RoPE's signal-to-noise ratio drops sharply on ultra-long sequences; Meta's solution is iRoPE: standard RoPE on layers 1, 2, and 3 to preserve local token order; switching to NoPE on layer 4, removing absolute positional encoding, letting attention heads do global perception on the entire causal mask. The MoE sparse-activation design means that although Scout has 109B total parameters, each token only needs to activate 17B — getting near 100B+ model knowledge capacity at 17B compute.
Benchmark results show: MMLU-Pro Maverick at 80.5 surpasses GPT-4o (78.0); ChartQA/DocVQA set new SOTA at the same scale; Scout maintains >99% accuracy on 10M token NIAH tests, while competitors start hitting walls in the 128K-1M range. But on pure STEM reasoning, OpenAI's o-series still leads.
Meta simultaneously open-sourced Llama Guard 4 (12B) and Prompt Guard 2 (86M), forming a four-layer safety pipeline. Llama 4 Scout's emergence reinforces a signal: the open-source model's competitive focus is shifting from "parameter count" to "efficiency density."