Zyphra released Zamba2-VL, the first VLM (Vision-Language Model) based on a Mamba2+Transformer hybrid architecture. The standout: TTFT (Time To First Token) cut by an order of magnitude (10×) compared to pure-Transformer VLMs, with no quality loss on multimodal benchmarks.
The "hybrid VLM" architecture: Zamba2-VL uses a Mamba2 backbone for the LLM component, with cross-attention layers to a vision encoder. The Mamba2 backbone gives O(n) inference complexity, vs the O(n²) of a pure Transformer, which translates directly to faster TTFT.
The technical details: the Mamba2 backbone has 7B parameters, with 4 cross-attention layers per 32 Mamba2 layers. The cross-attention layers attend to the vision encoder's output, allowing the Mamba2 to "see" the image. The training is a standard VLM pipeline (image-text contrastive, SFT, RLHF) with a hybrid architecture.
The benchmark: on the VLM benchmark (image captioning, VQA, multimodal reasoning), Zamba2-VL-7B scores on par with Qwen2.5-VL-7B and LLaVA-1.6-13B. The TTFT is 10× faster (150ms vs 1500ms for a 1K-token image description). The inference cost is also significantly lower.
The bigger takeaway: "Mamba2+Transformer hybrid" is the right architecture for efficient VLMs. The "pure Transformer" assumption is wasteful, and the "hybrid" approach gives 10× speedup with no quality loss. For the industry, this signals that the next generation of multimodal models will adopt hybrid architectures, and the "Mamba2 + cross-attention" pattern will be the standard.