Mixture-of-Experts has carried most of the capacity scaling in large models over the past two years, but one structural trade-off keeps getting routed around: for a single token, how many experts contribute knowledge, how many expert passes actually get computed, and how many expert-sized parameter sets must be built and stored — in existing designs these three quantities are locked together. AMap's September paper, IntBMoE (arXiv:2609.21346), proposes a decoupling scheme, and the paper states it is already deployed in AMap's generative recommendation system serving hundreds of millions of users.
Why the three quantities were locked
The paper defines them precisely. Participation is how many experts contribute knowledge to a token's output; execution is how much expert computation actually runs; materialization is how many expert-sized parameter sets must be built and stored. Three existing routes each get stuck on one: sparse routing keeps execution and materialization low but shrinks participation to a few experts per token; dense output-mixing restores full participation but execution grows with the expert count; parameter-merging keeps execution at one expert while materialization grows with the number of routing decisions. You could optimize at most two of the three — the third always got worse.
IntBMoE: compose first, execute sparsely
IntBMoE's core is block-level conditioning: block configurations come from a small learned codebook, and at each layer a lightweight hypernetwork merges every expert base in that layer's pool into composed experts. Participation is full by construction — every composed expert draws on the entire pool. Execution stays sparse — a router sends each token to only a few blocks. Materialization is bounded — the codebook, not the input, fixes how many blocks exist. A Dual-Path Residual Gating (DPRG) design couples two independently composed paths through multiplicative gating, completed by feature filtering and an always-active shared expert.
One practical detail deserves emphasis: blocks can be precomposed and cached, so with a fixed block configuration, inference compute is independent of the expert count. This is what separates it from dense-mixing approaches — capacity scaling no longer translates directly into an inference bill.
Production numbers, vendor-reported
Per the paper's own account, IntBMoE is fully deployed in AMap's generative recommendation system, serving hundreds of millions of users under a 60ms latency budget, with a 2.4% relative UVCTR gain in online A/B testing. On the experimental side, it shows consistent gains over representative sparse and dense MoE baselines on image classification, with language modeling and sequential recommendation validating generalization across domains. Caveat: the online figures are all vendor-reported from official A/B tests with no independent replication yet — read them with that filter on.
What the open-source repo gives, and what it doesn't
The code lives in the DreamX-Rec repository on GitHub (Apache 2.0, currently 121 stars). The repo hosts six AMap generative-recommendation works, with IntBMoE as its capacity-expansion module, shipping recommendation, NLP (MiniPile), and vision (ImageNet-1K) examples. Two sober notes: first, in the open-source implementation, BlockMoE's forward pass recomposes parameters on every call — the serving cache studied in the paper is not part of the public code. Second, the recommendation example trains single-task POI prediction, not the full multi-task objective. The online gains came from AMap's internal system, and there is a clearly marked distance between the public code and reproducing them — the README states this boundary itself.
For people working on inference architecture, the real increment in this paper is not the 2.4% — it is refining the MoE cost model from a single "expert count" knob into three independently tunable quantities. While the community simultaneously struggles with expert-parallel communication overhead and memory that cannot hold the full expert pool, "compose first, execute sparsely" offers a third path. The next milestone to watch is whether the community fills in the serving cache — that is the hinge on whether this design escapes AMap into the general LLM stack.
References: paper arxiv.org/abs/2609.21346; code github.com/AMAP-ML/DreamX-Rec