A new paper posted to arXiv on July 21 breaks the most expensive part of MoE training — optimizer state — into three buckets. The paper is titled "Where Should Optimizer State Live? Tiered State Allocation for Memory-Efficient Mixture-of-Experts Training", by Nuemaan Malik, with code open-sourced on GitHub (nuemaan/skewadam). Where the problem lies. A 6.78B-parameter MoE language model has only 12.6 GB of weights (bfloat16), but AdamW's first- and second-order moments take 50.6 GB — four times the parameters. The training peak is 81.4 GB, which doesn't fit on a 40 GB card. This is the most direct reason MoE training has been locked behind big-company walls. SkewAdam's key observation is simple: the three kinds of parameters in MoE shouldn't all get the same optimizer treatment. The dense backbone, which accounts for 5%, is responsible for most of the gradient signal; the experts, 95%, are sparsely activated; the router, less than 0.01%, only does routing decisions — they're not on the same scale in terms of gradient statistics or sample count. The paper assigns each of the three groups its most appropriate state format: the backbone keeps float32 moments plus a factored second moment; experts only keep a factored second moment (no moment, since they're sparsely selected by the router); the router, though small, is decision-sensitive and keeps a full second moment. The numbers are hard. Same initialization, same controlled comparison over 82M tokens: optimizer state drops from 50.6 GB to 1.29 GB, which is 2.6% of AdamW; training peak drops from 81.4 GB to 31.3 GB, fitting comfortably on a 40 GB accelerator. Validation perplexity: SkewAdam 108.4, AdamW 126.8, Muon 120.2, Lion 393.7; router load balance converges to within 1% of the uniform distribution. The paper does a key ablation: pulling all three states up to the backbone's same treatment, parameter memory balloons 20x and perplexity is essentially unchanged; switching to Adafactor (sharing factored but dropping moments), perplexity plateaus at a 40-point deficit. Two cuts make the conclusion clear: the saved memory comes from tiered design; the kept precision comes from not dropping moments. The closing line "where optimizer state lives matters at least as much as how much of it there is" is the paper's thesis. There are three layers of meaning for practitioners. First, the threshold for multi-B-parameter MoE training drops directly to a single 40 GB card — what used to require stacking A100/H100 80GB budgets can now be shouldered by consumer-grade 4090/5090 clusters, and the open-source community can bypass big-company compute walls for MoE post-training. Second, the tiered strategy is a way of thinking, not numbers: when new parameter subgroups emerge in MoE (shared experts, retrieval heads), you can keep splitting along "gradient sample count determines state precision". Third, optimizer state takes more memory than the weights themselves, so anyone wanting to compress memory should compress optimizer state first, then activations, then weights. The code is pure single-file PyTorch, drop-in to existing training scripts.