On November 13, a paper landed on arXiv under the ID 2611.18397, and the author block is not a familiar LLM vendor: it is the miHoYo Honkai AI R&D Team, with collaborators from the National University of Singapore, Tsinghua University, the University of Chinese Academy of Sciences, USTC, and Xi'an Jiaotong University. The model is called Gamer, a generalist vision-language-action (VLA) foundation model aimed at long-horizon, real-time game agents. It reads no game memory and calls no special API: it watches screen pixels and emits native keyboard-mouse actions, exactly like a human sitting in front of the monitor.

Three-stage training: watch recordings, learn instructions, then learn by doing

Gamer starts from Qwen3-VL-8B and trains in three stages. Stage one is continued pre-training (CPT) on tens of thousands of hours of device-level keyboard-mouse trajectories. The Honkai: Star Rail recording track alone holds roughly 68K sessions and 11K hours of raw video; after filtering corrupted, misaligned, or inactive segments, about 3.2K sessions and 1K hours survive for annotation, and the HSR portion of the CPT corpus ends up as roughly 149K aligned clips covering around 332 hours. Gameplay trajectories account for 85% of training tokens, with the remaining 15% from general multimodal data. The team also ran a controlled 120-configuration study (4 backbone sizes x 6 gameplay budgets x 5 data mixtures). The action interface samples one observation every 200 ms (5 Hz) and splits each interval into six keyboard-mouse sub-blocks, an effective control rate of roughly 30 Hz.

Stage two is supervised fine-tuning: GPT-5.6-Luna annotates sparse Think steps at key decision points, plus task instructions and memory summaries, stitching three instruction levels (local operations, subgoals, complete tasks) into the same trajectories. Stage three is TA-GRPO online reinforcement learning: 64 NVIDIA H200 GPUs serve 32 parallel game environments, each iteration collects 128 rollouts, and rewards come from environment-side verification, with no learned process reward model.

The scoreboard: 83.6% on side quests, 49.7% across seven minigame types

On two held-out Star Rail side quests, overall accuracy moves from 0.0% for the base model to 34.4% after CPT, 72.2% after SFT, and 80.0% after RL on Opportunity Makes the Thief, and from 13.9% to 47.9%, 69.1%, and 83.6% on Bestial Ferocity. Across 40 held-out minigame instances spanning seven gameplay types, the macro-average climbs from 3.2% for the base model to 15.0% for Gamer-CPT, 39.2% for Gamer-SFT, and 49.7% for Gamer-RL. In the same table, a row of API frontier models lands at GPT-6.1-Sol 17.1%, GPT-5.6-Sol 12.0%, GPT-5.6-Luna 11.5%, Claude-Opus-5.0 9.9%, GLM-5.3-Flash 8.2%, and 2.0% each for Gemini-3.8-Flash and DeepSeek-V4-Flash-Vision — all self-reported numbers from the paper. On four public game benchmarks, the 8B CPT policy averages a real-time score of 29.5 and the Qwen3.5-35B-A3B MoE variant reaches 31.5; the paper states that both exceed every general-purpose baseline listed in that table.

The hidden protagonist: an engine that cuts 1.31 seconds to 65 ms

Real-time games are brutally sensitive to inference latency, and this is where the in-house Streaming-vLLM engine earns its keep: it retains KV states across turns under memory compaction, verifies structural action tokens with a finite-state machine, and dispatches verified action blocks early. On an H200 running the Qwen3.5-35B-A3B MoE model, end-to-end latency drops from 1306 ms to 65 ms, which the paper calls a 20.1x speedup, while the step rate rises from 0.77 Hz to 15.36 Hz. The engine feeds straight back into training: when rollouts switch to standard vLLM control at 0.6 FPS, reward curves across eight navigation tasks collapse in later training stages. On streaming video understanding, Streaming-vLLM scores 83.29 on StreamingBench versus 83.21 for DSCache, with query latency falling from 0.53 s to 0.08 s.

miHoYo is not alone on this road

Training game agents on human keyboard-mouse trajectories plus scaling is not a one-off. Game-TARS (arXiv 2510.23691, October 2025) pre-trained on over 500B tokens with a unified keyboard-mouse action space. Another paper from January 2026 (arXiv 2601.04575) open-sourced 8300+ hours of high-quality human gameplay plus a full training recipe, with a model that runs real-time inference on a consumer GPU and performs competitively with human players. Different teams, different recipes, each independently landing on the same answer.

A few buckets of cold water

The paper lists three limitations of its own: evaluation still covers a selected set of environments; serving costs can drop further; and deep reasoning remains a bottleneck, with the authors sketching a layered System 2 plus System 1 architecture as a remedy. Distant transfer is not magic — the paper concedes that gains on the more distant GameWorld benchmark remain limited for CPT checkpoints. RL is not uniformly positive either: on Janus's Maze, one of the seven minigame types, success falls from 52.4% after SFT to 23.8% after RL. And while the GitHub repository is live, the Streaming-vLLM engine still sits on the repository TODO list, and the CPT checkpoints the paper says will be released are not yet visible. The claim of operational efficiency comparable to skilled players likewise remains the paper's own framing, not an independent measurement.

Games are environments with verifiable rewards built in, and no one knows better than a game company where thousands of repetitive tests are needed. When miHoYo starts training its own models, the first landing spot is probably automated QA and regression testing, not playing alongside you. The keyboard and mouse are becoming another universal action vocabulary alongside embodied AI — which game company steps in next?

Paper: https://arxiv.org/abs/2611.18397 | Project: https://mihoyo-gamer.github.io/ | Code: https://github.com/RainBowLuoCS/Gamer