arXiv 2606.20517 introduces Multi-LCB (Multi-Language LiveCodeBench), an extension of the popular LiveCodeBench that evaluates LLMs on programming tasks in 12 languages. The benchmark is used to evaluate 24 mainstream LLMs, revealing significant cross-language performance gaps.
The 12 languages: English, Chinese, Japanese, Korean, Russian, Spanish, Portuguese, French, German, Arabic, Hindi, and Vietnamese. The tasks are programming problems (algorithms, data structures, system design) translated and adapted for each language, with the same underlying difficulty.
The result: the average cross-language performance gap is 18.4% — i.e., the average LLM scores 18.4% lower on non-English tasks than on English tasks. The worst gap is 32% (for Vietnamese), and the best is 8% (for Chinese). The leaderboard shows that "good at English coding" does not imply "good at multi-language coding."
The "monster mirror" highlight: Multi-LCB exposes the "English bias" in current LLM coding evaluations. Most coding benchmarks are English-only, and the cross-language performance is hidden. The Multi-LCB results show that vendors claiming "best coding model" should be evaluated on multi-language performance, not just English.
The bigger takeaway: "multi-language evaluation" is essential for global LLM deployment. The "English-only benchmark" is misleading, and the industry needs multi-language standards. For the industry, this means LLM vendors targeting global markets (especially non-English markets) should prioritize multi-language training and evaluation, not just English SOTA.