Text-to-image models have gotten very good at looking plausible. LongBanana, released by researchers from South China Normal University, Waterloo, the Vector Institute, CMU, Tsinghua, HKUST and HKU together with KlingAI Research, tests a harder skill: given about 32 reference images per task, find the scattered evidence, bind it to the right entities, satisfy 20 constraints, and compose one coherent image. Several authors did the work during internships at KlingAI Research, and the project page marks the benchmark accepted to NeurIPS 2026.
What the exam tests
LongBanana contains 90 tasks in a 3-by-3 taxonomy — Scene Composition, Visual Appearance and Reference Reasoning — with ten cases per category. Each task carries about 32 reference images, 20 constraints and 50 evaluation items, for 4,670 expert-specified atomic checks in total. Cases are reverse-constructed: experts start from a realistic scenario, build an entity-relation graph, and align reference inventories, synthesis instructions and evaluation criteria to that same graph. At least 30 cases embed two or more hard negatives — visually convincing distractors that are simply wrong to use. The authors call it the first expert-designed and expert-verified benchmark for long-context multi-reference image synthesis.
Scores you can audit
The same graph grounds evaluation: node, edge and whole-graph obligations become checklist items for Content Fidelity, Relational Binding and Global Constraint Satisfaction. Three critics cover those three categories, with optional OCR, counting, layout and identity tools; every judgment records an evidence trace before aggregation, scored 0, 1 or 2. The human-audited pool spans 90 cases across 10 generation conditions — 900 outputs and 46,700 reviewed labels — and a 15% double-labeled subset shows 89.4% exact agreement with 0.83 quadratic weighted kappa. Image quality becomes an auditable chain of evidence rather than a vibe.
Agents expose their own weakness
The companion LongBanana-Gen framework proposes RPR — ReAct plus Plan-Exec plus Reflect — to retrieve evidence, organize generation and repair reference-grounded errors. The results cut both ways: minimal agent harnesses beat direct generation, and RPR lifts checklist averages across the reported harnesses; yet task pass rates drop for OpenCode and Hermes-Agent. Longer reasoning chains create more chances to cite the wrong evidence — more reasoning does not mean the right reference stays bound.
So what
The next bottleneck for image generation is not pixels but evidence: retrieval, binding and verification. The dataset is on Hugging Face (Kwai-Kolors/LongBanana), with code and data on the project page. Next time a model "tops the leaderboard" on text-to-image, ask one question first: can it pick the right image out of 32 references?