For the last few years, most evaluations of "spatial intelligence" have done the same thing: hand the model a single image or a short video clip, ask whether the sofa is to the left or right of the television, and declare victory on a correct answer. That format can prove a model recognizes objects. It cannot prove the model can place those objects into a metric 3D map. The paper arXiv:2610.26659 changes the exam format: no multiple choice. The model gets the camera and has to go explore the room on its own.

Task design: the model holds the camera

The benchmark was built by a Peking University team, with Zhaoxi Zhang as project lead and Weikang Li as corresponding author, jointly with Tianjin University. It is released under CC BY 4.0, and the code is open at https://github.com/JesseZZZZZ/SpatialScout .

The scene set comprises 80 indoor scenes containing 3,140 target objects, with 30 to 40 targets per scene, spanning diverse spatial layouts, object scales, and repeated category instances. The agent starts with one RGB image, an object inventory (instance identifier, category, equivalence group), plus camera intrinsics and poses. From there it works through ten shared tools: list_entities to read the inventory, get_camera_pose and get_camera_intrinsics for calibration, set_camera_position, move_camera and rotate_camera to steer, capture_image to grab frames, triangulate_observations for multi-view triangulation, submit_positions to write or revise object centers, and finish_mapping to stop. No tool reveals ground-truth target positions or evaluation scores.

The budget is tight. Each episode allows 100 model calls and 100 acquired images, including the initial one; once captures are exhausted, the remaining calls can only be used for reasoning and submission. Explicit captures render 512x512 RGB images, delivered as JPEGs with the longest side at most 384 pixels, so the agent has to convert delivered coordinates back onto the original image grid. Ground-truth centers are defined simply: the midpoint of the tight world-axis-aligned bounding box of the rendered geometry, with the union of link boxes for articulated objects.

Scores: the strongest model in the paper reaches only the low forties

The paper evaluates seven frontier models on the shared 80-scene set. All numbers below are the paper's official figures, object-micro over all targets:

  • Claude Opus 5.5 (xhigh): mean IoU 41.71, positive-overlap recall 77.36, 47.23% of targets above IoU 0.5, and only 3.25% above 0.9
  • Claude Sonnet 5.5: mean 32.26, IoU>0.5 at 33.60, IoU>0.9 at 1.40
  • GPT-6 Astra (xhigh): mean 33.16, IoU>0.5 at 35.15, IoU>0.9 at 1.62; dropping to the medium reasoning setting pulls it down to 31.18
  • GPT-6 Sol: mean 20.83, IoU>0.5 at 18.95, IoU>0.9 at 0.48
  • Qwen3.8 Max: mean 6.80, IoU>0.5 at 4.52, IoU>0.9 at 0.00
  • Kimi-K3: mean 4.22, IoU>0.5 at 2.77, IoU>0.9 at 0.00
  • DeepSeek V4.1 Flash: mean 3.48, IoU>0.5 at 0.92, IoU>0.9 at 0.00

Restricting the average to objects the models actually submitted lifts Claude Opus 5.5 to 46.69 and GPT-6 Astra to 41.16. That gap is exactly what omissions cost each model.

Tool-call patterns expose behavioral differences

The paper's tool-call frequency figures offer a more revealing signal than the scores alone. Claude Opus 5.5 makes the largest number of tool calls, and the authors read its broad exploration as the reason for better map quality. GPT-6 Astra reaches competitive mapping performance with noticeably fewer calls. In the opposite direction, DeepSeek V4.1 Flash and Qwen3.8 Max spend many turns moving the camera while observing and mapping far less. The paper interprets this as difficulty selecting informative viewpoints and difficulty converting visual observations into a coherent map. Uninformative views are not free either, since images consume substantial context length.

The dominant failure mode is omission, not offset

The paper analyzes 40 trajectories from four models on the same ten selected tasks. Reconstruction success requires both at least 90% object coverage and a mean center error of at most 0.2 meters over matched submitted objects. Only six trajectories clear both bars, and all six share the same profile: extensive multi-view geometric grounding, incremental submission that expands coverage, and subsequent refinement.

Among the 34 unsuccessful trajectories, incomplete coverage is the most common pattern, appearing in 26 of them. In 21 cases at least one submitted position stays substantially inaccurate despite a low triangulation residual, which the authors attribute to incorrect visual correspondences or wrong object-center selection, neither of which geometric consistency alone can resolve. Thirteen cases retain inaccurate positions alongside geometric-inconsistency warnings, and another overlapping set of thirteen terminates with at least ten response rounds still unused.

So what

The benchmark punctures the comfortable assumption that visual common sense equals spatial competence. Recognizing an object and getting left/right/front/back right is one kind of ability; putting its center into a shared coordinate frame at centimeter precision is another. The first can be partly guessed from statistical regularities in image-text alignment. The second demands cross-view association, geometric estimation and continuous revision all working at once.

What makes this interesting for practitioners is that the benchmark does hand the model a triangulation tool that estimates a 3D point from correspondences across at least two calibrated views, and reports the root-mean-square perpendicular line residual along with the maximum pairwise ray angle. The paper is explicit that a low residual establishes neither whether the marked pixels identify the same object feature nor how the estimated point relates to the object's center. The geometry pipeline can be very precise, yet nothing in that pipeline decides whether the model is even looking at the right thing. That is a fair description of the hardest blocker in embodied AI today.

The paper appeared on the Hugging Face Daily Papers board for October 22, 2026, and the repository is public for anyone who wants to reproduce the runs.