Preprint · 2026
UNC Chapel Hill · *Equal contribution
Many spatial questions are about what a model cannot see: what it would face after an instructed movement, or how objects seen in separate views relate to each other. Answering them takes spatial imagination, building an imagined state: the scene as it would appear from a viewpoint the model has not observed. Visual world models imagine this state by generating RGB views, largely because RGB is what they are built to generate, not because it is necessarily what reasoning needs. We ask which representation of the same imagined state actually helps.
We hold the imagined state fixed, change only how it is represented, and measure what helps spatial reasoning.
Visual world models increasingly support spatial reasoning by generating RGB views of unobserved states. However, RGB has become the default interface largely because it is well suited for visual generation, rather than necessarily being the most effective representation for downstream reasoning. Different spatial questions may benefit from different representations of the same imagined state: top-down geometry can facilitate global distance comparisons, while panoramic views provide broader context for reasoning about open space. To isolate the role of representation in spatial imagination, we introduce ImagiRep, a simulator-grounded benchmark of 922 questions spanning seven question types across two settings, Viewpoint Imagination and Cross-view Integration, providing oracle RGB and six alternative representations (e.g., depth, panoramic views, and top-down geometry) of the same unobserved state. This controlled comparison reveals that even accurate imagined RGB does not guarantee accurate spatial reasoning, and different questions benefit from different representations. Similar trends hold for representations derived from world-model-estimated states, even when estimated RGB alone degrades performance. Building on these findings, we introduce GEARS, a lightweight strategy that adaptively selects imagined evidence at test time, outperforming existing spatial imagination methods on both Viewpoint Imagination and Cross-view Integration. Our results suggest that effective spatial imagination depends not only on generating unseen views, but also on representing spatial information according to the needs of downstream reasoning.
Benchmark
Built on Matterport3D scenes with known geometry and camera poses, ImagiRep keeps the scene, the question and the answer options fixed and changes only how the unobserved state is shown to the model. The endpoint is the unobserved viewpoint a question is about: where the observer stands after the instructed movement, or the missing view between the given ones. Oracle representations are rendered from the true scene at the endpoint, so the imagination itself is error-free; estimated ones are rendered from a scene predicted by a world model.
From a single starting view and an instructed movement (one to three moves, 0.5–9 m), reason about the new viewpoint: open-space and nearest-obstacle direction, visibility, nearest and farthest object, and where a named object lies. 200 routes across 73 scenes.
From two or three partially overlapping views, combine evidence about objects that never appear together: cross-view nearest and farthest object, and the left-to-right order of objects from a given view. 143 view groups across 21 scenes.
Examples
Pick a question, try it yourself, and switch between representations of the same imagined state.
Findings
Accurate imagined RGB helps overall, but can hurt specific questions. Oracle endpoint RGB adds roughly 8–12 points over the observed views, yet reasoning stays far from perfect, and on Farthest Object questions GPT-6.1-sol drops from 68.4% to 58.2% even with the ground-truth view.
Alternative representations add complementary evidence, and no single one wins everywhere. Every representation recovers some errors made with oracle RGB. The best one depends on the question: top-down geometry for Farthest Object, panorama for Open-space Direction, depth for cross-view Nearest Object. Picking the right one per question (oracle selection) would reach 87.5% and 91.7% with GPT-6.1-sol.
The benefit survives imperfect imagination. With states estimated by world models (HY-World 2.0 and Stable Virtual Camera), estimated RGB alone lowers Viewpoint Imagination accuracy from 57.2% to 54.7%, while every alternative representation built from the same estimated scene lifts it to 61.7–65.6%.
Method
Gated Evidence and Adaptive Representation Selection is a lightweight test-time strategy. If the answers with and without the estimated endpoint RGB agree, GEARS keeps that answer. If they disagree, the reasoner chooses which evidence to trust, from the observed views alone to an estimated representation, and answers with it.
All methods use GPT-6.1-sol as the final reasoner and are evaluated on the same questions. MindJourney uses its released Search World Model; AVIC uses Stable Virtual Camera.
More examples
Citation
@article{zhao2026imagirep,
title = {Representing the Unseen: Rethinking Visual Imagination for Spatial Reasoning},
author = {Zhao, Zengqi and Wang, Zun and Yu, Shoubin and Bansal, Mohit and Zhang, Yue},
journal = {arXiv preprint},
year = {2026}
}