--- base_model: - OpenGVLab/InternVL3-2B license: apache-2.0 pipeline_tag: image-text-to-text library_name: transformers language: en --- **EN** | [中文](README_CN.md) # SenseNova-SI: Scaling Spatial Intelligence with Multimodal Foundation Models Code arXiv Code Leaderboard ## Paper The model was presented in the paper [Scaling Spatial Intelligence with Multimodal Foundation Models](https://arxiv.org/abs/2511.13719). ## Code The official code can be found on the [OpenSenseNova/SenseNova-SI GitHub repository](https://github.com/OpenSenseNova/SenseNova-SI). ## Overview Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation models to cultivate spatial intelligence within the **SenseNova-SI family**, built upon established multimodal foundations including visual understanding models (i.e., Qwen3-VL and InternVL3) and unified understanding and generation models (i.e., Bagel). We take a principled approach to constructing high-performing and robust spatial intelligence by systematically curating SenseNova-SI-8M: eight million diverse data samples under a rigorous taxonomy of spatial capabilities. SenseNova-SI demonstrates unprecedented performance across a broad range of spatial intelligence benchmarks: 68.7% on VSI-Bench, 43.3% on MMSI, 85.6% on MindCube, 54.6% on ViewSpatial, and 50.1% on SITE, while maintaining strong general multimodal understanding (e.g., 84.9% on MMBench-En). More importantly, we analyze the impact of data scaling, discuss early signs of emergent generalization capabilities enabled by diverse data training, analyze the risk of overfitting and language shortcuts, present a preliminary study on spatial chain-of-thought reasoning, and validate the potential downstream application. SenseNova-SI is an ongoing project, and this report will be updated continuously. All newly trained multimodal foundation models are publicly released to facilitate further research in this direction. *In the future, SenseNova-SI will be integrated with larger-scale in-house models.* ## Release Information Currently, we build SenseNova-SI upon popular open-source foundation models to maximize compatibility with existing research pipelines. In this release, we present [**SenseNova-SI-1.1-InternVL3-2B**](https://huggingface.co/sensenova/SenseNova-SI-1.1-InternVL3-2B) and [**SenseNova-SI-1.1-InternVL3-8B**](https://huggingface.co/sensenova/SenseNova-SI-1.1-InternVL3-8B), which achieve state-of-the-art performance among open-source models of comparable size across five recent spatial intelligence benchmarks: **VSI**, **MMSI**, **MindCube**, **ViewSpatial** and **SITE**.
Model VSI MMSI MindCube-Tiny ViewSpatial SITE
Open-source Models (~2B)
InternVL3-2B32.926.537.532.530.0
Qwen3-VL-2B-Instruct50.328.934.536.935.6
MindCube-3B-RawQA-SFT17.21.751.724.16.3
SpatialLadder-3B44.827.443.439.827.9
SpatialMLLM-4B46.326.133.434.618.0
VST-3B-SFT57.930.235.952.835.8
Cambrian-S-3B57.325.232.539.028.3
SenseNova-SI-1.1-InternVL3-2B 63.7 34.2 41.8 52.6 36.7
Open-source Models (~8B)
InternVL3-8B42.128.041.538.641.1
Qwen3-VL-8B-Instruct57.931.129.442.245.8
BAGEL-7B-MoT31.431.034.741.337.0
SpaceR-7B41.527.437.935.834.2
ViLaSR-7B44.630.235.135.738.7
VST-7B-SFT60.632.039.750.539.6
Cambrian-S-7B67.525.839.640.933.0
SenseNova-SI-1.1-InternVL3-8B 68.7 43.3 85.6 54.6 47.7
Proprietary Models
Gemini-2.5-pro-2025-0653.538.057.646.057.0
Grok-4-2025-07-0947.937.863.543.247.0
GPT-5-2025-08-0755.041.856.345.561.8
## 🛠️ QuickStart ### Installation We recommend using [uv](https://docs.astral.sh/uv/) to manage the environment. > uv installation guide: ```bash git clone git@github.com:OpenSenseNova/SenseNova-SI.git cd SenseNova-SI/ uv sync --extra cu124 # or one of [cu118|cu121|cu124|cu126|cu128|cu129], depending on your CUDA version uv sync source .venv/bin/activate ``` #### Hello World A simple image-free test to verify environment setup and download the model. ```bash python example.py \ --question "Hello" \ --model_path sensenova/SenseNova-SI-1.1-InternVL3-8B ``` ### Examples #### Example 1 This example is from the `Pos-Obj-Obj` subset of [MMSI-Bench](https://github.com/InternRobotics/MMSI-Bench): ```bash python example.py \ --image_paths examples/Q1_1.png examples/Q1_2.png \ --question " You are standing in front of the dice pattern and observing it. Where is the desk lamp approximately located relative to you? Options: A: 90 degrees counterclockwise, B: 90 degrees clockwise, C: 135 degrees counterclockwise, D: 135 degrees clockwise" \ --model_path sensenova/SenseNova-SI-1.1-InternVL3-8B # --model_path OpenGVLab/InternVL3-8B ```
Details of Example 1

Q: You are standing in front of the dice pattern and observing it. Where is the desk lamp approximately located relative to you? Options: A: 90 degrees counterclockwise, B: 90 degrees clockwise, C: 135 degrees counterclockwise, D: 135 degrees clockwise

First image Second image

GT: C

#### Example 2 This example is from the `Rotation` subset of [MindCube](https://mind-cube.github.io/): ```bash python example.py \ --image_paths examples/Q2_1.png examples/Q2_2.png \ --question " Based on these two views showing the same scene: in which direction did I move from the first view to the second view? A. Directly left B. Directly right C. Diagonally forward and right D. Diagonally forward and left" \ --model_path sensenova/SenseNova-SI-1.1-InternVL3-8B # --model_path OpenGVLab/InternVL3-8B ```
Details of Example 2

Q: Based on these two views showing the same scene: in which direction did I move from the first view to the second view? Directly left B. Directly right C. Diagonally forward and right D. Diagonally forward and left

First image Second image

GT: D

#### Test Multiple Questions in a Single Run Prepare a file similar to [examples/examples.jsonl](examples/examples.jsonl), where each line represents a single question. The model is loaded once and processes questions sequentially. The questions remain independent of each other. > For more details on the `jsonl` format, refer to the documentation for [Single-Image Data](https://internvl.readthedocs.io/en/latest/get_started/chat_data_format.html#single-image-data) and [Multi-Image Data](https://internvl.readthedocs.io/en/latest/get_started/chat_data_format.html#multi-image-data). ```bash python example.py \ --jsonl_path examples/examples.jsonl \ --model_path sensenova/SenseNova-SI-1.1-InternVL3-8B # --model_path OpenGVLab/InternVL3-8B ``` ### Evaluation To reproduce the benchmark results above, please refer to [EASI](https://github.com/EvolvingLMMs-Lab/EASI) to evaluate SenseNova-SI on mainstream spatial intelligence benchmarks. ## 🖊️ Citation ```bib @article{sensenova-si, title = {Scaling Spatial Intelligence with Multimodal Foundation Models}, author = {Cai, Zhongang and Wang, Ruisi and Gu, Chenyang and Pu, Fanyi and Xu, Junxiang and Wang, Yubo and Yin, Wanqi and Yang, Zhitao and Wei, Chen and Sun, Qingping and Zhou, Tongxi and Li, Jiaqi and Pang, Hui En and Qian, Oscar and Wei, Yukun and Lin, Zhiqian and Shi, Xuanke and Deng, Kewang and Han, Xiaoyang and Chen, Zukai and Fan, Xiangyu and Deng, Hanming and Lu, Lewei and Pan, Liang and Li, Bo and Liu, Ziwei and Wang, Quan and Lin, Dahua and Yang, Lei}, journal = {arXiv preprint arXiv:2511.13719}, year = {2025} } ```