File size: 1,472 Bytes
2410073 45fa3a1 2410073 3192870 2410073 a69c5eb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
---
license: apache-2.0
---
# DeskVision: Large Scale Desktop Region Captioning for Advanced GUI Agents
[\[💻Code\]](https://sh-code.mthreads.com/liang.yang/mt-gui)[\[📝Paper\]](https://arxiv.org/abs/2503.11170) [\[🤗Models\]](https://huggingface.co/caca9527/GUIExplorer)[\[🤗Data\]](https://huggingface.co/datasets/caca9527/DeskVision)
<div align=center><img width="800" height="400" src="https://sh-code.mthreads.com/liang.yang/mt-gui/-/raw/master/assets/overview.png?ref_type=heads"/></div>
🔥🔥🔥 We have open-sourced our self-developed GUI multimodal visual understanding model GUIExplorer, which is based on the model architecture of LLaVA OneVision 7B. It has basic GUI visual understanding capabilities, including regional OCR, Grounding, and single-step instruction execution capabilities. For details on how to train and use this model, please refer to the [\[💻Code\]](https://sh-code.mthreads.com/liang.yang/mt-gui).
**Citation**
If you use GUIExplorer for your research, please cite our [\[📝Paper\]](https://arxiv.org/abs/2503.11170)：
```bibtex
@misc{xu2025deskvisionlargescaledesktop,
title={DeskVision: Large Scale Desktop Region Captioning for Advanced GUI Agents},
author={Yibin Xu and Liang Yang and Hao Chen and Hua Wang and Zhi Chen and Yaohua Tang},
year={2025},
eprint={2503.11170},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2503.11170},
}
``` |