metadata

base_model:
  - Ultralytics/YOLO11
pipeline_tag: object-detection
library_name: ultralytics
tags:
  - yolov11
  - ultralytics
  - yolo
  - vision
  - object-detection
  - pytorch
  - ui
datasets:
  - MacPaw/Screen2AX-Group
license: agpl-3.0

🔍 YOLOv11l — UI Groups Detection

This model is a fine-tuned version of Ultralytics/YOLO11, trained to detect UI groups (e.g., toolbars, tab groups) in macOS application screenshots.

It is part of the Screen2AX project, a research-driven effort to generate accessibility metadata for macOS applications using vision-based techniques.

🧠 Task Overview

Task: Object Detection
Target: macOS UI groups
Supported Label(s):
```
['AXGroup']
```

This model detects higher-level UI groupings that are commonly used to structure accessible interfaces (e.g., AXGroup, AXTabGroup, AXToolbar, etc.).

🗂 Dataset

Training data: MacPaw/Screen2AX-Group

🚀 How to Use

🔧 Install Dependencies

pip install huggingface_hub ultralytics

🧪 Load the Model and Run Predictions

from huggingface_hub import hf_hub_download
from ultralytics import YOLO

# Download the model from the Hugging Face Hub
model_path = hf_hub_download(
    repo_id="MacPaw/yolov11l-ui-groups-detection",
    filename="ui-groups-detection.pt"
)

# Load and run prediction
model = YOLO(model_path)
results = model.predict("/path/to/your/image")

# Visualize or process results
results[0].show()

📜 License

This model is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0), inherited from the original YOLOv11 base model.

🔗 Related Projects

✍️ Citation

If you use this model, please cite the Screen2AX paper:

@misc{muryn2025screen2axvisionbasedapproachautomatic,
      title={Screen2AX: Vision-Based Approach for Automatic macOS Accessibility Generation}, 
      author={Viktor Muryn and Marta Sumyk and Mariya Hirna and Sofiya Garkot and Maksym Shamrai},
      year={2025},
      eprint={2507.16704},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2507.16704}, 
}

🌐 MacPaw Research

Learn more at https://research.macpaw.com