Works for all siglip2 versions except naflex. Why?
#27
by
pySilver
- opened
Hi! I'm trying to create ONNX version of this model: google/siglip2-so400m-patch16-naflex and it simply fails with the following error:
Conversion failed: Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1092, in from_pretrained config_class = CONFIG_MAPPING[config_dict["model_type"]] File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 794, in getitem raise KeyError(key) KeyError: 'siglip2'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/home/user/app/transformers.js/scripts/convert.py", line 456, in <module> main() File "/home/user/app/transformers.js/scripts/convert.py", line 211, in main config = AutoConfig.from_pretrained(model_id, **from_pretrained_kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1094, in from_pretrained raise ValueError( ValueError: The checkpoint you are trying to load has model type siglip2 but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
You can update Transformers with the command pip install --upgrade transformers. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command pip install git+https://github.com/huggingface/transformers.git
Other siglip2 family models converting just fine.
The reason seems to be that other siglip2 family models (e.g. google/siglip2-so400m-patch14-384) had "model_type": "siglip" in their config [1], while google/siglip2-so400m-patch16-naflex has "model_type": "siglip2" [2]. Checking transformers.js code [3][4] I see not reference to the siglip2 model type.
What you could do is to open a feature request on https://github.com/huggingface/transformers.js repository, requesting the addition of siglip2 on their list of supported models.