Error "Unable to load weights from pytorch checkpoint file for \path\to\pytorch_model.bin" when using model_zoo_importer.py
Hi,
When I use the conversion tool model_zoo_importer.py from Deep Java Library on this model I get the error:
Failed to load model: elastic/multilingual-e5-small-optimized. Unable to load weights from pytorch checkpoint file for '/path/to/pytorch_model.bin' at 'path/to/pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
And also other exceptions.
I can convert other HF sentence similarity models without issues (also intfloat/multilingual-e5-small). If I Google on this error some responses mention that files may be corrupt, but when I compare the locally cached files with the ones I download from "Files and versions" I see no binary differences. I tried both under Ubuntu 20.04 (WSL2) and Windows 11 (errors differ somewhat). Does anyone has any idea how to resolve this issue?
This is the complete output of the conversion:
maarten@XPS159520:~/projects/djl/extensions/tokenizers$ python3 src/main/python/model_zoo_importer.py -m elastic/multilingual-e5-small-optimized
config.json: 100%|██████████████████████████████████████████████████████████████████████| 672/672 [00:00<00:00, 169kB/s]
Loading model: elastic/multilingual-e5-small-optimized ...
tokenizer_config.json: 100%|████████████████████████████████████████████████████████████| 443/443 [00:00<00:00, 115kB/s]
sentencepiece.bpe.model: 100%|█████████████████████████████████████████████████████| 5.07M/5.07M [00:00<00:00, 74.8MB/s]
tokenizer.json: 100%|███████████████████████████████████████████████████████████████| 17.1M/17.1M [00:00<00:00, 112MB/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████████| 167/167 [00:00<00:00, 66.3kB/s]
pytorch_model.bin: 100%|██████████████████████████████████████████████████████████████| 412M/412M [00:03<00:00, 116MB/s]
Failed to load model: elastic/multilingual-e5-small-optimized.
Unable to load weights from pytorch checkpoint file for '/home/maarten/.cache/huggingface/hub/models--elastic--multilingual-e5-small-optimized/snapshots/2612fa238fbd4d6348b6aef85906b3d3d3a8fec3/pytorch_model.bin' at '/home/maarten/.cache/huggingface/hub/models--elastic--multilingual-e5-small-optimized/snapshots/2612fa238fbd4d6348b6aef85906b3d3d3a8fec3/pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
Traceback (most recent call last):
File "/home/maarten/.local/lib/python3.8/site-packages/transformers/modeling_utils.py", line 531, in load_state_dict
return torch.load(
File "/home/maarten/.local/lib/python3.8/site-packages/torch/serialization.py", line 1025, in load
raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
_pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported class torch.qint8
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/maarten/.local/lib/python3.8/site-packages/transformers/modeling_utils.py", line 540, in load_state_dict
if f.read(7) == "version":
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 128: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/maarten/projects/djl/extensions/tokenizers/src/main/python/huggingface_converter.py", line 45, in save_model
hf_pipeline = self.load_model(model_id)
File "/home/maarten/projects/djl/extensions/tokenizers/src/main/python/sentence_similarity_converter.py", line 45, in load_model
model = AutoModel.from_pretrained(model_id)
File "/home/maarten/.local/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
return model_class.from_pretrained(
File "/home/maarten/.local/lib/python3.8/site-packages/transformers/modeling_utils.py", line 3525, in from_pretrained
state_dict = load_state_dict(resolved_archive_file)
File "/home/maarten/.local/lib/python3.8/site-packages/transformers/modeling_utils.py", line 552, in load_state_dict
raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for '/home/maarten/.cache/huggingface/hub/models--elastic--multilingual-e5-small-optimized/snapshots/2612fa238fbd4d6348b6aef85906b3d3d3a8fec3/pytorch_model.bin' at '/home/maarten/.cache/huggingface/hub/models--elastic--multilingual-e5-small-optimized/snapshots/2612fa238fbd4d6348b6aef85906b3d3d3a8fec3/pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
elastic/multilingual-e5-small-optimized: Failed to load model
finished.
Interesting, it looks like there's a problem loading int8 types.Unsupported class torch.qint8
Thanks for reporting @mkroon68 . We'll have a look.
Hi Josh, great, thanks!
I also experience this error, on torch==2.7.1 and transformers==4.53.1 .
We are currently only providing support for this model within Elasticsearch itself. Unfortunately we can't validate support for all torch and transformer versions.