OpenMuQ

/

MuQ-MuLan-large

like 7

Follow

MuQ 8

Hi :)
The model.safetensors variant fails to load and causes a torch error because of missing keys.
Here is an example from diffrhythm. (the code in diffrhythm hasn't changed. It simply pulls now the safetensors variant instead of pytorch_model.bin)

Traceback (most recent call last):
  File "G:\pinokio\api\diffrhythm-fork.git\app\app.py", line 33, in <module>
    cfm, tokenizer, muq, vae = prepare_model(device)
  File "G:\pinokio\api\diffrhythm-fork.git\app\diffrhythm\infer\infer_utils.py", line 34, in prepare_model
    muq = MuQMuLan.from_pretrained("OpenMuQ/MuQ-MuLan-large")
  File "G:\pinokio\api\diffrhythm-fork.git\app\env\lib\site-packages\muq\muq_mulan\muq_mulan.py", line 91, in from_pretrained
    return super().from_pretrained(*args, cache_dir=cache_dir, **kwargs)
  File "G:\pinokio\api\diffrhythm-fork.git\app\env\lib\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "G:\pinokio\api\diffrhythm-fork.git\app\env\lib\site-packages\huggingface_hub\hub_mixin.py", line 566, in from_pretrained
    instance = cls._from_pretrained(
  File "G:\pinokio\api\diffrhythm-fork.git\app\env\lib\site-packages\huggingface_hub\hub_mixin.py", line 807, in _from_pretrained
    return cls._load_as_safetensor(model, model_file, map_location, strict)
  File "G:\pinokio\api\diffrhythm-fork.git\app\env\lib\site-packages\huggingface_hub\hub_mixin.py", line 842, in _load_as_safetensor
    safetensors.torch.load_model(model, model_file, strict=strict, device=map_location)  # type: ignore [arg-type]
  File "G:\pinokio\api\diffrhythm-fork.git\app\env\lib\site-packages\safetensors\torch.py", line 284, in load_model
    assert torch_missing == missing, f"{torch_missing} != {missing}"
AssertionError: set() != {'mulan.audio.model.model.conformer.pos_conv_embed.conv.parametrizations.weight.original1', 'mulan.audio.model.model.conformer.pos_conv_embed.conv.parametrizations.weight.origi
nal0'}

After checking the keys of model.safetensors I figured out these keys are missing in deed.

Is it possible that there was a problem during conversion?
Will you upload a safetensors variant with the new keys in the near future or should we switch back to the bin model instead?
Or is there anything else we could do to fix that problem?