deployment issue on hugging face endpoint inference
#26
by
Ideaentity21
- opened
Hi there,
i tried to deploy this model on hugging face endpoint inference and got this error:
RuntimeError: The size of tensor a (48) must match the size of tensor b (64) at non-singleton dimension 0
Instance Types Tested: Nvidia A100 80GB (single GPU) AND 2x Nvidia A100 80GB (multi-GPU)
how to resolve this issue?
looks like a more active discussion here: https://github.com/huggingface/text-generation-inference/issues/3071