https://huggingface.co/PerceptronAI/Isaac-0.1
#1396
by
davidxifeng
- opened
could you please quantize this model?
IsaacForConditionalGeneration is unfortunately not currently supported by llama.cpp and so the model can't currently be converted into a GGUF. Luckily the model is so small in size that it should easily be able to run on most computers. Just keep in mind that they stored the original model in F32 so just load it in F16 and it will probably fit as it is just a 3B model.
Thank you very much for the explanation!
davidxifeng
changed discussion status to
closed