SGLang: ValueError: Unknown quantization method: auto-round.
#3
by
ubergarm
- opened
Just built sglang from tip of main and seems like auto-round is not supported? Added issue comment here on project github repo: https://github.com/sgl-project/sglang/issues/9653#issuecomment-3289822243
Maybe better chance with vLLM for this quant, but they are having issues too (and i can't get it to run on vLLM either). https://huggingface.co/Intel/Qwen3-Next-80B-A3B-Instruct-int4-mixed-AutoRound/discussions/1