cpatonn

/

Qwen3-Next-80B-A3B-Instruct-AWQ-4bit

like 34

The fix for that problem is merged to vllm 2-3 days ago. Please install vllm latest nightly build using:

pip install -U vllm \
    --pre \
    --extra-index-url https://wheels.vllm.ai/nightly

Edit: I'm sorry, as the merged fix is not in nightly build yet. Please install vllm from source.

I updated your model and then it stopped working. I updated to the latest nightly already. The original model worked although it had a few issues. But I am at a loss on how to get this one to work. I will start fresh in a new env and see what is going on.

The fix for that problem is merged to vllm 2-3 days ago. Please install vllm latest nightly build using:

pip install -U vllm \
    --pre \
    --extra-index-url https://wheels.vllm.ai/nightly

Edit: I'm sorry, as the merged fix is not in nightly build yet. Please install vllm from source.

OK. Thanks for the update.