Works with vllm? Any recommendations or howtos?
#1
by
DrRos
- opened
Does it? Will it work on 48gb (2*24) vram?
Which cards? I've been having trouble emulating NVFP4 on my 2x3090 setup so you may run into issues there. AWQ is more compatible if you aren't on Blackwell IMO.
A5000's - suppose there will be problems as well