"RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x256 and 3072x256)"
I'm using a quantized t5 (Q4_K_M) from https://huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/tree/main
It works with dev, schnell, and other finetunes such as shuttle (quantized). But it will not work with chroma-unlocked-v24-Q4_K_M. Is this a known thing?
I can't test the full model since I only have 8GB of VRAM.
@popeyed This isn't Flux anymore. If you are using ComfyUI, go to the main repo for this model for instructions or wait a little for the implementation to ComfyUI core repo
I am having success (slowly) running the version 27 O4 KM, and Q5 models in Forge with my old pathetic P.C. (Core i5-3450 CPU @ 3.10GHz, 16.0 GB RAM, 8 GB GTX 1070FE) using "ForgeChroma".
However, if I try to run the newer version 29.5 models with the same quants which are actually smaller than the version 27 models and should be easier for my geriatric P.C. to run, I get:
"RuntimeError: mat1 and mat2 shapes cannot be multiplied (1024x64 and 128x3072)
mat1 and mat2 shapes cannot be multiplied (1024x64 and 128x3072)".
Is this a flaw or change in the newer models? Or do you believe it's the fault of the software I'm using (ForgeChroma)? Or is it simply because my old P.C. just can't handle them?
Any help/advice would be greatly appreciated! Thanks!
Linking below to the software in case you are unfamiliar.
That is because Forge code is old and don't keep stuff up to date. GGUF model changed in order to reduce size and improve other things. Did you try updating forgeChroma before posting this message?
Just finished talking with ForgeChroma maker and it's fixed in his latest update, I was not wanting to reinstall the mod (just about requires reinstalling Forge) so he showed which lines of code to edit. Super nice guy!