The IQ4_XS_L quant is meant to be run with IK_Llama.cpp or the incoming EsoCroK.Cpp, because it uses a Q6_0 quant specific to IK_Llama.cpp
Here's EsoCrok, compatible with the llama.cpp mainline quants AND Q6_0
https://github.com/Nexesenex/croco.cpp/releases/tag/v1.97200_b6119_RMv1.14.9m
The usual Croco has not been updated (yet?) to support properly GLM 4.5 (or OpenAI GPT OSS)
- Downloads last month
- 80
Hardware compatibility
Log In
to view the estimation
4-bit
16-bit
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for NexesQuants/zai-org_GLM-4.5-Air-bf16-iMat-IKL-CQ-GGUF
Base model
zai-org/GLM-4.5-Air