unsloth

/

GLM-4.6-GGUF

like 146

Follow

Unsloth AI 12k

We did a refresh of quants (quality of life updates) for GLM 4.5, 4.6 and 4.7

llama.cpp and other inference engines like LM Studio now support more features including but not limited to:

1. Non ascii decoding for tools (affects non English languages) For eg before the default (ensure_ascii=True) would cause "café" → "caf\u00e9", whilst now ensure_ascii=False would tokenize "café" → "café". I would re-download our quants if you use languages other than English.
2. Converts reasoning content parsing to original [0], [-1] from our changes of |first and |last. We used to change [0] to |first and [-1] to |last so we be compatible with LM Studio and llama-cli. With the upgrade of llama-cli to use llama-server, we can revert this. llama-server also didn't like |first, so we fixed it as well.

Also other changes:

1. (Ongoing process) Will add Ollama model files, so Ollama would function.
2. Added lot of tool calls in our calibration dataset - makes tool calling better, especially for smaller quants.
3. A bit more calibration data for GLM models., adding a teeny tiny bit more accurancy overall.

GGUFs which will be receive Quality of Life updates:
https://huggingface.co/unsloth/GLM-4.6-GGUF
https://huggingface.co/unsloth/GLM-4.5-GGUF
https://huggingface.co/unsloth/GLM-4.5-Air-GGUF
https://huggingface.co/unsloth/GLM-4.6V-GGUF
https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF
https://huggingface.co/unsloth/GLM-4.7-GGUF