My GGUF-IQ-ARM-Imatrix quants for inflatebot/MN-12B-Mag-Mell-R1.

Mag Mell R1 was tested with Temp 1.25 and MinP 0.2. This was fairly stable up to 10K, but this might be too "hot". If issues with coherency occur, try increasing MinP or decreasing Temperature.

Use the ChatML prompt format.

Downloads last month: 313

GGUF

Model size

12B params

Architecture

llama

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Base model

inflatebot/MN-12B-Mag-Mell-R1

Quantized

(28)

this model

Collection including Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Quantized Models (GGUF, IQ, Imatrix)

Collection

Various GGUF quantizations of small models. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 98 items • Updated Sep 2 • 67