Upload GGUF quantizations of heretic model

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct-2507-heretic-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct-2507-heretic-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct-2507-heretic-f16.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,95 @@
+---
+license: apache-2.0
+base_model: p-e-w/Qwen3-4B-Instruct-2507-heretic
+tags:
+  - gguf
+  - quantized
+  - heretic
+  - abliterated
+  - uncensored
+model_type: qwen2
+---
+
+# Qwen3-4B-Instruct-2507-heretic-GGUF
+
+GGUF quantized versions of [p-e-w/Qwen3-4B-Instruct-2507-heretic](https://huggingface.co/p-e-w/Qwen3-4B-Instruct-2507-heretic) from "The Bestiary" collection.
+
+## Model Description
+
+This is a GGUF conversion of the Qwen3-4B-Instruct-2507-heretic model, which is an abliterated (uncensored) version of Alibaba's Qwen2.5 4B Instruct model. The model has had its refusal mechanisms removed, making it more willing to engage with any prompt.
+
+**Original Model:** [p-e-w/Qwen3-4B-Instruct-2507-heretic](https://huggingface.co/p-e-w/Qwen3-4B-Instruct-2507-heretic)
+**Collection:** [The Bestiary by p-e-w](https://huggingface.co/collections/p-e-w/the-bestiary)
+
+## Quantization Formats
+
+This repository contains 4 quantization levels:
+
+| File | Size | Description | Use Case |
+|------|------|-------------|----------|
+| `qwen3-4b-instruct-2507-heretic-f16.gguf` | 7.5GB | Full 16-bit precision | Best quality, highest memory usage |
+| `qwen3-4b-instruct-2507-heretic-Q8_0.gguf` | 4.0GB | 8-bit quantization | High quality, good balance |
+| `qwen3-4b-instruct-2507-heretic-Q5_K_M.gguf` | 2.7GB | 5-bit quantization | Balanced quality/size |
+| `qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf` | 2.4GB | 4-bit quantization | Smallest size, good quality |
+
+**Recommended:** `Q4_K_M` for most users (best balance of quality and size)
+
+## Usage
+
+### With Ollama
+
+1. Download the GGUF file you want to use
+2. Create a Modelfile:
+
+```
+FROM ./qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf
+
+TEMPLATE """{{ if .System }}<|im_start|>system<|im_sep|>{{ .System }}<|im_end|>{{ end }}{{ if .Prompt }}<|im_start|>user<|im_sep|>{{ .Prompt }}<|im_end|>{{ end }}<|im_start|>assistant<|im_sep|>{{ .Response }}<|im_end|>"""
+
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER num_ctx 8192
+```
+
+3. Import to Ollama:
+```bash
+ollama create qwen3-4b-heretic:Q4_K_M -f Modelfile
+```
+
+4. Run:
+```bash
+ollama run qwen3-4b-heretic:Q4_K_M
+```
+
+### With llama.cpp
+
+```bash
+./llama-cli -m qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf -p "Your prompt here" -n 512
+```
+
+### With Open WebUI
+
+Once imported to Ollama, the model will automatically appear in the Open WebUI model dropdown.
+
+## Conversion Details
+
+- **Converted using:** llama.cpp (latest)
+- **Conversion date:** 2025-11-21
+- **Base format:** FP16 GGUF
+- **Quantization method:** llama-quantize
+
+## Important Note
+
+This is an uncensored model with refusal mechanisms removed. Use responsibly and in accordance with applicable laws and regulations.
+
+## License
+
+Inherits the Apache 2.0 license from the base Qwen model.
+
+## Credits
+
+- **Original model:** Alibaba (Qwen2.5)
+- **Abliteration:** p-e-w ([The Bestiary](https://huggingface.co/collections/p-e-w/the-bestiary))
+- **GGUF conversion:** cybrown

qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf6aacb6e2bb0e7758555e8842c1bb57955a11aedeb7809ef16e3bf806a2306e
+size 2497279104

qwen3-4b-instruct-2507-heretic-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67126e09d99ae1ea1921889175ef6e37834b1a160c5b53949ac7e1b349ba7615
+size 2889512064

qwen3-4b-instruct-2507-heretic-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77f8f224eab65c11855b9144d2a09f437daa241df1214056249c1f919488b07b
+size 4280403584

qwen3-4b-instruct-2507-heretic-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2ba00143246d0671e87d17be84a9af28cdfbbcbbf641307fe191f9c0c8cee82
+size 8051283584