A collection of GGUF for Illustrious models just to help low memory people and do some experiences:

What I noticed is that Q6 or lower, will increse the s/it to double, not sure why...

If you have a specific modelt hat you want the conversion, you can ask that I'll try to do it

===================================================================================

if you would like to help me, it seems that runpod has a Refer thing - https://runpod.io?ref=d2452mau

You get	I get
- A one-time credit of $5 when they sign up with your link and adds $10 for the first time - Instant access to Runpod's GPU resources	- A one-time credit of $5 when a user signs up with your link and adds $10 for the first time - Credits on referred user spend during their first 6 months. (5% Serverless and 3% Pods)

GGUF

Model size

3B params

Architecture

sdxl

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including BigDannyPt/Illustrious-GGUF-Collection