Quantization Details
- Quantization Method: AWQ
- Bits: 4
- Group Size: 128
- Quantization Tool: ms-swift
Memory Usage
| Type | GUI-Owl-7B | GUI-Owl-7B-AWQ-4bit |
|---|---|---|
| Memory Size | 16G GB | 6.5 GB |
Inference
Run with:
vllm serve JessieWei/GUI-Owl-7B-AWQ-4bit
Detailed parameter tuning refer to GUI-Owl-7B.
- Downloads last month
- 10
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support