README.md

---
language:
  - en
license: apache-2.0
tags:
  - granite
  - client-simulation
  - dialogue
  - bitsandbytes
  - 4-bit
  - unsloth
  - transformers
base_model: ibm-granite/granite-3.2-2b-instruct
pipeline_tag: text-generation
datasets:
  - merged_mental_health_dataset.jsonl
library_name: transformers
---
# Gradiant-ClientSim-v0.1

A 4-bit quantized client simulation model based on IBM Granite 3.2B, fine-tuned for client interaction and simulation tasks. This model is compatible with Huggingface Transformers and bitsandbytes for efficient inference.

## Model Details
- **Base Model:** IBM Granite 3.2B (Unsloth)
- **Precision:** 4-bit (safetensors, bitsandbytes)
- **Architecture:** Causal Language Model
- **Tokenizer:** Included (BPE)
- **Intended Use:** Client simulation, dialogue, and assistant tasks

## Files Included
- `model.safetensors` — Main model weights (4-bit)
- `config.json` — Model configuration
- `generation_config.json` — Generation parameters
- `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`, `special_tokens_map.json`, `added_tokens.json` — Tokenizer files

## Example Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model_id = "oneblackmage/Gradiant-ClientSim-v0.1"
bnb_config = BitsAndBytesConfig(load_in_4bit=True)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

prompt = "<|user>How can I improve my focus at work?\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Quantization
- This model is stored in 4-bit precision using [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) for efficient inference on modern GPUs.
- For best performance, use with `transformers` >= 4.45 and `bitsandbytes` >= 0.43.

## License
- See the LICENSE file or Huggingface model card for details.

## Citation
If you use this model, please cite the original IBM Granite model and this fine-tuned version.

---

For questions or issues, open an issue on the Huggingface repo or contact the maintainer.