README.md

---
license: mit
---

### bitnet_b1_58-3B-Coder

Code finetuned version of [bitnet_b1_58-3B](https://huggingface.co/1bitLLM/bitnet_b1_58-3B)

### Usage

```python
from tokenization_bitnet import BitnetTokenizer
from transformers import AutoModelForCausalLM
import torch
PROMPT = """### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
tokenizer = BitnetTokenizer.from_pretrained(
    "TechxGenus/bitnet_b1_58-3B-Coder",
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
    "TechxGenus/bitnet_b1_58-3B-Coder",
    torch_dtype=torch.float16,
    device_map="auto",
)
inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=2048)
print(tokenizer.decode(outputs[0]))
```

### Note

Model may sometimes make errors, produce misleading contents, or struggle to manage tasks that are not related to coding. It has undergone very limited testing. Additional safety testing should be performed before any real-world deployments.