README.md · TechxGenus/bitnet_b1

metadata

license: mit

bitnet_b1_58-3B-Coder

Code finetuned version of bitnet_b1_58-3B

Usage

from tokenization_bitnet import BitnetTokenizer
from transformers import AutoModelForCausalLM
import torch
PROMPT = """### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
tokenizer = BitnetTokenizer.from_pretrained(
    "TechxGenus/bitnet_b1_58-3B-Coder",
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
    "TechxGenus/bitnet_b1_58-3B-Coder",
    torch_dtype=torch.float16,
    device_map="auto",
)
inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=2048)
print(tokenizer.decode(outputs[0]))

Note

Model may sometimes make errors, produce misleading contents, or struggle to manage tasks that are not related to coding. It has undergone very limited testing. Additional safety testing should be performed before any real-world deployments.