---
license: apache-2.0
base_model: unsloth/Meta-Llama-3.1-8B-Instruct
tags:
- diffusion
- language-model
- llama
- text-generation
library_name: transformers
pipeline_tag: text-generation
---

# Llama-3.1-8B Diffusion Model (LAD)

This is a **Language Autoregressive Diffusion (LAD)** model based on Llama-3.1-8B-Instruct.

## Features
- 🎯 Dual mode: Autoregressive + Diffusion generation
- 🚀 Cosine noise schedule with 1000 timesteps  
- 🧠 LoRA fine-tuning (rank 32)
- ⚡ Custom diffusion components

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("rootxhacker/llama3-diffusion")
tokenizer = AutoTokenizer.from_pretrained("rootxhacker/llama3-diffusion")

# Generate text
inputs = tokenizer("The future of AI", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0]))
```

## Training Details
- Base: Meta-Llama-3.1-8B-Instruct
- Dataset: PatrickHaller/cosmopedia-v2-1B
- Framework: Unsloth + Custom Diffusion
- Context: 256 tokens
- Training: 60% AR + 40% Diffusion

Uploaded: 2025-06-08 23:13