---
title: Token Attention Visualizer
emoji: 🔍
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.42.0
app_file: app.py
pinned: false
license: apache-2.0
---

# Token Attention Visualizer

An interactive tool for visualizing attention patterns in Large Language Models during text generation.

## Features

- 🚀 **Real-time Generation**: Generate text with any Hugging Face model
- 🔍 **Attention Visualization**: Explore attention patterns with clear visual representations
- 📊 **Dual Normalization**: Choose between separate or joint attention normalization
- ⚡ **Smart Caching**: Fast response with intelligent result caching
- 🎯 **Token Selection**: Use dropdown menus to select and filter token connections
- 📈 **Step Navigation**: Navigate through generation steps
- 🎨 **Customizable Threshold**: Filter weak attention connections

## How It Works

The visualizer shows how tokens attend to each other during text generation:
- **Blue lines**: Attention from input tokens to output tokens
- **Orange curves**: Attention between output tokens
- **Line thickness**: Represents attention weight strength

## Usage

1. **Load a Model**: Enter a Hugging Face model name (default: HuggingFaceTB/SmolLM-135M-Instruct)
2. **Enter Prompt**: Type your input text
3. **Configure Settings**: Adjust max tokens, temperature, and normalization
4. **Generate**: Click to generate text and visualize attention
5. **Explore**: Use dropdown menus to select tokens and view their attention patterns

## Technical Details

- Built with Gradio for the interface
- Visualization system with dropdown-based token selection
- Supports any Hugging Face causal language model
- Optimized for smaller models like SmolLM for efficient deployment
- Implements efficient attention processing and caching

## Local Development

```bash
# Clone the repository
git clone <repo-url>
cd token-attention-viz

# Install dependencies
pip install -r requirements.txt

# Run the app
python app.py
```

## Deployment

This app is designed for easy deployment on Hugging Face Spaces. Simply:
1. Create a new Space
2. Upload the project files
3. The app will automatically start

## Requirements

- Python 3.8+
- 4GB+ RAM (SmolLM models are lightweight)
- GPU acceleration optional (works well on CPU)

## License

Apache 2.0