prompterminal/gpt2-compressed

GPT-2 XL Compressed Model

This is a compressed version of GPT-2 XL using the CompactifAI methodology - a novel compression technique based on quantum-inspired Tensor Networks.

🎯 Key Achievements

• 68.3% compression (1.56B → 494M parameters)
• 140 layers compressed using Matrix Product Operators (MPO)
• Knowledge distillation healing applied for quality recovery
• Functional model with maintained generation capabilities

📊 Compression Results

Metric	Original	Compressed	Reduction
Parameters	1.56B	494M	68.3%
Model Size	~6.2GB	~2.0GB	67.7%
Layers Compressed	0	140	-

🔬 Methodology

This model implements the CompactifAI approach from the paper: "CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks"

Compression Process:

1. Tensor Network Decomposition: Weight matrices decomposed into Matrix Product Operators (MPO)
2. Strategic Layer Selection: 140 layers across attention and MLP components
3. Bond Dimension Control: Controlled truncation via bond dimensions
4. Knowledge Distillation: Healing process to recover generation quality

🚀 Usage

from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Load the compressed model
model = GPT2LMHeadModel.from_pretrained("your-username/gpt2-compressed")
tokenizer = GPT2Tokenizer.from_pretrained("gpt2-xl")

# Generate text
input_text = "The future of artificial intelligence"
inputs = tokenizer.encode(input_text, return_tensors="pt")
outputs = model.generate(inputs, max_length=50, do_sample=True)
print(tokenizer.decode(outputs[0]))

📈 Performance

• Generation Quality: Maintained coherent text generation after healing
• Inference Speed: Improved due to reduced parameter count
• Memory Efficiency: 68.3% reduction in memory requirements

🔧 Technical Details

• Base Model: GPT-2 XL (1.56B parameters)
• Compression Method: Matrix Product Operator (MPO) decomposition
• Bond Dimensions: Varied per layer (16-64 range)
• Healing: Knowledge distillation from teacher model
• Framework: PyTorch + Transformers

🎓 Research Impact

This model represents:

• First open-source implementation of CompactifAI methodology
• Validation of tensor network compression at billion-parameter scale
• Proof-of-concept for edge deployment of large language models
• Foundation for democratizing LLM access

📚 Citation

If you use this model, please cite:

@article{compactifai2024,
  title={CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks},
  author={Tomut, Andrei and Jahromi, Saeed S. and others},
  journal={arXiv preprint arXiv:2401.14109},
  year={2024}
}

⚠️ Limitations

• Model may show some quality degradation compared to original GPT-2 XL
• Specific domains may be more affected than others
• Further fine-tuning may be needed for specialized applications

🤝 Contributing

This model is part of ongoing research into efficient LLM compression. Feedback and improvements are welcome!

---

This model was created using the CompactifAI methodology and represents a significant step forward in making large language models more accessible and efficient.