|
|
---
|
|
|
title: RLHF Pairwise Annotation Demo
|
|
|
emoji: 🎯
|
|
|
colorFrom: blue
|
|
|
colorTo: green
|
|
|
sdk: gradio
|
|
|
sdk_version: 4.44.0
|
|
|
app_file: app.py
|
|
|
pinned: false
|
|
|
license: mit
|
|
|
datasets:
|
|
|
- openbmb/UltraFeedback
|
|
|
---
|
|
|
|
|
|
# 🎯 AI Alignment: Binary Preference Annotation
|
|
|
|
|
|
This app simulates the data annotation process used in RLHF (Reinforcement Learning from Human Feedback) training. Users compare two AI completions and select which one is better.
|
|
|
|
|
|
## How it works
|
|
|
|
|
|
1. The app loads random examples from the UltraFeedback dataset
|
|
|
2. Users see a prompt and two AI completions
|
|
|
3. Users select which completion is better or skip if unsure
|
|
|
4. All annotations are saved to a public dataset for research purposes
|
|
|
|
