HUME: Measuring the Human-Model Performance Gap in Text Embedding Task
Paper
• 2510.10062 • Published
• 10
HUME
The HUME benchmark is designed to evaluate the performance of text embedding models and humans on a comparable set of tasks. This captures areas wh...
-
-
mteb/HUMEEmotionClassification
Viewer • Updated • 16k • 25 -
mteb/HUMEToxicConversationsClassification
Viewer • Updated • 8.05k • 30 -
mteb/HUMETweetSentimentExtractionClassification
Viewer • Updated • 27.5k • 24 -
mteb/HUMEMultilingualSentimentClassification
Viewer • Updated • 15.2k • 12 -
mteb/mteb-human-arxiv-clustering
Viewer • Updated • 1 • 12 -
mteb/mteb-human-reddit-clustering
Viewer • Updated • 1 • 13 -
mteb/mteb-human-wikicities-clustering
Viewer • Updated • 1 • 14 -
mteb/mteb-human-sib200-clustering
Viewer • Updated • 10 • 15 -
mteb/HUMECore17InstructionReranking
Viewer • Updated • 320 • 45 -
mteb/HUMENews21InstructionReranking
Viewer • Updated • 496 • 28 -
mteb/HUMERobust04InstructionReranking
Viewer • Updated • 784 • 26 -
mteb/HUMEWikipediaRerankingMultilingual
Viewer • Updated • 1.44k • 33 -
mteb/mteb-human-sickr-sts
Viewer • Updated • 40 • 65 -
mteb/mteb-human-sts12-sts
Viewer • Updated • 50 • 63 -
mteb/mteb-human-stsbenchmark-sts
Viewer • Updated • 50 • 39 -
mteb/mteb-human-sts22-sts
Viewer • Updated • 240 • 44