leaderboard_swe.csv

Rank,Model,Joint Accuracy,Categorical F1,Location Accuracy,Date
1,Gemini-2.5-Pro-Preview-05-06,0.050,0.148,0.238,2025-05-14
2,Gemini-2.5-Flash-Preview-04-17,0.000,0.213,0.060,2025-05-14
3,Llama-4-Maverick-17B-128E-Instruct,0.000,0.191,0.083,2025-05-14
4,GPT-4.1,0.000,0.166,0.000,2025-05-14
5,Llama-4-Scout-17B-16E-Instruct,0.000,0.050,0.000,2025-05-14
6,Open AI o1,CLE,CLE,CLE,2025-05-14
7,Open AI o3,CLE,CLE,CLE,2025-05-14
8,Anthropic Claude-3.7-Sonnet,CLE,CLE,CLE,2025-05-14