submit.md

LLM Hallucination Detection Leaderboard Submission Guidelines

Thank you for your interest in contributing to the LLM Hallucination Detection Leaderboard! We welcome submissions from researchers and practitioners who have built or finetuned language models that can be evaluated on our hallucination benchmarks.

---

1. What to Send

Please email ryan@kluster.ai with the subject line:

[Verify Leaderboard Submission] <Your-Model-Name>

Attach one ZIP file that contains all of the following:

1. model_card.md: A short Markdown file describing your model:
   • Name and version
   • Architecture / base model
   • Training or finetuning procedure
   • License
   • Intended use & known limitations
   • Contact information
2. results.csv: A CSV file with one row per prompt and one column per field (see schema below).
3. (Optional) extra_notes.md: Anything else you would like us to know (e.g., additional analysis).

---

2. CSV Schema

Column	Description
request	The exact input request provided to the model. This must follow the same request structure and prompt format as described in Details section.
response	The raw output produced by the model.
verify_response	The Verify judgment or explanation regarding hallucination.
verify_label	The final boolean / categorical label (e.g., TRUE, FALSE).
task	The benchmark or dataset name the sample comes from.

Important: Use UTF-8 encoding and do not add additional columns without prior discussion; extra information should go in the metadata field. You must use Verify by kluster.ai to ensure fairness in the leaderboard.

---

3. Evaluation Datasets

Run your model on the following public datasets and include all examples in your CSV. You can load them directly from Hugging Face:

Dataset	Hugging Face Link
HaluEval QA (qa_samples subet with Question and Knowledge column)	https://huggingface.co/datasets/pminervini/HaluEval
UltraChat	https://huggingface.co/datasets/kluster-ai/ultrachat-sampled

---

5. Example Row

request,response,verify_response,verify_label,task
"What is the capital of the UK?","London is the capital of the UK.","The statement is factually correct.",TRUE,TruthfulQA

---

6. Review Process

1. We will sanity-check the file format and reproduce a random subset.
2. If everything looks good, your scores will appear on the public leaderboard.
3. We may reach out for clarifications, please keep an eye on your inbox.

---

7. Contact

Questions? Email ryan@kluster.ai or join our Discord here.

We look forward to your submissions and to advancing reliable language models together!