lm-evaluation-harness/lm_eval/tasks/pile_10k/README.md

Pile-10k

Paper

Title: NeelNanda/pile-10k

Abstract: The first 10K elements of The Pile, useful for debugging models trained on it. See the HuggingFace page for the full Pile for more info. Inspired by stas' great resource doing the same for OpenWebText

Homepage: https://huggingface.co/datasets/NeelNanda/pile-10k

Citation

@misc{Nanda2022Pile10K,
  author = {Nanda, Neel},
  title = {{NeelNanda/pile-10k} \textendash\ Datasets at Hugging Face},
  year = {2022},
  howpublished = {\url{https://huggingface.co/datasets/NeelNanda/pile-10k}},
}

Groups and Tasks

Groups

• Not part of a group yet.

Tasks

• pile_10k: The first 10K elements of The Pile, useful for debugging models trained on it.

Checklist

For adding novel benchmarks/datasets to the library:

• Is the task an existing benchmark in the literature?
  • Have you referenced the original paper that introduced the task?
  • If yes, does the original paper provide a reference implementation? If so, have you checked against the reference implementation and documented how to run such a test?

If other tasks on this dataset are already supported:

• Is the "Main" variant of this task clearly denoted?
• Have you provided a short sentence in a README on what each new variant adds / evaluates?
• Have you noted which, if any, published evaluation setups are matched by this variant?