Update README.md
Browse files
README.md
CHANGED
@@ -76,22 +76,11 @@ The training corpus for OpenCodeReasoning-CPP-Nemotron-32B is [OpenCodeReasoning
|
|
76 |
|
77 |
|
78 |
## Evaluation Dataset
|
79 |
-
We used the datasets
|
80 |
* Data Collection Method: Hybrid: Automated, Human, Synthetic <br>
|
81 |
* Data Labeling Method: Hybrid: Automated, Human, Synthetic <br>
|
82 |
|
83 |
|
84 |
-
### [LiveCodeBench](https://huggingface.co/datasets/livecodebench/code_generation_lite)
|
85 |
-
| Easy | Medium | Hard | Avg. |
|
86 |
-
|:------|:------|:------|:-----|
|
87 |
-
| 98.4 | 77.2 | 30.4 | 61.7 |
|
88 |
-
|
89 |
-
### [CodeContests](https://huggingface.co/datasets/deepmind/code_contests)
|
90 |
-
| Public | Private | Generated | All |
|
91 |
-
|:--------|:--------|:----------|:----|
|
92 |
-
| 60.3 | 36.6 | 42.7 | 24.4|
|
93 |
-
|
94 |
-
|
95 |
## Inference
|
96 |
- **Engine:** vLLM <br>
|
97 |
- **Test Hardware** NVIDIA H100-80GB <br>
|
|
|
76 |
|
77 |
|
78 |
## Evaluation Dataset
|
79 |
+
We used the [IOI benchmark](https://huggingface.co/datasets/open-r1/ioi) to evaluate OpenCodeReasoning-CPP-Nemotron-32B. <br>
|
80 |
* Data Collection Method: Hybrid: Automated, Human, Synthetic <br>
|
81 |
* Data Labeling Method: Hybrid: Automated, Human, Synthetic <br>
|
82 |
|
83 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
84 |
## Inference
|
85 |
- **Engine:** vLLM <br>
|
86 |
- **Test Hardware** NVIDIA H100-80GB <br>
|