qnaug commited on
Commit
7400720
·
verified ·
1 Parent(s): a756c6f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -18
README.md CHANGED
@@ -462,26 +462,27 @@ You can finetune this model on your own dataset.
462
 
463
  #### Information Retrieval
464
 
465
- * Datasets: `miriad-eval-1kq-31kd` and `clean-val-only`
466
  * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
467
 
468
- | Metric | miriad-eval-1kq-31kd | clean-val-only |
469
- |:--------------------|:---------------------|:---------------|
470
- | cosine_accuracy@1 | 0.6595 | 0.8471 |
471
- | cosine_accuracy@3 | 0.8518 | 0.9328 |
472
- | cosine_accuracy@5 | 0.8865 | 0.9544 |
473
- | cosine_accuracy@10 | 0.9221 | 0.9772 |
474
- | cosine_precision@1 | 0.6595 | 0.8471 |
475
- | cosine_precision@3 | 0.2839 | 0.3109 |
476
- | cosine_precision@5 | 0.1773 | 0.1909 |
477
- | cosine_precision@10 | 0.0922 | 0.0977 |
478
- | cosine_recall@1 | 0.6595 | 0.8471 |
479
- | cosine_recall@3 | 0.8518 | 0.9328 |
480
- | cosine_recall@5 | 0.8865 | 0.9544 |
481
- | cosine_recall@10 | 0.9221 | 0.9772 |
482
- | **cosine_ndcg@10** | **0.8011** | **0.9151** |
483
- | cosine_mrr@10 | 0.7612 | 0.8949 |
484
- | cosine_map@100 | 0.7639 | 0.8957 |
 
485
 
486
  <!--
487
  ## Bias, Risks and Limitations
 
462
 
463
  #### Information Retrieval
464
 
465
+ * Datasets: `clean-val-only`
466
  * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
467
 
468
+ | Metric | Base model (clean) | Fine-tuned (clean) |
469
+ |:--------------------|:---------------------|:-------------------|
470
+ | cosine_accuracy@1 | 0.4425 | 0.8471 |
471
+ | cosine_accuracy@3 | 0.6182 | 0.9328 |
472
+ | cosine_accuracy@5 | 0.6779 | 0.9544 |
473
+ | cosine_accuracy@10 | 0.7668 | 0.9772 |
474
+ | cosine_precision@1 | 0.4425 | 0.8471 |
475
+ | cosine_precision@3 | 0.2061 | 0.3109 |
476
+ | cosine_precision@5 | 0.1356 | 0.1909 |
477
+ | cosine_precision@10 | 0.0767 | 0.0977 |
478
+ | cosine_recall@1 | 0.4425 | 0.8471 |
479
+ | cosine_recall@3 | 0.6182 | 0.9328 |
480
+ | cosine_recall@5 | 0.6779 | 0.9544 |
481
+ | cosine_recall@10 | 0.7668 | 0.9772 |
482
+ | **cosine_ndcg@10** | **0.5994** | **0.9151** |
483
+ | cosine_mrr@10 | 0.5465 | 0.8949 |
484
+ | cosine_map@100 | 0.5533 | 0.8957 |
485
+
486
 
487
  <!--
488
  ## Bias, Risks and Limitations