Hugging Face's logo

faodl
/
model_cca_multilabel_MiniLM-L12-70prop-data-augmented

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Model card Files
xet
Community

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/model_cca_multilabel_MiniLM-L12-70prop-data-augmented")
# Run inference
preds = model("Strengthen macro-fiscal resilience through risk-informed public investment planning, including scenario-based budgeting and contingent financing arrangements.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 69.0403 951

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (2, 2)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 20
  • body_learning_rate: (2e-05, 2e-05)
  • head_learning_rate: 2e-05
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • l2_weight: 0.01
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0001 1 0.2247 -
0.0065 50 0.2105 -
0.0130 100 0.1984 -
0.0195 150 0.1899 -
0.0260 200 0.1916 -
0.0325 250 0.1769 -
0.0390 300 0.1679 -
0.0455 350 0.1677 -
0.0520 400 0.1591 -
0.0585 450 0.1521 -
0.0650 500 0.1522 -
0.0715 550 0.1497 -
0.0780 600 0.1494 -
0.0845 650 0.1457 -
0.0910 700 0.1503 -
0.0975 750 0.1328 -
0.1040 800 0.1251 -
0.1105 850 0.1395 -
0.1170 900 0.1298 -
0.1235 950 0.1221 -
0.1300 1000 0.1313 -
0.1365 1050 0.1267 -
0.1429 1100 0.1367 -
0.1494 1150 0.1324 -
0.1559 1200 0.1201 -
0.1624 1250 0.1244 -
0.1689 1300 0.1231 -
0.1754 1350 0.1214 -
0.1819 1400 0.1098 -
0.1884 1450 0.1152 -
0.1949 1500 0.1149 -
0.2014 1550 0.1185 -
0.2079 1600 0.1123 -
0.2144 1650 0.1092 -
0.2209 1700 0.1097 -
0.2274 1750 0.1159 -
0.2339 1800 0.1076 -
0.2404 1850 0.114 -
0.2469 1900 0.1055 -
0.2534 1950 0.1033 -
0.2599 2000 0.1016 -
0.2664 2050 0.1004 -
0.2729 2100 0.0973 -
0.2794 2150 0.1051 -
0.2859 2200 0.0954 -
0.2924 2250 0.0998 -
0.2989 2300 0.0984 -
0.3054 2350 0.0906 -
0.3119 2400 0.0939 -
0.3184 2450 0.1023 -
0.3249 2500 0.0983 -
0.3314 2550 0.0952 -
0.3379 2600 0.099 -
0.3444 2650 0.0994 -
0.3509 2700 0.0975 -
0.3574 2750 0.0871 -
0.3639 2800 0.0969 -
0.3704 2850 0.0845 -
0.3769 2900 0.1007 -
0.3834 2950 0.0887 -
0.3899 3000 0.0807 -
0.3964 3050 0.0859 -
0.4029 3100 0.0826 -
0.4094 3150 0.0784 -
0.4159 3200 0.0851 -
0.4224 3250 0.0834 -
0.4288 3300 0.0922 -
0.4353 3350 0.0862 -
0.4418 3400 0.0856 -
0.4483 3450 0.0848 -
0.4548 3500 0.0735 -
0.4613 3550 0.0752 -
0.4678 3600 0.0881 -
0.4743 3650 0.0836 -
0.4808 3700 0.0808 -
0.4873 3750 0.0963 -
0.4938 3800 0.0816 -
0.5003 3850 0.0809 -
0.5068 3900 0.0833 -
0.5133 3950 0.0852 -
0.5198 4000 0.0788 -
0.5263 4050 0.0742 -
0.5328 4100 0.0693 -
0.5393 4150 0.0856 -
0.5458 4200 0.072 -
0.5523 4250 0.0805 -
0.5588 4300 0.0741 -
0.5653 4350 0.0845 -
0.5718 4400 0.0753 -
0.5783 4450 0.0814 -
0.5848 4500 0.0691 -
0.5913 4550 0.0823 -
0.5978 4600 0.0847 -
0.6043 4650 0.0714 -
0.6108 4700 0.0879 -
0.6173 4750 0.0711 -
0.6238 4800 0.0697 -
0.6303 4850 0.0741 -
0.6368 4900 0.0771 -
0.6433 4950 0.0837 -
0.6498 5000 0.0743 -
0.6563 5050 0.0755 -
0.6628 5100 0.0739 -
0.6693 5150 0.0816 -
0.6758 5200 0.0782 -
0.6823 5250 0.0755 -
0.6888 5300 0.0712 -
0.6953 5350 0.0639 -
0.7018 5400 0.0694 -
0.7083 5450 0.0806 -
0.7147 5500 0.071 -
0.7212 5550 0.0707 -
0.7277 5600 0.0751 -
0.7342 5650 0.0724 -
0.7407 5700 0.0688 -
0.7472 5750 0.067 -
0.7537 5800 0.0718 -
0.7602 5850 0.0681 -
0.7667 5900 0.0694 -
0.7732 5950 0.0693 -
0.7797 6000 0.0731 -
0.7862 6050 0.0626 -
0.7927 6100 0.0691 -
0.7992 6150 0.0711 -
0.8057 6200 0.0627 -
0.8122 6250 0.0726 -
0.8187 6300 0.068 -
0.8252 6350 0.0766 -
0.8317 6400 0.0617 -
0.8382 6450 0.0671 -
0.8447 6500 0.0645 -
0.8512 6550 0.0722 -
0.8577 6600 0.0751 -
0.8642 6650 0.0591 -
0.8707 6700 0.0664 -
0.8772 6750 0.0735 -
0.8837 6800 0.0709 -
0.8902 6850 0.0632 -
0.8967 6900 0.0679 -
0.9032 6950 0.0596 -
0.9097 7000 0.0676 -
0.9162 7050 0.066 -
0.9227 7100 0.069 -
0.9292 7150 0.0615 -
0.9357 7200 0.0579 -
0.9422 7250 0.0576 -
0.9487 7300 0.0558 -
0.9552 7350 0.0556 -
0.9617 7400 0.0637 -
0.9682 7450 0.0615 -
0.9747 7500 0.0677 -
0.9812 7550 0.0584 -
0.9877 7600 0.0661 -
0.9942 7650 0.0583 -
1.0006 7700 0.0639 -
1.0071 7750 0.0598 -
1.0136 7800 0.0586 -
1.0201 7850 0.055 -
1.0266 7900 0.0636 -
1.0331 7950 0.0623 -
1.0396 8000 0.0661 -
1.0461 8050 0.0633 -
1.0526 8100 0.056 -
1.0591 8150 0.0555 -
1.0656 8200 0.0608 -
1.0721 8250 0.0491 -
1.0786 8300 0.0592 -
1.0851 8350 0.0645 -
1.0916 8400 0.0553 -
1.0981 8450 0.0547 -
1.1046 8500 0.0494 -
1.1111 8550 0.0594 -
1.1176 8600 0.058 -
1.1241 8650 0.0589 -
1.1306 8700 0.0552 -
1.1371 8750 0.0554 -
1.1436 8800 0.0566 -
1.1501 8850 0.0558 -
1.1566 8900 0.0596 -
1.1631 8950 0.0551 -
1.1696 9000 0.061 -
1.1761 9050 0.0689 -
1.1826 9100 0.0565 -
1.1891 9150 0.0581 -
1.1956 9200 0.0606 -
1.2021 9250 0.057 -
1.2086 9300 0.0577 -
1.2151 9350 0.0629 -
1.2216 9400 0.0592 -
1.2281 9450 0.0547 -
1.2346 9500 0.0606 -
1.2411 9550 0.0588 -
1.2476 9600 0.0581 -
1.2541 9650 0.0624 -
1.2606 9700 0.0589 -
1.2671 9750 0.0646 -
1.2736 9800 0.0559 -
1.2801 9850 0.0594 -
1.2865 9900 0.0586 -
1.2930 9950 0.0552 -
1.2995 10000 0.0513 -
1.3060 10050 0.0565 -
1.3125 10100 0.0626 -
1.3190 10150 0.0483 -
1.3255 10200 0.0643 -
1.3320 10250 0.0524 -
1.3385 10300 0.0559 -
1.3450 10350 0.0589 -
1.3515 10400 0.0562 -
1.3580 10450 0.0592 -
1.3645 10500 0.047 -
1.3710 10550 0.0531 -
1.3775 10600 0.0506 -
1.3840 10650 0.0579 -
1.3905 10700 0.0569 -
1.3970 10750 0.0579 -
1.4035 10800 0.0504 -
1.4100 10850 0.0547 -
1.4165 10900 0.0497 -
1.4230 10950 0.0533 -
1.4295 11000 0.0488 -
1.4360 11050 0.0537 -
1.4425 11100 0.0544 -
1.4490 11150 0.0548 -
1.4555 11200 0.0475 -
1.4620 11250 0.0519 -
1.4685 11300 0.0568 -
1.4750 11350 0.0567 -
1.4815 11400 0.0473 -
1.4880 11450 0.0535 -
1.4945 11500 0.0531 -
1.5010 11550 0.0567 -
1.5075 11600 0.0529 -
1.5140 11650 0.0544 -
1.5205 11700 0.0612 -
1.5270 11750 0.055 -
1.5335 11800 0.0474 -
1.5400 11850 0.0572 -
1.5465 11900 0.0484 -
1.5530 11950 0.0553 -
1.5595 12000 0.0519 -
1.5660 12050 0.0565 -
1.5724 12100 0.0466 -
1.5789 12150 0.0502 -
1.5854 12200 0.0525 -
1.5919 12250 0.054 -
1.5984 12300 0.0556 -
1.6049 12350 0.0515 -
1.6114 12400 0.0476 -
1.6179 12450 0.0579 -
1.6244 12500 0.0567 -
1.6309 12550 0.0551 -
1.6374 12600 0.0518 -
1.6439 12650 0.0508 -
1.6504 12700 0.0503 -
1.6569 12750 0.0484 -
1.6634 12800 0.0531 -
1.6699 12850 0.0553 -
1.6764 12900 0.0588 -
1.6829 12950 0.0547 -
1.6894 13000 0.0587 -
1.6959 13050 0.0562 -
1.7024 13100 0.0558 -
1.7089 13150 0.0559 -
1.7154 13200 0.0547 -
1.7219 13250 0.059 -
1.7284 13300 0.053 -
1.7349 13350 0.0532 -
1.7414 13400 0.0552 -
1.7479 13450 0.0443 -
1.7544 13500 0.058 -
1.7609 13550 0.0503 -
1.7674 13600 0.0499 -
1.7739 13650 0.0478 -
1.7804 13700 0.0569 -
1.7869 13750 0.052 -
1.7934 13800 0.0458 -
1.7999 13850 0.0551 -
1.8064 13900 0.0567 -
1.8129 13950 0.0511 -
1.8194 14000 0.0546 -
1.8259 14050 0.058 -
1.8324 14100 0.0539 -
1.8389 14150 0.0544 -
1.8454 14200 0.061 -
1.8519 14250 0.0521 -
1.8583 14300 0.046 -
1.8648 14350 0.0494 -
1.8713 14400 0.0604 -
1.8778 14450 0.0543 -
1.8843 14500 0.0522 -
1.8908 14550 0.0533 -
1.8973 14600 0.0469 -
1.9038 14650 0.0525 -
1.9103 14700 0.0516 -
1.9168 14750 0.0485 -
1.9233 14800 0.0601 -
1.9298 14850 0.0487 -
1.9363 14900 0.0496 -
1.9428 14950 0.0529 -
1.9493 15000 0.054 -
1.9558 15050 0.0431 -
1.9623 15100 0.0449 -
1.9688 15150 0.0602 -
1.9753 15200 0.0447 -
1.9818 15250 0.0506 -
1.9883 15300 0.0503 -
1.9948 15350 0.0515 -

Framework Versions

  • Python: 3.12.12
  • SetFit: 1.1.3
  • Sentence Transformers: 5.1.1
  • Transformers: 4.57.1
  • PyTorch: 2.8.0+cu126
  • Datasets: 4.0.0
  • Tokenizers: 0.22.1

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
8
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
Text Classification
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for faodl/model_cca_multilabel_MiniLM-L12-70prop-data-augmented

Base model

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Finetuned
(265)
this model