Hugging Face's logo

CIRCL
/
cwe-parent-vulnerability-classification-roberta-base

Text Classification
Transformers
PyTorch
Safetensors
roberta
Generated from Trainer
Model card Files
xet
Community
cwe-parent-vulnerability-classification-roberta-base / README.md
cedricbonhomme's picture
cedricbonhomme
Update README.md
f6ea582 verified
preview code
|
raw
history blame contribute delete
4.43 kB
metadata
library_name: transformers
license: mit
base_model: roberta-base
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: cwe-parent-vulnerability-classification-roberta-base
    results: []
datasets:
  - CIRCL/vulnerability-cwe-patch

cwe-parent-vulnerability-classification-roberta-base

This model is a fine-tuned version of roberta-base.

It achieves the following results on the evaluation set:

  • Loss: 1.2078
  • Accuracy: 0.875
  • F1 Macro: 0.6248

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 40

Training results

Training Loss Epoch Step Validation Loss Accuracy F1 Macro
3.2699 1.0 25 3.1492 0.0341 0.0055
3.1972 2.0 50 2.9909 0.0114 0.0064
3.1211 3.0 75 3.0017 0.0341 0.0140
3.0888 4.0 100 3.0223 0.2841 0.0463
2.9467 5.0 125 2.9608 0.0114 0.0018
2.9851 6.0 150 2.8743 0.1932 0.0641
2.9083 7.0 175 2.7687 0.375 0.0963
2.7652 8.0 200 2.7049 0.4318 0.1953
2.6893 9.0 225 2.5547 0.4886 0.1952
2.5636 10.0 250 2.4970 0.5682 0.3314
2.477 11.0 275 2.3499 0.6136 0.3790
2.2936 12.0 300 2.2659 0.6364 0.3949
2.1369 13.0 325 2.1758 0.625 0.4002
2.0615 14.0 350 2.1015 0.6477 0.4169
1.9548 15.0 375 1.9444 0.6932 0.3972
1.7943 16.0 400 1.8892 0.6818 0.4210
1.6619 17.0 425 1.8439 0.6818 0.4149
1.5391 18.0 450 1.7247 0.7159 0.4848
1.4415 19.0 475 1.6650 0.7273 0.4749
1.2834 20.0 500 1.5743 0.7727 0.5574
1.2245 21.0 525 1.5396 0.7614 0.5373
1.1629 22.0 550 1.5005 0.7614 0.5350
1.0894 23.0 575 1.4478 0.7614 0.5383
0.9755 24.0 600 1.4335 0.7841 0.5599
0.9271 25.0 625 1.4195 0.7841 0.5562
0.8761 26.0 650 1.3740 0.8182 0.6015
0.8312 27.0 675 1.3479 0.8295 0.6086
0.7523 28.0 700 1.3379 0.8295 0.5948
0.718 29.0 725 1.2991 0.8295 0.5948
0.6819 30.0 750 1.3059 0.8409 0.6047
0.6771 31.0 775 1.2650 0.8636 0.6167
0.6267 32.0 800 1.2905 0.8523 0.6252
0.6068 33.0 825 1.2559 0.875 0.6248
0.5811 34.0 850 1.2371 0.875 0.6248
0.5579 35.0 875 1.2231 0.875 0.6248
0.5385 36.0 900 1.2342 0.875 0.6248
0.5334 37.0 925 1.2255 0.875 0.6248
0.4868 38.0 950 1.2223 0.875 0.6248
0.5228 39.0 975 1.2078 0.875 0.6248
0.5325 40.0 1000 1.2101 0.875 0.6248

Framework versions

  • Transformers 4.55.4
  • Pytorch 2.7.1+cu126
  • Datasets 4.0.0
  • Tokenizers 0.21.2