cwe-parent-vulnerability-classification-distilbert-base-uncased

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.3946
• Accuracy: 0.7416
• F1 Macro: 0.4136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro
3.2586	1.0	25	3.2817	0.0225	0.0037
3.1726	2.0	50	3.2811	0.0225	0.0037
3.1195	3.0	75	3.2705	0.0225	0.0037
3.0213	4.0	100	3.2327	0.0449	0.0325
3.0235	5.0	125	3.2046	0.2247	0.0830
2.931	6.0	150	3.2138	0.2697	0.0725
2.9104	7.0	175	3.1642	0.4382	0.1117
2.779	8.0	200	3.1058	0.4831	0.1095
2.7528	9.0	225	3.0725	0.5169	0.1238
2.6453	10.0	250	3.0537	0.5730	0.2290
2.6262	11.0	275	3.0154	0.5618	0.2103
2.4852	12.0	300	2.9611	0.5955	0.3516
2.3778	13.0	325	2.9121	0.5955	0.3256
2.3381	14.0	350	2.8414	0.6067	0.3057
2.2415	15.0	375	2.8161	0.6180	0.3610
2.0991	16.0	400	2.7636	0.6180	0.3520
2.0469	17.0	425	2.7049	0.6180	0.3175
1.9623	18.0	450	2.7100	0.6404	0.3767
1.921	19.0	475	2.6304	0.6404	0.3320
1.8045	20.0	500	2.6552	0.6404	0.3130
1.7417	21.0	525	2.5960	0.6517	0.3082
1.7004	22.0	550	2.5777	0.6404	0.3183
1.6295	23.0	575	2.5849	0.6742	0.3602
1.5823	24.0	600	2.5379	0.6742	0.345
1.4711	25.0	625	2.5262	0.6742	0.3512
1.4868	26.0	650	2.4962	0.7079	0.3970
1.4563	27.0	675	2.4695	0.6854	0.3438
1.3601	28.0	700	2.4549	0.6854	0.3438
1.2847	29.0	725	2.4691	0.7079	0.3688
1.2883	30.0	750	2.4587	0.7079	0.3966
1.316	31.0	775	2.4454	0.7191	0.3623
1.1845	32.0	800	2.4432	0.7416	0.4111
1.24	33.0	825	2.4280	0.7079	0.3980
1.1163	34.0	850	2.4179	0.7416	0.3871
1.1728	35.0	875	2.4326	0.7528	0.4201
1.1013	36.0	900	2.4116	0.7416	0.4136
1.1612	37.0	925	2.3985	0.7416	0.4136
1.1472	38.0	950	2.3956	0.7416	0.4136
1.069	39.0	975	2.3946	0.7416	0.4136
1.0865	40.0	1000	2.3983	0.7528	0.4201

Framework versions

• Transformers 4.55.4
• Pytorch 2.7.1+cu126
• Datasets 4.0.0
• Tokenizers 0.21.2