--- library_name: transformers license: mit base_model: roberta-base tags: - generated_from_trainer metrics: - accuracy model-index: - name: vulnerability-severity-classification-roberta-base results: [] datasets: - CIRCL/vulnerability-scores --- # vulnerability-severity-classification-roberta-base This model is a fine-tuned version of [roberta-base](https://huggingface.co/FacebookAI/roberta-base) on the dataset [CIRCL/vulnerability-scores](https://huggingface.co/datasets/CIRCL/vulnerability-scores). It achieves the following results on the evaluation set: - Loss: 0.5372 - Accuracy: 0.8138 ## Model description It is a classification model and is aimed to assist in classifying vulnerabilities by severity based on their descriptions. ## How to get started with the model ```python from transformers import AutoModelForSequenceClassification, AutoTokenizer import torch labels = ["low", "medium", "high", "critical"] model_name = "CIRCL/vulnerability-severity-classification-roberta-base" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSequenceClassification.from_pretrained(model_name) model.eval() test_description = "langchain_experimental 0.0.14 allows an attacker to bypass the CVE-2023-36258 fix and execute arbitrary code via the PALChain in the python exec method." inputs = tokenizer(test_description, return_tensors="pt", truncation=True, padding=True) # Run inference with torch.no_grad(): outputs = model(**inputs) predictions = torch.nn.functional.softmax(outputs.logits, dim=-1) # Print results print("Predictions:", predictions) predicted_class = torch.argmax(predictions, dim=-1).item() print("Predicted severity:", labels[predicted_class]) ``` ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - num_epochs: 5 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:------:|:---------------:|:--------:| | 0.7239 | 1.0 | 24058 | 0.6421 | 0.7359 | | 0.6718 | 2.0 | 48116 | 0.5911 | 0.7598 | | 0.5085 | 3.0 | 72174 | 0.5567 | 0.7878 | | 0.4282 | 4.0 | 96232 | 0.5377 | 0.8059 | | 0.3508 | 5.0 | 120290 | 0.5372 | 0.8138 | ### Framework versions - Transformers 4.49.0 - Pytorch 2.6.0+cu124 - Datasets 3.3.2 - Tokenizers 0.21.0