|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- sentiment-analysis |
|
|
- distilbert |
|
|
- text-classification |
|
|
- nlp |
|
|
- imdb |
|
|
- binary-classification |
|
|
license: mit |
|
|
datasets: |
|
|
- stanfordnlp/imdb |
|
|
language: |
|
|
- en |
|
|
metrics: |
|
|
- accuracy |
|
|
base_model: |
|
|
- distilbert/distilbert-base-uncased |
|
|
--- |
|
|
|
|
|
# Model Card for Model ID |
|
|
|
|
|
A fine-tuned DistilBERT model for binary sentiment analysis — predicting whether input text expresses a positive or negative sentiment. Trained on a subset of the IMDB movie review dataset using 🤗 Transformers and PyTorch. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
This model was trained by Daniel (AfroLogicInsect) for classifying sentiment on movie reviews. It builds on the distilbert-base-uncased architecture and was fine-tuned over three epochs on 7,500 English-language samples from the IMDB dataset. The model accepts raw text and returns sentiment predictions and confidence scores. |
|
|
|
|
|
- **Developed by:** Daniel 🇳🇬 (@AfroLogicInsect) |
|
|
- **Funded by:** [More Information Needed] |
|
|
- **Shared by:** [More Information Needed] |
|
|
- **Model type:** DistilBERT-based sequence classification |
|
|
- **Language(s) (NLP):** English |
|
|
- **License:** MIT |
|
|
- **Finetuned from model:** distilbert-base-uncased |
|
|
|
|
|
### Model Sources [optional] |
|
|
|
|
|
<!-- Provide the basic links for the model. --> |
|
|
|
|
|
- **Repository:** https://huggingface.co/AfroLogicInsect/sentiment-analysis-model_v2 |
|
|
- **Paper [optional]:** [More Information Needed] |
|
|
- **Demo [optional]:** [More Information Needed] |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
|
- Sentiment analysis of short texts, reviews, feedback forms, etc. |
|
|
- Embedding in web apps or chatbots to assess user mood or response tone |
|
|
|
|
|
|
|
|
### Downstream Use [optional] |
|
|
|
|
|
- Can be incorporated into feedback categorization pipelines |
|
|
- Extended to multilingual sentiment tasks with additional fine-tuning |
|
|
|
|
|
### Out-of-Scope Use |
|
|
|
|
|
- Not intended for clinical sentiment/emotion assessment |
|
|
- Doesn't capture sarcasm or highly ambiguous language reliably |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
|
|
- Biases may be inherited from the IMDB dataset (e.g. genre or cultural bias) |
|
|
- Model trained on movie reviews — performance may drop on domain-specific texts like legal or medical writing |
|
|
- Scores represent probabilities, not certainty |
|
|
|
|
|
### Recommendations |
|
|
|
|
|
- Use thresholding with score confidence if deploying in production |
|
|
- Consider further fine-tuning on in-domain data for robustness |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
```{python} |
|
|
from transformers import pipeline |
|
|
|
|
|
classifier = pipeline("sentiment-analysis", model="AfroLogicInsect/sentiment-analysis-model") |
|
|
result = classifier("Absolutely loved it!") |
|
|
print(result) |
|
|
``` |
|
|
|
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
- Subset of stanfordnlp/imdb |
|
|
- Balanced binary classes (positive and negative) |
|
|
- Sample size: ~15,000 training / 1,500 validation |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
|
|
##### Training arguments |
|
|
training_args = TrainingArguments( |
|
|
output_dir = "./sentiment-model-v2", |
|
|
num_train_epochs=3, |
|
|
per_device_train_batch_size=16, |
|
|
per_device_eval_batch_size=16, |
|
|
learning_rate=2e-5, # Explicit learning rate |
|
|
warmup_steps=100, # Reduced warmup |
|
|
weight_decay=0.01, |
|
|
logging_dir="./logs", |
|
|
logging_steps=50, |
|
|
eval_strategy="steps", |
|
|
eval_steps=200, # < 500: More frequent evaluation |
|
|
save_strategy="steps", |
|
|
save_steps=200, # match eval_steps |
|
|
load_best_model_at_end=True, |
|
|
metric_for_best_model="f1", |
|
|
greater_is_better=True, |
|
|
seed=42, # Reproducibility |
|
|
dataloader_drop_last=False, |
|
|
#remove_unused_columns=False, |
|
|
) |
|
|
|
|
|
##### Create trainer |
|
|
trainer = Trainer( |
|
|
model=model, |
|
|
args=training_args, |
|
|
train_dataset=train_dataset, |
|
|
eval_dataset=val_dataset, |
|
|
tokenizer=tokenizer, |
|
|
data_collator=data_collator, |
|
|
compute_metrics=compute_metrics, |
|
|
) |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
|
|
#### Testing Data |
|
|
|
|
|
- Validation set from IMDB subset |
|
|
|
|
|
#### Metrics |
|
|
|
|
|
|
|
|
Step Training Loss Validation Loss Accuracy F1 Precision Recall |
|
|
200 0.391100 0.344377 0.850000 0.863554 0.791991 0.949333 |
|
|
400 0.299000 0.304345 0.876000 0.865994 0.942006 0.801333 |
|
|
600 0.301700 0.298436 0.881333 0.888331 0.838863 0.944000 |
|
|
800 0.280700 0.260090 0.893333 0.897698 0.862408 0.936000 |
|
|
1000 0.173100 0.288142 0.899333 0.897766 0.911967 0.884000 |
|
|
1200 0.203700 0.263154 0.904667 0.905486 0.897772 0.913333 |
|
|
1400 0.186100 0.275240 0.904000 0.901370 0.926761 0.877333 |
|
|
1600 0.130400 0.291926 0.904667 0.903313 0.916324 0.890667 |
|
|
1800 0.158900 0.304814 0.908000 0.908488 0.903694 0.913333 |
|
|
2000 0.087900 0.332357 0.904000 0.905263 0.893506 0.917333 |
|
|
2200 0.119300 0.339073 0.908667 0.910399 0.893453 0.928000 |
|
|
2400 0.178100 0.366023 0.903333 0.905660 0.884371 0.928000 |
|
|
2600 0.072100 0.372015 0.909333 0.908356 0.918256 0.898667 |
|
|
2800 0.097700 0.368600 0.906667 0.908016 0.895078 0.921333 |
|
|
|
|
|
Final evaluation results: { |
|
|
'eval_loss': 0.3390733003616333, |
|
|
'eval_accuracy': 0.9086666666666666, |
|
|
'eval_f1': 0.9103989535644212, |
|
|
'eval_precision': 0.8934531450577664, |
|
|
'eval_recall': 0.928, |
|
|
'eval_runtime': 9.9181, |
|
|
'eval_samples_per_second': 151.239, |
|
|
'eval_steps_per_second': 9.478, 'epoch': 3.0 |
|
|
} |
|
|
|
|
|
|
|
|
### Results [Sample] |
|
|
|
|
|
#### ============================================================ |
|
|
#### TESTING FIXED MODEL |
|
|
#### ============================================================ |
|
|
|
|
|
Testing fixed model... |
|
|
Text Expected Predicted Confidence Match |
|
|
========================================================================================== |
|
|
I absolutely loved this movie! It was fantastic! positive positive 0.9959 ✓ |
|
|
This movie was terrible and boring. negative negative 0.9969 ✓ |
|
|
Amazing acting and great story! positive positive 0.9959 ✓ |
|
|
Worst film I've ever seen. negative negative 0.9950 ✓ |
|
|
Incredible cinematography and soundtrack. positive positive 0.9950 ✓ |
|
|
Complete waste of time and money. negative negative 0.9957 ✓ |
|
|
The movie was okay, nothing special. neutral negative 0.9915 N/A |
|
|
I enjoyed most of it. positive positive 0.9912 ✓ |
|
|
Pretty disappointing overall. negative negative 0.9936 ✓ |
|
|
Masterpiece of cinema! positive positive 0.9939 ✓ |
|
|
|
|
|
Overall Accuracy: 100.0% (9/9) |
|
|
|
|
|
## 🧪 Live Demo |
|
|
|
|
|
Try it out below! |
|
|
|
|
|
👉 [Launch Sentiment Analyzer](https://huggingface.co/spaces/AfroLogicInsect/sentiment-analysis-model-gradio) |
|
|
|
|
|
|
|
|
#### Summary |
|
|
|
|
|
The model performs well on balanced sentiment data and generalizes across a variety of movie review tones. Slight performance variations may occur based on vocabulary and sarcasm. |
|
|
|
|
|
|
|
|
## Environmental Impact |
|
|
|
|
|
Carbon footprint estimated using [ML Impact Calculator](https://mlco2.github.io/impact#compute) |
|
|
|
|
|
Hardware Type: GPU (single NVIDIA T4) |
|
|
Hours used: ~2.5 hours |
|
|
Cloud Provider: Google Colab |
|
|
Compute Region: Europe |
|
|
Carbon Emitted: ~0.3 kg CO₂eq |
|
|
|
|
|
## Technical Specifications [optional] |
|
|
|
|
|
### Model Architecture and Objective |
|
|
|
|
|
DistilBERT with a classification head trained for binary text classification. |
|
|
|
|
|
### Compute Infrastructure |
|
|
- Hardware: Google Colab (GPU-backed) |
|
|
- Software: Python, PyTorch, 🤗 Transformers, Hugging Face Hub |
|
|
|
|
|
## Citation |
|
|
|
|
|
**BibTeX:** |
|
|
|
|
|
[@misc{afrologicinsect2025sentiment, |
|
|
title = {AfroLogicInsect Sentiment Analysis Model}, |
|
|
author = {Akan Daniel}, |
|
|
year = {2025}, |
|
|
howpublished = {\url{https://huggingface.co/AfroLogicInsect/sentiment-analysis-model_v2}}, |
|
|
}] |
|
|
|
|
|
|
|
|
## Model Card Contact |
|
|
|
|
|
- Name: Daniel (@AfroLogicInsect) |
|
|
- Location: Lagos, Nigeria |
|
|
- Contact: GitHub / Hugging Face / email (danielamahtoday@gmail.com) |
