|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
- automatic_speech_recognition |
|
|
- asr |
|
|
- nlp |
|
|
- speech_to_text |
|
|
- low_resource |
|
|
metrics: |
|
|
- wer |
|
|
base_model: facebook/wav2vec2-large-xlsr-53 |
|
|
model-index: |
|
|
- name: pidgin-wav2vec2-xlsr53 |
|
|
results: [] |
|
|
datasets: |
|
|
- asr-nigerian-pidgin/nigerian-pidgin-1.0 |
|
|
pipeline_tag: automatic-speech-recognition |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# pidgin-wav2vec2-xlsr53 |
|
|
|
|
|
This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the [Nigerian Pidgin](https://huggingface.co/datasets/asr-nigerian-pidgin/nigerian-pidgin-1.0) dataset. |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 0.6907 |
|
|
- Wer: 0.3161 (val) |
|
|
|
|
|
## Model description |
|
|
|
|
|
*to be updated* |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
**Intended Uses**: |
|
|
- Best suited for automatic speech recognition (ASR) tasks on Nigerian Pidgin audio, such as speech-to-text conversion and related downstream tasks. |
|
|
- Academic research on low-resource and creole language ASR. |
|
|
|
|
|
**Known Limitations**: |
|
|
- Performance may degrade with dialectal variation, heavy code-switching, or noisy audio environments. |
|
|
- Model reflects biases present in the training dataset, which may affect accuracy on underrepresented demographics, phonetic variations or topics. |
|
|
- May struggle with rare words, numerals, and domain-specific terminology not well represented in the training set. |
|
|
- Not recommended for high-stakes domains (e.g., legal, medical) without domain-specific retraining/finetuning. |
|
|
|
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
The model was fine-tuned on the [Nigerian Pidgin ASR v1.0 dataset](https://huggingface.co/datasets/asr-nigerian-pidgin/nigerian-pidgin-1.0), consisting of over 4,200 utterances recorded by 10 native speakers (balanced across gender and age) using the LIG-Aikuma mobile platform. Recordings were collected in controlled environments to ensure high-quality audio. |
|
|
Performance: WER 7.4%(train), 31.6% (validation) / 29.6% (test), exceeding baseline benchmarks like QuartzNet and zero-shot XLSR. This results demonstrate the effectiveness of targeted fine-tuning for low-resource ASR. |
|
|
|
|
|
## Training procedure |
|
|
We fine-tuned the facebook/wav2vec2-large-xlsr-53 model using the Nigerian Pidgin ASR dataset, following the methodology outlined in the XLSR-53 paper. Training was performed on a single NVIDIA A100 GPU using the Hugging Face transformers library with fp16 mixed precision to accelerate computation and reduce memory usage. |
|
|
|
|
|
A key modification from the standard setup was unfreezing the feature encoder during fine-tuning. This adjustment yielded improved performance, lowering word error rates (WER) on both validation and test sets compared to the frozen-encoder approach. |
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 1e-4 |
|
|
- train_batch_size: 4 |
|
|
- eval_batch_size: 4 |
|
|
- seed: 3407 |
|
|
- gradient_accumulation_steps: 2 |
|
|
- total_train_batch_size: 8 |
|
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
|
- lr_scheduler_type: linear |
|
|
- lr_scheduler_warmup_steps: 1000 |
|
|
- num_epochs: 30 |
|
|
- mixed_precision_training: Native AMP |
|
|
|
|
|
This configuration balanced training stability, efficiency, and accuracy, allowing the model to adapt effectively to Nigerian Pidgin speech patterns despite the dataset’s limited size |
|
|
### Perfomance Comparision for Frozen Encoder and Unfrozen Encoder: |
|
|
| Encoder State | Val WER | Test WER | |
|
|
| ------------- | ------- | -------- | |
|
|
| Frozen | 0.332 | 0.436 | |
|
|
| Unfrozen | 0.3161 | 0.296 | |
|
|
|
|
|
|
|
|
### Training results(Unfrozen Model) |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Wer | |
|
|
|:-------------:|:-----:|:-----:|:---------------:|:------:| |
|
|
| 6.604 | 1.48 | 500 | 3.0540 | 1.0 | |
|
|
| 3.0176 | 2.95 | 1000 | 3.0035 | 1.0 | |
|
|
| 2.1071 | 4.43 | 1500 | 1.0811 | 0.6289 | |
|
|
| 1.1143 | 5.91 | 2000 | 0.8348 | 0.5017 | |
|
|
| 0.8501 | 7.39 | 2500 | 0.7707 | 0.4352 | |
|
|
| 0.7272 | 8.86 | 3000 | 0.7410 | 0.4075 | |
|
|
| 0.6038 | 10.34 | 3500 | 0.6283 | 0.3850 | |
|
|
| 0.5334 | 11.82 | 4000 | 0.6356 | 0.3701 | |
|
|
| 0.4645 | 13.29 | 4500 | 0.6243 | 0.3657 | |
|
|
| 0.4251 | 14.77 | 5000 | 0.6838 | 0.3492 | |
|
|
| 0.3801 | 16.25 | 5500 | 0.6619 | 0.3445 | |
|
|
| 0.3636 | 17.73 | 6000 | 0.6945 | 0.3360 | |
|
|
| 0.3366 | 19.2 | 6500 | 0.6108 | 0.3340 | |
|
|
| 0.3146 | 20.68 | 7000 | 0.6511 | 0.3273 | |
|
|
| 0.3003 | 22.16 | 7500 | 0.6815 | 0.3253 | |
|
|
| 0.2783 | 23.63 | 8000 | 0.6761 | 0.3215 | |
|
|
| 0.2601 | 25.11 | 8500 | 0.6762 | 0.3187 | |
|
|
| 0.2528 | 26.59 | 9000 | 0.6687 | 0.3194 | |
|
|
| 0.2409 | 28.06 | 9500 | 0.7064 | 0.3163 | |
|
|
| 0.2359 | 29.54 | 10000 | 0.6907 | 0.3161 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.37.2 |
|
|
- Pytorch 2.0.1+cu117 |
|
|
- Datasets 2.20.0 |
|
|
- Tokenizers 0.15.2 |
|
|
|
|
|
## Citation |
|
|
@misc{rufai2025endtoendtrainingautomaticspeech, |
|
|
title={Towards End-to-End Training of Automatic Speech Recognition for Nigerian Pidgin}, |
|
|
author={Amina Mardiyyah Rufai and Afolabi Abeeb and Esther Oduntan and Tayo Arulogun and Oluwabukola Adegboro and Daniel Ajisafe}, |
|
|
year={2025}, |
|
|
eprint={2010.11123}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={eess.AS}, |
|
|
url={https://arxiv.org/abs/2010.11123}, |
|
|
} |
