mt5-small-finetuned-amazon-en

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.3111
• Rouge1: 19.7134
• Rouge2: 11.6722
• Rougel: 18.6968
• Rougelsum: 18.8969

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
8.1374	1.0	563	3.7083	17.1951	8.5859	16.5981	16.8171
4.1857	2.0	1126	3.4477	18.0932	10.062	17.7225	17.8035
3.786	3.0	1689	3.3894	17.3401	9.4298	16.7955	16.9244
3.6013	4.0	2252	3.3642	19.3222	11.0203	18.5578	18.7428
3.4778	5.0	2815	3.3165	17.8766	9.8814	16.9851	17.422
3.3863	6.0	3378	3.3158	18.1618	9.7407	17.3244	17.5345
3.3278	7.0	3941	3.3108	19.7134	11.6722	18.6875	18.8903
3.2844	8.0	4504	3.3111	19.7134	11.6722	18.6968	18.8969

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 2.14.4
• Tokenizers 0.21.1