OKAI-midi-gen-v-004

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 10.2911
• Accuracy: 0.0003

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 16
• seed: 444
• gradient_accumulation_steps: 3
• total_train_batch_size: 24
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine_with_restarts
• lr_scheduler_warmup_ratio: 0.3
• training_steps: 2000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
10.2092	3.2283	100	10.2764	0.0004
9.7373	6.4565	200	10.0773	0.0020
9.2158	9.6848	300	9.9404	0.0023
8.8101	12.9130	400	9.9445	0.0027
8.3439	16.1304	500	9.9546	0.0016
7.7001	19.3587	600	9.9316	0.0012
6.9652	22.5870	700	9.9396	0.0007
6.2067	25.8152	800	9.9274	0.0007
5.5185	29.0326	900	9.9435	0.0007
4.8318	32.2609	1000	9.9918	0.0004
4.343	35.4891	1100	10.0255	0.0004
3.9477	38.7174	1200	10.0792	0.0004
3.5394	41.9457	1300	10.1247	0.0002
3.2964	45.1630	1400	10.1824	0.0003
3.0237	48.3913	1500	10.2247	0.0003
2.8621	51.6196	1600	10.2556	0.0003
2.765	54.8478	1700	10.2756	0.0001
2.7383	58.0652	1800	10.2867	0.0003
2.7324	61.2935	1900	10.2906	0.0003
2.6918	64.5217	2000	10.2911	0.0003

Framework versions

• Transformers 4.52.3
• Pytorch 2.6.0
• Datasets 3.6.0
• Tokenizers 0.21.1