Hugging Face's logo

ThomasTheMaker
/
pico-decoder-tiny-experiments

Safetensors
Model card Files
xet
Community
pico-decoder-tiny-experiments
File size: 64,726 Bytes 
a1a7208
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6b0f104
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
 
2025-08-31 17:03:52 - pico-train - INFO - Step 100000 -- 📊 Evaluation Results
2025-08-31 17:03:52 - pico-train - INFO - └── paloma: inf
2025-08-31 17:03:52 - pico-train - INFO - ==================================================
2025-08-31 17:03:52 - pico-train - INFO - ✨ Training Configuration
2025-08-31 17:03:52 - pico-train - INFO - ==================================================
2025-08-31 17:03:52 - pico-train - INFO - ╭─────────────────────────────────────────────────────╮
2025-08-31 17:03:52 - pico-train - INFO - │ checkpointing:                                      │
2025-08-31 17:03:52 - pico-train - INFO - │   checkpoints_dir: checkpoints                      │
2025-08-31 17:03:52 - pico-train - INFO - │   evaluation:                                       │
2025-08-31 17:03:52 - pico-train - INFO - │     eval_results_dir: eval_results                  │
2025-08-31 17:03:52 - pico-train - INFO - │   fabric_checkpoint_dir: fabric_state               │
2025-08-31 17:03:52 - pico-train - INFO - │   fabric_checkpoint_filename: checkpoint.pt         │
2025-08-31 17:03:52 - pico-train - INFO - │   hf_checkpoint:                                    │
2025-08-31 17:03:52 - pico-train - INFO - │     collection_slug: null                           │
2025-08-31 17:03:52 - pico-train - INFO - │     repo_id: ThomasTheMaker/pico-decoder-tiny       │
2025-08-31 17:03:52 - pico-train - INFO - │   learning_dynamics:                                │
2025-08-31 17:03:52 - pico-train - INFO - │     batch_size: 1                                   │
2025-08-31 17:03:52 - pico-train - INFO - │     eval_data: null                                 │
2025-08-31 17:03:52 - pico-train - INFO - │     layer_suffixes:                                 │
2025-08-31 17:03:52 - pico-train - INFO - │     - attention.v_proj                              │
2025-08-31 17:03:52 - pico-train - INFO - │     - attention.o_proj                              │
2025-08-31 17:03:52 - pico-train - INFO - │     - swiglu.w_2                                    │
2025-08-31 17:03:52 - pico-train - INFO - │     sequence_idx: -1                                │
2025-08-31 17:03:52 - pico-train - INFO - │   learning_dynamics_dir: learning_dynamics          │
2025-08-31 17:03:52 - pico-train - INFO - │   logs_dir: logs                                    │
2025-08-31 17:03:52 - pico-train - INFO - │   run_name: pico-decoder-tiny-dolma250M-v1          │
2025-08-31 17:03:52 - pico-train - INFO - │   runs_dir: runs                                    │
2025-08-31 17:03:52 - pico-train - INFO - │   save_every_n_steps: 2000                          │
2025-08-31 17:03:52 - pico-train - INFO - │   save_to_hf: false                                 │
2025-08-31 17:03:52 - pico-train - INFO - │   training:                                         │
2025-08-31 17:03:52 - pico-train - INFO - │     auto_resume: true                               │
2025-08-31 17:03:52 - pico-train - INFO - │ data:                                               │
2025-08-31 17:03:52 - pico-train - INFO - │   dataloader:                                       │
2025-08-31 17:03:52 - pico-train - INFO - │     batch_size: 16                                  │
2025-08-31 17:03:52 - pico-train - INFO - │   dataset:                                          │
2025-08-31 17:03:52 - pico-train - INFO - │     name: pico-lm/pretokenized-dolma                │
2025-08-31 17:03:52 - pico-train - INFO - │   tokenizer:                                        │
2025-08-31 17:03:52 - pico-train - INFO - │     name: allenai/OLMo-7B-0724-hf                   │
2025-08-31 17:03:52 - pico-train - INFO - │     vocab_size: 50304                               │
2025-08-31 17:03:52 - pico-train - INFO - │ evaluation:                                         │
2025-08-31 17:03:52 - pico-train - INFO - │   metrics:                                          │
2025-08-31 17:03:52 - pico-train - INFO - │   - paloma                                          │
2025-08-31 17:03:52 - pico-train - INFO - │   paloma:                                           │
2025-08-31 17:03:52 - pico-train - INFO - │     batch_size: 1                                   │
2025-08-31 17:03:52 - pico-train - INFO - │     dataset_name: pico-lm/pretokenized-paloma-tinsy │
2025-08-31 17:03:52 - pico-train - INFO - │     dataset_split: val                              │
2025-08-31 17:03:52 - pico-train - INFO - │     max_length: 2048                                │
2025-08-31 17:03:52 - pico-train - INFO - │ model:                                              │
2025-08-31 17:03:52 - pico-train - INFO - │   activation_hidden_dim: 384                        │
2025-08-31 17:03:52 - pico-train - INFO - │   attention_n_heads: 12                             │
2025-08-31 17:03:52 - pico-train - INFO - │   attention_n_kv_heads: 4                           │
2025-08-31 17:03:52 - pico-train - INFO - │   batch_size: 1024                                  │
2025-08-31 17:03:52 - pico-train - INFO - │   d_model: 96                                       │
2025-08-31 17:03:52 - pico-train - INFO - │   max_seq_len: 2048                                 │
2025-08-31 17:03:52 - pico-train - INFO - │   model_type: pico_decoder                          │
2025-08-31 17:03:52 - pico-train - INFO - │   n_layers: 12                                      │
2025-08-31 17:03:52 - pico-train - INFO - │   norm_eps: 1.0e-06                                 │
2025-08-31 17:03:52 - pico-train - INFO - │   position_emb_theta: 10000.0                       │
2025-08-31 17:03:52 - pico-train - INFO - │   vocab_size: 50304                                 │
2025-08-31 17:03:52 - pico-train - INFO - │ monitoring:                                         │
2025-08-31 17:03:52 - pico-train - INFO - │   logging:                                          │
2025-08-31 17:03:52 - pico-train - INFO - │     log_every_n_steps: 100                          │
2025-08-31 17:03:52 - pico-train - INFO - │     log_level: INFO                                 │
2025-08-31 17:03:52 - pico-train - INFO - │   save_to_wandb: false                              │
2025-08-31 17:03:52 - pico-train - INFO - │   wandb:                                            │
2025-08-31 17:03:52 - pico-train - INFO - │     entity: boymyc                                  │
2025-08-31 17:03:52 - pico-train - INFO - │     project: pico-decoder-tiny                      │
2025-08-31 17:03:52 - pico-train - INFO - │ training:                                           │
2025-08-31 17:03:52 - pico-train - INFO - │   fabric:                                           │
2025-08-31 17:03:52 - pico-train - INFO - │     accelerator: cuda                               │
2025-08-31 17:03:52 - pico-train - INFO - │     num_devices: 1                                  │
2025-08-31 17:03:52 - pico-train - INFO - │     num_nodes: 1                                    │
2025-08-31 17:03:52 - pico-train - INFO - │     precision: bf16-mixed                           │
2025-08-31 17:03:52 - pico-train - INFO - │   max_steps: 100000                                 │
2025-08-31 17:03:52 - pico-train - INFO - │   optimization:                                     │
2025-08-31 17:03:52 - pico-train - INFO - │     gradient_accumulation_steps: 1                  │
2025-08-31 17:03:52 - pico-train - INFO - │     lr: 0.0002                                      │
2025-08-31 17:03:52 - pico-train - INFO - │     lr_scheduler: cosine                            │
2025-08-31 17:03:52 - pico-train - INFO - │     lr_warmup_steps: 2000                           │
2025-08-31 17:03:52 - pico-train - INFO - │     optimizer: adamw                                │
2025-08-31 17:03:52 - pico-train - INFO - │                                                     │
2025-08-31 17:03:52 - pico-train - INFO - ╰─────────────────────────────────────────────────────╯
2025-08-31 17:03:52 - pico-train - INFO - ==================================================
2025-08-31 17:03:52 - pico-train - INFO - ⛭ Runtime Summary:
2025-08-31 17:03:52 - pico-train - INFO - ==================================================
2025-08-31 17:03:52 - pico-train - INFO - Starting from step: 100000
2025-08-31 17:03:52 - pico-train - INFO - Model Setup:
2025-08-31 17:03:52 - pico-train - INFO - └─ Total Parameters: 11,282,784
2025-08-31 17:03:52 - pico-train - INFO - └─ Trainable Parameters: 11,282,784
2025-08-31 17:03:52 - pico-train - INFO - Distributed Setup:
2025-08-31 17:03:52 - pico-train - INFO - └─ Number of Devices: 1
2025-08-31 17:03:52 - pico-train - INFO - └─ Device Type: NVIDIA H100 80GB HBM3
2025-08-31 17:03:52 - pico-train - INFO - └─ Available Memory: 85.03 GB
2025-08-31 17:03:52 - pico-train - INFO - Software Setup:
2025-08-31 17:03:52 - pico-train - INFO - └─ Python Version: 3.12.3
2025-08-31 17:03:52 - pico-train - INFO - └─ PyTorch Version: 2.8.0+cu128
2025-08-31 17:03:52 - pico-train - INFO - └─ CUDA Version: 12.8
2025-08-31 17:03:52 - pico-train - INFO - └─ Operating System: Linux 6.8.0-71-generic
2025-08-31 17:03:52 - pico-train - INFO - Batch Size Configuration:
2025-08-31 17:03:52 - pico-train - INFO - └─ Global Batch Size: 16
2025-08-31 17:03:52 - pico-train - INFO - └─ Per Device Batch Size: 16
2025-08-31 17:03:52 - pico-train - INFO - └─ Gradient Accumulation Steps: 1
2025-08-31 17:03:52 - pico-train - INFO - ==================================================
2025-08-31 17:03:52 - pico-train - INFO - Step 100000 -- 🔄 Training Metrics
2025-08-31 17:03:52 - pico-train - INFO - ├── Loss: 4.9432
2025-08-31 17:03:52 - pico-train - INFO - ├── Learning Rate: 2.00e-05
2025-08-31 17:03:52 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:03:52 - pico-train - INFO - Step 100000 -- 📈 Saving Learning Dynamics
2025-08-31 17:04:49 - pico-train - INFO - Step 100100 -- 🔄 Training Metrics
2025-08-31 17:04:49 - pico-train - INFO - ├── Loss: 4.7703
2025-08-31 17:04:49 - pico-train - INFO - ├── Learning Rate: 1.01e-04
2025-08-31 17:04:49 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:05:43 - pico-train - INFO - Step 100200 -- 🔄 Training Metrics
2025-08-31 17:05:43 - pico-train - INFO - ├── Loss: 4.8047
2025-08-31 17:05:43 - pico-train - INFO - ├── Learning Rate: 1.01e-04
2025-08-31 17:05:43 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:06:37 - pico-train - INFO - Step 100300 -- 🔄 Training Metrics
2025-08-31 17:06:37 - pico-train - INFO - ├── Loss: 4.8076
2025-08-31 17:06:37 - pico-train - INFO - ├── Learning Rate: 1.01e-04
2025-08-31 17:06:37 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:07:31 - pico-train - INFO - Step 100400 -- 🔄 Training Metrics
2025-08-31 17:07:31 - pico-train - INFO - ├── Loss: 4.7926
2025-08-31 17:07:31 - pico-train - INFO - ├── Learning Rate: 1.01e-04
2025-08-31 17:07:31 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:08:25 - pico-train - INFO - Step 100500 -- 🔄 Training Metrics
2025-08-31 17:08:25 - pico-train - INFO - ├── Loss: 4.8059
2025-08-31 17:08:25 - pico-train - INFO - ├── Learning Rate: 1.01e-04
2025-08-31 17:08:25 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:09:19 - pico-train - INFO - Step 100600 -- 🔄 Training Metrics
2025-08-31 17:09:19 - pico-train - INFO - ├── Loss: 4.7896
2025-08-31 17:09:19 - pico-train - INFO - ├── Learning Rate: 1.01e-04
2025-08-31 17:09:19 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:10:12 - pico-train - INFO - Step 100700 -- 🔄 Training Metrics
2025-08-31 17:10:12 - pico-train - INFO - ├── Loss: 4.8066
2025-08-31 17:10:12 - pico-train - INFO - ├── Learning Rate: 1.00e-04
2025-08-31 17:10:12 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:11:07 - pico-train - INFO - Step 100800 -- 🔄 Training Metrics
2025-08-31 17:11:07 - pico-train - INFO - ├── Loss: 4.7870
2025-08-31 17:11:07 - pico-train - INFO - ├── Learning Rate: 1.00e-04
2025-08-31 17:11:07 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:12:01 - pico-train - INFO - Step 100900 -- 🔄 Training Metrics
2025-08-31 17:12:01 - pico-train - INFO - ├── Loss: 4.7958
2025-08-31 17:12:01 - pico-train - INFO - ├── Learning Rate: 1.00e-04
2025-08-31 17:12:01 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:12:55 - pico-train - INFO - Step 101000 -- 🔄 Training Metrics
2025-08-31 17:12:55 - pico-train - INFO - ├── Loss: 4.8081
2025-08-31 17:12:55 - pico-train - INFO - ├── Learning Rate: 1.00e-04
2025-08-31 17:12:55 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:13:48 - pico-train - INFO - Step 101100 -- 🔄 Training Metrics
2025-08-31 17:13:48 - pico-train - INFO - ├── Loss: 4.8023
2025-08-31 17:13:48 - pico-train - INFO - ├── Learning Rate: 9.98e-05
2025-08-31 17:13:48 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:14:43 - pico-train - INFO - Step 101200 -- 🔄 Training Metrics
2025-08-31 17:14:43 - pico-train - INFO - ├── Loss: 4.7830
2025-08-31 17:14:43 - pico-train - INFO - ├── Learning Rate: 9.97e-05
2025-08-31 17:14:43 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:15:38 - pico-train - INFO - Step 101300 -- 🔄 Training Metrics
2025-08-31 17:15:38 - pico-train - INFO - ├── Loss: 4.8071
2025-08-31 17:15:38 - pico-train - INFO - ├── Learning Rate: 9.95e-05
2025-08-31 17:15:38 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:16:32 - pico-train - INFO - Step 101400 -- 🔄 Training Metrics
2025-08-31 17:16:32 - pico-train - INFO - ├── Loss: 4.8072
2025-08-31 17:16:32 - pico-train - INFO - ├── Learning Rate: 9.94e-05
2025-08-31 17:16:32 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:17:27 - pico-train - INFO - Step 101500 -- 🔄 Training Metrics
2025-08-31 17:17:27 - pico-train - INFO - ├── Loss: 4.8027
2025-08-31 17:17:27 - pico-train - INFO - ├── Learning Rate: 9.92e-05
2025-08-31 17:17:27 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:18:20 - pico-train - INFO - Step 101600 -- 🔄 Training Metrics
2025-08-31 17:18:20 - pico-train - INFO - ├── Loss: 4.7874
2025-08-31 17:18:20 - pico-train - INFO - ├── Learning Rate: 9.90e-05
2025-08-31 17:18:20 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:19:15 - pico-train - INFO - Step 101700 -- 🔄 Training Metrics
2025-08-31 17:19:15 - pico-train - INFO - ├── Loss: 4.7817
2025-08-31 17:19:15 - pico-train - INFO - ├── Learning Rate: 9.89e-05
2025-08-31 17:19:15 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:20:09 - pico-train - INFO - Step 101800 -- 🔄 Training Metrics
2025-08-31 17:20:09 - pico-train - INFO - ├── Loss: 4.8188
2025-08-31 17:20:09 - pico-train - INFO - ├── Learning Rate: 9.87e-05
2025-08-31 17:20:09 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:21:04 - pico-train - INFO - Step 101900 -- 🔄 Training Metrics
2025-08-31 17:21:04 - pico-train - INFO - ├── Loss: 4.7880
2025-08-31 17:21:04 - pico-train - INFO - ├── Learning Rate: 9.86e-05
2025-08-31 17:21:04 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 17:21:58 - pico-train - INFO - Step 102000 -- 💾 Saving Checkpoint
2025-08-31 18:00:17 - pico-train - INFO - Step 102000 -- 📊 Evaluation Results
2025-08-31 18:00:17 - pico-train - INFO - └── paloma: inf
2025-08-31 18:00:17 - pico-train - INFO - Step 102000 -- 🔄 Training Metrics
2025-08-31 18:00:17 - pico-train - INFO - ├── Loss: 4.8055
2025-08-31 18:00:17 - pico-train - INFO - ├── Learning Rate: 9.84e-05
2025-08-31 18:00:17 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:00:17 - pico-train - INFO - Step 102000 -- 📈 Saving Learning Dynamics
2025-08-31 18:01:13 - pico-train - INFO - Step 102100 -- 🔄 Training Metrics
2025-08-31 18:01:13 - pico-train - INFO - ├── Loss: 4.7742
2025-08-31 18:01:13 - pico-train - INFO - ├── Learning Rate: 9.83e-05
2025-08-31 18:01:13 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:02:07 - pico-train - INFO - Step 102200 -- 🔄 Training Metrics
2025-08-31 18:02:07 - pico-train - INFO - ├── Loss: 4.8050
2025-08-31 18:02:07 - pico-train - INFO - ├── Learning Rate: 9.81e-05
2025-08-31 18:02:07 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:03:01 - pico-train - INFO - Step 102300 -- 🔄 Training Metrics
2025-08-31 18:03:01 - pico-train - INFO - ├── Loss: 4.8066
2025-08-31 18:03:01 - pico-train - INFO - ├── Learning Rate: 9.79e-05
2025-08-31 18:03:01 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:03:57 - pico-train - INFO - Step 102400 -- 🔄 Training Metrics
2025-08-31 18:03:57 - pico-train - INFO - ├── Loss: 4.7865
2025-08-31 18:03:57 - pico-train - INFO - ├── Learning Rate: 9.78e-05
2025-08-31 18:03:57 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:04:50 - pico-train - INFO - Step 102500 -- 🔄 Training Metrics
2025-08-31 18:04:50 - pico-train - INFO - ├── Loss: 4.8019
2025-08-31 18:04:50 - pico-train - INFO - ├── Learning Rate: 9.76e-05
2025-08-31 18:04:50 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:05:45 - pico-train - INFO - Step 102600 -- 🔄 Training Metrics
2025-08-31 18:05:45 - pico-train - INFO - ├── Loss: 4.7948
2025-08-31 18:05:45 - pico-train - INFO - ├── Learning Rate: 9.75e-05
2025-08-31 18:05:45 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:06:39 - pico-train - INFO - Step 102700 -- 🔄 Training Metrics
2025-08-31 18:06:39 - pico-train - INFO - ├── Loss: 4.8006
2025-08-31 18:06:39 - pico-train - INFO - ├── Learning Rate: 9.73e-05
2025-08-31 18:06:39 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:07:33 - pico-train - INFO - Step 102800 -- 🔄 Training Metrics
2025-08-31 18:07:33 - pico-train - INFO - ├── Loss: 4.8049
2025-08-31 18:07:33 - pico-train - INFO - ├── Learning Rate: 9.71e-05
2025-08-31 18:07:33 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:08:27 - pico-train - INFO - Step 102900 -- 🔄 Training Metrics
2025-08-31 18:08:27 - pico-train - INFO - ├── Loss: 4.8086
2025-08-31 18:08:27 - pico-train - INFO - ├── Learning Rate: 9.70e-05
2025-08-31 18:08:27 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:09:21 - pico-train - INFO - Step 103000 -- 🔄 Training Metrics
2025-08-31 18:09:21 - pico-train - INFO - ├── Loss: 4.8154
2025-08-31 18:09:21 - pico-train - INFO - ├── Learning Rate: 9.68e-05
2025-08-31 18:09:21 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:10:15 - pico-train - INFO - Step 103100 -- 🔄 Training Metrics
2025-08-31 18:10:15 - pico-train - INFO - ├── Loss: 4.8232
2025-08-31 18:10:15 - pico-train - INFO - ├── Learning Rate: 9.67e-05
2025-08-31 18:10:15 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:11:10 - pico-train - INFO - Step 103200 -- 🔄 Training Metrics
2025-08-31 18:11:10 - pico-train - INFO - ├── Loss: 4.8032
2025-08-31 18:11:10 - pico-train - INFO - ├── Learning Rate: 9.65e-05
2025-08-31 18:11:10 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:12:05 - pico-train - INFO - Step 103300 -- 🔄 Training Metrics
2025-08-31 18:12:05 - pico-train - INFO - ├── Loss: 4.8157
2025-08-31 18:12:05 - pico-train - INFO - ├── Learning Rate: 9.64e-05
2025-08-31 18:12:05 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:13:00 - pico-train - INFO - Step 103400 -- 🔄 Training Metrics
2025-08-31 18:13:00 - pico-train - INFO - ├── Loss: 4.7903
2025-08-31 18:13:00 - pico-train - INFO - ├── Learning Rate: 9.62e-05
2025-08-31 18:13:00 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:13:54 - pico-train - INFO - Step 103500 -- 🔄 Training Metrics
2025-08-31 18:13:54 - pico-train - INFO - ├── Loss: 4.7786
2025-08-31 18:13:54 - pico-train - INFO - ├── Learning Rate: 9.60e-05
2025-08-31 18:13:54 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:14:48 - pico-train - INFO - Step 103600 -- 🔄 Training Metrics
2025-08-31 18:14:48 - pico-train - INFO - ├── Loss: 4.7962
2025-08-31 18:14:48 - pico-train - INFO - ├── Learning Rate: 9.59e-05
2025-08-31 18:14:48 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:15:43 - pico-train - INFO - Step 103700 -- 🔄 Training Metrics
2025-08-31 18:15:43 - pico-train - INFO - ├── Loss: 4.8097
2025-08-31 18:15:43 - pico-train - INFO - ├── Learning Rate: 9.57e-05
2025-08-31 18:15:43 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:16:37 - pico-train - INFO - Step 103800 -- 🔄 Training Metrics
2025-08-31 18:16:37 - pico-train - INFO - ├── Loss: 4.7613
2025-08-31 18:16:37 - pico-train - INFO - ├── Learning Rate: 9.56e-05
2025-08-31 18:16:37 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:17:31 - pico-train - INFO - Step 103900 -- 🔄 Training Metrics
2025-08-31 18:17:31 - pico-train - INFO - ├── Loss: 4.7992
2025-08-31 18:17:31 - pico-train - INFO - ├── Learning Rate: 9.54e-05
2025-08-31 18:17:31 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:18:25 - pico-train - INFO - Step 104000 -- 💾 Saving Checkpoint
2025-08-31 18:55:41 - pico-train - INFO - Step 104000 -- 📊 Evaluation Results
2025-08-31 18:55:41 - pico-train - INFO - └── paloma: inf
2025-08-31 18:55:41 - pico-train - INFO - Step 104000 -- 🔄 Training Metrics
2025-08-31 18:55:41 - pico-train - INFO - ├── Loss: 4.8021
2025-08-31 18:55:41 - pico-train - INFO - ├── Learning Rate: 9.52e-05
2025-08-31 18:55:41 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:55:41 - pico-train - INFO - Step 104000 -- 📈 Saving Learning Dynamics
2025-08-31 18:56:38 - pico-train - INFO - Step 104100 -- 🔄 Training Metrics
2025-08-31 18:56:38 - pico-train - INFO - ├── Loss: 4.7678
2025-08-31 18:56:38 - pico-train - INFO - ├── Learning Rate: 9.51e-05
2025-08-31 18:56:38 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:57:32 - pico-train - INFO - Step 104200 -- 🔄 Training Metrics
2025-08-31 18:57:32 - pico-train - INFO - ├── Loss: 4.7840
2025-08-31 18:57:32 - pico-train - INFO - ├── Learning Rate: 9.49e-05
2025-08-31 18:57:32 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:58:25 - pico-train - INFO - Step 104300 -- 🔄 Training Metrics
2025-08-31 18:58:25 - pico-train - INFO - ├── Loss: 4.7948
2025-08-31 18:58:25 - pico-train - INFO - ├── Learning Rate: 9.48e-05
2025-08-31 18:58:25 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 18:59:20 - pico-train - INFO - Step 104400 -- 🔄 Training Metrics
2025-08-31 18:59:20 - pico-train - INFO - ├── Loss: 4.7842
2025-08-31 18:59:20 - pico-train - INFO - ├── Learning Rate: 9.46e-05
2025-08-31 18:59:20 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:00:14 - pico-train - INFO - Step 104500 -- 🔄 Training Metrics
2025-08-31 19:00:14 - pico-train - INFO - ├── Loss: 4.7806
2025-08-31 19:00:14 - pico-train - INFO - ├── Learning Rate: 9.44e-05
2025-08-31 19:00:14 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:01:08 - pico-train - INFO - Step 104600 -- 🔄 Training Metrics
2025-08-31 19:01:08 - pico-train - INFO - ├── Loss: 4.8013
2025-08-31 19:01:08 - pico-train - INFO - ├── Learning Rate: 9.43e-05
2025-08-31 19:01:08 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:02:02 - pico-train - INFO - Step 104700 -- 🔄 Training Metrics
2025-08-31 19:02:02 - pico-train - INFO - ├── Loss: 4.7920
2025-08-31 19:02:02 - pico-train - INFO - ├── Learning Rate: 9.41e-05
2025-08-31 19:02:02 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:02:55 - pico-train - INFO - Step 104800 -- 🔄 Training Metrics
2025-08-31 19:02:55 - pico-train - INFO - ├── Loss: 4.7982
2025-08-31 19:02:55 - pico-train - INFO - ├── Learning Rate: 9.40e-05
2025-08-31 19:02:55 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:03:50 - pico-train - INFO - Step 104900 -- 🔄 Training Metrics
2025-08-31 19:03:50 - pico-train - INFO - ├── Loss: 4.7977
2025-08-31 19:03:50 - pico-train - INFO - ├── Learning Rate: 9.38e-05
2025-08-31 19:03:50 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:04:44 - pico-train - INFO - Step 105000 -- 🔄 Training Metrics
2025-08-31 19:04:44 - pico-train - INFO - ├── Loss: 4.7973
2025-08-31 19:04:44 - pico-train - INFO - ├── Learning Rate: 9.37e-05
2025-08-31 19:04:44 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:05:38 - pico-train - INFO - Step 105100 -- 🔄 Training Metrics
2025-08-31 19:05:38 - pico-train - INFO - ├── Loss: 4.7657
2025-08-31 19:05:38 - pico-train - INFO - ├── Learning Rate: 9.35e-05
2025-08-31 19:05:38 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:06:32 - pico-train - INFO - Step 105200 -- 🔄 Training Metrics
2025-08-31 19:06:32 - pico-train - INFO - ├── Loss: 4.7921
2025-08-31 19:06:32 - pico-train - INFO - ├── Learning Rate: 9.33e-05
2025-08-31 19:06:32 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:07:27 - pico-train - INFO - Step 105300 -- 🔄 Training Metrics
2025-08-31 19:07:27 - pico-train - INFO - ├── Loss: 4.7775
2025-08-31 19:07:27 - pico-train - INFO - ├── Learning Rate: 9.32e-05
2025-08-31 19:07:27 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:08:22 - pico-train - INFO - Step 105400 -- 🔄 Training Metrics
2025-08-31 19:08:22 - pico-train - INFO - ├── Loss: 4.7990
2025-08-31 19:08:22 - pico-train - INFO - ├── Learning Rate: 9.30e-05
2025-08-31 19:08:22 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:09:18 - pico-train - INFO - Step 105500 -- 🔄 Training Metrics
2025-08-31 19:09:18 - pico-train - INFO - ├── Loss: 4.7902
2025-08-31 19:09:18 - pico-train - INFO - ├── Learning Rate: 9.29e-05
2025-08-31 19:09:18 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:10:12 - pico-train - INFO - Step 105600 -- 🔄 Training Metrics
2025-08-31 19:10:12 - pico-train - INFO - ├── Loss: 4.7873
2025-08-31 19:10:12 - pico-train - INFO - ├── Learning Rate: 9.27e-05
2025-08-31 19:10:12 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:11:06 - pico-train - INFO - Step 105700 -- 🔄 Training Metrics
2025-08-31 19:11:06 - pico-train - INFO - ├── Loss: 4.7870
2025-08-31 19:11:06 - pico-train - INFO - ├── Learning Rate: 9.25e-05
2025-08-31 19:11:06 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:12:00 - pico-train - INFO - Step 105800 -- 🔄 Training Metrics
2025-08-31 19:12:00 - pico-train - INFO - ├── Loss: 4.7853
2025-08-31 19:12:00 - pico-train - INFO - ├── Learning Rate: 9.24e-05
2025-08-31 19:12:00 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:12:54 - pico-train - INFO - Step 105900 -- 🔄 Training Metrics
2025-08-31 19:12:54 - pico-train - INFO - ├── Loss: 4.7667
2025-08-31 19:12:54 - pico-train - INFO - ├── Learning Rate: 9.22e-05
2025-08-31 19:12:54 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:13:48 - pico-train - INFO - Step 106000 -- 💾 Saving Checkpoint
2025-08-31 19:52:30 - pico-train - INFO - Step 106000 -- 📊 Evaluation Results
2025-08-31 19:52:30 - pico-train - INFO - └── paloma: inf
2025-08-31 19:52:30 - pico-train - INFO - Step 106000 -- 🔄 Training Metrics
2025-08-31 19:52:30 - pico-train - INFO - ├── Loss: 4.7840
2025-08-31 19:52:30 - pico-train - INFO - ├── Learning Rate: 9.21e-05
2025-08-31 19:52:30 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:52:30 - pico-train - INFO - Step 106000 -- 📈 Saving Learning Dynamics
2025-08-31 19:53:27 - pico-train - INFO - Step 106100 -- 🔄 Training Metrics
2025-08-31 19:53:27 - pico-train - INFO - ├── Loss: 4.7948
2025-08-31 19:53:27 - pico-train - INFO - ├── Learning Rate: 9.19e-05
2025-08-31 19:53:27 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:54:22 - pico-train - INFO - Step 106200 -- 🔄 Training Metrics
2025-08-31 19:54:22 - pico-train - INFO - ├── Loss: 4.7926
2025-08-31 19:54:22 - pico-train - INFO - ├── Learning Rate: 9.18e-05
2025-08-31 19:54:22 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:55:18 - pico-train - INFO - Step 106300 -- 🔄 Training Metrics
2025-08-31 19:55:18 - pico-train - INFO - ├── Loss: 4.7992
2025-08-31 19:55:18 - pico-train - INFO - ├── Learning Rate: 9.16e-05
2025-08-31 19:55:18 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:56:13 - pico-train - INFO - Step 106400 -- 🔄 Training Metrics
2025-08-31 19:56:13 - pico-train - INFO - ├── Loss: 4.7733
2025-08-31 19:56:13 - pico-train - INFO - ├── Learning Rate: 9.14e-05
2025-08-31 19:56:13 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:57:08 - pico-train - INFO - Step 106500 -- 🔄 Training Metrics
2025-08-31 19:57:08 - pico-train - INFO - ├── Loss: 4.8039
2025-08-31 19:57:08 - pico-train - INFO - ├── Learning Rate: 9.13e-05
2025-08-31 19:57:08 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:58:03 - pico-train - INFO - Step 106600 -- 🔄 Training Metrics
2025-08-31 19:58:03 - pico-train - INFO - ├── Loss: 4.7913
2025-08-31 19:58:03 - pico-train - INFO - ├── Learning Rate: 9.11e-05
2025-08-31 19:58:03 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:58:58 - pico-train - INFO - Step 106700 -- 🔄 Training Metrics
2025-08-31 19:58:58 - pico-train - INFO - ├── Loss: 4.8052
2025-08-31 19:58:58 - pico-train - INFO - ├── Learning Rate: 9.10e-05
2025-08-31 19:58:58 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 19:59:54 - pico-train - INFO - Step 106800 -- 🔄 Training Metrics
2025-08-31 19:59:54 - pico-train - INFO - ├── Loss: 4.8040
2025-08-31 19:59:54 - pico-train - INFO - ├── Learning Rate: 9.08e-05
2025-08-31 19:59:54 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:00:49 - pico-train - INFO - Step 106900 -- 🔄 Training Metrics
2025-08-31 20:00:49 - pico-train - INFO - ├── Loss: 4.7788
2025-08-31 20:00:49 - pico-train - INFO - ├── Learning Rate: 9.07e-05
2025-08-31 20:00:49 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:01:44 - pico-train - INFO - Step 107000 -- 🔄 Training Metrics
2025-08-31 20:01:44 - pico-train - INFO - ├── Loss: 4.7795
2025-08-31 20:01:44 - pico-train - INFO - ├── Learning Rate: 9.05e-05
2025-08-31 20:01:44 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:02:39 - pico-train - INFO - Step 107100 -- 🔄 Training Metrics
2025-08-31 20:02:39 - pico-train - INFO - ├── Loss: 4.7627
2025-08-31 20:02:39 - pico-train - INFO - ├── Learning Rate: 9.03e-05
2025-08-31 20:02:39 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:03:34 - pico-train - INFO - Step 107200 -- 🔄 Training Metrics
2025-08-31 20:03:34 - pico-train - INFO - ├── Loss: 4.7699
2025-08-31 20:03:34 - pico-train - INFO - ├── Learning Rate: 9.02e-05
2025-08-31 20:03:34 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:04:29 - pico-train - INFO - Step 107300 -- 🔄 Training Metrics
2025-08-31 20:04:29 - pico-train - INFO - ├── Loss: 4.7992
2025-08-31 20:04:29 - pico-train - INFO - ├── Learning Rate: 9.00e-05
2025-08-31 20:04:29 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:05:25 - pico-train - INFO - Step 107400 -- 🔄 Training Metrics
2025-08-31 20:05:25 - pico-train - INFO - ├── Loss: 4.8061
2025-08-31 20:05:25 - pico-train - INFO - ├── Learning Rate: 8.99e-05
2025-08-31 20:05:25 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:06:20 - pico-train - INFO - Step 107500 -- 🔄 Training Metrics
2025-08-31 20:06:20 - pico-train - INFO - ├── Loss: 4.7807
2025-08-31 20:06:20 - pico-train - INFO - ├── Learning Rate: 8.97e-05
2025-08-31 20:06:20 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:07:15 - pico-train - INFO - Step 107600 -- 🔄 Training Metrics
2025-08-31 20:07:15 - pico-train - INFO - ├── Loss: 4.8012
2025-08-31 20:07:15 - pico-train - INFO - ├── Learning Rate: 8.95e-05
2025-08-31 20:07:15 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:08:11 - pico-train - INFO - Step 107700 -- 🔄 Training Metrics
2025-08-31 20:08:11 - pico-train - INFO - ├── Loss: 4.7822
2025-08-31 20:08:11 - pico-train - INFO - ├── Learning Rate: 8.94e-05
2025-08-31 20:08:11 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:09:06 - pico-train - INFO - Step 107800 -- 🔄 Training Metrics
2025-08-31 20:09:06 - pico-train - INFO - ├── Loss: 4.7630
2025-08-31 20:09:06 - pico-train - INFO - ├── Learning Rate: 8.92e-05
2025-08-31 20:09:06 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:10:01 - pico-train - INFO - Step 107900 -- 🔄 Training Metrics
2025-08-31 20:10:01 - pico-train - INFO - ├── Loss: 4.7928
2025-08-31 20:10:01 - pico-train - INFO - ├── Learning Rate: 8.91e-05
2025-08-31 20:10:01 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:10:56 - pico-train - INFO - Step 108000 -- 💾 Saving Checkpoint
2025-08-31 20:48:49 - pico-train - INFO - Step 108000 -- 📊 Evaluation Results
2025-08-31 20:48:49 - pico-train - INFO - └── paloma: inf
2025-08-31 20:48:50 - pico-train - INFO - Step 108000 -- 🔄 Training Metrics
2025-08-31 20:48:50 - pico-train - INFO - ├── Loss: 4.7865
2025-08-31 20:48:50 - pico-train - INFO - ├── Learning Rate: 8.89e-05
2025-08-31 20:48:50 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:48:50 - pico-train - INFO - Step 108000 -- 📈 Saving Learning Dynamics
2025-08-31 20:49:48 - pico-train - INFO - Step 108100 -- 🔄 Training Metrics
2025-08-31 20:49:48 - pico-train - INFO - ├── Loss: 4.7775
2025-08-31 20:49:48 - pico-train - INFO - ├── Learning Rate: 8.88e-05
2025-08-31 20:49:48 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:50:43 - pico-train - INFO - Step 108200 -- 🔄 Training Metrics
2025-08-31 20:50:43 - pico-train - INFO - ├── Loss: 4.7740
2025-08-31 20:50:43 - pico-train - INFO - ├── Learning Rate: 8.86e-05
2025-08-31 20:50:43 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:51:38 - pico-train - INFO - Step 108300 -- 🔄 Training Metrics
2025-08-31 20:51:38 - pico-train - INFO - ├── Loss: 4.7898
2025-08-31 20:51:38 - pico-train - INFO - ├── Learning Rate: 8.84e-05
2025-08-31 20:51:38 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:52:35 - pico-train - INFO - Step 108400 -- 🔄 Training Metrics
2025-08-31 20:52:35 - pico-train - INFO - ├── Loss: 4.8049
2025-08-31 20:52:35 - pico-train - INFO - ├── Learning Rate: 8.83e-05
2025-08-31 20:52:35 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:53:30 - pico-train - INFO - Step 108500 -- 🔄 Training Metrics
2025-08-31 20:53:30 - pico-train - INFO - ├── Loss: 4.7662
2025-08-31 20:53:30 - pico-train - INFO - ├── Learning Rate: 8.81e-05
2025-08-31 20:53:30 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:54:25 - pico-train - INFO - Step 108600 -- 🔄 Training Metrics
2025-08-31 20:54:25 - pico-train - INFO - ├── Loss: 4.7673
2025-08-31 20:54:25 - pico-train - INFO - ├── Learning Rate: 8.80e-05
2025-08-31 20:54:25 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:55:21 - pico-train - INFO - Step 108700 -- 🔄 Training Metrics
2025-08-31 20:55:21 - pico-train - INFO - ├── Loss: 4.8010
2025-08-31 20:55:21 - pico-train - INFO - ├── Learning Rate: 8.78e-05
2025-08-31 20:55:21 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:56:17 - pico-train - INFO - Step 108800 -- 🔄 Training Metrics
2025-08-31 20:56:17 - pico-train - INFO - ├── Loss: 4.7886
2025-08-31 20:56:17 - pico-train - INFO - ├── Learning Rate: 8.77e-05
2025-08-31 20:56:17 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:57:11 - pico-train - INFO - Step 108900 -- 🔄 Training Metrics
2025-08-31 20:57:11 - pico-train - INFO - ├── Loss: 4.7790
2025-08-31 20:57:11 - pico-train - INFO - ├── Learning Rate: 8.75e-05
2025-08-31 20:57:11 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:58:07 - pico-train - INFO - Step 109000 -- 🔄 Training Metrics
2025-08-31 20:58:07 - pico-train - INFO - ├── Loss: 4.7760
2025-08-31 20:58:07 - pico-train - INFO - ├── Learning Rate: 8.73e-05
2025-08-31 20:58:07 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:59:02 - pico-train - INFO - Step 109100 -- 🔄 Training Metrics
2025-08-31 20:59:02 - pico-train - INFO - ├── Loss: 4.7922
2025-08-31 20:59:02 - pico-train - INFO - ├── Learning Rate: 8.72e-05
2025-08-31 20:59:02 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 20:59:57 - pico-train - INFO - Step 109200 -- 🔄 Training Metrics
2025-08-31 20:59:57 - pico-train - INFO - ├── Loss: 4.7920
2025-08-31 20:59:57 - pico-train - INFO - ├── Learning Rate: 8.70e-05
2025-08-31 20:59:57 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:00:52 - pico-train - INFO - Step 109300 -- 🔄 Training Metrics
2025-08-31 21:00:52 - pico-train - INFO - ├── Loss: 4.8020
2025-08-31 21:00:52 - pico-train - INFO - ├── Learning Rate: 8.69e-05
2025-08-31 21:00:52 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:01:47 - pico-train - INFO - Step 109400 -- 🔄 Training Metrics
2025-08-31 21:01:47 - pico-train - INFO - ├── Loss: 4.7930
2025-08-31 21:01:47 - pico-train - INFO - ├── Learning Rate: 8.67e-05
2025-08-31 21:01:47 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:02:42 - pico-train - INFO - Step 109500 -- 🔄 Training Metrics
2025-08-31 21:02:42 - pico-train - INFO - ├── Loss: 4.8030
2025-08-31 21:02:42 - pico-train - INFO - ├── Learning Rate: 8.66e-05
2025-08-31 21:02:42 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:03:37 - pico-train - INFO - Step 109600 -- 🔄 Training Metrics
2025-08-31 21:03:37 - pico-train - INFO - ├── Loss: 4.7972
2025-08-31 21:03:37 - pico-train - INFO - ├── Learning Rate: 8.64e-05
2025-08-31 21:03:37 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:04:33 - pico-train - INFO - Step 109700 -- 🔄 Training Metrics
2025-08-31 21:04:33 - pico-train - INFO - ├── Loss: 4.7616
2025-08-31 21:04:33 - pico-train - INFO - ├── Learning Rate: 8.62e-05
2025-08-31 21:04:33 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:05:28 - pico-train - INFO - Step 109800 -- 🔄 Training Metrics
2025-08-31 21:05:28 - pico-train - INFO - ├── Loss: 4.7960
2025-08-31 21:05:28 - pico-train - INFO - ├── Learning Rate: 8.61e-05
2025-08-31 21:05:28 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:06:23 - pico-train - INFO - Step 109900 -- 🔄 Training Metrics
2025-08-31 21:06:23 - pico-train - INFO - ├── Loss: 4.7931
2025-08-31 21:06:23 - pico-train - INFO - ├── Learning Rate: 8.59e-05
2025-08-31 21:06:23 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:07:17 - pico-train - INFO - Step 110000 -- 💾 Saving Checkpoint
2025-08-31 21:46:40 - pico-train - INFO - Step 110000 -- 📊 Evaluation Results
2025-08-31 21:46:40 - pico-train - INFO - └── paloma: inf
2025-08-31 21:46:40 - pico-train - INFO - Step 110000 -- 🔄 Training Metrics
2025-08-31 21:46:40 - pico-train - INFO - ├── Loss: 4.7744
2025-08-31 21:46:40 - pico-train - INFO - ├── Learning Rate: 8.58e-05
2025-08-31 21:46:40 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:46:40 - pico-train - INFO - Step 110000 -- 📈 Saving Learning Dynamics
2025-08-31 21:47:38 - pico-train - INFO - Step 110100 -- 🔄 Training Metrics
2025-08-31 21:47:38 - pico-train - INFO - ├── Loss: 4.7719
2025-08-31 21:47:38 - pico-train - INFO - ├── Learning Rate: 8.56e-05
2025-08-31 21:47:38 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:48:33 - pico-train - INFO - Step 110200 -- 🔄 Training Metrics
2025-08-31 21:48:33 - pico-train - INFO - ├── Loss: 4.7851
2025-08-31 21:48:33 - pico-train - INFO - ├── Learning Rate: 8.55e-05
2025-08-31 21:48:33 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:49:28 - pico-train - INFO - Step 110300 -- 🔄 Training Metrics
2025-08-31 21:49:28 - pico-train - INFO - ├── Loss: 4.7918
2025-08-31 21:49:28 - pico-train - INFO - ├── Learning Rate: 8.53e-05
2025-08-31 21:49:28 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:50:23 - pico-train - INFO - Step 110400 -- 🔄 Training Metrics
2025-08-31 21:50:23 - pico-train - INFO - ├── Loss: 4.7676
2025-08-31 21:50:23 - pico-train - INFO - ├── Learning Rate: 8.51e-05
2025-08-31 21:50:23 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:51:18 - pico-train - INFO - Step 110500 -- 🔄 Training Metrics
2025-08-31 21:51:18 - pico-train - INFO - ├── Loss: 4.7964
2025-08-31 21:51:18 - pico-train - INFO - ├── Learning Rate: 8.50e-05
2025-08-31 21:51:18 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:52:14 - pico-train - INFO - Step 110600 -- 🔄 Training Metrics
2025-08-31 21:52:14 - pico-train - INFO - ├── Loss: 4.7715
2025-08-31 21:52:14 - pico-train - INFO - ├── Learning Rate: 8.48e-05
2025-08-31 21:52:14 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:53:09 - pico-train - INFO - Step 110700 -- 🔄 Training Metrics
2025-08-31 21:53:09 - pico-train - INFO - ├── Loss: 4.7874
2025-08-31 21:53:09 - pico-train - INFO - ├── Learning Rate: 8.47e-05
2025-08-31 21:53:09 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:54:04 - pico-train - INFO - Step 110800 -- 🔄 Training Metrics
2025-08-31 21:54:04 - pico-train - INFO - ├── Loss: 4.7858
2025-08-31 21:54:04 - pico-train - INFO - ├── Learning Rate: 8.45e-05
2025-08-31 21:54:04 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:54:59 - pico-train - INFO - Step 110900 -- 🔄 Training Metrics
2025-08-31 21:54:59 - pico-train - INFO - ├── Loss: 4.8073
2025-08-31 21:54:59 - pico-train - INFO - ├── Learning Rate: 8.44e-05
2025-08-31 21:54:59 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:55:55 - pico-train - INFO - Step 111000 -- 🔄 Training Metrics
2025-08-31 21:55:55 - pico-train - INFO - ├── Loss: 4.7721
2025-08-31 21:55:55 - pico-train - INFO - ├── Learning Rate: 8.42e-05
2025-08-31 21:55:55 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:56:50 - pico-train - INFO - Step 111100 -- 🔄 Training Metrics
2025-08-31 21:56:50 - pico-train - INFO - ├── Loss: 4.7713
2025-08-31 21:56:50 - pico-train - INFO - ├── Learning Rate: 8.40e-05
2025-08-31 21:56:50 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:57:45 - pico-train - INFO - Step 111200 -- 🔄 Training Metrics
2025-08-31 21:57:45 - pico-train - INFO - ├── Loss: 4.7687
2025-08-31 21:57:45 - pico-train - INFO - ├── Learning Rate: 8.39e-05
2025-08-31 21:57:45 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:58:40 - pico-train - INFO - Step 111300 -- 🔄 Training Metrics
2025-08-31 21:58:40 - pico-train - INFO - ├── Loss: 4.7715
2025-08-31 21:58:40 - pico-train - INFO - ├── Learning Rate: 8.37e-05
2025-08-31 21:58:40 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 21:59:36 - pico-train - INFO - Step 111400 -- 🔄 Training Metrics
2025-08-31 21:59:36 - pico-train - INFO - ├── Loss: 4.7610
2025-08-31 21:59:36 - pico-train - INFO - ├── Learning Rate: 8.36e-05
2025-08-31 21:59:36 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:00:31 - pico-train - INFO - Step 111500 -- 🔄 Training Metrics
2025-08-31 22:00:31 - pico-train - INFO - ├── Loss: 4.7668
2025-08-31 22:00:31 - pico-train - INFO - ├── Learning Rate: 8.34e-05
2025-08-31 22:00:31 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:01:26 - pico-train - INFO - Step 111600 -- 🔄 Training Metrics
2025-08-31 22:01:26 - pico-train - INFO - ├── Loss: 4.7816
2025-08-31 22:01:26 - pico-train - INFO - ├── Learning Rate: 8.33e-05
2025-08-31 22:01:26 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:02:21 - pico-train - INFO - Step 111700 -- 🔄 Training Metrics
2025-08-31 22:02:21 - pico-train - INFO - ├── Loss: 4.8025
2025-08-31 22:02:21 - pico-train - INFO - ├── Learning Rate: 8.31e-05
2025-08-31 22:02:21 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:03:16 - pico-train - INFO - Step 111800 -- 🔄 Training Metrics
2025-08-31 22:03:16 - pico-train - INFO - ├── Loss: 4.7870
2025-08-31 22:03:16 - pico-train - INFO - ├── Learning Rate: 8.29e-05
2025-08-31 22:03:16 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:04:12 - pico-train - INFO - Step 111900 -- 🔄 Training Metrics
2025-08-31 22:04:12 - pico-train - INFO - ├── Loss: 4.7972
2025-08-31 22:04:12 - pico-train - INFO - ├── Learning Rate: 8.28e-05
2025-08-31 22:04:12 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:05:06 - pico-train - INFO - Step 112000 -- 💾 Saving Checkpoint
2025-08-31 22:43:48 - pico-train - INFO - Step 112000 -- 📊 Evaluation Results
2025-08-31 22:43:48 - pico-train - INFO - └── paloma: inf
2025-08-31 22:43:49 - pico-train - INFO - Step 112000 -- 🔄 Training Metrics
2025-08-31 22:43:49 - pico-train - INFO - ├── Loss: 4.7665
2025-08-31 22:43:49 - pico-train - INFO - ├── Learning Rate: 8.26e-05
2025-08-31 22:43:49 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:43:49 - pico-train - INFO - Step 112000 -- 📈 Saving Learning Dynamics
2025-08-31 22:44:45 - pico-train - INFO - Step 112100 -- 🔄 Training Metrics
2025-08-31 22:44:45 - pico-train - INFO - ├── Loss: 4.7908
2025-08-31 22:44:45 - pico-train - INFO - ├── Learning Rate: 8.25e-05
2025-08-31 22:44:45 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:45:39 - pico-train - INFO - Step 112200 -- 🔄 Training Metrics
2025-08-31 22:45:39 - pico-train - INFO - ├── Loss: 4.7975
2025-08-31 22:45:39 - pico-train - INFO - ├── Learning Rate: 8.23e-05
2025-08-31 22:45:39 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:46:33 - pico-train - INFO - Step 112300 -- 🔄 Training Metrics
2025-08-31 22:46:33 - pico-train - INFO - ├── Loss: 4.7640
2025-08-31 22:46:33 - pico-train - INFO - ├── Learning Rate: 8.22e-05
2025-08-31 22:46:33 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:47:27 - pico-train - INFO - Step 112400 -- 🔄 Training Metrics
2025-08-31 22:47:27 - pico-train - INFO - ├── Loss: 4.7924
2025-08-31 22:47:27 - pico-train - INFO - ├── Learning Rate: 8.20e-05
2025-08-31 22:47:27 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:48:21 - pico-train - INFO - Step 112500 -- 🔄 Training Metrics
2025-08-31 22:48:21 - pico-train - INFO - ├── Loss: 4.7599
2025-08-31 22:48:21 - pico-train - INFO - ├── Learning Rate: 8.19e-05
2025-08-31 22:48:21 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:49:15 - pico-train - INFO - Step 112600 -- 🔄 Training Metrics
2025-08-31 22:49:15 - pico-train - INFO - ├── Loss: 4.7767
2025-08-31 22:49:15 - pico-train - INFO - ├── Learning Rate: 8.17e-05
2025-08-31 22:49:15 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:50:10 - pico-train - INFO - Step 112700 -- 🔄 Training Metrics
2025-08-31 22:50:10 - pico-train - INFO - ├── Loss: 4.7574
2025-08-31 22:50:10 - pico-train - INFO - ├── Learning Rate: 8.15e-05
2025-08-31 22:50:10 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:51:04 - pico-train - INFO - Step 112800 -- 🔄 Training Metrics
2025-08-31 22:51:04 - pico-train - INFO - ├── Loss: 4.7592
2025-08-31 22:51:04 - pico-train - INFO - ├── Learning Rate: 8.14e-05
2025-08-31 22:51:04 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:51:58 - pico-train - INFO - Step 112900 -- 🔄 Training Metrics
2025-08-31 22:51:58 - pico-train - INFO - ├── Loss: 4.7704
2025-08-31 22:51:58 - pico-train - INFO - ├── Learning Rate: 8.12e-05
2025-08-31 22:51:58 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:52:51 - pico-train - INFO - Step 113000 -- 🔄 Training Metrics
2025-08-31 22:52:51 - pico-train - INFO - ├── Loss: 4.7620
2025-08-31 22:52:51 - pico-train - INFO - ├── Learning Rate: 8.11e-05
2025-08-31 22:52:51 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:53:46 - pico-train - INFO - Step 113100 -- 🔄 Training Metrics
2025-08-31 22:53:46 - pico-train - INFO - ├── Loss: 4.7904
2025-08-31 22:53:46 - pico-train - INFO - ├── Learning Rate: 8.09e-05
2025-08-31 22:53:46 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:54:40 - pico-train - INFO - Step 113200 -- 🔄 Training Metrics
2025-08-31 22:54:40 - pico-train - INFO - ├── Loss: 4.7851
2025-08-31 22:54:40 - pico-train - INFO - ├── Learning Rate: 8.08e-05
2025-08-31 22:54:40 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:55:34 - pico-train - INFO - Step 113300 -- 🔄 Training Metrics
2025-08-31 22:55:34 - pico-train - INFO - ├── Loss: 4.7766
2025-08-31 22:55:34 - pico-train - INFO - ├── Learning Rate: 8.06e-05
2025-08-31 22:55:34 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:56:28 - pico-train - INFO - Step 113400 -- 🔄 Training Metrics
2025-08-31 22:56:28 - pico-train - INFO - ├── Loss: 4.7659
2025-08-31 22:56:28 - pico-train - INFO - ├── Learning Rate: 8.05e-05
2025-08-31 22:56:28 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:57:22 - pico-train - INFO - Step 113500 -- 🔄 Training Metrics
2025-08-31 22:57:22 - pico-train - INFO - ├── Loss: 4.7723
2025-08-31 22:57:22 - pico-train - INFO - ├── Learning Rate: 8.03e-05
2025-08-31 22:57:22 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:58:16 - pico-train - INFO - Step 113600 -- 🔄 Training Metrics
2025-08-31 22:58:16 - pico-train - INFO - ├── Loss: 4.7792
2025-08-31 22:58:16 - pico-train - INFO - ├── Learning Rate: 8.01e-05
2025-08-31 22:58:16 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 22:59:10 - pico-train - INFO - Step 113700 -- 🔄 Training Metrics
2025-08-31 22:59:10 - pico-train - INFO - ├── Loss: 4.7774
2025-08-31 22:59:10 - pico-train - INFO - ├── Learning Rate: 8.00e-05
2025-08-31 22:59:10 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:00:05 - pico-train - INFO - Step 113800 -- 🔄 Training Metrics
2025-08-31 23:00:05 - pico-train - INFO - ├── Loss: 4.7757
2025-08-31 23:00:05 - pico-train - INFO - ├── Learning Rate: 7.98e-05
2025-08-31 23:00:05 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:01:00 - pico-train - INFO - Step 113900 -- 🔄 Training Metrics
2025-08-31 23:01:00 - pico-train - INFO - ├── Loss: 4.7841
2025-08-31 23:01:00 - pico-train - INFO - ├── Learning Rate: 7.97e-05
2025-08-31 23:01:00 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:01:55 - pico-train - INFO - Step 114000 -- 💾 Saving Checkpoint
2025-08-31 23:42:10 - pico-train - INFO - Step 114000 -- 📊 Evaluation Results
2025-08-31 23:42:10 - pico-train - INFO - └── paloma: inf
2025-08-31 23:42:11 - pico-train - INFO - Step 114000 -- 🔄 Training Metrics
2025-08-31 23:42:11 - pico-train - INFO - ├── Loss: 4.7682
2025-08-31 23:42:11 - pico-train - INFO - ├── Learning Rate: 7.95e-05
2025-08-31 23:42:11 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:42:11 - pico-train - INFO - Step 114000 -- 📈 Saving Learning Dynamics
2025-08-31 23:43:07 - pico-train - INFO - Step 114100 -- 🔄 Training Metrics
2025-08-31 23:43:07 - pico-train - INFO - ├── Loss: 4.7732
2025-08-31 23:43:07 - pico-train - INFO - ├── Learning Rate: 7.94e-05
2025-08-31 23:43:07 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:44:01 - pico-train - INFO - Step 114200 -- 🔄 Training Metrics
2025-08-31 23:44:01 - pico-train - INFO - ├── Loss: 4.7557
2025-08-31 23:44:01 - pico-train - INFO - ├── Learning Rate: 7.92e-05
2025-08-31 23:44:01 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:44:55 - pico-train - INFO - Step 114300 -- 🔄 Training Metrics
2025-08-31 23:44:55 - pico-train - INFO - ├── Loss: 4.7418
2025-08-31 23:44:55 - pico-train - INFO - ├── Learning Rate: 7.91e-05
2025-08-31 23:44:55 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:45:49 - pico-train - INFO - Step 114400 -- 🔄 Training Metrics
2025-08-31 23:45:49 - pico-train - INFO - ├── Loss: 4.7622
2025-08-31 23:45:49 - pico-train - INFO - ├── Learning Rate: 7.89e-05
2025-08-31 23:45:49 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:46:44 - pico-train - INFO - Step 114500 -- 🔄 Training Metrics
2025-08-31 23:46:44 - pico-train - INFO - ├── Loss: 4.7745
2025-08-31 23:46:44 - pico-train - INFO - ├── Learning Rate: 7.87e-05
2025-08-31 23:46:44 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:47:38 - pico-train - INFO - Step 114600 -- 🔄 Training Metrics
2025-08-31 23:47:38 - pico-train - INFO - ├── Loss: 4.7669
2025-08-31 23:47:38 - pico-train - INFO - ├── Learning Rate: 7.86e-05
2025-08-31 23:47:38 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:48:32 - pico-train - INFO - Step 114700 -- 🔄 Training Metrics
2025-08-31 23:48:32 - pico-train - INFO - ├── Loss: 4.7675
2025-08-31 23:48:32 - pico-train - INFO - ├── Learning Rate: 7.84e-05
2025-08-31 23:48:32 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:49:26 - pico-train - INFO - Step 114800 -- 🔄 Training Metrics
2025-08-31 23:49:26 - pico-train - INFO - ├── Loss: 4.7752
2025-08-31 23:49:26 - pico-train - INFO - ├── Learning Rate: 7.83e-05
2025-08-31 23:49:26 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:50:20 - pico-train - INFO - Step 114900 -- 🔄 Training Metrics
2025-08-31 23:50:20 - pico-train - INFO - ├── Loss: 4.7719
2025-08-31 23:50:20 - pico-train - INFO - ├── Learning Rate: 7.81e-05
2025-08-31 23:50:20 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:51:14 - pico-train - INFO - Step 115000 -- 🔄 Training Metrics
2025-08-31 23:51:14 - pico-train - INFO - ├── Loss: 4.7845
2025-08-31 23:51:14 - pico-train - INFO - ├── Learning Rate: 7.80e-05
2025-08-31 23:51:14 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:52:10 - pico-train - INFO - Step 115100 -- 🔄 Training Metrics
2025-08-31 23:52:10 - pico-train - INFO - ├── Loss: 4.7656
2025-08-31 23:52:10 - pico-train - INFO - ├── Learning Rate: 7.78e-05
2025-08-31 23:52:10 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:53:05 - pico-train - INFO - Step 115200 -- 🔄 Training Metrics
2025-08-31 23:53:05 - pico-train - INFO - ├── Loss: 4.7658
2025-08-31 23:53:05 - pico-train - INFO - ├── Learning Rate: 7.77e-05
2025-08-31 23:53:05 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:54:00 - pico-train - INFO - Step 115300 -- 🔄 Training Metrics
2025-08-31 23:54:00 - pico-train - INFO - ├── Loss: 4.7655
2025-08-31 23:54:00 - pico-train - INFO - ├── Learning Rate: 7.75e-05
2025-08-31 23:54:00 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:54:55 - pico-train - INFO - Step 115400 -- 🔄 Training Metrics
2025-08-31 23:54:55 - pico-train - INFO - ├── Loss: 4.7857
2025-08-31 23:54:55 - pico-train - INFO - ├── Learning Rate: 7.74e-05
2025-08-31 23:54:55 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:55:50 - pico-train - INFO - Step 115500 -- 🔄 Training Metrics
2025-08-31 23:55:50 - pico-train - INFO - ├── Loss: 4.7564
2025-08-31 23:55:50 - pico-train - INFO - ├── Learning Rate: 7.72e-05
2025-08-31 23:55:50 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:56:45 - pico-train - INFO - Step 115600 -- 🔄 Training Metrics
2025-08-31 23:56:45 - pico-train - INFO - ├── Loss: 4.7920
2025-08-31 23:56:45 - pico-train - INFO - ├── Learning Rate: 7.70e-05
2025-08-31 23:56:45 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:57:40 - pico-train - INFO - Step 115700 -- 🔄 Training Metrics
2025-08-31 23:57:40 - pico-train - INFO - ├── Loss: 4.7652
2025-08-31 23:57:40 - pico-train - INFO - ├── Learning Rate: 7.69e-05
2025-08-31 23:57:40 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:58:35 - pico-train - INFO - Step 115800 -- 🔄 Training Metrics
2025-08-31 23:58:35 - pico-train - INFO - ├── Loss: 4.7682
2025-08-31 23:58:35 - pico-train - INFO - ├── Learning Rate: 7.67e-05
2025-08-31 23:58:35 - pico-train - INFO - └── Inf/NaN count: 0
2025-08-31 23:59:30 - pico-train - INFO - Step 115900 -- 🔄 Training Metrics
2025-08-31 23:59:30 - pico-train - INFO - ├── Loss: 4.7581
2025-08-31 23:59:30 - pico-train - INFO - ├── Learning Rate: 7.66e-05
2025-08-31 23:59:30 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:00:25 - pico-train - INFO - Step 116000 -- 💾 Saving Checkpoint
2025-09-01 00:38:43 - pico-train - INFO - Step 116000 -- 📊 Evaluation Results
2025-09-01 00:38:43 - pico-train - INFO - └── paloma: inf
2025-09-01 00:38:44 - pico-train - INFO - Step 116000 -- 🔄 Training Metrics
2025-09-01 00:38:44 - pico-train - INFO - ├── Loss: 4.7403
2025-09-01 00:38:44 - pico-train - INFO - ├── Learning Rate: 7.64e-05
2025-09-01 00:38:44 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:38:44 - pico-train - INFO - Step 116000 -- 📈 Saving Learning Dynamics
2025-09-01 00:39:41 - pico-train - INFO - Step 116100 -- 🔄 Training Metrics
2025-09-01 00:39:41 - pico-train - INFO - ├── Loss: 4.7747
2025-09-01 00:39:41 - pico-train - INFO - ├── Learning Rate: 7.63e-05
2025-09-01 00:39:41 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:40:35 - pico-train - INFO - Step 116200 -- 🔄 Training Metrics
2025-09-01 00:40:35 - pico-train - INFO - ├── Loss: 4.7540
2025-09-01 00:40:35 - pico-train - INFO - ├── Learning Rate: 7.61e-05
2025-09-01 00:40:35 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:41:28 - pico-train - INFO - Step 116300 -- 🔄 Training Metrics
2025-09-01 00:41:28 - pico-train - INFO - ├── Loss: 4.7835
2025-09-01 00:41:28 - pico-train - INFO - ├── Learning Rate: 7.60e-05
2025-09-01 00:41:28 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:42:23 - pico-train - INFO - Step 116400 -- 🔄 Training Metrics
2025-09-01 00:42:23 - pico-train - INFO - ├── Loss: 4.7547
2025-09-01 00:42:23 - pico-train - INFO - ├── Learning Rate: 7.58e-05
2025-09-01 00:42:23 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:43:17 - pico-train - INFO - Step 116500 -- 🔄 Training Metrics
2025-09-01 00:43:17 - pico-train - INFO - ├── Loss: 4.7698
2025-09-01 00:43:17 - pico-train - INFO - ├── Learning Rate: 7.57e-05
2025-09-01 00:43:17 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:44:11 - pico-train - INFO - Step 116600 -- 🔄 Training Metrics
2025-09-01 00:44:11 - pico-train - INFO - ├── Loss: 4.7806
2025-09-01 00:44:11 - pico-train - INFO - ├── Learning Rate: 7.55e-05
2025-09-01 00:44:11 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:45:05 - pico-train - INFO - Step 116700 -- 🔄 Training Metrics
2025-09-01 00:45:05 - pico-train - INFO - ├── Loss: 4.7499
2025-09-01 00:45:05 - pico-train - INFO - ├── Learning Rate: 7.53e-05
2025-09-01 00:45:05 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:45:59 - pico-train - INFO - Step 116800 -- 🔄 Training Metrics
2025-09-01 00:45:59 - pico-train - INFO - ├── Loss: 4.7724
2025-09-01 00:45:59 - pico-train - INFO - ├── Learning Rate: 7.52e-05
2025-09-01 00:45:59 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:46:54 - pico-train - INFO - Step 116900 -- 🔄 Training Metrics
2025-09-01 00:46:54 - pico-train - INFO - ├── Loss: 4.7790
2025-09-01 00:46:54 - pico-train - INFO - ├── Learning Rate: 7.50e-05
2025-09-01 00:46:54 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:48:04 - pico-train - INFO - Step 117000 -- 🔄 Training Metrics
2025-09-01 00:48:04 - pico-train - INFO - ├── Loss: 4.7729
2025-09-01 00:48:04 - pico-train - INFO - ├── Learning Rate: 7.49e-05
2025-09-01 00:48:04 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:49:16 - pico-train - INFO - Step 117100 -- 🔄 Training Metrics
2025-09-01 00:49:16 - pico-train - INFO - ├── Loss: 4.7553
2025-09-01 00:49:16 - pico-train - INFO - ├── Learning Rate: 7.47e-05
2025-09-01 00:49:16 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:50:30 - pico-train - INFO - Step 117200 -- 🔄 Training Metrics
2025-09-01 00:50:30 - pico-train - INFO - ├── Loss: 4.7685
2025-09-01 00:50:30 - pico-train - INFO - ├── Learning Rate: 7.46e-05
2025-09-01 00:50:30 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:51:43 - pico-train - INFO - Step 117300 -- 🔄 Training Metrics
2025-09-01 00:51:43 - pico-train - INFO - ├── Loss: 4.7522
2025-09-01 00:51:43 - pico-train - INFO - ├── Learning Rate: 7.44e-05
2025-09-01 00:51:43 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:52:56 - pico-train - INFO - Step 117400 -- 🔄 Training Metrics
2025-09-01 00:52:56 - pico-train - INFO - ├── Loss: 4.7770
2025-09-01 00:52:56 - pico-train - INFO - ├── Learning Rate: 7.43e-05
2025-09-01 00:52:56 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:54:05 - pico-train - INFO - Step 117500 -- 🔄 Training Metrics
2025-09-01 00:54:05 - pico-train - INFO - ├── Loss: 4.7848
2025-09-01 00:54:05 - pico-train - INFO - ├── Learning Rate: 7.41e-05
2025-09-01 00:54:05 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:55:00 - pico-train - INFO - Step 117600 -- 🔄 Training Metrics
2025-09-01 00:55:00 - pico-train - INFO - ├── Loss: 4.7610
2025-09-01 00:55:00 - pico-train - INFO - ├── Learning Rate: 7.40e-05
2025-09-01 00:55:00 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:55:56 - pico-train - INFO - Step 117700 -- 🔄 Training Metrics
2025-09-01 00:55:56 - pico-train - INFO - ├── Loss: 4.7598
2025-09-01 00:55:56 - pico-train - INFO - ├── Learning Rate: 7.38e-05
2025-09-01 00:55:56 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:56:52 - pico-train - INFO - Step 117800 -- 🔄 Training Metrics
2025-09-01 00:56:52 - pico-train - INFO - ├── Loss: 4.7555
2025-09-01 00:56:52 - pico-train - INFO - ├── Learning Rate: 7.37e-05
2025-09-01 00:56:52 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:57:47 - pico-train - INFO - Step 117900 -- 🔄 Training Metrics
2025-09-01 00:57:47 - pico-train - INFO - ├── Loss: 4.7664
2025-09-01 00:57:47 - pico-train - INFO - ├── Learning Rate: 7.35e-05
2025-09-01 00:57:47 - pico-train - INFO - └── Inf/NaN count: 0
2025-09-01 00:58:41 - pico-train - INFO - Step 118000 -- 💾 Saving Checkpoint