0: W0816 22:51:23.859000 1085788 torch/distributed/run.py:792] 
0: W0816 22:51:23.859000 1085788 torch/distributed/run.py:792] *****************************************
0: W0816 22:51:23.859000 1085788 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
0: W0816 22:51:23.859000 1085788 torch/distributed/run.py:792] *****************************************
3: W0816 22:51:23.859000 2505549 torch/distributed/run.py:792] 
3: W0816 22:51:23.859000 2505549 torch/distributed/run.py:792] *****************************************
3: W0816 22:51:23.859000 2505549 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
3: W0816 22:51:23.859000 2505549 torch/distributed/run.py:792] *****************************************
1: W0816 22:51:23.859000 2511227 torch/distributed/run.py:792] 
1: W0816 22:51:23.859000 2511227 torch/distributed/run.py:792] *****************************************
1: W0816 22:51:23.859000 2511227 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
1: W0816 22:51:23.859000 2511227 torch/distributed/run.py:792] *****************************************
2: W0816 22:51:23.859000 1864 torch/distributed/run.py:792] 
2: W0816 22:51:23.859000 1864 torch/distributed/run.py:792] *****************************************
2: W0816 22:51:23.859000 1864 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
2: W0816 22:51:23.859000 1864 torch/distributed/run.py:792] *****************************************
2: [2025-08-16 22:53:47,649] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:2076] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
2: [2025-08-16 22:53:47,649] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:2076] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
3: [2025-08-16 22:53:47,649] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:2505632] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
0: [2025-08-16 22:53:47,650] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:1085867] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
0: [2025-08-16 22:53:47,650] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:1085867] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
3: [2025-08-16 22:53:47,650] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:2505632] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
1: [2025-08-16 22:53:47,650] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:2511304] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
1: [2025-08-16 22:53:47,650] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:2511304] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
0: [2025-08-16 22:54:03,185] [INFO] [axolotl.cli.config.load_cfg:244] [PID:1085867] [RANK:0] config:
0: {
0:   "activation_offloading": false,
0:   "auto_resume_from_checkpoints": true,
0:   "axolotl_config_path": "/lustre/fswork/projects/rech/dgo/udv55np/train/tmp/1755377434871738899.yaml",
0:   "base_model": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-14B",
0:   "base_model_config": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-14B",
0:   "batch_size": 16,
0:   "bf16": true,
0:   "capabilities": {
0:     "bf16": true,
0:     "compute_capability": "sm_90",
0:     "fp8": false,
0:     "n_gpu": 16,
0:     "n_node": 1
0:   },
0:   "chat_template": "qwen_25",
0:   "dataloader_num_workers": 16,
0:   "dataloader_pin_memory": true,
0:   "dataloader_prefetch_factor": 256,
0:   "dataset_prepared_path": "/lustre/fsn1/projects/rech/dgo/udv55np/dataset/Qwen3-235B-A22B/Qwen2.5-14B/1",
0:   "dataset_processes": 192,
0:   "datasets": [
0:     {
0:       "chat_template": "tokenizer_default",
0:       "data_files": [
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0007.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0009.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0005.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0006.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0014.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0010.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0012.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0008.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0001.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0002.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0013.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0015.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0004.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0011.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0000.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking/0003.jsonl"
0:       ],
0:       "ds_type": "json",
0:       "field_messages": "conversations",
0:       "message_property_mappings": {
0:         "content": "content",
0:         "role": "role"
0:       },
0:       "path": "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/thinking",
0:       "trust_remote_code": false,
0:       "type": "chat_template"
0:     }
0:   ],
0:   "ddp": true,
0:   "deepspeed": {
0:     "bf16": {
0:       "enabled": true
0:     },
0:     "gradient_accumulation_steps": "auto",
0:     "gradient_clipping": "auto",
0:     "train_batch_size": "auto",
0:     "train_micro_batch_size_per_gpu": "auto",
0:     "wall_clock_breakdown": false,
0:     "zero_optimization": {
0:       "contiguous_gradients": true,
0:       "overlap_comm": true,
0:       "reduce_bucket_size": "auto",
0:       "stage": 3,
0:       "stage3_gather_16bit_weights_on_model_save": true,
0:       "stage3_param_persistence_threshold": "auto",
0:       "stage3_prefetch_bucket_size": "auto",
0:       "sub_group_size": 0
0:     }
0:   },
0:   "device": "cuda:0",
0:   "device_map": {
0:     "": 0
0:   },
0:   "env_capabilities": {
0:     "torch_version": "2.6.0"
0:   },
0:   "eval_batch_size": 1,
0:   "eval_causal_lm_metrics": [
0:     "sacrebleu",
0:     "comet",
0:     "ter",
0:     "chrf"
0:   ],
0:   "eval_max_new_tokens": 128,
0:   "eval_sample_packing": true,
0:   "eval_table_size": 0,
0:   "evals_per_epoch": 0,
0:   "flash_attention": true,
0:   "fp16": false,
0:   "gradient_accumulation_steps": 1,
0:   "gradient_checkpointing": true,
0:   "gradient_checkpointing_kwargs": {
0:     "use_reentrant": true
0:   },
0:   "learning_rate": 3e-06,
0:   "lisa_layers_attribute": "model.layers",
0:   "load_best_model_at_end": false,
0:   "load_in_4bit": false,
0:   "load_in_8bit": false,
0:   "local_rank": 0,
0:   "logging_steps": 10,
0:   "lora_dropout": 0.0,
0:   "loraplus_lr_embedding": 1e-06,
0:   "lr_scheduler": "warmup_stable_decay",
0:   "lr_scheduler_kwargs": {
0:     "min_lr_ratio": 0.1,
0:     "num_decay_steps": 300
0:   },
0:   "max_prompt_len": 512,
0:   "mean_resizing_embeddings": false,
0:   "micro_batch_size": 1,
0:   "model_config_type": "qwen2",
0:   "num_epochs": 1.0,
0:   "optimizer": "adamw_torch_fused",
0:   "output_dir": "/lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-14B/1",
0:   "pad_to_sequence_len": true,
0:   "pretrain_multipack_attn": true,
0:   "pretrain_multipack_buffer_size": 10000,
0:   "profiler_steps_start": 0,
0:   "qlora_sharded_model_loading": false,
0:   "ray_num_workers": 1,
0:   "resources_per_worker": {
0:     "GPU": 1
0:   },
0:   "sample_packing": true,
0:   "sample_packing_bin_size": 200,
0:   "sample_packing_group_size": 100000,
0:   "save_only_model": false,
0:   "save_safetensors": true,
0:   "save_steps": 0.2,
0:   "save_total_limit": 20,
0:   "sequence_len": 16384,
0:   "sequence_parallel_degree": 1,
0:   "shuffle_merged_datasets": true,
0:   "skip_prepare_dataset": false,
0:   "special_tokens": {
0:     "bos_token": "<|im_start|>",
0:     "eos_token": "<|im_end|>",
0:     "pad_token": "<|endoftext|>"
0:   },
0:   "strict": false,
0:   "tensor_parallel_size": 1,
0:   "tf32": false,
0:   "tiled_mlp_use_original_mlp": true,
0:   "tokenizer_config": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-14B",
0:   "torch_dtype": "torch.bfloat16",
0:   "train_on_inputs": false,
0:   "trl": {
0:     "log_completions": false,
0:     "mask_truncated_completions": false,
0:     "ref_model_mixup_alpha": 0.9,
0:     "ref_model_sync_steps": 64,
0:     "scale_rewards": true,
0:     "sync_ref_model": false,
0:     "use_vllm": false,
0:     "vllm_server_host": "0.0.0.0",
0:     "vllm_server_port": 8000
0:   },
0:   "use_ray": false,
0:   "use_tensorboard": true,
0:   "val_set_size": 0.0,
0:   "vllm": {
0:     "device": "auto",
0:     "dtype": "auto",
0:     "gpu_memory_utilization": 0.9,
0:     "host": "0.0.0.0",
0:     "port": 8000
0:   },
0:   "warmup_steps": 150,
0:   "weight_decay": 0.0,
0:   "world_size": 16
0: }
0: [2025-08-16 22:54:03,187] [INFO] [axolotl.cli.checks.check_user_token:35] [PID:1085867] [RANK:0] Skipping HuggingFace token verification because HF_HUB_OFFLINE is set to True. Only local files will be used.
0: [2025-08-16 22:54:04,314] [INFO] [axolotl.utils.data.shared.load_preprocessed_dataset:471] [PID:1085867] [RANK:0] Loading prepared dataset from disk at /lustre/fsn1/projects/rech/dgo/udv55np/dataset/Qwen3-235B-A22B/Qwen2.5-14B/1/fbf1716df9e81872f42eae1d294a3179...
0: [2025-08-16 22:58:18,051] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:435] [PID:1085867] [RANK:0] gather_len_batches: [235606, 235608, 235606, 235603, 235607, 235608, 235607, 235612, 235607, 235605, 235611, 235610, 235606, 235609, 235603, 235608]
0: [2025-08-16 22:58:18,102] [INFO] [axolotl.utils.trainer.calc_sample_packing_eff_est:496] [PID:1085867] [RANK:0] sample_packing_eff_est across ranks: [0.9936686158180237, 0.9936728477478027, 0.9936348795890808, 0.9936854839324951, 0.9936559796333313, 0.9936728477478027, 0.9936770796775818, 0.9936348795890808, 0.9936643838882446, 0.9936517477035522, 0.9936391115188599, 0.9936433434486389, 0.9936559796333313, 0.9936391115188599, 0.9936728477478027, 0.9936854839324951]
0: [2025-08-16 22:58:18,127] [INFO] [axolotl.utils.data.sft._prepare_standard_dataset:123] [PID:1085867] [RANK:0] Maximum number of steps set at 14725
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: 
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.88s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.88s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.88s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.97s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:12<00:37,  6.19s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [0
3: 
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [0
2: 
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.88s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.88s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.88s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.88s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [0
1: 
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  12%|█▎        | 1/8 [00:04<00:34,  4.89s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  25%|██▌       | 2/8 [00:11<00:36,  6.12s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [0
0: 0:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.99s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  7.00s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.28s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42
3: 0:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42
1: 0:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42
2: 0:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  38%|███▊      | 3/8 [00:19<00:34,  6.91s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  50%|█████     | 4/8 [00:26<00:27,  6.90s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  62%|██████▎   | 5/8 [00:34<00:21,  7.32s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42
0: <00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.33s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
0: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
0: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
1: <00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
3: <00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
2: <00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  75%|███████▌  | 6/8 [00:42<00:14,  7.34s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:49<00:07,  7.49s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
1: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
1: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
1: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
3: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
3: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
2: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
2: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
3: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
2: 
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  5.14s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:49<00:00,  6.25s/it]
0: 
Loading checkpoint shards:  88%|████████▊ | 7/8 [00:50<00:07,  7.51s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:51<00:00,  5.62s/it]
Loading checkpoint shards: 100%|██████████| 8/8 [00:51<00:00,  6.45s/it]
0: [2025-08-16 22:59:14,989] [INFO] [axolotl.loaders.model._configure_embedding_dtypes:317] [PID:1085867] [RANK:0] Converting modules to torch.bfloat16
0: [2025-08-16 22:59:43,101] [INFO] [axolotl.train.save_initial_configs:397] [PID:1085867] [RANK:0] Pre-saving tokenizer to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-14B/1...
0: [2025-08-16 22:59:43,509] [INFO] [axolotl.train.save_initial_configs:400] [PID:1085867] [RANK:0] Pre-saving model config to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-14B/1...
0: [2025-08-16 22:59:43,515] [INFO] [axolotl.train.determine_resume_checkpoint:144] [PID:1085867] [RANK:0] Using Auto-resume functionality to start with checkpoint at /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-14B/1/checkpoint-11780
0: [2025-08-16 22:59:43,515] [INFO] [axolotl.train.execute_training:221] [PID:1085867] [RANK:0] Starting trainer...
0: [2025-08-16 23:16:58,499] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:435] [PID:1085867] [RANK:0] gather_len_batches: [235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608, 235608]
0: Parameter Offload - Persistent parameters statistics: param_count = 241, numel = 840704
2: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
2: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
2: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
2: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
1: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
1: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
0: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
0: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
3: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
3: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
1: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
1: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
1: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
1: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
1: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
1: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
0: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
0: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
0: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
0: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
0: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
0: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
3: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
3: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
3: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
3: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
2: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
2: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
3: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
3: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
2: Warning: The following arguments do not match the ones in the `trainer_state.json` within the checkpoint directory: 
2: 	save_steps: 0.2 (from args) != 2945 (from trainer_state.json)
0: {'loss': 0.5311, 'grad_norm': 0.3750610478990628, 'learning_rate': 3e-06, 'epoch': 0.8}
0: 
  0%|          | 0/14725 [00:00<?, ?it/s]
 80%|████████  | 11781/14725 [03:03<00:45, 64.24it/s]
 80%|████████  | 11784/14725 [03:21<00:45, 64.24it/s]
 80%|████████  | 11785/14725 [03:26<00:54, 54.41it/s]
 80%|████████  | 11786/14725 [03:32<00:56, 51.59it/s]
 80%|████████  | 11787/14725 [03:38<01:01, 48.05it/s]
 80%|████████  | 11788/14725 [03:43<01:07, 43.83it/s]
 80%|████████  | 11789/14725 [03:49<01:15, 39.01it/s]
 80%|████████  | 11790/14725 [03:55<01:28, 33.33it/s]
                                                     

 80%|████████  | 11790/14725 [03:55<01:28, 33.33it/s]
 80%|████████  | 11791/14725 [04:01<01:45, 27.72it/s]
 80%|████████  | 11792/14725 [04:07<02:10, 22.45it/s]
 80%|████████  | 11793/14725 [04:12<02:44, 17.86it/s]
 80%|████████  | 11794/14725 [04:18<03:29, 13.99it/s]
 80%|███�
0: {'loss': 0.5224, 'grad_norm': 0.3787057620735715, 'learning_rate': 3e-06, 'epoch': 0.8}
0: �████  | 11795/14725 [04:24<04:46, 10.21it/s]
 80%|████████  | 11796/14725 [04:30<06:33,  7.44it/s]
 80%|████████  | 11797/14725 [04:37<09:08,  5.34it/s]
 80%|████████  | 11798/14725 [04:43<12:35,  3.87it/s]
 80%|████████  | 11799/14725 [04:49<17:29,  2.79it/s]
 80%|████████  | 11800/14725 [04:55<24:24,  2.00it/s]
                                                     

 80%|████████  | 11800/14725 [04:55<24:24,  2.00it/s]
 80%|████████  | 11801/14725 [05:01<33:01,  1.48it/s]
 80%|████████  | 11802/14725 [05:07<43:27,  1.12it/s]
 80%|████████  | 11803/14725 [05:13<57:59,  1.19s/it]
 80%|████████  | 11804/14725 [05:18<1:15:06,  1.54s/it]
 80%|████████  | 11805/14725 [05:24<1:35:18,  1.96s/it]
 80%|████████  | 11806/14725 [05:29<1:56:23,  2.39s/it]
 80%|████████  | 11807/14725 [05:35<2:18:17,  2.84s/i
0: {'loss': 0.5248, 'grad_norm': 0.40681280298643996, 'learning_rate': 3e-06, 'epoch': 0.8}
0: t]
 80%|████████  | 11808/14725 [05:40<2:40:13,  3.30s/it]
 80%|████████  | 11809/14725 [05:46<3:07:40,  3.86s/it]
 80%|████████  | 11810/14725 [05:52<3:34:20,  4.41s/it]
                                                       

 80%|████████  | 11810/14725 [05:52<3:34:20,  4.41s/it]
 80%|████████  | 11811/14725 [05:58<3:52:18,  4.78s/it]
 80%|████████  | 11812/14725 [06:04<4:05:18,  5.05s/it]
 80%|████████  | 11813/14725 [06:09<4:05:44,  5.06s/it]
 80%|████████  | 11814/14725 [06:15<4:12:58,  5.21s/it]
 80%|████████  | 11815/14725 [06:21<4:23:15,  5.43s/it]
 80%|████████  | 11816/14725 [06:27<4:32:08,  5.61s/it]
 80%|████████  | 11817/14725 [06:33<4:35:33,  5.69s/it]
 80%|████████  | 11818/14725 [06:39<4:36:21,  5.70s/it]
 80%|████████  | 11819/14725 [06:45<4:43:24,  5.85s/it]
 80%|███████�
0: {'loss': 0.5263, 'grad_norm': 0.3989093800354776, 'learning_rate': 3e-06, 'epoch': 0.8}
0: {'loss': 0.5314, 'grad_norm': 0.41890637151483184, 'learning_rate': 3e-06, 'epoch': 0.8}
0: �  | 11820/14725 [06:51<4:51:05,  6.01s/it]
                                                       

 80%|████████  | 11820/14725 [06:51<4:51:05,  6.01s/it]
 80%|████████  | 11821/14725 [06:57<4:50:06,  5.99s/it]
 80%|████████  | 11822/14725 [07:03<4:47:49,  5.95s/it]
 80%|████████  | 11823/14725 [07:09<4:50:33,  6.01s/it]
 80%|████████  | 11824/14725 [07:15<4:45:19,  5.90s/it]
 80%|████████  | 11825/14725 [07:20<4:42:44,  5.85s/it]
 80%|████████  | 11826/14725 [07:26<4:41:01,  5.82s/it]
 80%|████████  | 11827/14725 [07:32<4:47:14,  5.95s/it]
 80%|████████  | 11828/14725 [07:38<4:44:03,  5.88s/it]
 80%|████████  | 11829/14725 [07:43<4:35:19,  5.70s/it]
 80%|████████  | 11830/14725 [07:49<4:30:01,  5.60s/it]
                                                       

 80%|████████  | 11830/14725 [07:49<4:30:01,  5.60s/it]
 8
0: {'loss': 0.5341, 'grad_norm': 0.36442945539251825, 'learning_rate': 3e-06, 'epoch': 0.8}
0: 0%|████████  | 11831/14725 [07:55<4:32:25,  5.65s/it]
 80%|████████  | 11832/14725 [08:00<4:28:20,  5.57s/it]
 80%|████████  | 11833/14725 [08:06<4:31:46,  5.64s/it]
 80%|████████  | 11834/14725 [08:12<4:39:27,  5.80s/it]
 80%|████████  | 11835/14725 [08:17<4:34:26,  5.70s/it]
 80%|████████  | 11836/14725 [08:23<4:37:43,  5.77s/it]
 80%|████████  | 11837/14725 [08:30<4:43:28,  5.89s/it]
 80%|████████  | 11838/14725 [08:36<4:45:53,  5.94s/it]
 80%|████████  | 11839/14725 [08:41<4:40:07,  5.82s/it]
 80%|████████  | 11840/14725 [08:47<4:46:53,  5.97s/it]
                                                       

 80%|████████  | 11840/14725 [08:47<4:46:53,  5.97s/it]
 80%|████████  | 11841/14725 [08:54<4:49:29,  6.02s/it]
 80%|████████  | 11842/14725 [08:59<4:45:54,  5.95s/it]
 80%|████████  | 
0: {'loss': 0.516, 'grad_norm': 0.4266008372650705, 'learning_rate': 3e-06, 'epoch': 0.8}
0: 11843/14725 [09:05<4:48:27,  6.01s/it]
 80%|████████  | 11844/14725 [09:11<4:45:44,  5.95s/it]
 80%|████████  | 11845/14725 [09:17<4:40:28,  5.84s/it]
 80%|████████  | 11846/14725 [09:23<4:44:45,  5.93s/it]
 80%|████████  | 11847/14725 [09:29<4:46:30,  5.97s/it]
 80%|████████  | 11848/14725 [09:35<4:38:12,  5.80s/it]
 80%|████████  | 11849/14725 [09:40<4:33:06,  5.70s/it]
 80%|████████  | 11850/14725 [09:46<4:35:29,  5.75s/it]
                                                       

 80%|████████  | 11850/14725 [09:46<4:35:29,  5.75s/it]
 80%|████████  | 11851/14725 [09:52<4:38:46,  5.82s/it]
 80%|████████  | 11852/14725 [09:58<4:37:38,  5.80s/it]
 80%|████████  | 11853/14725 [10:03<4:31:58,  5.68s/it]
 81%|████████  | 11854/14725 [10:08<4:26:24,  5.57s/it]
 81%|████████  | 11855/14725 [10:15<4:37:24,  5.
0: {'loss': 0.534, 'grad_norm': 0.39895926430881834, 'learning_rate': 3e-06, 'epoch': 0.81}
0: 80s/it]
 81%|████████  | 11856/14725 [10:21<4:39:03,  5.84s/it]
 81%|████████  | 11857/14725 [10:26<4:37:38,  5.81s/it]
 81%|████████  | 11858/14725 [10:32<4:33:39,  5.73s/it]
 81%|████████  | 11859/14725 [10:37<4:32:22,  5.70s/it]
 81%|████████  | 11860/14725 [10:44<4:40:11,  5.87s/it]
                                                       

 81%|████████  | 11860/14725 [10:44<4:40:11,  5.87s/it]
 81%|████████  | 11861/14725 [10:50<4:38:50,  5.84s/it]
 81%|████████  | 11862/14725 [10:56<4:40:56,  5.89s/it]
 81%|████████  | 11863/14725 [11:01<4:35:22,  5.77s/it]
 81%|████████  | 11864/14725 [11:07<4:42:27,  5.92s/it]
 81%|████████  | 11865/14725 [11:13<4:35:55,  5.79s/it]
 81%|████████  | 11866/14725 [11:19<4:36:39,  5.81s/it]
 81%|████████  | 11867/14725 [11:25<4:40:26,  5.89s/it]
 81%|██████
0: {'loss': 0.5388, 'grad_norm': 0.40919771413736156, 'learning_rate': 3e-06, 'epoch': 0.81}
0: ██  | 11868/14725 [11:31<4:45:06,  5.99s/it]
 81%|████████  | 11869/14725 [11:37<4:48:30,  6.06s/it]
 81%|████████  | 11870/14725 [11:43<4:50:20,  6.10s/it]
                                                       

 81%|████████  | 11870/14725 [11:43<4:50:20,  6.10s/it]
 81%|████████  | 11871/14725 [11:49<4:44:47,  5.99s/it]
 81%|████████  | 11872/14725 [11:55<4:41:31,  5.92s/it]
 81%|████████  | 11873/14725 [12:01<4:38:55,  5.87s/it]
 81%|████████  | 11874/14725 [12:06<4:32:57,  5.74s/it]
 81%|████████  | 11875/14725 [12:11<4:28:43,  5.66s/it]
 81%|████████  | 11876/14725 [12:18<4:39:08,  5.88s/it]
 81%|████████  | 11877/14725 [12:24<4:39:23,  5.89s/it]
 81%|████████  | 11878/14725 [12:29<4:30:48,  5.71s/it]
 81%|████████  | 11879/14725 [12:35<4:31:55,  5.73s/it]
 81%|████████  | 11880/14725 [12:41<4:
0: {'loss': 0.5382, 'grad_norm': 0.3844646974429067, 'learning_rate': 3e-06, 'epoch': 0.81}
0: {'loss': 0.5178, 'grad_norm': 0.37865108255646757, 'learning_rate': 3e-06, 'epoch': 0.81}
0: 39:42,  5.90s/it]
                                                       

 81%|████████  | 11880/14725 [12:41<4:39:42,  5.90s/it]
 81%|████████  | 11881/14725 [12:47<4:37:04,  5.85s/it]
 81%|████████  | 11882/14725 [12:53<4:35:52,  5.82s/it]
 81%|████████  | 11883/14725 [12:58<4:32:42,  5.76s/it]
 81%|████████  | 11884/14725 [13:04<4:35:56,  5.83s/it]
 81%|████████  | 11885/14725 [13:10<4:32:30,  5.76s/it]
 81%|████████  | 11886/14725 [13:15<4:30:14,  5.71s/it]
 81%|████████  | 11887/14725 [13:22<4:36:23,  5.84s/it]
 81%|████████  | 11888/14725 [13:27<4:34:40,  5.81s/it]
 81%|████████  | 11889/14725 [13:33<4:30:02,  5.71s/it]
 81%|████████  | 11890/14725 [13:38<4:25:47,  5.63s/it]
                                                       

 81%|████████  | 11890/14725 [13:38<4:25:47,  5.63s/it]
 81%|███████�
0: {'loss': 0.5191, 'grad_norm': 0.38407137704570604, 'learning_rate': 3e-06, 'epoch': 0.81}
0: �  | 11891/14725 [13:45<4:35:36,  5.84s/it]
 81%|████████  | 11892/14725 [13:51<4:39:17,  5.92s/it]
 81%|████████  | 11893/14725 [13:57<4:41:00,  5.95s/it]
 81%|████████  | 11894/14725 [14:03<4:42:34,  5.99s/it]
 81%|████████  | 11895/14725 [14:09<4:41:04,  5.96s/it]
 81%|████████  | 11896/14725 [14:14<4:37:43,  5.89s/it]
 81%|████████  | 11897/14725 [14:20<4:31:08,  5.75s/it]
 81%|████████  | 11898/14725 [14:26<4:39:01,  5.92s/it]
 81%|████████  | 11899/14725 [14:32<4:32:57,  5.80s/it]
 81%|████████  | 11900/14725 [14:38<4:39:20,  5.93s/it]
                                                       

 81%|████████  | 11900/14725 [14:38<4:39:20,  5.93s/it]
 81%|████████  | 11901/14725 [14:43<4:34:34,  5.83s/it]
 81%|████████  | 11902/14725 [14:49<4:34:00,  5.82s/it]
 81%|████████  | 11903/14725 [14:55<4:32:43
0: {'loss': 0.5264, 'grad_norm': 0.4066129835047694, 'learning_rate': 3e-06, 'epoch': 0.81}
0: ,  5.80s/it]
 81%|████████  | 11904/14725 [15:01<4:29:11,  5.73s/it]
 81%|████████  | 11905/14725 [15:06<4:29:28,  5.73s/it]
 81%|████████  | 11906/14725 [15:13<4:37:47,  5.91s/it]
 81%|████████  | 11907/14725 [15:18<4:31:59,  5.79s/it]
 81%|████████  | 11908/14725 [15:25<4:39:38,  5.96s/it]
 81%|████████  | 11909/14725 [15:30<4:35:03,  5.86s/it]
 81%|████████  | 11910/14725 [15:36<4:39:11,  5.95s/it]
                                                       

 81%|████████  | 11910/14725 [15:36<4:39:11,  5.95s/it]
 81%|████████  | 11911/14725 [15:42<4:38:44,  5.94s/it]
 81%|████████  | 11912/14725 [15:48<4:34:00,  5.84s/it]
 81%|████████  | 11913/14725 [15:54<4:31:38,  5.80s/it]
 81%|████████  | 11914/14725 [16:00<4:34:26,  5.86s/it]
 81%|████████  | 11915/14725 [16:06<4:37:03,  5.92s/it]
 81%|████�
0: {'loss': 0.524, 'grad_norm': 0.40335693544917783, 'learning_rate': 3e-06, 'epoch': 0.81}
0: ��███  | 11916/14725 [16:12<4:43:18,  6.05s/it]
 81%|████████  | 11917/14725 [16:17<4:35:43,  5.89s/it]
 81%|████████  | 11918/14725 [16:24<4:37:36,  5.93s/it]
 81%|████████  | 11919/14725 [16:29<4:31:47,  5.81s/it]
 81%|████████  | 11920/14725 [16:35<4:38:04,  5.95s/it]
                                                       

 81%|████████  | 11920/14725 [16:35<4:38:04,  5.95s/it]
 81%|████████  | 11921/14725 [16:41<4:36:57,  5.93s/it]
 81%|████████  | 11922/14725 [16:47<4:31:37,  5.81s/it]
 81%|████████  | 11923/14725 [16:52<4:28:07,  5.74s/it]
 81%|████████  | 11924/14725 [16:58<4:32:06,  5.83s/it]
 81%|████████  | 11925/14725 [17:04<4:27:39,  5.74s/it]
 81%|████████  | 11926/14725 [17:09<4:22:18,  5.62s/it]
 81%|████████  | 11927/14725 [17:15<4:26:43,  5.72s/it]
 81%|████████  | 11928/14725 [17:
0: {'loss': 0.5259, 'grad_norm': 0.38295855813389035, 'learning_rate': 3e-06, 'epoch': 0.81}
0: 21<4:30:48,  5.81s/it]
 81%|████████  | 11929/14725 [17:27<4:33:13,  5.86s/it]
 81%|████████  | 11930/14725 [17:33<4:39:35,  6.00s/it]
                                                       

 81%|████████  | 11930/14725 [17:33<4:39:35,  6.00s/it]
 81%|████████  | 11931/14725 [17:39<4:36:21,  5.93s/it]
 81%|████████  | 11932/14725 [17:45<4:34:00,  5.89s/it]
 81%|████████  | 11933/14725 [17:51<4:30:25,  5.81s/it]
 81%|████████  | 11934/14725 [17:56<4:30:32,  5.82s/it]
 81%|████████  | 11935/14725 [18:03<4:38:49,  6.00s/it]
 81%|████████  | 11936/14725 [18:08<4:30:19,  5.82s/it]
 81%|████████  | 11937/14725 [18:14<4:29:12,  5.79s/it]
 81%|████████  | 11938/14725 [18:19<4:21:42,  5.63s/it]
 81%|████████  | 11939/14725 [18:26<4:29:26,  5.80s/it]
 81%|████████  | 11940/14725 [18:31<4:26:20,  5.74s/it]
        
0: {'loss': 0.5438, 'grad_norm': 0.3759190372908252, 'learning_rate': 3e-06, 'epoch': 0.81}
0: {'loss': 0.5401, 'grad_norm': 0.40370813560979874, 'learning_rate': 3e-06, 'epoch': 0.81}
0:                                                

 81%|████████  | 11940/14725 [18:31<4:26:20,  5.74s/it]
 81%|████████  | 11941/14725 [18:37<4:35:24,  5.94s/it]
 81%|████████  | 11942/14725 [18:43<4:35:18,  5.94s/it]
 81%|████████  | 11943/14725 [18:49<4:35:15,  5.94s/it]
 81%|████████  | 11944/14725 [18:55<4:27:43,  5.78s/it]
 81%|████████  | 11945/14725 [19:00<4:19:37,  5.60s/it]
 81%|████████  | 11946/14725 [19:06<4:22:05,  5.66s/it]
 81%|████████  | 11947/14725 [19:12<4:25:38,  5.74s/it]
 81%|████████  | 11948/14725 [19:18<4:31:13,  5.86s/it]
 81%|████████  | 11949/14725 [19:24<4:29:58,  5.84s/it]
 81%|████████  | 11950/14725 [19:30<4:34:39,  5.94s/it]
                                                       

 81%|████████  | 11950/14725 [19:30<4:34:39,  5.94s/it]
 81%|████████  | 11951/14725 [19:35<4:
0: {'loss': 0.5198, 'grad_norm': 0.3812390958520302, 'learning_rate': 3e-06, 'epoch': 0.81}
0: 28:29,  5.81s/it]
 81%|████████  | 11952/14725 [19:42<4:35:45,  5.97s/it]
 81%|████████  | 11953/14725 [19:48<4:39:14,  6.04s/it]
 81%|████████  | 11954/14725 [19:53<4:32:53,  5.91s/it]
 81%|████████  | 11955/14725 [19:59<4:30:47,  5.87s/it]
 81%|████████  | 11956/14725 [20:05<4:33:00,  5.92s/it]
 81%|████████  | 11957/14725 [20:11<4:27:53,  5.81s/it]
 81%|████████  | 11958/14725 [20:16<4:20:21,  5.65s/it]
 81%|████████  | 11959/14725 [20:22<4:18:50,  5.61s/it]
 81%|████████  | 11960/14725 [20:28<4:23:32,  5.72s/it]
                                                       

 81%|████████  | 11960/14725 [20:28<4:23:32,  5.72s/it]
 81%|████████  | 11961/14725 [20:33<4:20:34,  5.66s/it]
 81%|████████  | 11962/14725 [20:39<4:20:29,  5.66s/it]
 81%|████████  | 11963/14725 [20:44<4:21:50,  5.69s/it]
 81%|██�
0: {'loss': 0.521, 'grad_norm': 0.4090438346134969, 'learning_rate': 3e-06, 'epoch': 0.81}
0: �█████  | 11964/14725 [20:50<4:14:59,  5.54s/it]
 81%|████████▏ | 11965/14725 [20:56<4:23:29,  5.73s/it]
 81%|████████▏ | 11966/14725 [21:02<4:24:18,  5.75s/it]
 81%|████████▏ | 11967/14725 [21:07<4:18:51,  5.63s/it]
 81%|████████▏ | 11968/14725 [21:13<4:28:14,  5.84s/it]
 81%|████████▏ | 11969/14725 [21:19<4:28:25,  5.84s/it]
 81%|████████▏ | 11970/14725 [21:25<4:24:53,  5.77s/it]
                                                       

 81%|████████▏ | 11970/14725 [21:25<4:24:53,  5.77s/it]
 81%|████████▏ | 11971/14725 [21:31<4:24:31,  5.76s/it]
 81%|████████▏ | 11972/14725 [21:36<4:18:48,  5.64s/it]
 81%|████████▏ | 11973/14725 [21:42<4:29:15,  5.87s/it]
 81%|████████▏ | 11974/14725 [21:48<4:24:39,  5.77s/it]
 81%|████████▏ | 11975/14725 [21:54<4:30:41,  5.91s/it]
 81%|█████
0: {'loss': 0.5305, 'grad_norm': 0.39527392019217716, 'learning_rate': 3e-06, 'epoch': 0.81}
0: ███▏ | 11976/14725 [22:00<4:33:07,  5.96s/it]
 81%|████████▏ | 11977/14725 [22:06<4:37:23,  6.06s/it]
 81%|████████▏ | 11978/14725 [22:12<4:30:19,  5.90s/it]
 81%|████████▏ | 11979/14725 [22:18<4:30:39,  5.91s/it]
 81%|████████▏ | 11980/14725 [22:23<4:22:22,  5.73s/it]
                                                       

 81%|████████▏ | 11980/14725 [22:23<4:22:22,  5.73s/it]
 81%|████████▏ | 11981/14725 [22:29<4:23:45,  5.77s/it]
 81%|████████▏ | 11982/14725 [22:34<4:17:42,  5.64s/it]
 81%|████████▏ | 11983/14725 [22:40<4:11:53,  5.51s/it]
 81%|████████▏ | 11984/14725 [22:45<4:12:38,  5.53s/it]
 81%|████████▏ | 11985/14725 [22:51<4:18:41,  5.66s/it]
 81%|████████▏ | 11986/14725 [22:57<4:26:52,  5.85s/it]
 81%|████████▏ | 11987/14725 [23:03<4:27:46,  5.87s/it]
 81%|██████�
0: {'loss': 0.5266, 'grad_norm': 0.36927777769019887, 'learning_rate': 3e-06, 'epoch': 0.81}
0: �█▏ | 11988/14725 [23:09<4:19:33,  5.69s/it]
 81%|████████▏ | 11989/14725 [23:14<4:19:48,  5.70s/it]
 81%|████████▏ | 11990/14725 [23:20<4:14:32,  5.58s/it]
                                                       

 81%|████████▏ | 11990/14725 [23:20<4:14:32,  5.58s/it]
 81%|████████▏ | 11991/14725 [23:25<4:17:24,  5.65s/it]
 81%|████████▏ | 11992/14725 [23:32<4:23:18,  5.78s/it]
 81%|████████▏ | 11993/14725 [23:37<4:21:43,  5.75s/it]
 81%|████████▏ | 11994/14725 [23:43<4:15:13,  5.61s/it]
 81%|████████▏ | 11995/14725 [23:48<4:12:11,  5.54s/it]
 81%|████████▏ | 11996/14725 [23:54<4:23:04,  5.78s/it]
 81%|████████▏ | 11997/14725 [24:00<4:23:40,  5.80s/it]
 81%|████████▏ | 11998/14725 [24:06<4:27:10,  5.88s/it]
 81%|████████▏ | 11999/14725 [24:12<4:20:51,  5.74s/it]
 81%|████████�
0: {'loss': 0.541, 'grad_norm': 0.3906696354124452, 'learning_rate': 3e-06, 'epoch': 0.81}
0: {'loss': 0.5278, 'grad_norm': 0.3831066142659631, 'learning_rate': 3e-06, 'epoch': 0.82}
0: �� | 12000/14725 [24:17<4:19:07,  5.71s/it]
                                                       

 81%|████████▏ | 12000/14725 [24:17<4:19:07,  5.71s/it]
 82%|████████▏ | 12001/14725 [24:23<4:25:19,  5.84s/it]
 82%|████████▏ | 12002/14725 [24:29<4:22:42,  5.79s/it]
 82%|████████▏ | 12003/14725 [24:35<4:25:30,  5.85s/it]
 82%|████████▏ | 12004/14725 [24:41<4:26:43,  5.88s/it]
 82%|████████▏ | 12005/14725 [24:47<4:25:30,  5.86s/it]
 82%|████████▏ | 12006/14725 [24:53<4:30:23,  5.97s/it]
 82%|████████▏ | 12007/14725 [24:58<4:22:04,  5.79s/it]
 82%|████████▏ | 12008/14725 [25:05<4:28:32,  5.93s/it]
 82%|████████▏ | 12009/14725 [25:10<4:26:27,  5.89s/it]
 82%|████████▏ | 12010/14725 [25:16<4:23:47,  5.83s/it]
                                                       

 82%|████████▏ | 12010/14725 [25:1
0: {'loss': 0.5332, 'grad_norm': 0.3830074709538243, 'learning_rate': 3e-06, 'epoch': 0.82}
0: 6<4:23:47,  5.83s/it]
 82%|████████▏ | 12011/14725 [25:22<4:21:06,  5.77s/it]
 82%|████████▏ | 12012/14725 [25:27<4:19:10,  5.73s/it]
 82%|████████▏ | 12013/14725 [25:33<4:17:10,  5.69s/it]
 82%|████████▏ | 12014/14725 [25:39<4:18:32,  5.72s/it]
 82%|████████▏ | 12015/14725 [25:45<4:19:18,  5.74s/it]
 82%|████████▏ | 12016/14725 [25:50<4:19:21,  5.74s/it]
 82%|████████▏ | 12017/14725 [25:56<4:24:02,  5.85s/it]
 82%|████████▏ | 12018/14725 [26:02<4:22:13,  5.81s/it]
 82%|████████▏ | 12019/14725 [26:08<4:22:15,  5.82s/it]
 82%|████████▏ | 12020/14725 [26:14<4:29:18,  5.97s/it]
                                                       

 82%|████████▏ | 12020/14725 [26:14<4:29:18,  5.97s/it]
 82%|████████▏ | 12021/14725 [26:20<4:24:53,  5.88s/it]
 82%|████████▏ | 12022/14725 [26:26<4:2
0: {'loss': 0.5304, 'grad_norm': 0.3700344876425115, 'learning_rate': 3e-06, 'epoch': 0.82}
0: 1:59,  5.82s/it]
 82%|████████▏ | 12023/14725 [26:31<4:17:26,  5.72s/it]
 82%|████████▏ | 12024/14725 [26:37<4:18:47,  5.75s/it]
 82%|████████▏ | 12025/14725 [26:43<4:25:08,  5.89s/it]
 82%|████████▏ | 12026/14725 [26:49<4:21:00,  5.80s/it]
 82%|████████▏ | 12027/14725 [26:54<4:19:36,  5.77s/it]
 82%|████████▏ | 12028/14725 [27:00<4:14:10,  5.65s/it]
 82%|████████▏ | 12029/14725 [27:05<4:13:34,  5.64s/it]
 82%|████████▏ | 12030/14725 [27:11<4:15:48,  5.70s/it]
                                                       

 82%|████████▏ | 12030/14725 [27:11<4:15:48,  5.70s/it]
 82%|████████▏ | 12031/14725 [27:17<4:22:06,  5.84s/it]
 82%|████████▏ | 12032/14725 [27:23<4:19:01,  5.77s/it]
 82%|████████▏ | 12033/14725 [27:29<4:17:27,  5.74s/it]
 82%|████████▏ | 12034/14725 [27:34<4:10:13,
0: {'loss': 0.5223, 'grad_norm': 0.3858543773025916, 'learning_rate': 3e-06, 'epoch': 0.82}
0:   5.58s/it]
 82%|████████▏ | 12035/14725 [27:40<4:17:40,  5.75s/it]
 82%|████████▏ | 12036/14725 [27:46<4:18:10,  5.76s/it]
 82%|████████▏ | 12037/14725 [27:52<4:17:33,  5.75s/it]
 82%|████████▏ | 12038/14725 [27:58<4:25:34,  5.93s/it]
 82%|████████▏ | 12039/14725 [28:04<4:28:21,  5.99s/it]
 82%|████████▏ | 12040/14725 [28:10<4:21:53,  5.85s/it]
                                                       

 82%|████████▏ | 12040/14725 [28:10<4:21:53,  5.85s/it]
 82%|████████▏ | 12041/14725 [28:16<4:25:20,  5.93s/it]
 82%|████████▏ | 12042/14725 [28:22<4:26:55,  5.97s/it]
 82%|████████▏ | 12043/14725 [28:28<4:24:16,  5.91s/it]
 82%|████████▏ | 12044/14725 [28:33<4:20:02,  5.82s/it]
 82%|████████▏ | 12045/14725 [28:39<4:14:49,  5.71s/it]
 82%|████████▏ | 12046/14725 [28:45<4:17:58,  5.7
0: {'loss': 0.5262, 'grad_norm': 0.395656195852078, 'learning_rate': 3e-06, 'epoch': 0.82}
0: 8s/it]
 82%|████████▏ | 12047/14725 [28:50<4:11:59,  5.65s/it]
 82%|████████▏ | 12048/14725 [28:55<4:11:11,  5.63s/it]
 82%|████████▏ | 12049/14725 [29:02<4:20:40,  5.84s/it]
 82%|████████▏ | 12050/14725 [29:08<4:19:42,  5.83s/it]
                                                       

 82%|████████▏ | 12050/14725 [29:08<4:19:42,  5.83s/it]
 82%|████████▏ | 12051/14725 [29:14<4:20:52,  5.85s/it]
 82%|████████▏ | 12052/14725 [29:20<4:23:42,  5.92s/it]
 82%|████████▏ | 12053/14725 [29:26<4:27:13,  6.00s/it]
 82%|████████▏ | 12054/14725 [29:32<4:26:38,  5.99s/it]
 82%|████████▏ | 12055/14725 [29:38<4:24:52,  5.95s/it]
 82%|████████▏ | 12056/14725 [29:43<4:14:53,  5.73s/it]
 82%|████████▏ | 12057/14725 [29:48<4:11:03,  5.65s/it]
 82%|████████▏ | 12058/14725 [29:54<4:18:14,  5.81s/it
0: {'loss': 0.5272, 'grad_norm': 0.4184169819432591, 'learning_rate': 3e-06, 'epoch': 0.82}
0: ]
 82%|████████▏ | 12059/14725 [30:01<4:21:27,  5.88s/it]
 82%|████████▏ | 12060/14725 [30:06<4:20:02,  5.85s/it]
                                                       

 82%|████████▏ | 12060/14725 [30:06<4:20:02,  5.85s/it]
 82%|████████▏ | 12061/14725 [30:13<4:27:15,  6.02s/it]
 82%|████████▏ | 12062/14725 [30:18<4:23:19,  5.93s/it]
 82%|████████▏ | 12063/14725 [30:24<4:24:29,  5.96s/it]
 82%|████████▏ | 12064/14725 [30:31<4:30:00,  6.09s/it]
 82%|████████▏ | 12065/14725 [30:37<4:30:38,  6.10s/it]
 82%|████████▏ | 12066/14725 [30:43<4:35:00,  6.21s/it]
 82%|████████▏ | 12067/14725 [30:49<4:28:24,  6.06s/it]
 82%|████████▏ | 12068/14725 [30:55<4:25:38,  6.00s/it]
 82%|████████▏ | 12069/14725 [31:01<4:26:30,  6.02s/it]
 82%|████████▏ | 12070/14725 [31:07<4:19:53,  5.87s/it]
   
0: {'loss': 0.5282, 'grad_norm': 0.3814837171555291, 'learning_rate': 3e-06, 'epoch': 0.82}
0: {'loss': 0.5424, 'grad_norm': 0.37818545747288457, 'learning_rate': 3e-06, 'epoch': 0.82}
0:                                                     

 82%|████████▏ | 12070/14725 [31:07<4:19:53,  5.87s/it]
 82%|████████▏ | 12071/14725 [31:12<4:13:46,  5.74s/it]
 82%|████████▏ | 12072/14725 [31:18<4:14:48,  5.76s/it]
 82%|████████▏ | 12073/14725 [31:24<4:13:17,  5.73s/it]
 82%|████████▏ | 12074/14725 [31:29<4:09:22,  5.64s/it]
 82%|████████▏ | 12075/14725 [31:35<4:12:02,  5.71s/it]
 82%|████████▏ | 12076/14725 [31:41<4:18:46,  5.86s/it]
 82%|████████▏ | 12077/14725 [31:47<4:19:34,  5.88s/it]
 82%|████████▏ | 12078/14725 [31:53<4:22:37,  5.95s/it]
 82%|████████▏ | 12079/14725 [31:59<4:26:24,  6.04s/it]
 82%|████████▏ | 12080/14725 [32:05<4:23:38,  5.98s/it]
                                                       

 82%|████████▏ | 12080/14725 [32:05<4:23:38,  5.98s/it]
 82%|██████�
0: {'loss': 0.5258, 'grad_norm': 0.3970142825869889, 'learning_rate': 3e-06, 'epoch': 0.82}
0: �█▏ | 12081/14725 [32:10<4:14:49,  5.78s/it]
 82%|████████▏ | 12082/14725 [32:17<4:19:43,  5.90s/it]
 82%|████████▏ | 12083/14725 [32:23<4:19:26,  5.89s/it]
 82%|████████▏ | 12084/14725 [32:28<4:19:25,  5.89s/it]
 82%|████████▏ | 12085/14725 [32:34<4:17:33,  5.85s/it]
 82%|████████▏ | 12086/14725 [32:40<4:15:47,  5.82s/it]
 82%|████████▏ | 12087/14725 [32:46<4:15:38,  5.81s/it]
 82%|████████▏ | 12088/14725 [32:52<4:18:08,  5.87s/it]
 82%|████████▏ | 12089/14725 [32:57<4:13:34,  5.77s/it]
 82%|████████▏ | 12090/14725 [33:03<4:10:38,  5.71s/it]
                                                       

 82%|████████▏ | 12090/14725 [33:03<4:10:38,  5.71s/it]
 82%|████████▏ | 12091/14725 [33:08<4:09:33,  5.68s/it]
 82%|████████▏ | 12092/14725 [33:14<4:11:04,  5.72s/it]
 82%|████████�
0: {'loss': 0.5218, 'grad_norm': 0.4002119759013508, 'learning_rate': 3e-06, 'epoch': 0.82}
0: �� | 12093/14725 [33:20<4:05:14,  5.59s/it]
 82%|████████▏ | 12094/14725 [33:25<4:08:43,  5.67s/it]
 82%|████████▏ | 12095/14725 [33:31<4:14:09,  5.80s/it]
 82%|████████▏ | 12096/14725 [33:38<4:17:12,  5.87s/it]
 82%|████████▏ | 12097/14725 [33:44<4:24:19,  6.03s/it]
 82%|████████▏ | 12098/14725 [33:50<4:18:00,  5.89s/it]
 82%|████████▏ | 12099/14725 [33:56<4:23:11,  6.01s/it]
 82%|████████▏ | 12100/14725 [34:02<4:26:59,  6.10s/it]
                                                       

 82%|████████▏ | 12100/14725 [34:02<4:26:59,  6.10s/it]
 82%|████████▏ | 12101/14725 [34:08<4:25:17,  6.07s/it]
 82%|████████▏ | 12102/14725 [34:14<4:25:28,  6.07s/it]
 82%|████████▏ | 12103/14725 [34:20<4:26:37,  6.10s/it]
 82%|████████▏ | 12104/14725 [34:26<4:19:14,  5.93s/it]
 82%|████████▏ | 
0: {'loss': 0.5252, 'grad_norm': 0.42186515303294086, 'learning_rate': 3e-06, 'epoch': 0.82}
0: 12105/14725 [34:32<4:24:24,  6.06s/it]
 82%|████████▏ | 12106/14725 [34:38<4:22:10,  6.01s/it]
 82%|████████▏ | 12107/14725 [34:44<4:24:13,  6.06s/it]
 82%|████████▏ | 12108/14725 [34:51<4:26:47,  6.12s/it]
 82%|████████▏ | 12109/14725 [34:56<4:18:36,  5.93s/it]
 82%|████████▏ | 12110/14725 [35:02<4:21:42,  6.00s/it]
                                                       

 82%|████████▏ | 12110/14725 [35:02<4:21:42,  6.00s/it]
 82%|████████▏ | 12111/14725 [35:08<4:20:18,  5.97s/it]
 82%|████████▏ | 12112/14725 [35:14<4:23:50,  6.06s/it]
 82%|████████▏ | 12113/14725 [35:21<4:24:40,  6.08s/it]
 82%|████████▏ | 12114/14725 [35:26<4:12:33,  5.80s/it]
 82%|████████▏ | 12115/14725 [35:32<4:20:22,  5.99s/it]
 82%|████████▏ | 12116/14725 [35:38<4:15:57,  5.89s/it]
 82%|████████▏ | 12117
0: {'loss': 0.5281, 'grad_norm': 0.4131972835224758, 'learning_rate': 3e-06, 'epoch': 0.82}
0: /14725 [35:43<4:12:43,  5.81s/it]
 82%|████████▏ | 12118/14725 [35:48<4:03:06,  5.59s/it]
 82%|████████▏ | 12119/14725 [35:55<4:10:16,  5.76s/it]
 82%|████████▏ | 12120/14725 [36:01<4:16:05,  5.90s/it]
                                                       

 82%|████████▏ | 12120/14725 [36:01<4:16:05,  5.90s/it]
 82%|████████▏ | 12121/14725 [36:07<4:13:20,  5.84s/it]
 82%|████████▏ | 12122/14725 [36:13<4:15:03,  5.88s/it]
 82%|████████▏ | 12123/14725 [36:18<4:15:57,  5.90s/it]
 82%|████████▏ | 12124/14725 [36:24<4:15:31,  5.89s/it]
 82%|████████▏ | 12125/14725 [36:30<4:07:53,  5.72s/it]
 82%|████████▏ | 12126/14725 [36:36<4:12:42,  5.83s/it]
 82%|████████▏ | 12127/14725 [36:41<4:08:34,  5.74s/it]
 82%|████████▏ | 12128/14725 [36:47<4:02:51,  5.61s/it]
 82%|████████▏ | 12129/1472
0: {'loss': 0.5223, 'grad_norm': 0.3890172269300176, 'learning_rate': 3e-06, 'epoch': 0.82}
0: 5 [36:52<4:00:04,  5.55s/it]
 82%|████████▏ | 12130/14725 [36:58<4:02:34,  5.61s/it]
                                                       

 82%|████████▏ | 12130/14725 [36:58<4:02:34,  5.61s/it]
 82%|████████▏ | 12131/14725 [37:04<4:09:12,  5.76s/it]
 82%|████████▏ | 12132/14725 [37:09<4:04:39,  5.66s/it]
 82%|████████▏ | 12133/14725 [37:15<4:07:23,  5.73s/it]
 82%|████████▏ | 12134/14725 [37:21<4:09:42,  5.78s/it]
 82%|████████▏ | 12135/14725 [37:27<4:17:39,  5.97s/it]
 82%|████████▏ | 12136/14725 [37:34<4:19:27,  6.01s/it]
 82%|████████▏ | 12137/14725 [37:39<4:11:58,  5.84s/it]
 82%|████████▏ | 12138/14725 [37:44<4:04:38,  5.67s/it]
 82%|████████▏ | 12139/14725 [37:50<4:01:08,  5.59s/it]
 82%|████████▏ | 12140/14725 [37:55<3:56:57,  5.50s/it]
                                                  
0: {'loss': 0.5273, 'grad_norm': 0.37446695444378486, 'learning_rate': 3e-06, 'epoch': 0.82}
0: {'loss': 0.5301, 'grad_norm': 0.38538500982028795, 'learning_rate': 3e-06, 'epoch': 0.83}
0:      

 82%|████████▏ | 12140/14725 [37:55<3:56:57,  5.50s/it]
 82%|████████▏ | 12141/14725 [38:01<3:56:36,  5.49s/it]
 82%|████████▏ | 12142/14725 [38:06<3:55:51,  5.48s/it]
 82%|████████▏ | 12143/14725 [38:12<4:05:48,  5.71s/it]
 82%|████████▏ | 12144/14725 [38:17<3:58:40,  5.55s/it]
 82%|████████▏ | 12145/14725 [38:24<4:08:05,  5.77s/it]
 82%|████████▏ | 12146/14725 [38:29<4:04:46,  5.69s/it]
 82%|████████▏ | 12147/14725 [38:35<4:03:48,  5.67s/it]
 82%|████████▏ | 12148/14725 [38:40<4:03:20,  5.67s/it]
 83%|████████▎ | 12149/14725 [38:46<4:04:25,  5.69s/it]
 83%|████████▎ | 12150/14725 [38:52<4:05:15,  5.71s/it]
                                                       

 83%|████████▎ | 12150/14725 [38:52<4:05:15,  5.71s/it]
 83%|████████▎ | 12151/14725 [38:58<4:04:53,  5.71s/it
0: {'loss': 0.5269, 'grad_norm': 0.38064243088305155, 'learning_rate': 3e-06, 'epoch': 0.83}
0: ]
 83%|████████▎ | 12152/14725 [39:03<4:04:34,  5.70s/it]
 83%|████████▎ | 12153/14725 [39:09<3:58:47,  5.57s/it]
 83%|████████▎ | 12154/14725 [39:14<4:01:12,  5.63s/it]
 83%|████████▎ | 12155/14725 [39:20<4:01:26,  5.64s/it]
 83%|████████▎ | 12156/14725 [39:26<4:04:24,  5.71s/it]
 83%|████████▎ | 12157/14725 [39:32<4:09:47,  5.84s/it]
 83%|████████▎ | 12158/14725 [39:38<4:12:02,  5.89s/it]
 83%|████████▎ | 12159/14725 [39:44<4:17:46,  6.03s/it]
 83%|████████▎ | 12160/14725 [39:50<4:14:41,  5.96s/it]
                                                       

 83%|████████▎ | 12160/14725 [39:50<4:14:41,  5.96s/it]
 83%|████████▎ | 12161/14725 [39:56<4:08:27,  5.81s/it]
 83%|████████▎ | 12162/14725 [40:02<4:13:21,  5.93s/it]
 83%|████████▎ | 12163/14725 [40:08<4:16:25,  6.01s/it]
 83
0: {'loss': 0.526, 'grad_norm': 0.4255614672077294, 'learning_rate': 3e-06, 'epoch': 0.83}
0: %|████████▎ | 12164/14725 [40:13<4:08:39,  5.83s/it]
 83%|████████▎ | 12165/14725 [40:19<4:05:10,  5.75s/it]
 83%|████████▎ | 12166/14725 [40:25<4:04:17,  5.73s/it]
 83%|████████▎ | 12167/14725 [40:30<4:03:19,  5.71s/it]
 83%|████████▎ | 12168/14725 [40:36<4:05:28,  5.76s/it]
 83%|████████▎ | 12169/14725 [40:42<4:07:50,  5.82s/it]
 83%|████████▎ | 12170/14725 [40:48<4:03:21,  5.71s/it]
                                                       

 83%|████████▎ | 12170/14725 [40:48<4:03:21,  5.71s/it]
 83%|████████▎ | 12171/14725 [40:54<4:06:40,  5.79s/it]
 83%|████████▎ | 12172/14725 [41:00<4:09:33,  5.87s/it]
 83%|████████▎ | 12173/14725 [41:06<4:09:26,  5.86s/it]
 83%|████████▎ | 12174/14725 [41:11<4:08:11,  5.84s/it]
 83%|████████▎ | 12175/14725 [41:18<4:12:47,  5.95s/it]
 83%|█
0: {'loss': 0.5272, 'grad_norm': 0.39384828725177, 'learning_rate': 3e-06, 'epoch': 0.83}
0: ███████▎ | 12176/14725 [41:24<4:14:13,  5.98s/it]
 83%|████████▎ | 12177/14725 [41:29<4:06:03,  5.79s/it]
 83%|████████▎ | 12178/14725 [41:35<4:04:28,  5.76s/it]
 83%|████████▎ | 12179/14725 [41:40<4:01:30,  5.69s/it]
 83%|████████▎ | 12180/14725 [41:46<3:57:41,  5.60s/it]
                                                       

 83%|████████▎ | 12180/14725 [41:46<3:57:41,  5.60s/it]
 83%|████████▎ | 12181/14725 [41:51<3:59:59,  5.66s/it]
 83%|████████▎ | 12182/14725 [41:57<4:00:13,  5.67s/it]
 83%|████████▎ | 12183/14725 [42:03<3:59:36,  5.66s/it]
 83%|████████▎ | 12184/14725 [42:08<3:57:21,  5.60s/it]
 83%|████████▎ | 12185/14725 [42:14<4:02:56,  5.74s/it]
 83%|████████▎ | 12186/14725 [42:20<4:04:24,  5.78s/it]
 83%|████████▎ | 12187/14725 [42:26<4:04:48,  5.79s/it]
 83%|██�
0: {'loss': 0.5264, 'grad_norm': 0.39179202607329977, 'learning_rate': 3e-06, 'epoch': 0.83}
0: �█████▎ | 12188/14725 [42:31<3:58:46,  5.65s/it]
 83%|████████▎ | 12189/14725 [42:37<3:55:59,  5.58s/it]
 83%|████████▎ | 12190/14725 [42:43<4:08:10,  5.87s/it]
                                                       

 83%|████████▎ | 12190/14725 [42:43<4:08:10,  5.87s/it]
 83%|████████▎ | 12191/14725 [42:49<4:10:54,  5.94s/it]
 83%|████████▎ | 12192/14725 [42:55<4:08:18,  5.88s/it]
 83%|████████▎ | 12193/14725 [43:01<4:05:30,  5.82s/it]
 83%|████████▎ | 12194/14725 [43:06<4:01:58,  5.74s/it]
 83%|████████▎ | 12195/14725 [43:13<4:09:06,  5.91s/it]
 83%|████████▎ | 12196/14725 [43:19<4:12:01,  5.98s/it]
 83%|████████▎ | 12197/14725 [43:24<4:08:41,  5.90s/it]
 83%|████████▎ | 12198/14725 [43:31<4:15:23,  6.06s/it]
 83%|████████▎ | 12199/14725 [43:36<4:09:40,  5.93s/it]
 83%|████�
0: {'loss': 0.5286, 'grad_norm': 0.37656857586492093, 'learning_rate': 3e-06, 'epoch': 0.83}
0: {'loss': 0.5278, 'grad_norm': 0.37367559361646113, 'learning_rate': 3e-06, 'epoch': 0.83}
0: ��███▎ | 12200/14725 [43:42<4:03:16,  5.78s/it]
                                                       

 83%|████████▎ | 12200/14725 [43:42<4:03:16,  5.78s/it]
 83%|████████▎ | 12201/14725 [43:48<4:02:27,  5.76s/it]
 83%|████████▎ | 12202/14725 [43:53<3:57:31,  5.65s/it]
 83%|████████▎ | 12203/14725 [43:59<4:00:07,  5.71s/it]
 83%|████████▎ | 12204/14725 [44:05<4:06:54,  5.88s/it]
 83%|████████▎ | 12205/14725 [44:11<4:02:38,  5.78s/it]
 83%|████████▎ | 12206/14725 [44:16<3:58:18,  5.68s/it]
 83%|████████▎ | 12207/14725 [44:22<4:03:24,  5.80s/it]
 83%|████████▎ | 12208/14725 [44:29<4:09:51,  5.96s/it]
 83%|████████▎ | 12209/14725 [44:34<4:08:39,  5.93s/it]
 83%|████████▎ | 12210/14725 [44:40<4:04:54,  5.84s/it]
                                                       

 83%|████████▎ | 12210
0: {'loss': 0.5307, 'grad_norm': 0.4040714681654341, 'learning_rate': 3e-06, 'epoch': 0.83}
0: /14725 [44:40<4:04:54,  5.84s/it]
 83%|████████▎ | 12211/14725 [44:46<3:59:57,  5.73s/it]
 83%|████████▎ | 12212/14725 [44:52<4:04:34,  5.84s/it]
 83%|████████▎ | 12213/14725 [44:58<4:10:00,  5.97s/it]
 83%|████████▎ | 12214/14725 [45:04<4:08:35,  5.94s/it]
 83%|████████▎ | 12215/14725 [45:10<4:13:40,  6.06s/it]
 83%|████████▎ | 12216/14725 [45:17<4:18:22,  6.18s/it]
 83%|████████▎ | 12217/14725 [45:23<4:15:51,  6.12s/it]
 83%|████████▎ | 12218/14725 [45:28<4:10:57,  6.01s/it]
 83%|████████▎ | 12219/14725 [45:34<4:02:12,  5.80s/it]
 83%|████████▎ | 12220/14725 [45:39<4:01:20,  5.78s/it]
                                                       

 83%|████████▎ | 12220/14725 [45:39<4:01:20,  5.78s/it]
 83%|████████▎ | 12221/14725 [45:45<3:57:21,  5.69s/it]
 83%|████████▎ | 12222/1472
0: {'loss': 0.5272, 'grad_norm': 0.3871772610931612, 'learning_rate': 3e-06, 'epoch': 0.83}
0: 5 [45:51<4:00:39,  5.77s/it]
 83%|████████▎ | 12223/14725 [45:56<3:59:15,  5.74s/it]
 83%|████████▎ | 12224/14725 [46:03<4:08:01,  5.95s/it]
 83%|████████▎ | 12225/14725 [46:08<4:02:39,  5.82s/it]
 83%|████████▎ | 12226/14725 [46:14<4:01:20,  5.79s/it]
 83%|████████▎ | 12227/14725 [46:20<4:04:09,  5.86s/it]
 83%|████████▎ | 12228/14725 [46:26<4:00:20,  5.77s/it]
 83%|████████▎ | 12229/14725 [46:31<3:56:10,  5.68s/it]
 83%|████████▎ | 12230/14725 [46:37<3:55:35,  5.67s/it]
                                                       

 83%|████████▎ | 12230/14725 [46:37<3:55:35,  5.67s/it]
 83%|████████▎ | 12231/14725 [46:43<4:03:35,  5.86s/it]
 83%|████████▎ | 12232/14725 [46:49<4:07:43,  5.96s/it]
 83%|████████▎ | 12233/14725 [46:55<4:06:06,  5.93s/it]
 83%|████████▎ | 12234/14725 [47
0: {'loss': 0.5204, 'grad_norm': 0.354156961853953, 'learning_rate': 3e-06, 'epoch': 0.83}
0: :01<4:10:16,  6.03s/it]
 83%|████████▎ | 12235/14725 [47:08<4:11:55,  6.07s/it]
 83%|████████▎ | 12236/14725 [47:13<4:05:11,  5.91s/it]
 83%|████████▎ | 12237/14725 [47:19<4:04:34,  5.90s/it]
 83%|████████▎ | 12238/14725 [47:25<4:08:39,  6.00s/it]
 83%|████████▎ | 12239/14725 [47:31<4:04:37,  5.90s/it]
 83%|████████▎ | 12240/14725 [47:37<4:02:20,  5.85s/it]
                                                       

 83%|████████▎ | 12240/14725 [47:37<4:02:20,  5.85s/it]
 83%|████████▎ | 12241/14725 [47:43<4:07:40,  5.98s/it]
 83%|████████▎ | 12242/14725 [47:49<4:13:13,  6.12s/it]
 83%|████████▎ | 12243/14725 [47:56<4:13:50,  6.14s/it]
 83%|████████▎ | 12244/14725 [48:02<4:16:42,  6.21s/it]
 83%|████████▎ | 12245/14725 [48:08<4:10:56,  6.07s/it]
 83%|████████▎ | 12246/14725 [48:14<4
0: {'loss': 0.5267, 'grad_norm': 0.392102576929533, 'learning_rate': 3e-06, 'epoch': 0.83}
0: :09:31,  6.04s/it]
 83%|████████▎ | 12247/14725 [48:19<4:05:20,  5.94s/it]
 83%|████████▎ | 12248/14725 [48:25<4:05:06,  5.94s/it]
 83%|████████▎ | 12249/14725 [48:31<3:58:40,  5.78s/it]
 83%|████████▎ | 12250/14725 [48:36<3:53:37,  5.66s/it]
                                                       

 83%|████████▎ | 12250/14725 [48:36<3:53:37,  5.66s/it]
 83%|████████▎ | 12251/14725 [48:42<3:54:44,  5.69s/it]
 83%|████████▎ | 12252/14725 [48:48<3:55:23,  5.71s/it]
 83%|████████▎ | 12253/14725 [48:53<3:50:55,  5.60s/it]
 83%|████████▎ | 12254/14725 [48:59<3:58:32,  5.79s/it]
 83%|████████▎ | 12255/14725 [49:05<3:54:10,  5.69s/it]
 83%|████████▎ | 12256/14725 [49:10<3:51:02,  5.61s/it]
 83%|████████▎ | 12257/14725 [49:16<3:51:34,  5.63s/it]
 83%|████████▎ | 12258/14725 [49:21<3:51:5
0: {'loss': 0.5298, 'grad_norm': 0.4040806927845242, 'learning_rate': 3e-06, 'epoch': 0.83}
0: 9,  5.64s/it]
 83%|████████▎ | 12259/14725 [49:27<3:51:16,  5.63s/it]
 83%|████████▎ | 12260/14725 [49:33<3:53:00,  5.67s/it]
                                                       

 83%|████████▎ | 12260/14725 [49:33<3:53:00,  5.67s/it]
 83%|████████▎ | 12261/14725 [49:39<3:53:49,  5.69s/it]
 83%|████████▎ | 12262/14725 [49:44<3:46:44,  5.52s/it]
 83%|████████▎ | 12263/14725 [49:49<3:47:26,  5.54s/it]
 83%|████████▎ | 12264/14725 [49:56<3:57:59,  5.80s/it]
 83%|████████▎ | 12265/14725 [50:01<3:53:53,  5.70s/it]
 83%|████████▎ | 12266/14725 [50:07<3:52:31,  5.67s/it]
 83%|████████▎ | 12267/14725 [50:12<3:46:28,  5.53s/it]
 83%|████████▎ | 12268/14725 [50:18<3:49:29,  5.60s/it]
 83%|████████▎ | 12269/14725 [50:24<3:54:59,  5.74s/it]
 83%|████████▎ | 12270/14725 [50:29<3:54:34,  5
0: {'loss': 0.521, 'grad_norm': 0.39827252487422526, 'learning_rate': 3e-06, 'epoch': 0.83}
0: {'loss': 0.5221, 'grad_norm': 0.4080483395530573, 'learning_rate': 3e-06, 'epoch': 0.83}
0: .73s/it]
                                                       

 83%|████████▎ | 12270/14725 [50:29<3:54:34,  5.73s/it]
 83%|████████▎ | 12271/14725 [50:35<3:50:50,  5.64s/it]
 83%|████████▎ | 12272/14725 [50:40<3:48:34,  5.59s/it]
 83%|████████▎ | 12273/14725 [50:46<3:46:01,  5.53s/it]
 83%|████████▎ | 12274/14725 [50:51<3:45:29,  5.52s/it]
 83%|████████▎ | 12275/14725 [50:57<3:52:54,  5.70s/it]
 83%|████████▎ | 12276/14725 [51:03<3:51:23,  5.67s/it]
 83%|████████▎ | 12277/14725 [51:09<3:58:55,  5.86s/it]
 83%|████████▎ | 12278/14725 [51:15<3:58:46,  5.85s/it]
 83%|████████▎ | 12279/14725 [51:21<3:52:54,  5.71s/it]
 83%|████████▎ | 12280/14725 [51:26<3:48:52,  5.62s/it]
                                                       

 83%|████████▎ | 12280/14725 [51:26<3:48:52,  5.62s/it]
 83%|██�
0: {'loss': 0.5328, 'grad_norm': 0.3888330662898527, 'learning_rate': 3e-06, 'epoch': 0.83}
0: �█████▎ | 12281/14725 [51:31<3:48:19,  5.61s/it]
 83%|████████▎ | 12282/14725 [51:37<3:48:08,  5.60s/it]
 83%|████████▎ | 12283/14725 [51:43<3:46:01,  5.55s/it]
 83%|████████▎ | 12284/14725 [51:49<3:55:31,  5.79s/it]
 83%|████████▎ | 12285/14725 [51:55<3:53:48,  5.75s/it]
 83%|████████▎ | 12286/14725 [52:01<3:58:32,  5.87s/it]
 83%|████████▎ | 12287/14725 [52:06<3:53:19,  5.74s/it]
 83%|████████▎ | 12288/14725 [52:12<3:49:32,  5.65s/it]
 83%|████████▎ | 12289/14725 [52:17<3:42:39,  5.48s/it]
 83%|████████▎ | 12290/14725 [52:23<3:47:24,  5.60s/it]
                                                       

 83%|████████▎ | 12290/14725 [52:23<3:47:24,  5.60s/it]
 83%|████████▎ | 12291/14725 [52:29<3:52:32,  5.73s/it]
 83%|████████▎ | 12292/14725 [52:35<3:57:12,  5.85s/it]
 83%|████�
0: {'loss': 0.5285, 'grad_norm': 0.391526067913762, 'learning_rate': 3e-06, 'epoch': 0.84}
0: ��███▎ | 12293/14725 [52:41<4:01:38,  5.96s/it]
 83%|████████▎ | 12294/14725 [52:47<4:07:36,  6.11s/it]
 83%|████████▎ | 12295/14725 [52:53<4:05:00,  6.05s/it]
 84%|████████▎ | 12296/14725 [52:59<4:06:42,  6.09s/it]
 84%|████████▎ | 12297/14725 [53:05<4:03:07,  6.01s/it]
 84%|████████▎ | 12298/14725 [53:11<4:02:21,  5.99s/it]
 84%|████████▎ | 12299/14725 [53:17<4:01:24,  5.97s/it]
 84%|████████▎ | 12300/14725 [53:23<3:58:04,  5.89s/it]
                                                       

 84%|████████▎ | 12300/14725 [53:23<3:58:04,  5.89s/it]
 84%|████████▎ | 12301/14725 [53:29<4:01:58,  5.99s/it]
 84%|████████▎ | 12302/14725 [53:35<4:03:44,  6.04s/it]
 84%|████████▎ | 12303/14725 [53:41<4:01:23,  5.98s/it]
 84%|████████▎ | 12304/14725 [53:47<4:02:18,  6.01s/it]
 84%|██████
0: {'loss': 0.5265, 'grad_norm': 0.38677794567902596, 'learning_rate': 3e-06, 'epoch': 0.84}
0: ██▎ | 12305/14725 [53:53<4:00:51,  5.97s/it]
 84%|████████▎ | 12306/14725 [53:59<4:02:10,  6.01s/it]
 84%|████████▎ | 12307/14725 [54:05<4:00:30,  5.97s/it]
 84%|████████▎ | 12308/14725 [54:11<3:58:47,  5.93s/it]
 84%|████████▎ | 12309/14725 [54:17<3:58:12,  5.92s/it]
 84%|████████▎ | 12310/14725 [54:23<4:02:00,  6.01s/it]
                                                       

 84%|████████▎ | 12310/14725 [54:23<4:02:00,  6.01s/it]
 84%|████████▎ | 12311/14725 [54:28<3:52:13,  5.77s/it]
 84%|████████▎ | 12312/14725 [54:34<3:50:02,  5.72s/it]
 84%|████████▎ | 12313/14725 [54:40<3:50:44,  5.74s/it]
 84%|████████▎ | 12314/14725 [54:46<4:00:11,  5.98s/it]
 84%|████████▎ | 12315/14725 [54:51<3:49:56,  5.72s/it]
 84%|████████▎ | 12316/14725 [54:57<3:52:50,  5.80s/it]
 84%|███████�
0: {'loss': 0.531, 'grad_norm': 0.37718275168190857, 'learning_rate': 3e-06, 'epoch': 0.84}
0: �▎ | 12317/14725 [55:03<3:56:06,  5.88s/it]
 84%|████████▎ | 12318/14725 [55:09<3:58:45,  5.95s/it]
 84%|████████▎ | 12319/14725 [55:15<3:59:40,  5.98s/it]
 84%|████████▎ | 12320/14725 [55:21<3:55:12,  5.87s/it]
                                                       

 84%|████████▎ | 12320/14725 [55:21<3:55:12,  5.87s/it]
 84%|████████▎ | 12321/14725 [55:27<3:54:29,  5.85s/it]
 84%|████████▎ | 12322/14725 [55:32<3:49:24,  5.73s/it]
 84%|████████▎ | 12323/14725 [55:38<3:55:11,  5.87s/it]
 84%|████████▎ | 12324/14725 [55:45<3:59:47,  5.99s/it]
 84%|████████▎ | 12325/14725 [55:51<4:05:20,  6.13s/it]
 84%|████████▎ | 12326/14725 [55:57<4:03:41,  6.09s/it]
 84%|████████▎ | 12327/14725 [56:03<3:57:38,  5.95s/it]
 84%|████████▎ | 12328/14725 [56:08<3:52:53,  5.83s/it]
 84%|████████▎ 
0: {'loss': 0.528, 'grad_norm': 0.3706573214422352, 'learning_rate': 3e-06, 'epoch': 0.84}
0: | 12329/14725 [56:15<3:58:39,  5.98s/it]
 84%|████████▎ | 12330/14725 [56:20<3:53:40,  5.85s/it]
                                                       

 84%|████████▎ | 12330/14725 [56:20<3:53:40,  5.85s/it]
 84%|████████▎ | 12331/14725 [56:26<3:52:10,  5.82s/it]
 84%|████████▎ | 12332/14725 [56:32<3:51:09,  5.80s/it]
 84%|████████▍ | 12333/14725 [56:37<3:49:56,  5.77s/it]
 84%|████████▍ | 12334/14725 [56:44<3:56:07,  5.93s/it]
 84%|████████▍ | 12335/14725 [56:50<3:54:49,  5.90s/it]
 84%|████████▍ | 12336/14725 [56:55<3:50:34,  5.79s/it]
 84%|████████▍ | 12337/14725 [57:01<3:55:32,  5.92s/it]
 84%|████████▍ | 12338/14725 [57:08<3:59:05,  6.01s/it]
 84%|████████▍ | 12339/14725 [57:14<4:02:09,  6.09s/it]
 84%|████████▍ | 12340/14725 [57:19<3:52:52,  5.86s/it]
                                      
0: {'loss': 0.5311, 'grad_norm': 0.3771481102481653, 'learning_rate': 3e-06, 'epoch': 0.84}
0: {'loss': 0.532, 'grad_norm': 0.390589156035833, 'learning_rate': 3e-06, 'epoch': 0.84}
0:                  

 84%|████████▍ | 12340/14725 [57:19<3:52:52,  5.86s/it]
 84%|████████▍ | 12341/14725 [57:25<3:54:48,  5.91s/it]
 84%|████████▍ | 12342/14725 [57:31<3:58:17,  6.00s/it]
 84%|████████▍ | 12343/14725 [57:38<4:01:35,  6.09s/it]
 84%|████████▍ | 12344/14725 [57:44<4:03:23,  6.13s/it]
 84%|████████▍ | 12345/14725 [57:49<3:55:16,  5.93s/it]
 84%|████████▍ | 12346/14725 [57:56<3:58:20,  6.01s/it]
 84%|████████▍ | 12347/14725 [58:02<4:01:21,  6.09s/it]
 84%|████████▍ | 12348/14725 [58:08<4:00:40,  6.07s/it]
 84%|████████▍ | 12349/14725 [58:13<3:53:49,  5.90s/it]
 84%|████████▍ | 12350/14725 [58:20<3:56:13,  5.97s/it]
                                                       

 84%|████████▍ | 12350/14725 [58:20<3:56:13,  5.97s/it]
 84%|████████▍ | 12351/14725 [58:26<3:59:4
0: {'loss': 0.5339, 'grad_norm': 0.4079345276385702, 'learning_rate': 3e-06, 'epoch': 0.84}
0: 4,  6.06s/it]
 84%|████████▍ | 12352/14725 [58:32<4:04:29,  6.18s/it]
 84%|████████▍ | 12353/14725 [58:38<3:59:44,  6.06s/it]
 84%|████████▍ | 12354/14725 [58:44<3:54:54,  5.94s/it]
 84%|████████▍ | 12355/14725 [58:50<3:58:58,  6.05s/it]
 84%|████████▍ | 12356/14725 [58:56<4:00:23,  6.09s/it]
 84%|████████▍ | 12357/14725 [59:02<3:53:31,  5.92s/it]
 84%|████████▍ | 12358/14725 [59:07<3:47:53,  5.78s/it]
 84%|████████▍ | 12359/14725 [59:13<3:48:58,  5.81s/it]
 84%|████████▍ | 12360/14725 [59:19<3:50:17,  5.84s/it]
                                                       

 84%|████████▍ | 12360/14725 [59:19<3:50:17,  5.84s/it]
 84%|████████▍ | 12361/14725 [59:25<3:49:20,  5.82s/it]
 84%|████████▍ | 12362/14725 [59:31<3:55:17,  5.97s/it]
 84%|████████▍ | 12363/14725 [59:36<3:48:20,  5
0: {'loss': 0.5281, 'grad_norm': 0.3801055122306837, 'learning_rate': 3e-06, 'epoch': 0.84}
0: .80s/it]
 84%|████████▍ | 12364/14725 [59:42<3:45:09,  5.72s/it]
 84%|████████▍ | 12365/14725 [59:47<3:41:39,  5.64s/it]
 84%|████████▍ | 12366/14725 [59:53<3:41:00,  5.62s/it]
 84%|████████▍ | 12367/14725 [59:59<3:41:34,  5.64s/it]
 84%|████████▍ | 12368/14725 [1:00:04<3:42:07,  5.65s/it]
 84%|████████▍ | 12369/14725 [1:00:10<3:44:22,  5.71s/it]
 84%|████████▍ | 12370/14725 [1:00:15<3:38:56,  5.58s/it]
                                                         

 84%|████████▍ | 12370/14725 [1:00:15<3:38:56,  5.58s/it]
 84%|████████▍ | 12371/14725 [1:00:22<3:44:51,  5.73s/it]
 84%|████████▍ | 12372/14725 [1:00:28<3:47:07,  5.79s/it]
 84%|████████▍ | 12373/14725 [1:00:34<3:53:49,  5.96s/it]
 84%|████████▍ | 12374/14725 [1:00:39<3:49:14,  5.85s/it]
 84%|████████▍ | 12375/14725 [1:00
0: {'loss': 0.5271, 'grad_norm': 0.38433606833302375, 'learning_rate': 3e-06, 'epoch': 0.84}
0: :45<3:48:29,  5.83s/it]
 84%|████████▍ | 12376/14725 [1:00:51<3:52:56,  5.95s/it]
 84%|████████▍ | 12377/14725 [1:00:58<4:00:17,  6.14s/it]
 84%|████████▍ | 12378/14725 [1:01:04<3:53:37,  5.97s/it]
 84%|████████▍ | 12379/14725 [1:01:09<3:46:57,  5.80s/it]
 84%|████████▍ | 12380/14725 [1:01:14<3:40:37,  5.65s/it]
                                                         

 84%|████████▍ | 12380/14725 [1:01:14<3:40:37,  5.65s/it]
 84%|████████▍ | 12381/14725 [1:01:21<3:49:10,  5.87s/it]
 84%|████████▍ | 12382/14725 [1:01:27<3:55:35,  6.03s/it]
 84%|████████▍ | 12383/14725 [1:01:33<3:49:44,  5.89s/it]
 84%|████████▍ | 12384/14725 [1:01:38<3:46:07,  5.80s/it]
 84%|████████▍ | 12385/14725 [1:01:44<3:43:18,  5.73s/it]
 84%|████████▍ | 12386/14725 [1:01:50<3:47:52,  5.85s/it]
 84%|████████
0: {'loss': 0.5299, 'grad_norm': 0.39026372629519396, 'learning_rate': 3e-06, 'epoch': 0.84}
0: ▍ | 12387/14725 [1:01:56<3:47:58,  5.85s/it]
 84%|████████▍ | 12388/14725 [1:02:02<3:47:49,  5.85s/it]
 84%|████████▍ | 12389/14725 [1:02:07<3:44:51,  5.78s/it]
 84%|████████▍ | 12390/14725 [1:02:13<3:43:24,  5.74s/it]
                                                         

 84%|████████▍ | 12390/14725 [1:02:13<3:43:24,  5.74s/it]
 84%|████████▍ | 12391/14725 [1:02:19<3:47:51,  5.86s/it]
 84%|████████▍ | 12392/14725 [1:02:25<3:49:55,  5.91s/it]
 84%|████████▍ | 12393/14725 [1:02:31<3:53:50,  6.02s/it]
 84%|████████▍ | 12394/14725 [1:02:37<3:50:33,  5.93s/it]
 84%|████████▍ | 12395/14725 [1:02:43<3:48:53,  5.89s/it]
 84%|████████▍ | 12396/14725 [1:02:49<3:52:47,  6.00s/it]
 84%|████████▍ | 12397/14725 [1:02:55<3:48:35,  5.89s/it]
 84%|████████▍ | 12398/14725 [1:03:00<3:41:15,  5.71s/it]
 84%|�
0: {'loss': 0.5318, 'grad_norm': 0.4003150639950137, 'learning_rate': 3e-06, 'epoch': 0.84}
0: ��███████▍ | 12399/14725 [1:03:06<3:40:59,  5.70s/it]
 84%|████████▍ | 12400/14725 [1:03:11<3:41:09,  5.71s/it]
                                                         

 84%|████████▍ | 12400/14725 [1:03:11<3:41:09,  5.71s/it]
 84%|████████▍ | 12401/14725 [1:03:17<3:42:40,  5.75s/it]
 84%|████████▍ | 12402/14725 [1:03:23<3:46:38,  5.85s/it]
 84%|████████▍ | 12403/14725 [1:03:29<3:40:30,  5.70s/it]
 84%|████████▍ | 12404/14725 [1:03:35<3:42:38,  5.76s/it]
 84%|████████▍ | 12405/14725 [1:03:40<3:40:18,  5.70s/it]
 84%|████████▍ | 12406/14725 [1:03:46<3:42:46,  5.76s/it]
 84%|████████▍ | 12407/14725 [1:03:51<3:38:06,  5.65s/it]
 84%|████████▍ | 12408/14725 [1:03:58<3:44:04,  5.80s/it]
 84%|████████▍ | 12409/14725 [1:04:03<3:39:53,  5.70s/it]
 84%|████████▍ | 12410/14725 [1:04:09<3:3
0: {'loss': 0.5214, 'grad_norm': 0.3872429737735603, 'learning_rate': 3e-06, 'epoch': 0.84}
0: {'loss': 0.5249, 'grad_norm': 0.3822419163140829, 'learning_rate': 3e-06, 'epoch': 0.84}
0: 6:19,  5.61s/it]
                                                         

 84%|████████▍ | 12410/14725 [1:04:09<3:36:19,  5.61s/it]
 84%|████████▍ | 12411/14725 [1:04:14<3:36:16,  5.61s/it]
 84%|████████▍ | 12412/14725 [1:04:20<3:35:58,  5.60s/it]
 84%|████████▍ | 12413/14725 [1:04:26<3:39:11,  5.69s/it]
 84%|████████▍ | 12414/14725 [1:04:32<3:44:51,  5.84s/it]
 84%|████████▍ | 12415/14725 [1:04:37<3:40:04,  5.72s/it]
 84%|████████▍ | 12416/14725 [1:04:43<3:45:41,  5.86s/it]
 84%|████████▍ | 12417/14725 [1:04:49<3:44:38,  5.84s/it]
 84%|████████▍ | 12418/14725 [1:04:55<3:48:23,  5.94s/it]
 84%|████████▍ | 12419/14725 [1:05:01<3:49:22,  5.97s/it]
 84%|████████▍ | 12420/14725 [1:05:08<3:53:24,  6.08s/it]
                                                         

 84%|████████▍ | 12420/14725 [1:05:
0: {'loss': 0.5239, 'grad_norm': 0.3801489607582475, 'learning_rate': 3e-06, 'epoch': 0.84}
0: 08<3:53:24,  6.08s/it]
 84%|████████▍ | 12421/14725 [1:05:13<3:47:29,  5.92s/it]
 84%|████████▍ | 12422/14725 [1:05:19<3:41:40,  5.78s/it]
 84%|████████▍ | 12423/14725 [1:05:25<3:47:55,  5.94s/it]
 84%|████████▍ | 12424/14725 [1:05:31<3:47:43,  5.94s/it]
 84%|████████▍ | 12425/14725 [1:05:37<3:44:31,  5.86s/it]
 84%|████████▍ | 12426/14725 [1:05:43<3:44:14,  5.85s/it]
 84%|████████▍ | 12427/14725 [1:05:48<3:40:16,  5.75s/it]
 84%|████████▍ | 12428/14725 [1:05:54<3:45:02,  5.88s/it]
 84%|████████▍ | 12429/14725 [1:06:00<3:45:39,  5.90s/it]
 84%|████████▍ | 12430/14725 [1:06:06<3:49:08,  5.99s/it]
                                                         

 84%|████████▍ | 12430/14725 [1:06:06<3:49:08,  5.99s/it]
 84%|████████▍ | 12431/14725 [1:06:12<3:41:47,  5.80s/it]
 84%|████████�
0: {'loss': 0.5208, 'grad_norm': 0.38150387483953746, 'learning_rate': 3e-06, 'epoch': 0.84}
0: �� | 12432/14725 [1:06:17<3:33:58,  5.60s/it]
 84%|████████▍ | 12433/14725 [1:06:23<3:36:03,  5.66s/it]
 84%|████████▍ | 12434/14725 [1:06:29<3:39:18,  5.74s/it]
 84%|████████▍ | 12435/14725 [1:06:34<3:36:39,  5.68s/it]
 84%|████████▍ | 12436/14725 [1:06:40<3:34:35,  5.63s/it]
 84%|████████▍ | 12437/14725 [1:06:45<3:33:54,  5.61s/it]
 84%|████████▍ | 12438/14725 [1:06:51<3:37:21,  5.70s/it]
 84%|████████▍ | 12439/14725 [1:06:57<3:36:26,  5.68s/it]
 84%|████████▍ | 12440/14725 [1:07:03<3:38:05,  5.73s/it]
                                                         

 84%|████████▍ | 12440/14725 [1:07:03<3:38:05,  5.73s/it]
 84%|████████▍ | 12441/14725 [1:07:08<3:32:03,  5.57s/it]
 84%|████████▍ | 12442/14725 [1:07:13<3:32:11,  5.58s/it]
 85%|████████▍ | 12443/14725 [1:07:19<3:38:38,  5.75s/it]
 85%|�
0: {'loss': 0.5243, 'grad_norm': 0.39140479029802316, 'learning_rate': 3e-06, 'epoch': 0.85}
0: �███████▍ | 12444/14725 [1:07:25<3:38:07,  5.74s/it]
 85%|████████▍ | 12445/14725 [1:07:31<3:41:57,  5.84s/it]
 85%|████████▍ | 12446/14725 [1:07:40<4:18:30,  6.81s/it]
 85%|████████▍ | 12447/14725 [1:07:51<4:59:50,  7.90s/it]
 85%|████████▍ | 12448/14725 [1:07:56<4:31:56,  7.17s/it]
 85%|████████▍ | 12449/14725 [1:08:02<4:14:36,  6.71s/it]
 85%|████████▍ | 12450/14725 [1:08:08<4:03:55,  6.43s/it]
                                                         

 85%|████████▍ | 12450/14725 [1:08:08<4:03:55,  6.43s/it]
 85%|████████▍ | 12451/14725 [1:08:13<3:55:51,  6.22s/it]
 85%|████████▍ | 12452/14725 [1:08:19<3:47:19,  6.00s/it]
 85%|████████▍ | 12453/14725 [1:08:25<3:43:07,  5.89s/it]
 85%|████████▍ | 12454/14725 [1:08:31<3:44:20,  5.93s/it]
 85%|████████▍ | 12455/14725 [1:08:37<3:44
0: {'loss': 0.5383, 'grad_norm': 0.3928768818813287, 'learning_rate': 3e-06, 'epoch': 0.85}
0: :46,  5.94s/it]
 85%|████████▍ | 12456/14725 [1:08:43<3:48:11,  6.03s/it]
 85%|████████▍ | 12457/14725 [1:08:49<3:47:49,  6.03s/it]
 85%|████████▍ | 12458/14725 [1:08:54<3:43:46,  5.92s/it]
 85%|████████▍ | 12459/14725 [1:09:00<3:43:36,  5.92s/it]
 85%|████████▍ | 12460/14725 [1:09:06<3:44:48,  5.96s/it]
                                                         

 85%|████████▍ | 12460/14725 [1:09:06<3:44:48,  5.96s/it]
 85%|████████▍ | 12461/14725 [1:09:12<3:43:07,  5.91s/it]
 85%|████████▍ | 12462/14725 [1:09:18<3:40:36,  5.85s/it]
 85%|████████▍ | 12463/14725 [1:09:23<3:35:31,  5.72s/it]
 85%|████████▍ | 12464/14725 [1:09:29<3:31:21,  5.61s/it]
 85%|████████▍ | 12465/14725 [1:09:34<3:31:06,  5.60s/it]
 85%|████████▍ | 12466/14725 [1:09:40<3:31:53,  5.63s/it]
 85%|████████▍ | 12
0: {'loss': 0.5255, 'grad_norm': 0.39043339430652285, 'learning_rate': 3e-06, 'epoch': 0.85}
0: 467/14725 [1:09:45<3:26:48,  5.50s/it]
 85%|████████▍ | 12468/14725 [1:09:50<3:22:11,  5.38s/it]
 85%|████████▍ | 12469/14725 [1:09:56<3:29:53,  5.58s/it]
 85%|████████▍ | 12470/14725 [1:10:02<3:30:28,  5.60s/it]
                                                         

 85%|████████▍ | 12470/14725 [1:10:02<3:30:28,  5.60s/it]
 85%|████████▍ | 12471/14725 [1:10:08<3:34:29,  5.71s/it]
 85%|████████▍ | 12472/14725 [1:10:14<3:36:49,  5.77s/it]
 85%|████████▍ | 12473/14725 [1:10:20<3:41:10,  5.89s/it]
 85%|████████▍ | 12474/14725 [1:10:26<3:39:15,  5.84s/it]
 85%|████████▍ | 12475/14725 [1:10:31<3:35:43,  5.75s/it]
 85%|████████▍ | 12476/14725 [1:10:37<3:37:21,  5.80s/it]
 85%|████████▍ | 12477/14725 [1:10:43<3:36:21,  5.77s/it]
 85%|████████▍ | 12478/14725 [1:10:49<3:38:07,  5.82s/it]
 85%|███
0: {'loss': 0.513, 'grad_norm': 0.37914732858261707, 'learning_rate': 3e-06, 'epoch': 0.85}
0: █████▍ | 12479/14725 [1:10:54<3:32:45,  5.68s/it]
 85%|████████▍ | 12480/14725 [1:11:00<3:34:32,  5.73s/it]
                                                         

 85%|████████▍ | 12480/14725 [1:11:00<3:34:32,  5.73s/it]
 85%|████████▍ | 12481/14725 [1:11:06<3:31:12,  5.65s/it]
 85%|████████▍ | 12482/14725 [1:11:11<3:34:25,  5.74s/it]
 85%|████████▍ | 12483/14725 [1:11:18<3:38:50,  5.86s/it]
 85%|████████▍ | 12484/14725 [1:11:23<3:36:56,  5.81s/it]
 85%|████████▍ | 12485/14725 [1:11:29<3:31:23,  5.66s/it]
 85%|████████▍ | 12486/14725 [1:11:34<3:31:17,  5.66s/it]
 85%|████████▍ | 12487/14725 [1:11:40<3:37:26,  5.83s/it]
 85%|████████▍ | 12488/14725 [1:11:46<3:38:24,  5.86s/it]
 85%|████████▍ | 12489/14725 [1:11:52<3:36:57,  5.82s/it]
 85%|████████▍ | 12490/14725 [1:11:58<3:39:50,  5
0: {'loss': 0.5267, 'grad_norm': 0.3594272257010625, 'learning_rate': 3e-06, 'epoch': 0.85}
0: {'loss': 0.5419, 'grad_norm': 0.39910628722324637, 'learning_rate': 3e-06, 'epoch': 0.85}
0: .90s/it]
                                                         

 85%|████████▍ | 12490/14725 [1:11:58<3:39:50,  5.90s/it]
 85%|████████▍ | 12491/14725 [1:12:04<3:37:42,  5.85s/it]
 85%|████████▍ | 12492/14725 [1:12:09<3:33:13,  5.73s/it]
 85%|████████▍ | 12493/14725 [1:12:15<3:28:07,  5.59s/it]
 85%|████████▍ | 12494/14725 [1:12:21<3:33:26,  5.74s/it]
 85%|████████▍ | 12495/14725 [1:12:27<3:34:42,  5.78s/it]
 85%|████████▍ | 12496/14725 [1:12:33<3:38:51,  5.89s/it]
 85%|████████▍ | 12497/14725 [1:12:39<3:42:28,  5.99s/it]
 85%|████████▍ | 12498/14725 [1:12:45<3:39:04,  5.90s/it]
 85%|████████▍ | 12499/14725 [1:12:50<3:34:01,  5.77s/it]
 85%|████████▍ | 12500/14725 [1:12:56<3:38:11,  5.88s/it]
                                                         

 85%|████████▍ | 12500/14725 [1:12:56<3:38:
0: {'loss': 0.5301, 'grad_norm': 0.41001200199510346, 'learning_rate': 3e-06, 'epoch': 0.85}
0: 11,  5.88s/it]
 85%|████████▍ | 12501/14725 [1:13:02<3:39:18,  5.92s/it]
 85%|████████▍ | 12502/14725 [1:13:08<3:35:32,  5.82s/it]
 85%|████████▍ | 12503/14725 [1:13:13<3:32:21,  5.73s/it]
 85%|████████▍ | 12504/14725 [1:13:19<3:29:21,  5.66s/it]
 85%|████████▍ | 12505/14725 [1:13:25<3:34:13,  5.79s/it]
 85%|████████▍ | 12506/14725 [1:13:31<3:30:48,  5.70s/it]
 85%|████████▍ | 12507/14725 [1:13:37<3:34:45,  5.81s/it]
 85%|████████▍ | 12508/14725 [1:13:42<3:35:40,  5.84s/it]
 85%|████████▍ | 12509/14725 [1:13:48<3:35:03,  5.82s/it]
 85%|████████▍ | 12510/14725 [1:13:54<3:37:55,  5.90s/it]
                                                         

 85%|████████▍ | 12510/14725 [1:13:54<3:37:55,  5.90s/it]
 85%|████████▍ | 12511/14725 [1:14:01<3:41:42,  6.01s/it]
 85%|████████▍ | 125
0: {'loss': 0.5356, 'grad_norm': 0.38542796093949366, 'learning_rate': 3e-06, 'epoch': 0.85}
0: 12/14725 [1:14:06<3:37:36,  5.90s/it]
 85%|████████▍ | 12513/14725 [1:14:12<3:33:57,  5.80s/it]
 85%|████████▍ | 12514/14725 [1:14:18<3:38:19,  5.92s/it]
 85%|████████▍ | 12515/14725 [1:14:24<3:42:32,  6.04s/it]
 85%|████████▍ | 12516/14725 [1:14:30<3:42:33,  6.04s/it]
 85%|████████▌ | 12517/14725 [1:14:36<3:39:44,  5.97s/it]
 85%|████████▌ | 12518/14725 [1:14:42<3:36:35,  5.89s/it]
 85%|████████▌ | 12519/14725 [1:14:48<3:39:10,  5.96s/it]
 85%|████████▌ | 12520/14725 [1:14:54<3:43:42,  6.09s/it]
                                                         

 85%|████████▌ | 12520/14725 [1:14:54<3:43:42,  6.09s/it]
 85%|████████▌ | 12521/14725 [1:15:00<3:43:15,  6.08s/it]
 85%|████████▌ | 12522/14725 [1:15:07<3:45:02,  6.13s/it]
 85%|████████▌ | 12523/14725 [1:15:13<3:44:03,  6.10s/it]
 85%|███�
0: {'loss': 0.53, 'grad_norm': 0.37542753627484116, 'learning_rate': 3e-06, 'epoch': 0.85}
0: ��████▌ | 12524/14725 [1:15:19<3:45:45,  6.15s/it]
 85%|████████▌ | 12525/14725 [1:15:25<3:46:12,  6.17s/it]
 85%|████████▌ | 12526/14725 [1:15:31<3:43:24,  6.10s/it]
 85%|████████▌ | 12527/14725 [1:15:37<3:36:47,  5.92s/it]
 85%|████████▌ | 12528/14725 [1:15:42<3:31:02,  5.76s/it]
 85%|████████▌ | 12529/14725 [1:15:48<3:38:11,  5.96s/it]
 85%|████████▌ | 12530/14725 [1:15:54<3:32:21,  5.80s/it]
                                                         

 85%|████████▌ | 12530/14725 [1:15:54<3:32:21,  5.80s/it]
 85%|████████▌ | 12531/14725 [1:16:00<3:32:43,  5.82s/it]
 85%|████████▌ | 12532/14725 [1:16:05<3:31:22,  5.78s/it]
 85%|████████▌ | 12533/14725 [1:16:12<3:36:29,  5.93s/it]
 85%|████████▌ | 12534/14725 [1:16:17<3:29:08,  5.73s/it]
 85%|████████▌ | 12535/14725 [1:16:23<3:32:21,  5.
0: {'loss': 0.5331, 'grad_norm': 0.4037746507893962, 'learning_rate': 3e-06, 'epoch': 0.85}
0: 82s/it]
 85%|████████▌ | 12536/14725 [1:16:29<3:30:24,  5.77s/it]
 85%|████████▌ | 12537/14725 [1:16:34<3:26:34,  5.66s/it]
 85%|████████▌ | 12538/14725 [1:16:40<3:25:45,  5.64s/it]
 85%|████████▌ | 12539/14725 [1:16:45<3:24:22,  5.61s/it]
 85%|████████▌ | 12540/14725 [1:16:51<3:22:22,  5.56s/it]
                                                         

 85%|████████▌ | 12540/14725 [1:16:51<3:22:22,  5.56s/it]
 85%|████████▌ | 12541/14725 [1:16:57<3:26:38,  5.68s/it]
 85%|████████▌ | 12542/14725 [1:17:03<3:30:45,  5.79s/it]
 85%|████████▌ | 12543/14725 [1:17:09<3:32:47,  5.85s/it]
 85%|████████▌ | 12544/14725 [1:17:15<3:33:55,  5.88s/it]
 85%|████████▌ | 12545/14725 [1:17:21<3:37:25,  5.98s/it]
 85%|████████▌ | 12546/14725 [1:17:27<3:39:48,  6.05s/it]
 85%|████████▌ | 12547/1472
0: {'loss': 0.5223, 'grad_norm': 0.3764804088974058, 'learning_rate': 3e-06, 'epoch': 0.85}
0: 5 [1:17:33<3:43:34,  6.16s/it]
 85%|████████▌ | 12548/14725 [1:17:40<3:45:14,  6.21s/it]
 85%|████████▌ | 12549/14725 [1:17:45<3:34:25,  5.91s/it]
 85%|████████▌ | 12550/14725 [1:17:51<3:31:23,  5.83s/it]
                                                         

 85%|████████▌ | 12550/14725 [1:17:51<3:31:23,  5.83s/it]
 85%|████████▌ | 12551/14725 [1:17:57<3:31:52,  5.85s/it]
 85%|████████▌ | 12552/14725 [1:18:02<3:29:09,  5.78s/it]
 85%|████████▌ | 12553/14725 [1:18:08<3:33:26,  5.90s/it]
 85%|████████▌ | 12554/14725 [1:18:14<3:30:46,  5.83s/it]
 85%|████████▌ | 12555/14725 [1:18:19<3:24:37,  5.66s/it]
 85%|████████▌ | 12556/14725 [1:18:25<3:30:27,  5.82s/it]
 85%|████████▌ | 12557/14725 [1:18:31<3:32:07,  5.87s/it]
 85%|████████▌ | 12558/14725 [1:18:37<3:28:49,  5.78s/it]
 85%|█████�
0: {'loss': 0.518, 'grad_norm': 0.3805918137983058, 'learning_rate': 3e-06, 'epoch': 0.85}
0: �██▌ | 12559/14725 [1:18:43<3:29:34,  5.81s/it]
 85%|████████▌ | 12560/14725 [1:18:49<3:28:52,  5.79s/it]
                                                         

 85%|████████▌ | 12560/14725 [1:18:49<3:28:52,  5.79s/it]
 85%|████████▌ | 12561/14725 [1:18:55<3:34:58,  5.96s/it]
 85%|████████▌ | 12562/14725 [1:19:01<3:30:37,  5.84s/it]
 85%|████████▌ | 12563/14725 [1:19:06<3:28:17,  5.78s/it]
 85%|████████▌ | 12564/14725 [1:19:12<3:27:44,  5.77s/it]
 85%|████████▌ | 12565/14725 [1:19:17<3:24:36,  5.68s/it]
 85%|████████▌ | 12566/14725 [1:19:23<3:24:28,  5.68s/it]
 85%|████████▌ | 12567/14725 [1:19:28<3:20:44,  5.58s/it]
 85%|████████▌ | 12568/14725 [1:19:34<3:22:30,  5.63s/it]
 85%|████████▌ | 12569/14725 [1:19:40<3:25:16,  5.71s/it]
 85%|████████▌ | 12570/14725 [1:19:46<3:21:59,  5.62s/it]
0: {'loss': 0.5315, 'grad_norm': 0.3997230687023781, 'learning_rate': 3e-06, 'epoch': 0.85}
0: {'loss': 0.5303, 'grad_norm': 0.3696988913372491, 'learning_rate': 3e-06, 'epoch': 0.85}
0: 
                                                         

 85%|████████▌ | 12570/14725 [1:19:46<3:21:59,  5.62s/it]
 85%|████████▌ | 12571/14725 [1:19:51<3:22:20,  5.64s/it]
 85%|████████▌ | 12572/14725 [1:19:57<3:23:37,  5.67s/it]
 85%|████████▌ | 12573/14725 [1:20:03<3:26:54,  5.77s/it]
 85%|████████▌ | 12574/14725 [1:20:09<3:29:33,  5.85s/it]
 85%|████████▌ | 12575/14725 [1:20:15<3:27:17,  5.78s/it]
 85%|████████▌ | 12576/14725 [1:20:20<3:26:16,  5.76s/it]
 85%|████████▌ | 12577/14725 [1:20:27<3:33:04,  5.95s/it]
 85%|████████▌ | 12578/14725 [1:20:32<3:29:06,  5.84s/it]
 85%|████████▌ | 12579/14725 [1:20:38<3:25:34,  5.75s/it]
 85%|████████▌ | 12580/14725 [1:20:44<3:27:54,  5.82s/it]
                                                         

 85%|████████▌ | 12580/14725 [1:20:44<3:27:54,  5.8
0: {'loss': 0.5244, 'grad_norm': 0.3827850373350774, 'learning_rate': 3e-06, 'epoch': 0.86}
0: 2s/it]
 85%|████████▌ | 12581/14725 [1:20:49<3:21:10,  5.63s/it]
 85%|████████▌ | 12582/14725 [1:20:54<3:16:01,  5.49s/it]
 85%|████████▌ | 12583/14725 [1:21:00<3:16:38,  5.51s/it]
 85%|████████▌ | 12584/14725 [1:21:06<3:22:08,  5.66s/it]
 85%|████████▌ | 12585/14725 [1:21:11<3:16:48,  5.52s/it]
 85%|████████▌ | 12586/14725 [1:21:17<3:22:47,  5.69s/it]
 85%|████████▌ | 12587/14725 [1:21:23<3:22:58,  5.70s/it]
 85%|████████▌ | 12588/14725 [1:21:29<3:29:52,  5.89s/it]
 85%|████████▌ | 12589/14725 [1:21:35<3:32:09,  5.96s/it]
 86%|████████▌ | 12590/14725 [1:21:41<3:31:54,  5.96s/it]
                                                         

 86%|████████▌ | 12590/14725 [1:21:41<3:31:54,  5.96s/it]
 86%|████████▌ | 12591/14725 [1:21:47<3:33:20,  6.00s/it]
 86%|████████▌ | 12592/14725
0: {'loss': 0.5286, 'grad_norm': 0.38123667187142557, 'learning_rate': 3e-06, 'epoch': 0.86}
0:  [1:21:53<3:28:52,  5.88s/it]
 86%|████████▌ | 12593/14725 [1:21:59<3:31:11,  5.94s/it]
 86%|████████▌ | 12594/14725 [1:22:05<3:27:55,  5.85s/it]
 86%|████████▌ | 12595/14725 [1:22:10<3:25:36,  5.79s/it]
 86%|████████▌ | 12596/14725 [1:22:16<3:25:29,  5.79s/it]
 86%|████████▌ | 12597/14725 [1:22:21<3:22:15,  5.70s/it]
 86%|████████▌ | 12598/14725 [1:22:27<3:18:35,  5.60s/it]
 86%|████████▌ | 12599/14725 [1:22:33<3:20:54,  5.67s/it]
 86%|████████▌ | 12600/14725 [1:22:39<3:24:24,  5.77s/it]
                                                         

 86%|████████▌ | 12600/14725 [1:22:39<3:24:24,  5.77s/it]
 86%|████████▌ | 12601/14725 [1:22:45<3:25:38,  5.81s/it]
 86%|████████▌ | 12602/14725 [1:22:51<3:31:04,  5.97s/it]
 86%|████████▌ | 12603/14725 [1:22:56<3:24:29,  5.78s/it]
 86%|██████
0: {'loss': 0.5365, 'grad_norm': 0.36092333514165903, 'learning_rate': 3e-06, 'epoch': 0.86}
0: ██▌ | 12604/14725 [1:23:02<3:25:16,  5.81s/it]
 86%|████████▌ | 12605/14725 [1:23:08<3:23:23,  5.76s/it]
 86%|████████▌ | 12606/14725 [1:23:14<3:29:21,  5.93s/it]
 86%|████████▌ | 12607/14725 [1:23:20<3:33:27,  6.05s/it]
 86%|████████▌ | 12608/14725 [1:23:27<3:36:22,  6.13s/it]
 86%|████████▌ | 12609/14725 [1:23:32<3:28:09,  5.90s/it]
 86%|████████▌ | 12610/14725 [1:23:38<3:29:17,  5.94s/it]
                                                         

 86%|████████▌ | 12610/14725 [1:23:38<3:29:17,  5.94s/it]
 86%|████████▌ | 12611/14725 [1:23:44<3:32:00,  6.02s/it]
 86%|████████▌ | 12612/14725 [1:23:50<3:28:55,  5.93s/it]
 86%|████████▌ | 12613/14725 [1:23:56<3:26:17,  5.86s/it]
 86%|████████▌ | 12614/14725 [1:24:02<3:26:09,  5.86s/it]
 86%|████████▌ | 12615/14725 [1:24:07<3:20:13,  5.69s/it]
0: {'loss': 0.5268, 'grad_norm': 0.3791526917563628, 'learning_rate': 3e-06, 'epoch': 0.86}
0:  86%|████████▌ | 12616/14725 [1:24:13<3:20:53,  5.72s/it]
 86%|████████▌ | 12617/14725 [1:24:18<3:19:42,  5.68s/it]
 86%|████████▌ | 12618/14725 [1:24:24<3:22:43,  5.77s/it]
 86%|████████▌ | 12619/14725 [1:24:29<3:15:34,  5.57s/it]
 86%|████████▌ | 12620/14725 [1:24:36<3:23:21,  5.80s/it]
                                                         

 86%|████████▌ | 12620/14725 [1:24:36<3:23:21,  5.80s/it]
 86%|████████▌ | 12621/14725 [1:24:42<3:23:14,  5.80s/it]
 86%|████████▌ | 12622/14725 [1:24:47<3:22:11,  5.77s/it]
 86%|████████▌ | 12623/14725 [1:24:53<3:19:19,  5.69s/it]
 86%|████████▌ | 12624/14725 [1:24:59<3:22:07,  5.77s/it]
 86%|████████▌ | 12625/14725 [1:25:04<3:20:47,  5.74s/it]
 86%|████████▌ | 12626/14725 [1:25:10<3:20:24,  5.73s/it]
 86%|████████▌ | 12627/14725 [1:25:
0: {'loss': 0.527, 'grad_norm': 0.3743895385991036, 'learning_rate': 3e-06, 'epoch': 0.86}
0: 16<3:22:09,  5.78s/it]
 86%|████████▌ | 12628/14725 [1:25:22<3:28:35,  5.97s/it]
 86%|████████▌ | 12629/14725 [1:25:28<3:25:04,  5.87s/it]
 86%|████████▌ | 12630/14725 [1:25:34<3:25:06,  5.87s/it]
                                                         

 86%|████████▌ | 12630/14725 [1:25:34<3:25:06,  5.87s/it]
 86%|████████▌ | 12631/14725 [1:25:39<3:21:16,  5.77s/it]
 86%|████████▌ | 12632/14725 [1:25:45<3:21:31,  5.78s/it]
 86%|████████▌ | 12633/14725 [1:25:51<3:19:21,  5.72s/it]
 86%|████████▌ | 12634/14725 [1:25:56<3:17:53,  5.68s/it]
 86%|████████▌ | 12635/14725 [1:26:03<3:23:33,  5.84s/it]
 86%|████████▌ | 12636/14725 [1:26:08<3:19:40,  5.73s/it]
 86%|████████▌ | 12637/14725 [1:26:14<3:18:03,  5.69s/it]
 86%|████████▌ | 12638/14725 [1:26:19<3:18:53,  5.72s/it]
 86%|████████�
0: {'loss': 0.5143, 'grad_norm': 0.3959344051851138, 'learning_rate': 3e-06, 'epoch': 0.86}
0: �� | 12639/14725 [1:26:25<3:13:02,  5.55s/it]
 86%|████████▌ | 12640/14725 [1:26:30<3:16:15,  5.65s/it]
                                                         

 86%|████████▌ | 12640/14725 [1:26:30<3:16:15,  5.65s/it]
 86%|████████▌ | 12641/14725 [1:26:36<3:15:26,  5.63s/it]
 86%|████████▌ | 12642/14725 [1:26:42<3:14:51,  5.61s/it]
 86%|████████▌ | 12643/14725 [1:26:48<3:17:35,  5.69s/it]
 86%|████████▌ | 12644/14725 [1:26:53<3:16:02,  5.65s/it]
 86%|████████▌ | 12645/14725 [1:26:59<3:14:45,  5.62s/it]
 86%|████████▌ | 12646/14725 [1:27:04<3:11:49,  5.54s/it]
 86%|████████▌ | 12647/14725 [1:27:10<3:12:11,  5.55s/it]
 86%|████████▌ | 12648/14725 [1:27:15<3:08:32,  5.45s/it]
 86%|████████▌ | 12649/14725 [1:27:21<3:18:20,  5.73s/it]
 86%|████████▌ | 12650/14725 [1:27:27<3:24:33,  5.91s/it]
       
0: {'loss': 0.5188, 'grad_norm': 0.4276274504968776, 'learning_rate': 3e-06, 'epoch': 0.86}
0: {'loss': 0.5238, 'grad_norm': 0.390325362631331, 'learning_rate': 3e-06, 'epoch': 0.86}
0:                                                   

 86%|████████▌ | 12650/14725 [1:27:27<3:24:33,  5.91s/it]
 86%|████████▌ | 12651/14725 [1:27:33<3:18:07,  5.73s/it]
 86%|████████▌ | 12652/14725 [1:27:39<3:21:51,  5.84s/it]
 86%|████████▌ | 12653/14725 [1:27:45<3:25:01,  5.94s/it]
 86%|████████▌ | 12654/14725 [1:27:51<3:27:44,  6.02s/it]
 86%|████████▌ | 12655/14725 [1:27:57<3:24:29,  5.93s/it]
 86%|████████▌ | 12656/14725 [1:28:03<3:22:48,  5.88s/it]
 86%|████████▌ | 12657/14725 [1:28:09<3:26:52,  6.00s/it]
 86%|████████▌ | 12658/14725 [1:28:15<3:31:16,  6.13s/it]
 86%|████████▌ | 12659/14725 [1:28:21<3:23:40,  5.92s/it]
 86%|████████▌ | 12660/14725 [1:28:26<3:19:20,  5.79s/it]
                                                         

 86%|████████▌ | 12660/14725 [1:28:26<3:19:20,  5.79s/it]
 
0: {'loss': 0.5247, 'grad_norm': 0.4935307017746449, 'learning_rate': 3e-06, 'epoch': 0.86}
0: 86%|████████▌ | 12661/14725 [1:28:32<3:18:49,  5.78s/it]
 86%|████████▌ | 12662/14725 [1:28:38<3:19:42,  5.81s/it]
 86%|████████▌ | 12663/14725 [1:28:44<3:17:04,  5.73s/it]
 86%|████████▌ | 12664/14725 [1:28:50<3:20:01,  5.82s/it]
 86%|████████▌ | 12665/14725 [1:28:55<3:19:34,  5.81s/it]
 86%|████████▌ | 12666/14725 [1:29:01<3:20:17,  5.84s/it]
 86%|████████▌ | 12667/14725 [1:29:07<3:19:36,  5.82s/it]
 86%|████████▌ | 12668/14725 [1:29:13<3:19:25,  5.82s/it]
 86%|████████▌ | 12669/14725 [1:29:19<3:24:18,  5.96s/it]
 86%|████████▌ | 12670/14725 [1:29:26<3:28:18,  6.08s/it]
                                                         

 86%|████████▌ | 12670/14725 [1:29:26<3:28:18,  6.08s/it]
 86%|████████▌ | 12671/14725 [1:29:31<3:24:48,  5.98s/it]
 86%|████████▌ | 12672/14725 [1:29:3
0: {'loss': 0.5312, 'grad_norm': 0.40838671344069283, 'learning_rate': 3e-06, 'epoch': 0.86}
0: 7<3:20:37,  5.86s/it]
 86%|████████▌ | 12673/14725 [1:29:43<3:19:56,  5.85s/it]
 86%|████████▌ | 12674/14725 [1:29:49<3:20:46,  5.87s/it]
 86%|████████▌ | 12675/14725 [1:29:54<3:20:43,  5.88s/it]
 86%|████████▌ | 12676/14725 [1:30:00<3:13:44,  5.67s/it]
 86%|████████▌ | 12677/14725 [1:30:05<3:11:22,  5.61s/it]
 86%|████████▌ | 12678/14725 [1:30:11<3:12:23,  5.64s/it]
 86%|████████▌ | 12679/14725 [1:30:17<3:13:19,  5.67s/it]
 86%|████████▌ | 12680/14725 [1:30:23<3:19:23,  5.85s/it]
                                                         

 86%|████████▌ | 12680/14725 [1:30:23<3:19:23,  5.85s/it]
 86%|████████▌ | 12681/14725 [1:30:28<3:13:04,  5.67s/it]
 86%|████████▌ | 12682/14725 [1:30:34<3:11:40,  5.63s/it]
 86%|████████▌ | 12683/14725 [1:30:39<3:11:22,  5.62s/it]
 86%|████████�
0: {'loss': 0.531, 'grad_norm': 0.3979518267464433, 'learning_rate': 3e-06, 'epoch': 0.86}
0: � | 12684/14725 [1:30:45<3:09:04,  5.56s/it]
 86%|████████▌ | 12685/14725 [1:30:51<3:15:03,  5.74s/it]
 86%|████████▌ | 12686/14725 [1:30:57<3:15:53,  5.76s/it]
 86%|████████▌ | 12687/14725 [1:31:03<3:22:28,  5.96s/it]
 86%|████████▌ | 12688/14725 [1:31:09<3:18:14,  5.84s/it]
 86%|████████▌ | 12689/14725 [1:31:14<3:11:04,  5.63s/it]
 86%|████████▌ | 12690/14725 [1:31:19<3:08:28,  5.56s/it]
                                                         

 86%|████████▌ | 12690/14725 [1:31:19<3:08:28,  5.56s/it]
 86%|████████▌ | 12691/14725 [1:31:24<3:05:13,  5.46s/it]
 86%|████████▌ | 12692/14725 [1:31:30<3:05:16,  5.47s/it]
 86%|████████▌ | 12693/14725 [1:31:36<3:14:25,  5.74s/it]
 86%|████████▌ | 12694/14725 [1:31:41<3:07:43,  5.55s/it]
 86%|████████▌ | 12695/14725 [1:31:47<3:07:46,  5.55s/it]
 86%|█
0: {'loss': 0.5158, 'grad_norm': 0.3783672266015963, 'learning_rate': 3e-06, 'epoch': 0.86}
0: ███████▌ | 12696/14725 [1:31:53<3:08:13,  5.57s/it]
 86%|████████▌ | 12697/14725 [1:31:58<3:07:13,  5.54s/it]
 86%|████████▌ | 12698/14725 [1:32:04<3:14:50,  5.77s/it]
 86%|████████▌ | 12699/14725 [1:32:10<3:11:04,  5.66s/it]
 86%|████████▌ | 12700/14725 [1:32:15<3:09:47,  5.62s/it]
                                                         

 86%|████████▌ | 12700/14725 [1:32:15<3:09:47,  5.62s/it]
 86%|████████▋ | 12701/14725 [1:32:21<3:08:32,  5.59s/it]
 86%|████████▋ | 12702/14725 [1:32:26<3:09:23,  5.62s/it]
 86%|████████▋ | 12703/14725 [1:32:32<3:07:44,  5.57s/it]
 86%|████████▋ | 12704/14725 [1:32:37<3:07:27,  5.57s/it]
 86%|████████▋ | 12705/14725 [1:32:44<3:13:31,  5.75s/it]
 86%|████████▋ | 12706/14725 [1:32:49<3:10:56,  5.67s/it]
 86%|████████▋ | 12707/14725 [1:32:55<3:10:
0: {'loss': 0.5337, 'grad_norm': 0.3842227747723304, 'learning_rate': 3e-06, 'epoch': 0.86}
0: 58,  5.68s/it]
 86%|████████▋ | 12708/14725 [1:33:01<3:16:15,  5.84s/it]
 86%|████████▋ | 12709/14725 [1:33:07<3:17:14,  5.87s/it]
 86%|████████▋ | 12710/14725 [1:33:13<3:14:15,  5.78s/it]
                                                         

 86%|████████▋ | 12710/14725 [1:33:13<3:14:15,  5.78s/it]
 86%|████████▋ | 12711/14725 [1:33:19<3:19:11,  5.93s/it]
 86%|████████▋ | 12712/14725 [1:33:25<3:20:48,  5.99s/it]
 86%|████████▋ | 12713/14725 [1:33:31<3:24:31,  6.10s/it]
 86%|████████▋ | 12714/14725 [1:33:37<3:23:50,  6.08s/it]
 86%|████████▋ | 12715/14725 [1:33:44<3:26:10,  6.15s/it]
 86%|████████▋ | 12716/14725 [1:33:49<3:20:23,  5.98s/it]
 86%|████████▋ | 12717/14725 [1:33:55<3:13:44,  5.79s/it]
 86%|████████▋ | 12718/14725 [1:34:00<3:09:39,  5.67s/it]
 86%|████████▋ | 127
0: {'loss': 0.5221, 'grad_norm': 0.38510169241460984, 'learning_rate': 3e-06, 'epoch': 0.86}
0: 19/14725 [1:34:06<3:08:14,  5.63s/it]
 86%|████████▋ | 12720/14725 [1:34:11<3:04:33,  5.52s/it]
                                                         

 86%|████████▋ | 12720/14725 [1:34:11<3:04:33,  5.52s/it]
 86%|████████▋ | 12721/14725 [1:34:16<3:05:20,  5.55s/it]
 86%|████████▋ | 12722/14725 [1:34:22<3:07:13,  5.61s/it]
 86%|████████▋ | 12723/14725 [1:34:28<3:07:24,  5.62s/it]
 86%|████████▋ | 12724/14725 [1:34:33<3:06:00,  5.58s/it]
 86%|████████▋ | 12725/14725 [1:34:39<3:03:31,  5.51s/it]
 86%|████████▋ | 12726/14725 [1:34:44<3:02:19,  5.47s/it]
 86%|████████▋ | 12727/14725 [1:34:49<3:00:59,  5.44s/it]
 86%|████████▋ | 12728/14725 [1:34:55<3:00:22,  5.42s/it]
 86%|████████▋ | 12729/14725 [1:35:01<3:04:48,  5.56s/it]
 86%|████████▋ | 12730/14725 [1:35:07<3:11:14,  5.75s/it]
               
0: {'loss': 0.515, 'grad_norm': 0.3989603173470104, 'learning_rate': 3e-06, 'epoch': 0.86}
0: {'loss': 0.531, 'grad_norm': 0.38958858790453427, 'learning_rate': 3e-06, 'epoch': 0.87}
0:                                           

 86%|████████▋ | 12730/14725 [1:35:07<3:11:14,  5.75s/it]
 86%|████████▋ | 12731/14725 [1:35:13<3:13:39,  5.83s/it]
 86%|████████▋ | 12732/14725 [1:35:18<3:11:54,  5.78s/it]
 86%|████████▋ | 12733/14725 [1:35:25<3:15:47,  5.90s/it]
 86%|████████▋ | 12734/14725 [1:35:30<3:11:46,  5.78s/it]
 86%|████████▋ | 12735/14725 [1:35:36<3:13:39,  5.84s/it]
 86%|████████▋ | 12736/14725 [1:35:42<3:10:59,  5.76s/it]
 86%|████████▋ | 12737/14725 [1:35:48<3:11:45,  5.79s/it]
 87%|████████▋ | 12738/14725 [1:35:54<3:14:23,  5.87s/it]
 87%|████████▋ | 12739/14725 [1:36:00<3:18:53,  6.01s/it]
 87%|████████▋ | 12740/14725 [1:36:06<3:18:45,  6.01s/it]
                                                         

 87%|████████▋ | 12740/14725 [1:36:06<3:18:45,  6.01s/it]
 87%|█�
0: {'loss': 0.5212, 'grad_norm': 0.3962288508543022, 'learning_rate': 3e-06, 'epoch': 0.87}
0: ��██████▋ | 12741/14725 [1:36:12<3:22:55,  6.14s/it]
 87%|████████▋ | 12742/14725 [1:36:18<3:20:36,  6.07s/it]
 87%|████████▋ | 12743/14725 [1:36:24<3:18:28,  6.01s/it]
 87%|████████▋ | 12744/14725 [1:36:30<3:15:07,  5.91s/it]
 87%|████████▋ | 12745/14725 [1:36:36<3:14:03,  5.88s/it]
 87%|████████▋ | 12746/14725 [1:36:41<3:13:05,  5.85s/it]
 87%|████████▋ | 12747/14725 [1:36:48<3:14:43,  5.91s/it]
 87%|████████▋ | 12748/14725 [1:36:53<3:14:20,  5.90s/it]
 87%|████████▋ | 12749/14725 [1:36:59<3:09:04,  5.74s/it]
 87%|████████▋ | 12750/14725 [1:37:04<3:04:02,  5.59s/it]
                                                         

 87%|████████▋ | 12750/14725 [1:37:04<3:04:02,  5.59s/it]
 87%|████████▋ | 12751/14725 [1:37:10<3:03:56,  5.59s/it]
 87%|████████▋ | 12752/14725 [1:37:15<3:06:3
0: {'loss': 0.5307, 'grad_norm': 0.38304524795042977, 'learning_rate': 3e-06, 'epoch': 0.87}
0: 5,  5.67s/it]
 87%|████████▋ | 12753/14725 [1:37:21<3:02:48,  5.56s/it]
 87%|████████▋ | 12754/14725 [1:37:27<3:04:42,  5.62s/it]
 87%|████████▋ | 12755/14725 [1:37:32<3:01:19,  5.52s/it]
 87%|████████▋ | 12756/14725 [1:37:38<3:08:51,  5.75s/it]
 87%|████████▋ | 12757/14725 [1:37:44<3:13:44,  5.91s/it]
 87%|████████▋ | 12758/14725 [1:37:50<3:15:19,  5.96s/it]
 87%|████████▋ | 12759/14725 [1:37:57<3:17:16,  6.02s/it]
 87%|████████▋ | 12760/14725 [1:38:03<3:16:55,  6.01s/it]
                                                         

 87%|████████▋ | 12760/14725 [1:38:03<3:16:55,  6.01s/it]
 87%|████████▋ | 12761/14725 [1:38:08<3:14:01,  5.93s/it]
 87%|████████▋ | 12762/14725 [1:38:15<3:17:31,  6.04s/it]
 87%|████████▋ | 12763/14725 [1:38:20<3:12:28,  5.89s/it]
 87%|████████▋ | 1276
0: {'loss': 0.5285, 'grad_norm': 0.38417021760888886, 'learning_rate': 3e-06, 'epoch': 0.87}
0: 4/14725 [1:38:27<3:16:56,  6.03s/it]
 87%|████████▋ | 12765/14725 [1:38:32<3:13:03,  5.91s/it]
 87%|████████▋ | 12766/14725 [1:38:38<3:13:27,  5.93s/it]
 87%|████████▋ | 12767/14725 [1:38:43<3:06:43,  5.72s/it]
 87%|████████▋ | 12768/14725 [1:38:49<3:08:44,  5.79s/it]
 87%|████████▋ | 12769/14725 [1:38:55<3:08:42,  5.79s/it]
 87%|████████▋ | 12770/14725 [1:39:01<3:08:55,  5.80s/it]
                                                         

 87%|████████▋ | 12770/14725 [1:39:01<3:08:55,  5.80s/it]
 87%|████████▋ | 12771/14725 [1:39:06<3:06:31,  5.73s/it]
 87%|████████▋ | 12772/14725 [1:39:13<3:11:38,  5.89s/it]
 87%|████████▋ | 12773/14725 [1:39:18<3:10:06,  5.84s/it]
 87%|████████▋ | 12774/14725 [1:39:24<3:05:46,  5.71s/it]
 87%|████████▋ | 12775/14725 [1:39:30<3:09:16,  5.82s/it]
 87%|███�
0: {'loss': 0.5335, 'grad_norm': 0.40767047742159074, 'learning_rate': 3e-06, 'epoch': 0.87}
0: �████▋ | 12776/14725 [1:39:35<3:06:05,  5.73s/it]
 87%|████████▋ | 12777/14725 [1:39:42<3:12:13,  5.92s/it]
 87%|████████▋ | 12778/14725 [1:39:48<3:13:12,  5.95s/it]
 87%|████████▋ | 12779/14725 [1:39:54<3:17:00,  6.07s/it]
 87%|████████▋ | 12780/14725 [1:40:00<3:12:14,  5.93s/it]
                                                         

 87%|████████▋ | 12780/14725 [1:40:00<3:12:14,  5.93s/it]
 87%|████████▋ | 12781/14725 [1:40:05<3:04:51,  5.71s/it]
 87%|████████▋ | 12782/14725 [1:40:11<3:06:51,  5.77s/it]
 87%|████████▋ | 12783/14725 [1:40:17<3:09:02,  5.84s/it]
 87%|████████▋ | 12784/14725 [1:40:23<3:06:26,  5.76s/it]
 87%|████████▋ | 12785/14725 [1:40:28<3:03:29,  5.68s/it]
 87%|████████▋ | 12786/14725 [1:40:34<3:03:29,  5.68s/it]
 87%|████████▋ | 12787/14725 [1:40:40<3:08:00,  5.8
0: {'loss': 0.5209, 'grad_norm': 0.37855473764910375, 'learning_rate': 3e-06, 'epoch': 0.87}
0: 2s/it]
 87%|████████▋ | 12788/14725 [1:40:45<3:04:13,  5.71s/it]
 87%|████████▋ | 12789/14725 [1:40:51<3:01:06,  5.61s/it]
 87%|████████▋ | 12790/14725 [1:40:57<3:04:58,  5.74s/it]
                                                         

 87%|████████▋ | 12790/14725 [1:40:57<3:04:58,  5.74s/it]
 87%|████████▋ | 12791/14725 [1:41:02<3:02:14,  5.65s/it]
 87%|████████▋ | 12792/14725 [1:41:08<3:01:19,  5.63s/it]
 87%|████████▋ | 12793/14725 [1:41:14<3:02:50,  5.68s/it]
 87%|████████▋ | 12794/14725 [1:41:20<3:08:42,  5.86s/it]
 87%|████████▋ | 12795/14725 [1:41:26<3:11:10,  5.94s/it]
 87%|████████▋ | 12796/14725 [1:41:32<3:13:21,  6.01s/it]
 87%|████████▋ | 12797/14725 [1:41:38<3:08:11,  5.86s/it]
 87%|████████▋ | 12798/14725 [1:41:43<3:06:55,  5.82s/it]
 87%|████████▋ | 12799/14725
0: {'loss': 0.5358, 'grad_norm': 0.3993726406994716, 'learning_rate': 3e-06, 'epoch': 0.87}
0:  [1:41:49<3:07:01,  5.83s/it]
 87%|████████▋ | 12800/14725 [1:41:55<3:06:11,  5.80s/it]
                                                         

 87%|████████▋ | 12800/14725 [1:41:55<3:06:11,  5.80s/it]
 87%|████████▋ | 12801/14725 [1:42:00<3:02:58,  5.71s/it]
 87%|████████▋ | 12802/14725 [1:42:06<3:04:47,  5.77s/it]
 87%|████████▋ | 12803/14725 [1:42:12<3:06:15,  5.81s/it]
 87%|████████▋ | 12804/14725 [1:42:18<3:04:55,  5.78s/it]
 87%|████████▋ | 12805/14725 [1:42:23<3:00:11,  5.63s/it]
 87%|████████▋ | 12806/14725 [1:42:29<3:00:58,  5.66s/it]
 87%|████████▋ | 12807/14725 [1:42:34<2:57:14,  5.54s/it]
 87%|████████▋ | 12808/14725 [1:42:40<2:58:44,  5.59s/it]
 87%|████████▋ | 12809/14725 [1:42:46<3:01:17,  5.68s/it]
 87%|████████▋ | 12810/14725 [1:42:52<3:05:09,  5.80s/it]
                       
0: {'loss': 0.5156, 'grad_norm': 0.41207589851506277, 'learning_rate': 3e-06, 'epoch': 0.87}
0: {'loss': 0.5206, 'grad_norm': 0.36972094695951435, 'learning_rate': 3e-06, 'epoch': 0.87}
0:                                   

 87%|████████▋ | 12810/14725 [1:42:52<3:05:09,  5.80s/it]
 87%|████████▋ | 12811/14725 [1:42:57<3:02:41,  5.73s/it]
 87%|████████▋ | 12812/14725 [1:43:03<2:59:51,  5.64s/it]
 87%|████████▋ | 12813/14725 [1:43:08<2:58:23,  5.60s/it]
 87%|████████▋ | 12814/14725 [1:43:15<3:04:05,  5.78s/it]
 87%|████████▋ | 12815/14725 [1:43:20<2:58:11,  5.60s/it]
 87%|████████▋ | 12816/14725 [1:43:25<2:59:14,  5.63s/it]
 87%|████████▋ | 12817/14725 [1:43:31<3:00:27,  5.67s/it]
 87%|████████▋ | 12818/14725 [1:43:38<3:06:52,  5.88s/it]
 87%|████████▋ | 12819/14725 [1:43:44<3:09:34,  5.97s/it]
 87%|████████▋ | 12820/14725 [1:43:49<3:03:40,  5.79s/it]
                                                         

 87%|████████▋ | 12820/14725 [1:43:49<3:03:40,  5.79s/it]
 87%|████
0: {'loss': 0.5255, 'grad_norm': 0.38924686844853856, 'learning_rate': 3e-06, 'epoch': 0.87}
0: ████▋ | 12821/14725 [1:43:54<2:57:46,  5.60s/it]
 87%|████████▋ | 12822/14725 [1:44:00<2:59:43,  5.67s/it]
 87%|████████▋ | 12823/14725 [1:44:06<2:58:28,  5.63s/it]
 87%|████████▋ | 12824/14725 [1:44:11<2:57:13,  5.59s/it]
 87%|████████▋ | 12825/14725 [1:44:17<2:55:35,  5.54s/it]
 87%|████████▋ | 12826/14725 [1:44:22<2:52:08,  5.44s/it]
 87%|████████▋ | 12827/14725 [1:44:28<2:59:25,  5.67s/it]
 87%|████████▋ | 12828/14725 [1:44:34<2:59:13,  5.67s/it]
 87%|████████▋ | 12829/14725 [1:44:39<2:58:52,  5.66s/it]
 87%|████████▋ | 12830/14725 [1:44:45<2:58:26,  5.65s/it]
                                                         

 87%|████████▋ | 12830/14725 [1:44:45<2:58:26,  5.65s/it]
 87%|████████▋ | 12831/14725 [1:44:51<3:01:36,  5.75s/it]
 87%|████████▋ | 12832/14725 [1:44:57<3:00:17,  5.71
0: {'loss': 0.5212, 'grad_norm': 0.37994003467903853, 'learning_rate': 3e-06, 'epoch': 0.87}
0: s/it]
 87%|████████▋ | 12833/14725 [1:45:02<2:58:12,  5.65s/it]
 87%|████████▋ | 12834/14725 [1:45:07<2:51:45,  5.45s/it]
 87%|████████▋ | 12835/14725 [1:45:12<2:49:30,  5.38s/it]
 87%|████████▋ | 12836/14725 [1:45:19<2:58:17,  5.66s/it]
 87%|████████▋ | 12837/14725 [1:45:24<2:57:27,  5.64s/it]
 87%|████████▋ | 12838/14725 [1:45:30<2:57:37,  5.65s/it]
 87%|████████▋ | 12839/14725 [1:45:36<3:05:06,  5.89s/it]
 87%|████████▋ | 12840/14725 [1:45:42<3:00:30,  5.75s/it]
                                                         

 87%|████████▋ | 12840/14725 [1:45:42<3:00:30,  5.75s/it]
 87%|████████▋ | 12841/14725 [1:45:48<3:00:59,  5.76s/it]
 87%|████████▋ | 12842/14725 [1:45:53<3:01:18,  5.78s/it]
 87%|████████▋ | 12843/14725 [1:45:59<3:01:06,  5.77s/it]
 87%|████████▋ | 12844/14725 
0: {'loss': 0.518, 'grad_norm': 0.36651195852138346, 'learning_rate': 3e-06, 'epoch': 0.87}
0: [1:46:05<2:59:16,  5.72s/it]
 87%|████████▋ | 12845/14725 [1:46:10<2:56:11,  5.62s/it]
 87%|████████▋ | 12846/14725 [1:46:16<2:56:51,  5.65s/it]
 87%|████████▋ | 12847/14725 [1:46:22<3:00:03,  5.75s/it]
 87%|████████▋ | 12848/14725 [1:46:27<2:56:42,  5.65s/it]
 87%|████████▋ | 12849/14725 [1:46:33<2:58:41,  5.72s/it]
 87%|████████▋ | 12850/14725 [1:46:39<2:58:28,  5.71s/it]
                                                         

 87%|████████▋ | 12850/14725 [1:46:39<2:58:28,  5.71s/it]
 87%|████████▋ | 12851/14725 [1:46:44<2:56:05,  5.64s/it]
 87%|████████▋ | 12852/14725 [1:46:50<2:56:47,  5.66s/it]
 87%|████████▋ | 12853/14725 [1:46:55<2:53:54,  5.57s/it]
 87%|████████▋ | 12854/14725 [1:47:02<3:01:35,  5.82s/it]
 87%|████████▋ | 12855/14725 [1:47:07<3:00:02,  5.78s/it]
 87%|██████�
0: {'loss': 0.5279, 'grad_norm': 0.36546841627496107, 'learning_rate': 3e-06, 'epoch': 0.87}
0: ��█▋ | 12856/14725 [1:47:13<2:58:48,  5.74s/it]
 87%|████████▋ | 12857/14725 [1:47:19<3:03:25,  5.89s/it]
 87%|████████▋ | 12858/14725 [1:47:25<3:02:23,  5.86s/it]
 87%|████████▋ | 12859/14725 [1:47:31<3:02:01,  5.85s/it]
 87%|████████▋ | 12860/14725 [1:47:37<3:04:28,  5.93s/it]
                                                         

 87%|████████▋ | 12860/14725 [1:47:37<3:04:28,  5.93s/it]
 87%|████████▋ | 12861/14725 [1:47:43<3:01:32,  5.84s/it]
 87%|████████▋ | 12862/14725 [1:47:48<2:59:56,  5.80s/it]
 87%|████████▋ | 12863/14725 [1:47:54<3:01:19,  5.84s/it]
 87%|████████▋ | 12864/14725 [1:48:00<3:02:32,  5.89s/it]
 87%|████████▋ | 12865/14725 [1:48:06<3:05:04,  5.97s/it]
 87%|████████▋ | 12866/14725 [1:48:13<3:07:37,  6.06s/it]
 87%|████████▋ | 12867/14725 [1:48:19<3:05:52,  6.00s/it]
 
0: {'loss': 0.5279, 'grad_norm': 0.3935433133395605, 'learning_rate': 3e-06, 'epoch': 0.87}
0: 87%|████████▋ | 12868/14725 [1:48:25<3:08:22,  6.09s/it]
 87%|████████▋ | 12869/14725 [1:48:31<3:06:22,  6.03s/it]
 87%|████████▋ | 12870/14725 [1:48:37<3:06:30,  6.03s/it]
                                                         

 87%|████████▋ | 12870/14725 [1:48:37<3:06:30,  6.03s/it]
 87%|████████▋ | 12871/14725 [1:48:42<3:02:48,  5.92s/it]
 87%|████████▋ | 12872/14725 [1:48:48<2:57:48,  5.76s/it]
 87%|████████▋ | 12873/14725 [1:48:54<2:57:35,  5.75s/it]
 87%|████████▋ | 12874/14725 [1:49:00<2:59:48,  5.83s/it]
 87%|████████▋ | 12875/14725 [1:49:06<3:03:05,  5.94s/it]
 87%|████████▋ | 12876/14725 [1:49:12<3:05:26,  6.02s/it]
 87%|████████▋ | 12877/14725 [1:49:17<3:00:01,  5.84s/it]
 87%|████████▋ | 12878/14725 [1:49:24<3:04:18,  5.99s/it]
 87%|████████▋ | 12879/14725 [1:49:2
0: {'loss': 0.5261, 'grad_norm': 0.415328701028454, 'learning_rate': 3e-06, 'epoch': 0.87}
0: 9<2:57:33,  5.77s/it]
 87%|████████▋ | 12880/14725 [1:49:35<2:55:47,  5.72s/it]
                                                         

 87%|████████▋ | 12880/14725 [1:49:35<2:55:47,  5.72s/it]
 87%|████████▋ | 12881/14725 [1:49:41<2:59:49,  5.85s/it]
 87%|████████▋ | 12882/14725 [1:49:46<2:57:09,  5.77s/it]
 87%|████████▋ | 12883/14725 [1:49:52<2:54:06,  5.67s/it]
 87%|████████▋ | 12884/14725 [1:49:57<2:51:10,  5.58s/it]
 88%|████████▊ | 12885/14725 [1:50:03<2:57:13,  5.78s/it]
 88%|████████▊ | 12886/14725 [1:50:09<2:58:52,  5.84s/it]
 88%|████████▊ | 12887/14725 [1:50:15<2:55:59,  5.75s/it]
 88%|████████▊ | 12888/14725 [1:50:21<2:56:18,  5.76s/it]
 88%|████████▊ | 12889/14725 [1:50:26<2:53:42,  5.68s/it]
 88%|████████▊ | 12890/14725 [1:50:31<2:49:05,  5.53s/it]
                               
0: {'loss': 0.5304, 'grad_norm': 0.38551587430647133, 'learning_rate': 3e-06, 'epoch': 0.88}
0: {'loss': 0.531, 'grad_norm': 0.3818893177550598, 'learning_rate': 3e-06, 'epoch': 0.88}
0:                           

 88%|████████▊ | 12890/14725 [1:50:31<2:49:05,  5.53s/it]
 88%|████████▊ | 12891/14725 [1:50:37<2:51:30,  5.61s/it]
 88%|████████▊ | 12892/14725 [1:50:43<2:57:43,  5.82s/it]
 88%|████████▊ | 12893/14725 [1:50:49<2:53:27,  5.68s/it]
 88%|████████▊ | 12894/14725 [1:50:55<2:54:40,  5.72s/it]
 88%|████████▊ | 12895/14725 [1:51:00<2:53:59,  5.70s/it]
 88%|████████▊ | 12896/14725 [1:51:06<2:52:34,  5.66s/it]
 88%|████████▊ | 12897/14725 [1:51:12<2:58:09,  5.85s/it]
 88%|████████▊ | 12898/14725 [1:51:18<3:00:25,  5.93s/it]
 88%|████████▊ | 12899/14725 [1:51:24<2:58:31,  5.87s/it]
 88%|████████▊ | 12900/14725 [1:51:29<2:54:02,  5.72s/it]
                                                         

 88%|████████▊ | 12900/14725 [1:51:29<2:54:02,  5.72s/it]
 88%|██████�
0: {'loss': 0.529, 'grad_norm': 0.3855559467837739, 'learning_rate': 3e-06, 'epoch': 0.88}
0: �█▊ | 12901/14725 [1:51:35<2:52:18,  5.67s/it]
 88%|████████▊ | 12902/14725 [1:51:40<2:49:50,  5.59s/it]
 88%|████████▊ | 12903/14725 [1:51:46<2:47:08,  5.50s/it]
 88%|████████▊ | 12904/14725 [1:51:51<2:49:03,  5.57s/it]
 88%|████████▊ | 12905/14725 [1:51:57<2:45:40,  5.46s/it]
 88%|████████▊ | 12906/14725 [1:52:02<2:47:22,  5.52s/it]
 88%|████████▊ | 12907/14725 [1:52:08<2:48:18,  5.55s/it]
 88%|████████▊ | 12908/14725 [1:52:14<2:49:30,  5.60s/it]
 88%|████████▊ | 12909/14725 [1:52:19<2:50:05,  5.62s/it]
 88%|████████▊ | 12910/14725 [1:52:25<2:48:58,  5.59s/it]
                                                         

 88%|████████▊ | 12910/14725 [1:52:25<2:48:58,  5.59s/it]
 88%|████████▊ | 12911/14725 [1:52:30<2:47:40,  5.55s/it]
 88%|████████▊ | 12912/14725 [1:52:36<2:53:30,  5.74s/it]
 8
0: {'loss': 0.516, 'grad_norm': 0.4122354527047955, 'learning_rate': 3e-06, 'epoch': 0.88}
0: 8%|████████▊ | 12913/14725 [1:52:43<2:58:06,  5.90s/it]
 88%|████████▊ | 12914/14725 [1:52:49<3:01:44,  6.02s/it]
 88%|████████▊ | 12915/14725 [1:52:55<3:05:00,  6.13s/it]
 88%|████████▊ | 12916/14725 [1:53:01<2:56:58,  5.87s/it]
 88%|████████▊ | 12917/14725 [1:53:06<2:53:56,  5.77s/it]
 88%|████████▊ | 12918/14725 [1:53:12<2:53:24,  5.76s/it]
 88%|████████▊ | 12919/14725 [1:53:18<2:57:50,  5.91s/it]
 88%|████████▊ | 12920/14725 [1:53:24<2:56:24,  5.86s/it]
                                                         

 88%|████████▊ | 12920/14725 [1:53:24<2:56:24,  5.86s/it]
 88%|████████▊ | 12921/14725 [1:53:29<2:53:42,  5.78s/it]
 88%|████████▊ | 12922/14725 [1:53:36<2:57:55,  5.92s/it]
 88%|████████▊ | 12923/14725 [1:53:42<3:01:02,  6.03s/it]
 88%|████████▊ | 12924/14725 [1:53:48
0: {'loss': 0.5308, 'grad_norm': 0.3993155417997744, 'learning_rate': 3e-06, 'epoch': 0.88}
0: <2:57:30,  5.91s/it]
 88%|████████▊ | 12925/14725 [1:53:53<2:54:18,  5.81s/it]
 88%|████████▊ | 12926/14725 [1:53:59<2:54:29,  5.82s/it]
 88%|████████▊ | 12927/14725 [1:54:05<2:53:49,  5.80s/it]
 88%|████████▊ | 12928/14725 [1:54:10<2:51:45,  5.74s/it]
 88%|████████▊ | 12929/14725 [1:54:16<2:47:24,  5.59s/it]
 88%|████████▊ | 12930/14725 [1:54:22<2:53:58,  5.82s/it]
                                                         

 88%|████████▊ | 12930/14725 [1:54:22<2:53:58,  5.82s/it]
 88%|████████▊ | 12931/14725 [1:54:28<2:51:51,  5.75s/it]
 88%|████████▊ | 12932/14725 [1:54:33<2:51:58,  5.75s/it]
 88%|████████▊ | 12933/14725 [1:54:40<2:58:58,  5.99s/it]
 88%|████████▊ | 12934/14725 [1:54:46<3:02:04,  6.10s/it]
 88%|████████▊ | 12935/14725 [1:54:52<2:54:44,  5.86s/it]
 88%|████████▊
0: {'loss': 0.5279, 'grad_norm': 0.37047908242794386, 'learning_rate': 3e-06, 'epoch': 0.88}
0:  | 12936/14725 [1:54:58<3:00:42,  6.06s/it]
 88%|████████▊ | 12937/14725 [1:55:04<3:00:34,  6.06s/it]
 88%|████████▊ | 12938/14725 [1:55:10<2:56:08,  5.91s/it]
 88%|████████▊ | 12939/14725 [1:55:15<2:54:47,  5.87s/it]
 88%|████████▊ | 12940/14725 [1:55:21<2:52:08,  5.79s/it]
                                                         

 88%|████████▊ | 12940/14725 [1:55:21<2:52:08,  5.79s/it]
 88%|████████▊ | 12941/14725 [1:55:27<2:51:16,  5.76s/it]
 88%|████████▊ | 12942/14725 [1:55:33<2:51:07,  5.76s/it]
 88%|████████▊ | 12943/14725 [1:55:38<2:47:37,  5.64s/it]
 88%|████████▊ | 12944/14725 [1:55:43<2:45:44,  5.58s/it]
 88%|████████▊ | 12945/14725 [1:55:49<2:46:48,  5.62s/it]
 88%|████████▊ | 12946/14725 [1:55:55<2:49:23,  5.71s/it]
 88%|████████▊ | 12947/14725 [1:56:01<2:54:40,  5.89s/it]
 88%|█�
0: {'loss': 0.522, 'grad_norm': 0.3648198185566628, 'learning_rate': 3e-06, 'epoch': 0.88}
0: ��██████▊ | 12948/14725 [1:56:07<2:53:05,  5.84s/it]
 88%|████████▊ | 12949/14725 [1:56:13<2:51:21,  5.79s/it]
 88%|████████▊ | 12950/14725 [1:56:18<2:50:39,  5.77s/it]
                                                         

 88%|████████▊ | 12950/14725 [1:56:18<2:50:39,  5.77s/it]
 88%|████████▊ | 12951/14725 [1:56:25<2:55:43,  5.94s/it]
 88%|████████▊ | 12952/14725 [1:56:31<2:57:04,  5.99s/it]
 88%|████████▊ | 12953/14725 [1:56:37<2:56:15,  5.97s/it]
 88%|████████▊ | 12954/14725 [1:56:42<2:50:27,  5.78s/it]
 88%|████████▊ | 12955/14725 [1:56:48<2:49:32,  5.75s/it]
 88%|████████▊ | 12956/14725 [1:56:54<2:49:28,  5.75s/it]
 88%|████████▊ | 12957/14725 [1:56:59<2:48:53,  5.73s/it]
 88%|████████▊ | 12958/14725 [1:57:06<2:54:27,  5.92s/it]
 88%|████████▊ | 12959/14725 [1:57:12<2:55:3
0: {'loss': 0.5282, 'grad_norm': 0.37886351383065187, 'learning_rate': 3e-06, 'epoch': 0.88}
0: 2,  5.96s/it]
 88%|████████▊ | 12960/14725 [1:57:17<2:53:56,  5.91s/it]
                                                         

 88%|████████▊ | 12960/14725 [1:57:17<2:53:56,  5.91s/it]
 88%|████████▊ | 12961/14725 [1:57:23<2:53:12,  5.89s/it]
 88%|████████▊ | 12962/14725 [1:57:29<2:51:06,  5.82s/it]
 88%|████████▊ | 12963/14725 [1:57:35<2:49:23,  5.77s/it]
 88%|████████▊ | 12964/14725 [1:57:40<2:45:58,  5.65s/it]
 88%|████████▊ | 12965/14725 [1:57:45<2:43:24,  5.57s/it]
 88%|████████▊ | 12966/14725 [1:57:51<2:45:20,  5.64s/it]
 88%|████████▊ | 12967/14725 [1:57:57<2:45:52,  5.66s/it]
 88%|████████▊ | 12968/14725 [1:58:03<2:45:50,  5.66s/it]
 88%|████████▊ | 12969/14725 [1:58:09<2:50:40,  5.83s/it]
 88%|████████▊ | 12970/14725 [1:58:14<2:43:53,  5.60s/it]
                                       
0: {'loss': 0.5254, 'grad_norm': 0.3880744400905477, 'learning_rate': 3e-06, 'epoch': 0.88}
0: {'loss': 0.5133, 'grad_norm': 0.3894045052718756, 'learning_rate': 3e-06, 'epoch': 0.88}
0:                   

 88%|████████▊ | 12970/14725 [1:58:14<2:43:53,  5.60s/it]
 88%|████████▊ | 12971/14725 [1:58:20<2:44:51,  5.64s/it]
 88%|████████▊ | 12972/14725 [1:58:26<2:49:21,  5.80s/it]
 88%|████████▊ | 12973/14725 [1:58:32<2:54:35,  5.98s/it]
 88%|████████▊ | 12974/14725 [1:58:38<2:51:02,  5.86s/it]
 88%|████████▊ | 12975/14725 [1:58:43<2:44:57,  5.66s/it]
 88%|████████▊ | 12976/14725 [1:58:49<2:45:55,  5.69s/it]
 88%|████████▊ | 12977/14725 [1:58:54<2:41:23,  5.54s/it]
 88%|████████▊ | 12978/14725 [1:59:00<2:45:28,  5.68s/it]
 88%|████████▊ | 12979/14725 [1:59:06<2:49:08,  5.81s/it]
 88%|████████▊ | 12980/14725 [1:59:12<2:49:36,  5.83s/it]
                                                         

 88%|████████▊ | 12980/14725 [1:59:12<2:49:36,  5.83s/it]
 88%|████████▊ 
0: {'loss': 0.5246, 'grad_norm': 0.3940480495386162, 'learning_rate': 3e-06, 'epoch': 0.88}
0: | 12981/14725 [1:59:18<2:51:53,  5.91s/it]
 88%|████████▊ | 12982/14725 [1:59:24<2:50:00,  5.85s/it]
 88%|████████▊ | 12983/14725 [1:59:29<2:46:36,  5.74s/it]
 88%|████████▊ | 12984/14725 [1:59:35<2:48:37,  5.81s/it]
 88%|████████▊ | 12985/14725 [1:59:41<2:51:34,  5.92s/it]
 88%|████████▊ | 12986/14725 [1:59:47<2:49:23,  5.84s/it]
 88%|████████▊ | 12987/14725 [1:59:53<2:47:05,  5.77s/it]
 88%|████████▊ | 12988/14725 [1:59:59<2:49:53,  5.87s/it]
 88%|████████▊ | 12989/14725 [2:00:04<2:47:08,  5.78s/it]
 88%|████████▊ | 12990/14725 [2:00:10<2:43:20,  5.65s/it]
                                                         

 88%|████████▊ | 12990/14725 [2:00:10<2:43:20,  5.65s/it]
 88%|████████▊ | 12991/14725 [2:00:15<2:43:22,  5.65s/it]
 88%|████████▊ | 12992/14725 [2:00:21<2:48:04,  5.82s/it]
 88%|█�
0: {'loss': 0.5294, 'grad_norm': 0.38783940468316, 'learning_rate': 3e-06, 'epoch': 0.88}
0: �██████▊ | 12993/14725 [2:00:27<2:46:38,  5.77s/it]
 88%|████████▊ | 12994/14725 [2:00:33<2:47:10,  5.79s/it]
 88%|████████▊ | 12995/14725 [2:00:39<2:49:56,  5.89s/it]
 88%|████████▊ | 12996/14725 [2:00:45<2:52:23,  5.98s/it]
 88%|████████▊ | 12997/14725 [2:00:51<2:53:12,  6.01s/it]
 88%|████████▊ | 12998/14725 [2:00:57<2:52:52,  6.01s/it]
 88%|████████▊ | 12999/14725 [2:01:03<2:52:35,  6.00s/it]
 88%|████████▊ | 13000/14725 [2:01:09<2:53:46,  6.04s/it]
                                                         

 88%|████████▊ | 13000/14725 [2:01:09<2:53:46,  6.04s/it]
 88%|████████▊ | 13001/14725 [2:01:16<2:54:00,  6.06s/it]
 88%|████████▊ | 13002/14725 [2:01:21<2:47:33,  5.83s/it]
 88%|████████▊ | 13003/14725 [2:01:27<2:47:06,  5.82s/it]
 88%|████████▊ | 13004/14725 [2:01:33<2:47:08
0: {'loss': 0.5163, 'grad_norm': 0.3806342480563682, 'learning_rate': 3e-06, 'epoch': 0.88}
0: ,  5.83s/it]
 88%|████████▊ | 13005/14725 [2:01:38<2:47:34,  5.85s/it]
 88%|████████▊ | 13006/14725 [2:01:44<2:44:33,  5.74s/it]
 88%|████████▊ | 13007/14725 [2:01:50<2:44:25,  5.74s/it]
 88%|████████▊ | 13008/14725 [2:01:55<2:44:35,  5.75s/it]
 88%|████████▊ | 13009/14725 [2:02:01<2:46:57,  5.84s/it]
 88%|████████▊ | 13010/14725 [2:02:08<2:51:39,  6.01s/it]
                                                         

 88%|████████▊ | 13010/14725 [2:02:08<2:51:39,  6.01s/it]
 88%|████████▊ | 13011/14725 [2:02:13<2:48:33,  5.90s/it]
 88%|████████▊ | 13012/14725 [2:02:20<2:51:49,  6.02s/it]
 88%|████████▊ | 13013/14725 [2:02:26<2:54:26,  6.11s/it]
 88%|████████▊ | 13014/14725 [2:02:32<2:48:36,  5.91s/it]
 88%|████████▊ | 13015/14725 [2:02:38<2:51:05,  6.00s/it]
 88%|████████▊ | 13016
0: {'loss': 0.5206, 'grad_norm': 0.3923654981349062, 'learning_rate': 3e-06, 'epoch': 0.88}
0: /14725 [2:02:44<2:50:31,  5.99s/it]
 88%|████████▊ | 13017/14725 [2:02:49<2:47:13,  5.87s/it]
 88%|████████▊ | 13018/14725 [2:02:55<2:45:26,  5.82s/it]
 88%|████████▊ | 13019/14725 [2:03:01<2:45:33,  5.82s/it]
 88%|████████▊ | 13020/14725 [2:03:07<2:45:35,  5.83s/it]
                                                         

 88%|████████▊ | 13020/14725 [2:03:07<2:45:35,  5.83s/it]
 88%|████████▊ | 13021/14725 [2:03:13<2:46:43,  5.87s/it]
 88%|████████▊ | 13022/14725 [2:03:18<2:45:28,  5.83s/it]
 88%|████████▊ | 13023/14725 [2:03:24<2:46:16,  5.86s/it]
 88%|████████▊ | 13024/14725 [2:03:30<2:48:35,  5.95s/it]
 88%|████████▊ | 13025/14725 [2:03:36<2:44:14,  5.80s/it]
 88%|████████▊ | 13026/14725 [2:03:41<2:38:56,  5.61s/it]
 88%|████████▊ | 13027/14725 [2:03:47<2:43:33,  5.78s/it]
 88%|████
0: {'loss': 0.5292, 'grad_norm': 0.38229344870861637, 'learning_rate': 3e-06, 'epoch': 0.88}
0: ████▊ | 13028/14725 [2:03:53<2:44:23,  5.81s/it]
 88%|████████▊ | 13029/14725 [2:03:59<2:45:21,  5.85s/it]
 88%|████████▊ | 13030/14725 [2:04:04<2:40:00,  5.66s/it]
                                                         

 88%|████████▊ | 13030/14725 [2:04:04<2:40:00,  5.66s/it]
 88%|████████▊ | 13031/14725 [2:04:10<2:43:15,  5.78s/it]
 89%|████████▊ | 13032/14725 [2:04:16<2:42:44,  5.77s/it]
 89%|████████▊ | 13033/14725 [2:04:22<2:44:07,  5.82s/it]
 89%|████████▊ | 13034/14725 [2:04:28<2:45:05,  5.86s/it]
 89%|████████▊ | 13035/14725 [2:04:34<2:46:15,  5.90s/it]
 89%|████████▊ | 13036/14725 [2:04:40<2:43:27,  5.81s/it]
 89%|████████▊ | 13037/14725 [2:04:45<2:38:39,  5.64s/it]
 89%|████████▊ | 13038/14725 [2:04:51<2:38:35,  5.64s/it]
 89%|████████▊ | 13039/14725 [2:04:56<2:37:59,  5.62
0: {'loss': 0.5151, 'grad_norm': 0.3857455432476398, 'learning_rate': 3e-06, 'epoch': 0.89}
0: s/it]
 89%|████████▊ | 13040/14725 [2:05:02<2:43:42,  5.83s/it]
                                                         

 89%|████████▊ | 13040/14725 [2:05:02<2:43:42,  5.83s/it]
 89%|████████▊ | 13041/14725 [2:05:08<2:41:46,  5.76s/it]
 89%|████████▊ | 13042/14725 [2:05:14<2:41:45,  5.77s/it]
 89%|████████▊ | 13043/14725 [2:05:19<2:39:35,  5.69s/it]
 89%|████████▊ | 13044/14725 [2:05:25<2:39:35,  5.70s/it]
 89%|████████▊ | 13045/14725 [2:05:30<2:36:40,  5.60s/it]
 89%|████████▊ | 13046/14725 [2:05:35<2:31:03,  5.40s/it]
 89%|████████▊ | 13047/14725 [2:05:41<2:34:26,  5.52s/it]
 89%|████████▊ | 13048/14725 [2:05:47<2:33:28,  5.49s/it]
 89%|████████▊ | 13049/14725 [2:05:52<2:36:42,  5.61s/it]
 89%|████████▊ | 13050/14725 [2:05:58<2:38:30,  5.68s/it]
                                               
0: {'loss': 0.5154, 'grad_norm': 0.3873415325579951, 'learning_rate': 3e-06, 'epoch': 0.89}
0: {'loss': 0.5249, 'grad_norm': 0.38295426970776375, 'learning_rate': 3e-06, 'epoch': 0.89}
0:           

 89%|████████▊ | 13050/14725 [2:05:58<2:38:30,  5.68s/it]
 89%|████████▊ | 13051/14725 [2:06:04<2:37:57,  5.66s/it]
 89%|████████▊ | 13052/14725 [2:06:10<2:38:21,  5.68s/it]
 89%|████████▊ | 13053/14725 [2:06:15<2:35:37,  5.58s/it]
 89%|████████▊ | 13054/14725 [2:06:20<2:34:07,  5.53s/it]
 89%|████████▊ | 13055/14725 [2:06:26<2:38:07,  5.68s/it]
 89%|████████▊ | 13056/14725 [2:06:32<2:35:07,  5.58s/it]
 89%|████████▊ | 13057/14725 [2:06:37<2:36:24,  5.63s/it]
 89%|████████▊ | 13058/14725 [2:06:44<2:42:47,  5.86s/it]
 89%|████████▊ | 13059/14725 [2:06:49<2:38:48,  5.72s/it]
 89%|████████▊ | 13060/14725 [2:06:55<2:38:28,  5.71s/it]
                                                         

 89%|████████▊ | 13060/14725 [2:06:55<2:38:28,  5.71s/it]
 89%|████████▊ | 13061/
0: {'loss': 0.533, 'grad_norm': 0.4217438018560299, 'learning_rate': 3e-06, 'epoch': 0.89}
0: 14725 [2:07:01<2:42:15,  5.85s/it]
 89%|████████▊ | 13062/14725 [2:07:07<2:42:26,  5.86s/it]
 89%|████████▊ | 13063/14725 [2:07:13<2:40:24,  5.79s/it]
 89%|████████▊ | 13064/14725 [2:07:18<2:36:19,  5.65s/it]
 89%|████████▊ | 13065/14725 [2:07:24<2:41:16,  5.83s/it]
 89%|████████▊ | 13066/14725 [2:07:30<2:41:43,  5.85s/it]
 89%|████████▊ | 13067/14725 [2:07:36<2:39:09,  5.76s/it]
 89%|████████▊ | 13068/14725 [2:07:41<2:37:20,  5.70s/it]
 89%|████████▉ | 13069/14725 [2:07:47<2:40:59,  5.83s/it]
 89%|████████▉ | 13070/14725 [2:07:53<2:38:03,  5.73s/it]
                                                         

 89%|████████▉ | 13070/14725 [2:07:53<2:38:03,  5.73s/it]
 89%|████████▉ | 13071/14725 [2:07:58<2:36:02,  5.66s/it]
 89%|████████▉ | 13072/14725 [2:08:05<2:40:04,  5.81s/it]
 89%|████�
0: {'loss': 0.5243, 'grad_norm': 0.42096373367255957, 'learning_rate': 3e-06, 'epoch': 0.89}
0: ��███▉ | 13073/14725 [2:08:11<2:42:26,  5.90s/it]
 89%|████████▉ | 13074/14725 [2:08:16<2:39:17,  5.79s/it]
 89%|████████▉ | 13075/14725 [2:08:22<2:39:34,  5.80s/it]
 89%|████████▉ | 13076/14725 [2:08:28<2:37:00,  5.71s/it]
 89%|████████▉ | 13077/14725 [2:08:33<2:36:55,  5.71s/it]
 89%|████████▉ | 13078/14725 [2:08:39<2:34:33,  5.63s/it]
 89%|████████▉ | 13079/14725 [2:08:45<2:38:59,  5.80s/it]
 89%|████████▉ | 13080/14725 [2:08:51<2:42:55,  5.94s/it]
                                                         

 89%|████████▉ | 13080/14725 [2:08:51<2:42:55,  5.94s/it]
 89%|████████▉ | 13081/14725 [2:08:57<2:40:56,  5.87s/it]
 89%|████████▉ | 13082/14725 [2:09:03<2:43:33,  5.97s/it]
 89%|████████▉ | 13083/14725 [2:09:09<2:44:09,  6.00s/it]
 89%|████████▉ | 13084/14725 [2:09:15<2:40:08,  5.86s
0: {'loss': 0.5346, 'grad_norm': 0.38071749139657857, 'learning_rate': 3e-06, 'epoch': 0.89}
0: /it]
 89%|████████▉ | 13085/14725 [2:09:20<2:38:31,  5.80s/it]
 89%|████████▉ | 13086/14725 [2:09:27<2:43:40,  5.99s/it]
 89%|████████▉ | 13087/14725 [2:09:33<2:42:46,  5.96s/it]
 89%|████████▉ | 13088/14725 [2:09:39<2:45:31,  6.07s/it]
 89%|████████▉ | 13089/14725 [2:09:45<2:46:08,  6.09s/it]
 89%|████████▉ | 13090/14725 [2:09:51<2:46:03,  6.09s/it]
                                                         

 89%|████████▉ | 13090/14725 [2:09:51<2:46:03,  6.09s/it]
 89%|████████▉ | 13091/14725 [2:09:56<2:38:14,  5.81s/it]
 89%|████████▉ | 13092/14725 [2:10:02<2:38:56,  5.84s/it]
 89%|████████▉ | 13093/14725 [2:10:08<2:36:47,  5.76s/it]
 89%|████████▉ | 13094/14725 [2:10:14<2:39:36,  5.87s/it]
 89%|████████▉ | 13095/14725 [2:10:20<2:41:24,  5.94s/it]
 89%|████████▉ | 13096/14725 [
0: {'loss': 0.526, 'grad_norm': 0.3899064612140416, 'learning_rate': 3e-06, 'epoch': 0.89}
0: 2:10:26<2:40:55,  5.93s/it]
 89%|████████▉ | 13097/14725 [2:10:31<2:36:32,  5.77s/it]
 89%|████████▉ | 13098/14725 [2:10:38<2:39:56,  5.90s/it]
 89%|████████▉ | 13099/14725 [2:10:43<2:39:49,  5.90s/it]
 89%|████████▉ | 13100/14725 [2:10:49<2:38:34,  5.85s/it]
                                                         

 89%|████████▉ | 13100/14725 [2:10:49<2:38:34,  5.85s/it]
 89%|████████▉ | 13101/14725 [2:10:55<2:36:25,  5.78s/it]
 89%|████████▉ | 13102/14725 [2:11:01<2:38:51,  5.87s/it]
 89%|████████▉ | 13103/14725 [2:11:07<2:37:38,  5.83s/it]
 89%|████████▉ | 13104/14725 [2:11:12<2:37:39,  5.84s/it]
 89%|████████▉ | 13105/14725 [2:11:18<2:38:23,  5.87s/it]
 89%|████████▉ | 13106/14725 [2:11:24<2:37:31,  5.84s/it]
 89%|████████▉ | 13107/14725 [2:11:30<2:40:27,  5.95s/it]
 89%|██████�
0: {'loss': 0.528, 'grad_norm': 0.3607003600850719, 'learning_rate': 3e-06, 'epoch': 0.89}
0: �█▉ | 13108/14725 [2:11:37<2:42:01,  6.01s/it]
 89%|████████▉ | 13109/14725 [2:11:43<2:43:15,  6.06s/it]
 89%|████████▉ | 13110/14725 [2:11:48<2:40:17,  5.96s/it]
                                                         

 89%|████████▉ | 13110/14725 [2:11:48<2:40:17,  5.96s/it]
 89%|████████▉ | 13111/14725 [2:11:54<2:38:31,  5.89s/it]
 89%|████████▉ | 13112/14725 [2:12:00<2:39:27,  5.93s/it]
 89%|████████▉ | 13113/14725 [2:12:06<2:40:13,  5.96s/it]
 89%|████████▉ | 13114/14725 [2:12:13<2:43:15,  6.08s/it]
 89%|████████▉ | 13115/14725 [2:12:19<2:44:56,  6.15s/it]
 89%|████████▉ | 13116/14725 [2:12:25<2:46:14,  6.20s/it]
 89%|████████▉ | 13117/14725 [2:12:31<2:43:24,  6.10s/it]
 89%|████████▉ | 13118/14725 [2:12:37<2:39:59,  5.97s/it]
 89%|████████▉ | 13119/14725 [2:12:43<2:42:45,  6.08s/it]
 8
0: {'loss': 0.5314, 'grad_norm': 0.37511939057048027, 'learning_rate': 3e-06, 'epoch': 0.89}
0: 9%|████████▉ | 13120/14725 [2:12:49<2:39:04,  5.95s/it]
                                                         

 89%|████████▉ | 13120/14725 [2:12:49<2:39:04,  5.95s/it]
 89%|████████▉ | 13121/14725 [2:12:54<2:37:30,  5.89s/it]
 89%|████████▉ | 13122/14725 [2:13:00<2:36:20,  5.85s/it]
 89%|████████▉ | 13123/14725 [2:13:06<2:32:18,  5.70s/it]
 89%|████████▉ | 13124/14725 [2:13:11<2:29:08,  5.59s/it]
 89%|████████▉ | 13125/14725 [2:13:17<2:31:38,  5.69s/it]
 89%|████████▉ | 13126/14725 [2:13:22<2:30:33,  5.65s/it]
 89%|████████▉ | 13127/14725 [2:13:28<2:32:02,  5.71s/it]
 89%|████████▉ | 13128/14725 [2:13:34<2:35:58,  5.86s/it]
 89%|████████▉ | 13129/14725 [2:13:40<2:36:12,  5.87s/it]
 89%|████████▉ | 13130/14725 [2:13:46<2:36:15,  5.88s/it]
                                                       
0: {'loss': 0.5181, 'grad_norm': 0.3979667476103468, 'learning_rate': 3e-06, 'epoch': 0.89}
0: {'loss': 0.5337, 'grad_norm': 0.37585504214909343, 'learning_rate': 3e-06, 'epoch': 0.89}
0:   

 89%|████████▉ | 13130/14725 [2:13:46<2:36:15,  5.88s/it]
 89%|████████▉ | 13131/14725 [2:13:53<2:39:06,  5.99s/it]
 89%|████████▉ | 13132/14725 [2:13:59<2:41:42,  6.09s/it]
 89%|████████▉ | 13133/14725 [2:14:04<2:36:19,  5.89s/it]
 89%|████████▉ | 13134/14725 [2:14:11<2:39:08,  6.00s/it]
 89%|████████▉ | 13135/14725 [2:14:17<2:40:56,  6.07s/it]
 89%|████████▉ | 13136/14725 [2:14:22<2:37:12,  5.94s/it]
 89%|████████▉ | 13137/14725 [2:14:29<2:40:40,  6.07s/it]
 89%|████████▉ | 13138/14725 [2:14:34<2:33:36,  5.81s/it]
 89%|████████▉ | 13139/14725 [2:14:40<2:35:28,  5.88s/it]
 89%|████████▉ | 13140/14725 [2:14:46<2:35:58,  5.90s/it]
                                                         

 89%|████████▉ | 13140/14725 [2:14:46<2:35:58,  5.90s/it]
 89%|████████▉ | 13141/14725 [2
0: {'loss': 0.5167, 'grad_norm': 0.3878513887530698, 'learning_rate': 3e-06, 'epoch': 0.89}
0: :14:52<2:33:31,  5.82s/it]
 89%|████████▉ | 13142/14725 [2:14:57<2:31:16,  5.73s/it]
 89%|████████▉ | 13143/14725 [2:15:03<2:29:48,  5.68s/it]
 89%|████████▉ | 13144/14725 [2:15:09<2:30:57,  5.73s/it]
 89%|████████▉ | 13145/14725 [2:15:14<2:30:53,  5.73s/it]
 89%|████████▉ | 13146/14725 [2:15:20<2:31:17,  5.75s/it]
 89%|████████▉ | 13147/14725 [2:15:25<2:28:21,  5.64s/it]
 89%|████████▉ | 13148/14725 [2:15:32<2:31:38,  5.77s/it]
 89%|████████▉ | 13149/14725 [2:15:38<2:35:45,  5.93s/it]
 89%|████████▉ | 13150/14725 [2:15:44<2:35:40,  5.93s/it]
                                                         

 89%|████████▉ | 13150/14725 [2:15:44<2:35:40,  5.93s/it]
 89%|████████▉ | 13151/14725 [2:15:49<2:30:50,  5.75s/it]
 89%|████████▉ | 13152/14725 [2:15:55<2:31:47,  5.79s/it]
 89%|███████
0: {'loss': 0.5257, 'grad_norm': 0.4692396780474859, 'learning_rate': 3e-06, 'epoch': 0.89}
0: █▉ | 13153/14725 [2:16:01<2:35:22,  5.93s/it]
 89%|████████▉ | 13154/14725 [2:16:07<2:34:04,  5.88s/it]
 89%|████████▉ | 13155/14725 [2:16:13<2:31:41,  5.80s/it]
 89%|████████▉ | 13156/14725 [2:16:18<2:27:41,  5.65s/it]
 89%|████████▉ | 13157/14725 [2:16:23<2:24:57,  5.55s/it]
 89%|████████▉ | 13158/14725 [2:16:29<2:24:34,  5.54s/it]
 89%|████████▉ | 13159/14725 [2:16:35<2:30:38,  5.77s/it]
 89%|████████▉ | 13160/14725 [2:16:41<2:29:59,  5.75s/it]
                                                         

 89%|████████▉ | 13160/14725 [2:16:41<2:29:59,  5.75s/it]
 89%|████████▉ | 13161/14725 [2:16:47<2:32:39,  5.86s/it]
 89%|████████▉ | 13162/14725 [2:16:52<2:30:58,  5.80s/it]
 89%|████████▉ | 13163/14725 [2:16:58<2:27:59,  5.68s/it]
 89%|████████▉ | 13164/14725 [2:17:04<2:29:55,  5.76s/it]
 89
0: {'loss': 0.5344, 'grad_norm': 0.3923851727407977, 'learning_rate': 3e-06, 'epoch': 0.89}
0: %|████████▉ | 13165/14725 [2:17:10<2:29:13,  5.74s/it]
 89%|████████▉ | 13166/14725 [2:17:16<2:33:42,  5.92s/it]
 89%|████████▉ | 13167/14725 [2:17:21<2:31:22,  5.83s/it]
 89%|████████▉ | 13168/14725 [2:17:27<2:30:41,  5.81s/it]
 89%|████████▉ | 13169/14725 [2:17:33<2:27:13,  5.68s/it]
 89%|████████▉ | 13170/14725 [2:17:39<2:31:29,  5.85s/it]
                                                         

 89%|████████▉ | 13170/14725 [2:17:39<2:31:29,  5.85s/it]
 89%|████████▉ | 13171/14725 [2:17:45<2:30:49,  5.82s/it]
 89%|████████▉ | 13172/14725 [2:17:50<2:27:09,  5.69s/it]
 89%|████████▉ | 13173/14725 [2:17:56<2:29:55,  5.80s/it]
 89%|████████▉ | 13174/14725 [2:18:02<2:29:32,  5.79s/it]
 89%|████████▉ | 13175/14725 [2:18:08<2:33:53,  5.96s/it]
 89%|████████▉ | 13176/14725 [2:18:15<
0: {'loss': 0.5292, 'grad_norm': 0.3719869944454638, 'learning_rate': 3e-06, 'epoch': 0.9}
0: 2:37:19,  6.09s/it]
 89%|████████▉ | 13177/14725 [2:18:21<2:38:28,  6.14s/it]
 89%|████████▉ | 13178/14725 [2:18:26<2:31:26,  5.87s/it]
 90%|████████▉ | 13179/14725 [2:18:32<2:33:43,  5.97s/it]
 90%|████████▉ | 13180/14725 [2:18:39<2:35:52,  6.05s/it]
                                                         

 90%|████████▉ | 13180/14725 [2:18:39<2:35:52,  6.05s/it]
 90%|████████▉ | 13181/14725 [2:18:45<2:37:42,  6.13s/it]
 90%|████████▉ | 13182/14725 [2:18:51<2:37:12,  6.11s/it]
 90%|████████▉ | 13183/14725 [2:18:57<2:34:40,  6.02s/it]
 90%|████████▉ | 13184/14725 [2:19:02<2:32:26,  5.94s/it]
 90%|████████▉ | 13185/14725 [2:19:08<2:29:09,  5.81s/it]
 90%|████████▉ | 13186/14725 [2:19:14<2:29:20,  5.82s/it]
 90%|████████▉ | 13187/14725 [2:19:19<2:26:33,  5.72s/it]
 90%|████████▉ 
0: {'loss': 0.5229, 'grad_norm': 0.38653900045237716, 'learning_rate': 3e-06, 'epoch': 0.9}
0: | 13188/14725 [2:19:25<2:27:15,  5.75s/it]
 90%|████████▉ | 13189/14725 [2:19:31<2:29:56,  5.86s/it]
 90%|████████▉ | 13190/14725 [2:19:37<2:29:41,  5.85s/it]
                                                         

 90%|████████▉ | 13190/14725 [2:19:37<2:29:41,  5.85s/it]
 90%|████████▉ | 13191/14725 [2:19:43<2:29:45,  5.86s/it]
 90%|████████▉ | 13192/14725 [2:19:49<2:32:12,  5.96s/it]
 90%|████████▉ | 13193/14725 [2:19:55<2:28:30,  5.82s/it]
 90%|████████▉ | 13194/14725 [2:20:00<2:23:48,  5.64s/it]
 90%|████████▉ | 13195/14725 [2:20:06<2:27:32,  5.79s/it]
 90%|████████▉ | 13196/14725 [2:20:11<2:23:07,  5.62s/it]
 90%|████████▉ | 13197/14725 [2:20:17<2:23:10,  5.62s/it]
 90%|████████▉ | 13198/14725 [2:20:23<2:27:57,  5.81s/it]
 90%|████████▉ | 13199/14725 [2:20:29<2:26:29,  5.76s/it]
 90%|█�
0: {'loss': 0.5301, 'grad_norm': 0.3927726463458516, 'learning_rate': 3e-06, 'epoch': 0.9}
0: {'loss': 0.5174, 'grad_norm': 0.38537373530778385, 'learning_rate': 3e-06, 'epoch': 0.9}
0: �██████▉ | 13200/14725 [2:20:34<2:25:00,  5.71s/it]
                                                         

 90%|████████▉ | 13200/14725 [2:20:34<2:25:00,  5.71s/it]
 90%|████████▉ | 13201/14725 [2:20:40<2:25:18,  5.72s/it]
 90%|████████▉ | 13202/14725 [2:20:46<2:30:31,  5.93s/it]
 90%|████████▉ | 13203/14725 [2:20:52<2:29:19,  5.89s/it]
 90%|████████▉ | 13204/14725 [2:20:58<2:25:47,  5.75s/it]
 90%|████████▉ | 13205/14725 [2:21:04<2:27:37,  5.83s/it]
 90%|████████▉ | 13206/14725 [2:21:10<2:27:56,  5.84s/it]
 90%|████████▉ | 13207/14725 [2:21:15<2:27:08,  5.82s/it]
 90%|████████▉ | 13208/14725 [2:21:22<2:29:53,  5.93s/it]
 90%|████████▉ | 13209/14725 [2:21:27<2:29:15,  5.91s/it]
 90%|████████▉ | 13210/14725 [2:21:33<2:30:14,  5.95s/it]
                                                         

 90%
0: {'loss': 0.5169, 'grad_norm': 0.3867847444701874, 'learning_rate': 3e-06, 'epoch': 0.9}
0: |████████▉ | 13210/14725 [2:21:33<2:30:14,  5.95s/it]
 90%|████████▉ | 13211/14725 [2:21:39<2:28:07,  5.87s/it]
 90%|████████▉ | 13212/14725 [2:21:44<2:24:20,  5.72s/it]
 90%|████████▉ | 13213/14725 [2:21:50<2:23:06,  5.68s/it]
 90%|████████▉ | 13214/14725 [2:21:56<2:22:57,  5.68s/it]
 90%|████████▉ | 13215/14725 [2:22:02<2:24:40,  5.75s/it]
 90%|████████▉ | 13216/14725 [2:22:07<2:23:38,  5.71s/it]
 90%|████████▉ | 13217/14725 [2:22:14<2:29:03,  5.93s/it]
 90%|████████▉ | 13218/14725 [2:22:19<2:25:54,  5.81s/it]
 90%|████████▉ | 13219/14725 [2:22:25<2:24:58,  5.78s/it]
 90%|████████▉ | 13220/14725 [2:22:30<2:22:04,  5.66s/it]
                                                         

 90%|████████▉ | 13220/14725 [2:22:30<2:22:04,  5.66s/it]
 90%|████████▉ | 13221/14725 [2:22:36<2
0: {'loss': 0.5226, 'grad_norm': 0.3978383436919936, 'learning_rate': 3e-06, 'epoch': 0.9}
0: :22:26,  5.68s/it]
 90%|████████▉ | 13222/14725 [2:22:41<2:19:33,  5.57s/it]
 90%|████████▉ | 13223/14725 [2:22:47<2:18:18,  5.53s/it]
 90%|████████▉ | 13224/14725 [2:22:52<2:19:18,  5.57s/it]
 90%|████████▉ | 13225/14725 [2:22:58<2:20:41,  5.63s/it]
 90%|████████▉ | 13226/14725 [2:23:04<2:20:34,  5.63s/it]
 90%|████████▉ | 13227/14725 [2:23:09<2:19:30,  5.59s/it]
 90%|████████▉ | 13228/14725 [2:23:15<2:19:26,  5.59s/it]
 90%|████████▉ | 13229/14725 [2:23:20<2:18:04,  5.54s/it]
 90%|████████▉ | 13230/14725 [2:23:26<2:20:30,  5.64s/it]
                                                         

 90%|████████▉ | 13230/14725 [2:23:26<2:20:30,  5.64s/it]
 90%|████████▉ | 13231/14725 [2:23:32<2:21:51,  5.70s/it]
 90%|████████▉ | 13232/14725 [2:23:37<2:18:55,  5.58s/it]
 90%|████████▉ |
0: {'loss': 0.5158, 'grad_norm': 0.41574576388009876, 'learning_rate': 3e-06, 'epoch': 0.9}
0:  13233/14725 [2:23:43<2:16:45,  5.50s/it]
 90%|████████▉ | 13234/14725 [2:23:49<2:19:01,  5.59s/it]
 90%|████████▉ | 13235/14725 [2:23:54<2:19:23,  5.61s/it]
 90%|████████▉ | 13236/14725 [2:24:00<2:22:53,  5.76s/it]
 90%|████████▉ | 13237/14725 [2:24:06<2:23:36,  5.79s/it]
 90%|████████▉ | 13238/14725 [2:24:12<2:22:02,  5.73s/it]
 90%|████████▉ | 13239/14725 [2:24:18<2:25:09,  5.86s/it]
 90%|████████▉ | 13240/14725 [2:24:24<2:29:18,  6.03s/it]
                                                         

 90%|████████▉ | 13240/14725 [2:24:24<2:29:18,  6.03s/it]
 90%|████████▉ | 13241/14725 [2:24:30<2:30:18,  6.08s/it]
 90%|████████▉ | 13242/14725 [2:24:36<2:29:27,  6.05s/it]
 90%|████████▉ | 13243/14725 [2:24:43<2:29:51,  6.07s/it]
 90%|████████▉ | 13244/14725 [2:24:49<2:29:22,  6.05s/it]
 90%|██
0: {'loss': 0.5424, 'grad_norm': 0.39042562782463247, 'learning_rate': 3e-06, 'epoch': 0.9}
0: ██████▉ | 13245/14725 [2:24:55<2:28:27,  6.02s/it]
 90%|████████▉ | 13246/14725 [2:25:00<2:26:52,  5.96s/it]
 90%|████████▉ | 13247/14725 [2:25:06<2:22:18,  5.78s/it]
 90%|████████▉ | 13248/14725 [2:25:12<2:25:26,  5.91s/it]
 90%|████████▉ | 13249/14725 [2:25:18<2:26:52,  5.97s/it]
 90%|████████▉ | 13250/14725 [2:25:24<2:30:03,  6.10s/it]
                                                         

 90%|████████▉ | 13250/14725 [2:25:24<2:30:03,  6.10s/it]
 90%|████████▉ | 13251/14725 [2:25:30<2:28:49,  6.06s/it]
 90%|████████▉ | 13252/14725 [2:25:36<2:23:18,  5.84s/it]
 90%|█████████ | 13253/14725 [2:25:42<2:25:37,  5.94s/it]
 90%|█████████ | 13254/14725 [2:25:48<2:23:21,  5.85s/it]
 90%|█████████ | 13255/14725 [2:25:53<2:19:43,  5.70s/it]
 90%|█████████ | 13256/14725 [2:25:59<2:22:09,
0: {'loss': 0.5272, 'grad_norm': 0.3873032660503769, 'learning_rate': 3e-06, 'epoch': 0.9}
0:   5.81s/it]
 90%|█████████ | 13257/14725 [2:26:05<2:21:03,  5.77s/it]
 90%|█████████ | 13258/14725 [2:26:10<2:19:29,  5.71s/it]
 90%|█████████ | 13259/14725 [2:26:16<2:19:59,  5.73s/it]
 90%|█████████ | 13260/14725 [2:26:22<2:22:27,  5.83s/it]
                                                         

 90%|█████████ | 13260/14725 [2:26:22<2:22:27,  5.83s/it]
 90%|█████████ | 13261/14725 [2:26:28<2:22:40,  5.85s/it]
 90%|█████████ | 13262/14725 [2:26:34<2:21:46,  5.81s/it]
 90%|█████████ | 13263/14725 [2:26:39<2:18:05,  5.67s/it]
 90%|█████████ | 13264/14725 [2:26:44<2:15:50,  5.58s/it]
 90%|█████████ | 13265/14725 [2:26:50<2:13:47,  5.50s/it]
 90%|█████████ | 13266/14725 [2:26:55<2:14:18,  5.52s/it]
 90%|█████████ | 13267/14725 [2:27:01<2:17:58,  5.68s/it]
 90%|█████████ | 13268/
0: {'loss': 0.5324, 'grad_norm': 0.39066767403789426, 'learning_rate': 3e-06, 'epoch': 0.9}
0: 14725 [2:27:07<2:19:40,  5.75s/it]
 90%|█████████ | 13269/14725 [2:27:14<2:24:48,  5.97s/it]
 90%|█████████ | 13270/14725 [2:27:19<2:21:18,  5.83s/it]
                                                         

 90%|█████████ | 13270/14725 [2:27:19<2:21:18,  5.83s/it]
 90%|█████████ | 13271/14725 [2:27:24<2:17:09,  5.66s/it]
 90%|█████████ | 13272/14725 [2:27:30<2:16:45,  5.65s/it]
 90%|█████████ | 13273/14725 [2:27:36<2:20:47,  5.82s/it]
 90%|█████████ | 13274/14725 [2:27:42<2:18:23,  5.72s/it]
 90%|█████████ | 13275/14725 [2:27:47<2:16:38,  5.65s/it]
 90%|█████████ | 13276/14725 [2:27:53<2:13:51,  5.54s/it]
 90%|█████████ | 13277/14725 [2:27:58<2:14:48,  5.59s/it]
 90%|█████████ | 13278/14725 [2:28:04<2:14:36,  5.58s/it]
 90%|█████████ | 13279/14725 [2:28:10<2:18:41,  5.75s/it]
 90%|████�
0: {'loss': 0.5277, 'grad_norm': 0.3853517952970942, 'learning_rate': 3e-06, 'epoch': 0.9}
0: {'loss': 0.519, 'grad_norm': 0.39544978324100594, 'learning_rate': 3e-06, 'epoch': 0.9}
0: ��████ | 13280/14725 [2:28:15<2:15:22,  5.62s/it]
                                                         

 90%|█████████ | 13280/14725 [2:28:15<2:15:22,  5.62s/it]
 90%|█████████ | 13281/14725 [2:28:21<2:12:57,  5.52s/it]
 90%|█████████ | 13282/14725 [2:28:26<2:13:57,  5.57s/it]
 90%|█████████ | 13283/14725 [2:28:32<2:14:05,  5.58s/it]
 90%|█████████ | 13284/14725 [2:28:38<2:19:21,  5.80s/it]
 90%|█████████ | 13285/14725 [2:28:44<2:22:34,  5.94s/it]
 90%|█████████ | 13286/14725 [2:28:50<2:20:47,  5.87s/it]
 90%|█████████ | 13287/14725 [2:28:57<2:24:12,  6.02s/it]
 90%|█████████ | 13288/14725 [2:29:02<2:20:58,  5.89s/it]
 90%|█████████ | 13289/14725 [2:29:09<2:25:53,  6.10s/it]
 90%|█████████ | 13290/14725 [2:29:15<2:25:28,  6.08s/it]
                                                         

 90%|██�
0: {'loss': 0.5356, 'grad_norm': 0.37933821298835324, 'learning_rate': 3e-06, 'epoch': 0.9}
0: ��██████ | 13290/14725 [2:29:15<2:25:28,  6.08s/it]
 90%|█████████ | 13291/14725 [2:29:21<2:25:28,  6.09s/it]
 90%|█████████ | 13292/14725 [2:29:27<2:26:45,  6.14s/it]
 90%|█████████ | 13293/14725 [2:29:33<2:23:32,  6.01s/it]
 90%|█████████ | 13294/14725 [2:29:38<2:20:04,  5.87s/it]
 90%|█████████ | 13295/14725 [2:29:45<2:22:44,  5.99s/it]
 90%|█████████ | 13296/14725 [2:29:50<2:21:21,  5.94s/it]
 90%|█████████ | 13297/14725 [2:29:57<2:24:27,  6.07s/it]
 90%|█████████ | 13298/14725 [2:30:03<2:23:11,  6.02s/it]
 90%|█████████ | 13299/14725 [2:30:08<2:18:46,  5.84s/it]
 90%|█████████ | 13300/14725 [2:30:14<2:15:48,  5.72s/it]
                                                         

 90%|█████████ | 13300/14725 [2:30:14<2:15:48,  5.72s/it]
 90%|█████████ | 13301/14725 [2:30:19<2:16:29, 
0: {'loss': 0.5256, 'grad_norm': 0.373268029337924, 'learning_rate': 3e-06, 'epoch': 0.9}
0:  5.75s/it]
 90%|█████████ | 13302/14725 [2:30:25<2:18:24,  5.84s/it]
 90%|█████████ | 13303/14725 [2:30:31<2:16:53,  5.78s/it]
 90%|█████████ | 13304/14725 [2:30:37<2:20:53,  5.95s/it]
 90%|█████████ | 13305/14725 [2:30:43<2:17:45,  5.82s/it]
 90%|█████████ | 13306/14725 [2:30:48<2:15:25,  5.73s/it]
 90%|█████████ | 13307/14725 [2:30:54<2:16:14,  5.76s/it]
 90%|█████████ | 13308/14725 [2:31:00<2:14:51,  5.71s/it]
 90%|█████████ | 13309/14725 [2:31:06<2:16:54,  5.80s/it]
 90%|█████████ | 13310/14725 [2:31:12<2:16:00,  5.77s/it]
                                                         

 90%|█████████ | 13310/14725 [2:31:12<2:16:00,  5.77s/it]
 90%|█████████ | 13311/14725 [2:31:17<2:14:07,  5.69s/it]
 90%|█████████ | 13312/14725 [2:31:23<2:18:01,  5.86s/it]
 90%|█████████ | 13313/1
0: {'loss': 0.5305, 'grad_norm': 0.3865748368545771, 'learning_rate': 3e-06, 'epoch': 0.9}
0: 4725 [2:31:30<2:21:43,  6.02s/it]
 90%|█████████ | 13314/14725 [2:31:36<2:19:38,  5.94s/it]
 90%|█████████ | 13315/14725 [2:31:42<2:21:11,  6.01s/it]
 90%|█████████ | 13316/14725 [2:31:47<2:17:55,  5.87s/it]
 90%|█████████ | 13317/14725 [2:31:53<2:18:51,  5.92s/it]
 90%|█████████ | 13318/14725 [2:31:59<2:16:30,  5.82s/it]
 90%|█████████ | 13319/14725 [2:32:04<2:14:46,  5.75s/it]
 90%|█████████ | 13320/14725 [2:32:10<2:11:55,  5.63s/it]
                                                         

 90%|█████████ | 13320/14725 [2:32:10<2:11:55,  5.63s/it]
 90%|█████████ | 13321/14725 [2:32:16<2:16:43,  5.84s/it]
 90%|█████████ | 13322/14725 [2:32:22<2:15:48,  5.81s/it]
 90%|█████████ | 13323/14725 [2:32:28<2:15:09,  5.78s/it]
 90%|█████████ | 13324/14725 [2:32:33<2:12:25,  5.67s/it]
 90%|████�
0: {'loss': 0.5179, 'grad_norm': 0.3649777419517571, 'learning_rate': 3e-06, 'epoch': 0.91}
0: �████ | 13325/14725 [2:32:39<2:14:07,  5.75s/it]
 90%|█████████ | 13326/14725 [2:32:45<2:18:08,  5.92s/it]
 91%|█████████ | 13327/14725 [2:32:51<2:17:46,  5.91s/it]
 91%|█████████ | 13328/14725 [2:32:58<2:21:43,  6.09s/it]
 91%|█████████ | 13329/14725 [2:33:03<2:18:34,  5.96s/it]
 91%|█████████ | 13330/14725 [2:33:08<2:12:33,  5.70s/it]
                                                         

 91%|█████████ | 13330/14725 [2:33:08<2:12:33,  5.70s/it]
 91%|█████████ | 13331/14725 [2:33:14<2:14:33,  5.79s/it]
 91%|█████████ | 13332/14725 [2:33:20<2:14:31,  5.79s/it]
 91%|█████████ | 13333/14725 [2:33:27<2:19:16,  6.00s/it]
 91%|█████████ | 13334/14725 [2:33:32<2:16:28,  5.89s/it]
 91%|█████████ | 13335/14725 [2:33:38<2:16:47,  5.90s/it]
 91%|█████████ | 13336/14725 [2:33:44<2:17:00,  5.92s/
0: {'loss': 0.5161, 'grad_norm': 0.3841161034394869, 'learning_rate': 3e-06, 'epoch': 0.91}
0: it]
 91%|█████████ | 13337/14725 [2:33:50<2:16:29,  5.90s/it]
 91%|█████████ | 13338/14725 [2:33:55<2:10:13,  5.63s/it]
 91%|█████████ | 13339/14725 [2:34:01<2:11:36,  5.70s/it]
 91%|█████████ | 13340/14725 [2:34:07<2:10:45,  5.66s/it]
                                                         

 91%|█████████ | 13340/14725 [2:34:07<2:10:45,  5.66s/it]
 91%|█████████ | 13341/14725 [2:34:12<2:10:04,  5.64s/it]
 91%|█████████ | 13342/14725 [2:34:18<2:08:59,  5.60s/it]
 91%|█████████ | 13343/14725 [2:34:23<2:08:33,  5.58s/it]
 91%|█████████ | 13344/14725 [2:34:29<2:09:43,  5.64s/it]
 91%|█████████ | 13345/14725 [2:34:35<2:12:30,  5.76s/it]
 91%|█████████ | 13346/14725 [2:34:40<2:07:20,  5.54s/it]
 91%|█████████ | 13347/14725 [2:34:45<2:05:57,  5.48s/it]
 91%|█████████ | 13348/14725 [2
0: {'loss': 0.5165, 'grad_norm': 0.3764738763571604, 'learning_rate': 3e-06, 'epoch': 0.91}
0: :34:51<2:04:21,  5.42s/it]
 91%|█████████ | 13349/14725 [2:34:57<2:09:07,  5.63s/it]
 91%|█████████ | 13350/14725 [2:35:02<2:07:53,  5.58s/it]
                                                         

 91%|█████████ | 13350/14725 [2:35:02<2:07:53,  5.58s/it]
 91%|█████████ | 13351/14725 [2:35:08<2:11:53,  5.76s/it]
 91%|█████████ | 13352/14725 [2:35:14<2:12:31,  5.79s/it]
 91%|█████████ | 13353/14725 [2:35:20<2:09:29,  5.66s/it]
 91%|█████████ | 13354/14725 [2:35:26<2:11:13,  5.74s/it]
 91%|█████████ | 13355/14725 [2:35:31<2:11:17,  5.75s/it]
 91%|█████████ | 13356/14725 [2:35:37<2:13:29,  5.85s/it]
 91%|█████████ | 13357/14725 [2:35:43<2:12:28,  5.81s/it]
 91%|█████████ | 13358/14725 [2:35:49<2:11:56,  5.79s/it]
 91%|█████████ | 13359/14725 [2:35:54<2:10:38,  5.74s/it]
 91%|███████
0: {'loss': 0.5198, 'grad_norm': 0.37637731896633325, 'learning_rate': 3e-06, 'epoch': 0.91}
0: {'loss': 0.5342, 'grad_norm': 0.38021781492202744, 'learning_rate': 3e-06, 'epoch': 0.91}
0: ██ | 13360/14725 [2:36:00<2:08:08,  5.63s/it]
                                                         

 91%|█████████ | 13360/14725 [2:36:00<2:08:08,  5.63s/it]
 91%|█████████ | 13361/14725 [2:36:06<2:09:14,  5.69s/it]
 91%|█████████ | 13362/14725 [2:36:12<2:11:10,  5.77s/it]
 91%|█████████ | 13363/14725 [2:36:17<2:08:56,  5.68s/it]
 91%|█████████ | 13364/14725 [2:36:23<2:07:37,  5.63s/it]
 91%|█████████ | 13365/14725 [2:36:28<2:09:28,  5.71s/it]
 91%|█████████ | 13366/14725 [2:36:34<2:10:04,  5.74s/it]
 91%|█████████ | 13367/14725 [2:36:40<2:09:40,  5.73s/it]
 91%|█████████ | 13368/14725 [2:36:45<2:06:28,  5.59s/it]
 91%|█████████ | 13369/14725 [2:36:51<2:08:53,  5.70s/it]
 91%|█████████ | 13370/14725 [2:36:57<2:10:32,  5.78s/it]
                                                         

 91%|█████
0: {'loss': 0.5307, 'grad_norm': 0.4064143281630885, 'learning_rate': 3e-06, 'epoch': 0.91}
0: ████ | 13370/14725 [2:36:57<2:10:32,  5.78s/it]
 91%|█████████ | 13371/14725 [2:37:04<2:14:20,  5.95s/it]
 91%|█████████ | 13372/14725 [2:37:10<2:14:41,  5.97s/it]
 91%|█████████ | 13373/14725 [2:37:15<2:13:28,  5.92s/it]
 91%|█████████ | 13374/14725 [2:37:22<2:16:00,  6.04s/it]
 91%|█████████ | 13375/14725 [2:37:27<2:09:24,  5.75s/it]
 91%|█████████ | 13376/14725 [2:37:32<2:08:15,  5.70s/it]
 91%|█████████ | 13377/14725 [2:37:39<2:11:46,  5.87s/it]
 91%|█████████ | 13378/14725 [2:37:44<2:07:18,  5.67s/it]
 91%|█████████ | 13379/14725 [2:37:50<2:08:30,  5.73s/it]
 91%|█████████ | 13380/14725 [2:37:56<2:10:33,  5.82s/it]
                                                         

 91%|█████████ | 13380/14725 [2:37:56<2:10:33,  5.82s/it]
 91%|█████████ | 13381/14725 [2:38:01<2:05:57,  5.62s/i
0: {'loss': 0.5283, 'grad_norm': 0.37639043632480873, 'learning_rate': 3e-06, 'epoch': 0.91}
0: t]
 91%|█████████ | 13382/14725 [2:38:06<2:03:18,  5.51s/it]
 91%|█████████ | 13383/14725 [2:38:12<2:07:53,  5.72s/it]
 91%|█████████ | 13384/14725 [2:38:18<2:08:30,  5.75s/it]
 91%|█████████ | 13385/14725 [2:38:24<2:08:02,  5.73s/it]
 91%|█████████ | 13386/14725 [2:38:29<2:06:10,  5.65s/it]
 91%|█████████ | 13387/14725 [2:38:35<2:05:57,  5.65s/it]
 91%|█████████ | 13388/14725 [2:38:40<2:03:30,  5.54s/it]
 91%|█████████ | 13389/14725 [2:38:46<2:02:43,  5.51s/it]
 91%|█████████ | 13390/14725 [2:38:51<2:02:17,  5.50s/it]
                                                         

 91%|█████████ | 13390/14725 [2:38:51<2:02:17,  5.50s/it]
 91%|█████████ | 13391/14725 [2:38:57<2:02:24,  5.51s/it]
 91%|█████████ | 13392/14725 [2:39:02<2:03:07,  5.54s/it]
 91%|█████████ | 13393/14725 [2:
0: {'loss': 0.5221, 'grad_norm': 0.36354800167151274, 'learning_rate': 3e-06, 'epoch': 0.91}
0: 39:09<2:07:36,  5.75s/it]
 91%|█████████ | 13394/14725 [2:39:14<2:06:25,  5.70s/it]
 91%|█████████ | 13395/14725 [2:39:20<2:05:25,  5.66s/it]
 91%|█████████ | 13396/14725 [2:39:25<2:05:32,  5.67s/it]
 91%|█████████ | 13397/14725 [2:39:31<2:05:00,  5.65s/it]
 91%|█████████ | 13398/14725 [2:39:37<2:09:30,  5.86s/it]
 91%|█████████ | 13399/14725 [2:39:43<2:06:21,  5.72s/it]
 91%|█████████ | 13400/14725 [2:39:49<2:09:08,  5.85s/it]
                                                         

 91%|█████████ | 13400/14725 [2:39:49<2:09:08,  5.85s/it]
 91%|█████████ | 13401/14725 [2:39:54<2:07:17,  5.77s/it]
 91%|█████████ | 13402/14725 [2:40:00<2:07:24,  5.78s/it]
 91%|█████████ | 13403/14725 [2:40:06<2:07:43,  5.80s/it]
 91%|█████████ | 13404/14725 [2:40:12<2:07:29,  5.79s/it]
 91%|███████�
0: {'loss': 0.5311, 'grad_norm': 0.3815755325953391, 'learning_rate': 3e-06, 'epoch': 0.91}
0: ��█ | 13405/14725 [2:40:18<2:07:58,  5.82s/it]
 91%|█████████ | 13406/14725 [2:40:24<2:11:28,  5.98s/it]
 91%|█████████ | 13407/14725 [2:40:30<2:07:35,  5.81s/it]
 91%|█████████ | 13408/14725 [2:40:36<2:11:08,  5.97s/it]
 91%|█████████ | 13409/14725 [2:40:41<2:05:43,  5.73s/it]
 91%|█████████ | 13410/14725 [2:40:47<2:04:58,  5.70s/it]
                                                         

 91%|█████████ | 13410/14725 [2:40:47<2:04:58,  5.70s/it]
 91%|█████████ | 13411/14725 [2:40:52<2:03:11,  5.63s/it]
 91%|█████████ | 13412/14725 [2:40:58<2:05:43,  5.75s/it]
 91%|█████████ | 13413/14725 [2:41:04<2:08:46,  5.89s/it]
 91%|█████████ | 13414/14725 [2:41:10<2:06:56,  5.81s/it]
 91%|█████████ | 13415/14725 [2:41:16<2:06:08,  5.78s/it]
 91%|█████████ | 13416/14725 [2:41:22<2:09:47,  5.95s/it]
 91%
0: {'loss': 0.5261, 'grad_norm': 0.3823267953868133, 'learning_rate': 3e-06, 'epoch': 0.91}
0: |█████████ | 13417/14725 [2:41:28<2:10:01,  5.96s/it]
 91%|█████████ | 13418/14725 [2:41:34<2:09:33,  5.95s/it]
 91%|█████████ | 13419/14725 [2:41:40<2:06:53,  5.83s/it]
 91%|█████████ | 13420/14725 [2:41:46<2:09:12,  5.94s/it]
                                                         

 91%|█████████ | 13420/14725 [2:41:46<2:09:12,  5.94s/it]
 91%|█████████ | 13421/14725 [2:41:52<2:08:50,  5.93s/it]
 91%|█████████ | 13422/14725 [2:41:57<2:03:58,  5.71s/it]
 91%|█████████ | 13423/14725 [2:42:03<2:06:15,  5.82s/it]
 91%|█████████ | 13424/14725 [2:42:08<2:02:09,  5.63s/it]
 91%|█████████ | 13425/14725 [2:42:13<2:00:12,  5.55s/it]
 91%|█████████ | 13426/14725 [2:42:20<2:03:51,  5.72s/it]
 91%|█████████ | 13427/14725 [2:42:25<2:02:16,  5.65s/it]
 91%|█████████ | 13428/14725 [2:42:30<2
0: {'loss': 0.5277, 'grad_norm': 0.3581110127464882, 'learning_rate': 3e-06, 'epoch': 0.91}
0: :00:08,  5.56s/it]
 91%|█████████ | 13429/14725 [2:42:36<2:02:43,  5.68s/it]
 91%|█████████ | 13430/14725 [2:42:43<2:07:57,  5.93s/it]
                                                         

 91%|█████████ | 13430/14725 [2:42:43<2:07:57,  5.93s/it]
 91%|█████████ | 13431/14725 [2:42:49<2:10:07,  6.03s/it]
 91%|█████████ | 13432/14725 [2:42:55<2:11:51,  6.12s/it]
 91%|█████████ | 13433/14725 [2:43:01<2:06:35,  5.88s/it]
 91%|█████████ | 13434/14725 [2:43:07<2:06:17,  5.87s/it]
 91%|█████████ | 13435/14725 [2:43:12<2:03:08,  5.73s/it]
 91%|█████████ | 13436/14725 [2:43:18<2:01:25,  5.65s/it]
 91%|█████████▏| 13437/14725 [2:43:23<2:00:49,  5.63s/it]
 91%|█████████▏| 13438/14725 [2:43:29<2:00:29,  5.62s/it]
 91%|█████████▏| 13439/14725 [2:43:34<1:58:20,  5.52s/it]
 91%|███████�
0: {'loss': 0.5322, 'grad_norm': 0.4323923427952663, 'learning_rate': 3e-06, 'epoch': 0.91}
0: �█▏| 13440/14725 [2:43:40<2:00:59,  5.65s/it]
                                                         

 91%|█████████▏| 13440/14725 [2:43:40<2:00:59,  5.65s/it]
 91%|█████████▏| 13441/14725 [2:43:46<2:05:02,  5.84s/it]
 91%|█████████▏| 13442/14725 [2:43:52<2:04:49,  5.84s/it]
 91%|█████████▏| 13443/14725 [2:43:58<2:04:36,  5.83s/it]
 91%|█████████▏| 13444/14725 [2:44:04<2:05:55,  5.90s/it]
 91%|█████████▏| 13445/14725 [2:44:09<2:02:50,  5.76s/it]
 91%|█████████▏| 13446/14725 [2:44:16<2:06:12,  5.92s/it]
 91%|█████████▏| 13447/14725 [2:44:22<2:08:52,  6.05s/it]
 91%|█████████▏| 13448/14725 [2:44:28<2:07:52,  6.01s/it]
 91%|█████████▏| 13449/14725 [2:44:33<2:04:22,  5.85s/it]
 91%|█████████▏| 13450/14725 [2:44:39<2:03:22,  5.81s/it]
                                                         
0: {'loss': 0.5342, 'grad_norm': 0.4099113868208228, 'learning_rate': 3e-06, 'epoch': 0.91}
0: {'loss': 0.5255, 'grad_norm': 0.37989412213698454, 'learning_rate': 3e-06, 'epoch': 0.91}
0: 

 91%|█████████▏| 13450/14725 [2:44:39<2:03:22,  5.81s/it]
 91%|█████████▏| 13451/14725 [2:44:45<2:03:56,  5.84s/it]
 91%|█████████▏| 13452/14725 [2:44:51<2:02:54,  5.79s/it]
 91%|█████████▏| 13453/14725 [2:44:57<2:03:18,  5.82s/it]
 91%|█████████▏| 13454/14725 [2:45:02<2:01:40,  5.74s/it]
 91%|█████████▏| 13455/14725 [2:45:08<2:00:01,  5.67s/it]
 91%|█████████▏| 13456/14725 [2:45:13<1:58:18,  5.59s/it]
 91%|█████████▏| 13457/14725 [2:45:19<1:59:43,  5.67s/it]
 91%|█████████▏| 13458/14725 [2:45:25<2:01:42,  5.76s/it]
 91%|█████████▏| 13459/14725 [2:45:31<2:02:43,  5.82s/it]
 91%|█████████▏| 13460/14725 [2:45:36<2:01:20,  5.76s/it]
                                                         

 91%|█████████▏| 13460/14725 [2:45:36<2:01:20,  5.76s/it]
 91%|███████�
0: {'loss': 0.5299, 'grad_norm': 0.3831706834644564, 'learning_rate': 3e-06, 'epoch': 0.91}
0: ��█▏| 13461/14725 [2:45:42<2:00:00,  5.70s/it]
 91%|█████████▏| 13462/14725 [2:45:48<1:58:56,  5.65s/it]
 91%|█████████▏| 13463/14725 [2:45:53<1:59:45,  5.69s/it]
 91%|█████████▏| 13464/14725 [2:45:59<1:56:42,  5.55s/it]
 91%|█████████▏| 13465/14725 [2:46:05<1:59:46,  5.70s/it]
 91%|█████████▏| 13466/14725 [2:46:10<1:58:17,  5.64s/it]
 91%|█████████▏| 13467/14725 [2:46:16<2:00:23,  5.74s/it]
 91%|█████████▏| 13468/14725 [2:46:22<1:59:26,  5.70s/it]
 91%|█████████▏| 13469/14725 [2:46:28<2:01:23,  5.80s/it]
 91%|█████████▏| 13470/14725 [2:46:33<2:01:10,  5.79s/it]
                                                         

 91%|█████████▏| 13470/14725 [2:46:33<2:01:10,  5.79s/it]
 91%|█████████▏| 13471/14725 [2:46:39<1:59:40,  5.73s/it]
 91%|█████████▏| 13472/14725 [2:46:4
0: {'loss': 0.5293, 'grad_norm': 0.39110838024069955, 'learning_rate': 3e-06, 'epoch': 0.92}
0: 5<2:01:15,  5.81s/it]
 91%|█████████▏| 13473/14725 [2:46:51<2:01:21,  5.82s/it]
 92%|█████████▏| 13474/14725 [2:46:56<1:57:40,  5.64s/it]
 92%|█████████▏| 13475/14725 [2:47:02<1:57:41,  5.65s/it]
 92%|█████████▏| 13476/14725 [2:47:08<1:59:50,  5.76s/it]
 92%|█████████▏| 13477/14725 [2:47:13<1:58:25,  5.69s/it]
 92%|█████████▏| 13478/14725 [2:47:19<1:58:37,  5.71s/it]
 92%|█████████▏| 13479/14725 [2:47:25<1:58:21,  5.70s/it]
 92%|█████████▏| 13480/14725 [2:47:31<2:01:23,  5.85s/it]
                                                         

 92%|█████████▏| 13480/14725 [2:47:31<2:01:23,  5.85s/it]
 92%|█████████▏| 13481/14725 [2:47:36<1:58:23,  5.71s/it]
 92%|█████████▏| 13482/14725 [2:47:42<2:00:53,  5.84s/it]
 92%|█████████▏| 13483/14725 [2:47:48<2:00:34,  5.82s/it]
 92%|�
0: {'loss': 0.528, 'grad_norm': 0.3858803665853733, 'learning_rate': 3e-06, 'epoch': 0.92}
0: �████████▏| 13484/14725 [2:47:54<2:02:42,  5.93s/it]
 92%|█████████▏| 13485/14725 [2:48:00<2:02:01,  5.90s/it]
 92%|█████████▏| 13486/14725 [2:48:06<2:00:11,  5.82s/it]
 92%|█████████▏| 13487/14725 [2:48:12<2:01:58,  5.91s/it]
 92%|█████████▏| 13488/14725 [2:48:18<2:02:25,  5.94s/it]
 92%|█████████▏| 13489/14725 [2:48:24<2:03:35,  6.00s/it]
 92%|█████████▏| 13490/14725 [2:48:30<2:05:24,  6.09s/it]
                                                         

 92%|█████████▏| 13490/14725 [2:48:31<2:05:24,  6.09s/it]
 92%|█████████▏| 13491/14725 [2:48:36<2:01:22,  5.90s/it]
 92%|█████████▏| 13492/14725 [2:48:41<1:58:54,  5.79s/it]
 92%|█████████▏| 13493/14725 [2:48:48<2:00:45,  5.88s/it]
 92%|█████████▏| 13494/14725 [2:48:53<1:58:26,  5.77s/it]
 92%|█████████▏|
0: {'loss': 0.5196, 'grad_norm': 0.4048473848268619, 'learning_rate': 3e-06, 'epoch': 0.92}
0:  13495/14725 [2:48:59<1:58:43,  5.79s/it]
 92%|█████████▏| 13496/14725 [2:49:04<1:57:12,  5.72s/it]
 92%|█████████▏| 13497/14725 [2:49:11<2:00:10,  5.87s/it]
 92%|█████████▏| 13498/14725 [2:49:17<2:00:46,  5.91s/it]
 92%|█████████▏| 13499/14725 [2:49:22<1:55:30,  5.65s/it]
 92%|█████████▏| 13500/14725 [2:49:27<1:55:06,  5.64s/it]
                                                         

 92%|█████████▏| 13500/14725 [2:49:27<1:55:06,  5.64s/it]
 92%|█████████▏| 13501/14725 [2:49:34<1:59:05,  5.84s/it]
 92%|█████████▏| 13502/14725 [2:49:39<1:56:24,  5.71s/it]
 92%|█████████▏| 13503/14725 [2:49:44<1:54:05,  5.60s/it]
 92%|█████████▏| 13504/14725 [2:49:50<1:51:46,  5.49s/it]
 92%|█████████▏| 13505/14725 [2:49:55<1:50:11,  5.42s/it]
 92%|█████████▏| 13506/14725 [2:50:00<1:50:22
0: {'loss': 0.5238, 'grad_norm': 0.381522354451478, 'learning_rate': 3e-06, 'epoch': 0.92}
0: ,  5.43s/it]
 92%|█████████▏| 13507/14725 [2:50:07<1:55:30,  5.69s/it]
 92%|█████████▏| 13508/14725 [2:50:12<1:56:11,  5.73s/it]
 92%|█████████▏| 13509/14725 [2:50:19<1:57:52,  5.82s/it]
 92%|█████████▏| 13510/14725 [2:50:24<1:58:39,  5.86s/it]
                                                         

 92%|█████████▏| 13510/14725 [2:50:24<1:58:39,  5.86s/it]
 92%|█████████▏| 13511/14725 [2:50:31<1:59:55,  5.93s/it]
 92%|█████████▏| 13512/14725 [2:50:37<2:00:43,  5.97s/it]
 92%|█████████▏| 13513/14725 [2:50:43<2:02:46,  6.08s/it]
 92%|█████████▏| 13514/14725 [2:50:48<1:58:36,  5.88s/it]
 92%|█████████▏| 13515/14725 [2:50:54<1:59:40,  5.93s/it]
 92%|█████████▏| 13516/14725 [2:51:01<2:02:16,  6.07s/it]
 92%|█████████▏| 13517/14725 [2:51:06<1:58:50,  5.90s/it]
 92%|███�
0: {'loss': 0.5295, 'grad_norm': 0.36516268630857346, 'learning_rate': 3e-06, 'epoch': 0.92}
0: �█████▏| 13518/14725 [2:51:12<1:56:50,  5.81s/it]
 92%|█████████▏| 13519/14725 [2:51:18<1:56:33,  5.80s/it]
 92%|█████████▏| 13520/14725 [2:51:24<1:59:42,  5.96s/it]
                                                         

 92%|█████████▏| 13520/14725 [2:51:24<1:59:42,  5.96s/it]
 92%|█████████▏| 13521/14725 [2:51:29<1:55:10,  5.74s/it]
 92%|█████████▏| 13522/14725 [2:51:35<1:55:41,  5.77s/it]
 92%|█████████▏| 13523/14725 [2:51:41<1:57:33,  5.87s/it]
 92%|█████████▏| 13524/14725 [2:51:47<1:56:46,  5.83s/it]
 92%|█████████▏| 13525/14725 [2:51:53<1:55:20,  5.77s/it]
 92%|█████████▏| 13526/14725 [2:51:59<1:57:01,  5.86s/it]
 92%|█████████▏| 13527/14725 [2:52:04<1:54:24,  5.73s/it]
 92%|█████████▏| 13528/14725 [2:52:10<1:57:41,  5.90s/it]
 92%|█████████▏| 13529/14
0: {'loss': 0.5381, 'grad_norm': 0.38632565173340405, 'learning_rate': 3e-06, 'epoch': 0.92}
0: 725 [2:52:16<1:58:43,  5.96s/it]
 92%|█████████▏| 13530/14725 [2:52:22<1:56:19,  5.84s/it]
                                                         

 92%|█████████▏| 13530/14725 [2:52:22<1:56:19,  5.84s/it]
 92%|█████████▏| 13531/14725 [2:52:28<1:55:20,  5.80s/it]
 92%|█████████▏| 13532/14725 [2:52:34<1:58:17,  5.95s/it]
 92%|█████████▏| 13533/14725 [2:52:40<1:58:59,  5.99s/it]
 92%|█████████▏| 13534/14725 [2:52:46<1:58:38,  5.98s/it]
 92%|█████████▏| 13535/14725 [2:52:52<1:55:59,  5.85s/it]
 92%|█████████▏| 13536/14725 [2:52:57<1:55:00,  5.80s/it]
 92%|█████████▏| 13537/14725 [2:53:03<1:53:03,  5.71s/it]
 92%|█████████▏| 13538/14725 [2:53:09<1:54:36,  5.79s/it]
 92%|█████████▏| 13539/14725 [2:53:14<1:53:20,  5.73s/it]
 92%|█████████▏| 13540/14725 [2:53:20<1:54:23,  5.79s/
0: {'loss': 0.5157, 'grad_norm': 0.38926520093318556, 'learning_rate': 3e-06, 'epoch': 0.92}
0: {'loss': 0.5215, 'grad_norm': 0.3901063251029311, 'learning_rate': 3e-06, 'epoch': 0.92}
0: it]
                                                         

 92%|█████████▏| 13540/14725 [2:53:20<1:54:23,  5.79s/it]
 92%|█████████▏| 13541/14725 [2:53:26<1:56:45,  5.92s/it]
 92%|█████████▏| 13542/14725 [2:53:32<1:54:24,  5.80s/it]
 92%|█████████▏| 13543/14725 [2:53:38<1:52:57,  5.73s/it]
 92%|█████████▏| 13544/14725 [2:53:43<1:51:41,  5.67s/it]
 92%|█████████▏| 13545/14725 [2:53:49<1:50:34,  5.62s/it]
 92%|█████████▏| 13546/14725 [2:53:55<1:54:10,  5.81s/it]
 92%|█████████▏| 13547/14725 [2:54:01<1:53:57,  5.80s/it]
 92%|█████████▏| 13548/14725 [2:54:06<1:52:27,  5.73s/it]
 92%|█████████▏| 13549/14725 [2:54:12<1:52:11,  5.72s/it]
 92%|█████████▏| 13550/14725 [2:54:18<1:54:52,  5.87s/it]
                                                         

 92%|█████████▏| 13550/1
0: {'loss': 0.5238, 'grad_norm': 0.3849472447775367, 'learning_rate': 3e-06, 'epoch': 0.92}
0: 4725 [2:54:18<1:54:52,  5.87s/it]
 92%|█████████▏| 13551/14725 [2:54:24<1:55:24,  5.90s/it]
 92%|█████████▏| 13552/14725 [2:54:30<1:54:22,  5.85s/it]
 92%|█████████▏| 13553/14725 [2:54:36<1:54:33,  5.86s/it]
 92%|█████████▏| 13554/14725 [2:54:41<1:51:24,  5.71s/it]
 92%|█████████▏| 13555/14725 [2:54:47<1:53:57,  5.84s/it]
 92%|█████████▏| 13556/14725 [2:54:54<1:56:20,  5.97s/it]
 92%|█████████▏| 13557/14725 [2:55:00<1:57:40,  6.05s/it]
 92%|█████████▏| 13558/14725 [2:55:05<1:52:57,  5.81s/it]
 92%|█████████▏| 13559/14725 [2:55:10<1:50:05,  5.67s/it]
 92%|█████████▏| 13560/14725 [2:55:16<1:51:40,  5.75s/it]
                                                         

 92%|█████████▏| 13560/14725 [2:55:16<1:51:40,  5.75s/it]
 92%|█████████▏| 13561/14725 [2:55:22<1:53:08,  5.83s
0: {'loss': 0.5213, 'grad_norm': 0.39648926658905154, 'learning_rate': 3e-06, 'epoch': 0.92}
0: /it]
 92%|█████████▏| 13562/14725 [2:55:28<1:52:06,  5.78s/it]
 92%|█████████▏| 13563/14725 [2:55:33<1:50:12,  5.69s/it]
 92%|█████████▏| 13564/14725 [2:55:39<1:49:28,  5.66s/it]
 92%|█████████▏| 13565/14725 [2:55:45<1:53:22,  5.86s/it]
 92%|█████████▏| 13566/14725 [2:55:51<1:50:02,  5.70s/it]
 92%|█████████▏| 13567/14725 [2:55:56<1:49:27,  5.67s/it]
 92%|█████████▏| 13568/14725 [2:56:02<1:48:57,  5.65s/it]
 92%|█████████▏| 13569/14725 [2:56:08<1:51:13,  5.77s/it]
 92%|█████████▏| 13570/14725 [2:56:14<1:53:02,  5.87s/it]
                                                         

 92%|█████████▏| 13570/14725 [2:56:14<1:53:02,  5.87s/it]
 92%|█████████▏| 13571/14725 [2:56:20<1:52:49,  5.87s/it]
 92%|█████████▏| 13572/14725 [2:56:25<1:49:58,  5.72s/it]
 92%|██████�
0: {'loss': 0.5299, 'grad_norm': 0.4294818316552482, 'learning_rate': 3e-06, 'epoch': 0.92}
0: ��██▏| 13573/14725 [2:56:31<1:49:23,  5.70s/it]
 92%|█████████▏| 13574/14725 [2:56:36<1:46:13,  5.54s/it]
 92%|█████████▏| 13575/14725 [2:56:42<1:47:43,  5.62s/it]
 92%|█████████▏| 13576/14725 [2:56:48<1:47:28,  5.61s/it]
 92%|█████████▏| 13577/14725 [2:56:54<1:50:53,  5.80s/it]
 92%|█████████▏| 13578/14725 [2:57:00<1:51:34,  5.84s/it]
 92%|█████████▏| 13579/14725 [2:57:05<1:50:10,  5.77s/it]
 92%|█████████▏| 13580/14725 [2:57:11<1:49:12,  5.72s/it]
                                                         

 92%|█████████▏| 13580/14725 [2:57:11<1:49:12,  5.72s/it]
 92%|█████████▏| 13581/14725 [2:57:17<1:49:57,  5.77s/it]
 92%|█████████▏| 13582/14725 [2:57:23<1:51:14,  5.84s/it]
 92%|█████████▏| 13583/14725 [2:57:29<1:52:18,  5.90s/it]
 92%|█████████▏| 13584/14725 [2:5
0: {'loss': 0.5323, 'grad_norm': 0.41897200430588777, 'learning_rate': 3e-06, 'epoch': 0.92}
0: 7:35<1:53:56,  5.99s/it]
 92%|█████████▏| 13585/14725 [2:57:41<1:53:44,  5.99s/it]
 92%|█████████▏| 13586/14725 [2:57:48<1:56:37,  6.14s/it]
 92%|█████████▏| 13587/14725 [2:57:54<1:57:38,  6.20s/it]
 92%|█████████▏| 13588/14725 [2:58:00<1:58:06,  6.23s/it]
 92%|█████████▏| 13589/14725 [2:58:06<1:58:40,  6.27s/it]
 92%|█████████▏| 13590/14725 [2:58:13<1:58:17,  6.25s/it]
                                                         

 92%|█████████▏| 13590/14725 [2:58:13<1:58:17,  6.25s/it]
 92%|█████████▏| 13591/14725 [2:58:19<1:57:41,  6.23s/it]
 92%|█████████▏| 13592/14725 [2:58:24<1:53:51,  6.03s/it]
 92%|█████████▏| 13593/14725 [2:58:30<1:52:09,  5.94s/it]
 92%|█████████▏| 13594/14725 [2:58:36<1:52:58,  5.99s/it]
 92%|█████████▏| 13595/14725 [2:58:42<1:48:51,  5.78s/it]
 92%
0: {'loss': 0.5154, 'grad_norm': 0.3988611416663777, 'learning_rate': 3e-06, 'epoch': 0.92}
0: |█████████▏| 13596/14725 [2:58:47<1:49:08,  5.80s/it]
 92%|█████████▏| 13597/14725 [2:58:53<1:49:54,  5.85s/it]
 92%|█████████▏| 13598/14725 [2:59:00<1:52:16,  5.98s/it]
 92%|█████████▏| 13599/14725 [2:59:05<1:48:33,  5.78s/it]
 92%|█████████▏| 13600/14725 [2:59:11<1:49:44,  5.85s/it]
                                                         

 92%|█████████▏| 13600/14725 [2:59:11<1:49:44,  5.85s/it]
 92%|█████████▏| 13601/14725 [2:59:17<1:49:11,  5.83s/it]
 92%|█████████▏| 13602/14725 [2:59:23<1:50:12,  5.89s/it]
 92%|█████████▏| 13603/14725 [2:59:29<1:50:56,  5.93s/it]
 92%|█████████▏| 13604/14725 [2:59:35<1:53:32,  6.08s/it]
 92%|█████████▏| 13605/14725 [2:59:41<1:52:08,  6.01s/it]
 92%|█████████▏| 13606/14725 [2:59:47<1:49:33,  5.87s/it]
 92%|█████████�
0: {'loss': 0.5298, 'grad_norm': 0.3943000170322222, 'learning_rate': 3e-06, 'epoch': 0.92}
0: ��| 13607/14725 [2:59:52<1:47:01,  5.74s/it]
 92%|█████████▏| 13608/14725 [2:59:58<1:46:50,  5.74s/it]
 92%|█████████▏| 13609/14725 [3:00:04<1:49:04,  5.86s/it]
 92%|█████████▏| 13610/14725 [3:00:10<1:47:24,  5.78s/it]
                                                         

 92%|█████████▏| 13610/14725 [3:00:10<1:47:24,  5.78s/it]
 92%|█████████▏| 13611/14725 [3:00:15<1:47:38,  5.80s/it]
 92%|█████████▏| 13612/14725 [3:00:22<1:50:32,  5.96s/it]
 92%|█████████▏| 13613/14725 [3:00:28<1:49:58,  5.93s/it]
 92%|█████████▏| 13614/14725 [3:00:34<1:49:43,  5.93s/it]
 92%|█████████▏| 13615/14725 [3:00:39<1:48:36,  5.87s/it]
 92%|█████████▏| 13616/14725 [3:00:45<1:49:58,  5.95s/it]
 92%|█████████▏| 13617/14725 [3:00:52<1:51:36,  6.04s/it]
 92%|█████████▏| 13618/14725 [3:00:58<1:51
0: {'loss': 0.537, 'grad_norm': 0.4044988316000102, 'learning_rate': 3e-06, 'epoch': 0.92}
0: :24,  6.04s/it]
 92%|█████████▏| 13619/14725 [3:01:04<1:51:53,  6.07s/it]
 92%|█████████▏| 13620/14725 [3:01:10<1:49:46,  5.96s/it]
                                                         

 92%|█████████▏| 13620/14725 [3:01:10<1:49:46,  5.96s/it]
 93%|█████████▎| 13621/14725 [3:01:15<1:48:31,  5.90s/it]
 93%|█████████▎| 13622/14725 [3:01:22<1:51:08,  6.05s/it]
 93%|█████████▎| 13623/14725 [3:01:27<1:47:20,  5.84s/it]
 93%|█████████▎| 13624/14725 [3:01:33<1:45:47,  5.76s/it]
 93%|█████████▎| 13625/14725 [3:01:39<1:47:22,  5.86s/it]
 93%|█████████▎| 13626/14725 [3:01:44<1:45:14,  5.75s/it]
 93%|█████████▎| 13627/14725 [3:01:50<1:43:54,  5.68s/it]
 93%|█████████▎| 13628/14725 [3:01:56<1:46:11,  5.81s/it]
 93%|█████████▎| 13629/14725 [3:02:02<1:48:31,  5.94s/it]
 93%|██�
0: {'loss': 0.525, 'grad_norm': 0.3917107358735829, 'learning_rate': 3e-06, 'epoch': 0.93}
0: �██████▎| 13630/14725 [3:02:08<1:46:10,  5.82s/it]
                                                         

 93%|█████████▎| 13630/14725 [3:02:08<1:46:10,  5.82s/it]
 93%|█████████▎| 13631/14725 [3:02:13<1:45:38,  5.79s/it]
 93%|█████████▎| 13632/14725 [3:02:20<1:47:53,  5.92s/it]
 93%|█████████▎| 13633/14725 [3:02:25<1:44:15,  5.73s/it]
 93%|█████████▎| 13634/14725 [3:02:30<1:43:23,  5.69s/it]
 93%|█████████▎| 13635/14725 [3:02:36<1:44:36,  5.76s/it]
 93%|█████████▎| 13636/14725 [3:02:42<1:45:33,  5.82s/it]
 93%|█████████▎| 13637/14725 [3:02:48<1:44:43,  5.78s/it]
 93%|█████████▎| 13638/14725 [3:02:54<1:47:43,  5.95s/it]
 93%|█████████▎| 13639/14725 [3:03:00<1:44:43,  5.79s/it]
 93%|█████████▎| 13640/14725 [3:03:05<1:42:14,  5.65s/it]
                                          
0: {'loss': 0.5213, 'grad_norm': 0.36999801177356345, 'learning_rate': 3e-06, 'epoch': 0.93}
0: {'loss': 0.525, 'grad_norm': 0.3937836422698898, 'learning_rate': 3e-06, 'epoch': 0.93}
0:                

 93%|█████████▎| 13640/14725 [3:03:05<1:42:14,  5.65s/it]
 93%|█████████▎| 13641/14725 [3:03:11<1:43:17,  5.72s/it]
 93%|█████████▎| 13642/14725 [3:03:17<1:44:36,  5.80s/it]
 93%|█████████▎| 13643/14725 [3:03:22<1:42:49,  5.70s/it]
 93%|█████████▎| 13644/14725 [3:03:28<1:42:07,  5.67s/it]
 93%|█████████▎| 13645/14725 [3:03:34<1:43:37,  5.76s/it]
 93%|█████████▎| 13646/14725 [3:03:40<1:44:46,  5.83s/it]
 93%|█████████▎| 13647/14725 [3:03:46<1:46:54,  5.95s/it]
 93%|█████████▎| 13648/14725 [3:03:52<1:46:23,  5.93s/it]
 93%|█████████▎| 13649/14725 [3:03:58<1:43:57,  5.80s/it]
 93%|█████████▎| 13650/14725 [3:04:03<1:42:12,  5.70s/it]
                                                         

 93%|█████████▎| 13650/14725 [3:04:03<1:42:12,  5.70s/it]
 93%|██�
0: {'loss': 0.5291, 'grad_norm': 0.38493748479549517, 'learning_rate': 3e-06, 'epoch': 0.93}
0: ��██████▎| 13651/14725 [3:04:09<1:41:49,  5.69s/it]
 93%|█████████▎| 13652/14725 [3:04:15<1:43:39,  5.80s/it]
 93%|█████████▎| 13653/14725 [3:04:21<1:46:58,  5.99s/it]
 93%|█████████▎| 13654/14725 [3:04:27<1:48:20,  6.07s/it]
 93%|█████████▎| 13655/14725 [3:04:33<1:47:59,  6.06s/it]
 93%|█████████▎| 13656/14725 [3:04:39<1:44:26,  5.86s/it]
 93%|█████████▎| 13657/14725 [3:04:45<1:44:33,  5.87s/it]
 93%|█████████▎| 13658/14725 [3:04:51<1:44:19,  5.87s/it]
 93%|█████████▎| 13659/14725 [3:04:57<1:47:56,  6.08s/it]
 93%|█████████▎| 13660/14725 [3:05:02<1:42:45,  5.79s/it]
                                                         

 93%|█████████▎| 13660/14725 [3:05:02<1:42:45,  5.79s/it]
 93%|█████████▎| 13661/14725 [3:05:08<1:43:34,  5.84s/it]
 93%|█████████▎| 1366
0: {'loss': 0.5232, 'grad_norm': 0.37288375710555965, 'learning_rate': 3e-06, 'epoch': 0.93}
0: 2/14725 [3:05:15<1:46:30,  6.01s/it]
 93%|█████████▎| 13663/14725 [3:05:20<1:44:07,  5.88s/it]
 93%|█████████▎| 13664/14725 [3:05:26<1:44:52,  5.93s/it]
 93%|█████████▎| 13665/14725 [3:05:32<1:42:29,  5.80s/it]
 93%|█████████▎| 13666/14725 [3:05:37<1:40:52,  5.71s/it]
 93%|█████████▎| 13667/14725 [3:05:43<1:40:18,  5.69s/it]
 93%|█████████▎| 13668/14725 [3:05:48<1:39:07,  5.63s/it]
 93%|█████████▎| 13669/14725 [3:05:54<1:38:46,  5.61s/it]
 93%|█████████▎| 13670/14725 [3:06:00<1:42:11,  5.81s/it]
                                                         

 93%|█████████▎| 13670/14725 [3:06:00<1:42:11,  5.81s/it]
 93%|█████████▎| 13671/14725 [3:06:06<1:43:31,  5.89s/it]
 93%|█████████▎| 13672/14725 [3:06:12<1:41:00,  5.76s/it]
 93%|█████████▎| 13673/14725 [3:06:18<1:41:02,  5.
0: {'loss': 0.5231, 'grad_norm': 0.383808499820886, 'learning_rate': 3e-06, 'epoch': 0.93}
0: 76s/it]
 93%|█████████▎| 13674/14725 [3:06:23<1:37:33,  5.57s/it]
 93%|█████████▎| 13675/14725 [3:06:28<1:37:10,  5.55s/it]
 93%|█████████▎| 13676/14725 [3:06:34<1:39:30,  5.69s/it]
 93%|█████████▎| 13677/14725 [3:06:40<1:42:18,  5.86s/it]
 93%|█████████▎| 13678/14725 [3:06:46<1:40:50,  5.78s/it]
 93%|█████████▎| 13679/14725 [3:06:52<1:41:35,  5.83s/it]
 93%|█████████▎| 13680/14725 [3:06:58<1:41:29,  5.83s/it]
                                                         

 93%|█████████▎| 13680/14725 [3:06:58<1:41:29,  5.83s/it]
 93%|█████████▎| 13681/14725 [3:07:04<1:43:00,  5.92s/it]
 93%|█████████▎| 13682/14725 [3:07:10<1:41:57,  5.87s/it]
 93%|█████████▎| 13683/14725 [3:07:15<1:41:12,  5.83s/it]
 93%|█████████▎| 13684/14725 [3:07:21<1:41:46,  5.87s/it]
 93%|█████�
0: {'loss': 0.5227, 'grad_norm': 0.3919658251932925, 'learning_rate': 3e-06, 'epoch': 0.93}
0: ��███▎| 13685/14725 [3:07:27<1:39:39,  5.75s/it]
 93%|█████████▎| 13686/14725 [3:07:33<1:42:19,  5.91s/it]
 93%|█████████▎| 13687/14725 [3:07:39<1:39:33,  5.75s/it]
 93%|█████████▎| 13688/14725 [3:07:45<1:41:05,  5.85s/it]
 93%|█████████▎| 13689/14725 [3:07:50<1:38:08,  5.68s/it]
 93%|█████████▎| 13690/14725 [3:07:55<1:36:23,  5.59s/it]
                                                         

 93%|█████████▎| 13690/14725 [3:07:55<1:36:23,  5.59s/it]
 93%|█████████▎| 13691/14725 [3:08:01<1:39:14,  5.76s/it]
 93%|█████████▎| 13692/14725 [3:08:07<1:36:44,  5.62s/it]
 93%|█████████▎| 13693/14725 [3:08:13<1:37:45,  5.68s/it]
 93%|█████████▎| 13694/14725 [3:08:18<1:37:13,  5.66s/it]
 93%|█████████▎| 13695/14725 [3:08:24<1:40:04,  5.83s/it]
 93%|█████████▎| 13696/14725 [
0: {'loss': 0.5127, 'grad_norm': 0.3719756765018114, 'learning_rate': 3e-06, 'epoch': 0.93}
0: 3:08:30<1:38:39,  5.75s/it]
 93%|█████████▎| 13697/14725 [3:08:36<1:39:44,  5.82s/it]
 93%|█████████▎| 13698/14725 [3:08:42<1:40:15,  5.86s/it]
 93%|█████████▎| 13699/14725 [3:08:47<1:38:04,  5.74s/it]
 93%|█████████▎| 13700/14725 [3:08:53<1:36:45,  5.66s/it]
                                                         

 93%|█████████▎| 13700/14725 [3:08:53<1:36:45,  5.66s/it]
 93%|█████████▎| 13701/14725 [3:08:58<1:36:26,  5.65s/it]
 93%|█████████▎| 13702/14725 [3:09:04<1:35:27,  5.60s/it]
 93%|█████████▎| 13703/14725 [3:09:10<1:36:29,  5.66s/it]
 93%|█████████▎| 13704/14725 [3:09:15<1:35:54,  5.64s/it]
 93%|█████████▎| 13705/14725 [3:09:21<1:36:17,  5.66s/it]
 93%|█████████▎| 13706/14725 [3:09:27<1:38:10,  5.78s/it]
 93%|█████████▎| 13707/14725 [3:09:33<1:36:19,  5.68s/it]
 
0: {'loss': 0.5261, 'grad_norm': 0.4108836199546628, 'learning_rate': 3e-06, 'epoch': 0.93}
0: 93%|█████████▎| 13708/14725 [3:09:39<1:39:11,  5.85s/it]
 93%|█████████▎| 13709/14725 [3:09:45<1:41:08,  5.97s/it]
 93%|█████████▎| 13710/14725 [3:09:51<1:41:45,  6.01s/it]
                                                         

 93%|█████████▎| 13710/14725 [3:09:51<1:41:45,  6.01s/it]
 93%|█████████▎| 13711/14725 [3:09:57<1:40:04,  5.92s/it]
 93%|█████████▎| 13712/14725 [3:10:03<1:39:10,  5.87s/it]
 93%|█████████▎| 13713/14725 [3:10:09<1:39:16,  5.89s/it]
 93%|█████████▎| 13714/14725 [3:10:14<1:36:16,  5.71s/it]
 93%|█████████▎| 13715/14725 [3:10:20<1:36:26,  5.73s/it]
 93%|█████████▎| 13716/14725 [3:10:25<1:35:21,  5.67s/it]
 93%|█████████▎| 13717/14725 [3:10:31<1:37:10,  5.78s/it]
 93%|█████████▎| 13718/14725 [3:10:37<1:36:09,  5.73s/it]
 93%|████████�
0: {'loss': 0.5204, 'grad_norm': 0.3875185943049664, 'learning_rate': 3e-06, 'epoch': 0.93}
0: ��▎| 13719/14725 [3:10:42<1:35:20,  5.69s/it]
 93%|█████████▎| 13720/14725 [3:10:48<1:36:11,  5.74s/it]
                                                         

 93%|█████████▎| 13720/14725 [3:10:48<1:36:11,  5.74s/it]
 93%|█████████▎| 13721/14725 [3:10:54<1:36:20,  5.76s/it]
 93%|█████████▎| 13722/14725 [3:11:00<1:35:53,  5.74s/it]
 93%|█████████▎| 13723/14725 [3:11:06<1:36:26,  5.77s/it]
 93%|█████████▎| 13724/14725 [3:11:12<1:36:44,  5.80s/it]
 93%|█████████▎| 13725/14725 [3:11:17<1:36:02,  5.76s/it]
 93%|█████████▎| 13726/14725 [3:11:23<1:37:43,  5.87s/it]
 93%|█████████▎| 13727/14725 [3:11:30<1:39:17,  5.97s/it]
 93%|█████████▎| 13728/14725 [3:11:35<1:34:55,  5.71s/it]
 93%|█████████▎| 13729/14725 [3:11:40<1:35:23,  5.75s/it]
 93%|█████████▎| 13730/14725 [3:11:46<1
0: {'loss': 0.5324, 'grad_norm': 0.4142202644293291, 'learning_rate': 3e-06, 'epoch': 0.93}
0: {'loss': 0.5184, 'grad_norm': 0.3800594374133856, 'learning_rate': 3e-06, 'epoch': 0.93}
0: :32:52,  5.60s/it]
                                                         

 93%|█████████▎| 13730/14725 [3:11:46<1:32:52,  5.60s/it]
 93%|█████████▎| 13731/14725 [3:11:52<1:34:57,  5.73s/it]
 93%|█████████▎| 13732/14725 [3:11:58<1:37:17,  5.88s/it]
 93%|█████████▎| 13733/14725 [3:12:04<1:39:01,  5.99s/it]
 93%|█████████▎| 13734/14725 [3:12:10<1:35:41,  5.79s/it]
 93%|█████████▎| 13735/14725 [3:12:15<1:32:30,  5.61s/it]
 93%|█████████▎| 13736/14725 [3:12:21<1:35:01,  5.76s/it]
 93%|█████████▎| 13737/14725 [3:12:27<1:34:54,  5.76s/it]
 93%|█████████▎| 13738/14725 [3:12:32<1:33:44,  5.70s/it]
 93%|█████████▎| 13739/14725 [3:12:38<1:34:35,  5.76s/it]
 93%|█████████▎| 13740/14725 [3:12:44<1:34:34,  5.76s/it]
                                                         

 93%|████████
0: {'loss': 0.5206, 'grad_norm': 0.4030399689994678, 'learning_rate': 3e-06, 'epoch': 0.93}
0: █▎| 13740/14725 [3:12:44<1:34:34,  5.76s/it]
 93%|█████████▎| 13741/14725 [3:12:50<1:34:49,  5.78s/it]
 93%|█████████▎| 13742/14725 [3:12:56<1:35:32,  5.83s/it]
 93%|█████████▎| 13743/14725 [3:13:01<1:35:01,  5.81s/it]
 93%|█████████▎| 13744/14725 [3:13:08<1:37:43,  5.98s/it]
 93%|█████████▎| 13745/14725 [3:13:13<1:36:27,  5.91s/it]
 93%|█████████▎| 13746/14725 [3:13:20<1:37:28,  5.97s/it]
 93%|█████████▎| 13747/14725 [3:13:26<1:39:22,  6.10s/it]
 93%|█████████▎| 13748/14725 [3:13:32<1:39:05,  6.09s/it]
 93%|█████████▎| 13749/14725 [3:13:38<1:36:39,  5.94s/it]
 93%|█████████▎| 13750/14725 [3:13:43<1:33:37,  5.76s/it]
                                                         

 93%|█████████▎| 13750/14725 [3:13:43<1:33:37,  5.76s/it]
 93%|█████████▎| 13751/14725 [3:13:48<
0: {'loss': 0.519, 'grad_norm': 0.3874744679936435, 'learning_rate': 3e-06, 'epoch': 0.93}
0: 1:31:14,  5.62s/it]
 93%|█████████▎| 13752/14725 [3:13:54<1:31:58,  5.67s/it]
 93%|█████████▎| 13753/14725 [3:14:00<1:31:11,  5.63s/it]
 93%|█████████▎| 13754/14725 [3:14:06<1:32:31,  5.72s/it]
 93%|█████████▎| 13755/14725 [3:14:11<1:31:06,  5.64s/it]
 93%|█████████▎| 13756/14725 [3:14:17<1:33:07,  5.77s/it]
 93%|█████████▎| 13757/14725 [3:14:23<1:35:29,  5.92s/it]
 93%|█████████▎| 13758/14725 [3:14:30<1:37:51,  6.07s/it]
 93%|█████████▎| 13759/14725 [3:14:35<1:35:30,  5.93s/it]
 93%|█████████▎| 13760/14725 [3:14:41<1:34:42,  5.89s/it]
                                                         

 93%|█████████▎| 13760/14725 [3:14:41<1:34:42,  5.89s/it]
 93%|█████████▎| 13761/14725 [3:14:46<1:31:30,  5.70s/it]
 93%|█████████▎| 13762/14725 [3:14:53<1:33:57,  5.85s/it]
 93%|█�
0: {'loss': 0.5326, 'grad_norm': 0.37936039245634867, 'learning_rate': 3e-06, 'epoch': 0.94}
0: ��███████▎| 13763/14725 [3:14:58<1:33:28,  5.83s/it]
 93%|█████████▎| 13764/14725 [3:15:04<1:33:54,  5.86s/it]
 93%|█████████▎| 13765/14725 [3:15:10<1:31:32,  5.72s/it]
 93%|█████████▎| 13766/14725 [3:15:16<1:32:27,  5.78s/it]
 93%|█████████▎| 13767/14725 [3:15:21<1:29:55,  5.63s/it]
 94%|█████████▎| 13768/14725 [3:15:27<1:33:53,  5.89s/it]
 94%|█████████▎| 13769/14725 [3:15:33<1:33:54,  5.89s/it]
 94%|█████████▎| 13770/14725 [3:15:39<1:34:09,  5.92s/it]
                                                         

 94%|█████████▎| 13770/14725 [3:15:39<1:34:09,  5.92s/it]
 94%|█████████▎| 13771/14725 [3:15:45<1:35:01,  5.98s/it]
 94%|█████████▎| 13772/14725 [3:15:51<1:33:42,  5.90s/it]
 94%|█████████▎| 13773/14725 [3:15:57<1:33:34,  5.90s/it]
 94%|█████████▎| 1
0: {'loss': 0.5204, 'grad_norm': 0.3658090667943685, 'learning_rate': 3e-06, 'epoch': 0.94}
0: 3774/14725 [3:16:03<1:31:58,  5.80s/it]
 94%|█████████▎| 13775/14725 [3:16:09<1:35:28,  6.03s/it]
 94%|█████████▎| 13776/14725 [3:16:15<1:32:50,  5.87s/it]
 94%|█████████▎| 13777/14725 [3:16:20<1:32:40,  5.87s/it]
 94%|█████████▎| 13778/14725 [3:16:26<1:31:52,  5.82s/it]
 94%|█████████▎| 13779/14725 [3:16:32<1:30:51,  5.76s/it]
 94%|█████████▎| 13780/14725 [3:16:37<1:29:17,  5.67s/it]
                                                         

 94%|█████████▎| 13780/14725 [3:16:37<1:29:17,  5.67s/it]
 94%|█████████▎| 13781/14725 [3:16:43<1:28:40,  5.64s/it]
 94%|█████████▎| 13782/14725 [3:16:49<1:32:14,  5.87s/it]
 94%|█████████▎| 13783/14725 [3:16:55<1:30:00,  5.73s/it]
 94%|█████████▎| 13784/14725 [3:17:00<1:29:58,  5.74s/it]
 94%|█████████▎| 13785/14725 [3:17:07<1:31:31, 
0: {'loss': 0.5348, 'grad_norm': 0.37911051041587973, 'learning_rate': 3e-06, 'epoch': 0.94}
0:  5.84s/it]
 94%|█████████▎| 13786/14725 [3:17:13<1:32:31,  5.91s/it]
 94%|█████████▎| 13787/14725 [3:17:18<1:31:03,  5.82s/it]
 94%|█████████▎| 13788/14725 [3:17:23<1:28:28,  5.67s/it]
 94%|█████████▎| 13789/14725 [3:17:29<1:27:46,  5.63s/it]
 94%|█████████▎| 13790/14725 [3:17:35<1:28:03,  5.65s/it]
                                                         

 94%|█████████▎| 13790/14725 [3:17:35<1:28:03,  5.65s/it]
 94%|█████████▎| 13791/14725 [3:17:41<1:29:55,  5.78s/it]
 94%|█████████▎| 13792/14725 [3:17:46<1:28:45,  5.71s/it]
 94%|█████████▎| 13793/14725 [3:17:52<1:29:52,  5.79s/it]
 94%|█████████▎| 13794/14725 [3:17:58<1:29:16,  5.75s/it]
 94%|█████████▎| 13795/14725 [3:18:05<1:33:02,  6.00s/it]
 94%|█████████▎| 13796/14725 [3:18:10<1:32:16,  5.96s/it]
 94%|████�
0: {'loss': 0.5194, 'grad_norm': 0.38833274840172494, 'learning_rate': 3e-06, 'epoch': 0.94}
0: ��████▎| 13797/14725 [3:18:16<1:29:30,  5.79s/it]
 94%|█████████▎| 13798/14725 [3:18:21<1:27:34,  5.67s/it]
 94%|█████████▎| 13799/14725 [3:18:27<1:26:08,  5.58s/it]
 94%|█████████▎| 13800/14725 [3:18:33<1:28:40,  5.75s/it]
                                                         

 94%|█████████▎| 13800/14725 [3:18:33<1:28:40,  5.75s/it]
 94%|█████████▎| 13801/14725 [3:18:38<1:25:38,  5.56s/it]
 94%|█████████▎| 13802/14725 [3:18:44<1:28:10,  5.73s/it]
 94%|█████████▎| 13803/14725 [3:18:50<1:30:55,  5.92s/it]
 94%|█████████▎| 13804/14725 [3:18:56<1:27:41,  5.71s/it]
 94%|█████████▍| 13805/14725 [3:19:02<1:29:33,  5.84s/it]
 94%|█████████▍| 13806/14725 [3:19:08<1:30:42,  5.92s/it]
 94%|█████████▍| 13807/14725 [3:19:13<1:27:03,  5.69s/it]
 94%|█████████▍| 13808/1472
0: {'loss': 0.5285, 'grad_norm': 0.3749494737254719, 'learning_rate': 3e-06, 'epoch': 0.94}
0: 5 [3:19:19<1:27:59,  5.76s/it]
 94%|█████████▍| 13809/14725 [3:19:25<1:29:50,  5.88s/it]
 94%|█████████▍| 13810/14725 [3:19:31<1:28:02,  5.77s/it]
                                                         

 94%|█████████▍| 13810/14725 [3:19:31<1:28:02,  5.77s/it]
 94%|█████████▍| 13811/14725 [3:19:37<1:29:31,  5.88s/it]
 94%|█████████▍| 13812/14725 [3:19:42<1:28:04,  5.79s/it]
 94%|█████████▍| 13813/14725 [3:19:48<1:27:03,  5.73s/it]
 94%|█████████▍| 13814/14725 [3:19:54<1:27:01,  5.73s/it]
 94%|█████████▍| 13815/14725 [3:20:00<1:29:56,  5.93s/it]
 94%|█████████▍| 13816/14725 [3:20:06<1:27:55,  5.80s/it]
 94%|█████████▍| 13817/14725 [3:20:11<1:27:04,  5.75s/it]
 94%|█████████▍| 13818/14725 [3:20:17<1:28:58,  5.89s/it]
 94%|█████████▍| 13819/14725 [3:20:23<1:29:48,  5.95s/it
0: {'loss': 0.5302, 'grad_norm': 0.3695449686392507, 'learning_rate': 3e-06, 'epoch': 0.94}
0: ]
 94%|█████████▍| 13820/14725 [3:20:29<1:28:50,  5.89s/it]
                                                         

 94%|█████████▍| 13820/14725 [3:20:29<1:28:50,  5.89s/it]
 94%|█████████▍| 13821/14725 [3:20:34<1:25:51,  5.70s/it]
 94%|█████████▍| 13822/14725 [3:20:40<1:25:36,  5.69s/it]
 94%|█████████▍| 13823/14725 [3:20:46<1:24:30,  5.62s/it]
 94%|█████████▍| 13824/14725 [3:20:51<1:24:11,  5.61s/it]
 94%|█████████▍| 13825/14725 [3:20:56<1:22:42,  5.51s/it]
 94%|█████████▍| 13826/14725 [3:21:02<1:22:53,  5.53s/it]
 94%|█████████▍| 13827/14725 [3:21:08<1:24:33,  5.65s/it]
 94%|█████████▍| 13828/14725 [3:21:13<1:23:48,  5.61s/it]
 94%|█████████▍| 13829/14725 [3:21:20<1:27:37,  5.87s/it]
 94%|█████████▍| 13830/14725 [3:21:26<1:27:01,  5.83s/it]
                           
0: {'loss': 0.5131, 'grad_norm': 0.3748303173396113, 'learning_rate': 3e-06, 'epoch': 0.94}
0: {'loss': 0.5268, 'grad_norm': 0.40397409743718027, 'learning_rate': 3e-06, 'epoch': 0.94}
0:                               

 94%|█████████▍| 13830/14725 [3:21:26<1:27:01,  5.83s/it]
 94%|█████████▍| 13831/14725 [3:21:31<1:26:03,  5.78s/it]
 94%|█████████▍| 13832/14725 [3:21:37<1:27:09,  5.86s/it]
 94%|█████████▍| 13833/14725 [3:21:43<1:27:03,  5.86s/it]
 94%|█████████▍| 13834/14725 [3:21:49<1:27:02,  5.86s/it]
 94%|█████████▍| 13835/14725 [3:21:55<1:27:12,  5.88s/it]
 94%|█████████▍| 13836/14725 [3:22:01<1:25:29,  5.77s/it]
 94%|█████████▍| 13837/14725 [3:22:06<1:25:43,  5.79s/it]
 94%|█████████▍| 13838/14725 [3:22:12<1:25:12,  5.76s/it]
 94%|█████████▍| 13839/14725 [3:22:18<1:24:23,  5.72s/it]
 94%|█████████▍| 13840/14725 [3:22:24<1:26:20,  5.85s/it]
                                                         

 94%|█████████▍| 13840/14725 [3:22:24<1:26:20,  5.85s/i
0: {'loss': 0.5368, 'grad_norm': 0.3805867198643524, 'learning_rate': 3e-06, 'epoch': 0.94}
0: t]
 94%|█████████▍| 13841/14725 [3:22:30<1:28:35,  6.01s/it]
 94%|█████████▍| 13842/14725 [3:22:36<1:26:39,  5.89s/it]
 94%|█████████▍| 13843/14725 [3:22:42<1:25:46,  5.83s/it]
 94%|█████████▍| 13844/14725 [3:22:47<1:24:12,  5.73s/it]
 94%|█████████▍| 13845/14725 [3:22:53<1:23:12,  5.67s/it]
 94%|█████████▍| 13846/14725 [3:22:59<1:24:28,  5.77s/it]
 94%|█████████▍| 13847/14725 [3:23:04<1:22:58,  5.67s/it]
 94%|█████████▍| 13848/14725 [3:23:10<1:23:37,  5.72s/it]
 94%|█████████▍| 13849/14725 [3:23:16<1:25:05,  5.83s/it]
 94%|█████████▍| 13850/14725 [3:23:22<1:24:40,  5.81s/it]
                                                         

 94%|█████████▍| 13850/14725 [3:23:22<1:24:40,  5.81s/it]
 94%|█████████▍| 13851/14725 [3:23:27<1:23:26,  5.73s/it]
 94%|███████
0: {'loss': 0.5296, 'grad_norm': 0.3773104168003147, 'learning_rate': 3e-06, 'epoch': 0.94}
0: ██▍| 13852/14725 [3:23:33<1:25:00,  5.84s/it]
 94%|█████████▍| 13853/14725 [3:23:39<1:23:37,  5.75s/it]
 94%|█████████▍| 13854/14725 [3:23:45<1:26:09,  5.93s/it]
 94%|█████████▍| 13855/14725 [3:23:52<1:28:05,  6.08s/it]
 94%|█████████▍| 13856/14725 [3:23:57<1:26:17,  5.96s/it]
 94%|█████████▍| 13857/14725 [3:24:03<1:25:28,  5.91s/it]
 94%|█████████▍| 13858/14725 [3:24:09<1:24:48,  5.87s/it]
 94%|█████████▍| 13859/14725 [3:24:14<1:22:53,  5.74s/it]
 94%|█████████▍| 13860/14725 [3:24:20<1:22:30,  5.72s/it]
                                                         

 94%|█████████▍| 13860/14725 [3:24:20<1:22:30,  5.72s/it]
 94%|█████████▍| 13861/14725 [3:24:26<1:21:57,  5.69s/it]
 94%|█████████▍| 13862/14725 [3:24:32<1:24:25,  5.87s/it]
 94%|█████████▍| 13863/14725 [3:24:
0: {'loss': 0.5246, 'grad_norm': 0.3726273732323296, 'learning_rate': 3e-06, 'epoch': 0.94}
0: 38<1:23:39,  5.82s/it]
 94%|█████████▍| 13864/14725 [3:24:44<1:23:54,  5.85s/it]
 94%|█████████▍| 13865/14725 [3:24:48<1:19:57,  5.58s/it]
 94%|█████████▍| 13866/14725 [3:24:54<1:21:03,  5.66s/it]
 94%|█████████▍| 13867/14725 [3:25:00<1:22:57,  5.80s/it]
 94%|█████████▍| 13868/14725 [3:25:06<1:22:51,  5.80s/it]
 94%|█████████▍| 13869/14725 [3:25:12<1:23:40,  5.86s/it]
 94%|█████████▍| 13870/14725 [3:25:18<1:23:28,  5.86s/it]
                                                         

 94%|█████████▍| 13870/14725 [3:25:18<1:23:28,  5.86s/it]
 94%|█████████▍| 13871/14725 [3:25:24<1:21:25,  5.72s/it]
 94%|█████████▍| 13872/14725 [3:25:29<1:20:10,  5.64s/it]
 94%|█████████▍| 13873/14725 [3:25:35<1:20:15,  5.65s/it]
 94%|█████████▍| 13874/14725 [3:25:40<1:20:48,  5.70s/it]
 94%|�
0: {'loss': 0.5116, 'grad_norm': 0.38188602486691425, 'learning_rate': 3e-06, 'epoch': 0.94}
0: ��████████▍| 13875/14725 [3:25:46<1:22:00,  5.79s/it]
 94%|█████████▍| 13876/14725 [3:25:52<1:21:45,  5.78s/it]
 94%|█████████▍| 13877/14725 [3:25:58<1:19:41,  5.64s/it]
 94%|█████████▍| 13878/14725 [3:26:03<1:18:26,  5.56s/it]
 94%|█████████▍| 13879/14725 [3:26:09<1:18:43,  5.58s/it]
 94%|█████████▍| 13880/14725 [3:26:15<1:21:04,  5.76s/it]
                                                         

 94%|█████████▍| 13880/14725 [3:26:15<1:21:04,  5.76s/it]
 94%|█████████▍| 13881/14725 [3:26:21<1:21:47,  5.81s/it]
 94%|█████████▍| 13882/14725 [3:26:26<1:20:28,  5.73s/it]
 94%|█████████▍| 13883/14725 [3:26:32<1:20:26,  5.73s/it]
 94%|█████████▍| 13884/14725 [3:26:37<1:19:01,  5.64s/it]
 94%|█████████▍| 13885/14725 [3:26:43<1:17:59,  5.57s/it]
 94%|█████████▍
0: {'loss': 0.5317, 'grad_norm': 0.3794194745498408, 'learning_rate': 3e-06, 'epoch': 0.94}
0: | 13886/14725 [3:26:49<1:21:02,  5.80s/it]
 94%|█████████▍| 13887/14725 [3:26:54<1:17:57,  5.58s/it]
 94%|█████████▍| 13888/14725 [3:27:00<1:19:35,  5.71s/it]
 94%|█████████▍| 13889/14725 [3:27:06<1:18:51,  5.66s/it]
 94%|█████████▍| 13890/14725 [3:27:12<1:21:17,  5.84s/it]
                                                         

 94%|█████████▍| 13890/14725 [3:27:12<1:21:17,  5.84s/it]
 94%|█████████▍| 13891/14725 [3:27:18<1:20:12,  5.77s/it]
 94%|█████████▍| 13892/14725 [3:27:23<1:18:11,  5.63s/it]
 94%|█████████▍| 13893/14725 [3:27:28<1:17:58,  5.62s/it]
 94%|█████████▍| 13894/14725 [3:27:35<1:20:02,  5.78s/it]
 94%|█████████▍| 13895/14725 [3:27:41<1:21:15,  5.87s/it]
 94%|█████████▍| 13896/14725 [3:27:47<1:21:28,  5.90s/it]
 94%|█████████▍| 13897/14725 [3:27:53<1:21:0
0: {'loss': 0.531, 'grad_norm': 0.39716859310492003, 'learning_rate': 3e-06, 'epoch': 0.94}
0: 2,  5.87s/it]
 94%|█████████▍| 13898/14725 [3:27:58<1:19:01,  5.73s/it]
 94%|█████████▍| 13899/14725 [3:28:03<1:17:34,  5.63s/it]
 94%|█████████▍| 13900/14725 [3:28:10<1:20:24,  5.85s/it]
                                                         

 94%|█████████▍| 13900/14725 [3:28:10<1:20:24,  5.85s/it]
 94%|█████████▍| 13901/14725 [3:28:15<1:19:38,  5.80s/it]
 94%|█████████▍| 13902/14725 [3:28:21<1:17:57,  5.68s/it]
 94%|█████████▍| 13903/14725 [3:28:26<1:17:19,  5.64s/it]
 94%|█████████▍| 13904/14725 [3:28:32<1:17:38,  5.67s/it]
 94%|█████████▍| 13905/14725 [3:28:38<1:20:02,  5.86s/it]
 94%|█████████▍| 13906/14725 [3:28:44<1:20:03,  5.86s/it]
 94%|█████████▍| 13907/14725 [3:28:50<1:21:11,  5.96s/it]
 94%|█████████▍| 13908/14725 [3:28:56<1:21:17,  5.97s/it]
 94%|███�
0: {'loss': 0.5352, 'grad_norm': 0.4001206035085356, 'learning_rate': 3e-06, 'epoch': 0.94}
0: ��█████▍| 13909/14725 [3:29:03<1:22:35,  6.07s/it]
 94%|█████████▍| 13910/14725 [3:29:08<1:18:24,  5.77s/it]
                                                         

 94%|█████████▍| 13910/14725 [3:29:08<1:18:24,  5.77s/it]
 94%|█████████▍| 13911/14725 [3:29:14<1:18:36,  5.79s/it]
 94%|█████████▍| 13912/14725 [3:29:19<1:17:40,  5.73s/it]
 94%|█████████▍| 13913/14725 [3:29:25<1:16:47,  5.67s/it]
 94%|█████████▍| 13914/14725 [3:29:31<1:17:01,  5.70s/it]
 94%|█████████▍| 13915/14725 [3:29:36<1:16:27,  5.66s/it]
 95%|█████████▍| 13916/14725 [3:29:42<1:19:20,  5.88s/it]
 95%|█████████▍| 13917/14725 [3:29:49<1:20:10,  5.95s/it]
 95%|█████████▍| 13918/14725 [3:29:55<1:20:40,  6.00s/it]
 95%|█████████▍| 13919/14725 [3:30:01<1:19:52,  5.95s/it]
 95%|█████████▍| 13920/1
0: {'loss': 0.5223, 'grad_norm': 0.39693671669213604, 'learning_rate': 3e-06, 'epoch': 0.95}
0: {'loss': 0.5206, 'grad_norm': 0.36680960632708465, 'learning_rate': 3e-06, 'epoch': 0.95}
0: 4725 [3:30:06<1:19:19,  5.91s/it]
                                                         

 95%|█████████▍| 13920/14725 [3:30:06<1:19:19,  5.91s/it]
 95%|█████████▍| 13921/14725 [3:30:12<1:17:20,  5.77s/it]
 95%|█████████▍| 13922/14725 [3:30:17<1:16:08,  5.69s/it]
 95%|█████████▍| 13923/14725 [3:30:23<1:16:59,  5.76s/it]
 95%|█████████▍| 13924/14725 [3:30:29<1:16:53,  5.76s/it]
 95%|█████████▍| 13925/14725 [3:30:35<1:17:50,  5.84s/it]
 95%|█████████▍| 13926/14725 [3:30:41<1:17:52,  5.85s/it]
 95%|█████████▍| 13927/14725 [3:30:47<1:17:47,  5.85s/it]
 95%|█████████▍| 13928/14725 [3:30:52<1:15:59,  5.72s/it]
 95%|█████████▍| 13929/14725 [3:30:58<1:14:49,  5.64s/it]
 95%|█████████▍| 13930/14725 [3:31:04<1:17:53,  5.88s/it]
                                                         

 95%|███
0: {'loss': 0.5208, 'grad_norm': 0.3965203292761195, 'learning_rate': 3e-06, 'epoch': 0.95}
0: ██████▍| 13930/14725 [3:31:04<1:17:53,  5.88s/it]
 95%|█████████▍| 13931/14725 [3:31:09<1:15:14,  5.69s/it]
 95%|█████████▍| 13932/14725 [3:31:15<1:14:34,  5.64s/it]
 95%|█████████▍| 13933/14725 [3:31:21<1:16:42,  5.81s/it]
 95%|█████████▍| 13934/14725 [3:31:27<1:18:25,  5.95s/it]
 95%|█████████▍| 13935/14725 [3:31:33<1:16:30,  5.81s/it]
 95%|█████████▍| 13936/14725 [3:31:38<1:15:21,  5.73s/it]
 95%|█████████▍| 13937/14725 [3:31:44<1:15:56,  5.78s/it]
 95%|█████████▍| 13938/14725 [3:31:50<1:17:25,  5.90s/it]
 95%|█████████▍| 13939/14725 [3:31:56<1:16:59,  5.88s/it]
 95%|█████████▍| 13940/14725 [3:32:02<1:17:04,  5.89s/it]
                                                         

 95%|█████████▍| 13940/14725 [3:32:02<1:17:04,  5.89s/it]
 95%|█████████▍| 13941/
0: {'loss': 0.5369, 'grad_norm': 0.4354095255199643, 'learning_rate': 3e-06, 'epoch': 0.95}
0: 14725 [3:32:08<1:17:49,  5.96s/it]
 95%|█████████▍| 13942/14725 [3:32:14<1:17:27,  5.94s/it]
 95%|█████████▍| 13943/14725 [3:32:20<1:15:42,  5.81s/it]
 95%|█████████▍| 13944/14725 [3:32:25<1:13:40,  5.66s/it]
 95%|█████████▍| 13945/14725 [3:32:31<1:16:07,  5.86s/it]
 95%|█████████▍| 13946/14725 [3:32:38<1:17:42,  5.99s/it]
 95%|█████████▍| 13947/14725 [3:32:43<1:15:36,  5.83s/it]
 95%|█████████▍| 13948/14725 [3:32:49<1:15:15,  5.81s/it]
 95%|█████████▍| 13949/14725 [3:32:54<1:14:42,  5.78s/it]
 95%|█████████▍| 13950/14725 [3:33:00<1:14:21,  5.76s/it]
                                                         

 95%|█████████▍| 13950/14725 [3:33:00<1:14:21,  5.76s/it]
 95%|█████████▍| 13951/14725 [3:33:06<1:15:14,  5.83s/it]
 95%|█████████▍| 13952/14725 [3:33:12<1:15:59,  5.90
0: {'loss': 0.5312, 'grad_norm': 0.3826239118570705, 'learning_rate': 3e-06, 'epoch': 0.95}
0: s/it]
 95%|█████████▍| 13953/14725 [3:33:18<1:13:48,  5.74s/it]
 95%|█████████▍| 13954/14725 [3:33:24<1:15:44,  5.89s/it]
 95%|█████████▍| 13955/14725 [3:33:30<1:16:12,  5.94s/it]
 95%|█████████▍| 13956/14725 [3:33:35<1:13:48,  5.76s/it]
 95%|█████████▍| 13957/14725 [3:33:41<1:13:38,  5.75s/it]
 95%|█████████▍| 13958/14725 [3:33:47<1:13:54,  5.78s/it]
 95%|█████████▍| 13959/14725 [3:33:52<1:12:07,  5.65s/it]
 95%|█████████▍| 13960/14725 [3:33:58<1:10:42,  5.55s/it]
                                                         

 95%|█████████▍| 13960/14725 [3:33:58<1:10:42,  5.55s/it]
 95%|█████████▍| 13961/14725 [3:34:03<1:11:14,  5.59s/it]
 95%|█████████▍| 13962/14725 [3:34:09<1:12:07,  5.67s/it]
 95%|█████████▍| 13963/14725 [3:34:15<1:13:24,  5.78s/it]
 95%|██████
0: {'loss': 0.5184, 'grad_norm': 0.37972183508308577, 'learning_rate': 3e-06, 'epoch': 0.95}
0: ███▍| 13964/14725 [3:34:21<1:13:19,  5.78s/it]
 95%|█████████▍| 13965/14725 [3:34:26<1:11:24,  5.64s/it]
 95%|█████████▍| 13966/14725 [3:34:32<1:11:45,  5.67s/it]
 95%|█████████▍| 13967/14725 [3:34:37<1:10:40,  5.59s/it]
 95%|█████████▍| 13968/14725 [3:34:44<1:12:56,  5.78s/it]
 95%|█████████▍| 13969/14725 [3:34:49<1:12:26,  5.75s/it]
 95%|█████████▍| 13970/14725 [3:34:55<1:12:31,  5.76s/it]
                                                         

 95%|█████████▍| 13970/14725 [3:34:55<1:12:31,  5.76s/it]
 95%|█████████▍| 13971/14725 [3:35:00<1:11:10,  5.66s/it]
 95%|█████████▍| 13972/14725 [3:35:06<1:12:20,  5.76s/it]
 95%|█████████▍| 13973/14725 [3:35:12<1:12:26,  5.78s/it]
 95%|█████████▍| 13974/14725 [3:35:19<1:14:58,  5.99s/it]
 95%|█████████▍| 13975/14725 [3:
0: {'loss': 0.5167, 'grad_norm': 0.3832440794479363, 'learning_rate': 3e-06, 'epoch': 0.95}
0: 35:25<1:14:53,  5.99s/it]
 95%|█████████▍| 13976/14725 [3:35:31<1:14:35,  5.98s/it]
 95%|█████████▍| 13977/14725 [3:35:36<1:13:05,  5.86s/it]
 95%|█████████▍| 13978/14725 [3:35:42<1:10:51,  5.69s/it]
 95%|█████████▍| 13979/14725 [3:35:47<1:10:40,  5.68s/it]
 95%|█████████▍| 13980/14725 [3:35:53<1:10:10,  5.65s/it]
                                                         

 95%|█████████▍| 13980/14725 [3:35:53<1:10:10,  5.65s/it]
 95%|█████████▍| 13981/14725 [3:35:58<1:08:25,  5.52s/it]
 95%|█████████▍| 13982/14725 [3:36:04<1:08:34,  5.54s/it]
 95%|█████████▍| 13983/14725 [3:36:09<1:09:25,  5.61s/it]
 95%|█████████▍| 13984/14725 [3:36:16<1:11:35,  5.80s/it]
 95%|█████████▍| 13985/14725 [3:36:21<1:10:57,  5.75s/it]
 95%|█████████▍| 13986/14725 [3:36:27<1:10:19,  5.71s/it]
 95
0: {'loss': 0.526, 'grad_norm': 0.4007713777164006, 'learning_rate': 3e-06, 'epoch': 0.95}
0: %|█████████▍| 13987/14725 [3:36:33<1:09:51,  5.68s/it]
 95%|█████████▍| 13988/14725 [3:36:38<1:08:59,  5.62s/it]
 95%|█████████▌| 13989/14725 [3:36:44<1:10:57,  5.78s/it]
 95%|█████████▌| 13990/14725 [3:36:50<1:11:00,  5.80s/it]
                                                         

 95%|█████████▌| 13990/14725 [3:36:50<1:11:00,  5.80s/it]
 95%|█████████▌| 13991/14725 [3:36:56<1:10:46,  5.79s/it]
 95%|█████████▌| 13992/14725 [3:37:02<1:11:25,  5.85s/it]
 95%|█████████▌| 13993/14725 [3:37:08<1:11:23,  5.85s/it]
 95%|█████████▌| 13994/14725 [3:37:13<1:10:59,  5.83s/it]
 95%|█████████▌| 13995/14725 [3:37:20<1:13:09,  6.01s/it]
 95%|█████████▌| 13996/14725 [3:37:25<1:11:46,  5.91s/it]
 95%|█████████▌| 13997/14725 [3:37:31<1:09:59,  5.77s/it]
 95%|█████████
0: {'loss': 0.5213, 'grad_norm': 0.4072229406725453, 'learning_rate': 3e-06, 'epoch': 0.95}
0: ▌| 13998/14725 [3:37:36<1:09:09,  5.71s/it]
 95%|█████████▌| 13999/14725 [3:37:43<1:11:19,  5.89s/it]
 95%|█████████▌| 14000/14725 [3:37:48<1:09:14,  5.73s/it]
                                                         

 95%|█████████▌| 14000/14725 [3:37:48<1:09:14,  5.73s/it]
 95%|█████████▌| 14001/14725 [3:37:53<1:07:42,  5.61s/it]
 95%|█████████▌| 14002/14725 [3:38:00<1:09:48,  5.79s/it]
 95%|█████████▌| 14003/14725 [3:38:06<1:11:34,  5.95s/it]
 95%|█████████▌| 14004/14725 [3:38:12<1:11:19,  5.94s/it]
 95%|█████████▌| 14005/14725 [3:38:18<1:10:46,  5.90s/it]
 95%|█████████▌| 14006/14725 [3:38:24<1:10:39,  5.90s/it]
 95%|█████████▌| 14007/14725 [3:38:30<1:11:02,  5.94s/it]
 95%|█████████▌| 14008/14725 [3:38:35<1:09:00,  5.77s/it]
 95%|█████████▌| 14009/14725 [3:38:41<1:0
0: {'loss': 0.5092, 'grad_norm': 0.40185319364572136, 'learning_rate': 3e-06, 'epoch': 0.95}
0: 8:47,  5.77s/it]
 95%|█████████▌| 14010/14725 [3:38:47<1:09:38,  5.84s/it]
                                                         

 95%|█████████▌| 14010/14725 [3:38:47<1:09:38,  5.84s/it]
 95%|█████████▌| 14011/14725 [3:38:53<1:09:58,  5.88s/it]
 95%|█████████▌| 14012/14725 [3:38:59<1:09:32,  5.85s/it]
 95%|█████████▌| 14013/14725 [3:39:05<1:10:55,  5.98s/it]
 95%|█████████▌| 14014/14725 [3:39:10<1:08:31,  5.78s/it]
 95%|█████████▌| 14015/14725 [3:39:16<1:09:59,  5.91s/it]
 95%|█████████▌| 14016/14725 [3:39:23<1:10:41,  5.98s/it]
 95%|█████████▌| 14017/14725 [3:39:28<1:09:45,  5.91s/it]
 95%|█████████▌| 14018/14725 [3:39:34<1:07:43,  5.75s/it]
 95%|█████████▌| 14019/14725 [3:39:39<1:07:37,  5.75s/it]
 95%|█████████▌| 14020/14725 [3:39:45<1:07:05,  5.71s/it]
            
0: {'loss': 0.5365, 'grad_norm': 0.35219405076947913, 'learning_rate': 3e-06, 'epoch': 0.95}
0: {'loss': 0.5318, 'grad_norm': 0.40571788818387033, 'learning_rate': 3e-06, 'epoch': 0.95}
0:                                              

 95%|█████████▌| 14020/14725 [3:39:45<1:07:05,  5.71s/it]
 95%|█████████▌| 14021/14725 [3:39:51<1:07:45,  5.77s/it]
 95%|█████████▌| 14022/14725 [3:39:56<1:05:39,  5.60s/it]
 95%|█████████▌| 14023/14725 [3:40:01<1:04:34,  5.52s/it]
 95%|█████████▌| 14024/14725 [3:40:07<1:05:24,  5.60s/it]
 95%|█████████▌| 14025/14725 [3:40:13<1:05:47,  5.64s/it]
 95%|█████████▌| 14026/14725 [3:40:19<1:07:50,  5.82s/it]
 95%|█████████▌| 14027/14725 [3:40:25<1:07:22,  5.79s/it]
 95%|█████████▌| 14028/14725 [3:40:31<1:08:09,  5.87s/it]
 95%|█████████▌| 14029/14725 [3:40:37<1:07:55,  5.86s/it]
 95%|█████████▌| 14030/14725 [3:40:43<1:07:25,  5.82s/it]
                                                         

 95%|█████████▌| 14030/14725 [3:40:43<1:
0: {'loss': 0.5374, 'grad_norm': 0.44194979471718276, 'learning_rate': 3e-06, 'epoch': 0.95}
0: 07:25,  5.82s/it]
 95%|█████████▌| 14031/14725 [3:40:48<1:05:14,  5.64s/it]
 95%|█████████▌| 14032/14725 [3:40:54<1:06:47,  5.78s/it]
 95%|█████████▌| 14033/14725 [3:41:00<1:08:10,  5.91s/it]
 95%|█████████▌| 14034/14725 [3:41:06<1:08:03,  5.91s/it]
 95%|█████████▌| 14035/14725 [3:41:12<1:07:28,  5.87s/it]
 95%|█████████▌| 14036/14725 [3:41:18<1:07:38,  5.89s/it]
 95%|█████████▌| 14037/14725 [3:41:24<1:07:07,  5.85s/it]
 95%|█████████▌| 14038/14725 [3:41:30<1:08:22,  5.97s/it]
 95%|█████████▌| 14039/14725 [3:41:35<1:06:55,  5.85s/it]
 95%|█████████▌| 14040/14725 [3:41:41<1:06:30,  5.83s/it]
                                                         

 95%|█████████▌| 14040/14725 [3:41:41<1:06:30,  5.83s/it]
 95%|█████████▌| 14041/14725 [3:41:47<1:06:48,  5.86s/it]
 95%|██
0: {'loss': 0.5284, 'grad_norm': 0.36403830437392, 'learning_rate': 3e-06, 'epoch': 0.95}
0: ███████▌| 14042/14725 [3:41:53<1:07:50,  5.96s/it]
 95%|█████████▌| 14043/14725 [3:41:59<1:07:46,  5.96s/it]
 95%|█████████▌| 14044/14725 [3:42:05<1:06:57,  5.90s/it]
 95%|█████████▌| 14045/14725 [3:42:11<1:05:52,  5.81s/it]
 95%|█████████▌| 14046/14725 [3:42:16<1:04:59,  5.74s/it]
 95%|█████████▌| 14047/14725 [3:42:22<1:04:53,  5.74s/it]
 95%|█████████▌| 14048/14725 [3:42:28<1:06:57,  5.93s/it]
 95%|█████████▌| 14049/14725 [3:42:33<1:04:06,  5.69s/it]
 95%|█████████▌| 14050/14725 [3:42:39<1:04:51,  5.76s/it]
                                                         

 95%|█████████▌| 14050/14725 [3:42:39<1:04:51,  5.76s/it]
 95%|█████████▌| 14051/14725 [3:42:45<1:04:41,  5.76s/it]
 95%|█████████▌| 14052/14725 [3:42:51<1:05:38,  5.85s/it]
 95%|█████████▌| 140
0: {'loss': 0.5293, 'grad_norm': 0.4022892472928177, 'learning_rate': 3e-06, 'epoch': 0.95}
0: 53/14725 [3:42:57<1:06:13,  5.91s/it]
 95%|█████████▌| 14054/14725 [3:43:03<1:06:53,  5.98s/it]
 95%|█████████▌| 14055/14725 [3:43:09<1:06:29,  5.95s/it]
 95%|█████████▌| 14056/14725 [3:43:15<1:06:55,  6.00s/it]
 95%|█████████▌| 14057/14725 [3:43:21<1:05:45,  5.91s/it]
 95%|█████████▌| 14058/14725 [3:43:27<1:06:57,  6.02s/it]
 95%|█████████▌| 14059/14725 [3:43:33<1:05:26,  5.90s/it]
 95%|█████████▌| 14060/14725 [3:43:39<1:06:40,  6.02s/it]
                                                         

 95%|█████████▌| 14060/14725 [3:43:39<1:06:40,  6.02s/it]
 95%|█████████▌| 14061/14725 [3:43:45<1:04:56,  5.87s/it]
 95%|█████████▌| 14062/14725 [3:43:51<1:04:57,  5.88s/it]
 96%|█████████▌| 14063/14725 [3:43:57<1:05:43,  5.96s/it]
 96%|█████████▌| 14064/14725 [3:44:03<1:05:53,  5
0: {'loss': 0.5261, 'grad_norm': 0.3944231646838829, 'learning_rate': 3e-06, 'epoch': 0.96}
0: .98s/it]
 96%|█████████▌| 14065/14725 [3:44:09<1:05:20,  5.94s/it]
 96%|█████████▌| 14066/14725 [3:44:15<1:06:09,  6.02s/it]
 96%|█████████▌| 14067/14725 [3:44:21<1:06:05,  6.03s/it]
 96%|█████████▌| 14068/14725 [3:44:26<1:04:21,  5.88s/it]
 96%|█████████▌| 14069/14725 [3:44:32<1:03:59,  5.85s/it]
 96%|█████████▌| 14070/14725 [3:44:38<1:03:13,  5.79s/it]
                                                         

 96%|█████████▌| 14070/14725 [3:44:38<1:03:13,  5.79s/it]
 96%|█████████▌| 14071/14725 [3:44:44<1:04:26,  5.91s/it]
 96%|█████████▌| 14072/14725 [3:44:49<1:01:37,  5.66s/it]
 96%|█████████▌| 14073/14725 [3:44:55<1:01:28,  5.66s/it]
 96%|█████████▌| 14074/14725 [3:45:00<1:00:19,  5.56s/it]
 96%|█████████▌| 14075/14725 [3:45:06<59:53,  5.53s/it]  
 96%|█████
0: {'loss': 0.5241, 'grad_norm': 0.4117926702704799, 'learning_rate': 3e-06, 'epoch': 0.96}
0: ████▌| 14076/14725 [3:45:11<59:44,  5.52s/it]
 96%|█████████▌| 14077/14725 [3:45:17<1:00:13,  5.58s/it]
 96%|█████████▌| 14078/14725 [3:45:23<1:00:54,  5.65s/it]
 96%|█████████▌| 14079/14725 [3:45:29<1:02:58,  5.85s/it]
 96%|█████████▌| 14080/14725 [3:45:35<1:02:00,  5.77s/it]
                                                         

 96%|█████████▌| 14080/14725 [3:45:35<1:02:00,  5.77s/it]
 96%|█████████▌| 14081/14725 [3:45:41<1:02:38,  5.84s/it]
 96%|█████████▌| 14082/14725 [3:45:46<1:01:31,  5.74s/it]
 96%|█████████▌| 14083/14725 [3:45:51<1:00:26,  5.65s/it]
 96%|█████████▌| 14084/14725 [3:45:57<59:07,  5.53s/it]  
 96%|█████████▌| 14085/14725 [3:46:03<1:00:38,  5.69s/it]
 96%|█████████▌| 14086/14725 [3:46:08<58:44,  5.52s/it]  
 96%|█████████▌| 14087/14725 [3
0: {'loss': 0.5169, 'grad_norm': 0.38272033405002287, 'learning_rate': 3e-06, 'epoch': 0.96}
0: :46:14<59:33,  5.60s/it]
 96%|█████████▌| 14088/14725 [3:46:20<1:00:12,  5.67s/it]
 96%|█████████▌| 14089/14725 [3:46:25<58:57,  5.56s/it]  
 96%|█████████▌| 14090/14725 [3:46:31<1:00:22,  5.71s/it]
                                                         

 96%|█████████▌| 14090/14725 [3:46:31<1:00:22,  5.71s/it]
 96%|█████████▌| 14091/14725 [3:46:37<1:00:46,  5.75s/it]
 96%|█████████▌| 14092/14725 [3:46:43<1:01:08,  5.80s/it]
 96%|█████████▌| 14093/14725 [3:46:49<1:01:40,  5.86s/it]
 96%|█████████▌| 14094/14725 [3:46:55<1:02:41,  5.96s/it]
 96%|█████████▌| 14095/14725 [3:47:01<1:01:45,  5.88s/it]
 96%|█████████▌| 14096/14725 [3:47:07<1:03:22,  6.04s/it]
 96%|█████████▌| 14097/14725 [3:47:13<1:03:09,  6.03s/it]
 96%|█████████▌| 14098/14725 [3:47:19<1:02:26,  5.98s/it]
 96%
0: {'loss': 0.5245, 'grad_norm': 0.3991875475095195, 'learning_rate': 3e-06, 'epoch': 0.96}
0: |█████████▌| 14099/14725 [3:47:24<1:00:04,  5.76s/it]
 96%|█████████▌| 14100/14725 [3:47:30<1:00:04,  5.77s/it]
                                                         

 96%|█████████▌| 14100/14725 [3:47:30<1:00:04,  5.77s/it]
 96%|█████████▌| 14101/14725 [3:47:36<1:00:48,  5.85s/it]
 96%|█████████▌| 14102/14725 [3:47:41<58:51,  5.67s/it]  
 96%|█████████▌| 14103/14725 [3:47:47<58:44,  5.67s/it]
 96%|█████████▌| 14104/14725 [3:47:53<1:00:09,  5.81s/it]
 96%|█████████▌| 14105/14725 [3:47:59<1:01:19,  5.94s/it]
 96%|█████████▌| 14106/14725 [3:48:05<1:01:03,  5.92s/it]
 96%|█████████▌| 14107/14725 [3:48:11<1:01:20,  5.96s/it]
 96%|█████████▌| 14108/14725 [3:48:17<1:01:43,  6.00s/it]
 96%|█████████▌| 14109/14725 [3:48:23<59:41,  5.81s/it]  
 96%|█████████▌
0: {'loss': 0.5299, 'grad_norm': 0.3945738384188514, 'learning_rate': 3e-06, 'epoch': 0.96}
0: {'loss': 0.5157, 'grad_norm': 0.37808022897140253, 'learning_rate': 3e-06, 'epoch': 0.96}
0: | 14110/14725 [3:48:29<1:00:34,  5.91s/it]
                                                         

 96%|█████████▌| 14110/14725 [3:48:29<1:00:34,  5.91s/it]
 96%|█████████▌| 14111/14725 [3:48:35<1:01:18,  5.99s/it]
 96%|█████████▌| 14112/14725 [3:48:40<59:46,  5.85s/it]  
 96%|█████████▌| 14113/14725 [3:48:46<1:00:02,  5.89s/it]
 96%|█████████▌| 14114/14725 [3:48:52<59:56,  5.89s/it]  
 96%|█████████▌| 14115/14725 [3:48:58<58:43,  5.78s/it]
 96%|█████████▌| 14116/14725 [3:49:04<59:41,  5.88s/it]
 96%|█████████▌| 14117/14725 [3:49:10<1:00:44,  5.99s/it]
 96%|█████████▌| 14118/14725 [3:49:16<1:00:45,  6.01s/it]
 96%|█████████▌| 14119/14725 [3:49:22<59:47,  5.92s/it]  
 96%|█████████▌| 14120/14725 [3:49:28<59:21,  5.89s/it]
                                                       

 96%|██�
0: {'loss': 0.5234, 'grad_norm': 0.3601885534570663, 'learning_rate': 3e-06, 'epoch': 0.96}
0: �██████▌| 14120/14725 [3:49:28<59:21,  5.89s/it]
 96%|█████████▌| 14121/14725 [3:49:33<57:20,  5.70s/it]
 96%|█████████▌| 14122/14725 [3:49:38<56:09,  5.59s/it]
 96%|█████████▌| 14123/14725 [3:49:44<57:33,  5.74s/it]
 96%|█████████▌| 14124/14725 [3:49:50<56:42,  5.66s/it]
 96%|█████████▌| 14125/14725 [3:49:55<55:18,  5.53s/it]
 96%|█████████▌| 14126/14725 [3:50:01<54:57,  5.50s/it]
 96%|█████████▌| 14127/14725 [3:50:07<56:57,  5.71s/it]
 96%|█████████▌| 14128/14725 [3:50:12<56:02,  5.63s/it]
 96%|█████████▌| 14129/14725 [3:50:18<56:17,  5.67s/it]
 96%|█████████▌| 14130/14725 [3:50:24<55:59,  5.65s/it]
                                                       

 96%|█████████▌| 14130/14725 [3:50:24<55:59,  5.65s/it]
 96%|█████████▌| 14131/14725 [3:50:29<56:42,  5.
0: {'loss': 0.5191, 'grad_norm': 0.35478011902494316, 'learning_rate': 3e-06, 'epoch': 0.96}
0: 73s/it]
 96%|█████████▌| 14132/14725 [3:50:35<57:12,  5.79s/it]
 96%|█████████▌| 14133/14725 [3:50:41<56:53,  5.77s/it]
 96%|█████████▌| 14134/14725 [3:50:47<56:01,  5.69s/it]
 96%|█████████▌| 14135/14725 [3:50:52<56:06,  5.71s/it]
 96%|█████████▌| 14136/14725 [3:50:58<57:05,  5.82s/it]
 96%|█████████▌| 14137/14725 [3:51:04<56:56,  5.81s/it]
 96%|█████████▌| 14138/14725 [3:51:10<56:58,  5.82s/it]
 96%|█████████▌| 14139/14725 [3:51:16<56:48,  5.82s/it]
 96%|█████████▌| 14140/14725 [3:51:21<55:46,  5.72s/it]
                                                       

 96%|█████████▌| 14140/14725 [3:51:21<55:46,  5.72s/it]
 96%|█████████▌| 14141/14725 [3:51:27<55:11,  5.67s/it]
 96%|█████████▌| 14142/14725 [3:51:32<54:34,  5.62s/it]
 96%|█████████▌| 14143/1472
0: {'loss': 0.5289, 'grad_norm': 0.3931549474576158, 'learning_rate': 3e-06, 'epoch': 0.96}
0: 5 [3:51:39<55:52,  5.76s/it]
 96%|█████████▌| 14144/14725 [3:51:44<55:38,  5.75s/it]
 96%|█████████▌| 14145/14725 [3:51:49<53:31,  5.54s/it]
 96%|█████████▌| 14146/14725 [3:51:55<54:06,  5.61s/it]
 96%|█████████▌| 14147/14725 [3:52:01<53:42,  5.58s/it]
 96%|█████████▌| 14148/14725 [3:52:06<54:32,  5.67s/it]
 96%|█████████▌| 14149/14725 [3:52:12<55:28,  5.78s/it]
 96%|█████████▌| 14150/14725 [3:52:18<53:43,  5.61s/it]
                                                       

 96%|█████████▌| 14150/14725 [3:52:18<53:43,  5.61s/it]
 96%|█████████▌| 14151/14725 [3:52:23<53:19,  5.57s/it]
 96%|█████████▌| 14152/14725 [3:52:29<54:25,  5.70s/it]
 96%|█████████▌| 14153/14725 [3:52:35<54:46,  5.75s/it]
 96%|█████████▌| 14154/14725 [3:52:40<53:18,  5.60s/it]
 96%|███████
0: {'loss': 0.518, 'grad_norm': 0.3883688849121606, 'learning_rate': 3e-06, 'epoch': 0.96}
0: ██▌| 14155/14725 [3:52:47<55:05,  5.80s/it]
 96%|█████████▌| 14156/14725 [3:52:52<54:06,  5.71s/it]
 96%|█████████▌| 14157/14725 [3:52:58<54:27,  5.75s/it]
 96%|█████████▌| 14158/14725 [3:53:04<55:40,  5.89s/it]
 96%|█████████▌| 14159/14725 [3:53:10<55:44,  5.91s/it]
 96%|█████████▌| 14160/14725 [3:53:16<56:40,  6.02s/it]
                                                       

 96%|█████████▌| 14160/14725 [3:53:16<56:40,  6.02s/it]
 96%|█████████▌| 14161/14725 [3:53:22<56:52,  6.05s/it]
 96%|█████████▌| 14162/14725 [3:53:28<56:29,  6.02s/it]
 96%|█████████▌| 14163/14725 [3:53:34<55:42,  5.95s/it]
 96%|█████████▌| 14164/14725 [3:53:40<56:32,  6.05s/it]
 96%|█████████▌| 14165/14725 [3:53:46<54:35,  5.85s/it]
 96%|█████████▌| 14166/14725 [3:53:52<54:22,  5.84s/it]
 96%|
0: {'loss': 0.5299, 'grad_norm': 0.38104397440433857, 'learning_rate': 3e-06, 'epoch': 0.96}
0: █████████▌| 14167/14725 [3:53:57<53:31,  5.76s/it]
 96%|█████████▌| 14168/14725 [3:54:03<54:41,  5.89s/it]
 96%|█████████▌| 14169/14725 [3:54:09<54:21,  5.87s/it]
 96%|█████████▌| 14170/14725 [3:54:15<52:59,  5.73s/it]
                                                       

 96%|█████████▌| 14170/14725 [3:54:15<52:59,  5.73s/it]
 96%|█████████▌| 14171/14725 [3:54:21<53:54,  5.84s/it]
 96%|█████████▌| 14172/14725 [3:54:26<53:04,  5.76s/it]
 96%|█████████▋| 14173/14725 [3:54:33<54:16,  5.90s/it]
 96%|█████████▋| 14174/14725 [3:54:39<54:19,  5.92s/it]
 96%|█████████▋| 14175/14725 [3:54:45<54:41,  5.97s/it]
 96%|█████████▋| 14176/14725 [3:54:50<53:21,  5.83s/it]
 96%|█████████▋| 14177/14725 [3:54:56<54:18,  5.95s/it]
 96%|█████████▋| 14178/14725 [3:55:02<53
0: {'loss': 0.5316, 'grad_norm': 0.3908815238306709, 'learning_rate': 3e-06, 'epoch': 0.96}
0: :51,  5.91s/it]
 96%|█████████▋| 14179/14725 [3:55:08<52:47,  5.80s/it]
 96%|█████████▋| 14180/14725 [3:55:14<53:13,  5.86s/it]
                                                       

 96%|█████████▋| 14180/14725 [3:55:14<53:13,  5.86s/it]
 96%|█████████▋| 14181/14725 [3:55:20<53:31,  5.90s/it]
 96%|█████████▋| 14182/14725 [3:55:26<54:08,  5.98s/it]
 96%|█████████▋| 14183/14725 [3:55:32<53:31,  5.93s/it]
 96%|█████████▋| 14184/14725 [3:55:37<52:49,  5.86s/it]
 96%|█████████▋| 14185/14725 [3:55:44<53:41,  5.97s/it]
 96%|█████████▋| 14186/14725 [3:55:50<54:40,  6.09s/it]
 96%|█████████▋| 14187/14725 [3:55:56<54:01,  6.02s/it]
 96%|█████████▋| 14188/14725 [3:56:02<53:19,  5.96s/it]
 96%|█████████▋| 14189/14725 [3:56:07<51:52,  5.81s/it]
 96%|█████████▋| 14
0: {'loss': 0.5203, 'grad_norm': 0.37624629459841086, 'learning_rate': 3e-06, 'epoch': 0.96}
0: {'loss': 0.5269, 'grad_norm': 0.39154154006448916, 'learning_rate': 3e-06, 'epoch': 0.96}
0: 190/14725 [3:56:13<52:44,  5.92s/it]
                                                       

 96%|█████████▋| 14190/14725 [3:56:13<52:44,  5.92s/it]
 96%|█████████▋| 14191/14725 [3:56:19<52:20,  5.88s/it]
 96%|█████████▋| 14192/14725 [3:56:25<52:50,  5.95s/it]
 96%|█████████▋| 14193/14725 [3:56:31<52:06,  5.88s/it]
 96%|█████████▋| 14194/14725 [3:56:36<50:31,  5.71s/it]
 96%|█████████▋| 14195/14725 [3:56:42<51:35,  5.84s/it]
 96%|█████████▋| 14196/14725 [3:56:48<50:55,  5.78s/it]
 96%|█████████▋| 14197/14725 [3:56:53<49:48,  5.66s/it]
 96%|█████████▋| 14198/14725 [3:56:59<49:23,  5.62s/it]
 96%|█████████▋| 14199/14725 [3:57:04<49:12,  5.61s/it]
 96%|█████████▋| 14200/14725 [3:57:11<50:18,  5.75s/it]
                                                       

 96%|█████████▋| 
0: {'loss': 0.5237, 'grad_norm': 0.3946055746830924, 'learning_rate': 3e-06, 'epoch': 0.97}
0: 14200/14725 [3:57:11<50:18,  5.75s/it]
 96%|█████████▋| 14201/14725 [3:57:16<49:44,  5.69s/it]
 96%|█████████▋| 14202/14725 [3:57:22<51:12,  5.87s/it]
 96%|█████████▋| 14203/14725 [3:57:29<51:52,  5.96s/it]
 96%|█████████▋| 14204/14725 [3:57:35<53:55,  6.21s/it]
 96%|█████████▋| 14205/14725 [3:57:41<51:15,  5.91s/it]
 96%|█████████▋| 14206/14725 [3:57:46<49:52,  5.77s/it]
 96%|█████████▋| 14207/14725 [3:57:52<49:34,  5.74s/it]
 96%|█████████▋| 14208/14725 [3:57:57<49:22,  5.73s/it]
 96%|█████████▋| 14209/14725 [3:58:03<50:01,  5.82s/it]
 97%|█████████▋| 14210/14725 [3:58:09<49:33,  5.77s/it]
                                                       

 97%|█████████▋| 14210/14725 [3:58:09<49:33,  5.77s/it]
 97%|█████████▋| 14211/14725 [3:58:15<50:13,  5.86s/it]
 97%|███�
0: {'loss': 0.5224, 'grad_norm': 0.3891186337105166, 'learning_rate': 3e-06, 'epoch': 0.97}
0: �█████▋| 14212/14725 [3:58:21<49:50,  5.83s/it]
 97%|█████████▋| 14213/14725 [3:58:26<48:54,  5.73s/it]
 97%|█████████▋| 14214/14725 [3:58:32<48:59,  5.75s/it]
 97%|█████████▋| 14215/14725 [3:58:39<50:22,  5.93s/it]
 97%|█████████▋| 14216/14725 [3:58:45<51:10,  6.03s/it]
 97%|█████████▋| 14217/14725 [3:58:51<50:24,  5.95s/it]
 97%|█████████▋| 14218/14725 [3:58:56<49:22,  5.84s/it]
 97%|█████████▋| 14219/14725 [3:59:02<48:09,  5.71s/it]
 97%|█████████▋| 14220/14725 [3:59:07<48:31,  5.76s/it]
                                                       

 97%|█████████▋| 14220/14725 [3:59:07<48:31,  5.76s/it]
 97%|█████████▋| 14221/14725 [3:59:13<47:48,  5.69s/it]
 97%|█████████▋| 14222/14725 [3:59:19<47:53,  5.71s/it]
 97%|█████████▋| 14223/14725 [3:59:25<48:45,  5.83s
0: {'loss': 0.5308, 'grad_norm': 0.3838682001783511, 'learning_rate': 3e-06, 'epoch': 0.97}
0: /it]
 97%|█████████▋| 14224/14725 [3:59:30<48:04,  5.76s/it]
 97%|█████████▋| 14225/14725 [3:59:36<47:53,  5.75s/it]
 97%|█████████▋| 14226/14725 [3:59:42<47:28,  5.71s/it]
 97%|█████████▋| 14227/14725 [3:59:48<48:40,  5.86s/it]
 97%|█████████▋| 14228/14725 [3:59:54<48:23,  5.84s/it]
 97%|█████████▋| 14229/14725 [4:00:00<47:59,  5.80s/it]
 97%|█████████▋| 14230/14725 [4:00:05<48:12,  5.84s/it]
                                                       

 97%|█████████▋| 14230/14725 [4:00:05<48:12,  5.84s/it]
 97%|█████████▋| 14231/14725 [4:00:11<48:31,  5.89s/it]
 97%|█████████▋| 14232/14725 [4:00:17<48:19,  5.88s/it]
 97%|█████████▋| 14233/14725 [4:00:24<49:01,  5.98s/it]
 97%|█████████▋| 14234/14725 [4:00:29<47:55,  5.86s/it]
 97%|█████████▋| 14235/14725 [
0: {'loss': 0.5274, 'grad_norm': 0.3667241483579689, 'learning_rate': 3e-06, 'epoch': 0.97}
0: 4:00:35<48:52,  5.98s/it]
 97%|█████████▋| 14236/14725 [4:00:42<49:15,  6.04s/it]
 97%|█████████▋| 14237/14725 [4:00:47<48:31,  5.97s/it]
 97%|█████████▋| 14238/14725 [4:00:53<47:46,  5.89s/it]
 97%|█████████▋| 14239/14725 [4:00:59<47:11,  5.83s/it]
 97%|█████████▋| 14240/14725 [4:01:04<46:52,  5.80s/it]
                                                       

 97%|█████████▋| 14240/14725 [4:01:04<46:52,  5.80s/it]
 97%|█████████▋| 14241/14725 [4:01:10<46:58,  5.82s/it]
 97%|█████████▋| 14242/14725 [4:01:16<47:18,  5.88s/it]
 97%|█████████▋| 14243/14725 [4:01:22<45:56,  5.72s/it]
 97%|█████████▋| 14244/14725 [4:01:27<44:27,  5.54s/it]
 97%|█████████▋| 14245/14725 [4:01:33<46:13,  5.78s/it]
 97%|█████████▋| 14246/14725 [4:01:39<46:15,  5.79s/it]
 97%|████████
0: {'loss': 0.5092, 'grad_norm': 0.3709771893377644, 'learning_rate': 3e-06, 'epoch': 0.97}
0: █▋| 14247/14725 [4:01:45<45:57,  5.77s/it]
 97%|█████████▋| 14248/14725 [4:01:50<44:48,  5.64s/it]
 97%|█████████▋| 14249/14725 [4:01:55<43:37,  5.50s/it]
 97%|█████████▋| 14250/14725 [4:02:01<45:20,  5.73s/it]
                                                       

 97%|█████████▋| 14250/14725 [4:02:01<45:20,  5.73s/it]
 97%|█████████▋| 14251/14725 [4:02:07<44:28,  5.63s/it]
 97%|█████████▋| 14252/14725 [4:02:12<43:52,  5.56s/it]
 97%|█████████▋| 14253/14725 [4:02:19<45:22,  5.77s/it]
 97%|█████████▋| 14254/14725 [4:02:25<46:28,  5.92s/it]
 97%|█████████▋| 14255/14725 [4:02:30<45:13,  5.77s/it]
 97%|█████████▋| 14256/14725 [4:02:36<46:12,  5.91s/it]
 97%|█████████▋| 14257/14725 [4:02:43<46:30,  5.96s/it]
 97%|█████████▋| 14258/14725 [4:02:49<46:30,  5.98s/it]
 97%|█
0: {'loss': 0.534, 'grad_norm': 0.388326796813921, 'learning_rate': 3e-06, 'epoch': 0.97}
0: ████████▋| 14259/14725 [4:02:54<46:04,  5.93s/it]
 97%|█████████▋| 14260/14725 [4:03:00<46:21,  5.98s/it]
                                                       

 97%|█████████▋| 14260/14725 [4:03:00<46:21,  5.98s/it]
 97%|█████████▋| 14261/14725 [4:03:06<45:48,  5.92s/it]
 97%|█████████▋| 14262/14725 [4:03:11<44:07,  5.72s/it]
 97%|█████████▋| 14263/14725 [4:03:17<44:07,  5.73s/it]
 97%|█████████▋| 14264/14725 [4:03:23<44:46,  5.83s/it]
 97%|█████████▋| 14265/14725 [4:03:29<44:06,  5.75s/it]
 97%|█████████▋| 14266/14725 [4:03:34<43:39,  5.71s/it]
 97%|█████████▋| 14267/14725 [4:03:40<43:05,  5.64s/it]
 97%|█████████▋| 14268/14725 [4:03:46<43:02,  5.65s/it]
 97%|█████████▋| 14269/14725 [4:03:52<43:32,  5.73s/it]
 97%|█████████▋| 14270/14725 [4:03:58<44:18
0: {'loss': 0.5173, 'grad_norm': 0.3793029795742923, 'learning_rate': 3e-06, 'epoch': 0.97}
0: {'loss': 0.5424, 'grad_norm': 0.6836643094993814, 'learning_rate': 3e-06, 'epoch': 0.97}
0: ,  5.84s/it]
                                                       

 97%|█████████▋| 14270/14725 [4:03:58<44:18,  5.84s/it]
 97%|█████████▋| 14271/14725 [4:04:04<45:21,  6.00s/it]
 97%|█████████▋| 14272/14725 [4:04:09<44:04,  5.84s/it]
 97%|█████████▋| 14273/14725 [4:04:16<44:59,  5.97s/it]
 97%|█████████▋| 14274/14725 [4:04:22<45:19,  6.03s/it]
 97%|█████████▋| 14275/14725 [4:04:28<45:43,  6.10s/it]
 97%|█████████▋| 14276/14725 [4:04:34<44:52,  6.00s/it]
 97%|█████████▋| 14277/14725 [4:04:40<45:36,  6.11s/it]
 97%|█████████▋| 14278/14725 [4:04:46<45:03,  6.05s/it]
 97%|█████████▋| 14279/14725 [4:04:52<44:56,  6.05s/it]
 97%|█████████▋| 14280/14725 [4:04:59<45:28,  6.13s/it]
                                                       

 97%|█████████▋| 14280/14725 [4:04:59<45:
0: {'loss': 0.5275, 'grad_norm': 0.40480483366368863, 'learning_rate': 3e-06, 'epoch': 0.97}
0: 28,  6.13s/it]
 97%|█████████▋| 14281/14725 [4:05:04<44:46,  6.05s/it]
 97%|█████████▋| 14282/14725 [4:05:10<43:09,  5.84s/it]
 97%|█████████▋| 14283/14725 [4:05:16<44:01,  5.98s/it]
 97%|█████████▋| 14284/14725 [4:05:22<43:01,  5.85s/it]
 97%|█████████▋| 14285/14725 [4:05:27<42:40,  5.82s/it]
 97%|█████████▋| 14286/14725 [4:05:33<41:58,  5.74s/it]
 97%|█████████▋| 14287/14725 [4:05:38<40:58,  5.61s/it]
 97%|█████████▋| 14288/14725 [4:05:44<40:55,  5.62s/it]
 97%|█████████▋| 14289/14725 [4:05:50<40:45,  5.61s/it]
 97%|█████████▋| 14290/14725 [4:05:55<40:23,  5.57s/it]
                                                       

 97%|█████████▋| 14290/14725 [4:05:55<40:23,  5.57s/it]
 97%|█████████▋| 14291/14725 [4:06:01<40:27,  5.59s/it]
 97%|█████████▋| 142
0: {'loss': 0.5213, 'grad_norm': 0.368880102063277, 'learning_rate': 3e-06, 'epoch': 0.97}
0: 92/14725 [4:06:06<40:19,  5.59s/it]
 97%|█████████▋| 14293/14725 [4:06:12<40:33,  5.63s/it]
 97%|█████████▋| 14294/14725 [4:06:17<39:59,  5.57s/it]
 97%|█████████▋| 14295/14725 [4:06:23<40:50,  5.70s/it]
 97%|█████████▋| 14296/14725 [4:06:29<40:49,  5.71s/it]
 97%|█████████▋| 14297/14725 [4:06:35<41:10,  5.77s/it]
 97%|█████████▋| 14298/14725 [4:06:41<40:36,  5.71s/it]
 97%|█████████▋| 14299/14725 [4:06:46<40:31,  5.71s/it]
 97%|█████████▋| 14300/14725 [4:06:52<39:52,  5.63s/it]
                                                       

 97%|█████████▋| 14300/14725 [4:06:52<39:52,  5.63s/it]
 97%|█████████▋| 14301/14725 [4:06:57<39:58,  5.66s/it]
 97%|█████████▋| 14302/14725 [4:07:03<40:22,  5.73s/it]
 97%|█████████▋| 14303/14725 [4:07:09<39:53,  5.67s/it]
 97%|████�
0: {'loss': 0.5207, 'grad_norm': 0.39124427454758925, 'learning_rate': 3e-06, 'epoch': 0.97}
0: �████▋| 14304/14725 [4:07:15<40:16,  5.74s/it]
 97%|█████████▋| 14305/14725 [4:07:20<39:48,  5.69s/it]
 97%|█████████▋| 14306/14725 [4:07:26<40:18,  5.77s/it]
 97%|█████████▋| 14307/14725 [4:07:31<38:58,  5.59s/it]
 97%|█████████▋| 14308/14725 [4:07:38<40:29,  5.83s/it]
 97%|█████████▋| 14309/14725 [4:07:43<39:01,  5.63s/it]
 97%|█████████▋| 14310/14725 [4:07:49<40:20,  5.83s/it]
                                                       

 97%|█████████▋| 14310/14725 [4:07:49<40:20,  5.83s/it]
 97%|█████████▋| 14311/14725 [4:07:55<39:56,  5.79s/it]
 97%|█████████▋| 14312/14725 [4:08:01<39:41,  5.77s/it]
 97%|█████████▋| 14313/14725 [4:08:06<38:54,  5.67s/it]
 97%|█████████▋| 14314/14725 [4:08:12<39:38,  5.79s/it]
 97%|█████████▋| 14315/14725 [4:08:18<40:02,  5.86s/it
0: {'loss': 0.5239, 'grad_norm': 0.37411684275854434, 'learning_rate': 3e-06, 'epoch': 0.97}
0: ]
 97%|█████████▋| 14316/14725 [4:08:25<40:55,  6.00s/it]
 97%|█████████▋| 14317/14725 [4:08:30<39:43,  5.84s/it]
 97%|█████████▋| 14318/14725 [4:08:35<38:44,  5.71s/it]
 97%|█████████▋| 14319/14725 [4:08:41<38:49,  5.74s/it]
 97%|█████████▋| 14320/14725 [4:08:47<39:04,  5.79s/it]
                                                       

 97%|█████████▋| 14320/14725 [4:08:47<39:04,  5.79s/it]
 97%|█████████▋| 14321/14725 [4:08:52<37:33,  5.58s/it]
 97%|█████████▋| 14322/14725 [4:08:58<38:25,  5.72s/it]
 97%|█████████▋| 14323/14725 [4:09:05<39:52,  5.95s/it]
 97%|█████████▋| 14324/14725 [4:09:11<39:52,  5.97s/it]
 97%|█████████▋| 14325/14725 [4:09:16<38:21,  5.75s/it]
 97%|█████████▋| 14326/14725 [4:09:22<38:06,  5.73s/it]
 97%|█████████▋| 14327/14725 [4:0
0: {'loss': 0.524, 'grad_norm': 0.39050555051969876, 'learning_rate': 3e-06, 'epoch': 0.97}
0: 9:27<37:06,  5.59s/it]
 97%|█████████▋| 14328/14725 [4:09:32<36:39,  5.54s/it]
 97%|█████████▋| 14329/14725 [4:09:38<36:15,  5.49s/it]
 97%|█████████▋| 14330/14725 [4:09:43<36:25,  5.53s/it]
                                                       

 97%|█████████▋| 14330/14725 [4:09:43<36:25,  5.53s/it]
 97%|█████████▋| 14331/14725 [4:09:49<36:43,  5.59s/it]
 97%|█████████▋| 14332/14725 [4:09:55<38:00,  5.80s/it]
 97%|█████████▋| 14333/14725 [4:10:01<37:25,  5.73s/it]
 97%|█████████▋| 14334/14725 [4:10:06<36:45,  5.64s/it]
 97%|█████████▋| 14335/14725 [4:10:12<37:01,  5.70s/it]
 97%|█████████▋| 14336/14725 [4:10:18<35:58,  5.55s/it]
 97%|█████████▋| 14337/14725 [4:10:24<37:47,  5.84s/it]
 97%|█████████▋| 14338/14725 [4:10:30<38:30,  5.97s/it]
 97%|█████████
0: {'loss': 0.5193, 'grad_norm': 0.3852765416846069, 'learning_rate': 3e-06, 'epoch': 0.97}
0: ▋| 14339/14725 [4:10:36<38:04,  5.92s/it]
 97%|█████████▋| 14340/14725 [4:10:42<38:38,  6.02s/it]
                                                       

 97%|█████████▋| 14340/14725 [4:10:42<38:38,  6.02s/it]
 97%|█████████▋| 14341/14725 [4:10:48<38:11,  5.97s/it]
 97%|█████████▋| 14342/14725 [4:10:55<38:50,  6.08s/it]
 97%|█████████▋| 14343/14725 [4:11:01<39:07,  6.15s/it]
 97%|█████████▋| 14344/14725 [4:11:07<38:44,  6.10s/it]
 97%|█████████▋| 14345/14725 [4:11:13<38:05,  6.01s/it]
 97%|█████████▋| 14346/14725 [4:11:18<37:05,  5.87s/it]
 97%|█████████▋| 14347/14725 [4:11:24<37:38,  5.97s/it]
 97%|█████████▋| 14348/14725 [4:11:30<37:07,  5.91s/it]
 97%|█████████▋| 14349/14725 [4:11:36<37:05,  5.92s/it]
 97%|█████████▋| 14350/14725 [4:11:41<35:35,  5.69s/it]
           
0: {'loss': 0.5235, 'grad_norm': 0.389039815617287, 'learning_rate': 3e-06, 'epoch': 0.97}
0: {'loss': 0.5316, 'grad_norm': 0.38006435734132926, 'learning_rate': 3e-06, 'epoch': 0.98}
0:                                             

 97%|█████████▋| 14350/14725 [4:11:41<35:35,  5.69s/it]
 97%|█████████▋| 14351/14725 [4:11:48<36:33,  5.86s/it]
 97%|█████████▋| 14352/14725 [4:11:54<36:52,  5.93s/it]
 97%|█████████▋| 14353/14725 [4:12:00<36:54,  5.95s/it]
 97%|█████████▋| 14354/14725 [4:12:05<36:18,  5.87s/it]
 97%|█████████▋| 14355/14725 [4:12:11<36:43,  5.95s/it]
 97%|█████████▋| 14356/14725 [4:12:18<37:15,  6.06s/it]
 98%|█████████▊| 14357/14725 [4:12:23<36:10,  5.90s/it]
 98%|█████████▊| 14358/14725 [4:12:29<36:36,  5.99s/it]
 98%|█████████▊| 14359/14725 [4:12:35<36:19,  5.96s/it]
 98%|█████████▊| 14360/14725 [4:12:42<37:05,  6.10s/it]
                                                       

 98%|█████████▊| 14360/14725 [4:12:42<37:05,  6.10s/it]
 98%|█�
0: {'loss': 0.5214, 'grad_norm': 0.37603198265745263, 'learning_rate': 3e-06, 'epoch': 0.98}
0: ��███████▊| 14361/14725 [4:12:48<36:42,  6.05s/it]
 98%|█████████▊| 14362/14725 [4:12:53<35:43,  5.91s/it]
 98%|█████████▊| 14363/14725 [4:12:59<35:49,  5.94s/it]
 98%|█████████▊| 14364/14725 [4:13:05<34:47,  5.78s/it]
 98%|█████████▊| 14365/14725 [4:13:11<34:57,  5.83s/it]
 98%|█████████▊| 14366/14725 [4:13:17<34:56,  5.84s/it]
 98%|█████████▊| 14367/14725 [4:13:22<34:22,  5.76s/it]
 98%|█████████▊| 14368/14725 [4:13:28<34:43,  5.84s/it]
 98%|█████████▊| 14369/14725 [4:13:34<34:56,  5.89s/it]
 98%|█████████▊| 14370/14725 [4:13:40<34:16,  5.79s/it]
                                                       

 98%|█████████▊| 14370/14725 [4:13:40<34:16,  5.79s/it]
 98%|█████████▊| 14371/14725 [4:13:45<33:56,  5.75s/it]
 98%|█████████▊| 14372/14725 [4:13:51<32:55,
0: {'loss': 0.5245, 'grad_norm': 0.3744842391363422, 'learning_rate': 3e-06, 'epoch': 0.98}
0:   5.60s/it]
 98%|█████████▊| 14373/14725 [4:13:57<33:29,  5.71s/it]
 98%|█████████▊| 14374/14725 [4:14:03<34:15,  5.86s/it]
 98%|█████████▊| 14375/14725 [4:14:08<33:15,  5.70s/it]
 98%|█████████▊| 14376/14725 [4:14:14<33:39,  5.79s/it]
 98%|█████████▊| 14377/14725 [4:14:20<33:47,  5.83s/it]
 98%|█████████▊| 14378/14725 [4:14:26<34:27,  5.96s/it]
 98%|█████████▊| 14379/14725 [4:14:32<34:29,  5.98s/it]
 98%|█████████▊| 14380/14725 [4:14:38<34:15,  5.96s/it]
                                                       

 98%|█████████▊| 14380/14725 [4:14:38<34:15,  5.96s/it]
 98%|█████████▊| 14381/14725 [4:14:44<33:49,  5.90s/it]
 98%|█████████▊| 14382/14725 [4:14:49<32:58,  5.77s/it]
 98%|█████████▊| 14383/14725 [4:14:55<32:38,  5.73s/it]
 98%|█████████▊| 14384/
0: {'loss': 0.5153, 'grad_norm': 0.38832747764260056, 'learning_rate': 3e-06, 'epoch': 0.98}
0: 14725 [4:15:01<33:45,  5.94s/it]
 98%|█████████▊| 14385/14725 [4:15:07<32:13,  5.69s/it]
 98%|█████████▊| 14386/14725 [4:15:12<31:50,  5.64s/it]
 98%|█████████▊| 14387/14725 [4:15:17<30:41,  5.45s/it]
 98%|█████████▊| 14388/14725 [4:15:23<31:03,  5.53s/it]
 98%|█████████▊| 14389/14725 [4:15:28<30:35,  5.46s/it]
 98%|█████████▊| 14390/14725 [4:15:34<31:07,  5.57s/it]
                                                       

 98%|█████████▊| 14390/14725 [4:15:34<31:07,  5.57s/it]
 98%|█████████▊| 14391/14725 [4:15:40<31:41,  5.69s/it]
 98%|█████████▊| 14392/14725 [4:15:46<31:49,  5.73s/it]
 98%|█████████▊| 14393/14725 [4:15:52<32:14,  5.83s/it]
 98%|█████████▊| 14394/14725 [4:15:58<31:56,  5.79s/it]
 98%|█████████▊| 14395/14725 [4:16:03<30:55,  5.62s/it]
 98%|█████�
0: {'loss': 0.524, 'grad_norm': 0.39290696260722696, 'learning_rate': 3e-06, 'epoch': 0.98}
0: �███▊| 14396/14725 [4:16:08<30:39,  5.59s/it]
 98%|█████████▊| 14397/14725 [4:16:14<30:06,  5.51s/it]
 98%|█████████▊| 14398/14725 [4:16:20<30:56,  5.68s/it]
 98%|█████████▊| 14399/14725 [4:16:26<31:42,  5.84s/it]
 98%|█████████▊| 14400/14725 [4:16:32<31:53,  5.89s/it]
                                                       

 98%|█████████▊| 14400/14725 [4:16:32<31:53,  5.89s/it]
 98%|█████████▊| 14401/14725 [4:16:38<31:45,  5.88s/it]
 98%|█████████▊| 14402/14725 [4:16:43<30:46,  5.72s/it]
 98%|█████████▊| 14403/14725 [4:16:49<30:57,  5.77s/it]
 98%|█████████▊| 14404/14725 [4:16:55<30:51,  5.77s/it]
 98%|█████████▊| 14405/14725 [4:17:01<30:58,  5.81s/it]
 98%|█████████▊| 14406/14725 [4:17:07<31:31,  5.93s/it]
 98%|█████████▊| 14407/14725 [4:17:12<30:16,  5.71s/it]
 
0: {'loss': 0.5273, 'grad_norm': 0.39652416499558485, 'learning_rate': 3e-06, 'epoch': 0.98}
0: 98%|█████████▊| 14408/14725 [4:17:18<30:28,  5.77s/it]
 98%|█████████▊| 14409/14725 [4:17:23<29:45,  5.65s/it]
 98%|█████████▊| 14410/14725 [4:17:29<30:17,  5.77s/it]
                                                       

 98%|█████████▊| 14410/14725 [4:17:29<30:17,  5.77s/it]
 98%|█████████▊| 14411/14725 [4:17:36<31:00,  5.93s/it]
 98%|█████████▊| 14412/14725 [4:17:42<31:14,  5.99s/it]
 98%|█████████▊| 14413/14725 [4:17:47<30:17,  5.83s/it]
 98%|█████████▊| 14414/14725 [4:17:53<29:53,  5.77s/it]
 98%|█████████▊| 14415/14725 [4:17:58<29:22,  5.68s/it]
 98%|█████████▊| 14416/14725 [4:18:04<29:00,  5.63s/it]
 98%|█████████▊| 14417/14725 [4:18:10<29:59,  5.84s/it]
 98%|█████████▊| 14418/14725 [4:18:16<29:58,  5.86s/it]
 98%|█████████▊| 14419/14725 [4:18:2
0: {'loss': 0.5168, 'grad_norm': 0.37615995876436586, 'learning_rate': 3e-06, 'epoch': 0.98}
0: 2<30:22,  5.96s/it]
 98%|█████████▊| 14420/14725 [4:18:28<29:25,  5.79s/it]
                                                       

 98%|█████████▊| 14420/14725 [4:18:28<29:25,  5.79s/it]
 98%|█████████▊| 14421/14725 [4:18:34<29:39,  5.85s/it]
 98%|█████████▊| 14422/14725 [4:18:39<28:51,  5.72s/it]
 98%|█████████▊| 14423/14725 [4:18:45<28:51,  5.73s/it]
 98%|█████████▊| 14424/14725 [4:18:51<29:25,  5.87s/it]
 98%|█████████▊| 14425/14725 [4:18:57<28:57,  5.79s/it]
 98%|█████████▊| 14426/14725 [4:19:03<29:29,  5.92s/it]
 98%|█████████▊| 14427/14725 [4:19:09<29:03,  5.85s/it]
 98%|█████████▊| 14428/14725 [4:19:14<28:02,  5.67s/it]
 98%|█████████▊| 14429/14725 [4:19:19<27:37,  5.60s/it]
 98%|█████████▊| 14430/14725 [4:19:25<27:30,  5.60s/it]
                                   
0: {'loss': 0.5241, 'grad_norm': 0.3825393959344549, 'learning_rate': 2.998815820633459e-06, 'epoch': 0.98}
0: {'loss': 0.5043, 'grad_norm': 0.3682513699578065, 'learning_rate': 2.9855176495000345e-06, 'epoch': 0.98}
0:                     

 98%|█████████▊| 14430/14725 [4:19:25<27:30,  5.60s/it]
 98%|█████████▊| 14431/14725 [4:19:30<27:30,  5.61s/it]
 98%|█████████▊| 14432/14725 [4:19:36<27:38,  5.66s/it]
 98%|█████████▊| 14433/14725 [4:19:42<27:00,  5.55s/it]
 98%|█████████▊| 14434/14725 [4:19:48<28:03,  5.79s/it]
 98%|█████████▊| 14435/14725 [4:19:54<28:09,  5.82s/it]
 98%|█████████▊| 14436/14725 [4:20:00<27:54,  5.79s/it]
 98%|█████████▊| 14437/14725 [4:20:06<28:24,  5.92s/it]
 98%|█████████▊| 14438/14725 [4:20:11<27:09,  5.68s/it]
 98%|█████████▊| 14439/14725 [4:20:17<27:39,  5.80s/it]
 98%|█████████▊| 14440/14725 [4:20:23<27:13,  5.73s/it]
                                                       

 98%|█████████▊| 14440/14725 [4:20:23<27:13,  5.73s/it]
 98%|█████████�
0: {'loss': 0.5361, 'grad_norm': 0.37509030698374823, 'learning_rate': 2.9575872675236522e-06, 'epoch': 0.98}
0: ��| 14441/14725 [4:20:28<27:25,  5.79s/it]
 98%|█████████▊| 14442/14725 [4:20:34<26:53,  5.70s/it]
 98%|█████████▊| 14443/14725 [4:20:40<26:38,  5.67s/it]
 98%|█████████▊| 14444/14725 [4:20:45<26:18,  5.62s/it]
 98%|█████████▊| 14445/14725 [4:20:51<26:54,  5.77s/it]
 98%|█████████▊| 14446/14725 [4:20:57<26:25,  5.68s/it]
 98%|█████████▊| 14447/14725 [4:21:02<25:54,  5.59s/it]
 98%|█████████▊| 14448/14725 [4:21:08<26:36,  5.76s/it]
 98%|█████████▊| 14449/14725 [4:21:14<26:08,  5.68s/it]
 98%|█████████▊| 14450/14725 [4:21:19<25:29,  5.56s/it]
                                                       

 98%|█████████▊| 14450/14725 [4:21:19<25:29,  5.56s/it]
 98%|█████████▊| 14451/14725 [4:21:25<25:51,  5.66s/it]
 98%|█████████▊| 14452/14725 [4:21:30<25:19,  5.57s/it]
 98%|██�
0: {'loss': 0.5192, 'grad_norm': 0.39181150530789516, 'learning_rate': 2.915330685814054e-06, 'epoch': 0.98}
0: ��██████▊| 14453/14725 [4:21:36<25:30,  5.63s/it]
 98%|█████████▊| 14454/14725 [4:21:42<26:18,  5.83s/it]
 98%|█████████▊| 14455/14725 [4:21:48<25:33,  5.68s/it]
 98%|█████████▊| 14456/14725 [4:21:54<26:19,  5.87s/it]
 98%|█████████▊| 14457/14725 [4:21:59<25:29,  5.71s/it]
 98%|█████████▊| 14458/14725 [4:22:05<25:21,  5.70s/it]
 98%|█████████▊| 14459/14725 [4:22:11<25:47,  5.82s/it]
 98%|█████████▊| 14460/14725 [4:22:17<25:25,  5.76s/it]
                                                       

 98%|█████████▊| 14460/14725 [4:22:17<25:25,  5.76s/it]
 98%|█████████▊| 14461/14725 [4:22:22<24:38,  5.60s/it]
 98%|█████████▊| 14462/14725 [4:22:27<24:12,  5.52s/it]
 98%|█████████▊| 14463/14725 [4:22:33<24:26,  5.60s/it]
 98%|█████████▊| 14464/14725 [4:22:39<25:07,  5
0: {'loss': 0.526, 'grad_norm': 0.3833305512200044, 'learning_rate': 2.859210876323207e-06, 'epoch': 0.98}
0: .78s/it]
 98%|█████████▊| 14465/14725 [4:22:45<25:39,  5.92s/it]
 98%|█████████▊| 14466/14725 [4:22:51<25:23,  5.88s/it]
 98%|█████████▊| 14467/14725 [4:22:57<25:10,  5.86s/it]
 98%|█████████▊| 14468/14725 [4:23:03<25:35,  5.98s/it]
 98%|█████████▊| 14469/14725 [4:23:09<25:04,  5.88s/it]
 98%|█████████▊| 14470/14725 [4:23:15<25:00,  5.89s/it]
                                                       

 98%|█████████▊| 14470/14725 [4:23:15<25:00,  5.89s/it]
 98%|█████████▊| 14471/14725 [4:23:20<24:20,  5.75s/it]
 98%|█████████▊| 14472/14725 [4:23:26<23:38,  5.61s/it]
 98%|█████████▊| 14473/14725 [4:23:31<23:49,  5.67s/it]
 98%|█████████▊| 14474/14725 [4:23:38<24:23,  5.83s/it]
 98%|█████████▊| 14475/14725 [4:23:44<24:39,  5.92s/it]
 98%|█████████▊| 14476/147
0: {'loss': 0.5275, 'grad_norm': 0.37142474000541714, 'learning_rate': 2.7898426994277207e-06, 'epoch': 0.98}
0: 25 [4:23:50<24:38,  5.94s/it]
 98%|█████████▊| 14477/14725 [4:23:55<23:45,  5.75s/it]
 98%|█████████▊| 14478/14725 [4:24:00<23:05,  5.61s/it]
 98%|█████████▊| 14479/14725 [4:24:06<23:23,  5.71s/it]
 98%|█████████▊| 14480/14725 [4:24:12<23:33,  5.77s/it]
                                                       

 98%|█████████▊| 14480/14725 [4:24:12<23:33,  5.77s/it]
 98%|█████████▊| 14481/14725 [4:24:17<22:55,  5.64s/it]
 98%|█████████▊| 14482/14725 [4:24:23<23:03,  5.69s/it]
 98%|█████████▊| 14483/14725 [4:24:29<23:02,  5.71s/it]
 98%|█████████▊| 14484/14725 [4:24:35<22:44,  5.66s/it]
 98%|█████████▊| 14485/14725 [4:24:41<23:01,  5.76s/it]
 98%|█████████▊| 14486/14725 [4:24:47<23:34,  5.92s/it]
 98%|█████████▊| 14487/14725 [4:24:52<23:00,  5.80s/it]
 98%|██████�
0: {'loss': 0.5262, 'grad_norm': 0.37557927696246335, 'learning_rate': 2.707986167389884e-06, 'epoch': 0.98}
0: �██▊| 14488/14725 [4:24:58<22:58,  5.82s/it]
 98%|█████████▊| 14489/14725 [4:25:05<23:36,  6.00s/it]
 98%|█████████▊| 14490/14725 [4:25:10<22:36,  5.77s/it]
                                                       

 98%|█████████▊| 14490/14725 [4:25:10<22:36,  5.77s/it]
 98%|█████████▊| 14491/14725 [4:25:15<22:11,  5.69s/it]
 98%|█████████▊| 14492/14725 [4:25:22<22:58,  5.92s/it]
 98%|█████████▊| 14493/14725 [4:25:28<22:39,  5.86s/it]
 98%|█████████▊| 14494/14725 [4:25:34<23:10,  6.02s/it]
 98%|█████████▊| 14495/14725 [4:25:40<22:56,  5.99s/it]
 98%|█████████▊| 14496/14725 [4:25:46<22:47,  5.97s/it]
 98%|█████████▊| 14497/14725 [4:25:51<22:12,  5.84s/it]
 98%|█████████▊| 14498/14725 [4:25:57<22:21,  5.91s/it]
 98%|█████████▊| 14499/14725 [4:26:03<22:24,  5.95s/it]
 98%
0: {'loss': 0.5252, 'grad_norm': 0.3660909542743449, 'learning_rate': 2.6145381175042847e-06, 'epoch': 0.98}
0: {'loss': 0.5112, 'grad_norm': 0.3596556561631063, 'learning_rate': 2.5105223861607313e-06, 'epoch': 0.99}
0: |█████████▊| 14500/14725 [4:26:09<21:30,  5.74s/it]
                                                       

 98%|█████████▊| 14500/14725 [4:26:09<21:30,  5.74s/it]
 98%|█████████▊| 14501/14725 [4:26:15<21:54,  5.87s/it]
 98%|█████████▊| 14502/14725 [4:26:20<21:17,  5.73s/it]
 98%|█████████▊| 14503/14725 [4:26:26<21:00,  5.68s/it]
 98%|█████████▊| 14504/14725 [4:26:32<21:14,  5.77s/it]
 99%|█████████▊| 14505/14725 [4:26:37<21:01,  5.73s/it]
 99%|█████████▊| 14506/14725 [4:26:43<20:30,  5.62s/it]
 99%|█████████▊| 14507/14725 [4:26:49<20:39,  5.68s/it]
 99%|█████████▊| 14508/14725 [4:26:54<20:41,  5.72s/it]
 99%|█████████▊| 14509/14725 [4:27:00<20:19,  5.65s/it]
 99%|█████████▊| 14510/14725 [4:27:06<20:43,  5.78s/it]
                                                       

 9
0: {'loss': 0.5246, 'grad_norm': 0.3813525690038011, 'learning_rate': 2.3970785914785146e-06, 'epoch': 0.99}
0: 9%|█████████▊| 14510/14725 [4:27:06<20:43,  5.78s/it]
 99%|█████████▊| 14511/14725 [4:27:11<20:09,  5.65s/it]
 99%|█████████▊| 14512/14725 [4:27:17<20:05,  5.66s/it]
 99%|█████████▊| 14513/14725 [4:27:23<20:19,  5.75s/it]
 99%|█████████▊| 14514/14725 [4:27:29<20:58,  5.97s/it]
 99%|█████████▊| 14515/14725 [4:27:35<20:22,  5.82s/it]
 99%|█████████▊| 14516/14725 [4:27:40<19:54,  5.71s/it]
 99%|█████████▊| 14517/14725 [4:27:46<19:44,  5.70s/it]
 99%|█████████▊| 14518/14725 [4:27:52<19:32,  5.67s/it]
 99%|█████████▊| 14519/14725 [4:27:57<19:05,  5.56s/it]
 99%|█████████▊| 14520/14725 [4:28:03<19:39,  5.75s/it]
                                                       

 99%|█████████▊| 14520/14725 [4:28:03<19:39,  5.75s/it]
 99%|█████████▊| 14521/14725 [4:28:09
0: {'loss': 0.5397, 'grad_norm': 0.36219114276334646, 'learning_rate': 2.2754496474118135e-06, 'epoch': 0.99}
0: <19:48,  5.83s/it]
 99%|█████████▊| 14522/14725 [4:28:15<20:09,  5.96s/it]
 99%|█████████▊| 14523/14725 [4:28:21<19:48,  5.88s/it]
 99%|█████████▊| 14524/14725 [4:28:27<19:12,  5.73s/it]
 99%|█████████▊| 14525/14725 [4:28:32<19:05,  5.73s/it]
 99%|█████████▊| 14526/14725 [4:28:38<18:41,  5.63s/it]
 99%|█████████▊| 14527/14725 [4:28:44<18:53,  5.72s/it]
 99%|█████████▊| 14528/14725 [4:28:49<18:55,  5.76s/it]
 99%|█████████▊| 14529/14725 [4:28:55<18:50,  5.77s/it]
 99%|█████████▊| 14530/14725 [4:29:01<18:32,  5.71s/it]
                                                       

 99%|█████████▊| 14530/14725 [4:29:01<18:32,  5.71s/it]
 99%|█████████▊| 14531/14725 [4:29:07<18:45,  5.80s/it]
 99%|█████████▊| 14532/14725 [4:29:13<18:53,  5.87s/it]
 99%|█████████▊|
0: {'loss': 0.5213, 'grad_norm': 0.36410533767087977, 'learning_rate': 2.1469681461243154e-06, 'epoch': 0.99}
0:  14533/14725 [4:29:18<18:33,  5.80s/it]
 99%|█████████▊| 14534/14725 [4:29:25<19:00,  5.97s/it]
 99%|█████████▊| 14535/14725 [4:29:31<18:56,  5.98s/it]
 99%|█████████▊| 14536/14725 [4:29:37<19:01,  6.04s/it]
 99%|█████████▊| 14537/14725 [4:29:43<19:14,  6.14s/it]
 99%|█████████▊| 14538/14725 [4:29:49<18:44,  6.01s/it]
 99%|█████████▊| 14539/14725 [4:29:54<18:02,  5.82s/it]
 99%|█████████▊| 14540/14725 [4:30:00<17:48,  5.77s/it]
                                                       

 99%|█████████▊| 14540/14725 [4:30:00<17:48,  5.77s/it]
 99%|█████████▉| 14541/14725 [4:30:06<17:31,  5.72s/it]
 99%|█████████▉| 14542/14725 [4:30:12<17:47,  5.83s/it]
 99%|█████████▉| 14543/14725 [4:30:18<17:54,  5.91s/it]
 99%|█████████▉| 14544/14725 [4:30:23<17:28,  5.79s/it]
 99%|███�
0: {'loss': 0.5267, 'grad_norm': 0.3913519198981987, 'learning_rate': 2.0130417578306083e-06, 'epoch': 0.99}
0: ��█████▉| 14545/14725 [4:30:29<17:06,  5.70s/it]
 99%|█████████▉| 14546/14725 [4:30:35<17:02,  5.71s/it]
 99%|█████████▉| 14547/14725 [4:30:41<17:14,  5.81s/it]
 99%|█████████▉| 14548/14725 [4:30:47<17:19,  5.87s/it]
 99%|█████████▉| 14549/14725 [4:30:53<17:17,  5.90s/it]
 99%|█████████▉| 14550/14725 [4:30:59<17:12,  5.90s/it]
                                                       

 99%|█████████▉| 14550/14725 [4:30:59<17:12,  5.90s/it]
 99%|█████████▉| 14551/14725 [4:31:04<16:55,  5.84s/it]
 99%|█████████▉| 14552/14725 [4:31:10<16:58,  5.89s/it]
 99%|█████████▉| 14553/14725 [4:31:16<16:54,  5.90s/it]
 99%|█████████▉| 14554/14725 [4:31:22<16:46,  5.88s/it]
 99%|█████████▉| 14555/14725 [4:31:28<16:31,  5.83s/it]
 99%|█████████▉| 14556/14725 [4:31:34<16:50,  5.98
0: {'loss': 0.5292, 'grad_norm': 0.3915789196734901, 'learning_rate': 1.8751378080667381e-06, 'epoch': 0.99}
0: s/it]
 99%|█████████▉| 14557/14725 [4:31:40<16:29,  5.89s/it]
 99%|█████████▉| 14558/14725 [4:31:45<15:58,  5.74s/it]
 99%|█████████▉| 14559/14725 [4:31:51<15:58,  5.77s/it]
 99%|█████████▉| 14560/14725 [4:31:56<15:34,  5.66s/it]
                                                       

 99%|█████████▉| 14560/14725 [4:31:56<15:34,  5.66s/it]
 99%|█████████▉| 14561/14725 [4:32:02<15:40,  5.74s/it]
 99%|█████████▉| 14562/14725 [4:32:08<15:27,  5.69s/it]
 99%|█████████▉| 14563/14725 [4:32:14<15:45,  5.83s/it]
 99%|█████████▉| 14564/14725 [4:32:20<15:55,  5.93s/it]
 99%|█████████▉| 14565/14725 [4:32:25<15:14,  5.72s/it]
 99%|█████████▉| 14566/14725 [4:32:31<14:38,  5.53s/it]
 99%|█████████▉| 14567/14725 [4:32:36<14:45,  5.60s/it]
 99%|█████████▉| 14568/14725 
0: {'loss': 0.5242, 'grad_norm': 0.3754196219246912, 'learning_rate': 1.7347672013645732e-06, 'epoch': 0.99}
0: [4:32:42<14:51,  5.68s/it]
 99%|█████████▉| 14569/14725 [4:32:49<15:16,  5.87s/it]
 99%|█████████▉| 14570/14725 [4:32:54<15:04,  5.84s/it]
                                                       

 99%|█████████▉| 14570/14725 [4:32:54<15:04,  5.84s/it]
 99%|█████████▉| 14571/14725 [4:33:00<14:55,  5.82s/it]
 99%|█████████▉| 14572/14725 [4:33:06<15:09,  5.95s/it]
 99%|█████████▉| 14573/14725 [4:33:12<14:52,  5.87s/it]
 99%|█████████▉| 14574/14725 [4:33:18<14:50,  5.90s/it]
 99%|█████████▉| 14575/14725 [4:33:23<14:24,  5.76s/it]
 99%|█████████▉| 14576/14725 [4:33:28<13:49,  5.57s/it]
 99%|█████████▉| 14577/14725 [4:33:35<14:19,  5.81s/it]
 99%|█████████▉| 14578/14725 [4:33:41<14:33,  5.94s/it]
 99%|█████████▉| 14579/14725 [4:33:46<14:01,  5.77s/it]
 99%|███████�
0: {'loss': 0.5094, 'grad_norm': 0.38407592545670843, 'learning_rate': 1.5934678674655809e-06, 'epoch': 0.99}
0: {'loss': 0.5207, 'grad_norm': 0.3680461106395497, 'learning_rate': 1.4527879114407447e-06, 'epoch': 0.99}
0: �█▉| 14580/14725 [4:33:52<14:01,  5.80s/it]
                                                       

 99%|█████████▉| 14580/14725 [4:33:52<14:01,  5.80s/it]
 99%|█████████▉| 14581/14725 [4:33:58<14:05,  5.87s/it]
 99%|█████████▉| 14582/14725 [4:34:04<13:52,  5.82s/it]
 99%|█████████▉| 14583/14725 [4:34:10<13:52,  5.86s/it]
 99%|█████████▉| 14584/14725 [4:34:16<13:53,  5.91s/it]
 99%|█████████▉| 14585/14725 [4:34:22<13:46,  5.90s/it]
 99%|█████████▉| 14586/14725 [4:34:28<13:58,  6.03s/it]
 99%|█████████▉| 14587/14725 [4:34:34<13:57,  6.07s/it]
 99%|█████████▉| 14588/14725 [4:34:40<13:26,  5.89s/it]
 99%|█████████▉| 14589/14725 [4:34:46<13:35,  6.00s/it]
 99%|█████████▉| 14590/14725 [4:34:52<13:17,  5.91s/it]
                                                       

 99%|███████
0: {'loss': 0.5233, 'grad_norm': 0.3518109293164943, 'learning_rate': 1.3142686523274465e-06, 'epoch': 0.99}
0: ██▉| 14590/14725 [4:34:52<13:17,  5.91s/it]
 99%|█████████▉| 14591/14725 [4:34:58<13:24,  6.00s/it]
 99%|█████████▉| 14592/14725 [4:35:04<13:10,  5.95s/it]
 99%|█████████▉| 14593/14725 [4:35:10<13:19,  6.06s/it]
 99%|█████████▉| 14594/14725 [4:35:16<13:07,  6.01s/it]
 99%|█████████▉| 14595/14725 [4:35:22<12:44,  5.88s/it]
 99%|█████████▉| 14596/14725 [4:35:27<12:15,  5.71s/it]
 99%|█████████▉| 14597/14725 [4:35:33<12:29,  5.86s/it]
 99%|█████████▉| 14598/14725 [4:35:39<12:05,  5.71s/it]
 99%|█████████▉| 14599/14725 [4:35:45<12:22,  5.89s/it]
 99%|█████████▉| 14600/14725 [4:35:50<11:53,  5.71s/it]
                                                       

 99%|█████████▉| 14600/14725 [4:35:50<11:53,  5.71s/it]
 99%|█████████▉| 14601/14725 [4:35:56<11:58,  5.79s/it]
 99%|
0: {'loss': 0.5279, 'grad_norm': 0.35473286630413453, 'learning_rate': 1.1794277361155496e-06, 'epoch': 0.99}
0: █████████▉| 14602/14725 [4:36:02<12:02,  5.87s/it]
 99%|█████████▉| 14603/14725 [4:36:08<11:56,  5.87s/it]
 99%|█████████▉| 14604/14725 [4:36:14<12:00,  5.95s/it]
 99%|█████████▉| 14605/14725 [4:36:21<12:06,  6.05s/it]
 99%|█████████▉| 14606/14725 [4:36:27<12:12,  6.15s/it]
 99%|█████████▉| 14607/14725 [4:36:33<12:18,  6.26s/it]
 99%|█████████▉| 14608/14725 [4:36:40<12:12,  6.26s/it]
 99%|█████████▉| 14609/14725 [4:36:46<12:03,  6.24s/it]
 99%|█████████▉| 14610/14725 [4:36:51<11:32,  6.02s/it]
                                                       

 99%|█████████▉| 14610/14725 [4:36:51<11:32,  6.02s/it]
 99%|█████████▉| 14611/14725 [4:36:58<11:30,  6.06s/it]
 99%|█████████▉| 14612/14725 [4:37:03<11:13,  5.96s/it]
 99%|█████████▉| 14613/14725 [4:37:09<10
0: {'loss': 0.5127, 'grad_norm': 0.3702721639947587, 'learning_rate': 1.0497425081003483e-06, 'epoch': 0.99}
0: :59,  5.89s/it]
 99%|█████████▉| 14614/14725 [4:37:15<10:48,  5.84s/it]
 99%|█████████▉| 14615/14725 [4:37:21<10:53,  5.94s/it]
 99%|█████████▉| 14616/14725 [4:37:27<11:00,  6.06s/it]
 99%|█████████▉| 14617/14725 [4:37:33<10:36,  5.89s/it]
 99%|█████████▉| 14618/14725 [4:37:38<10:19,  5.79s/it]
 99%|█████████▉| 14619/14725 [4:37:44<09:56,  5.63s/it]
 99%|█████████▉| 14620/14725 [4:37:50<10:02,  5.74s/it]
                                                       

 99%|█████████▉| 14620/14725 [4:37:50<10:02,  5.74s/it]
 99%|█████████▉| 14621/14725 [4:37:56<10:11,  5.88s/it]
 99%|█████████▉| 14622/14725 [4:38:02<10:05,  5.88s/it]
 99%|█████████▉| 14623/14725 [4:38:07<09:51,  5.79s/it]
 99%|█████████▉| 14624/14725 [4:38:13<09:38,  5.73s/it]
 99%|█████████▉| 14
0: {'loss': 0.5288, 'grad_norm': 0.3517411315343397, 'learning_rate': 9.266338267783543e-07, 'epoch': 0.99}
0: 625/14725 [4:38:19<09:42,  5.83s/it]
 99%|█████████▉| 14626/14725 [4:38:25<09:35,  5.81s/it]
 99%|█████████▉| 14627/14725 [4:38:30<09:22,  5.74s/it]
 99%|█████████▉| 14628/14725 [4:38:35<09:03,  5.61s/it]
 99%|█████████▉| 14629/14725 [4:38:41<09:04,  5.67s/it]
 99%|█████████▉| 14630/14725 [4:38:47<09:09,  5.79s/it]
                                                       

 99%|█████████▉| 14630/14725 [4:38:47<09:09,  5.79s/it]
 99%|█████████▉| 14631/14725 [4:38:53<09:09,  5.85s/it]
 99%|█████████▉| 14632/14725 [4:38:59<09:02,  5.83s/it]
 99%|█████████▉| 14633/14725 [4:39:05<09:08,  5.97s/it]
 99%|█████████▉| 14634/14725 [4:39:11<08:48,  5.81s/it]
 99%|█████████▉| 14635/14725 [4:39:16<08:25,  5.61s/it]
 99%|█████████▉| 14636/14725 [4:39:22<08:17,  5.59s/it]
 99%|████�
0: {'loss': 0.5095, 'grad_norm': 0.34172216776782466, 'learning_rate': 8.114504966242812e-07, 'epoch': 0.99}
0: ��████▉| 14637/14725 [4:39:28<08:23,  5.72s/it]
 99%|█████████▉| 14638/14725 [4:39:33<08:11,  5.65s/it]
 99%|█████████▉| 14639/14725 [4:39:39<08:08,  5.67s/it]
 99%|█████████▉| 14640/14725 [4:39:44<07:59,  5.64s/it]
                                                       

 99%|█████████▉| 14640/14725 [4:39:44<07:59,  5.64s/it]
 99%|█████████▉| 14641/14725 [4:39:50<07:46,  5.56s/it]
 99%|█████████▉| 14642/14725 [4:39:56<07:47,  5.63s/it]
 99%|█████████▉| 14643/14725 [4:40:02<07:59,  5.85s/it]
 99%|█████████▉| 14644/14725 [4:40:07<07:48,  5.78s/it]
 99%|█████████▉| 14645/14725 [4:40:13<07:41,  5.77s/it]
 99%|█████████▉| 14646/14725 [4:40:19<07:47,  5.92s/it]
 99%|█████████▉| 14647/14725 [4:40:26<07:49,  6.02s/it]
 99%|█████████▉| 14648/14725 [4:40:31<07:32,  5.88s/i
0: {'loss': 0.523, 'grad_norm': 0.35572556341368744, 'learning_rate': 7.054544903069566e-07, 'epoch': 0.99}
0: t]
 99%|█████████▉| 14649/14725 [4:40:37<07:30,  5.93s/it]
 99%|█████████▉| 14650/14725 [4:40:43<07:12,  5.76s/it]
                                                       

 99%|█████████▉| 14650/14725 [4:40:43<07:12,  5.76s/it]
 99%|█████████▉| 14651/14725 [4:40:49<07:13,  5.86s/it]
100%|█████████▉| 14652/14725 [4:40:54<07:02,  5.79s/it]
100%|█████████▉| 14653/14725 [4:41:00<06:58,  5.81s/it]
100%|█████████▉| 14654/14725 [4:41:07<07:04,  5.97s/it]
100%|█████████▉| 14655/14725 [4:41:13<06:58,  5.98s/it]
100%|█████████▉| 14656/14725 [4:41:18<06:46,  5.89s/it]
100%|█████████▉| 14657/14725 [4:41:24<06:36,  5.83s/it]
100%|█████████▉| 14658/14725 [4:41:30<06:33,  5.88s/it]
100%|█████████▉| 14659/14725 [4:41:35<06:16,  5.71s/it]
100%|█████████▉| 14660/14725 [4:
0: {'loss': 0.5197, 'grad_norm': 0.3506957506754727, 'learning_rate': 6.098071222526847e-07, 'epoch': 1.0}
0: {'loss': 0.5188, 'grad_norm': 0.33625863189738897, 'learning_rate': 5.255563250413658e-07, 'epoch': 1.0}
0: 41:41<06:17,  5.81s/it]
                                                       

100%|█████████▉| 14660/14725 [4:41:41<06:17,  5.81s/it]
100%|█████████▉| 14661/14725 [4:41:47<06:05,  5.72s/it]
100%|█████████▉| 14662/14725 [4:41:53<06:04,  5.79s/it]
100%|█████████▉| 14663/14725 [4:41:58<05:51,  5.67s/it]
100%|█████████▉| 14664/14725 [4:42:04<05:49,  5.73s/it]
100%|█████████▉| 14665/14725 [4:42:10<05:46,  5.78s/it]
100%|█████████▉| 14666/14725 [4:42:16<05:42,  5.80s/it]
100%|█████████▉| 14667/14725 [4:42:21<05:27,  5.64s/it]
100%|█████████▉| 14668/14725 [4:42:27<05:28,  5.76s/it]
100%|█████████▉| 14669/14725 [4:42:33<05:25,  5.81s/it]
100%|█████████▉| 14670/14725 [4:42:39<05:17,  5.77s/it]
                                                       

100%|█████████▉| 14670/14725 [
0: {'loss': 0.5245, 'grad_norm': 0.3602008545123055, 'learning_rate': 4.5362516803786016e-07, 'epoch': 1.0}
0: 4:42:39<05:17,  5.77s/it]
100%|█████████▉| 14671/14725 [4:42:45<05:18,  5.90s/it]
100%|█████████▉| 14672/14725 [4:42:51<05:11,  5.87s/it]
100%|█████████▉| 14673/14725 [4:42:56<05:03,  5.83s/it]
100%|█████████▉| 14674/14725 [4:43:02<04:51,  5.72s/it]
100%|█████████▉| 14675/14725 [4:43:07<04:42,  5.66s/it]
100%|█████████▉| 14676/14725 [4:43:13<04:40,  5.73s/it]
100%|█████████▉| 14677/14725 [4:43:19<04:35,  5.74s/it]
100%|█████████▉| 14678/14725 [4:43:25<04:34,  5.84s/it]
100%|█████████▉| 14679/14725 [4:43:31<04:31,  5.90s/it]
100%|█████████▉| 14680/14725 [4:43:37<04:20,  5.78s/it]
                                                       

100%|█████████▉| 14680/14725 [4:43:37<04:20,  5.78s/it]
100%|█████████▉| 14681/14725 [4:43:43<04:14,  5.79s/it]
100%|████████
0: {'loss': 0.5184, 'grad_norm': 0.3459317954918365, 'learning_rate': 3.948017440508607e-07, 'epoch': 1.0}
0: █▉| 14682/14725 [4:43:49<04:17,  5.99s/it]
100%|█████████▉| 14683/14725 [4:43:54<04:03,  5.80s/it]
100%|█████████▉| 14684/14725 [4:44:01<04:02,  5.91s/it]
100%|█████████▉| 14685/14725 [4:44:07<03:58,  5.97s/it]
100%|█████████▉| 14686/14725 [4:44:12<03:47,  5.84s/it]
100%|█████████▉| 14687/14725 [4:44:18<03:44,  5.90s/it]
100%|█████████▉| 14688/14725 [4:44:24<03:36,  5.86s/it]
100%|█████████▉| 14689/14725 [4:44:30<03:29,  5.81s/it]
100%|█████████▉| 14690/14725 [4:44:36<03:26,  5.90s/it]
                                                       

100%|█████████▉| 14690/14725 [4:44:36<03:26,  5.90s/it]
100%|█████████▉| 14691/14725 [4:44:41<03:17,  5.81s/it]
100%|█████████▉| 14692/14725 [4:44:47<03:10,  5.78s/it]
100%|█████████▉| 14693/14725 [4:44:53<03:02,  5.70s/it]
100%|█
0: {'loss': 0.5123, 'grad_norm': 0.3473841195563995, 'learning_rate': 3.497305348231616e-07, 'epoch': 1.0}
0: ████████▉| 14694/14725 [4:44:58<02:56,  5.68s/it]
100%|█████████▉| 14695/14725 [4:45:04<02:52,  5.76s/it]
100%|█████████▉| 14696/14725 [4:45:10<02:47,  5.77s/it]
100%|█████████▉| 14697/14725 [4:45:16<02:43,  5.85s/it]
100%|█████████▉| 14698/14725 [4:45:22<02:38,  5.88s/it]
100%|█████████▉| 14699/14725 [4:45:28<02:30,  5.79s/it]
100%|█████████▉| 14700/14725 [4:45:34<02:26,  5.87s/it]
                                                       

100%|█████████▉| 14700/14725 [4:45:34<02:26,  5.87s/it]
100%|█████████▉| 14701/14725 [4:45:39<02:18,  5.75s/it]
100%|█████████▉| 14702/14725 [4:45:44<02:09,  5.62s/it]
100%|█████████▉| 14703/14725 [4:45:51<02:06,  5.76s/it]
100%|█████████▉| 14704/14725 [4:45:57<02:04,  5.92s/it]
100%|█████████▉| 14705/14725 [4:46:03<01:59
0: {'loss': 0.527, 'grad_norm': 0.3442297253342162, 'learning_rate': 3.189053499548184e-07, 'epoch': 1.0}
0: ,  5.96s/it]
100%|█████████▉| 14706/14725 [4:46:08<01:50,  5.82s/it]
100%|█████████▉| 14707/14725 [4:46:14<01:42,  5.67s/it]
100%|█████████▉| 14708/14725 [4:46:20<01:39,  5.84s/it]
100%|█████████▉| 14709/14725 [4:46:25<01:31,  5.75s/it]
100%|█████████▉| 14710/14725 [4:46:31<01:25,  5.69s/it]
                                                       

100%|█████████▉| 14710/14725 [4:46:31<01:25,  5.69s/it]
100%|█████████▉| 14711/14725 [4:46:37<01:21,  5.79s/it]
100%|█████████▉| 14712/14725 [4:46:43<01:14,  5.70s/it]
100%|█████████▉| 14713/14725 [4:46:48<01:08,  5.75s/it]
100%|█████████▉| 14714/14725 [4:46:54<01:02,  5.64s/it]
100%|█████████▉| 14715/14725 [4:46:59<00:56,  5.63s/it]
100%|█████████▉| 14716/14725 [4:47:05<00:50,  5.64s/it]
100%|█████████▉| 14717
0: {'loss': 0.5331, 'grad_norm': 0.3649898817606929, 'learning_rate': 3.026639166218334e-07, 'epoch': 1.0}
0: {'train_runtime': 17322.386, 'train_samples_per_second': 13.601, 'train_steps_per_second': 0.85, 'train_loss': 0.10514562906352692, 'epoch': 1.0}
0: /14725 [4:47:11<00:46,  5.79s/it]
100%|█████████▉| 14718/14725 [4:47:17<00:40,  5.85s/it]
100%|█████████▉| 14719/14725 [4:47:23<00:34,  5.83s/it]
100%|█████████▉| 14720/14725 [4:47:29<00:28,  5.76s/it]
                                                       

100%|█████████▉| 14720/14725 [4:47:29<00:28,  5.76s/it]
100%|█████████▉| 14721/14725 [4:47:35<00:23,  5.86s/it]
100%|█████████▉| 14722/14725 [4:47:40<00:17,  5.72s/it]
100%|█████████▉| 14723/14725 [4:47:46<00:11,  5.84s/it]
100%|█████████▉| 14724/14725 [4:47:52<00:05,  5.77s/it]
100%|██████████| 14725/14725 [4:48:01<00:00,  6.89s/it]
                                                       

100%|██████████| 14725/14725 [4:48:42<00:00,  6.89s/it]
100%|██████████| 14725/14725 [4:48:42<00:00,  1.18s/it]
0: [2025-08-17 04:06:05,373] [INFO] [axolotl.train.save_trained_model:246] [PID:1085867] [RANK:0] Training completed! Saving trained model to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-14B/1.
0: [2025-08-17 04:06:34,067] [INFO] [axolotl.train.save_trained_model:331] [PID:1085867] [RANK:0] Model successfully saved to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-14B/1