2: W0820 16:57:24.058000 1721248 torch/distributed/run.py:792] 
2: W0820 16:57:24.058000 1721248 torch/distributed/run.py:792] *****************************************
2: W0820 16:57:24.058000 1721248 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
2: W0820 16:57:24.058000 1721248 torch/distributed/run.py:792] *****************************************
1: W0820 16:57:24.060000 848800 torch/distributed/run.py:792] 
1: W0820 16:57:24.060000 848800 torch/distributed/run.py:792] *****************************************
1: W0820 16:57:24.060000 848800 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
1: W0820 16:57:24.060000 848800 torch/distributed/run.py:792] *****************************************
3: W0820 16:57:24.060000 1434017 torch/distributed/run.py:792] 
3: W0820 16:57:24.060000 1434017 torch/distributed/run.py:792] *****************************************
3: W0820 16:57:24.060000 1434017 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
3: W0820 16:57:24.060000 1434017 torch/distributed/run.py:792] *****************************************
0: W0820 16:57:24.108000 195210 torch/distributed/run.py:792] 
0: W0820 16:57:24.108000 195210 torch/distributed/run.py:792] *****************************************
0: W0820 16:57:24.108000 195210 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
0: W0820 16:57:24.108000 195210 torch/distributed/run.py:792] *****************************************
2: [2025-08-20 16:57:58,011] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:1721323] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
2: [2025-08-20 16:57:58,012] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:1721323] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
3: [2025-08-20 16:57:58,017] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:1434093] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
3: [2025-08-20 16:57:58,017] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:1434093] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
1: [2025-08-20 16:57:58,018] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:848876] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
1: [2025-08-20 16:57:58,018] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:848876] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
0: [2025-08-20 16:57:58,100] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:195289] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
0: [2025-08-20 16:57:58,101] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:195289] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
0: [2025-08-20 16:58:03,378] [INFO] [axolotl.cli.config.load_cfg:244] [PID:195289] [RANK:0] config:
0: {
0:   "activation_offloading": false,
0:   "auto_resume_from_checkpoints": true,
0:   "axolotl_config_path": "/lustre/fswork/projects/rech/dgo/udv55np/train/tmp/1755698958402216459.yaml",
0:   "base_model": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-1.5B",
0:   "base_model_config": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-1.5B",
0:   "batch_size": 16,
0:   "bf16": true,
0:   "capabilities": {
0:     "bf16": true,
0:     "compute_capability": "sm_90",
0:     "fp8": false,
0:     "n_gpu": 16,
0:     "n_node": 1
0:   },
0:   "chat_template": "qwen_25",
0:   "dataloader_num_workers": 16,
0:   "dataloader_pin_memory": true,
0:   "dataloader_prefetch_factor": 256,
0:   "dataset_prepared_path": "/lustre/fsn1/projects/rech/dgo/udv55np/dataset/Qwen3-235B-A22B/Qwen2.5-1.5B/mix_0",
0:   "dataset_processes": 192,
0:   "datasets": [
0:     {
0:       "chat_template": "tokenizer_default",
0:       "data_files": [
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0007.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0009.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0005.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0006.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0014.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0010.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0012.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0008.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0001.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0002.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0013.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0015.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0004.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0011.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0000.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0003.jsonl"
0:       ],
0:       "ds_type": "json",
0:       "field_messages": "conversations",
0:       "message_property_mappings": {
0:         "content": "content",
0:         "role": "role"
0:       },
0:       "path": "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking",
0:       "trust_remote_code": false,
0:       "type": "chat_template"
0:     }
0:   ],
0:   "ddp": true,
0:   "deepspeed": {
0:     "bf16": {
0:       "enabled": true
0:     },
0:     "gradient_accumulation_steps": "auto",
0:     "gradient_clipping": "auto",
0:     "train_batch_size": "auto",
0:     "train_micro_batch_size_per_gpu": "auto",
0:     "wall_clock_breakdown": false,
0:     "zero_optimization": {
0:       "contiguous_gradients": true,
0:       "overlap_comm": true,
0:       "reduce_bucket_size": "auto",
0:       "stage": 3,
0:       "stage3_gather_16bit_weights_on_model_save": true,
0:       "stage3_param_persistence_threshold": "auto",
0:       "stage3_prefetch_bucket_size": "auto",
0:       "sub_group_size": 0
0:     }
0:   },
0:   "device": "cuda:0",
0:   "device_map": {
0:     "": 0
0:   },
0:   "env_capabilities": {
0:     "torch_version": "2.6.0"
0:   },
0:   "eval_batch_size": 1,
0:   "eval_causal_lm_metrics": [
0:     "sacrebleu",
0:     "comet",
0:     "ter",
0:     "chrf"
0:   ],
0:   "eval_max_new_tokens": 128,
0:   "eval_sample_packing": true,
0:   "eval_table_size": 0,
0:   "evals_per_epoch": 0,
0:   "flash_attention": true,
0:   "fp16": false,
0:   "gradient_accumulation_steps": 1,
0:   "gradient_checkpointing": true,
0:   "gradient_checkpointing_kwargs": {
0:     "use_reentrant": true
0:   },
0:   "learning_rate": 7e-06,
0:   "lisa_layers_attribute": "model.layers",
0:   "load_best_model_at_end": false,
0:   "load_in_4bit": false,
0:   "load_in_8bit": false,
0:   "local_rank": 0,
0:   "logging_steps": 10,
0:   "lora_dropout": 0.0,
0:   "loraplus_lr_embedding": 1e-06,
0:   "lr_scheduler": "warmup_stable_decay",
0:   "lr_scheduler_kwargs": {
0:     "min_lr_ratio": 0.1,
0:     "num_decay_steps": 300
0:   },
0:   "max_prompt_len": 512,
0:   "mean_resizing_embeddings": false,
0:   "micro_batch_size": 1,
0:   "model_config_type": "qwen2",
0:   "num_epochs": 1.0,
0:   "optimizer": "adamw_torch_fused",
0:   "output_dir": "/lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-1.5B/0",
0:   "pad_to_sequence_len": true,
0:   "pretrain_multipack_attn": true,
0:   "pretrain_multipack_buffer_size": 10000,
0:   "profiler_steps_start": 0,
0:   "qlora_sharded_model_loading": false,
0:   "ray_num_workers": 1,
0:   "resources_per_worker": {
0:     "GPU": 1
0:   },
0:   "sample_packing": true,
0:   "sample_packing_bin_size": 200,
0:   "sample_packing_group_size": 100000,
0:   "save_only_model": false,
0:   "save_safetensors": true,
0:   "save_steps": 0.2,
0:   "save_total_limit": 20,
0:   "sequence_len": 16384,
0:   "sequence_parallel_degree": 1,
0:   "shuffle_merged_datasets": true,
0:   "skip_prepare_dataset": false,
0:   "special_tokens": {
0:     "bos_token": "<|im_start|>",
0:     "eos_token": "<|im_end|>",
0:     "pad_token": "<|endoftext|>"
0:   },
0:   "strict": false,
0:   "tensor_parallel_size": 1,
0:   "tf32": false,
0:   "tiled_mlp_use_original_mlp": true,
0:   "tokenizer_config": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-1.5B",
0:   "torch_dtype": "torch.bfloat16",
0:   "train_on_inputs": false,
0:   "trl": {
0:     "log_completions": false,
0:     "mask_truncated_completions": false,
0:     "ref_model_mixup_alpha": 0.9,
0:     "ref_model_sync_steps": 64,
0:     "scale_rewards": true,
0:     "sync_ref_model": false,
0:     "use_vllm": false,
0:     "vllm_server_host": "0.0.0.0",
0:     "vllm_server_port": 8000
0:   },
0:   "use_ray": false,
0:   "use_tensorboard": true,
0:   "val_set_size": 0.0,
0:   "vllm": {
0:     "device": "auto",
0:     "dtype": "auto",
0:     "gpu_memory_utilization": 0.9,
0:     "host": "0.0.0.0",
0:     "port": 8000
0:   },
0:   "warmup_steps": 150,
0:   "weight_decay": 0.0,
0:   "world_size": 16
0: }
0: [2025-08-20 16:58:03,380] [INFO] [axolotl.cli.checks.check_user_token:35] [PID:195289] [RANK:0] Skipping HuggingFace token verification because HF_HUB_OFFLINE is set to True. Only local files will be used.
1: [2025-08-20 16:58:03,652] [INFO] [axolotl.utils.data.sft._load_raw_datasets:310] [PID:848876] [RANK:0] Loading raw datasets...
1: [2025-08-20 16:58:03,935] [INFO] [axolotl.utils.data.wrappers.get_dataset_wrapper:88] [PID:848876] [RANK:0] Loading dataset: /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking with base_type: chat_template and prompt_style: None
1: 
Dropping Long Sequences (>16384) (num_proc=192):   0%|          | 0/1393784 [00:00<?, ? examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   0%|          | 1000/1393784 [00:01<33:13, 698.53 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   1%|          | 13000/1393784 [00:01<01:59, 11511.18 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   2%|▏         | 32000/1393784 [00:01<00:42, 31950.43 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   4%|▎         | 49000/1393784 [00:01<00:26, 51019.80 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   5%|▍         | 65000/1393784 [00:01<00:19, 68386.87 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   6%|▌         | 82000/1393784 [00:01<00:15, 87276.07 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   7%|▋         | 102000/1393784 [00:02<00:11, 110737.49 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   9%|▊         | 120000/1393784 [00:02<00:10, 125750.00 ex
1: amples/s]
Dropping Long Sequences (>16384) (num_proc=192):  10%|▉         | 138000/1393784 [00:02<00:09, 136842.03 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  11%|█         | 155000/1393784 [00:02<00:08, 139328.17 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  12%|█▏        | 172000/1393784 [00:02<00:08, 145776.77 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  14%|█▎        | 189000/1393784 [00:02<00:08, 143940.66 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  15%|█▍        | 205000/1393784 [00:02<00:08, 145243.53 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  16%|█▌        | 221000/1393784 [00:02<00:07, 147806.51 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  17%|█▋        | 237000/1393784 [00:02<00:07, 149615.41 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  18%|█▊        | 255000/1393784 [00:03<00:07, 156281.11 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  
1: 22%|██▏       | 308000/1393784 [00:03<00:04, 259546.92 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  25%|██▍       | 345000/1393784 [00:03<00:03, 290229.96 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  34%|███▍      | 473560/1393784 [00:03<00:01, 578677.44 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  47%|████▋     | 653759/1393784 [00:03<00:00, 924997.28 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  54%|█████▎    | 747358/1393784 [00:03<00:00, 681028.05 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  59%|█████▉    | 825957/1393784 [00:03<00:01, 560255.98 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  64%|██████▍   | 892294/1393784 [00:04<00:00, 518621.21 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  68%|██████▊   | 950850/1393784 [00:04<00:00, 495287.16 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  72%|�
1: �██████▏  | 1004961/1393784 [00:04<00:00, 459776.29 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  76%|███████▌  | 1054517/1393784 [00:04<00:00, 442192.38 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  80%|███████▉  | 1110848/1393784 [00:04<00:00, 468623.99 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  83%|████████▎ | 1160474/1393784 [00:04<00:00, 425479.53 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  87%|████████▋ | 1206546/1393784 [00:04<00:00, 428331.71 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  90%|████████▉ | 1251431/1393784 [00:04<00:00, 411800.12 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  93%|█████████▎| 1294021/1393784 [00:05<00:00, 393956.13 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  96%|█████████▌| 1334611/1393784 [00:05<00:00, 377648.18 exampl
1: es/s]
Dropping Long Sequences (>16384) (num_proc=192):  98%|█████████▊| 1372863/1393784 [00:05<00:00, 316032.14 examples/s]
Dropping Long Sequences (>16384) (num_proc=192): 100%|██████████| 1393784/1393784 [00:06<00:00, 231110.38 examples/s]
1: 
Drop Samples with Zero Trainable Tokens (num_proc=192):   0%|          | 0/1393229 [00:00<?, ? examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   0%|          | 1000/1393229 [00:02<47:12, 491.53 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   1%|          | 11000/1393229 [00:02<03:18, 6970.35 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   1%|▏         | 18000/1393229 [00:02<01:55, 11912.62 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   2%|▏         | 28000/1393229 [00:02<01:05, 20906.68 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   3%|▎         | 37000/1393229 [00:02<00:45, 29770.35 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   3%|▎         | 45000/1393229 [00:02<00:36, 36703.64 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   4%|▍         | 53000/1393229 [00:02<00:30, 44294.03 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192): 
1:   5%|▍         | 68000/1393229 [00:02<00:20, 65465.81 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   6%|▌         | 81000/1393229 [00:02<00:16, 77719.98 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   7%|▋         | 96000/1393229 [00:03<00:13, 93758.83 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   8%|▊         | 111000/1393229 [00:03<00:12, 105999.26 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   9%|▉         | 125000/1393229 [00:03<00:11, 112335.60 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  10%|▉         | 138000/1393229 [00:03<00:11, 111640.41 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  11%|█         | 151000/1393229 [00:03<00:10, 115741.55 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  12%|█▏        | 164000/1393229 [00:03<00:11, 111226.25 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  13%|█▎        |
1:  176000/1393229 [00:03<00:10, 111815.76 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  13%|█▎        | 188000/1393229 [00:03<00:10, 112064.58 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  14%|█▍        | 201000/1393229 [00:03<00:10, 115051.80 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  15%|█▌        | 213000/1393229 [00:04<00:10, 115853.36 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  16%|█▋        | 227000/1393229 [00:04<00:09, 120328.30 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  17%|█▋        | 240000/1393229 [00:04<00:09, 116875.07 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  18%|█▊        | 257000/1393229 [00:04<00:08, 129645.60 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  20%|█▉        | 273000/1393229 [00:04<00:08, 137077.30 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  29%|██▉      
1:  | 401000/1393229 [00:04<00:02, 459096.50 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  32%|███▏      | 448000/1393229 [00:04<00:02, 369273.36 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  35%|███▌      | 489000/1393229 [00:04<00:02, 345645.25 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  38%|███▊      | 527000/1393229 [00:05<00:02, 310598.17 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  41%|████      | 568000/1393229 [00:05<00:02, 330773.08 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  43%|████▎     | 604000/1393229 [00:05<00:02, 330396.87 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  46%|████▌     | 639000/1393229 [00:05<00:02, 310045.17 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  48%|████▊     | 672000/1393229 [00:05<00:02, 309308.56 examples/s]
Drop Samples with Zero Trainable Tokens 
1: (num_proc=192):  51%|█████     | 707000/1393229 [00:05<00:02, 315409.15 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  53%|█████▎    | 740000/1393229 [00:05<00:02, 315554.41 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  56%|█████▌    | 779000/1393229 [00:05<00:01, 332093.98 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  58%|█████▊    | 813000/1393229 [00:05<00:01, 326026.69 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  61%|██████    | 851000/1393229 [00:06<00:01, 338202.60 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  64%|██████▎   | 886771/1393229 [00:06<00:01, 337775.40 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  66%|██████▌   | 921341/1393229 [00:06<00:01, 335234.50 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  70%|██████▉   | 971140/1393229 [00:06<0
1: 0:01, 381058.32 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  72%|███████▏  | 1009995/1393229 [00:06<00:01, 374131.63 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  75%|███████▌  | 1048051/1393229 [00:06<00:00, 374885.66 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  78%|███████▊  | 1085906/1393229 [00:06<00:00, 375772.01 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  81%|████████  | 1124990/1393229 [00:06<00:00, 379621.32 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  84%|████████▎ | 1165786/1393229 [00:06<00:00, 387787.37 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  86%|████████▋ | 1204653/1393229 [00:06<00:00, 362726.33 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  89%|████████▉ | 1241701/1393229 [00:07<00:00, 364080.00 examples/s]
Dro
1: p Samples with Zero Trainable Tokens (num_proc=192):  92%|█████████▏| 1279053/1393229 [00:07<00:00, 364981.42 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  95%|█████████▍| 1317893/1393229 [00:07<00:00, 371313.95 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  97%|█████████▋| 1355269/1393229 [00:07<00:00, 329874.42 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192): 100%|█████████▉| 1389693/1393229 [00:07<00:00, 245667.67 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192): 100%|██████████| 1393229/1393229 [00:08<00:00, 173446.72 examples/s]
1: 
Add position_id column (Sample Packing) (num_proc=192):   0%|          | 0/1393229 [00:00<?, ? examples/s]
Add position_id column (Sample Packing) (num_proc=192):   0%|          | 1000/1393229 [00:01<44:13, 524.71 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   1%|          | 13000/1393229 [00:02<02:37, 8789.95 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   2%|▏         | 25000/1393229 [00:02<01:12, 18783.19 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   3%|▎         | 41000/1393229 [00:02<00:39, 34650.00 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   4%|▍         | 53000/1393229 [00:02<00:28, 46363.43 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   5%|▍         | 66000/1393229 [00:02<00:22, 59924.48 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   6%|▌         | 81000/1393229 [00:02<00:17, 76355.74 examples/s]
Add position_id column (Sample Packing) (num_proc=192): 
1:   7%|▋         | 95000/1393229 [00:02<00:14, 88477.80 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   8%|▊         | 111000/1393229 [00:02<00:12, 104186.94 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  10%|▉         | 138000/1393229 [00:02<00:08, 143123.23 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  13%|█▎        | 175000/1393229 [00:02<00:06, 197682.73 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  14%|█▍        | 198000/1393229 [00:03<00:06, 199095.41 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  16%|█▋        | 228000/1393229 [00:03<00:05, 221405.35 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  18%|█▊        | 252000/1393229 [00:03<00:05, 205693.22 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  20%|██        | 280000/1393229 [00:03<00:04, 223833.24 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  22%|█
1: █▏       | 304000/1393229 [00:03<00:05, 216996.34 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  24%|██▎       | 328000/1393229 [00:03<00:04, 219425.99 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  25%|██▌       | 351000/1393229 [00:03<00:04, 218702.22 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  27%|██▋       | 374000/1393229 [00:03<00:04, 221428.86 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  29%|██▉       | 407000/1393229 [00:03<00:03, 252125.97 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  31%|███       | 433000/1393229 [00:04<00:04, 224288.92 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  33%|███▎      | 458000/1393229 [00:04<00:04, 226018.80 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  35%|███▍      | 482000/1393229 [00:04<00:03, 229140.56 examples/s]
Add position_id column (Sample Packing) (num
1: _proc=192):  36%|███▋      | 506000/1393229 [00:04<00:03, 227739.73 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  38%|███▊      | 530000/1393229 [00:04<00:03, 227394.46 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  40%|███▉      | 553000/1393229 [00:04<00:03, 223700.94 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  42%|████▏     | 581000/1393229 [00:04<00:03, 239101.38 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  44%|████▍     | 612000/1393229 [00:04<00:03, 256154.33 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  46%|████▌     | 638000/1393229 [00:04<00:03, 222486.50 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  48%|████▊     | 664000/1393229 [00:05<00:03, 230199.27 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  49%|████▉     | 688000/1393229 [00:05<00:03, 232235.42 examples/s]
Ad
1: d position_id column (Sample Packing) (num_proc=192):  51%|█████     | 713000/1393229 [00:05<00:02, 231734.68 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  53%|█████▎    | 737000/1393229 [00:05<00:02, 225406.55 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  55%|█████▍    | 760000/1393229 [00:05<00:02, 224129.97 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  57%|█████▋    | 797000/1393229 [00:05<00:02, 258210.69 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  59%|█████▉    | 823000/1393229 [00:05<00:02, 229700.29 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  61%|██████    | 847000/1393229 [00:05<00:02, 230896.36 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  63%|██████▎   | 871000/1393229 [00:05<00:02, 225915.70 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  65%|████�
1: ��█▍   | 899000/1393229 [00:06<00:02, 239417.86 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  66%|██████▋   | 924000/1393229 [00:06<00:02, 226795.91 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  68%|██████▊   | 947000/1393229 [00:06<00:02, 217707.91 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  71%|███████   | 986000/1393229 [00:06<00:01, 261497.75 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  73%|███████▎  | 1013257/1393229 [00:06<00:01, 241365.20 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  75%|███████▍  | 1038285/1393229 [00:06<00:01, 222065.38 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  77%|███████▋  | 1067570/1393229 [00:06<00:01, 239927.97 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  78%|███████▊  | 1092341/1393229 [00:06<00:01, 2402
1: 05.89 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  80%|████████  | 1117168/1393229 [00:07<00:01, 241748.77 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  82%|████████▏ | 1141738/1393229 [00:07<00:01, 238660.81 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  85%|████████▍ | 1184051/1393229 [00:07<00:00, 290021.23 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  87%|████████▋ | 1214187/1393229 [00:07<00:00, 280249.25 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  89%|████████▉ | 1242579/1393229 [00:07<00:00, 255994.12 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  91%|█████████ | 1268909/1393229 [00:07<00:00, 253265.27 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  93%|█████████▎| 1296469/1393229 [00:07<00:00, 258248.92 examples/s]
Add p
1: osition_id column (Sample Packing) (num_proc=192):  95%|█████████▍| 1322845/1393229 [00:07<00:00, 256461.70 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  97%|█████████▋| 1348965/1393229 [00:07<00:00, 241187.69 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  99%|█████████▊| 1374341/1393229 [00:08<00:00, 214450.09 examples/s]
Add position_id column (Sample Packing) (num_proc=192): 100%|██████████| 1393229/1393229 [00:08<00:00, 161319.20 examples/s]
1: 
Saving the dataset (0/192 shards):   0%|          | 0/1393229 [00:00<?, ? examples/s]
Saving the dataset (0/192 shards):   0%|          | 4000/1393229 [00:01<10:36, 2184.15 examples/s]
Saving the dataset (1/192 shards):   7%|▋         | 95257/1393229 [00:01<09:54, 2184.15 examples/s]
Saving the dataset (2/192 shards):   8%|▊         | 110771/1393229 [00:01<09:47, 2184.15 examples/s]
Saving the dataset (3/192 shards):   8%|▊         | 117028/1393229 [00:01<09:44, 2184.15 examples/s]
Saving the dataset (4/192 shards):   8%|▊         | 117028/1393229 [00:01<09:44, 2184.15 examples/s]
Saving the dataset (5/192 shards):   9%|▉         | 131542/1393229 [00:01<09:37, 2184.15 examples/s]
Saving the dataset (6/192 shards):  10%|█         | 140056/1393229 [00:01<09:33, 2184.15 examples/s]
Saving the dataset (7/192 shards):  10%|█         | 140056/1393229 [00:01<09:33, 2184.15 examples/s]
Saving the dataset (8/192 shards):  10%|█         | 143056/1393229 [00:01<09:32, 2184.15 examples/s]
Saving the data
1: set (9/192 shards):  11%|█         | 152313/1393229 [00:01<09:28, 2184.15 examples/s]
Saving the dataset (10/192 shards):  12%|█▏        | 172570/1393229 [00:01<09:18, 2184.15 examples/s]
Saving the dataset (11/192 shards):  16%|█▌        | 221341/1393229 [00:01<08:56, 2184.15 examples/s]
Saving the dataset (12/192 shards):  16%|█▌        | 223341/1393229 [00:01<08:55, 2184.15 examples/s]
Saving the dataset (13/192 shards):  17%|█▋        | 232112/1393229 [00:01<08:51, 2184.15 examples/s]
Saving the dataset (14/192 shards):  17%|█▋        | 236112/1393229 [00:01<08:49, 2184.15 examples/s]
Saving the dataset (15/192 shards):  17%|█▋        | 241112/1393229 [00:01<08:47, 2184.15 examples/s]
Saving the dataset (16/192 shards):  17%|█▋        | 241112/1393229 [00:01<08:47, 2184.15 examples/s]
Saving the dataset (17/192 shards):  19%|█▉        | 267883/1393229 [00:01<08:35, 2184.15 examples/s]
Saving the dataset (18/192 shards):  19%|█▉        | 267883/1393229 [00:01<08:35, 21
1: 84.15 examples/s]
Saving the dataset (19/192 shards):  19%|█▉        | 269883/1393229 [00:01<08:34, 2184.15 examples/s]
Saving the dataset (20/192 shards):  20%|██        | 282397/1393229 [00:01<08:28, 2184.15 examples/s]
Saving the dataset (21/192 shards):  21%|██        | 295911/1393229 [00:01<08:22, 2184.15 examples/s]
Saving the dataset (22/192 shards):  22%|██▏       | 310168/1393229 [00:01<08:15, 2184.15 examples/s]
Saving the dataset (23/192 shards):  23%|██▎       | 320682/1393229 [00:01<08:11, 2184.15 examples/s]
Saving the dataset (24/192 shards):  23%|██▎       | 320939/1393229 [00:01<08:10, 2184.15 examples/s]
Saving the dataset (25/192 shards):  23%|██▎       | 322939/1393229 [00:01<08:10, 2184.15 examples/s]
Saving the dataset (26/192 shards):  24%|██▎       | 327939/1393229 [00:01<08:07, 2184.15 examples/s]
Saving the dataset (27/192 shards):  24%|██▍       | 331939/1393229 [00:01<08:05, 2184.15 examples/s]
Saving the dataset (28/192 shards):  25%
1: |██▌       | 348453/1393229 [00:01<07:58, 2184.15 examples/s]
Saving the dataset (29/192 shards):  26%|██▌       | 358710/1393229 [00:01<07:53, 2184.15 examples/s]
Saving the dataset (30/192 shards):  26%|██▌       | 363710/1393229 [00:01<07:51, 2184.15 examples/s]
Saving the dataset (31/192 shards):  28%|██▊       | 384481/1393229 [00:01<07:41, 2184.15 examples/s]
Saving the dataset (32/192 shards):  28%|██▊       | 386738/1393229 [00:01<07:40, 2184.15 examples/s]
Saving the dataset (33/192 shards):  28%|██▊       | 394738/1393229 [00:01<07:37, 2184.15 examples/s]
Saving the dataset (34/192 shards):  29%|██▊       | 400252/1393229 [00:01<07:34, 2184.15 examples/s]
Saving the dataset (35/192 shards):  29%|██▉       | 407252/1393229 [00:01<07:31, 2184.15 examples/s]
Saving the dataset (36/192 shards):  29%|██▉       | 407252/1393229 [00:01<07:31, 2184.15 examples/s]
Saving the dataset (37/192 shards):  31%|███       | 428766/1393229 [00:01<07:21, 2184
1: .15 examples/s]
Saving the dataset (38/192 shards):  32%|███▏      | 442537/1393229 [00:01<07:15, 2184.15 examples/s]
Saving the dataset (39/192 shards):  32%|███▏      | 449794/1393229 [00:01<07:11, 2184.15 examples/s]
Saving the dataset (40/192 shards):  32%|███▏      | 449794/1393229 [00:01<07:11, 2184.15 examples/s]
Saving the dataset (41/192 shards):  33%|███▎      | 456051/1393229 [00:01<07:09, 2184.15 examples/s]
Saving the dataset (42/192 shards):  34%|███▎      | 467051/1393229 [00:01<07:04, 2184.15 examples/s]
Saving the dataset (43/192 shards):  35%|███▍      | 480822/1393229 [00:01<06:57, 2184.15 examples/s]
Saving the dataset (44/192 shards):  35%|███▍      | 485336/1393229 [00:01<06:55, 2184.15 examples/s]
Saving the dataset (45/192 shards):  35%|███▍      | 485336/1393229 [00:01<06:55, 2184.15 examples/s]
Saving the dataset (46/192 shards):  35%|███▍      | 487336/1393229 [00:01<06:54, 2184.15 examples/s]
Saving the dataset
1:  (47/192 shards):  35%|███▌      | 488593/1393229 [00:01<06:54, 2184.15 examples/s]
Saving the dataset (48/192 shards):  35%|███▌      | 490593/1393229 [00:01<06:53, 2184.15 examples/s]
Saving the dataset (49/192 shards):  36%|███▌      | 498850/1393229 [00:01<06:49, 2184.15 examples/s]
Saving the dataset (50/192 shards):  36%|███▌      | 500364/1393229 [00:01<06:48, 2184.15 examples/s]
Saving the dataset (51/192 shards):  36%|███▋      | 505621/1393229 [00:01<06:46, 2184.15 examples/s]
Saving the dataset (52/192 shards):  37%|███▋      | 514878/1393229 [00:01<06:42, 2184.15 examples/s]
Saving the dataset (53/192 shards):  38%|███▊      | 527135/1393229 [00:01<06:36, 2184.15 examples/s]
Saving the dataset (54/192 shards):  38%|███▊      | 527135/1393229 [00:01<06:36, 2184.15 examples/s]
Saving the dataset (55/192 shards):  38%|███▊      | 531135/1393229 [00:01<06:34, 2184.15 examples/s]
Saving the dataset (56/192 shards):  39%|███�
1: �      | 545649/1393229 [00:01<06:28, 2184.15 examples/s]
Saving the dataset (57/192 shards):  40%|███▉      | 551163/1393229 [00:01<06:25, 2184.15 examples/s]
Saving the dataset (58/192 shards):  40%|████      | 560420/1393229 [00:01<06:21, 2184.15 examples/s]
Saving the dataset (59/192 shards):  40%|████      | 560420/1393229 [00:01<06:21, 2184.15 examples/s]
Saving the dataset (60/192 shards):  41%|████      | 564934/1393229 [00:01<06:19, 2184.15 examples/s]
Saving the dataset (61/192 shards):  41%|████▏     | 577448/1393229 [00:01<06:13, 2184.15 examples/s]
Saving the dataset (62/192 shards):  42%|████▏     | 579962/1393229 [00:01<06:12, 2184.15 examples/s]
Saving the dataset (63/192 shards):  42%|████▏     | 584962/1393229 [00:01<06:10, 2184.15 examples/s]
Saving the dataset (64/192 shards):  42%|████▏     | 584962/1393229 [00:01<06:10, 2184.15 examples/s]
Saving the dataset (65/192 shards):  42%|████▏     | 584962/1393229 
1: [00:01<06:10, 2184.15 examples/s]
Saving the dataset (66/192 shards):  43%|████▎     | 594219/1393229 [00:01<06:05, 2184.15 examples/s]
Saving the dataset (67/192 shards):  43%|████▎     | 594476/1393229 [00:01<06:05, 2184.15 examples/s]
Saving the dataset (68/192 shards):  43%|████▎     | 594476/1393229 [00:01<06:05, 2184.15 examples/s]
Saving the dataset (69/192 shards):  43%|████▎     | 602733/1393229 [00:01<06:01, 2184.15 examples/s]
Saving the dataset (70/192 shards):  44%|████▍     | 614504/1393229 [00:01<05:56, 2184.15 examples/s]
Saving the dataset (71/192 shards):  44%|████▍     | 619504/1393229 [00:01<05:54, 2184.15 examples/s]
Saving the dataset (72/192 shards):  44%|████▍     | 619504/1393229 [00:01<05:54, 2184.15 examples/s]
Saving the dataset (73/192 shards):  45%|████▌     | 627018/1393229 [00:01<05:50, 2184.15 examples/s]
Saving the dataset (74/192 shards):  46%|████▌     | 640532/1393229 [00:01<05:44, 21
1: 84.15 examples/s]
Saving the dataset (75/192 shards):  46%|████▋     | 646532/1393229 [00:01<05:41, 2184.15 examples/s]
Saving the dataset (76/192 shards):  46%|████▋     | 646532/1393229 [00:01<05:41, 2184.15 examples/s]
Saving the dataset (77/192 shards):  47%|████▋     | 661045/1393229 [00:01<05:35, 2184.15 examples/s]
Saving the dataset (78/192 shards):  48%|████▊     | 667045/1393229 [00:01<05:32, 2184.15 examples/s]
Saving the dataset (79/192 shards):  48%|████▊     | 675301/1393229 [00:01<05:28, 2184.15 examples/s]
Saving the dataset (80/192 shards):  49%|████▉     | 685069/1393229 [00:01<05:24, 2184.15 examples/s]
Saving the dataset (81/192 shards):  49%|████▉     | 685069/1393229 [00:01<05:24, 2184.15 examples/s]
Saving the dataset (82/192 shards):  49%|████▉     | 688069/1393229 [00:01<05:22, 2184.15 examples/s]
Saving the dataset (83/192 shards):  51%|█████     | 710837/1393229 [00:01<05:12, 2184.15 examples/s
1: ]
Saving the dataset (84/192 shards):  51%|█████     | 710837/1393229 [00:01<05:12, 2184.15 examples/s]
Saving the dataset (85/192 shards):  51%|█████▏    | 717093/1393229 [00:01<05:09, 2184.15 examples/s]
Saving the dataset (86/192 shards):  52%|█████▏    | 727861/1393229 [00:01<05:04, 2184.15 examples/s]
Saving the dataset (87/192 shards):  53%|█████▎    | 738117/1393229 [00:01<04:59, 2184.15 examples/s]
Saving the dataset (88/192 shards):  53%|█████▎    | 738117/1393229 [00:01<04:59, 2184.15 examples/s]
Saving the dataset (89/192 shards):  53%|█████▎    | 741117/1393229 [00:01<04:58, 2184.15 examples/s]
Saving the dataset (90/192 shards):  53%|█████▎    | 741117/1393229 [00:01<04:58, 2184.15 examples/s]
Saving the dataset (91/192 shards):  54%|█████▍    | 758885/1393229 [00:01<04:50, 2184.15 examples/s]
Saving the dataset (92/192 shards):  55%|█████▌    | 770141/1393229 [00:01<04:45, 2184.15 examples/s
1: ]
Saving the dataset (93/192 shards):  56%|█████▌    | 774397/1393229 [00:01<04:43, 2184.15 examples/s]
Saving the dataset (94/192 shards):  56%|█████▌    | 774397/1393229 [00:01<04:43, 2184.15 examples/s]
Saving the dataset (95/192 shards):  56%|█████▌    | 775653/1393229 [00:01<04:42, 2184.15 examples/s]
Saving the dataset (96/192 shards):  56%|█████▌    | 778653/1393229 [00:01<04:41, 2184.15 examples/s]
Saving the dataset (97/192 shards):  57%|█████▋    | 798909/1393229 [00:01<04:32, 2184.15 examples/s]
Saving the dataset (98/192 shards):  59%|█████▉    | 819677/1393229 [00:01<04:22, 2184.15 examples/s]
Saving the dataset (99/192 shards):  59%|█████▉    | 819677/1393229 [00:01<04:22, 2184.15 examples/s]
Saving the dataset (100/192 shards):  59%|█████▉    | 823677/1393229 [00:01<04:20, 2184.15 examples/s]
Saving the dataset (101/192 shards):  61%|██████    | 848445/1393229 [00:01<04:09, 2184.15 exampl
1: es/s]
Saving the dataset (102/192 shards):  61%|██████    | 851445/1393229 [00:01<04:08, 2184.15 examples/s]
Saving the dataset (103/192 shards):  61%|██████▏   | 854701/1393229 [00:01<04:06, 2184.15 examples/s]
Saving the dataset (104/192 shards):  62%|██████▏   | 857701/1393229 [00:01<04:05, 2184.15 examples/s]
Saving the dataset (105/192 shards):  63%|██████▎   | 883957/1393229 [00:01<03:53, 2184.15 examples/s]
Saving the dataset (106/192 shards):  65%|██████▍   | 905469/1393229 [00:01<03:43, 2184.15 examples/s]
Saving the dataset (107/192 shards):  65%|██████▌   | 909469/1393229 [00:01<03:41, 2184.15 examples/s]
Saving the dataset (108/192 shards):  66%|██████▋   | 923981/1393229 [00:01<03:34, 2184.15 examples/s]
Saving the dataset (109/192 shards):  66%|██████▋   | 926237/1393229 [00:01<03:33, 2184.15 examples/s]
Saving the dataset (110/192 shards):  68%|██████▊   | 944005/1393229 [
1: 00:01<03:25, 2184.15 examples/s]
Saving the dataset (111/192 shards):  68%|██████▊   | 944005/1393229 [00:01<03:25, 2184.15 examples/s]
Saving the dataset (112/192 shards):  68%|██████▊   | 950261/1393229 [00:01<03:22, 2184.15 examples/s]
Saving the dataset (113/192 shards):  68%|██████▊   | 950261/1393229 [00:01<03:22, 2184.15 examples/s]
Saving the dataset (113/192 shards):  69%|██████▊   | 957261/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (114/192 shards):  70%|██████▉   | 969517/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (115/192 shards):  70%|██████▉   | 972773/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (116/192 shards):  71%|███████   | 985773/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (117/192 shards):  72%|███████▏  | 999285/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (118/192 shards):  73%|�
1: ��██████▎  | 1014053/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (119/192 shards):  73%|███████▎  | 1021053/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (120/192 shards):  74%|███████▎  | 1026309/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (121/192 shards):  74%|███████▍  | 1028309/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (122/192 shards):  75%|███████▍  | 1039565/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (123/192 shards):  75%|███████▌  | 1046565/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (124/192 shards):  76%|███████▌  | 1062333/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (125/192 shards):  76%|███████▋  | 1064589/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (126/192 shards):  76%|███████▋  | 1064589/1393229 [00:0
1: 1<00:00, 691737.78 examples/s]
Saving the dataset (127/192 shards):  78%|███████▊  | 1087101/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (128/192 shards):  78%|███████▊  | 1091613/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (129/192 shards):  78%|███████▊  | 1092869/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (130/192 shards):  79%|███████▉  | 1099381/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (131/192 shards):  79%|███████▉  | 1103381/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (132/192 shards):  79%|███████▉  | 1104637/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (133/192 shards):  79%|███████▉  | 1106637/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (134/192 shards):  79%|███████▉  | 1106637/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the datase
1: t (135/192 shards):  80%|███████▉  | 1111893/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (136/192 shards):  80%|███████▉  | 1113893/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (137/192 shards):  82%|████████▏ | 1140661/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (138/192 shards):  82%|████████▏ | 1145661/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (139/192 shards):  83%|████████▎ | 1150661/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (140/192 shards):  83%|████████▎ | 1162917/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (141/192 shards):  84%|████████▍ | 1177173/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (142/192 shards):  87%|████████▋ | 1210941/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (143/192 shards):  87%|███�
1: �████▋ | 1210941/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (144/192 shards):  88%|████████▊ | 1220453/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (145/192 shards):  88%|████████▊ | 1229477/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (146/192 shards):  88%|████████▊ | 1230733/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (147/192 shards):  88%|████████▊ | 1232733/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (148/192 shards):  89%|████████▉ | 1236989/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (149/192 shards):  89%|████████▉ | 1236989/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (150/192 shards):  89%|████████▉ | 1246501/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (151/192 shards):  89%|████████▉ | 1246501/13932
1: 29 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (152/192 shards):  89%|████████▉ | 1246501/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (153/192 shards):  89%|████████▉ | 1246501/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (154/192 shards):  90%|████████▉ | 1250501/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (155/192 shards):  91%|█████████ | 1268013/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (156/192 shards):  91%|█████████ | 1268013/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (157/192 shards):  92%|█████████▏| 1285037/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (158/192 shards):  93%|█████████▎| 1293805/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (159/192 shards):  93%|█████████▎| 1293805/1393229 [00:01<00:00, 691737.78
1:  examples/s]
Saving the dataset (160/192 shards):  93%|█████████▎| 1293805/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (161/192 shards):  93%|█████████▎| 1293805/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (162/192 shards):  93%|█████████▎| 1296061/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (163/192 shards):  94%|█████████▎| 1303317/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (164/192 shards):  94%|█████████▍| 1307573/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (165/192 shards):  94%|█████████▍| 1307829/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (166/192 shards):  94%|█████████▍| 1307829/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (167/192 shards):  95%|█████████▍| 1322085/1393229 [00:01<00:00, 691737.78 examples/s]
Sav
1: ing the dataset (168/192 shards):  95%|█████████▌| 1325853/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (169/192 shards):  95%|█████████▌| 1327109/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (170/192 shards):  95%|█████████▌| 1327109/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (171/192 shards):  95%|█████████▌| 1327109/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (172/192 shards):  96%|█████████▋| 1342109/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (173/192 shards):  97%|█████████▋| 1352133/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (174/192 shards):  97%|█████████▋| 1354645/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (175/192 shards):  97%|█████████▋| 1354645/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset 
1: (176/192 shards):  97%|█████████▋| 1354645/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (177/192 shards):  98%|█████████▊| 1359157/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (178/192 shards):  98%|█████████▊| 1359157/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (179/192 shards):  98%|█████████▊| 1359157/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (180/192 shards):  98%|█████████▊| 1359157/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (181/192 shards):  98%|█████████▊| 1362669/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (182/192 shards):  98%|█████████▊| 1366669/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (183/192 shards):  99%|█████████▉| 1377181/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (184/192 shards)
1: :  99%|█████████▉| 1377181/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (185/192 shards):  99%|█████████▉| 1381949/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (186/192 shards):  99%|█████████▉| 1385949/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (187/192 shards):  99%|█████████▉| 1385949/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (188/192 shards): 100%|█████████▉| 1390717/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (189/192 shards): 100%|█████████▉| 1390717/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (190/192 shards): 100%|█████████▉| 1390717/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (191/192 shards): 100%|██████████| 1393229/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (192/192 shards): 100%|███
1: ███████| 1393229/1393229 [00:01<00:00, 691737.78 examples/s]
Saving the dataset (192/192 shards): 100%|██████████| 1393229/1393229 [00:02<00:00, 680816.69 examples/s]
0: [2025-08-20 16:58:53,532] [INFO] [axolotl.utils.data.shared.load_preprocessed_dataset:471] [PID:195289] [RANK:0] Loading prepared dataset from disk at /lustre/fsn1/projects/rech/dgo/udv55np/dataset/Qwen3-235B-A22B/Qwen2.5-1.5B/mix_0/afb8b5e73dec9399897e1acae9582100...
0: [2025-08-20 17:00:05,609] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:435] [PID:195289] [RANK:0] gather_len_batches: [47616, 47617, 47617, 47617, 47616, 47616, 47617, 47615, 47616, 47616, 47616, 47618, 47617, 47617, 47617, 47616]
0: [2025-08-20 17:00:05,656] [INFO] [axolotl.utils.trainer.calc_sample_packing_eff_est:496] [PID:195289] [RANK:0] sample_packing_eff_est across ranks: [0.9987691640853882, 0.9987481832504272, 0.9987691640853882, 0.9987691640853882, 0.9987901449203491, 0.9988111257553101, 0.9987901449203491, 0.9987272620201111, 0.9988111257553101, 0.9987901449203491, 0.9987691640853882, 0.9987901449203491, 0.9987691640853882, 0.9987691640853882, 0.9987901449203491, 0.9987481832504272]
0: [2025-08-20 17:00:05,671] [INFO] [axolotl.utils.data.sft._prepare_standard_dataset:123] [PID:195289] [RANK:0] Maximum number of steps set at 2975
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: [2025-08-20 17:00:09,473] [INFO] [axolotl.loaders.model._configure_embedding_dtypes:317] [PID:195289] [RANK:0] Converting modules to torch.bfloat16
0: [2025-08-20 17:00:15,030] [INFO] [axolotl.train.save_initial_configs:397] [PID:195289] [RANK:0] Pre-saving tokenizer to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-1.5B/0...
0: [2025-08-20 17:00:15,209] [INFO] [axolotl.train.save_initial_configs:400] [PID:195289] [RANK:0] Pre-saving model config to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-1.5B/0...
0: [2025-08-20 17:00:15,233] [INFO] [axolotl.train.execute_training:221] [PID:195289] [RANK:0] Starting trainer...
0: [2025-08-20 17:04:19,865] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:435] [PID:195289] [RANK:0] gather_len_batches: [47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616, 47616]
0: Parameter Offload - Persistent parameters statistics: param_count = 141, numel = 144896
0: {'loss': 0.9496, 'grad_norm': 2.920503055831775, 'learning_rate': 1.0779999999999999e-06, 'epoch': 0.0}
0: 
  0%|          | 0/2975 [00:00<?, ?it/s]
  0%|          | 1/2975 [03:43<184:36:37, 223.47s/it]
  0%|          | 2/2975 [03:46<77:30:19, 93.85s/it]  
  0%|          | 3/2975 [03:47<42:36:16, 51.61s/it]
  0%|          | 4/2975 [03:49<26:09:50, 31.70s/it]
  0%|          | 5/2975 [03:49<16:58:49, 20.58s/it]
  0%|          | 6/2975 [03:50<11:26:32, 13.87s/it]
  0%|          | 7/2975 [03:51<7:55:51,  9.62s/it] 
  0%|          | 8/2975 [03:52<5:37:39,  6.83s/it]
  0%|          | 9/2975 [03:53<4:05:18,  4.96s/it]
  0%|          | 10/2975 [03:54<3:02:29,  3.69s/it]
                                                   

  0%|          | 10/2975 [03:54<3:02:29,  3.69s/it]
  0%|          | 11/2975 [03:55<2:19:41,  2.83s/it]
  0%|          | 12/2975 [03:55<1:50:08,  2.23s/it]
  0%|          | 13/2975 [03:56<1:29:32,  1.81s/it]
  0%|          | 14/2975 [03:57<1:15:21,  1.53s/it]
  1%|          | 15/2975 [03:58<1:05:17,  1.32s/it]
  1%|          | 16/2975 [03:59<58:26,  1.19s/it]  
  1%|          | 17/2975 [04:00<53:29,  1.0
0: {'loss': 0.8782, 'grad_norm': 1.7490545831327262, 'learning_rate': 1.498e-06, 'epoch': 0.01}
0: {'loss': 0.8191, 'grad_norm': 1.0007436412116684, 'learning_rate': 1.918e-06, 'epoch': 0.01}
0: 9s/it]
  1%|          | 18/2975 [04:01<50:10,  1.02s/it]
  1%|          | 19/2975 [04:01<47:49,  1.03it/s]
  1%|          | 20/2975 [04:02<46:09,  1.07it/s]
                                                 

  1%|          | 20/2975 [04:02<46:09,  1.07it/s]
  1%|          | 21/2975 [04:03<45:06,  1.09it/s]
  1%|          | 22/2975 [04:04<44:08,  1.11it/s]
  1%|          | 23/2975 [04:05<44:07,  1.11it/s]
  1%|          | 24/2975 [04:06<43:41,  1.13it/s]
  1%|          | 25/2975 [04:07<46:04,  1.07it/s]
  1%|          | 26/2975 [04:08<44:57,  1.09it/s]
  1%|          | 27/2975 [04:09<44:04,  1.11it/s]
  1%|          | 28/2975 [04:09<43:34,  1.13it/s]
  1%|          | 29/2975 [04:10<43:08,  1.14it/s]
  1%|          | 30/2975 [04:11<42:50,  1.15it/s]
                                                 

  1%|          | 30/2975 [04:11<42:50,  1.15it/s]
  1%|          | 31/2975 [04:12<42:45,  1.15it/s]
  1%|          | 32/2975 [04:13<42:31,  1.15it/s]
  1%|          | 33/2975 [04:14<42:31,  1.15it/s]
  1%|          
0: {'loss': 0.7604, 'grad_norm': 0.7258523123693066, 'learning_rate': 2.338e-06, 'epoch': 0.01}
0: {'loss': 0.727, 'grad_norm': 0.5879124801244597, 'learning_rate': 2.758e-06, 'epoch': 0.02}
0: | 34/2975 [04:15<42:25,  1.16it/s]
  1%|          | 35/2975 [04:16<42:24,  1.16it/s]
  1%|          | 36/2975 [04:16<42:22,  1.16it/s]
  1%|          | 37/2975 [04:17<42:18,  1.16it/s]
  1%|▏         | 38/2975 [04:18<42:16,  1.16it/s]
  1%|▏         | 39/2975 [04:19<42:08,  1.16it/s]
  1%|▏         | 40/2975 [04:20<42:05,  1.16it/s]
                                                 

  1%|▏         | 40/2975 [04:20<42:05,  1.16it/s]
  1%|▏         | 41/2975 [04:21<42:06,  1.16it/s]
  1%|▏         | 42/2975 [04:22<42:02,  1.16it/s]
  1%|▏         | 43/2975 [04:22<41:59,  1.16it/s]
  1%|▏         | 44/2975 [04:23<41:59,  1.16it/s]
  2%|▏         | 45/2975 [04:24<41:58,  1.16it/s]
  2%|▏         | 46/2975 [04:25<41:57,  1.16it/s]
  2%|▏         | 47/2975 [04:26<41:58,  1.16it/s]
  2%|▏         | 48/2975 [04:27<41:57,  1.16it/s]
  2%|▏         | 49/2975 [04:28<41:59,  1.16it/s]
  2%|▏         | 50/2975 [04:28<42:23,  1.15it/s]
                                                 

  2%|▏ 
0: {'loss': 0.7018, 'grad_norm': 0.5713316218735138, 'learning_rate': 3.1779999999999995e-06, 'epoch': 0.02}
0:         | 50/2975 [04:28<42:23,  1.15it/s]
  2%|▏         | 51/2975 [04:29<42:16,  1.15it/s]
  2%|▏         | 52/2975 [04:30<42:10,  1.15it/s]
  2%|▏         | 53/2975 [04:31<42:06,  1.16it/s]
  2%|▏         | 54/2975 [04:32<42:02,  1.16it/s]
  2%|▏         | 55/2975 [04:33<42:01,  1.16it/s]
  2%|▏         | 56/2975 [04:34<41:55,  1.16it/s]
  2%|▏         | 57/2975 [04:34<41:54,  1.16it/s]
  2%|▏         | 58/2975 [04:35<41:49,  1.16it/s]
  2%|▏         | 59/2975 [04:36<41:53,  1.16it/s]
  2%|▏         | 60/2975 [04:37<41:48,  1.16it/s]
                                                 

  2%|▏         | 60/2975 [04:37<41:48,  1.16it/s]
  2%|▏         | 61/2975 [04:38<41:49,  1.16it/s]
  2%|▏         | 62/2975 [04:39<41:50,  1.16it/s]
  2%|▏         | 63/2975 [04:40<41:46,  1.16it/s]
  2%|▏         | 64/2975 [04:40<41:46,  1.16it/s]
  2%|▏         | 65/2975 [04:41<41:44,  1.16it/s]
  2%|▏         | 66/2975 [04:42<41:48,  1.16it/s]
  2%|▏         | 67/2975 [04:43<41:52,  1.16
0: {'loss': 0.6978, 'grad_norm': 0.5442589702537813, 'learning_rate': 3.598e-06, 'epoch': 0.02}
0: {'loss': 0.7063, 'grad_norm': 0.559027518904327, 'learning_rate': 4.0179999999999995e-06, 'epoch': 0.03}
0: it/s]
  2%|▏         | 68/2975 [04:44<41:50,  1.16it/s]
  2%|▏         | 69/2975 [04:45<41:52,  1.16it/s]
  2%|▏         | 70/2975 [04:46<41:49,  1.16it/s]
                                                 

  2%|▏         | 70/2975 [04:46<41:49,  1.16it/s]
  2%|▏         | 71/2975 [04:47<41:47,  1.16it/s]
  2%|▏         | 72/2975 [04:47<41:44,  1.16it/s]
  2%|▏         | 73/2975 [04:49<45:06,  1.07it/s]
  2%|▏         | 74/2975 [04:49<43:56,  1.10it/s]
  3%|▎         | 75/2975 [04:50<43:14,  1.12it/s]
  3%|▎         | 76/2975 [04:51<42:39,  1.13it/s]
  3%|▎         | 77/2975 [04:52<42:22,  1.14it/s]
  3%|▎         | 78/2975 [04:53<42:02,  1.15it/s]
  3%|▎         | 79/2975 [04:54<41:50,  1.15it/s]
  3%|▎         | 80/2975 [04:55<41:40,  1.16it/s]
                                                 

  3%|▎         | 80/2975 [04:55<41:40,  1.16it/s]
  3%|▎         | 81/2975 [04:55<41:37,  1.16it/s]
  3%|▎         | 82/2975 [04:56<41:37,  1.16it/s]
  3%|▎         | 83/2975 [04:5
0: {'loss': 0.6618, 'grad_norm': 0.5558787839653073, 'learning_rate': 4.438e-06, 'epoch': 0.03}
0: 7<41:27,  1.16it/s]
  3%|▎         | 84/2975 [04:58<41:30,  1.16it/s]
  3%|▎         | 85/2975 [04:59<41:23,  1.16it/s]
  3%|▎         | 86/2975 [05:00<41:25,  1.16it/s]
  3%|▎         | 87/2975 [05:01<41:20,  1.16it/s]
  3%|▎         | 88/2975 [05:01<41:21,  1.16it/s]
  3%|▎         | 89/2975 [05:02<41:25,  1.16it/s]
  3%|▎         | 90/2975 [05:03<41:21,  1.16it/s]
                                                 

  3%|▎         | 90/2975 [05:03<41:21,  1.16it/s]
  3%|▎         | 91/2975 [05:04<41:26,  1.16it/s]
  3%|▎         | 92/2975 [05:05<41:23,  1.16it/s]
  3%|▎         | 93/2975 [05:06<41:22,  1.16it/s]
  3%|▎         | 94/2975 [05:07<41:20,  1.16it/s]
  3%|▎         | 95/2975 [05:07<41:17,  1.16it/s]
  3%|▎         | 96/2975 [05:08<41:20,  1.16it/s]
  3%|▎         | 97/2975 [05:09<41:14,  1.16it/s]
  3%|▎         | 98/2975 [05:10<41:20,  1.16it/s]
  3%|▎         | 99/2975 [05:11<41:31,  1.15it/s]
  3%|▎         | 100/2975 [05:12<42:40,  1.12it/s]
                
0: {'loss': 0.6903, 'grad_norm': 0.5605280195228449, 'learning_rate': 4.858000000000001e-06, 'epoch': 0.03}
0: {'loss': 0.6727, 'grad_norm': 0.5223783835560211, 'learning_rate': 5.278e-06, 'epoch': 0.04}
0:                                   

  3%|▎         | 100/2975 [05:12<42:40,  1.12it/s]
  3%|▎         | 101/2975 [05:13<42:20,  1.13it/s]
  3%|▎         | 102/2975 [05:14<41:57,  1.14it/s]
  3%|▎         | 103/2975 [05:14<41:46,  1.15it/s]
  3%|▎         | 104/2975 [05:15<41:33,  1.15it/s]
  4%|▎         | 105/2975 [05:16<41:28,  1.15it/s]
  4%|▎         | 106/2975 [05:17<41:28,  1.15it/s]
  4%|▎         | 107/2975 [05:18<41:36,  1.15it/s]
  4%|▎         | 108/2975 [05:19<41:26,  1.15it/s]
  4%|▎         | 109/2975 [05:20<41:25,  1.15it/s]
  4%|▎         | 110/2975 [05:20<41:13,  1.16it/s]
                                                  

  4%|▎         | 110/2975 [05:20<41:13,  1.16it/s]
  4%|▎         | 111/2975 [05:21<41:16,  1.16it/s]
  4%|▍         | 112/2975 [05:22<41:12,  1.16it/s]
  4%|▍         | 113/2975 [05:23<41:06,  1.16it/s]
  4%|▍         | 114/2975 [05:24<41:19,  1.15it/s]
  4%|▍         | 115/2975 [05:25<41:11,  1.16it/s]
  4%|▍         | 116/2975 [05:26<
0: {'loss': 0.6638, 'grad_norm': 0.5550817308699073, 'learning_rate': 5.6979999999999995e-06, 'epoch': 0.04}
0: {'loss': 0.6512, 'grad_norm': 0.5583402000941641, 'learning_rate': 6.118e-06, 'epoch': 0.04}
0: 44:34,  1.07it/s]
  4%|▍         | 117/2975 [05:27<43:33,  1.09it/s]
  4%|▍         | 118/2975 [05:28<42:43,  1.11it/s]
  4%|▍         | 119/2975 [05:28<42:08,  1.13it/s]
  4%|▍         | 120/2975 [05:29<41:43,  1.14it/s]
                                                  

  4%|▍         | 120/2975 [05:29<41:43,  1.14it/s]
  4%|▍         | 121/2975 [05:30<41:30,  1.15it/s]
  4%|▍         | 122/2975 [05:31<41:32,  1.14it/s]
  4%|▍         | 123/2975 [05:32<41:20,  1.15it/s]
  4%|▍         | 124/2975 [05:33<41:10,  1.15it/s]
  4%|▍         | 125/2975 [05:34<41:05,  1.16it/s]
  4%|▍         | 126/2975 [05:35<40:57,  1.16it/s]
  4%|▍         | 127/2975 [05:35<41:06,  1.15it/s]
  4%|▍         | 128/2975 [05:36<41:02,  1.16it/s]
  4%|▍         | 129/2975 [05:37<41:01,  1.16it/s]
  4%|▍         | 130/2975 [05:38<40:55,  1.16it/s]
                                                  

  4%|▍         | 130/2975 [05:38<40:55,  1.16it/s]
  4%|▍         | 131/2975 [05:39<40:52,  1.16it/s]
 
0: {'loss': 0.6416, 'grad_norm': 0.549344445420887, 'learning_rate': 6.538e-06, 'epoch': 0.05}
0:  4%|▍         | 132/2975 [05:40<40:52,  1.16it/s]
  4%|▍         | 133/2975 [05:41<40:50,  1.16it/s]
  5%|▍         | 134/2975 [05:41<41:10,  1.15it/s]
  5%|▍         | 135/2975 [05:42<41:06,  1.15it/s]
  5%|▍         | 136/2975 [05:43<41:06,  1.15it/s]
  5%|▍         | 137/2975 [05:44<40:58,  1.15it/s]
  5%|▍         | 138/2975 [05:45<40:53,  1.16it/s]
  5%|▍         | 139/2975 [05:46<41:03,  1.15it/s]
  5%|▍         | 140/2975 [05:47<40:53,  1.16it/s]
                                                  

  5%|▍         | 140/2975 [05:47<40:53,  1.16it/s]
  5%|▍         | 141/2975 [05:47<40:51,  1.16it/s]
  5%|▍         | 142/2975 [05:48<40:46,  1.16it/s]
  5%|▍         | 143/2975 [05:49<40:44,  1.16it/s]
  5%|▍         | 144/2975 [05:50<40:42,  1.16it/s]
  5%|▍         | 145/2975 [05:51<41:43,  1.13it/s]
  5%|▍         | 146/2975 [05:52<41:26,  1.14it/s]
  5%|▍         | 147/2975 [05:53<42:31,  1.11it/s]
  5%|▍         | 148/2975 [05:54<41:57,  1.12it/s]
  5%|▌         | 
0: {'loss': 0.6263, 'grad_norm': 0.520622967520438, 'learning_rate': 6.958e-06, 'epoch': 0.05}
0: {'loss': 0.6363, 'grad_norm': 0.5288517291756368, 'learning_rate': 7e-06, 'epoch': 0.05}
0: 149/2975 [05:55<43:49,  1.07it/s]
  5%|▌         | 150/2975 [05:56<45:06,  1.04it/s]
                                                  

  5%|▌         | 150/2975 [05:56<45:06,  1.04it/s]
  5%|▌         | 151/2975 [05:57<43:47,  1.07it/s]
  5%|▌         | 152/2975 [05:57<42:51,  1.10it/s]
  5%|▌         | 153/2975 [05:58<42:06,  1.12it/s]
  5%|▌         | 154/2975 [05:59<42:04,  1.12it/s]
  5%|▌         | 155/2975 [06:00<41:35,  1.13it/s]
  5%|▌         | 156/2975 [06:01<41:14,  1.14it/s]
  5%|▌         | 157/2975 [06:02<41:04,  1.14it/s]
  5%|▌         | 158/2975 [06:03<41:10,  1.14it/s]
  5%|▌         | 159/2975 [06:04<40:58,  1.15it/s]
  5%|▌         | 160/2975 [06:04<40:48,  1.15it/s]
                                                  

  5%|▌         | 160/2975 [06:04<40:48,  1.15it/s]
  5%|▌         | 161/2975 [06:05<40:52,  1.15it/s]
  5%|▌         | 162/2975 [06:06<40:40,  1.15it/s]
  5%|▌         | 163/2975 [06:07<40:37,  1.15it/s]
  6%|▌         | 164/2975 [06:08<40:
0: {'loss': 0.6268, 'grad_norm': 0.5082943681346523, 'learning_rate': 7e-06, 'epoch': 0.06}
0: {'loss': 0.6339, 'grad_norm': 0.5005901603221945, 'learning_rate': 7e-06, 'epoch': 0.06}
0: 36,  1.15it/s]
  6%|▌         | 165/2975 [06:09<40:29,  1.16it/s]
  6%|▌         | 166/2975 [06:10<40:24,  1.16it/s]
  6%|▌         | 167/2975 [06:10<40:18,  1.16it/s]
  6%|▌         | 168/2975 [06:11<40:18,  1.16it/s]
  6%|▌         | 169/2975 [06:12<40:19,  1.16it/s]
  6%|▌         | 170/2975 [06:13<40:18,  1.16it/s]
                                                  

  6%|▌         | 170/2975 [06:13<40:18,  1.16it/s]
  6%|▌         | 171/2975 [06:14<40:22,  1.16it/s]
  6%|▌         | 172/2975 [06:15<40:17,  1.16it/s]
  6%|▌         | 173/2975 [06:16<40:18,  1.16it/s]
  6%|▌         | 174/2975 [06:17<40:21,  1.16it/s]
  6%|▌         | 175/2975 [06:17<40:19,  1.16it/s]
  6%|▌         | 176/2975 [06:18<40:18,  1.16it/s]
  6%|▌         | 177/2975 [06:19<40:10,  1.16it/s]
  6%|▌         | 178/2975 [06:20<40:11,  1.16it/s]
  6%|▌         | 179/2975 [06:21<40:37,  1.15it/s]
  6%|▌         | 180/2975 [06:22<40:31,  1.15it/s]
                                                  

  6%
0: {'loss': 0.6381, 'grad_norm': 0.44342420470663996, 'learning_rate': 7e-06, 'epoch': 0.06}
0: |▌         | 180/2975 [06:22<40:31,  1.15it/s]
  6%|▌         | 181/2975 [06:23<40:44,  1.14it/s]
  6%|▌         | 182/2975 [06:23<40:35,  1.15it/s]
  6%|▌         | 183/2975 [06:24<40:29,  1.15it/s]
  6%|▌         | 184/2975 [06:25<40:22,  1.15it/s]
  6%|▌         | 185/2975 [06:26<40:20,  1.15it/s]
  6%|▋         | 186/2975 [06:27<40:16,  1.15it/s]
  6%|▋         | 187/2975 [06:28<40:15,  1.15it/s]
  6%|▋         | 188/2975 [06:29<41:53,  1.11it/s]
  6%|▋         | 189/2975 [06:30<41:19,  1.12it/s]
  6%|▋         | 190/2975 [06:31<40:58,  1.13it/s]
                                                  

  6%|▋         | 190/2975 [06:31<40:58,  1.13it/s]
  6%|▋         | 191/2975 [06:31<40:46,  1.14it/s]
  6%|▋         | 192/2975 [06:32<40:30,  1.14it/s]
  6%|▋         | 193/2975 [06:33<40:23,  1.15it/s]
  7%|▋         | 194/2975 [06:34<40:17,  1.15it/s]
  7%|▋         | 195/2975 [06:35<40:09,  1.15it/s]
  7%|▋         | 196/2975 [06:36<40:06,  1.15it/s]
  7%|▋         | 197
0: {'loss': 0.6089, 'grad_norm': 0.44376226300962707, 'learning_rate': 7e-06, 'epoch': 0.07}
0: {'loss': 0.624, 'grad_norm': 0.4466735489732627, 'learning_rate': 7e-06, 'epoch': 0.07}
0: /2975 [06:37<40:04,  1.16it/s]
  7%|▋         | 198/2975 [06:37<40:04,  1.16it/s]
  7%|▋         | 199/2975 [06:38<40:01,  1.16it/s]
  7%|▋         | 200/2975 [06:39<39:56,  1.16it/s]
                                                  

  7%|▋         | 200/2975 [06:39<39:56,  1.16it/s]
  7%|▋         | 201/2975 [06:40<39:56,  1.16it/s]
  7%|▋         | 202/2975 [06:41<39:52,  1.16it/s]
  7%|▋         | 203/2975 [06:42<39:51,  1.16it/s]
  7%|▋         | 204/2975 [06:43<39:52,  1.16it/s]
  7%|▋         | 205/2975 [06:43<39:48,  1.16it/s]
  7%|▋         | 206/2975 [06:44<39:50,  1.16it/s]
  7%|▋         | 207/2975 [06:45<39:47,  1.16it/s]
  7%|▋         | 208/2975 [06:46<39:47,  1.16it/s]
  7%|▋         | 209/2975 [06:47<40:17,  1.14it/s]
  7%|▋         | 210/2975 [06:48<40:06,  1.15it/s]
                                                  

  7%|▋         | 210/2975 [06:48<40:06,  1.15it/s]
  7%|▋         | 211/2975 [06:49<40:01,  1.15it/s]
  7%|▋         | 212/2975 [06:50<39:55,
0: {'loss': 0.6292, 'grad_norm': 0.423732415128663, 'learning_rate': 7e-06, 'epoch': 0.07}
0:   1.15it/s]
  7%|▋         | 213/2975 [06:50<39:49,  1.16it/s]
  7%|▋         | 214/2975 [06:51<42:16,  1.09it/s]
  7%|▋         | 215/2975 [06:52<41:28,  1.11it/s]
  7%|▋         | 216/2975 [06:53<40:54,  1.12it/s]
  7%|▋         | 217/2975 [06:54<40:33,  1.13it/s]
  7%|▋         | 218/2975 [06:55<40:18,  1.14it/s]
  7%|▋         | 219/2975 [06:56<40:06,  1.15it/s]
  7%|▋         | 220/2975 [06:57<39:55,  1.15it/s]
                                                  

  7%|▋         | 220/2975 [06:57<39:55,  1.15it/s]
  7%|▋         | 221/2975 [06:57<39:49,  1.15it/s]
  7%|▋         | 222/2975 [06:58<39:44,  1.15it/s]
  7%|▋         | 223/2975 [06:59<39:42,  1.16it/s]
  8%|▊         | 224/2975 [07:00<39:38,  1.16it/s]
  8%|▊         | 225/2975 [07:01<39:34,  1.16it/s]
  8%|▊         | 226/2975 [07:02<39:54,  1.15it/s]
  8%|▊         | 227/2975 [07:03<39:45,  1.15it/s]
  8%|▊         | 228/2975 [07:04<39:39,  1.15it/s]
  8%|▊         | 229/2975 [07:04<39:38,  1.15it/s]
  8%|�
0: {'loss': 0.6265, 'grad_norm': 0.43061783772550144, 'learning_rate': 7e-06, 'epoch': 0.08}
0: {'loss': 0.6092, 'grad_norm': 0.4034238381680657, 'learning_rate': 7e-06, 'epoch': 0.08}
0: ��         | 230/2975 [07:05<39:38,  1.15it/s]
                                                  

  8%|▊         | 230/2975 [07:05<39:38,  1.15it/s]
  8%|▊         | 231/2975 [07:06<39:35,  1.16it/s]
  8%|▊         | 232/2975 [07:07<42:19,  1.08it/s]
  8%|▊         | 233/2975 [07:08<41:22,  1.10it/s]
  8%|▊         | 234/2975 [07:09<40:49,  1.12it/s]
  8%|▊         | 235/2975 [07:10<40:23,  1.13it/s]
  8%|▊         | 236/2975 [07:11<40:07,  1.14it/s]
  8%|▊         | 237/2975 [07:12<39:53,  1.14it/s]
  8%|▊         | 238/2975 [07:12<39:44,  1.15it/s]
  8%|▊         | 239/2975 [07:13<39:37,  1.15it/s]
  8%|▊         | 240/2975 [07:14<39:31,  1.15it/s]
                                                  

  8%|▊         | 240/2975 [07:14<39:31,  1.15it/s]
  8%|▊         | 241/2975 [07:15<39:31,  1.15it/s]
  8%|▊         | 242/2975 [07:16<39:39,  1.15it/s]
  8%|▊         | 243/2975 [07:17<39:31,  1.15it/s]
  8%|▊         | 244/2975 [07:18<39:29,  1.15it/s]
  8%|▊         | 245/29
0: {'loss': 0.6223, 'grad_norm': 0.4254962672899275, 'learning_rate': 7e-06, 'epoch': 0.08}
0: {'loss': 0.6081, 'grad_norm': 0.4135340032771683, 'learning_rate': 7e-06, 'epoch': 0.09}
0: 75 [07:18<39:26,  1.15it/s]
  8%|▊         | 246/2975 [07:19<39:22,  1.15it/s]
  8%|▊         | 247/2975 [07:20<41:27,  1.10it/s]
  8%|▊         | 248/2975 [07:21<41:00,  1.11it/s]
  8%|▊         | 249/2975 [07:22<40:33,  1.12it/s]
  8%|▊         | 250/2975 [07:23<40:09,  1.13it/s]
                                                  

  8%|▊         | 250/2975 [07:23<40:09,  1.13it/s]
  8%|▊         | 251/2975 [07:24<39:53,  1.14it/s]
  8%|▊         | 252/2975 [07:25<39:38,  1.14it/s]
  9%|▊         | 253/2975 [07:26<39:29,  1.15it/s]
  9%|▊         | 254/2975 [07:26<39:24,  1.15it/s]
  9%|▊         | 255/2975 [07:27<39:21,  1.15it/s]
  9%|▊         | 256/2975 [07:28<39:16,  1.15it/s]
  9%|▊         | 257/2975 [07:29<39:25,  1.15it/s]
  9%|▊         | 258/2975 [07:30<39:20,  1.15it/s]
  9%|▊         | 259/2975 [07:31<39:14,  1.15it/s]
  9%|▊         | 260/2975 [07:32<39:14,  1.15it/s]
                                                  

  9%|▊         | 260/2975 [07:32<39:14,  1
0: {'loss': 0.5997, 'grad_norm': 0.4143035217670216, 'learning_rate': 7e-06, 'epoch': 0.09}
0: .15it/s]
  9%|▉         | 261/2975 [07:32<39:20,  1.15it/s]
  9%|▉         | 262/2975 [07:33<39:12,  1.15it/s]
  9%|▉         | 263/2975 [07:34<39:19,  1.15it/s]
  9%|▉         | 264/2975 [07:35<39:12,  1.15it/s]
  9%|▉         | 265/2975 [07:36<39:07,  1.15it/s]
  9%|▉         | 266/2975 [07:37<39:05,  1.16it/s]
  9%|▉         | 267/2975 [07:38<38:58,  1.16it/s]
  9%|▉         | 268/2975 [07:39<39:01,  1.16it/s]
  9%|▉         | 269/2975 [07:39<39:01,  1.16it/s]
  9%|▉         | 270/2975 [07:40<41:13,  1.09it/s]
                                                  

  9%|▉         | 270/2975 [07:40<41:13,  1.09it/s]
  9%|▉         | 271/2975 [07:41<40:36,  1.11it/s]
  9%|▉         | 272/2975 [07:42<40:04,  1.12it/s]
  9%|▉         | 273/2975 [07:43<39:44,  1.13it/s]
  9%|▉         | 274/2975 [07:44<39:27,  1.14it/s]
  9%|▉         | 275/2975 [07:45<39:18,  1.14it/s]
  9%|▉         | 276/2975 [07:46<39:14,  1.15it/s]
  9%|▉         | 277/2975 [07:47<39:06,  1.15it/s]
  9%|▉ 
0: {'loss': 0.615, 'grad_norm': 0.40053808126708446, 'learning_rate': 7e-06, 'epoch': 0.09}
0: {'loss': 0.6231, 'grad_norm': 0.4155865813700936, 'learning_rate': 7e-06, 'epoch': 0.1}
0:         | 278/2975 [07:47<39:05,  1.15it/s]
  9%|▉         | 279/2975 [07:48<39:03,  1.15it/s]
  9%|▉         | 280/2975 [07:49<38:57,  1.15it/s]
                                                  

  9%|▉         | 280/2975 [07:49<38:57,  1.15it/s]
  9%|▉         | 281/2975 [07:50<38:59,  1.15it/s]
  9%|▉         | 282/2975 [07:51<38:58,  1.15it/s]
 10%|▉         | 283/2975 [07:52<38:52,  1.15it/s]
 10%|▉         | 284/2975 [07:53<39:13,  1.14it/s]
 10%|▉         | 285/2975 [07:53<39:04,  1.15it/s]
 10%|▉         | 286/2975 [07:54<38:53,  1.15it/s]
 10%|▉         | 287/2975 [07:55<38:54,  1.15it/s]
 10%|▉         | 288/2975 [07:56<38:54,  1.15it/s]
 10%|▉         | 289/2975 [07:57<38:50,  1.15it/s]
 10%|▉         | 290/2975 [07:58<38:49,  1.15it/s]
                                                  

 10%|▉         | 290/2975 [07:58<38:49,  1.15it/s]
 10%|▉         | 291/2975 [07:59<38:46,  1.15it/s]
 10%|▉         | 292/2975 [08:00<39:27,  1.13it/s]
 10%|▉         | 293/2975 
0: {'loss': 0.612, 'grad_norm': 0.42033441213096595, 'learning_rate': 7e-06, 'epoch': 0.1}
0: [08:00<39:09,  1.14it/s]
 10%|▉         | 294/2975 [08:01<39:01,  1.15it/s]
 10%|▉         | 295/2975 [08:02<38:54,  1.15it/s]
 10%|▉         | 296/2975 [08:03<38:45,  1.15it/s]
 10%|▉         | 297/2975 [08:04<38:41,  1.15it/s]
 10%|█         | 298/2975 [08:05<38:36,  1.16it/s]
 10%|█         | 299/2975 [08:06<38:35,  1.16it/s]
 10%|█         | 300/2975 [08:06<38:35,  1.16it/s]
                                                  

 10%|█         | 300/2975 [08:06<38:35,  1.16it/s]
 10%|█         | 301/2975 [08:07<40:24,  1.10it/s]
 10%|█         | 302/2975 [08:08<39:47,  1.12it/s]
 10%|█         | 303/2975 [08:09<39:24,  1.13it/s]
 10%|█         | 304/2975 [08:10<39:10,  1.14it/s]
 10%|█         | 305/2975 [08:11<39:02,  1.14it/s]
 10%|█         | 306/2975 [08:12<38:55,  1.14it/s]
 10%|█         | 307/2975 [08:13<38:44,  1.15it/s]
 10%|█         | 308/2975 [08:14<38:34,  1.15it/s]
 10%|█         | 309/2975 [08:14<38:31,  1.15it/s]
 10%|█         | 310/2975 [08:15<38:26,  1.1
0: {'loss': 0.6263, 'grad_norm': 0.440252154372947, 'learning_rate': 7e-06, 'epoch': 0.1}
0: {'loss': 0.6087, 'grad_norm': 0.41134758413752537, 'learning_rate': 7e-06, 'epoch': 0.11}
0: 6it/s]
                                                  

 10%|█         | 310/2975 [08:15<38:26,  1.16it/s]
 10%|█         | 311/2975 [08:16<38:26,  1.16it/s]
 10%|█         | 312/2975 [08:17<38:23,  1.16it/s]
 11%|█         | 313/2975 [08:18<38:21,  1.16it/s]
 11%|█         | 314/2975 [08:19<38:23,  1.15it/s]
 11%|█         | 315/2975 [08:20<38:21,  1.16it/s]
 11%|█         | 316/2975 [08:20<38:17,  1.16it/s]
 11%|█         | 317/2975 [08:21<38:19,  1.16it/s]
 11%|█         | 318/2975 [08:22<38:17,  1.16it/s]
 11%|█         | 319/2975 [08:23<38:14,  1.16it/s]
 11%|█         | 320/2975 [08:24<38:14,  1.16it/s]
                                                  

 11%|█         | 320/2975 [08:24<38:14,  1.16it/s]
 11%|█         | 321/2975 [08:25<38:13,  1.16it/s]
 11%|█         | 322/2975 [08:26<38:12,  1.16it/s]
 11%|█         | 323/2975 [08:27<38:09,  1.16it/s]
 11%|█         | 324/2975 [08:27<38:08,  1.16it/s]
 11%|█         | 325/2975 [08:28<38:08,  1.16it/s]
 11%|█    
0: {'loss': 0.6179, 'grad_norm': 0.4148203215861643, 'learning_rate': 7e-06, 'epoch': 0.11}
0: {'loss': 0.6051, 'grad_norm': 0.4257472172362986, 'learning_rate': 7e-06, 'epoch': 0.11}
0:      | 326/2975 [08:29<38:04,  1.16it/s]
 11%|█         | 327/2975 [08:30<38:04,  1.16it/s]
 11%|█         | 328/2975 [08:31<38:06,  1.16it/s]
 11%|█         | 329/2975 [08:32<38:06,  1.16it/s]
 11%|█         | 330/2975 [08:33<38:05,  1.16it/s]
                                                  

 11%|█         | 330/2975 [08:33<38:05,  1.16it/s]
 11%|█         | 331/2975 [08:33<38:04,  1.16it/s]
 11%|█         | 332/2975 [08:34<38:05,  1.16it/s]
 11%|█         | 333/2975 [08:35<38:04,  1.16it/s]
 11%|█         | 334/2975 [08:36<38:03,  1.16it/s]
 11%|█▏        | 335/2975 [08:37<38:17,  1.15it/s]
 11%|█▏        | 336/2975 [08:38<38:10,  1.15it/s]
 11%|█▏        | 337/2975 [08:39<38:03,  1.16it/s]
 11%|█▏        | 338/2975 [08:39<37:59,  1.16it/s]
 11%|█▏        | 339/2975 [08:40<37:53,  1.16it/s]
 11%|█▏        | 340/2975 [08:41<37:53,  1.16it/s]
                                                  

 11%|█▏        | 340/2975 [08:41<37:53,  1.16it/s]
 11%|█▏      
0: {'loss': 0.6193, 'grad_norm': 0.402696888448364, 'learning_rate': 7e-06, 'epoch': 0.12}
0:   | 341/2975 [08:42<37:56,  1.16it/s]
 11%|█▏        | 342/2975 [08:43<37:53,  1.16it/s]
 12%|█▏        | 343/2975 [08:44<37:52,  1.16it/s]
 12%|█▏        | 344/2975 [08:45<37:49,  1.16it/s]
 12%|█▏        | 345/2975 [08:46<37:51,  1.16it/s]
 12%|█▏        | 346/2975 [08:46<37:52,  1.16it/s]
 12%|█▏        | 347/2975 [08:47<37:49,  1.16it/s]
 12%|█▏        | 348/2975 [08:48<37:47,  1.16it/s]
 12%|█▏        | 349/2975 [08:49<37:45,  1.16it/s]
 12%|█▏        | 350/2975 [08:50<37:46,  1.16it/s]
                                                  

 12%|█▏        | 350/2975 [08:50<37:46,  1.16it/s]
 12%|█▏        | 351/2975 [08:51<37:48,  1.16it/s]
 12%|█▏        | 352/2975 [08:52<37:47,  1.16it/s]
 12%|█▏        | 353/2975 [08:52<37:43,  1.16it/s]
 12%|█▏        | 354/2975 [08:53<37:43,  1.16it/s]
 12%|█▏        | 355/2975 [08:54<37:40,  1.16it/s]
 12%|█▏        | 356/2975 [08:55<37:41,  1.16it/s]
 12%|█▏        | 357/2975 [08:56<37:42,  1.16it/s]
0: {'loss': 0.6078, 'grad_norm': 0.4113573319097285, 'learning_rate': 7e-06, 'epoch': 0.12}
0: {'loss': 0.6066, 'grad_norm': 0.4061719258624911, 'learning_rate': 7e-06, 'epoch': 0.12}
0: 
 12%|█▏        | 358/2975 [08:57<37:39,  1.16it/s]
 12%|█▏        | 359/2975 [08:58<37:37,  1.16it/s]
 12%|█▏        | 360/2975 [08:59<37:49,  1.15it/s]
                                                  

 12%|█▏        | 360/2975 [08:59<37:49,  1.15it/s]
 12%|█▏        | 361/2975 [09:00<40:17,  1.08it/s]
 12%|█▏        | 362/2975 [09:00<40:01,  1.09it/s]
 12%|█▏        | 363/2975 [09:01<39:12,  1.11it/s]
 12%|█▏        | 364/2975 [09:02<38:42,  1.12it/s]
 12%|█▏        | 365/2975 [09:03<38:19,  1.13it/s]
 12%|█▏        | 366/2975 [09:04<38:22,  1.13it/s]
 12%|█▏        | 367/2975 [09:05<38:05,  1.14it/s]
 12%|█▏        | 368/2975 [09:06<37:58,  1.14it/s]
 12%|█▏        | 369/2975 [09:07<37:48,  1.15it/s]
 12%|█▏        | 370/2975 [09:07<37:44,  1.15it/s]
                                                  

 12%|█▏        | 370/2975 [09:07<37:44,  1.15it/s]
 12%|█▏        | 371/2975 [09:08<37:38,  1.15it/s]
 13%|█▎        | 372/2975 [09:09<37
0: {'loss': 0.5931, 'grad_norm': 0.45324729696496974, 'learning_rate': 7e-06, 'epoch': 0.13}
0: :33,  1.16it/s]
 13%|█▎        | 373/2975 [09:10<37:28,  1.16it/s]
 13%|█▎        | 374/2975 [09:11<37:26,  1.16it/s]
 13%|█▎        | 375/2975 [09:12<39:23,  1.10it/s]
 13%|█▎        | 376/2975 [09:13<38:48,  1.12it/s]
 13%|█▎        | 377/2975 [09:14<38:20,  1.13it/s]
 13%|█▎        | 378/2975 [09:14<38:02,  1.14it/s]
 13%|█▎        | 379/2975 [09:15<37:49,  1.14it/s]
 13%|█▎        | 380/2975 [09:16<37:39,  1.15it/s]
                                                  

 13%|█▎        | 380/2975 [09:16<37:39,  1.15it/s]
 13%|█▎        | 381/2975 [09:17<37:37,  1.15it/s]
 13%|█▎        | 382/2975 [09:18<37:33,  1.15it/s]
 13%|█▎        | 383/2975 [09:19<37:29,  1.15it/s]
 13%|█▎        | 384/2975 [09:20<37:22,  1.16it/s]
 13%|█▎        | 385/2975 [09:20<37:16,  1.16it/s]
 13%|█▎        | 386/2975 [09:21<37:16,  1.16it/s]
 13%|█▎        | 387/2975 [09:22<37:12,  1.16it/s]
 13%|█▎        | 388/2975 [09:23<37:12,  1.16it/s]
 13%|█▎        | 
0: {'loss': 0.6012, 'grad_norm': 0.4033772603246502, 'learning_rate': 7e-06, 'epoch': 0.13}
0: {'loss': 0.6086, 'grad_norm': 0.4012612645993246, 'learning_rate': 7e-06, 'epoch': 0.13}
0: 389/2975 [09:24<37:09,  1.16it/s]
 13%|█▎        | 390/2975 [09:25<37:08,  1.16it/s]
                                                  

 13%|█▎        | 390/2975 [09:25<37:08,  1.16it/s]
 13%|█▎        | 391/2975 [09:26<37:12,  1.16it/s]
 13%|█▎        | 392/2975 [09:27<37:08,  1.16it/s]
 13%|█▎        | 393/2975 [09:27<37:08,  1.16it/s]
 13%|█▎        | 394/2975 [09:28<37:07,  1.16it/s]
 13%|█▎        | 395/2975 [09:29<37:12,  1.16it/s]
 13%|█▎        | 396/2975 [09:30<37:08,  1.16it/s]
 13%|█▎        | 397/2975 [09:31<37:15,  1.15it/s]
 13%|█▎        | 398/2975 [09:32<37:12,  1.15it/s]
 13%|█▎        | 399/2975 [09:33<37:10,  1.15it/s]
 13%|█▎        | 400/2975 [09:33<37:05,  1.16it/s]
                                                  

 13%|█▎        | 400/2975 [09:33<37:05,  1.16it/s]
 13%|█▎        | 401/2975 [09:34<37:08,  1.16it/s]
 14%|█▎        | 402/2975 [09:35<37:05,  1.16it/s]
 14%|█▎        | 403/2975 [09:36<37:03,  1.16it/s]
 14%|�
0: {'loss': 0.5979, 'grad_norm': 0.40463500185296497, 'learning_rate': 7e-06, 'epoch': 0.14}
0: ��▎        | 404/2975 [09:37<37:05,  1.16it/s]
 14%|█▎        | 405/2975 [09:38<37:00,  1.16it/s]
 14%|█▎        | 406/2975 [09:39<37:02,  1.16it/s]
 14%|█▎        | 407/2975 [09:40<36:59,  1.16it/s]
 14%|█▎        | 408/2975 [09:40<36:57,  1.16it/s]
 14%|█▎        | 409/2975 [09:41<36:58,  1.16it/s]
 14%|█▍        | 410/2975 [09:42<36:53,  1.16it/s]
                                                  

 14%|█▍        | 410/2975 [09:42<36:53,  1.16it/s]
 14%|█▍        | 411/2975 [09:43<36:55,  1.16it/s]
 14%|█▍        | 412/2975 [09:44<36:55,  1.16it/s]
 14%|█▍        | 413/2975 [09:45<36:52,  1.16it/s]
 14%|█▍        | 414/2975 [09:46<36:54,  1.16it/s]
 14%|█▍        | 415/2975 [09:46<36:53,  1.16it/s]
 14%|█▍        | 416/2975 [09:47<36:52,  1.16it/s]
 14%|█▍        | 417/2975 [09:48<36:52,  1.16it/s]
 14%|█▍        | 418/2975 [09:49<36:50,  1.16it/s]
 14%|█▍        | 419/2975 [09:50<36:52,  1.16it/s]
 14%|█▍        | 420/2975 [09:51<36:51,
0: {'loss': 0.6157, 'grad_norm': 0.4117177152769887, 'learning_rate': 7e-06, 'epoch': 0.14}
0: {'loss': 0.5943, 'grad_norm': 0.40366124999099273, 'learning_rate': 7e-06, 'epoch': 0.14}
0:   1.16it/s]
                                                  

 14%|█▍        | 420/2975 [09:51<36:51,  1.16it/s]
 14%|█▍        | 421/2975 [09:52<36:48,  1.16it/s]
 14%|█▍        | 422/2975 [09:52<36:52,  1.15it/s]
 14%|█▍        | 423/2975 [09:53<36:50,  1.15it/s]
 14%|█▍        | 424/2975 [09:54<36:47,  1.16it/s]
 14%|█▍        | 425/2975 [09:55<36:47,  1.15it/s]
 14%|█▍        | 426/2975 [09:56<36:46,  1.16it/s]
 14%|█▍        | 427/2975 [09:57<37:09,  1.14it/s]
 14%|█▍        | 428/2975 [09:58<37:00,  1.15it/s]
 14%|█▍        | 429/2975 [09:59<36:55,  1.15it/s]
 14%|█▍        | 430/2975 [09:59<36:47,  1.15it/s]
                                                  

 14%|█▍        | 430/2975 [09:59<36:47,  1.15it/s]
 14%|█▍        | 431/2975 [10:00<36:45,  1.15it/s]
 15%|█▍        | 432/2975 [10:01<36:42,  1.15it/s]
 15%|█▍        | 433/2975 [10:02<36:40,  1.16it/s]
 15%|█▍        | 434/2975 [10:03<36:39,  1.16it/s]
 15%|█▍        | 435/297
0: {'loss': 0.6033, 'grad_norm': 0.41875694041629263, 'learning_rate': 7e-06, 'epoch': 0.15}
0: {'loss': 0.5929, 'grad_norm': 0.4095404677390203, 'learning_rate': 7e-06, 'epoch': 0.15}
0: 5 [10:04<36:56,  1.15it/s]
 15%|█▍        | 436/2975 [10:05<37:09,  1.14it/s]
 15%|█▍        | 437/2975 [10:06<37:15,  1.14it/s]
 15%|█▍        | 438/2975 [10:06<37:41,  1.12it/s]
 15%|█▍        | 439/2975 [10:07<37:36,  1.12it/s]
 15%|█▍        | 440/2975 [10:08<37:33,  1.12it/s]
                                                  

 15%|█▍        | 440/2975 [10:08<37:33,  1.12it/s]
 15%|█▍        | 441/2975 [10:09<37:35,  1.12it/s]
 15%|█▍        | 442/2975 [10:10<37:35,  1.12it/s]
 15%|█▍        | 443/2975 [10:11<37:35,  1.12it/s]
 15%|█▍        | 444/2975 [10:12<37:35,  1.12it/s]
 15%|█▍        | 445/2975 [10:13<37:36,  1.12it/s]
 15%|█▍        | 446/2975 [10:14<37:34,  1.12it/s]
 15%|█▌        | 447/2975 [10:15<40:14,  1.05it/s]
 15%|█▌        | 448/2975 [10:16<39:24,  1.07it/s]
 15%|█▌        | 449/2975 [10:16<38:46,  1.09it/s]
 15%|█▌        | 450/2975 [10:17<38:21,  1.10it/s]
                                                  

 15%|█▌  
0: {'loss': 0.5883, 'grad_norm': 0.40994416643433074, 'learning_rate': 7e-06, 'epoch': 0.15}
0:       | 450/2975 [10:17<38:21,  1.10it/s]
 15%|█▌        | 451/2975 [10:18<38:04,  1.10it/s]
 15%|█▌        | 452/2975 [10:19<37:55,  1.11it/s]
 15%|█▌        | 453/2975 [10:20<37:48,  1.11it/s]
 15%|█▌        | 454/2975 [10:21<37:42,  1.11it/s]
 15%|█▌        | 455/2975 [10:22<37:37,  1.12it/s]
 15%|█▌        | 456/2975 [10:23<37:30,  1.12it/s]
 15%|█▌        | 457/2975 [10:24<37:22,  1.12it/s]
 15%|█▌        | 458/2975 [10:24<37:02,  1.13it/s]
 15%|█▌        | 459/2975 [10:25<36:46,  1.14it/s]
 15%|█▌        | 460/2975 [10:26<37:21,  1.12it/s]
                                                  

 15%|█▌        | 460/2975 [10:26<37:21,  1.12it/s]
 15%|█▌        | 461/2975 [10:27<37:00,  1.13it/s]
 16%|█▌        | 462/2975 [10:28<36:43,  1.14it/s]
 16%|█▌        | 463/2975 [10:29<36:34,  1.14it/s]
 16%|█▌        | 464/2975 [10:30<36:23,  1.15it/s]
 16%|█▌        | 465/2975 [10:31<36:15,  1.15it/s]
 16%|█▌        | 466/2975 [10:31<36:13,  1.15i
0: {'loss': 0.5984, 'grad_norm': 0.4380445388040044, 'learning_rate': 7e-06, 'epoch': 0.16}
0: {'loss': 0.5919, 'grad_norm': 0.4474924553364105, 'learning_rate': 7e-06, 'epoch': 0.16}
0: t/s]
 16%|█▌        | 467/2975 [10:32<36:11,  1.15it/s]
 16%|█▌        | 468/2975 [10:33<36:09,  1.16it/s]
 16%|█▌        | 469/2975 [10:34<37:18,  1.12it/s]
 16%|█▌        | 470/2975 [10:35<36:55,  1.13it/s]
                                                  

 16%|█▌        | 470/2975 [10:35<36:55,  1.13it/s]
 16%|█▌        | 471/2975 [10:36<37:06,  1.12it/s]
 16%|█▌        | 472/2975 [10:37<36:48,  1.13it/s]
 16%|█▌        | 473/2975 [10:38<36:30,  1.14it/s]
 16%|█▌        | 474/2975 [10:38<36:25,  1.14it/s]
 16%|█▌        | 475/2975 [10:39<36:15,  1.15it/s]
 16%|█▌        | 476/2975 [10:40<36:12,  1.15it/s]
 16%|█▌        | 477/2975 [10:41<37:16,  1.12it/s]
 16%|█▌        | 478/2975 [10:42<36:54,  1.13it/s]
 16%|█▌        | 479/2975 [10:43<36:38,  1.14it/s]
 16%|█▌        | 480/2975 [10:44<36:42,  1.13it/s]
                                                  

 16%|█▌        | 480/2975 [10:44<36:42,  1.13it/s]
 16%|█▌        | 481/2975 [10:4
0: {'loss': 0.5977, 'grad_norm': 0.40903268248970576, 'learning_rate': 7e-06, 'epoch': 0.16}
0: 5<36:28,  1.14it/s]
 16%|█▌        | 482/2975 [10:46<36:17,  1.14it/s]
 16%|█▌        | 483/2975 [10:46<36:09,  1.15it/s]
 16%|█▋        | 484/2975 [10:47<36:04,  1.15it/s]
 16%|█▋        | 485/2975 [10:48<36:00,  1.15it/s]
 16%|█▋        | 486/2975 [10:49<35:54,  1.16it/s]
 16%|█▋        | 487/2975 [10:50<37:06,  1.12it/s]
 16%|█▋        | 488/2975 [10:51<36:40,  1.13it/s]
 16%|█▋        | 489/2975 [10:52<36:22,  1.14it/s]
 16%|█▋        | 490/2975 [10:53<36:10,  1.15it/s]
                                                  

 16%|█▋        | 490/2975 [10:53<36:10,  1.15it/s]
 17%|█▋        | 491/2975 [10:53<36:04,  1.15it/s]
 17%|█▋        | 492/2975 [10:54<35:57,  1.15it/s]
 17%|█▋        | 493/2975 [10:55<35:52,  1.15it/s]
 17%|█▋        | 494/2975 [10:56<35:50,  1.15it/s]
 17%|█▋        | 495/2975 [10:57<36:21,  1.14it/s]
 17%|█▋        | 496/2975 [10:58<36:09,  1.14it/s]
 17%|█▋        | 497/2975 [10:59<35:56,  1.15it/s]
 17%|█▋      
0: {'loss': 0.5926, 'grad_norm': 0.4358599301506949, 'learning_rate': 7e-06, 'epoch': 0.17}
0: {'loss': 0.5997, 'grad_norm': 0.42596292936153557, 'learning_rate': 7e-06, 'epoch': 0.17}
0:   | 498/2975 [10:59<35:48,  1.15it/s]
 17%|█▋        | 499/2975 [11:00<35:45,  1.15it/s]
 17%|█▋        | 500/2975 [11:01<35:41,  1.16it/s]
                                                  

 17%|█▋        | 500/2975 [11:01<35:41,  1.16it/s]
 17%|█▋        | 501/2975 [11:02<35:41,  1.16it/s]
 17%|█▋        | 502/2975 [11:03<36:08,  1.14it/s]
 17%|█▋        | 503/2975 [11:04<35:57,  1.15it/s]
 17%|█▋        | 504/2975 [11:05<35:49,  1.15it/s]
 17%|█▋        | 505/2975 [11:06<36:40,  1.12it/s]
 17%|█▋        | 506/2975 [11:06<36:20,  1.13it/s]
 17%|█▋        | 507/2975 [11:07<36:07,  1.14it/s]
 17%|█▋        | 508/2975 [11:08<35:55,  1.14it/s]
 17%|█▋        | 509/2975 [11:09<35:48,  1.15it/s]
 17%|█▋        | 510/2975 [11:10<37:03,  1.11it/s]
                                                  

 17%|█▋        | 510/2975 [11:10<37:03,  1.11it/s]
 17%|█▋        | 511/2975 [11:11<37:47,  1.09it/s]
 17%|█▋        | 512/2975 [11:12<37:08,  1.11it/s]
 1
0: {'loss': 0.593, 'grad_norm': 0.41141960153225726, 'learning_rate': 7e-06, 'epoch': 0.17}
0: 7%|█▋        | 513/2975 [11:13<36:37,  1.12it/s]
 17%|█▋        | 514/2975 [11:14<36:17,  1.13it/s]
 17%|█▋        | 515/2975 [11:14<35:58,  1.14it/s]
 17%|█▋        | 516/2975 [11:15<35:48,  1.14it/s]
 17%|█▋        | 517/2975 [11:16<35:42,  1.15it/s]
 17%|█▋        | 518/2975 [11:17<35:35,  1.15it/s]
 17%|█▋        | 519/2975 [11:18<35:32,  1.15it/s]
 17%|█▋        | 520/2975 [11:19<35:28,  1.15it/s]
                                                  

 17%|█▋        | 520/2975 [11:19<35:28,  1.15it/s]
 18%|█▊        | 521/2975 [11:20<35:28,  1.15it/s]
 18%|█▊        | 522/2975 [11:21<35:32,  1.15it/s]
 18%|█▊        | 523/2975 [11:21<35:27,  1.15it/s]
 18%|█▊        | 524/2975 [11:22<35:25,  1.15it/s]
 18%|█▊        | 525/2975 [11:23<35:20,  1.16it/s]
 18%|█▊        | 526/2975 [11:24<35:18,  1.16it/s]
 18%|█▊        | 527/2975 [11:25<35:17,  1.16it/s]
 18%|█▊        | 528/2975 [11:26<35:15,  1.16it/s]
 18%|█▊        | 529/2975 [11:27<35
0: {'loss': 0.6058, 'grad_norm': 0.41519939913463394, 'learning_rate': 7e-06, 'epoch': 0.18}
0: {'loss': 0.5911, 'grad_norm': 0.4220963720930229, 'learning_rate': 7e-06, 'epoch': 0.18}
0: :13,  1.16it/s]
 18%|█▊        | 530/2975 [11:27<35:10,  1.16it/s]
                                                  

 18%|█▊        | 530/2975 [11:27<35:10,  1.16it/s]
 18%|█▊        | 531/2975 [11:28<35:16,  1.15it/s]
 18%|█▊        | 532/2975 [11:29<35:15,  1.15it/s]
 18%|█▊        | 533/2975 [11:30<35:13,  1.16it/s]
 18%|█▊        | 534/2975 [11:31<35:10,  1.16it/s]
 18%|█▊        | 535/2975 [11:32<35:07,  1.16it/s]
 18%|█▊        | 536/2975 [11:33<35:07,  1.16it/s]
 18%|█▊        | 537/2975 [11:34<35:07,  1.16it/s]
 18%|█▊        | 538/2975 [11:34<35:06,  1.16it/s]
 18%|█▊        | 539/2975 [11:35<35:08,  1.16it/s]
 18%|█▊        | 540/2975 [11:36<35:08,  1.15it/s]
                                                  

 18%|█▊        | 540/2975 [11:36<35:08,  1.15it/s]
 18%|█▊        | 541/2975 [11:37<35:19,  1.15it/s]
 18%|█▊        | 542/2975 [11:38<35:15,  1.15it/s]
 18%|█▊        | 543/2975 [11:39<35:11,  1.15it/s]
 18%|█▊        | 544
0: {'loss': 0.578, 'grad_norm': 0.43396480698185175, 'learning_rate': 7e-06, 'epoch': 0.18}
0: /2975 [11:40<35:03,  1.16it/s]
 18%|█▊        | 545/2975 [11:40<34:58,  1.16it/s]
 18%|█▊        | 546/2975 [11:41<34:54,  1.16it/s]
 18%|█▊        | 547/2975 [11:42<34:55,  1.16it/s]
 18%|█▊        | 548/2975 [11:43<34:55,  1.16it/s]
 18%|█▊        | 549/2975 [11:44<34:53,  1.16it/s]
 18%|█▊        | 550/2975 [11:45<34:53,  1.16it/s]
                                                  

 18%|█▊        | 550/2975 [11:45<34:53,  1.16it/s]
 19%|█▊        | 551/2975 [11:46<34:54,  1.16it/s]
 19%|█▊        | 552/2975 [11:46<34:52,  1.16it/s]
 19%|█▊        | 553/2975 [11:48<37:17,  1.08it/s]
 19%|█▊        | 554/2975 [11:48<36:30,  1.11it/s]
 19%|█▊        | 555/2975 [11:49<35:57,  1.12it/s]
 19%|█▊        | 556/2975 [11:50<35:29,  1.14it/s]
 19%|█▊        | 557/2975 [11:51<35:16,  1.14it/s]
 19%|█▉        | 558/2975 [11:52<35:06,  1.15it/s]
 19%|█▉        | 559/2975 [11:53<34:57,  1.15it/s]
 19%|█▉        | 560/2975 [11:54<34:59,  1.15it/s]
      
0: {'loss': 0.5961, 'grad_norm': 0.43649517430280765, 'learning_rate': 7e-06, 'epoch': 0.19}
0: {'loss': 0.5958, 'grad_norm': 0.39966114618564863, 'learning_rate': 7e-06, 'epoch': 0.19}
0:                                             

 19%|█▉        | 560/2975 [11:54<34:59,  1.15it/s]
 19%|█▉        | 561/2975 [11:55<35:36,  1.13it/s]
 19%|█▉        | 562/2975 [11:55<35:20,  1.14it/s]
 19%|█▉        | 563/2975 [11:56<35:06,  1.14it/s]
 19%|█▉        | 564/2975 [11:57<34:59,  1.15it/s]
 19%|█▉        | 565/2975 [11:58<34:50,  1.15it/s]
 19%|█▉        | 566/2975 [11:59<34:47,  1.15it/s]
 19%|█▉        | 567/2975 [12:00<34:45,  1.15it/s]
 19%|█▉        | 568/2975 [12:01<34:39,  1.16it/s]
 19%|█▉        | 569/2975 [12:01<34:42,  1.16it/s]
 19%|█▉        | 570/2975 [12:02<34:37,  1.16it/s]
                                                  

 19%|█▉        | 570/2975 [12:02<34:37,  1.16it/s]
 19%|█▉        | 571/2975 [12:03<34:38,  1.16it/s]
 19%|█▉        | 572/2975 [12:04<34:39,  1.16it/s]
 19%|█▉        | 573/2975 [12:05<34:34,  1.16it/s]
 19%|█▉        | 574/2975 [12:06<34:38,  1.16it/s]
 19%|█▉        | 575/2975 [12:07<34:34,  1
0: {'loss': 0.5905, 'grad_norm': 0.3989458106912554, 'learning_rate': 7e-06, 'epoch': 0.19}
0: {'loss': 0.5875, 'grad_norm': 0.41094766369822733, 'learning_rate': 7e-06, 'epoch': 0.2}
0: .16it/s]
 19%|█▉        | 576/2975 [12:07<34:32,  1.16it/s]
 19%|█▉        | 577/2975 [12:08<34:34,  1.16it/s]
 19%|█▉        | 578/2975 [12:09<34:33,  1.16it/s]
 19%|█▉        | 579/2975 [12:10<34:30,  1.16it/s]
 19%|█▉        | 580/2975 [12:11<34:26,  1.16it/s]
                                                  

 19%|█▉        | 580/2975 [12:11<34:26,  1.16it/s]
 20%|█▉        | 581/2975 [12:12<34:26,  1.16it/s]
 20%|█▉        | 582/2975 [12:13<34:31,  1.16it/s]
 20%|█▉        | 583/2975 [12:14<34:27,  1.16it/s]
 20%|█▉        | 584/2975 [12:14<34:25,  1.16it/s]
 20%|█▉        | 585/2975 [12:15<34:28,  1.16it/s]
 20%|█▉        | 586/2975 [12:16<34:23,  1.16it/s]
 20%|█▉        | 587/2975 [12:17<34:26,  1.16it/s]
 20%|█▉        | 588/2975 [12:18<34:22,  1.16it/s]
 20%|█▉        | 589/2975 [12:19<34:18,  1.16it/s]
 20%|█▉        | 590/2975 [12:20<34:22,  1.16it/s]
                                                  

 20%|█▉        | 590/2975 [
0: {'loss': 0.5641, 'grad_norm': 0.38909436032849143, 'learning_rate': 7e-06, 'epoch': 0.2}
0: 12:20<34:22,  1.16it/s]
 20%|█▉        | 591/2975 [12:20<34:59,  1.14it/s]
 20%|█▉        | 592/2975 [12:21<34:44,  1.14it/s]
 20%|█▉        | 593/2975 [12:22<37:15,  1.07it/s]
 20%|█▉        | 594/2975 [12:23<36:20,  1.09it/s]
 20%|██        | 595/2975 [12:24<35:38,  1.11it/s]
 20%|██        | 596/2975 [12:30<1:34:03,  2.37s/it]
 20%|██        | 597/2975 [12:31<1:16:04,  1.92s/it]
 20%|██        | 598/2975 [12:32<1:03:32,  1.60s/it]
 20%|██        | 599/2975 [12:33<54:40,  1.38s/it]  
 20%|██        | 600/2975 [12:33<48:29,  1.23s/it]
                                                  

 20%|██        | 600/2975 [12:33<48:29,  1.23s/it]
 20%|██        | 601/2975 [12:34<44:11,  1.12s/it]
 20%|██        | 602/2975 [12:35<41:08,  1.04s/it]
 20%|██        | 603/2975 [12:36<39:04,  1.01it/s]
 20%|██        | 604/2975 [12:37<39:17,  1.01it/s]
 20%|██        | 605/2975 [12:38<37:43,  1.05it/s]
 20%|██        | 606/2975 [12:39<36:34,  1.08it/s]
 20%|
0: {'loss': 0.5857, 'grad_norm': 0.39835177230942787, 'learning_rate': 7e-06, 'epoch': 0.2}
0: {'loss': 0.5974, 'grad_norm': 0.396196937961162, 'learning_rate': 7e-06, 'epoch': 0.21}
0: ██        | 607/2975 [12:40<35:47,  1.10it/s]
 20%|██        | 608/2975 [12:40<35:14,  1.12it/s]
 20%|██        | 609/2975 [12:41<34:52,  1.13it/s]
 21%|██        | 610/2975 [12:42<34:34,  1.14it/s]
                                                  

 21%|██        | 610/2975 [12:42<34:34,  1.14it/s]
 21%|██        | 611/2975 [12:43<34:25,  1.14it/s]
 21%|██        | 612/2975 [12:44<34:15,  1.15it/s]
 21%|██        | 613/2975 [12:45<34:11,  1.15it/s]
 21%|██        | 614/2975 [12:46<34:22,  1.14it/s]
 21%|██        | 615/2975 [12:47<34:16,  1.15it/s]
 21%|██        | 616/2975 [12:47<34:07,  1.15it/s]
 21%|██        | 617/2975 [12:48<34:06,  1.15it/s]
 21%|██        | 618/2975 [12:49<33:59,  1.16it/s]
 21%|██        | 619/2975 [12:50<33:58,  1.16it/s]
 21%|██        | 620/2975 [12:51<34:00,  1.15it/s]
                                                  

 21%|██        | 620/2975 [12:51<34:00,  1.15it/s]
 21%|██        | 621/2975 [12:52<33:57,  
0: {'loss': 0.58, 'grad_norm': 0.38344801342431184, 'learning_rate': 7e-06, 'epoch': 0.21}
0: 1.16it/s]
 21%|██        | 622/2975 [12:53<33:55,  1.16it/s]
 21%|██        | 623/2975 [12:53<33:53,  1.16it/s]
 21%|██        | 624/2975 [12:54<33:52,  1.16it/s]
 21%|██        | 625/2975 [12:55<34:00,  1.15it/s]
 21%|██        | 626/2975 [12:56<33:56,  1.15it/s]
 21%|██        | 627/2975 [12:57<33:53,  1.15it/s]
 21%|██        | 628/2975 [12:58<33:52,  1.15it/s]
 21%|██        | 629/2975 [12:59<33:50,  1.16it/s]
 21%|██        | 630/2975 [13:00<33:48,  1.16it/s]
                                                  

 21%|██        | 630/2975 [13:00<33:48,  1.16it/s]
 21%|██        | 631/2975 [13:00<33:49,  1.15it/s]
 21%|██        | 632/2975 [13:01<33:47,  1.16it/s]
 21%|██▏       | 633/2975 [13:02<33:47,  1.15it/s]
 21%|██▏       | 634/2975 [13:03<33:44,  1.16it/s]
 21%|██▏       | 635/2975 [13:04<34:49,  1.12it/s]
 21%|██▏       | 636/2975 [13:05<36:29,  1.07it/s]
 21%|██▏       | 637/2975 [13:06<35:40,  1.09it/s]
 21%|██▏   
0: {'loss': 0.5818, 'grad_norm': 0.5046617031105433, 'learning_rate': 7e-06, 'epoch': 0.22}
0: {'loss': 0.5824, 'grad_norm': 0.4054485192825751, 'learning_rate': 7e-06, 'epoch': 0.22}
0:     | 638/2975 [13:07<35:07,  1.11it/s]
 21%|██▏       | 639/2975 [13:08<34:40,  1.12it/s]
 22%|██▏       | 640/2975 [13:09<36:13,  1.07it/s]
                                                  

 22%|██▏       | 640/2975 [13:09<36:13,  1.07it/s]
 22%|██▏       | 641/2975 [13:09<35:28,  1.10it/s]
 22%|██▏       | 642/2975 [13:10<34:55,  1.11it/s]
 22%|██▏       | 643/2975 [13:11<34:29,  1.13it/s]
 22%|██▏       | 644/2975 [13:12<34:11,  1.14it/s]
 22%|██▏       | 645/2975 [13:13<34:00,  1.14it/s]
 22%|██▏       | 646/2975 [13:14<33:50,  1.15it/s]
 22%|██▏       | 647/2975 [13:15<33:43,  1.15it/s]
 22%|██▏       | 648/2975 [13:16<33:39,  1.15it/s]
 22%|██▏       | 649/2975 [13:16<33:36,  1.15it/s]
 22%|██▏       | 650/2975 [13:17<33:35,  1.15it/s]
                                                  

 22%|██▏       | 650/2975 [13:17<33:35,  1.15it/s]
 22%|██▏       | 651/2975 [13:18<33:33,  1.15it/s]
 22%|██▏       | 65
0: {'loss': 0.5758, 'grad_norm': 0.401632317804301, 'learning_rate': 7e-06, 'epoch': 0.22}
0: 2/2975 [13:19<33:31,  1.15it/s]
 22%|██▏       | 653/2975 [13:20<33:28,  1.16it/s]
 22%|██▏       | 654/2975 [13:21<33:28,  1.16it/s]
 22%|██▏       | 655/2975 [13:22<33:26,  1.16it/s]
 22%|██▏       | 656/2975 [13:22<33:26,  1.16it/s]
 22%|██▏       | 657/2975 [13:23<33:26,  1.16it/s]
 22%|██▏       | 658/2975 [13:24<33:45,  1.14it/s]
 22%|██▏       | 659/2975 [13:25<33:38,  1.15it/s]
 22%|██▏       | 660/2975 [13:26<33:32,  1.15it/s]
                                                  

 22%|██▏       | 660/2975 [13:26<33:32,  1.15it/s]
 22%|██▏       | 661/2975 [13:27<33:28,  1.15it/s]
 22%|██▏       | 662/2975 [13:28<33:25,  1.15it/s]
 22%|██▏       | 663/2975 [13:29<35:00,  1.10it/s]
 22%|██▏       | 664/2975 [13:30<34:30,  1.12it/s]
 22%|██▏       | 665/2975 [13:30<35:06,  1.10it/s]
 22%|██▏       | 666/2975 [13:31<34:44,  1.11it/s]
 22%|██▏       | 667/2975 [13:32<34:15,  1.12it/s]
 22%|██▏       | 668/2
0: {'loss': 0.5962, 'grad_norm': 0.4206354219577991, 'learning_rate': 7e-06, 'epoch': 0.23}
0: {'loss': 0.5835, 'grad_norm': 0.39234462765749656, 'learning_rate': 7e-06, 'epoch': 0.23}
0: 975 [13:33<33:56,  1.13it/s]
 22%|██▏       | 669/2975 [13:34<33:41,  1.14it/s]
 23%|██▎       | 670/2975 [13:35<33:30,  1.15it/s]
                                                  

 23%|██▎       | 670/2975 [13:35<33:30,  1.15it/s]
 23%|██▎       | 671/2975 [13:36<33:27,  1.15it/s]
 23%|██▎       | 672/2975 [13:37<33:32,  1.14it/s]
 23%|██▎       | 673/2975 [13:37<33:24,  1.15it/s]
 23%|██▎       | 674/2975 [13:38<33:19,  1.15it/s]
 23%|██▎       | 675/2975 [13:39<33:17,  1.15it/s]
 23%|██▎       | 676/2975 [13:40<35:28,  1.08it/s]
 23%|██▎       | 677/2975 [13:41<34:44,  1.10it/s]
 23%|██▎       | 678/2975 [13:42<34:16,  1.12it/s]
 23%|██▎       | 679/2975 [13:43<33:56,  1.13it/s]
 23%|██▎       | 680/2975 [13:44<33:43,  1.13it/s]
                                                  

 23%|██▎       | 680/2975 [13:44<33:43,  1.13it/s]
 23%|██▎       | 681/2975 [13:45<33:33,  1.14it/s]
 23%|██▎       | 682/2975 [13:
0: {'loss': 0.5991, 'grad_norm': 0.43418170336510764, 'learning_rate': 7e-06, 'epoch': 0.23}
0: 45<33:27,  1.14it/s]
 23%|██▎       | 683/2975 [13:46<33:17,  1.15it/s]
 23%|██▎       | 684/2975 [13:47<33:10,  1.15it/s]
 23%|██▎       | 685/2975 [13:48<33:07,  1.15it/s]
 23%|██▎       | 686/2975 [13:49<33:05,  1.15it/s]
 23%|██▎       | 687/2975 [13:50<36:23,  1.05it/s]
 23%|██▎       | 688/2975 [13:51<35:20,  1.08it/s]
 23%|██▎       | 689/2975 [13:52<34:35,  1.10it/s]
 23%|██▎       | 690/2975 [13:53<34:08,  1.12it/s]
                                                  

 23%|██▎       | 690/2975 [13:53<34:08,  1.12it/s]
 23%|██▎       | 691/2975 [13:54<34:26,  1.11it/s]
 23%|██▎       | 692/2975 [13:54<34:00,  1.12it/s]
 23%|██▎       | 693/2975 [13:55<33:34,  1.13it/s]
 23%|██▎       | 694/2975 [13:56<33:24,  1.14it/s]
 23%|██▎       | 695/2975 [13:57<33:12,  1.14it/s]
 23%|██▎       | 696/2975 [13:58<33:04,  1.15it/s]
 23%|██▎       | 697/2975 [13:59<33:02,  1.15it/s]
 23%|██▎       | 698/2975 [14:00<
0: {'loss': 0.5839, 'grad_norm': 0.43132501271421114, 'learning_rate': 7e-06, 'epoch': 0.24}
0: {'loss': 0.5777, 'grad_norm': 0.42636237638177044, 'learning_rate': 7e-06, 'epoch': 0.24}
0: 32:52,  1.15it/s]
 23%|██▎       | 699/2975 [14:00<32:50,  1.15it/s]
 24%|██▎       | 700/2975 [14:01<32:46,  1.16it/s]
                                                  

 24%|██▎       | 700/2975 [14:01<32:46,  1.16it/s]
 24%|██▎       | 701/2975 [14:02<32:44,  1.16it/s]
 24%|██▎       | 702/2975 [14:03<32:44,  1.16it/s]
 24%|██▎       | 703/2975 [14:04<32:38,  1.16it/s]
 24%|██▎       | 704/2975 [14:05<32:41,  1.16it/s]
 24%|██▎       | 705/2975 [14:06<32:41,  1.16it/s]
 24%|██▎       | 706/2975 [14:07<32:37,  1.16it/s]
 24%|██▍       | 707/2975 [14:07<32:40,  1.16it/s]
 24%|██▍       | 708/2975 [14:08<32:34,  1.16it/s]
 24%|██▍       | 709/2975 [14:09<32:35,  1.16it/s]
 24%|██▍       | 710/2975 [14:10<32:33,  1.16it/s]
                                                  

 24%|██▍       | 710/2975 [14:10<32:33,  1.16it/s]
 24%|██▍       | 711/2975 [14:11<32:33,  1.16it/s]
 24%|██▍       | 712/2975 [14:12<32:37,  
0: {'loss': 0.5838, 'grad_norm': 0.4222659310919221, 'learning_rate': 7e-06, 'epoch': 0.24}
0: 1.16it/s]
 24%|██▍       | 713/2975 [14:13<32:37,  1.16it/s]
 24%|██▍       | 714/2975 [14:13<32:35,  1.16it/s]
 24%|██▍       | 715/2975 [14:14<32:37,  1.15it/s]
 24%|██▍       | 716/2975 [14:15<32:31,  1.16it/s]
 24%|██▍       | 717/2975 [14:16<32:32,  1.16it/s]
 24%|██▍       | 718/2975 [14:17<32:32,  1.16it/s]
 24%|██▍       | 719/2975 [14:18<32:28,  1.16it/s]
 24%|██▍       | 720/2975 [14:19<32:30,  1.16it/s]
                                                  

 24%|██▍       | 720/2975 [14:19<32:30,  1.16it/s]
 24%|██▍       | 721/2975 [14:19<32:29,  1.16it/s]
 24%|██▍       | 722/2975 [14:20<32:27,  1.16it/s]
 24%|██▍       | 723/2975 [14:21<32:28,  1.16it/s]
 24%|██▍       | 724/2975 [14:22<32:24,  1.16it/s]
 24%|██▍       | 725/2975 [14:23<32:26,  1.16it/s]
 24%|██▍       | 726/2975 [14:24<32:21,  1.16it/s]
 24%|██▍       | 727/2975 [14:25<32:18,  1.16it/s]
 24%|██▍       | 728/2975 [14:26<32:16,  1.1
0: {'loss': 0.5987, 'grad_norm': 0.40689041905902407, 'learning_rate': 7e-06, 'epoch': 0.25}
0: {'loss': 0.5883, 'grad_norm': 0.4258591166267513, 'learning_rate': 7e-06, 'epoch': 0.25}
0: 6it/s]
 25%|██▍       | 729/2975 [14:26<32:13,  1.16it/s]
 25%|██▍       | 730/2975 [14:27<32:14,  1.16it/s]
                                                  

 25%|██▍       | 730/2975 [14:27<32:14,  1.16it/s]
 25%|██▍       | 731/2975 [14:28<32:12,  1.16it/s]
 25%|██▍       | 732/2975 [14:29<32:13,  1.16it/s]
 25%|██▍       | 733/2975 [14:30<34:05,  1.10it/s]
 25%|██▍       | 734/2975 [14:31<33:34,  1.11it/s]
 25%|██▍       | 735/2975 [14:32<33:06,  1.13it/s]
 25%|██▍       | 736/2975 [14:33<32:58,  1.13it/s]
 25%|██▍       | 737/2975 [14:33<33:00,  1.13it/s]
 25%|██▍       | 738/2975 [14:34<32:46,  1.14it/s]
 25%|██▍       | 739/2975 [14:35<32:34,  1.14it/s]
 25%|██▍       | 740/2975 [14:36<32:26,  1.15it/s]
                                                  

 25%|██▍       | 740/2975 [14:36<32:26,  1.15it/s]
 25%|██▍       | 741/2975 [14:37<32:19,  1.15it/s]
 25%|██▍       | 742/2975 [14:38<32:13,  1.15it/s]
 
0: {'loss': 0.5857, 'grad_norm': 0.40899130477521906, 'learning_rate': 7e-06, 'epoch': 0.25}
0: 25%|██▍       | 743/2975 [14:39<32:11,  1.16it/s]
 25%|██▌       | 744/2975 [14:40<33:07,  1.12it/s]
 25%|██▌       | 745/2975 [14:40<32:47,  1.13it/s]
 25%|██▌       | 746/2975 [14:41<32:32,  1.14it/s]
 25%|██▌       | 747/2975 [14:42<32:23,  1.15it/s]
 25%|██▌       | 748/2975 [14:43<32:12,  1.15it/s]
 25%|██▌       | 749/2975 [14:44<32:08,  1.15it/s]
 25%|██▌       | 750/2975 [14:45<32:05,  1.16it/s]
                                                  

 25%|██▌       | 750/2975 [14:45<32:05,  1.16it/s]
 25%|██▌       | 751/2975 [14:46<32:04,  1.16it/s]
 25%|██▌       | 752/2975 [14:47<32:04,  1.15it/s]
 25%|██▌       | 753/2975 [14:47<32:02,  1.16it/s]
 25%|██▌       | 754/2975 [14:48<31:59,  1.16it/s]
 25%|██▌       | 755/2975 [14:49<31:55,  1.16it/s]
 25%|██▌       | 756/2975 [14:50<31:51,  1.16it/s]
 25%|██▌       | 757/2975 [14:51<31:53,  1.16it/s]
 25%|██▌       | 758/2975 [14:52<31:52,  1.16it/s]
 26%
0: {'loss': 0.5832, 'grad_norm': 0.41970897933487855, 'learning_rate': 7e-06, 'epoch': 0.26}
0: {'loss': 0.5815, 'grad_norm': 0.404926494807898, 'learning_rate': 7e-06, 'epoch': 0.26}
0: |██▌       | 759/2975 [14:53<31:53,  1.16it/s]
 26%|██▌       | 760/2975 [14:53<31:52,  1.16it/s]
                                                  

 26%|██▌       | 760/2975 [14:53<31:52,  1.16it/s]
 26%|██▌       | 761/2975 [14:54<31:49,  1.16it/s]
 26%|██▌       | 762/2975 [14:55<31:50,  1.16it/s]
 26%|██▌       | 763/2975 [14:56<31:49,  1.16it/s]
 26%|██▌       | 764/2975 [14:57<32:04,  1.15it/s]
 26%|██▌       | 765/2975 [14:58<32:00,  1.15it/s]
 26%|██▌       | 766/2975 [14:59<31:57,  1.15it/s]
 26%|██▌       | 767/2975 [14:59<31:50,  1.16it/s]
 26%|██▌       | 768/2975 [15:00<31:49,  1.16it/s]
 26%|██▌       | 769/2975 [15:01<31:43,  1.16it/s]
 26%|██▌       | 770/2975 [15:02<31:44,  1.16it/s]
                                                  

 26%|██▌       | 770/2975 [15:02<31:44,  1.16it/s]
 26%|██▌       | 771/2975 [15:03<31:51,  1.15it/s]
 26%|██▌       | 772/2975 [15:04<31:44,  1.16it/s]
 26%|██�
0: {'loss': 0.5966, 'grad_norm': 0.3979553147568994, 'learning_rate': 7e-06, 'epoch': 0.26}
0: ��       | 773/2975 [15:05<31:43,  1.16it/s]
 26%|██▌       | 774/2975 [15:06<31:41,  1.16it/s]
 26%|██▌       | 775/2975 [15:06<31:40,  1.16it/s]
 26%|██▌       | 776/2975 [15:07<31:39,  1.16it/s]
 26%|██▌       | 777/2975 [15:08<31:34,  1.16it/s]
 26%|██▌       | 778/2975 [15:09<31:34,  1.16it/s]
 26%|██▌       | 779/2975 [15:10<31:34,  1.16it/s]
 26%|██▌       | 780/2975 [15:11<31:33,  1.16it/s]
                                                  

 26%|██▌       | 780/2975 [15:11<31:33,  1.16it/s]
 26%|██▋       | 781/2975 [15:12<31:35,  1.16it/s]
 26%|██▋       | 782/2975 [15:12<31:29,  1.16it/s]
 26%|██▋       | 783/2975 [15:13<31:32,  1.16it/s]
 26%|██▋       | 784/2975 [15:14<31:30,  1.16it/s]
 26%|██▋       | 785/2975 [15:15<31:29,  1.16it/s]
 26%|██▋       | 786/2975 [15:16<31:28,  1.16it/s]
 26%|██▋       | 787/2975 [15:17<31:24,  1.16it/s]
 26%|██▋       | 788/2975 [15:18<31:35,  1.15it/s]
 27%|██▋ 
0: {'loss': 0.5884, 'grad_norm': 0.4215409476549146, 'learning_rate': 7e-06, 'epoch': 0.27}
0: {'loss': 0.5765, 'grad_norm': 0.41704439168393653, 'learning_rate': 7e-06, 'epoch': 0.27}
0:       | 789/2975 [15:18<31:33,  1.15it/s]
 27%|██▋       | 790/2975 [15:19<31:27,  1.16it/s]
                                                  

 27%|██▋       | 790/2975 [15:19<31:27,  1.16it/s]
 27%|██▋       | 791/2975 [15:20<31:28,  1.16it/s]
 27%|██▋       | 792/2975 [15:21<32:34,  1.12it/s]
 27%|██▋       | 793/2975 [15:22<32:13,  1.13it/s]
 27%|██▋       | 794/2975 [15:23<31:59,  1.14it/s]
 27%|██▋       | 795/2975 [15:24<31:45,  1.14it/s]
 27%|██▋       | 796/2975 [15:25<31:38,  1.15it/s]
 27%|██▋       | 797/2975 [15:25<31:32,  1.15it/s]
 27%|██▋       | 798/2975 [15:26<31:27,  1.15it/s]
 27%|██▋       | 799/2975 [15:27<31:25,  1.15it/s]
 27%|██▋       | 800/2975 [15:28<31:20,  1.16it/s]
                                                  

 27%|██▋       | 800/2975 [15:28<31:20,  1.16it/s]
 27%|██▋       | 801/2975 [15:29<31:18,  1.16it/s]
 27%|██▋       | 802/2975 [15:30<31:14,  1.16it/s]
 27%|██▋       | 
0: {'loss': 0.5868, 'grad_norm': 0.4231405745960955, 'learning_rate': 7e-06, 'epoch': 0.27}
0: 803/2975 [15:31<31:13,  1.16it/s]
 27%|██▋       | 804/2975 [15:32<31:11,  1.16it/s]
 27%|██▋       | 805/2975 [15:32<31:16,  1.16it/s]
 27%|██▋       | 806/2975 [15:33<31:11,  1.16it/s]
 27%|██▋       | 807/2975 [15:34<31:11,  1.16it/s]
 27%|██▋       | 808/2975 [15:35<31:11,  1.16it/s]
 27%|██▋       | 809/2975 [15:36<31:11,  1.16it/s]
 27%|██▋       | 810/2975 [15:37<31:08,  1.16it/s]
                                                  

 27%|██▋       | 810/2975 [15:37<31:08,  1.16it/s]
 27%|██▋       | 811/2975 [15:38<31:11,  1.16it/s]
 27%|██▋       | 812/2975 [15:38<31:10,  1.16it/s]
 27%|██▋       | 813/2975 [15:39<31:08,  1.16it/s]
 27%|██▋       | 814/2975 [15:40<31:06,  1.16it/s]
 27%|██▋       | 815/2975 [15:41<31:05,  1.16it/s]
 27%|██▋       | 816/2975 [15:42<31:03,  1.16it/s]
 27%|██▋       | 817/2975 [15:43<31:02,  1.16it/s]
 27%|██▋       | 818/2975 [15:44<31:01,  1.16it/s]
 28%|██▊       | 819
0: {'loss': 0.5788, 'grad_norm': 0.4018893834611393, 'learning_rate': 7e-06, 'epoch': 0.28}
0: {'loss': 0.5855, 'grad_norm': 0.426953014154181, 'learning_rate': 7e-06, 'epoch': 0.28}
0: /2975 [15:45<31:23,  1.14it/s]
 28%|██▊       | 820/2975 [15:45<31:15,  1.15it/s]
                                                  

 28%|██▊       | 820/2975 [15:45<31:15,  1.15it/s]
 28%|██▊       | 821/2975 [15:46<31:10,  1.15it/s]
 28%|██▊       | 822/2975 [15:47<31:03,  1.16it/s]
 28%|██▊       | 823/2975 [15:48<31:00,  1.16it/s]
 28%|██▊       | 824/2975 [15:49<30:57,  1.16it/s]
 28%|██▊       | 825/2975 [15:50<30:55,  1.16it/s]
 28%|██▊       | 826/2975 [15:51<30:52,  1.16it/s]
 28%|██▊       | 827/2975 [15:51<30:51,  1.16it/s]
 28%|██▊       | 828/2975 [15:52<31:44,  1.13it/s]
 28%|██▊       | 829/2975 [15:53<31:26,  1.14it/s]
 28%|██▊       | 830/2975 [15:54<31:15,  1.14it/s]
                                                  

 28%|██▊       | 830/2975 [15:54<31:15,  1.14it/s]
 28%|██▊       | 831/2975 [15:55<31:07,  1.15it/s]
 28%|██▊       | 832/2975 [15:56<31:01,  1.15it/s]
 28%|██▊       | 833/2975 [1
0: {'loss': 0.5833, 'grad_norm': 0.41055914477288274, 'learning_rate': 7e-06, 'epoch': 0.28}
0: 5:57<30:57,  1.15it/s]
 28%|██▊       | 834/2975 [15:58<30:52,  1.16it/s]
 28%|██▊       | 835/2975 [15:58<30:49,  1.16it/s]
 28%|██▊       | 836/2975 [15:59<30:45,  1.16it/s]
 28%|██▊       | 837/2975 [16:00<30:43,  1.16it/s]
 28%|██▊       | 838/2975 [16:01<30:43,  1.16it/s]
 28%|██▊       | 839/2975 [16:02<30:40,  1.16it/s]
 28%|██▊       | 840/2975 [16:03<30:43,  1.16it/s]
                                                  

 28%|██▊       | 840/2975 [16:03<30:43,  1.16it/s]
 28%|██▊       | 841/2975 [16:04<32:35,  1.09it/s]
 28%|██▊       | 842/2975 [16:05<32:01,  1.11it/s]
 28%|██▊       | 843/2975 [16:05<31:37,  1.12it/s]
 28%|██▊       | 844/2975 [16:06<31:19,  1.13it/s]
 28%|██▊       | 845/2975 [16:07<31:06,  1.14it/s]
 28%|██▊       | 846/2975 [16:08<32:50,  1.08it/s]
 28%|██▊       | 847/2975 [16:09<32:10,  1.10it/s]
 29%|██▊       | 848/2975 [16:10<31:39,  1.12it/s]
 29%|██▊       | 849/2975 [16:1
0: {'loss': 0.5821, 'grad_norm': 0.44483450274079234, 'learning_rate': 7e-06, 'epoch': 0.29}
0: {'loss': 0.5778, 'grad_norm': 0.41251676275326365, 'learning_rate': 7e-06, 'epoch': 0.29}
0: 1<31:19,  1.13it/s]
 29%|██▊       | 850/2975 [16:12<31:06,  1.14it/s]
                                                  

 29%|██▊       | 850/2975 [16:12<31:06,  1.14it/s]
 29%|██▊       | 851/2975 [16:13<30:59,  1.14it/s]
 29%|██▊       | 852/2975 [16:13<30:53,  1.15it/s]
 29%|██▊       | 853/2975 [16:14<31:39,  1.12it/s]
 29%|██▊       | 854/2975 [16:15<31:20,  1.13it/s]
 29%|██▊       | 855/2975 [16:16<31:06,  1.14it/s]
 29%|██▉       | 856/2975 [16:17<30:56,  1.14it/s]
 29%|██▉       | 857/2975 [16:18<30:51,  1.14it/s]
 29%|██▉       | 858/2975 [16:19<30:41,  1.15it/s]
 29%|██▉       | 859/2975 [16:20<30:40,  1.15it/s]
 29%|██▉       | 860/2975 [16:20<30:36,  1.15it/s]
                                                  

 29%|██▉       | 860/2975 [16:20<30:36,  1.15it/s]
 29%|██▉       | 861/2975 [16:21<30:32,  1.15it/s]
 29%|██▉       | 862/2975 [16:22<33:01,  1.07it/s]
 29%|██▉       | 863/2975 [16:23<33:30,
0: {'loss': 0.5936, 'grad_norm': 0.434024291684814, 'learning_rate': 7e-06, 'epoch': 0.29}
0:   1.05it/s]
 29%|██▉       | 864/2975 [16:24<32:35,  1.08it/s]
 29%|██▉       | 865/2975 [16:25<31:50,  1.10it/s]
 29%|██▉       | 866/2975 [16:26<31:25,  1.12it/s]
 29%|██▉       | 867/2975 [16:27<31:08,  1.13it/s]
 29%|██▉       | 868/2975 [16:28<30:51,  1.14it/s]
 29%|██▉       | 869/2975 [16:29<30:44,  1.14it/s]
 29%|██▉       | 870/2975 [16:29<30:34,  1.15it/s]
                                                  

 29%|██▉       | 870/2975 [16:29<30:34,  1.15it/s]
 29%|██▉       | 871/2975 [16:30<30:30,  1.15it/s]
 29%|██▉       | 872/2975 [16:31<30:29,  1.15it/s]
 29%|██▉       | 873/2975 [16:32<30:20,  1.15it/s]
 29%|██▉       | 874/2975 [16:33<30:19,  1.15it/s]
 29%|██▉       | 875/2975 [16:34<30:18,  1.15it/s]
 29%|██▉       | 876/2975 [16:35<30:15,  1.16it/s]
 29%|██▉       | 877/2975 [16:35<30:16,  1.15it/s]
 30%|██▉       | 878/2975 [16:36<30:11,  1.16it/s]
 30%|██▉       | 879/2975 [16:37<30:11,  1
0: {'loss': 0.5969, 'grad_norm': 0.4185729087473412, 'learning_rate': 7e-06, 'epoch': 0.3}
0: {'loss': 0.5825, 'grad_norm': 0.44109369482230865, 'learning_rate': 7e-06, 'epoch': 0.3}
0: .16it/s]
 30%|██▉       | 880/2975 [16:38<30:11,  1.16it/s]
                                                  

 30%|██▉       | 880/2975 [16:38<30:11,  1.16it/s]
 30%|██▉       | 881/2975 [16:39<30:09,  1.16it/s]
 30%|██▉       | 882/2975 [16:40<30:11,  1.16it/s]
 30%|██▉       | 883/2975 [16:41<30:05,  1.16it/s]
 30%|██▉       | 884/2975 [16:42<30:03,  1.16it/s]
 30%|██▉       | 885/2975 [16:42<30:03,  1.16it/s]
 30%|██▉       | 886/2975 [16:43<30:10,  1.15it/s]
 30%|██▉       | 887/2975 [16:44<30:05,  1.16it/s]
 30%|██▉       | 888/2975 [16:45<30:02,  1.16it/s]
 30%|██▉       | 889/2975 [16:46<29:59,  1.16it/s]
 30%|██▉       | 890/2975 [16:47<30:03,  1.16it/s]
                                                  

 30%|██▉       | 890/2975 [16:47<30:03,  1.16it/s]
 30%|██▉       | 891/2975 [16:48<30:00,  1.16it/s]
 30%|██▉       | 892/2975 [16:48<30:00,  1.16it/s]
 30%|███       | 893/2975 [16:49<29:59,  1.16it/s]
0: {'loss': 0.5835, 'grad_norm': 0.4243172035005087, 'learning_rate': 7e-06, 'epoch': 0.3}
0: 
 30%|███       | 894/2975 [16:50<29:53,  1.16it/s]
 30%|███       | 895/2975 [16:51<29:55,  1.16it/s]
 30%|███       | 896/2975 [16:52<29:51,  1.16it/s]
 30%|███       | 897/2975 [16:53<29:51,  1.16it/s]
 30%|███       | 898/2975 [16:54<29:54,  1.16it/s]
 30%|███       | 899/2975 [16:54<29:51,  1.16it/s]
 30%|███       | 900/2975 [16:55<29:51,  1.16it/s]
                                                  

 30%|███       | 900/2975 [16:55<29:51,  1.16it/s]
 30%|███       | 901/2975 [16:56<29:49,  1.16it/s]
 30%|███       | 902/2975 [16:57<29:49,  1.16it/s]
 30%|███       | 903/2975 [16:58<29:49,  1.16it/s]
 30%|███       | 904/2975 [16:59<29:45,  1.16it/s]
 30%|███       | 905/2975 [17:00<29:46,  1.16it/s]
 30%|███       | 906/2975 [17:01<29:42,  1.16it/s]
 30%|███       | 907/2975 [17:01<29:43,  1.16it/s]
 31%|███       | 908/2975 [17:02<29:42,  1.16it/s]
 31%|███       | 909/2975 [17:03<29:41,  1.16it/s]
 3
0: {'loss': 0.5775, 'grad_norm': 0.40319735994427935, 'learning_rate': 7e-06, 'epoch': 0.31}
0: {'loss': 0.5741, 'grad_norm': 0.4154611354833376, 'learning_rate': 7e-06, 'epoch': 0.31}
0: 1%|███       | 910/2975 [17:04<29:40,  1.16it/s]
                                                  

 31%|███       | 910/2975 [17:04<29:40,  1.16it/s]
 31%|███       | 911/2975 [17:05<29:39,  1.16it/s]
 31%|███       | 912/2975 [17:06<29:39,  1.16it/s]
 31%|███       | 913/2975 [17:07<29:38,  1.16it/s]
 31%|███       | 914/2975 [17:07<29:36,  1.16it/s]
 31%|███       | 915/2975 [17:08<29:35,  1.16it/s]
 31%|███       | 916/2975 [17:09<29:32,  1.16it/s]
 31%|███       | 917/2975 [17:10<29:31,  1.16it/s]
 31%|███       | 918/2975 [17:11<29:30,  1.16it/s]
 31%|███       | 919/2975 [17:12<29:31,  1.16it/s]
 31%|███       | 920/2975 [17:13<29:32,  1.16it/s]
                                                  

 31%|███       | 920/2975 [17:13<29:32,  1.16it/s]
 31%|███       | 921/2975 [17:13<29:31,  1.16it/s]
 31%|███       | 922/2975 [17:14<29:32,  1.16it/s]
 31%|███       | 923/2975 [17:15<29:32,  1.16it/s]
 31%|█�
0: {'loss': 0.5729, 'grad_norm': 0.4116668974557861, 'learning_rate': 7e-06, 'epoch': 0.31}
0: �█       | 924/2975 [17:16<29:31,  1.16it/s]
 31%|███       | 925/2975 [17:17<29:30,  1.16it/s]
 31%|███       | 926/2975 [17:18<29:43,  1.15it/s]
 31%|███       | 927/2975 [17:19<29:37,  1.15it/s]
 31%|███       | 928/2975 [17:20<29:33,  1.15it/s]
 31%|███       | 929/2975 [17:20<29:29,  1.16it/s]
 31%|███▏      | 930/2975 [17:21<29:30,  1.16it/s]
                                                  

 31%|███▏      | 930/2975 [17:21<29:30,  1.16it/s]
 31%|███▏      | 931/2975 [17:22<29:30,  1.15it/s]
 31%|███▏      | 932/2975 [17:23<29:28,  1.16it/s]
 31%|███▏      | 933/2975 [17:24<29:27,  1.16it/s]
 31%|███▏      | 934/2975 [17:25<29:25,  1.16it/s]
 31%|███▏      | 935/2975 [17:26<29:25,  1.16it/s]
 31%|███▏      | 936/2975 [17:26<29:23,  1.16it/s]
 31%|███▏      | 937/2975 [17:27<29:22,  1.16it/s]
 32%|███▏      | 938/2975 [17:28<29:21,  1.16it/s]
 32%|███▏      | 939/2975 [17:29<29:17,  1
0: {'loss': 0.573, 'grad_norm': 0.4181821215208397, 'learning_rate': 7e-06, 'epoch': 0.32}
0: {'loss': 0.576, 'grad_norm': 0.41793029917876107, 'learning_rate': 7e-06, 'epoch': 0.32}
0: .16it/s]
 32%|███▏      | 940/2975 [17:30<29:16,  1.16it/s]
                                                  

 32%|███▏      | 940/2975 [17:30<29:16,  1.16it/s]
 32%|███▏      | 941/2975 [17:31<29:16,  1.16it/s]
 32%|███▏      | 942/2975 [17:32<29:15,  1.16it/s]
 32%|███▏      | 943/2975 [17:32<29:17,  1.16it/s]
 32%|███▏      | 944/2975 [17:33<29:14,  1.16it/s]
 32%|███▏      | 945/2975 [17:34<29:13,  1.16it/s]
 32%|███▏      | 946/2975 [17:35<29:11,  1.16it/s]
 32%|███▏      | 947/2975 [17:36<29:08,  1.16it/s]
 32%|███▏      | 948/2975 [17:37<29:09,  1.16it/s]
 32%|███▏      | 949/2975 [17:38<29:07,  1.16it/s]
 32%|███▏      | 950/2975 [17:39<29:08,  1.16it/s]
                                                  

 32%|███▏      | 950/2975 [17:39<29:08,  1.16it/s]
 32%|███▏      | 951/2975 [17:39<29:07,  1.16it/s]
 32%|███▏      | 952/2975 [17:40<29:06,  1.16it/s]
 32%|███▏      | 9
0: {'loss': 0.5731, 'grad_norm': 0.3988703063356054, 'learning_rate': 7e-06, 'epoch': 0.32}
0: 53/2975 [17:41<29:05,  1.16it/s]
 32%|███▏      | 954/2975 [17:42<29:03,  1.16it/s]
 32%|███▏      | 955/2975 [17:43<29:02,  1.16it/s]
 32%|███▏      | 956/2975 [17:44<29:01,  1.16it/s]
 32%|███▏      | 957/2975 [17:45<28:59,  1.16it/s]
 32%|███▏      | 958/2975 [17:45<28:58,  1.16it/s]
 32%|███▏      | 959/2975 [17:46<28:55,  1.16it/s]
 32%|███▏      | 960/2975 [17:47<28:56,  1.16it/s]
                                                  

 32%|███▏      | 960/2975 [17:47<28:56,  1.16it/s]
 32%|███▏      | 961/2975 [17:48<28:58,  1.16it/s]
 32%|███▏      | 962/2975 [17:49<28:55,  1.16it/s]
 32%|███▏      | 963/2975 [17:50<28:55,  1.16it/s]
 32%|███▏      | 964/2975 [17:51<28:55,  1.16it/s]
 32%|███▏      | 965/2975 [17:51<28:55,  1.16it/s]
 32%|███▏      | 966/2975 [17:52<28:55,  1.16it/s]
 33%|███▎      | 967/2975 [17:53<28:52,  1.16it/s]
 33%|███▎      | 968/2975 [17:54<28:53,  1.16i
0: {'loss': 0.5796, 'grad_norm': 0.4156612201341967, 'learning_rate': 7e-06, 'epoch': 0.33}
0: {'loss': 0.5757, 'grad_norm': 0.415565344808143, 'learning_rate': 7e-06, 'epoch': 0.33}
0: t/s]
 33%|███▎      | 969/2975 [17:55<28:50,  1.16it/s]
 33%|███▎      | 970/2975 [17:56<28:49,  1.16it/s]
                                                  

 33%|███▎      | 970/2975 [17:56<28:49,  1.16it/s]
 33%|███▎      | 971/2975 [17:57<28:53,  1.16it/s]
 33%|███▎      | 972/2975 [17:58<28:49,  1.16it/s]
 33%|███▎      | 973/2975 [17:58<28:48,  1.16it/s]
 33%|███▎      | 974/2975 [17:59<28:48,  1.16it/s]
 33%|███▎      | 975/2975 [18:00<28:45,  1.16it/s]
 33%|███▎      | 976/2975 [18:01<28:45,  1.16it/s]
 33%|███▎      | 977/2975 [18:02<28:54,  1.15it/s]
 33%|███▎      | 978/2975 [18:03<28:50,  1.15it/s]
 33%|███▎      | 979/2975 [18:04<28:49,  1.15it/s]
 33%|███▎      | 980/2975 [18:04<28:45,  1.16it/s]
                                                  

 33%|███▎      | 980/2975 [18:04<28:45,  1.16it/s]
 33%|███▎      | 981/2975 [18:05<28:47,  1.15it/s]
 33%|███▎      | 982/2
0: {'loss': 0.5769, 'grad_norm': 0.4327133173008329, 'learning_rate': 7e-06, 'epoch': 0.33}
0: 975 [18:06<28:43,  1.16it/s]
 33%|███▎      | 983/2975 [18:07<28:41,  1.16it/s]
 33%|███▎      | 984/2975 [18:08<28:40,  1.16it/s]
 33%|███▎      | 985/2975 [18:09<28:37,  1.16it/s]
 33%|███▎      | 986/2975 [18:10<28:36,  1.16it/s]
 33%|███▎      | 987/2975 [18:10<28:34,  1.16it/s]
 33%|███▎      | 988/2975 [18:11<28:35,  1.16it/s]
 33%|███▎      | 989/2975 [18:12<28:33,  1.16it/s]
 33%|███▎      | 990/2975 [18:13<28:33,  1.16it/s]
                                                  

 33%|███▎      | 990/2975 [18:13<28:33,  1.16it/s]
 33%|███▎      | 991/2975 [18:14<28:34,  1.16it/s]
 33%|███▎      | 992/2975 [18:15<28:32,  1.16it/s]
 33%|███▎      | 993/2975 [18:16<28:32,  1.16it/s]
 33%|███▎      | 994/2975 [18:17<28:34,  1.16it/s]
 33%|███▎      | 995/2975 [18:17<28:29,  1.16it/s]
 33%|███▎      | 996/2975 [18:18<28:26,  1.16it/s]
 34%|███▎      | 997/2975 [18:19<28:24,  1.16it/s]
0: {'loss': 0.5732, 'grad_norm': 0.4077596106130701, 'learning_rate': 7e-06, 'epoch': 0.34}
0: {'loss': 0.5672, 'grad_norm': 0.40293282965169264, 'learning_rate': 7e-06, 'epoch': 0.34}
0: 
 34%|███▎      | 998/2975 [18:20<28:22,  1.16it/s]
 34%|███▎      | 999/2975 [18:21<28:23,  1.16it/s]
 34%|███▎      | 1000/2975 [18:22<28:22,  1.16it/s]
                                                   

 34%|███▎      | 1000/2975 [18:22<28:22,  1.16it/s]
 34%|███▎      | 1001/2975 [18:23<28:35,  1.15it/s]
 34%|███▎      | 1002/2975 [18:23<28:31,  1.15it/s]
 34%|███▎      | 1003/2975 [18:24<28:52,  1.14it/s]
 34%|███▎      | 1004/2975 [18:25<28:40,  1.15it/s]
 34%|███▍      | 1005/2975 [18:26<28:32,  1.15it/s]
 34%|███▍      | 1006/2975 [18:27<28:26,  1.15it/s]
 34%|███▍      | 1007/2975 [18:28<28:23,  1.15it/s]
 34%|███▍      | 1008/2975 [18:29<28:22,  1.16it/s]
 34%|███▍      | 1009/2975 [18:30<28:18,  1.16it/s]
 34%|███▍      | 1010/2975 [18:30<28:15,  1.16it/s]
                                                   

 34%|███▍      | 1010/2975 [18:30<28:15,  1.16it/s]
 34%|███▍  
0: {'loss': 0.5756, 'grad_norm': 0.41043301517347724, 'learning_rate': 7e-06, 'epoch': 0.34}
0:     | 1011/2975 [18:31<28:45,  1.14it/s]
 34%|███▍      | 1012/2975 [18:32<28:34,  1.15it/s]
 34%|███▍      | 1013/2975 [18:33<28:29,  1.15it/s]
 34%|███▍      | 1014/2975 [18:34<28:24,  1.15it/s]
 34%|███▍      | 1015/2975 [18:35<28:19,  1.15it/s]
 34%|███▍      | 1016/2975 [18:36<28:18,  1.15it/s]
 34%|███▍      | 1017/2975 [18:36<28:15,  1.15it/s]
 34%|███▍      | 1018/2975 [18:37<28:12,  1.16it/s]
 34%|███▍      | 1019/2975 [18:38<28:11,  1.16it/s]
 34%|███▍      | 1020/2975 [18:39<28:09,  1.16it/s]
                                                   

 34%|███▍      | 1020/2975 [18:39<28:09,  1.16it/s]
 34%|███▍      | 1021/2975 [18:40<28:10,  1.16it/s]
 34%|███▍      | 1022/2975 [18:41<28:08,  1.16it/s]
 34%|███▍      | 1023/2975 [18:42<28:05,  1.16it/s]
 34%|███▍      | 1024/2975 [18:43<28:05,  1.16it/s]
 34%|███▍      | 1025/2975 [18:43<28:02,  1.16it/s]
 34%|███▍      | 1026/
0: {'loss': 0.5791, 'grad_norm': 0.41389530665926955, 'learning_rate': 7e-06, 'epoch': 0.35}
0: 2975 [18:44<28:00,  1.16it/s]
 35%|███▍      | 1027/2975 [18:45<28:01,  1.16it/s]
 35%|███▍      | 1028/2975 [18:46<28:01,  1.16it/s]
 35%|███▍      | 1029/2975 [18:47<28:01,  1.16it/s]
 35%|███▍      | 1030/2975 [18:48<28:01,  1.16it/s]
                                                   

 35%|███▍      | 1030/2975 [18:48<28:01,  1.16it/s]
 35%|███▍      | 1031/2975 [18:49<28:02,  1.16it/s]
 35%|███▍      | 1032/2975 [18:49<28:01,  1.16it/s]
 35%|███▍      | 1033/2975 [18:50<27:58,  1.16it/s]
 35%|███▍      | 1034/2975 [18:51<27:58,  1.16it/s]
 35%|███▍      | 1035/2975 [18:52<27:55,  1.16it/s]
 35%|███▍      | 1036/2975 [18:53<27:55,  1.16it/s]
 35%|███▍      | 1037/2975 [18:54<27:56,  1.16it/s]
 35%|███▍      | 1038/2975 [18:55<27:56,  1.16it/s]
 35%|███▍      | 1039/2975 [18:55<27:55,  1.16it/s]
 35%|███▍      | 1040/2975 [18:56<27:52,  1.16it/s]
                                         
0: {'loss': 0.5786, 'grad_norm': 0.4271922396867151, 'learning_rate': 7e-06, 'epoch': 0.35}
0: {'loss': 0.5854, 'grad_norm': 0.40952397601830853, 'learning_rate': 7e-06, 'epoch': 0.35}
0:           

 35%|███▍      | 1040/2975 [18:56<27:52,  1.16it/s]
 35%|███▍      | 1041/2975 [18:57<27:52,  1.16it/s]
 35%|███▌      | 1042/2975 [18:58<27:51,  1.16it/s]
 35%|███▌      | 1043/2975 [18:59<27:48,  1.16it/s]
 35%|███▌      | 1044/2975 [19:00<27:48,  1.16it/s]
 35%|███▌      | 1045/2975 [19:01<27:43,  1.16it/s]
 35%|███▌      | 1046/2975 [19:02<27:42,  1.16it/s]
 35%|███▌      | 1047/2975 [19:02<27:44,  1.16it/s]
 35%|███▌      | 1048/2975 [19:03<27:42,  1.16it/s]
 35%|███▌      | 1049/2975 [19:04<27:43,  1.16it/s]
 35%|███▌      | 1050/2975 [19:05<27:40,  1.16it/s]
                                                   

 35%|███▌      | 1050/2975 [19:05<27:40,  1.16it/s]
 35%|███▌      | 1051/2975 [19:06<27:39,  1.16it/s]
 35%|███▌      | 1052/2975 [19:07<27:40,  1.16it/s]
 35%|███▌      | 1053/2975 [19:08<27:38,  1.16it/s]
 35%|███▌      | 1054/2975 [19:08<27:39,  1.16it/s]
0: {'loss': 0.5896, 'grad_norm': 0.4124543509760127, 'learning_rate': 7e-06, 'epoch': 0.36}
0: 
 35%|███▌      | 1055/2975 [19:09<27:37,  1.16it/s]
 35%|███▌      | 1056/2975 [19:10<27:35,  1.16it/s]
 36%|███▌      | 1057/2975 [19:11<29:52,  1.07it/s]
 36%|███▌      | 1058/2975 [19:12<29:12,  1.09it/s]
 36%|███▌      | 1059/2975 [19:13<28:41,  1.11it/s]
 36%|███▌      | 1060/2975 [19:14<28:20,  1.13it/s]
                                                   

 36%|███▌      | 1060/2975 [19:14<28:20,  1.13it/s]
 36%|███▌      | 1061/2975 [19:15<28:07,  1.13it/s]
 36%|███▌      | 1062/2975 [19:16<28:26,  1.12it/s]
 36%|███▌      | 1063/2975 [19:17<28:06,  1.13it/s]
 36%|███▌      | 1064/2975 [19:17<27:55,  1.14it/s]
 36%|███▌      | 1065/2975 [19:18<29:14,  1.09it/s]
 36%|███▌      | 1066/2975 [19:19<28:45,  1.11it/s]
 36%|███▌      | 1067/2975 [19:20<28:20,  1.12it/s]
 36%|███▌      | 1068/2975 [19:21<28:04,  1.13it/s]
 36%|███▌      | 1069/2975 [19:22<27:54,  1.14it/s]
 36%|█�
0: {'loss': 0.5816, 'grad_norm': 0.41243450166257145, 'learning_rate': 7e-06, 'epoch': 0.36}
0: {'loss': 0.5817, 'grad_norm': 0.40619789289272457, 'learning_rate': 7e-06, 'epoch': 0.36}
0: �█▌      | 1070/2975 [19:23<27:44,  1.14it/s]
                                                   

 36%|███▌      | 1070/2975 [19:23<27:44,  1.14it/s]
 36%|███▌      | 1071/2975 [19:24<27:41,  1.15it/s]
 36%|███▌      | 1072/2975 [19:24<27:34,  1.15it/s]
 36%|███▌      | 1073/2975 [19:25<27:32,  1.15it/s]
 36%|███▌      | 1074/2975 [19:26<28:32,  1.11it/s]
 36%|███▌      | 1075/2975 [19:27<28:10,  1.12it/s]
 36%|███▌      | 1076/2975 [19:28<27:56,  1.13it/s]
 36%|███▌      | 1077/2975 [19:29<27:47,  1.14it/s]
 36%|███▌      | 1078/2975 [19:30<27:38,  1.14it/s]
 36%|███▋      | 1079/2975 [19:31<27:31,  1.15it/s]
 36%|███▋      | 1080/2975 [19:31<27:24,  1.15it/s]
                                                   

 36%|███▋      | 1080/2975 [19:31<27:24,  1.15it/s]
 36%|███▋      | 1081/2975 [19:32<27:23,  1.15it/s]
 36%|███▋      | 1082/2975 [19:33<27:20,  1.15it/s]
 36%|███▋      | 108
0: {'loss': 0.5805, 'grad_norm': 0.4341308347663893, 'learning_rate': 7e-06, 'epoch': 0.37}
0: 3/2975 [19:34<27:17,  1.16it/s]
 36%|███▋      | 1084/2975 [19:35<27:16,  1.16it/s]
 36%|███▋      | 1085/2975 [19:36<27:14,  1.16it/s]
 37%|███▋      | 1086/2975 [19:37<27:11,  1.16it/s]
 37%|███▋      | 1087/2975 [19:38<27:13,  1.16it/s]
 37%|███▋      | 1088/2975 [19:38<27:50,  1.13it/s]
 37%|███▋      | 1089/2975 [19:39<27:37,  1.14it/s]
 37%|███▋      | 1090/2975 [19:40<27:28,  1.14it/s]
                                                   

 37%|███▋      | 1090/2975 [19:40<27:28,  1.14it/s]
 37%|███▋      | 1091/2975 [19:41<27:24,  1.15it/s]
 37%|███▋      | 1092/2975 [19:42<27:19,  1.15it/s]
 37%|███▋      | 1093/2975 [19:43<27:33,  1.14it/s]
 37%|███▋      | 1094/2975 [19:44<27:25,  1.14it/s]
 37%|███▋      | 1095/2975 [19:45<27:21,  1.15it/s]
 37%|███▋      | 1096/2975 [19:45<27:13,  1.15it/s]
 37%|███▋      | 1097/2975 [19:46<27:10,  1.15it/s]
 37%|███▋      | 1098/2975 [19:
0: {'loss': 0.5751, 'grad_norm': 0.4046248877795063, 'learning_rate': 7e-06, 'epoch': 0.37}
0: {'loss': 0.5904, 'grad_norm': 0.3957538904800882, 'learning_rate': 7e-06, 'epoch': 0.37}
0: 47<27:06,  1.15it/s]
 37%|███▋      | 1099/2975 [19:48<27:04,  1.15it/s]
 37%|███▋      | 1100/2975 [19:49<27:03,  1.16it/s]
                                                   

 37%|███▋      | 1100/2975 [19:49<27:03,  1.16it/s]
 37%|███▋      | 1101/2975 [19:50<27:48,  1.12it/s]
 37%|███▋      | 1102/2975 [19:51<27:35,  1.13it/s]
 37%|███▋      | 1103/2975 [19:52<27:22,  1.14it/s]
 37%|███▋      | 1104/2975 [19:52<27:14,  1.14it/s]
 37%|███▋      | 1105/2975 [19:53<27:10,  1.15it/s]
 37%|███▋      | 1106/2975 [19:54<27:03,  1.15it/s]
 37%|███▋      | 1107/2975 [19:55<27:00,  1.15it/s]
 37%|███▋      | 1108/2975 [19:56<26:58,  1.15it/s]
 37%|███▋      | 1109/2975 [19:57<26:54,  1.16it/s]
 37%|███▋      | 1110/2975 [19:58<26:54,  1.16it/s]
                                                   

 37%|███▋      | 1110/2975 [19:58<26:54,  1.16it/s]
 37%|███▋      | 1111/2975 [19:58<26:52,  1.16it/
0: {'loss': 0.5757, 'grad_norm': 0.399347536404926, 'learning_rate': 7e-06, 'epoch': 0.38}
0: s]
 37%|███▋      | 1112/2975 [19:59<26:50,  1.16it/s]
 37%|███▋      | 1113/2975 [20:00<27:10,  1.14it/s]
 37%|███▋      | 1114/2975 [20:01<27:06,  1.14it/s]
 37%|███▋      | 1115/2975 [20:02<26:58,  1.15it/s]
 38%|███▊      | 1116/2975 [20:03<27:04,  1.14it/s]
 38%|███▊      | 1117/2975 [20:04<26:58,  1.15it/s]
 38%|███▊      | 1118/2975 [20:05<26:53,  1.15it/s]
 38%|███▊      | 1119/2975 [20:05<26:51,  1.15it/s]
 38%|███▊      | 1120/2975 [20:06<26:48,  1.15it/s]
                                                   

 38%|███▊      | 1120/2975 [20:06<26:48,  1.15it/s]
 38%|███▊      | 1121/2975 [20:07<26:45,  1.15it/s]
 38%|███▊      | 1122/2975 [20:08<26:46,  1.15it/s]
 38%|███▊      | 1123/2975 [20:09<26:43,  1.16it/s]
 38%|███▊      | 1124/2975 [20:10<28:30,  1.08it/s]
 38%|███▊      | 1125/2975 [20:11<27:56,  1.10it/s]
 38%|███▊      | 1126/2975 [20:12<27:29,  1.12it/s]
 38%|█
0: {'loss': 0.5764, 'grad_norm': 0.4105822230900647, 'learning_rate': 7e-06, 'epoch': 0.38}
0: {'loss': 0.5801, 'grad_norm': 0.4278209078596019, 'learning_rate': 7e-06, 'epoch': 0.38}
0: ██▊      | 1127/2975 [20:13<27:15,  1.13it/s]
 38%|███▊      | 1128/2975 [20:13<27:04,  1.14it/s]
 38%|███▊      | 1129/2975 [20:14<26:57,  1.14it/s]
 38%|███▊      | 1130/2975 [20:15<26:50,  1.15it/s]
                                                   

 38%|███▊      | 1130/2975 [20:15<26:50,  1.15it/s]
 38%|███▊      | 1131/2975 [20:16<26:45,  1.15it/s]
 38%|███▊      | 1132/2975 [20:17<26:39,  1.15it/s]
 38%|███▊      | 1133/2975 [20:18<26:38,  1.15it/s]
 38%|███▊      | 1134/2975 [20:19<26:34,  1.15it/s]
 38%|███▊      | 1135/2975 [20:19<26:34,  1.15it/s]
 38%|███▊      | 1136/2975 [20:20<26:35,  1.15it/s]
 38%|███▊      | 1137/2975 [20:21<26:30,  1.16it/s]
 38%|███▊      | 1138/2975 [20:22<26:33,  1.15it/s]
 38%|███▊      | 1139/2975 [20:23<28:30,  1.07it/s]
 38%|███▊      | 1140/2975 [20:24<27:48,  1.10it/s]
                                                   

 38%|███▊      | 1
0: {'loss': 0.5704, 'grad_norm': 0.39882547835696996, 'learning_rate': 7e-06, 'epoch': 0.39}
0: 140/2975 [20:24<27:48,  1.10it/s]
 38%|███▊      | 1141/2975 [20:25<27:26,  1.11it/s]
 38%|███▊      | 1142/2975 [20:26<27:05,  1.13it/s]
 38%|███▊      | 1143/2975 [20:27<26:51,  1.14it/s]
 38%|███▊      | 1144/2975 [20:27<26:44,  1.14it/s]
 38%|███▊      | 1145/2975 [20:28<26:35,  1.15it/s]
 39%|███▊      | 1146/2975 [20:29<26:31,  1.15it/s]
 39%|███▊      | 1147/2975 [20:30<26:28,  1.15it/s]
 39%|███▊      | 1148/2975 [20:31<26:23,  1.15it/s]
 39%|███▊      | 1149/2975 [20:32<26:59,  1.13it/s]
 39%|███▊      | 1150/2975 [20:33<26:45,  1.14it/s]
                                                   

 39%|███▊      | 1150/2975 [20:33<26:45,  1.14it/s]
 39%|███▊      | 1151/2975 [20:34<26:48,  1.13it/s]
 39%|███▊      | 1152/2975 [20:34<26:39,  1.14it/s]
 39%|███▉      | 1153/2975 [20:35<26:29,  1.15it/s]
 39%|███▉      | 1154/2975 [20:36<26:25,  1.15it/s]
 39%|███▉      | 1155/2975 [2
0: {'loss': 0.5726, 'grad_norm': 0.40072336104523915, 'learning_rate': 7e-06, 'epoch': 0.39}
0: 0:37<26:21,  1.15it/s]
 39%|███▉      | 1156/2975 [20:38<26:33,  1.14it/s]
 39%|███▉      | 1157/2975 [20:39<26:27,  1.15it/s]
 39%|███▉      | 1158/2975 [20:40<27:59,  1.08it/s]
 39%|███▉      | 1159/2975 [20:41<27:25,  1.10it/s]
 39%|███▉      | 1160/2975 [20:42<27:00,  1.12it/s]
                                                   

 39%|███▉      | 1160/2975 [20:42<27:00,  1.12it/s]
 39%|███▉      | 1161/2975 [20:42<26:45,  1.13it/s]
 39%|███▉      | 1162/2975 [20:43<26:33,  1.14it/s]
 39%|███▉      | 1163/2975 [20:44<26:23,  1.14it/s]
 39%|███▉      | 1164/2975 [20:45<26:17,  1.15it/s]
 39%|███▉      | 1165/2975 [20:46<26:15,  1.15it/s]
 39%|███▉      | 1166/2975 [20:47<26:13,  1.15it/s]
 39%|███▉      | 1167/2975 [20:48<26:08,  1.15it/s]
 39%|███▉      | 1168/2975 [20:49<26:06,  1.15it/s]
 39%|███▉      | 1169/2975 [20:49<26:28,  1.14it/s]
 39%|███▉      | 1170/2975 [20:50<26:20,
0: {'loss': 0.5876, 'grad_norm': 0.4096808763181043, 'learning_rate': 7e-06, 'epoch': 0.39}
0: {'loss': 0.5646, 'grad_norm': 0.3777949743692005, 'learning_rate': 7e-06, 'epoch': 0.4}
0:   1.14it/s]
                                                   

 39%|███▉      | 1170/2975 [20:50<26:20,  1.14it/s]
 39%|███▉      | 1171/2975 [20:51<26:14,  1.15it/s]
 39%|███▉      | 1172/2975 [20:52<26:09,  1.15it/s]
 39%|███▉      | 1173/2975 [20:53<26:06,  1.15it/s]
 39%|███▉      | 1174/2975 [20:54<26:03,  1.15it/s]
 39%|███▉      | 1175/2975 [20:55<26:00,  1.15it/s]
 40%|███▉      | 1176/2975 [20:55<25:56,  1.16it/s]
 40%|███▉      | 1177/2975 [20:56<26:05,  1.15it/s]
 40%|███▉      | 1178/2975 [20:57<26:01,  1.15it/s]
 40%|███▉      | 1179/2975 [20:58<25:55,  1.15it/s]
 40%|███▉      | 1180/2975 [20:59<25:53,  1.16it/s]
                                                   

 40%|███▉      | 1180/2975 [20:59<25:53,  1.16it/s]
 40%|███▉      | 1181/2975 [21:00<25:54,  1.15it/s]
 40%|███▉      | 1182/2975 [21:01<25:49,  1.16it/s]
 40%|███▉      | 1183/2975 [21:02<25:48,  1.16it/s]
 40%|�
0: {'loss': 0.5663, 'grad_norm': 0.4018384323010857, 'learning_rate': 7e-06, 'epoch': 0.4}
0: ��██▉      | 1184/2975 [21:02<25:46,  1.16it/s]
 40%|███▉      | 1185/2975 [21:03<25:47,  1.16it/s]
 40%|███▉      | 1186/2975 [21:04<25:46,  1.16it/s]
 40%|███▉      | 1187/2975 [21:05<25:45,  1.16it/s]
 40%|███▉      | 1188/2975 [21:06<25:45,  1.16it/s]
 40%|███▉      | 1189/2975 [21:07<25:43,  1.16it/s]
 40%|████      | 1190/2975 [21:08<25:43,  1.16it/s]
                                                   

 40%|████      | 1190/2975 [21:08<25:43,  1.16it/s]
 40%|████      | 1191/2975 [21:13<1:05:11,  2.19s/it]
 40%|████      | 1192/2975 [21:14<53:18,  1.79s/it]  
 40%|████      | 1193/2975 [21:15<45:00,  1.52s/it]
 40%|████      | 1194/2975 [21:15<39:12,  1.32s/it]
 40%|████      | 1195/2975 [21:16<35:07,  1.18s/it]
 40%|████      | 1196/2975 [21:17<32:15,  1.09s/it]
 40%|████      | 1197/2975 [21:18<30:14,  1.02s/it]
 40%|████      | 1198/2975 [21:19<28:48,  1.03it/s]
 40%|██�
0: {'loss': 0.5704, 'grad_norm': 0.40808153778806194, 'learning_rate': 7e-06, 'epoch': 0.4}
0: {'loss': 0.578, 'grad_norm': 0.42309928197576674, 'learning_rate': 7e-06, 'epoch': 0.41}
0: �█      | 1199/2975 [21:20<27:48,  1.06it/s]
 40%|████      | 1200/2975 [21:21<27:06,  1.09it/s]
                                                   

 40%|████      | 1200/2975 [21:21<27:06,  1.09it/s]
 40%|████      | 1201/2975 [21:22<26:36,  1.11it/s]
 40%|████      | 1202/2975 [21:22<26:16,  1.12it/s]
 40%|████      | 1203/2975 [21:23<26:00,  1.14it/s]
 40%|████      | 1204/2975 [21:24<25:50,  1.14it/s]
 41%|████      | 1205/2975 [21:25<25:43,  1.15it/s]
 41%|████      | 1206/2975 [21:26<25:35,  1.15it/s]
 41%|████      | 1207/2975 [21:27<25:34,  1.15it/s]
 41%|████      | 1208/2975 [21:28<25:31,  1.15it/s]
 41%|████      | 1209/2975 [21:29<26:40,  1.10it/s]
 41%|████      | 1210/2975 [21:29<26:16,  1.12it/s]
                                                   

 41%|████      | 1210/2975 [21:29<26:16,  1.12it/s]
 41%|████      | 1211/2975 [21:30<27:10,  1.08it/s]
 41%|████      | 1212/2
0: {'loss': 0.5858, 'grad_norm': 0.44312809751680365, 'learning_rate': 7e-06, 'epoch': 0.41}
0: 975 [21:31<26:35,  1.11it/s]
 41%|████      | 1213/2975 [21:32<26:10,  1.12it/s]
 41%|████      | 1214/2975 [21:33<25:54,  1.13it/s]
 41%|████      | 1215/2975 [21:34<25:42,  1.14it/s]
 41%|████      | 1216/2975 [21:35<25:36,  1.14it/s]
 41%|████      | 1217/2975 [21:36<25:30,  1.15it/s]
 41%|████      | 1218/2975 [21:36<25:23,  1.15it/s]
 41%|████      | 1219/2975 [21:37<25:20,  1.16it/s]
 41%|████      | 1220/2975 [21:38<25:16,  1.16it/s]
                                                   

 41%|████      | 1220/2975 [21:38<25:16,  1.16it/s]
 41%|████      | 1221/2975 [21:39<25:20,  1.15it/s]
 41%|████      | 1222/2975 [21:40<25:18,  1.15it/s]
 41%|████      | 1223/2975 [21:41<25:14,  1.16it/s]
 41%|████      | 1224/2975 [21:42<25:14,  1.16it/s]
 41%|████      | 1225/2975 [21:43<25:11,  1.16it/s]
 41%|████      | 1226/2975 [21:43<25:11,  1.16it/s]
 41%|████      | 1227/2975 [21:44<
0: {'loss': 0.5833, 'grad_norm': 0.3999265505188534, 'learning_rate': 7e-06, 'epoch': 0.41}
0: {'loss': 0.5595, 'grad_norm': 0.4157682502027386, 'learning_rate': 7e-06, 'epoch': 0.42}
0: 25:10,  1.16it/s]
 41%|████▏     | 1228/2975 [21:45<25:06,  1.16it/s]
 41%|████▏     | 1229/2975 [21:46<25:08,  1.16it/s]
 41%|████▏     | 1230/2975 [21:47<25:04,  1.16it/s]
                                                   

 41%|████▏     | 1230/2975 [21:47<25:04,  1.16it/s]
 41%|████▏     | 1231/2975 [21:48<27:13,  1.07it/s]
 41%|████▏     | 1232/2975 [21:49<26:33,  1.09it/s]
 41%|████▏     | 1233/2975 [21:50<26:08,  1.11it/s]
 41%|████▏     | 1234/2975 [21:51<27:29,  1.06it/s]
 42%|████▏     | 1235/2975 [21:52<26:44,  1.08it/s]
 42%|████▏     | 1236/2975 [21:52<26:11,  1.11it/s]
 42%|████▏     | 1237/2975 [21:53<25:49,  1.12it/s]
 42%|████▏     | 1238/2975 [21:54<25:31,  1.13it/s]
 42%|████▏     | 1239/2975 [21:55<25:20,  1.14it/s]
 42%|████▏     | 1240/2975 [21:56<25:14,  1.15it/s]
                                                   

 42%|████▏     | 1240/
0: {'loss': 0.5699, 'grad_norm': 0.4095041085785252, 'learning_rate': 7e-06, 'epoch': 0.42}
0: 2975 [21:56<25:14,  1.15it/s]
 42%|████▏     | 1241/2975 [21:57<25:09,  1.15it/s]
 42%|████▏     | 1242/2975 [21:58<25:04,  1.15it/s]
 42%|████▏     | 1243/2975 [21:58<25:00,  1.15it/s]
 42%|████▏     | 1244/2975 [21:59<24:56,  1.16it/s]
 42%|████▏     | 1245/2975 [22:00<24:54,  1.16it/s]
 42%|████▏     | 1246/2975 [22:01<24:50,  1.16it/s]
 42%|████▏     | 1247/2975 [22:02<24:48,  1.16it/s]
 42%|████▏     | 1248/2975 [22:03<24:48,  1.16it/s]
 42%|████▏     | 1249/2975 [22:04<24:46,  1.16it/s]
 42%|████▏     | 1250/2975 [22:05<24:48,  1.16it/s]
                                                   

 42%|████▏     | 1250/2975 [22:05<24:48,  1.16it/s]
 42%|████▏     | 1251/2975 [22:05<24:49,  1.16it/s]
 42%|████▏     | 1252/2975 [22:06<24:47,  1.16it/s]
 42%|████▏     | 1253/2975 [22:07<24:46,  1.16it/s]
 42%|████▏     | 1254/2975 [22:08<24:49,  1.16it/s]
 42%|██
0: {'loss': 0.5612, 'grad_norm': 0.4161356621560543, 'learning_rate': 7e-06, 'epoch': 0.42}
0: ██▏     | 1255/2975 [22:09<24:48,  1.16it/s]
 42%|████▏     | 1256/2975 [22:10<24:45,  1.16it/s]
 42%|████▏     | 1257/2975 [22:11<24:42,  1.16it/s]
 42%|████▏     | 1258/2975 [22:11<24:44,  1.16it/s]
 42%|████▏     | 1259/2975 [22:12<24:41,  1.16it/s]
 42%|████▏     | 1260/2975 [22:13<24:41,  1.16it/s]
                                                   

 42%|████▏     | 1260/2975 [22:13<24:41,  1.16it/s]
 42%|████▏     | 1261/2975 [22:14<24:42,  1.16it/s]
 42%|████▏     | 1262/2975 [22:15<24:41,  1.16it/s]
 42%|████▏     | 1263/2975 [22:16<24:40,  1.16it/s]
 42%|████▏     | 1264/2975 [22:17<24:37,  1.16it/s]
 43%|████▎     | 1265/2975 [22:17<24:35,  1.16it/s]
 43%|████▎     | 1266/2975 [22:18<24:35,  1.16it/s]
 43%|████▎     | 1267/2975 [22:19<24:34,  1.16it/s]
 43%|████▎     | 1268/2975 [22:20<24:33,  1.16it/s]
 43%|████▎     | 1269/2975 [22:21<24:32,  
0: {'loss': 0.5658, 'grad_norm': 0.4182308246047162, 'learning_rate': 7e-06, 'epoch': 0.43}
0: {'loss': 0.5681, 'grad_norm': 0.44156130121882586, 'learning_rate': 7e-06, 'epoch': 0.43}
0: 1.16it/s]
 43%|████▎     | 1270/2975 [22:22<24:31,  1.16it/s]
                                                   

 43%|████▎     | 1270/2975 [22:22<24:31,  1.16it/s]
 43%|████▎     | 1271/2975 [22:23<24:33,  1.16it/s]
 43%|████▎     | 1272/2975 [22:24<24:30,  1.16it/s]
 43%|████▎     | 1273/2975 [22:24<24:30,  1.16it/s]
 43%|████▎     | 1274/2975 [22:25<24:27,  1.16it/s]
 43%|████▎     | 1275/2975 [22:26<24:25,  1.16it/s]
 43%|████▎     | 1276/2975 [22:27<24:26,  1.16it/s]
 43%|████▎     | 1277/2975 [22:28<24:24,  1.16it/s]
 43%|████▎     | 1278/2975 [22:29<24:27,  1.16it/s]
 43%|████▎     | 1279/2975 [22:30<24:26,  1.16it/s]
 43%|████▎     | 1280/2975 [22:30<24:53,  1.14it/s]
                                                   

 43%|████▎     | 1280/2975 [22:30<24:53,  1.14it/s]
 43%|████▎     | 1281/2975 [22:31<24:42,  1.14it/s]
 43%|████▎     | 1282/2975 [22
0: {'loss': 0.5805, 'grad_norm': 0.41356421999393106, 'learning_rate': 7e-06, 'epoch': 0.43}
0: :32<24:33,  1.15it/s]
 43%|████▎     | 1283/2975 [22:33<24:27,  1.15it/s]
 43%|████▎     | 1284/2975 [22:34<24:23,  1.16it/s]
 43%|████▎     | 1285/2975 [22:35<24:22,  1.16it/s]
 43%|████▎     | 1286/2975 [22:36<24:18,  1.16it/s]
 43%|████▎     | 1287/2975 [22:37<24:16,  1.16it/s]
 43%|████▎     | 1288/2975 [22:37<24:13,  1.16it/s]
 43%|████▎     | 1289/2975 [22:38<24:12,  1.16it/s]
 43%|████▎     | 1290/2975 [22:39<24:32,  1.14it/s]
                                                   

 43%|████▎     | 1290/2975 [22:39<24:32,  1.14it/s]
 43%|████▎     | 1291/2975 [22:40<24:28,  1.15it/s]
 43%|████▎     | 1292/2975 [22:41<24:58,  1.12it/s]
 43%|████▎     | 1293/2975 [22:42<24:44,  1.13it/s]
 43%|████▎     | 1294/2975 [22:43<24:33,  1.14it/s]
 44%|████▎     | 1295/2975 [22:44<24:36,  1.14it/s]
 44%|████▎     | 1296/2975 [22:44<24:27,  1.14it/s]
 44%|████�
0: {'loss': 0.5563, 'grad_norm': 0.45669754705727994, 'learning_rate': 7e-06, 'epoch': 0.44}
0: {'loss': 0.5677, 'grad_norm': 0.386633319698303, 'learning_rate': 7e-06, 'epoch': 0.44}
0: �     | 1297/2975 [22:45<24:21,  1.15it/s]
 44%|████▎     | 1298/2975 [22:46<24:16,  1.15it/s]
 44%|████▎     | 1299/2975 [22:47<24:10,  1.16it/s]
 44%|████▎     | 1300/2975 [22:48<24:08,  1.16it/s]
                                                   

 44%|████▎     | 1300/2975 [22:48<24:08,  1.16it/s]
 44%|████▎     | 1301/2975 [22:49<24:08,  1.16it/s]
 44%|████▍     | 1302/2975 [22:50<24:04,  1.16it/s]
 44%|████▍     | 1303/2975 [22:50<24:05,  1.16it/s]
 44%|████▍     | 1304/2975 [22:51<24:03,  1.16it/s]
 44%|████▍     | 1305/2975 [22:52<24:16,  1.15it/s]
 44%|████▍     | 1306/2975 [22:53<24:11,  1.15it/s]
 44%|████▍     | 1307/2975 [22:54<24:08,  1.15it/s]
 44%|████▍     | 1308/2975 [22:55<24:04,  1.15it/s]
 44%|████▍     | 1309/2975 [22:56<24:01,  1.16it/s]
 44%|████▍     | 1310/2975 [22:57<23:56,  1.16it/s]
                                                   

 44%|�
0: {'loss': 0.5625, 'grad_norm': 0.4023339517570007, 'learning_rate': 7e-06, 'epoch': 0.44}
0: �███▍     | 1310/2975 [22:57<23:56,  1.16it/s]
 44%|████▍     | 1311/2975 [22:57<23:56,  1.16it/s]
 44%|████▍     | 1312/2975 [22:58<23:54,  1.16it/s]
 44%|████▍     | 1313/2975 [22:59<23:52,  1.16it/s]
 44%|████▍     | 1314/2975 [23:00<23:51,  1.16it/s]
 44%|████▍     | 1315/2975 [23:01<23:49,  1.16it/s]
 44%|████▍     | 1316/2975 [23:02<23:49,  1.16it/s]
 44%|████▍     | 1317/2975 [23:03<23:48,  1.16it/s]
 44%|████▍     | 1318/2975 [23:03<23:48,  1.16it/s]
 44%|████▍     | 1319/2975 [23:04<23:48,  1.16it/s]
 44%|████▍     | 1320/2975 [23:05<23:47,  1.16it/s]
                                                   

 44%|████▍     | 1320/2975 [23:05<23:47,  1.16it/s]
 44%|████▍     | 1321/2975 [23:06<23:49,  1.16it/s]
 44%|████▍     | 1322/2975 [23:07<23:56,  1.15it/s]
 44%|████▍     | 1323/2975 [23:08<23:51,  1.15it/s]
 45%|████▍     | 1324/2975 [23:09<23:4
0: {'loss': 0.5713, 'grad_norm': 0.4032885862014425, 'learning_rate': 7e-06, 'epoch': 0.45}
0: 9,  1.16it/s]
 45%|████▍     | 1325/2975 [23:09<23:45,  1.16it/s]
 45%|████▍     | 1326/2975 [23:10<23:43,  1.16it/s]
 45%|████▍     | 1327/2975 [23:11<23:41,  1.16it/s]
 45%|████▍     | 1328/2975 [23:12<23:40,  1.16it/s]
 45%|████▍     | 1329/2975 [23:13<23:41,  1.16it/s]
 45%|████▍     | 1330/2975 [23:14<23:38,  1.16it/s]
                                                   

 45%|████▍     | 1330/2975 [23:14<23:38,  1.16it/s]
 45%|████▍     | 1331/2975 [23:15<23:40,  1.16it/s]
 45%|████▍     | 1332/2975 [23:16<23:39,  1.16it/s]
 45%|████▍     | 1333/2975 [23:16<23:36,  1.16it/s]
 45%|████▍     | 1334/2975 [23:17<23:36,  1.16it/s]
 45%|████▍     | 1335/2975 [23:18<23:36,  1.16it/s]
 45%|████▍     | 1336/2975 [23:19<24:17,  1.12it/s]
 45%|████▍     | 1337/2975 [23:20<24:02,  1.14it/s]
 45%|████▍     | 1338/2975 [23:21<23:53,  1.14it/s]
 45%|████▌     | 
0: {'loss': 0.5699, 'grad_norm': 0.40785606126230783, 'learning_rate': 7e-06, 'epoch': 0.45}
0: {'loss': 0.5579, 'grad_norm': 0.41341495908406617, 'learning_rate': 7e-06, 'epoch': 0.45}
0: 1339/2975 [23:22<23:45,  1.15it/s]
 45%|████▌     | 1340/2975 [23:23<23:40,  1.15it/s]
                                                   

 45%|████▌     | 1340/2975 [23:23<23:40,  1.15it/s]
 45%|████▌     | 1341/2975 [23:23<23:38,  1.15it/s]
 45%|████▌     | 1342/2975 [23:24<23:35,  1.15it/s]
 45%|████▌     | 1343/2975 [23:25<23:32,  1.16it/s]
 45%|████▌     | 1344/2975 [23:26<23:28,  1.16it/s]
 45%|████▌     | 1345/2975 [23:27<23:25,  1.16it/s]
 45%|████▌     | 1346/2975 [23:28<23:23,  1.16it/s]
 45%|████▌     | 1347/2975 [23:29<23:22,  1.16it/s]
 45%|████▌     | 1348/2975 [23:29<23:20,  1.16it/s]
 45%|████▌     | 1349/2975 [23:30<23:18,  1.16it/s]
 45%|████▌     | 1350/2975 [23:31<23:17,  1.16it/s]
                                                   

 45%|████▌     | 1350/2975 [23:31<23:17,  1.16it/s]
 45%|████▌     | 1351/2975 [23:32<23:18,  1.16it/s]
 45%|███�
0: {'loss': 0.585, 'grad_norm': 0.4205108540658371, 'learning_rate': 7e-06, 'epoch': 0.46}
0: ��▌     | 1352/2975 [23:33<23:16,  1.16it/s]
 45%|████▌     | 1353/2975 [23:34<23:17,  1.16it/s]
 46%|████▌     | 1354/2975 [23:35<23:17,  1.16it/s]
 46%|████▌     | 1355/2975 [23:35<23:18,  1.16it/s]
 46%|████▌     | 1356/2975 [23:36<23:17,  1.16it/s]
 46%|████▌     | 1357/2975 [23:37<23:14,  1.16it/s]
 46%|████▌     | 1358/2975 [23:38<23:16,  1.16it/s]
 46%|████▌     | 1359/2975 [23:39<23:15,  1.16it/s]
 46%|████▌     | 1360/2975 [23:40<23:13,  1.16it/s]
                                                   

 46%|████▌     | 1360/2975 [23:40<23:13,  1.16it/s]
 46%|████▌     | 1361/2975 [23:41<23:12,  1.16it/s]
 46%|████▌     | 1362/2975 [23:41<23:10,  1.16it/s]
 46%|████▌     | 1363/2975 [23:42<23:12,  1.16it/s]
 46%|████▌     | 1364/2975 [23:43<23:10,  1.16it/s]
 46%|████▌     | 1365/2975 [23:44<23:09,  1.16it/s]
 46%|████▌     | 1366/2975 [23:45<23:07,  1.16
0: {'loss': 0.5764, 'grad_norm': 0.419756620289106, 'learning_rate': 7e-06, 'epoch': 0.46}
0: it/s]
 46%|████▌     | 1367/2975 [23:46<23:07,  1.16it/s]
 46%|████▌     | 1368/2975 [23:47<23:09,  1.16it/s]
 46%|████▌     | 1369/2975 [23:48<23:31,  1.14it/s]
 46%|████▌     | 1370/2975 [23:48<23:23,  1.14it/s]
                                                   

 46%|████▌     | 1370/2975 [23:48<23:23,  1.14it/s]
 46%|████▌     | 1371/2975 [23:49<23:17,  1.15it/s]
 46%|████▌     | 1372/2975 [23:50<23:14,  1.15it/s]
 46%|████▌     | 1373/2975 [23:51<23:09,  1.15it/s]
 46%|████▌     | 1374/2975 [23:52<23:06,  1.15it/s]
 46%|████▌     | 1375/2975 [23:53<23:13,  1.15it/s]
 46%|████▋     | 1376/2975 [23:54<23:05,  1.15it/s]
 46%|████▋     | 1377/2975 [23:54<23:04,  1.15it/s]
 46%|████▋     | 1378/2975 [23:55<23:00,  1.16it/s]
 46%|████▋     | 1379/2975 [23:56<23:00,  1.16it/s]
 46%|████▋     | 1380/2975 [23:57<22:59,  1.16it/s]
                                   
0: {'loss': 0.5717, 'grad_norm': 0.4048207663989343, 'learning_rate': 7e-06, 'epoch': 0.46}
0: {'loss': 0.5878, 'grad_norm': 0.39924268557499715, 'learning_rate': 7e-06, 'epoch': 0.47}
0:                 

 46%|████▋     | 1380/2975 [23:57<22:59,  1.16it/s]
 46%|████▋     | 1381/2975 [23:58<23:07,  1.15it/s]
 46%|████▋     | 1382/2975 [23:59<23:01,  1.15it/s]
 46%|████▋     | 1383/2975 [24:00<23:02,  1.15it/s]
 47%|████▋     | 1384/2975 [24:01<22:56,  1.16it/s]
 47%|████▋     | 1385/2975 [24:01<22:54,  1.16it/s]
 47%|████▋     | 1386/2975 [24:02<22:52,  1.16it/s]
 47%|████▋     | 1387/2975 [24:03<22:50,  1.16it/s]
 47%|████▋     | 1388/2975 [24:04<22:51,  1.16it/s]
 47%|████▋     | 1389/2975 [24:05<24:10,  1.09it/s]
 47%|████▋     | 1390/2975 [24:06<23:44,  1.11it/s]
                                                   

 47%|████▋     | 1390/2975 [24:06<23:44,  1.11it/s]
 47%|████▋     | 1391/2975 [24:07<23:27,  1.13it/s]
 47%|████▋     | 1392/2975 [24:08<23:14,  1.14it/s]
 47%|████▋     | 1393/2975 [24:08<23:03,  1.14it/s]
 47%|████▋   
0: {'loss': 0.5697, 'grad_norm': 0.3960128295713862, 'learning_rate': 7e-06, 'epoch': 0.47}
0:   | 1394/2975 [24:09<22:57,  1.15it/s]
 47%|████▋     | 1395/2975 [24:10<22:51,  1.15it/s]
 47%|████▋     | 1396/2975 [24:11<22:46,  1.16it/s]
 47%|████▋     | 1397/2975 [24:12<22:44,  1.16it/s]
 47%|████▋     | 1398/2975 [24:13<22:43,  1.16it/s]
 47%|████▋     | 1399/2975 [24:14<22:41,  1.16it/s]
 47%|████▋     | 1400/2975 [24:15<22:40,  1.16it/s]
                                                   

 47%|████▋     | 1400/2975 [24:15<22:40,  1.16it/s]
 47%|████▋     | 1401/2975 [24:15<22:43,  1.15it/s]
 47%|████▋     | 1402/2975 [24:16<22:41,  1.16it/s]
 47%|████▋     | 1403/2975 [24:17<22:40,  1.16it/s]
 47%|████▋     | 1404/2975 [24:18<22:37,  1.16it/s]
 47%|████▋     | 1405/2975 [24:19<22:36,  1.16it/s]
 47%|████▋     | 1406/2975 [24:20<22:33,  1.16it/s]
 47%|████▋     | 1407/2975 [24:21<22:31,  1.16it/s]
 47%|████▋     | 1408/2975 [24:21<22:31,  1.16it/s]
 4
0: {'loss': 0.5732, 'grad_norm': 0.39688233974624904, 'learning_rate': 7e-06, 'epoch': 0.47}
0: {'loss': 0.5689, 'grad_norm': 0.4025950160365982, 'learning_rate': 7e-06, 'epoch': 0.48}
0: 7%|████▋     | 1409/2975 [24:22<22:31,  1.16it/s]
 47%|████▋     | 1410/2975 [24:23<22:32,  1.16it/s]
                                                   

 47%|████▋     | 1410/2975 [24:23<22:32,  1.16it/s]
 47%|████▋     | 1411/2975 [24:24<22:31,  1.16it/s]
 47%|████▋     | 1412/2975 [24:25<22:31,  1.16it/s]
 47%|████▋     | 1413/2975 [24:26<22:30,  1.16it/s]
 48%|████▊     | 1414/2975 [24:27<22:27,  1.16it/s]
 48%|████▊     | 1415/2975 [24:27<22:26,  1.16it/s]
 48%|████▊     | 1416/2975 [24:28<22:26,  1.16it/s]
 48%|████▊     | 1417/2975 [24:29<22:26,  1.16it/s]
 48%|████▊     | 1418/2975 [24:30<22:25,  1.16it/s]
 48%|████▊     | 1419/2975 [24:31<22:23,  1.16it/s]
 48%|████▊     | 1420/2975 [24:32<22:21,  1.16it/s]
                                                   

 48%|████▊     | 1420/2975 [24:32<22:21,  1.16it/s]
 48%|████▊     | 1421/2975 [24:33<22:22,  
0: {'loss': 0.5495, 'grad_norm': 0.4059482403856615, 'learning_rate': 7e-06, 'epoch': 0.48}
0: 1.16it/s]
 48%|████▊     | 1422/2975 [24:34<22:22,  1.16it/s]
 48%|████▊     | 1423/2975 [24:34<22:20,  1.16it/s]
 48%|████▊     | 1424/2975 [24:35<22:18,  1.16it/s]
 48%|████▊     | 1425/2975 [24:36<22:17,  1.16it/s]
 48%|████▊     | 1426/2975 [24:37<22:16,  1.16it/s]
 48%|████▊     | 1427/2975 [24:38<22:15,  1.16it/s]
 48%|████▊     | 1428/2975 [24:39<22:15,  1.16it/s]
 48%|████▊     | 1429/2975 [24:40<22:13,  1.16it/s]
 48%|████▊     | 1430/2975 [24:40<22:12,  1.16it/s]
                                                   

 48%|████▊     | 1430/2975 [24:40<22:12,  1.16it/s]
 48%|████▊     | 1431/2975 [24:41<22:12,  1.16it/s]
 48%|████▊     | 1432/2975 [24:42<22:13,  1.16it/s]
 48%|████▊     | 1433/2975 [24:43<22:12,  1.16it/s]
 48%|████▊     | 1434/2975 [24:44<22:10,  1.16it/s]
 48%|████▊     | 1435/2975 [24:45<22:09,  1.16it/s]
 48%|████▊     | 1436
0: {'loss': 0.5582, 'grad_norm': 0.3854665678148656, 'learning_rate': 7e-06, 'epoch': 0.48}
0: /2975 [24:46<22:12,  1.16it/s]
 48%|████▊     | 1437/2975 [24:46<22:11,  1.16it/s]
 48%|████▊     | 1438/2975 [24:47<22:14,  1.15it/s]
 48%|████▊     | 1439/2975 [24:48<22:12,  1.15it/s]
 48%|████▊     | 1440/2975 [24:49<22:11,  1.15it/s]
                                                   

 48%|████▊     | 1440/2975 [24:49<22:11,  1.15it/s]
 48%|████▊     | 1441/2975 [24:50<22:18,  1.15it/s]
 48%|████▊     | 1442/2975 [24:51<22:13,  1.15it/s]
 49%|████▊     | 1443/2975 [24:52<22:11,  1.15it/s]
 49%|████▊     | 1444/2975 [24:53<22:08,  1.15it/s]
 49%|████▊     | 1445/2975 [24:53<22:07,  1.15it/s]
 49%|████▊     | 1446/2975 [24:54<22:05,  1.15it/s]
 49%|████▊     | 1447/2975 [24:55<22:17,  1.14it/s]
 49%|████▊     | 1448/2975 [24:56<22:13,  1.14it/s]
 49%|████▊     | 1449/2975 [24:57<22:09,  1.15it/s]
 49%|████▊     | 1450/2975 [24:58<22:04,  1.15it/s]
          
0: {'loss': 0.5832, 'grad_norm': 0.4326503042921453, 'learning_rate': 7e-06, 'epoch': 0.49}
0: {'loss': 0.5609, 'grad_norm': 0.3870176562312668, 'learning_rate': 7e-06, 'epoch': 0.49}
0:                                          

 49%|████▊     | 1450/2975 [24:58<22:04,  1.15it/s]
 49%|████▉     | 1451/2975 [24:59<22:02,  1.15it/s]
 49%|████▉     | 1452/2975 [25:00<22:01,  1.15it/s]
 49%|████▉     | 1453/2975 [25:00<22:01,  1.15it/s]
 49%|████▉     | 1454/2975 [25:01<21:59,  1.15it/s]
 49%|████▉     | 1455/2975 [25:02<21:58,  1.15it/s]
 49%|████▉     | 1456/2975 [25:03<21:54,  1.16it/s]
 49%|████▉     | 1457/2975 [25:04<21:54,  1.15it/s]
 49%|████▉     | 1458/2975 [25:05<21:52,  1.16it/s]
 49%|████▉     | 1459/2975 [25:06<21:50,  1.16it/s]
 49%|████▉     | 1460/2975 [25:06<21:51,  1.15it/s]
                                                   

 49%|████▉     | 1460/2975 [25:06<21:51,  1.15it/s]
 49%|████▉     | 1461/2975 [25:07<21:52,  1.15it/s]
 49%|████▉     | 1462/2975 [25:08<21:50,  1.15it/s]
 49%|████▉     | 1463/2975 [25:09<21:52,  1.15it/s
0: {'loss': 0.5683, 'grad_norm': 0.4297852013376354, 'learning_rate': 7e-06, 'epoch': 0.49}
0: ]
 49%|████▉     | 1464/2975 [25:10<21:48,  1.15it/s]
 49%|████▉     | 1465/2975 [25:11<21:47,  1.15it/s]
 49%|████▉     | 1466/2975 [25:12<21:46,  1.15it/s]
 49%|████▉     | 1467/2975 [25:13<21:43,  1.16it/s]
 49%|████▉     | 1468/2975 [25:14<23:04,  1.09it/s]
 49%|████▉     | 1469/2975 [25:14<22:39,  1.11it/s]
 49%|████▉     | 1470/2975 [25:15<22:23,  1.12it/s]
                                                   

 49%|████▉     | 1470/2975 [25:15<22:23,  1.12it/s]
 49%|████▉     | 1471/2975 [25:16<22:12,  1.13it/s]
 49%|████▉     | 1472/2975 [25:17<22:02,  1.14it/s]
 50%|████▉     | 1473/2975 [25:18<21:57,  1.14it/s]
 50%|████▉     | 1474/2975 [25:19<21:51,  1.14it/s]
 50%|████▉     | 1475/2975 [25:20<21:44,  1.15it/s]
 50%|████▉     | 1476/2975 [25:20<21:41,  1.15it/s]
 50%|████▉     | 1477/2975 [25:21<21:37,  1.15it/s]
 50%|████▉     | 1478/2975 [2
0: {'loss': 0.5616, 'grad_norm': 0.4331284511699335, 'learning_rate': 7e-06, 'epoch': 0.5}
0: {'loss': 0.5553, 'grad_norm': 0.3915282192968263, 'learning_rate': 7e-06, 'epoch': 0.5}
0: 5:22<21:36,  1.15it/s]
 50%|████▉     | 1479/2975 [25:23<21:34,  1.16it/s]
 50%|████▉     | 1480/2975 [25:24<21:32,  1.16it/s]
                                                   

 50%|████▉     | 1480/2975 [25:24<21:32,  1.16it/s]
 50%|████▉     | 1481/2975 [25:25<21:35,  1.15it/s]
 50%|████▉     | 1482/2975 [25:26<21:32,  1.15it/s]
 50%|████▉     | 1483/2975 [25:27<21:30,  1.16it/s]
 50%|████▉     | 1484/2975 [25:27<21:28,  1.16it/s]
 50%|████▉     | 1485/2975 [25:28<21:26,  1.16it/s]
 50%|████▉     | 1486/2975 [25:29<21:26,  1.16it/s]
 50%|████▉     | 1487/2975 [25:30<21:26,  1.16it/s]
 50%|█████     | 1488/2975 [25:31<21:23,  1.16it/s]
 50%|█████     | 1489/2975 [25:32<21:21,  1.16it/s]
 50%|█████     | 1490/2975 [25:33<21:19,  1.16it/s]
                                                   

 50%|█████     | 1490/2975 [25:33<21:19,  1.16it/s]
 50%|█████     | 
0: {'loss': 0.5568, 'grad_norm': 0.4029221543968464, 'learning_rate': 7e-06, 'epoch': 0.5}
0: 1491/2975 [25:33<21:19,  1.16it/s]
 50%|█████     | 1492/2975 [25:34<21:18,  1.16it/s]
 50%|█████     | 1493/2975 [25:35<21:16,  1.16it/s]
 50%|█████     | 1494/2975 [25:36<21:15,  1.16it/s]
 50%|█████     | 1495/2975 [25:37<21:14,  1.16it/s]
 50%|█████     | 1496/2975 [25:38<21:14,  1.16it/s]
 50%|█████     | 1497/2975 [25:39<21:13,  1.16it/s]
 50%|█████     | 1498/2975 [25:39<21:12,  1.16it/s]
 50%|█████     | 1499/2975 [25:41<22:47,  1.08it/s]
 50%|█████     | 1500/2975 [25:41<22:16,  1.10it/s]
                                                   

 50%|█████     | 1500/2975 [25:41<22:16,  1.10it/s]
 50%|█████     | 1501/2975 [25:42<21:56,  1.12it/s]
 50%|█████     | 1502/2975 [25:43<23:25,  1.05it/s]
 51%|█████     | 1503/2975 [25:44<22:44,  1.08it/s]
 51%|█████     | 1504/2975 [25:45<22:12,  1.10it/s]
 51%|█████     | 1505/2975 [25:46<21:56,  1.12it/s]
 51%|�
0: {'loss': 0.5724, 'grad_norm': 0.421220831579463, 'learning_rate': 7e-06, 'epoch': 0.51}
0: ��████     | 1506/2975 [25:47<21:40,  1.13it/s]
 51%|█████     | 1507/2975 [25:48<23:09,  1.06it/s]
 51%|█████     | 1508/2975 [25:49<22:32,  1.08it/s]
 51%|█████     | 1509/2975 [25:50<22:04,  1.11it/s]
 51%|█████     | 1510/2975 [25:51<21:52,  1.12it/s]
                                                   

 51%|█████     | 1510/2975 [25:51<21:52,  1.12it/s]
 51%|█████     | 1511/2975 [25:51<21:40,  1.13it/s]
 51%|█████     | 1512/2975 [25:52<21:28,  1.14it/s]
 51%|█████     | 1513/2975 [25:53<21:18,  1.14it/s]
 51%|█████     | 1514/2975 [25:54<21:16,  1.14it/s]
 51%|█████     | 1515/2975 [25:55<21:08,  1.15it/s]
 51%|█████     | 1516/2975 [25:56<22:48,  1.07it/s]
 51%|█████     | 1517/2975 [25:57<22:16,  1.09it/s]
 51%|█████     | 1518/2975 [25:58<21:50,  1.11it/s]
 51%|█████     | 1519/2975 [25:59<21:32,  1.13it/s]
 51%|█████     | 1520/2975 [25:59<21:
0: {'loss': 0.5719, 'grad_norm': 0.4068837701186281, 'learning_rate': 7e-06, 'epoch': 0.51}
0: {'loss': 0.5681, 'grad_norm': 0.4034396735213433, 'learning_rate': 7e-06, 'epoch': 0.51}
0: 21,  1.14it/s]
                                                   

 51%|█████     | 1520/2975 [25:59<21:21,  1.14it/s]
 51%|█████     | 1521/2975 [26:00<21:12,  1.14it/s]
 51%|█████     | 1522/2975 [26:01<21:07,  1.15it/s]
 51%|█████     | 1523/2975 [26:02<21:02,  1.15it/s]
 51%|█████     | 1524/2975 [26:03<20:58,  1.15it/s]
 51%|█████▏    | 1525/2975 [26:04<20:58,  1.15it/s]
 51%|█████▏    | 1526/2975 [26:05<20:55,  1.15it/s]
 51%|█████▏    | 1527/2975 [26:05<20:53,  1.16it/s]
 51%|█████▏    | 1528/2975 [26:06<20:50,  1.16it/s]
 51%|█████▏    | 1529/2975 [26:07<20:50,  1.16it/s]
 51%|█████▏    | 1530/2975 [26:08<20:51,  1.15it/s]
                                                   

 51%|█████▏    | 1530/2975 [26:08<20:51,  1.15it/s]
 51%|█████▏    | 1531/2975 [26:09<20:51,  1.15it/s]
 51%|█████▏    | 1532/2975 [26:10<20:49,  1.15it/s]
 52%|████
0: {'loss': 0.5593, 'grad_norm': 0.4057349482768572, 'learning_rate': 7e-06, 'epoch': 0.52}
0: █▏    | 1533/2975 [26:11<20:50,  1.15it/s]
 52%|█████▏    | 1534/2975 [26:12<20:48,  1.15it/s]
 52%|█████▏    | 1535/2975 [26:12<20:48,  1.15it/s]
 52%|█████▏    | 1536/2975 [26:13<20:49,  1.15it/s]
 52%|█████▏    | 1537/2975 [26:14<20:48,  1.15it/s]
 52%|█████▏    | 1538/2975 [26:15<20:47,  1.15it/s]
 52%|█████▏    | 1539/2975 [26:16<20:45,  1.15it/s]
 52%|█████▏    | 1540/2975 [26:17<20:45,  1.15it/s]
                                                   

 52%|█████▏    | 1540/2975 [26:17<20:45,  1.15it/s]
 52%|█████▏    | 1541/2975 [26:18<21:27,  1.11it/s]
 52%|█████▏    | 1542/2975 [26:19<22:06,  1.08it/s]
 52%|█████▏    | 1543/2975 [26:20<21:40,  1.10it/s]
 52%|█████▏    | 1544/2975 [26:20<21:20,  1.12it/s]
 52%|█████▏    | 1545/2975 [26:21<21:08,  1.13it/s]
 52%|█████▏    | 1546/2975 [26:22<20:57,  1.14it/s]
 52%|█████▏    |
0: {'loss': 0.5505, 'grad_norm': 0.43083414643961787, 'learning_rate': 7e-06, 'epoch': 0.52}
0:  1547/2975 [26:23<20:52,  1.14it/s]
 52%|█████▏    | 1548/2975 [26:24<20:45,  1.15it/s]
 52%|█████▏    | 1549/2975 [26:25<20:39,  1.15it/s]
 52%|█████▏    | 1550/2975 [26:26<20:37,  1.15it/s]
                                                   

 52%|█████▏    | 1550/2975 [26:26<20:37,  1.15it/s]
 52%|█████▏    | 1551/2975 [26:26<20:34,  1.15it/s]
 52%|█████▏    | 1552/2975 [26:27<20:30,  1.16it/s]
 52%|█████▏    | 1553/2975 [26:28<20:28,  1.16it/s]
 52%|█████▏    | 1554/2975 [26:29<20:27,  1.16it/s]
 52%|█████▏    | 1555/2975 [26:30<21:50,  1.08it/s]
 52%|█████▏    | 1556/2975 [26:31<21:23,  1.11it/s]
 52%|█████▏    | 1557/2975 [26:32<21:15,  1.11it/s]
 52%|█████▏    | 1558/2975 [26:33<20:59,  1.12it/s]
 52%|█████▏    | 1559/2975 [26:34<20:46,  1.14it/s]
 52%|█████▏    | 1560/2975 [26:34<20:38,  1.14it/s]
                                       
0: {'loss': 0.5574, 'grad_norm': 0.41435977734969165, 'learning_rate': 7e-06, 'epoch': 0.52}
0: {'loss': 0.5641, 'grad_norm': 0.4082820260563453, 'learning_rate': 7e-06, 'epoch': 0.53}
0:             

 52%|█████▏    | 1560/2975 [26:34<20:38,  1.14it/s]
 52%|█████▏    | 1561/2975 [26:35<20:34,  1.15it/s]
 53%|█████▎    | 1562/2975 [26:36<20:29,  1.15it/s]
 53%|█████▎    | 1563/2975 [26:37<20:26,  1.15it/s]
 53%|█████▎    | 1564/2975 [26:38<20:22,  1.15it/s]
 53%|█████▎    | 1565/2975 [26:39<20:21,  1.15it/s]
 53%|█████▎    | 1566/2975 [26:40<20:19,  1.16it/s]
 53%|█████▎    | 1567/2975 [26:40<20:15,  1.16it/s]
 53%|█████▎    | 1568/2975 [26:41<20:23,  1.15it/s]
 53%|█████▎    | 1569/2975 [26:42<20:20,  1.15it/s]
 53%|█████▎    | 1570/2975 [26:43<20:27,  1.14it/s]
                                                   

 53%|█████▎    | 1570/2975 [26:43<20:27,  1.14it/s]
 53%|█████▎    | 1571/2975 [26:44<20:23,  1.15it/s]
 53%|█████▎    | 1572/2975 [26:45<20:21,  1.15it/s]
 53%|█████▎    | 1573/2975 [26:46<20:16,  1.15it/
0: {'loss': 0.5646, 'grad_norm': 0.40816329547280783, 'learning_rate': 7e-06, 'epoch': 0.53}
0: s]
 53%|█████▎    | 1574/2975 [26:47<20:13,  1.15it/s]
 53%|█████▎    | 1575/2975 [26:47<20:11,  1.16it/s]
 53%|█████▎    | 1576/2975 [26:48<20:08,  1.16it/s]
 53%|█████▎    | 1577/2975 [26:49<20:10,  1.16it/s]
 53%|█████▎    | 1578/2975 [26:50<20:05,  1.16it/s]
 53%|█████▎    | 1579/2975 [26:51<20:06,  1.16it/s]
 53%|█████▎    | 1580/2975 [26:52<20:04,  1.16it/s]
                                                   

 53%|█████▎    | 1580/2975 [26:52<20:04,  1.16it/s]
 53%|█████▎    | 1581/2975 [26:53<20:08,  1.15it/s]
 53%|█████▎    | 1582/2975 [26:54<20:06,  1.15it/s]
 53%|█████▎    | 1583/2975 [26:54<20:03,  1.16it/s]
 53%|█████▎    | 1584/2975 [26:55<20:02,  1.16it/s]
 53%|█████▎    | 1585/2975 [26:56<20:02,  1.16it/s]
 53%|█████▎    | 1586/2975 [26:57<20:02,  1.16it/s]
 53%|█████▎    | 1587/2975 [26:58<20:10,  1.15it/s]
 53%|█
0: {'loss': 0.5678, 'grad_norm': 0.4010583384310771, 'learning_rate': 7e-06, 'epoch': 0.53}
0: {'loss': 0.5634, 'grad_norm': 0.40195436628055986, 'learning_rate': 7e-06, 'epoch': 0.54}
0: ████▎    | 1588/2975 [26:59<20:07,  1.15it/s]
 53%|█████▎    | 1589/2975 [27:00<20:01,  1.15it/s]
 53%|█████▎    | 1590/2975 [27:00<19:58,  1.16it/s]
                                                   

 53%|█████▎    | 1590/2975 [27:00<19:58,  1.16it/s]
 53%|█████▎    | 1591/2975 [27:01<19:58,  1.15it/s]
 54%|█████▎    | 1592/2975 [27:02<19:54,  1.16it/s]
 54%|█████▎    | 1593/2975 [27:03<19:54,  1.16it/s]
 54%|█████▎    | 1594/2975 [27:04<19:52,  1.16it/s]
 54%|█████▎    | 1595/2975 [27:05<19:51,  1.16it/s]
 54%|█████▎    | 1596/2975 [27:06<19:49,  1.16it/s]
 54%|█████▎    | 1597/2975 [27:06<19:46,  1.16it/s]
 54%|█████▎    | 1598/2975 [27:07<19:47,  1.16it/s]
 54%|█████▎    | 1599/2975 [27:08<19:45,  1.16it/s]
 54%|█████▍    | 1600/2975 [27:09<19:45,  1.16it/s]
                                                   

 54%|█████▍    | 1
0: {'loss': 0.5588, 'grad_norm': 0.403342105243361, 'learning_rate': 7e-06, 'epoch': 0.54}
0: 600/2975 [27:09<19:45,  1.16it/s]
 54%|█████▍    | 1601/2975 [27:10<19:47,  1.16it/s]
 54%|█████▍    | 1602/2975 [27:11<19:44,  1.16it/s]
 54%|█████▍    | 1603/2975 [27:12<19:44,  1.16it/s]
 54%|█████▍    | 1604/2975 [27:13<19:42,  1.16it/s]
 54%|█████▍    | 1605/2975 [27:13<19:42,  1.16it/s]
 54%|█████▍    | 1606/2975 [27:14<19:46,  1.15it/s]
 54%|█████▍    | 1607/2975 [27:15<19:42,  1.16it/s]
 54%|█████▍    | 1608/2975 [27:16<19:42,  1.16it/s]
 54%|█████▍    | 1609/2975 [27:17<19:41,  1.16it/s]
 54%|█████▍    | 1610/2975 [27:18<19:39,  1.16it/s]
                                                   

 54%|█████▍    | 1610/2975 [27:18<19:39,  1.16it/s]
 54%|█████▍    | 1611/2975 [27:19<19:38,  1.16it/s]
 54%|█████▍    | 1612/2975 [27:19<19:35,  1.16it/s]
 54%|█████▍    | 1613/2975 [27:20<19:35,  1.16it/s]
 54%|█████▍    | 1614/2975 [2
0: {'loss': 0.5594, 'grad_norm': 0.39167586198618015, 'learning_rate': 7e-06, 'epoch': 0.54}
0: 7:21<19:35,  1.16it/s]
 54%|█████▍    | 1615/2975 [27:22<19:34,  1.16it/s]
 54%|█████▍    | 1616/2975 [27:23<19:34,  1.16it/s]
 54%|█████▍    | 1617/2975 [27:24<19:33,  1.16it/s]
 54%|█████▍    | 1618/2975 [27:25<19:32,  1.16it/s]
 54%|█████▍    | 1619/2975 [27:25<19:31,  1.16it/s]
 54%|█████▍    | 1620/2975 [27:26<19:29,  1.16it/s]
                                                   

 54%|█████▍    | 1620/2975 [27:26<19:29,  1.16it/s]
 54%|█████▍    | 1621/2975 [27:27<19:29,  1.16it/s]
 55%|█████▍    | 1622/2975 [27:28<19:29,  1.16it/s]
 55%|█████▍    | 1623/2975 [27:29<21:01,  1.07it/s]
 55%|█████▍    | 1624/2975 [27:30<20:32,  1.10it/s]
 55%|█████▍    | 1625/2975 [27:31<20:12,  1.11it/s]
 55%|█████▍    | 1626/2975 [27:32<19:57,  1.13it/s]
 55%|█████▍    | 1627/2975 [27:33<19:45,  1.14it/s]
 55%|█████▍    | 1628/2975 [27:33<19:39,
0: {'loss': 0.5702, 'grad_norm': 0.4040855228959046, 'learning_rate': 7e-06, 'epoch': 0.55}
0: {'loss': 0.5628, 'grad_norm': 0.432371511147914, 'learning_rate': 7e-06, 'epoch': 0.55}
0:   1.14it/s]
 55%|█████▍    | 1629/2975 [27:34<19:35,  1.15it/s]
 55%|█████▍    | 1630/2975 [27:35<19:29,  1.15it/s]
                                                   

 55%|█████▍    | 1630/2975 [27:35<19:29,  1.15it/s]
 55%|█████▍    | 1631/2975 [27:36<19:28,  1.15it/s]
 55%|█████▍    | 1632/2975 [27:37<19:24,  1.15it/s]
 55%|█████▍    | 1633/2975 [27:38<19:23,  1.15it/s]
 55%|█████▍    | 1634/2975 [27:39<19:21,  1.15it/s]
 55%|█████▍    | 1635/2975 [27:40<19:18,  1.16it/s]
 55%|█████▍    | 1636/2975 [27:40<19:19,  1.16it/s]
 55%|█████▌    | 1637/2975 [27:41<19:18,  1.15it/s]
 55%|█████▌    | 1638/2975 [27:42<19:16,  1.16it/s]
 55%|█████▌    | 1639/2975 [27:43<19:18,  1.15it/s]
 55%|█████▌    | 1640/2975 [27:44<19:14,  1.16it/s]
                                                   

 55%|█████▌    | 1640/2975 [27:44<19:14,  1.16it/s]
 55%|█�
0: {'loss': 0.5664, 'grad_norm': 0.4091954446435366, 'learning_rate': 7e-06, 'epoch': 0.55}
0: �███▌    | 1641/2975 [27:45<19:14,  1.16it/s]
 55%|█████▌    | 1642/2975 [27:46<19:11,  1.16it/s]
 55%|█████▌    | 1643/2975 [27:46<19:09,  1.16it/s]
 55%|█████▌    | 1644/2975 [27:47<19:10,  1.16it/s]
 55%|█████▌    | 1645/2975 [27:48<19:07,  1.16it/s]
 55%|█████▌    | 1646/2975 [27:49<19:07,  1.16it/s]
 55%|█████▌    | 1647/2975 [27:50<19:05,  1.16it/s]
 55%|█████▌    | 1648/2975 [27:51<19:03,  1.16it/s]
 55%|█████▌    | 1649/2975 [27:52<19:04,  1.16it/s]
 55%|█████▌    | 1650/2975 [27:52<19:02,  1.16it/s]
                                                   

 55%|█████▌    | 1650/2975 [27:52<19:02,  1.16it/s]
 55%|█████▌    | 1651/2975 [27:54<20:21,  1.08it/s]
 56%|█████▌    | 1652/2975 [27:54<19:56,  1.11it/s]
 56%|█████▌    | 1653/2975 [27:55<19:38,  1.12it/s]
 56%|█████▌    | 1654/2975 [27:56<19:30,  1.13it/s]
 56%|█████�
0: {'loss': 0.5686, 'grad_norm': 0.39347281820088204, 'learning_rate': 7e-06, 'epoch': 0.56}
0: ��    | 1655/2975 [27:57<19:22,  1.14it/s]
 56%|█████▌    | 1656/2975 [27:58<19:13,  1.14it/s]
 56%|█████▌    | 1657/2975 [27:59<19:07,  1.15it/s]
 56%|█████▌    | 1658/2975 [28:00<19:02,  1.15it/s]
 56%|█████▌    | 1659/2975 [28:00<18:58,  1.16it/s]
 56%|█████▌    | 1660/2975 [28:01<18:57,  1.16it/s]
                                                   

 56%|█████▌    | 1660/2975 [28:01<18:57,  1.16it/s]
 56%|█████▌    | 1661/2975 [28:02<18:57,  1.16it/s]
 56%|█████▌    | 1662/2975 [28:03<18:55,  1.16it/s]
 56%|█████▌    | 1663/2975 [28:04<18:55,  1.16it/s]
 56%|█████▌    | 1664/2975 [28:05<18:52,  1.16it/s]
 56%|█████▌    | 1665/2975 [28:06<19:02,  1.15it/s]
 56%|█████▌    | 1666/2975 [28:07<18:59,  1.15it/s]
 56%|█████▌    | 1667/2975 [28:07<18:59,  1.15it/s]
 56%|█████▌    | 1668/2975 [28:08<18:54,  1.15it/s]
 56%|█████▌    | 166
0: {'loss': 0.5591, 'grad_norm': 0.4050726797166232, 'learning_rate': 7e-06, 'epoch': 0.56}
0: {'loss': 0.5665, 'grad_norm': 0.4059998078746424, 'learning_rate': 7e-06, 'epoch': 0.56}
0: 9/2975 [28:09<18:52,  1.15it/s]
 56%|█████▌    | 1670/2975 [28:10<18:48,  1.16it/s]
                                                   

 56%|█████▌    | 1670/2975 [28:10<18:48,  1.16it/s]
 56%|█████▌    | 1671/2975 [28:11<18:49,  1.15it/s]
 56%|█████▌    | 1672/2975 [28:12<18:46,  1.16it/s]
 56%|█████▌    | 1673/2975 [28:13<18:44,  1.16it/s]
 56%|█████▋    | 1674/2975 [28:13<18:43,  1.16it/s]
 56%|█████▋    | 1675/2975 [28:14<18:40,  1.16it/s]
 56%|█████▋    | 1676/2975 [28:15<18:39,  1.16it/s]
 56%|█████▋    | 1677/2975 [28:16<18:38,  1.16it/s]
 56%|█████▋    | 1678/2975 [28:17<18:38,  1.16it/s]
 56%|█████▋    | 1679/2975 [28:18<20:08,  1.07it/s]
 56%|█████▋    | 1680/2975 [28:19<19:41,  1.10it/s]
                                                   

 56%|█████▋    | 1680/2975 [28:19<19:41,  1.10it/s]
 57%|█████▋    | 1681/2975 [28:20<19:22,  
0: {'loss': 0.5598, 'grad_norm': 0.3932040808779393, 'learning_rate': 7e-06, 'epoch': 0.57}
0: 1.11it/s]
 57%|█████▋    | 1682/2975 [28:21<19:09,  1.12it/s]
 57%|█████▋    | 1683/2975 [28:21<18:59,  1.13it/s]
 57%|█████▋    | 1684/2975 [28:22<18:50,  1.14it/s]
 57%|█████▋    | 1685/2975 [28:23<18:43,  1.15it/s]
 57%|█████▋    | 1686/2975 [28:24<20:05,  1.07it/s]
 57%|█████▋    | 1687/2975 [28:25<19:35,  1.10it/s]
 57%|█████▋    | 1688/2975 [28:26<19:13,  1.12it/s]
 57%|█████▋    | 1689/2975 [28:27<18:57,  1.13it/s]
 57%|█████▋    | 1690/2975 [28:28<18:46,  1.14it/s]
                                                   

 57%|█████▋    | 1690/2975 [28:28<18:46,  1.14it/s]
 57%|█████▋    | 1691/2975 [28:29<18:41,  1.14it/s]
 57%|█████▋    | 1692/2975 [28:29<18:34,  1.15it/s]
 57%|█████▋    | 1693/2975 [28:30<18:30,  1.15it/s]
 57%|█████▋    | 1694/2975 [28:31<18:27,  1.16it/s]
 57%|█████▋    | 1695/2975 [28:32<18:25,  1.16it/s]
 
0: {'loss': 0.5454, 'grad_norm': 0.3837821321204197, 'learning_rate': 7e-06, 'epoch': 0.57}
0: 57%|█████▋    | 1696/2975 [28:33<18:24,  1.16it/s]
 57%|█████▋    | 1697/2975 [28:34<18:22,  1.16it/s]
 57%|█████▋    | 1698/2975 [28:35<18:21,  1.16it/s]
 57%|█████▋    | 1699/2975 [28:35<18:19,  1.16it/s]
 57%|█████▋    | 1700/2975 [28:36<18:40,  1.14it/s]
                                                   

 57%|█████▋    | 1700/2975 [28:36<18:40,  1.14it/s]
 57%|█████▋    | 1701/2975 [28:37<18:33,  1.14it/s]
 57%|█████▋    | 1702/2975 [28:38<18:28,  1.15it/s]
 57%|█████▋    | 1703/2975 [28:39<18:24,  1.15it/s]
 57%|█████▋    | 1704/2975 [28:40<18:20,  1.15it/s]
 57%|█████▋    | 1705/2975 [28:41<18:17,  1.16it/s]
 57%|█████▋    | 1706/2975 [28:42<19:39,  1.08it/s]
 57%|█████▋    | 1707/2975 [28:43<19:12,  1.10it/s]
 57%|█████▋    | 1708/2975 [28:44<19:51,  1.06it/s]
 57%|█████▋    | 1709/2975 [28:44<19:19,  1.09it/s]
 57%|██�
0: {'loss': 0.5674, 'grad_norm': 0.39828327409289, 'learning_rate': 7e-06, 'epoch': 0.57}
0: {'loss': 0.5721, 'grad_norm': 0.38824843939076636, 'learning_rate': 7e-06, 'epoch': 0.58}
0: ��██▋    | 1710/2975 [28:45<18:58,  1.11it/s]
                                                   

 57%|█████▋    | 1710/2975 [28:45<18:58,  1.11it/s]
 58%|█████▊    | 1711/2975 [28:46<18:45,  1.12it/s]
 58%|█████▊    | 1712/2975 [28:47<18:52,  1.11it/s]
 58%|█████▊    | 1713/2975 [28:48<18:36,  1.13it/s]
 58%|█████▊    | 1714/2975 [28:49<18:27,  1.14it/s]
 58%|█████▊    | 1715/2975 [28:50<18:20,  1.14it/s]
 58%|█████▊    | 1716/2975 [28:51<18:15,  1.15it/s]
 58%|█████▊    | 1717/2975 [28:51<18:14,  1.15it/s]
 58%|█████▊    | 1718/2975 [28:52<18:11,  1.15it/s]
 58%|█████▊    | 1719/2975 [28:53<18:08,  1.15it/s]
 58%|█████▊    | 1720/2975 [28:54<18:06,  1.15it/s]
                                                   

 58%|█████▊    | 1720/2975 [28:54<18:06,  1.15it/s]
 58%|█████▊    | 1721/2975 [28:55<18:05,  1.16it/s]
 58%|█████▊    | 1722/
0: {'loss': 0.5482, 'grad_norm': 0.39933126300596383, 'learning_rate': 7e-06, 'epoch': 0.58}
0: 2975 [28:56<18:04,  1.16it/s]
 58%|█████▊    | 1723/2975 [28:57<18:06,  1.15it/s]
 58%|█████▊    | 1724/2975 [28:58<18:02,  1.16it/s]
 58%|█████▊    | 1725/2975 [28:58<18:01,  1.16it/s]
 58%|█████▊    | 1726/2975 [28:59<18:25,  1.13it/s]
 58%|█████▊    | 1727/2975 [29:00<18:18,  1.14it/s]
 58%|█████▊    | 1728/2975 [29:01<18:10,  1.14it/s]
 58%|█████▊    | 1729/2975 [29:02<18:06,  1.15it/s]
 58%|█████▊    | 1730/2975 [29:03<19:31,  1.06it/s]
                                                   

 58%|█████▊    | 1730/2975 [29:03<19:31,  1.06it/s]
 58%|█████▊    | 1731/2975 [29:04<19:04,  1.09it/s]
 58%|█████▊    | 1732/2975 [29:05<18:40,  1.11it/s]
 58%|█████▊    | 1733/2975 [29:06<18:31,  1.12it/s]
 58%|█████▊    | 1734/2975 [29:06<18:20,  1.13it/s]
 58%|█████▊    | 1735/2975 [29:07<18:09,  1.14it/s]
 58%|█████▊    | 1736/2975 [29:08
0: {'loss': 0.5724, 'grad_norm': 0.4193204568073356, 'learning_rate': 7e-06, 'epoch': 0.58}
0: <18:03,  1.14it/s]
 58%|█████▊    | 1737/2975 [29:09<18:15,  1.13it/s]
 58%|█████▊    | 1738/2975 [29:10<18:06,  1.14it/s]
 58%|█████▊    | 1739/2975 [29:11<17:58,  1.15it/s]
 58%|█████▊    | 1740/2975 [29:12<17:55,  1.15it/s]
                                                   

 58%|█████▊    | 1740/2975 [29:12<17:55,  1.15it/s]
 59%|█████▊    | 1741/2975 [29:13<17:51,  1.15it/s]
 59%|█████▊    | 1742/2975 [29:13<17:49,  1.15it/s]
 59%|█████▊    | 1743/2975 [29:14<17:46,  1.15it/s]
 59%|█████▊    | 1744/2975 [29:15<17:43,  1.16it/s]
 59%|█████▊    | 1745/2975 [29:16<17:42,  1.16it/s]
 59%|█████▊    | 1746/2975 [29:17<17:38,  1.16it/s]
 59%|█████▊    | 1747/2975 [29:18<17:38,  1.16it/s]
 59%|█████▉    | 1748/2975 [29:19<17:38,  1.16it/s]
 59%|█████▉    | 1749/2975 [29:19<17:37,  1.16it/s]
 59%|█████▉    | 1750/2975 [29:20<17:38,  1.
0: {'loss': 0.5609, 'grad_norm': 0.398111854931816, 'learning_rate': 7e-06, 'epoch': 0.59}
0: {'loss': 0.5566, 'grad_norm': 0.4056668732444875, 'learning_rate': 7e-06, 'epoch': 0.59}
0: 16it/s]
                                                   

 59%|█████▉    | 1750/2975 [29:20<17:38,  1.16it/s]
 59%|█████▉    | 1751/2975 [29:21<18:24,  1.11it/s]
 59%|█████▉    | 1752/2975 [29:22<18:38,  1.09it/s]
 59%|█████▉    | 1753/2975 [29:23<18:17,  1.11it/s]
 59%|█████▉    | 1754/2975 [29:24<18:02,  1.13it/s]
 59%|█████▉    | 1755/2975 [29:25<17:54,  1.14it/s]
 59%|█████▉    | 1756/2975 [29:26<18:58,  1.07it/s]
 59%|█████▉    | 1757/2975 [29:27<18:31,  1.10it/s]
 59%|█████▉    | 1758/2975 [29:28<18:12,  1.11it/s]
 59%|█████▉    | 1759/2975 [29:28<17:57,  1.13it/s]
 59%|█████▉    | 1760/2975 [29:29<17:49,  1.14it/s]
                                                   

 59%|█████▉    | 1760/2975 [29:29<17:49,  1.14it/s]
 59%|█████▉    | 1761/2975 [29:30<17:40,  1.14it/s]
 59%|█████▉    | 1762/2975 [29:31<17:35,  1.15it/s]
 59%|███
0: {'loss': 0.5498, 'grad_norm': 0.3883113014686659, 'learning_rate': 7e-06, 'epoch': 0.59}
0: ██▉    | 1763/2975 [29:32<17:34,  1.15it/s]
 59%|█████▉    | 1764/2975 [29:33<17:29,  1.15it/s]
 59%|█████▉    | 1765/2975 [29:34<17:44,  1.14it/s]
 59%|█████▉    | 1766/2975 [29:35<17:38,  1.14it/s]
 59%|█████▉    | 1767/2975 [29:35<17:39,  1.14it/s]
 59%|█████▉    | 1768/2975 [29:36<17:32,  1.15it/s]
 59%|█████▉    | 1769/2975 [29:37<18:09,  1.11it/s]
 59%|█████▉    | 1770/2975 [29:38<17:54,  1.12it/s]
                                                   

 59%|█████▉    | 1770/2975 [29:38<17:54,  1.12it/s]
 60%|█████▉    | 1771/2975 [29:39<17:43,  1.13it/s]
 60%|█████▉    | 1772/2975 [29:40<18:33,  1.08it/s]
 60%|█████▉    | 1773/2975 [29:41<18:11,  1.10it/s]
 60%|█████▉    | 1774/2975 [29:42<17:57,  1.11it/s]
 60%|█████▉    | 1775/2975 [29:43<17:43,  1.13it/s]
 60%|█████▉    | 1776/2975 [29:44<17:34,  1.14it/s]
 60%|█████▉  
0: {'loss': 0.5648, 'grad_norm': 0.39528061604176284, 'learning_rate': 7e-06, 'epoch': 0.6}
0:   | 1777/2975 [29:44<17:25,  1.15it/s]
 60%|█████▉    | 1778/2975 [29:45<17:20,  1.15it/s]
 60%|█████▉    | 1779/2975 [29:46<17:17,  1.15it/s]
 60%|█████▉    | 1780/2975 [29:47<17:12,  1.16it/s]
                                                   

 60%|█████▉    | 1780/2975 [29:47<17:12,  1.16it/s]
 60%|█████▉    | 1781/2975 [29:48<17:12,  1.16it/s]
 60%|█████▉    | 1782/2975 [29:49<17:12,  1.16it/s]
 60%|█████▉    | 1783/2975 [29:50<17:10,  1.16it/s]
 60%|█████▉    | 1784/2975 [29:50<17:09,  1.16it/s]
 60%|██████    | 1785/2975 [29:51<17:08,  1.16it/s]
 60%|██████    | 1786/2975 [29:56<42:59,  2.17s/it]
 60%|██████    | 1787/2975 [29:57<35:12,  1.78s/it]
 60%|██████    | 1788/2975 [29:58<29:45,  1.50s/it]
 60%|██████    | 1789/2975 [29:59<25:55,  1.31s/it]
 60%|██████    | 1790/2975 [30:00<23:16,  1.18s/it]
                                    
0: {'loss': 0.5576, 'grad_norm': 0.4078923319660163, 'learning_rate': 7e-06, 'epoch': 0.6}
0: {'loss': 0.5655, 'grad_norm': 0.4036931255817094, 'learning_rate': 7e-06, 'epoch': 0.6}
0:                

 60%|██████    | 1790/2975 [30:00<23:16,  1.18s/it]
 60%|██████    | 1791/2975 [30:01<21:22,  1.08s/it]
 60%|██████    | 1792/2975 [30:02<20:03,  1.02s/it]
 60%|██████    | 1793/2975 [30:03<19:07,  1.03it/s]
 60%|██████    | 1794/2975 [30:03<18:29,  1.06it/s]
 60%|██████    | 1795/2975 [30:04<18:02,  1.09it/s]
 60%|██████    | 1796/2975 [30:05<17:41,  1.11it/s]
 60%|██████    | 1797/2975 [30:06<17:26,  1.13it/s]
 60%|██████    | 1798/2975 [30:07<17:16,  1.14it/s]
 60%|██████    | 1799/2975 [30:08<17:09,  1.14it/s]
 61%|██████    | 1800/2975 [30:09<17:08,  1.14it/s]
                                                   

 61%|██████    | 1800/2975 [30:09<17:08,  1.14it/s]
 61%|██████    | 1801/2975 [30:09<17:04,  1.15it/s]
 61%|██████    | 1802/2975 [30:10<17:00,  1.15it/s]
 61%|██████    | 1803/2975 [30:11<16:57,  1.15
0: {'loss': 0.5663, 'grad_norm': 0.40792793988169607, 'learning_rate': 7e-06, 'epoch': 0.61}
0: it/s]
 61%|██████    | 1804/2975 [30:12<16:54,  1.15it/s]
 61%|██████    | 1805/2975 [30:13<17:18,  1.13it/s]
 61%|██████    | 1806/2975 [30:14<17:12,  1.13it/s]
 61%|██████    | 1807/2975 [30:15<17:10,  1.13it/s]
 61%|██████    | 1808/2975 [30:16<17:04,  1.14it/s]
 61%|██████    | 1809/2975 [30:17<17:22,  1.12it/s]
 61%|██████    | 1810/2975 [30:17<17:10,  1.13it/s]
                                                   

 61%|██████    | 1810/2975 [30:17<17:10,  1.13it/s]
 61%|██████    | 1811/2975 [30:18<17:03,  1.14it/s]
 61%|██████    | 1812/2975 [30:19<16:57,  1.14it/s]
 61%|██████    | 1813/2975 [30:20<16:51,  1.15it/s]
 61%|██████    | 1814/2975 [30:21<16:49,  1.15it/s]
 61%|██████    | 1815/2975 [30:22<16:47,  1.15it/s]
 61%|██████    | 1816/2975 [30:23<16:44,  1.15it/s]
 61%|██████    | 1817/2975 [30:23<16:43,  1.15it/s]
 61%|
0: {'loss': 0.5617, 'grad_norm': 0.4145737266922681, 'learning_rate': 7e-06, 'epoch': 0.61}
0: {'loss': 0.5622, 'grad_norm': 0.3896354933521576, 'learning_rate': 7e-06, 'epoch': 0.61}
0: ██████    | 1818/2975 [30:24<16:41,  1.16it/s]
 61%|██████    | 1819/2975 [30:25<16:40,  1.16it/s]
 61%|██████    | 1820/2975 [30:26<16:39,  1.16it/s]
                                                   

 61%|██████    | 1820/2975 [30:26<16:39,  1.16it/s]
 61%|██████    | 1821/2975 [30:27<16:39,  1.15it/s]
 61%|██████    | 1822/2975 [30:28<16:37,  1.16it/s]
 61%|██████▏   | 1823/2975 [30:29<16:34,  1.16it/s]
 61%|██████▏   | 1824/2975 [30:29<16:32,  1.16it/s]
 61%|██████▏   | 1825/2975 [30:30<16:31,  1.16it/s]
 61%|██████▏   | 1826/2975 [30:31<16:30,  1.16it/s]
 61%|██████▏   | 1827/2975 [30:32<16:30,  1.16it/s]
 61%|██████▏   | 1828/2975 [30:33<16:28,  1.16it/s]
 61%|██████▏   | 1829/2975 [30:34<16:28,  1.16it/s]
 62%|██████▏   | 1830/2975 [30:35<16:27,  1.16it/s]
                                                   

 62%|██
0: {'loss': 0.5574, 'grad_norm': 0.3891532264301837, 'learning_rate': 7e-06, 'epoch': 0.62}
0: ████▏   | 1830/2975 [30:35<16:27,  1.16it/s]
 62%|██████▏   | 1831/2975 [30:36<16:27,  1.16it/s]
 62%|██████▏   | 1832/2975 [30:36<16:26,  1.16it/s]
 62%|██████▏   | 1833/2975 [30:37<16:36,  1.15it/s]
 62%|██████▏   | 1834/2975 [30:38<16:31,  1.15it/s]
 62%|██████▏   | 1835/2975 [30:39<16:28,  1.15it/s]
 62%|██████▏   | 1836/2975 [30:40<17:10,  1.10it/s]
 62%|██████▏   | 1837/2975 [30:41<16:55,  1.12it/s]
 62%|██████▏   | 1838/2975 [30:42<16:44,  1.13it/s]
 62%|██████▏   | 1839/2975 [30:43<16:37,  1.14it/s]
 62%|██████▏   | 1840/2975 [30:43<16:31,  1.14it/s]
                                                   

 62%|██████▏   | 1840/2975 [30:43<16:31,  1.14it/s]
 62%|██████▏   | 1841/2975 [30:44<16:27,  1.15it/s]
 62%|██████▏   | 1842/2975 [30:45<16:23,  1.15it/s]
 62%|██████▏   | 1843/2975 [30:46<16:20,  1.
0: {'loss': 0.5479, 'grad_norm': 0.39085626719204736, 'learning_rate': 7e-06, 'epoch': 0.62}
0: 15it/s]
 62%|██████▏   | 1844/2975 [30:47<16:19,  1.15it/s]
 62%|██████▏   | 1845/2975 [30:48<16:16,  1.16it/s]
 62%|██████▏   | 1846/2975 [30:49<16:14,  1.16it/s]
 62%|██████▏   | 1847/2975 [30:49<16:12,  1.16it/s]
 62%|██████▏   | 1848/2975 [30:50<16:11,  1.16it/s]
 62%|██████▏   | 1849/2975 [30:51<16:44,  1.12it/s]
 62%|██████▏   | 1850/2975 [30:52<16:32,  1.13it/s]
                                                   

 62%|██████▏   | 1850/2975 [30:52<16:32,  1.13it/s]
 62%|██████▏   | 1851/2975 [30:53<16:26,  1.14it/s]
 62%|██████▏   | 1852/2975 [30:54<16:20,  1.15it/s]
 62%|██████▏   | 1853/2975 [30:55<16:15,  1.15it/s]
 62%|██████▏   | 1854/2975 [30:56<16:14,  1.15it/s]
 62%|██████▏   | 1855/2975 [30:56<16:10,  1.15it/s]
 62%|██████▏   | 1856/2975 [30:57<16:09,  1.15it/s]
 62%|██████▏   | 1857/297
0: {'loss': 0.5658, 'grad_norm': 0.40539954082947327, 'learning_rate': 7e-06, 'epoch': 0.62}
0: 5 [30:58<16:13,  1.15it/s]
 62%|██████▏   | 1858/2975 [30:59<16:23,  1.14it/s]
 62%|██████▏   | 1859/2975 [31:00<16:15,  1.14it/s]
 63%|██████▎   | 1860/2975 [31:01<16:11,  1.15it/s]
                                                   

 63%|██████▎   | 1860/2975 [31:01<16:11,  1.15it/s]
 63%|██████▎   | 1861/2975 [31:02<16:10,  1.15it/s]
 63%|██████▎   | 1862/2975 [31:03<16:06,  1.15it/s]
 63%|██████▎   | 1863/2975 [31:03<16:03,  1.15it/s]
 63%|██████▎   | 1864/2975 [31:04<16:00,  1.16it/s]
 63%|██████▎   | 1865/2975 [31:05<15:58,  1.16it/s]
 63%|██████▎   | 1866/2975 [31:06<15:58,  1.16it/s]
 63%|██████▎   | 1867/2975 [31:07<15:56,  1.16it/s]
 63%|██████▎   | 1868/2975 [31:08<15:56,  1.16it/s]
 63%|██████▎   | 1869/2975 [31:09<15:55,  1.16it/s]
 63%|██████▎   | 1870/2975 [31:10<15:53,  1.16it/s]
                    
0: {'loss': 0.5806, 'grad_norm': 0.3947871965460968, 'learning_rate': 7e-06, 'epoch': 0.63}
0: {'loss': 0.5536, 'grad_norm': 0.3925122106914771, 'learning_rate': 7e-06, 'epoch': 0.63}
0:                                

 63%|██████▎   | 1870/2975 [31:10<15:53,  1.16it/s]
 63%|██████▎   | 1871/2975 [31:10<16:03,  1.15it/s]
 63%|██████▎   | 1872/2975 [31:11<16:01,  1.15it/s]
 63%|██████▎   | 1873/2975 [31:12<15:58,  1.15it/s]
 63%|██████▎   | 1874/2975 [31:13<15:56,  1.15it/s]
 63%|██████▎   | 1875/2975 [31:14<15:54,  1.15it/s]
 63%|██████▎   | 1876/2975 [31:15<15:50,  1.16it/s]
 63%|██████▎   | 1877/2975 [31:16<15:49,  1.16it/s]
 63%|██████▎   | 1878/2975 [31:16<15:47,  1.16it/s]
 63%|██████▎   | 1879/2975 [31:17<15:46,  1.16it/s]
 63%|██████▎   | 1880/2975 [31:18<15:46,  1.16it/s]
                                                   

 63%|██████▎   | 1880/2975 [31:18<15:46,  1.16it/s]
 63%|██████▎   | 1881/2975 [31:19<15:44,  1.16it/s]
 63%|██████▎   | 1882/2975 [31:20<15:42,  1.16it/s]
 63%|███
0: {'loss': 0.5526, 'grad_norm': 0.37722990439301946, 'learning_rate': 7e-06, 'epoch': 0.64}
0: ███▎   | 1883/2975 [31:21<15:41,  1.16it/s]
 63%|██████▎   | 1884/2975 [31:22<15:39,  1.16it/s]
 63%|██████▎   | 1885/2975 [31:23<16:14,  1.12it/s]
 63%|██████▎   | 1886/2975 [31:23<16:02,  1.13it/s]
 63%|██████▎   | 1887/2975 [31:24<15:54,  1.14it/s]
 63%|██████▎   | 1888/2975 [31:25<15:49,  1.14it/s]
 63%|██████▎   | 1889/2975 [31:26<15:45,  1.15it/s]
 64%|██████▎   | 1890/2975 [31:27<15:42,  1.15it/s]
                                                   

 64%|██████▎   | 1890/2975 [31:27<15:42,  1.15it/s]
 64%|██████▎   | 1891/2975 [31:28<15:39,  1.15it/s]
 64%|██████▎   | 1892/2975 [31:29<15:37,  1.16it/s]
 64%|██████▎   | 1893/2975 [31:29<15:36,  1.16it/s]
 64%|██████▎   | 1894/2975 [31:30<15:33,  1.16it/s]
 64%|██████▎   | 1895/2975 [31:31<15:32,  1.16it/s]
 64%|██████▎   | 1896/2975 [31:32<15:32,  1.16i
0: {'loss': 0.563, 'grad_norm': 0.41896408747540165, 'learning_rate': 7e-06, 'epoch': 0.64}
0: t/s]
 64%|██████▍   | 1897/2975 [31:33<15:30,  1.16it/s]
 64%|██████▍   | 1898/2975 [31:34<15:30,  1.16it/s]
 64%|██████▍   | 1899/2975 [31:35<15:43,  1.14it/s]
 64%|██████▍   | 1900/2975 [31:36<15:37,  1.15it/s]
                                                   

 64%|██████▍   | 1900/2975 [31:36<15:37,  1.15it/s]
 64%|██████▍   | 1901/2975 [31:36<15:36,  1.15it/s]
 64%|██████▍   | 1902/2975 [31:37<15:33,  1.15it/s]
 64%|██████▍   | 1903/2975 [31:38<15:29,  1.15it/s]
 64%|██████▍   | 1904/2975 [31:39<15:28,  1.15it/s]
 64%|██████▍   | 1905/2975 [31:40<15:25,  1.16it/s]
 64%|██████▍   | 1906/2975 [31:41<15:24,  1.16it/s]
 64%|██████▍   | 1907/2975 [31:42<15:24,  1.16it/s]
 64%|██████▍   | 1908/2975 [31:42<15:21,  1.16it/s]
 64%|██████▍   | 1909/2975 [31:43<15:26,  1.15it/s]
 64%|██████▍   | 1910/2975 [
0: {'loss': 0.5844, 'grad_norm': 0.39344732430437723, 'learning_rate': 7e-06, 'epoch': 0.64}
0: {'loss': 0.5684, 'grad_norm': 0.40704663621715076, 'learning_rate': 7e-06, 'epoch': 0.65}
0: 31:44<15:26,  1.15it/s]
                                                   

 64%|██████▍   | 1910/2975 [31:44<15:26,  1.15it/s]
 64%|██████▍   | 1911/2975 [31:45<15:22,  1.15it/s]
 64%|██████▍   | 1912/2975 [31:46<15:22,  1.15it/s]
 64%|██████▍   | 1913/2975 [31:47<15:21,  1.15it/s]
 64%|██████▍   | 1914/2975 [31:48<15:18,  1.15it/s]
 64%|██████▍   | 1915/2975 [31:49<15:17,  1.16it/s]
 64%|██████▍   | 1916/2975 [31:49<15:14,  1.16it/s]
 64%|██████▍   | 1917/2975 [31:50<15:13,  1.16it/s]
 64%|██████▍   | 1918/2975 [31:51<15:13,  1.16it/s]
 65%|██████▍   | 1919/2975 [31:52<15:10,  1.16it/s]
 65%|██████▍   | 1920/2975 [31:53<15:11,  1.16it/s]
                                                   

 65%|██████▍   | 1920/2975 [31:53<15:11,  1.16it/s]
 65%|██████▍   | 1921/2975 [31:54<15:14,  1.15it/s]
 65%|██████▍   | 1922/
0: {'loss': 0.581, 'grad_norm': 0.39414672947721546, 'learning_rate': 7e-06, 'epoch': 0.65}
0: 2975 [31:55<16:06,  1.09it/s]
 65%|██████▍   | 1923/2975 [31:56<15:48,  1.11it/s]
 65%|██████▍   | 1924/2975 [31:57<15:34,  1.13it/s]
 65%|██████▍   | 1925/2975 [31:57<15:27,  1.13it/s]
 65%|██████▍   | 1926/2975 [31:58<15:18,  1.14it/s]
 65%|██████▍   | 1927/2975 [31:59<15:13,  1.15it/s]
 65%|██████▍   | 1928/2975 [32:00<15:10,  1.15it/s]
 65%|██████▍   | 1929/2975 [32:01<16:10,  1.08it/s]
 65%|██████▍   | 1930/2975 [32:02<15:49,  1.10it/s]
                                                   

 65%|██████▍   | 1930/2975 [32:02<15:49,  1.10it/s]
 65%|██████▍   | 1931/2975 [32:03<15:32,  1.12it/s]
 65%|██████▍   | 1932/2975 [32:04<15:21,  1.13it/s]
 65%|██████▍   | 1933/2975 [32:04<15:12,  1.14it/s]
 65%|██████▌   | 1934/2975 [32:05<15:15,  1.14it/s]
 65%|██████▌   | 1935/2975 [32:06<16:13,  1.07it/s]
 65%|████
0: {'loss': 0.5544, 'grad_norm': 0.41275820069085883, 'learning_rate': 7e-06, 'epoch': 0.65}
0: ██▌   | 1936/2975 [32:07<15:50,  1.09it/s]
 65%|██████▌   | 1937/2975 [32:08<15:31,  1.11it/s]
 65%|██████▌   | 1938/2975 [32:09<15:20,  1.13it/s]
 65%|██████▌   | 1939/2975 [32:10<15:09,  1.14it/s]
 65%|██████▌   | 1940/2975 [32:11<15:03,  1.15it/s]
                                                   

 65%|██████▌   | 1940/2975 [32:11<15:03,  1.15it/s]
 65%|██████▌   | 1941/2975 [32:12<14:59,  1.15it/s]
 65%|██████▌   | 1942/2975 [32:12<14:54,  1.15it/s]
 65%|██████▌   | 1943/2975 [32:13<14:54,  1.15it/s]
 65%|██████▌   | 1944/2975 [32:14<14:49,  1.16it/s]
 65%|██████▌   | 1945/2975 [32:15<15:11,  1.13it/s]
 65%|██████▌   | 1946/2975 [32:16<15:02,  1.14it/s]
 65%|██████▌   | 1947/2975 [32:17<14:57,  1.14it/s]
 65%|██████▌   | 1948/2975 [32:18<14:50,  1.15it/s]
 66%|██████▌   | 1949/2975 [32:19<14:49,  1.15it/s
0: {'loss': 0.5436, 'grad_norm': 0.424079766559486, 'learning_rate': 7e-06, 'epoch': 0.66}
0: {'loss': 0.5463, 'grad_norm': 0.39409835497060103, 'learning_rate': 7e-06, 'epoch': 0.66}
0: ]
 66%|██████▌   | 1950/2975 [32:19<14:46,  1.16it/s]
                                                   

 66%|██████▌   | 1950/2975 [32:19<14:46,  1.16it/s]
 66%|██████▌   | 1951/2975 [32:20<14:44,  1.16it/s]
 66%|██████▌   | 1952/2975 [32:21<14:46,  1.15it/s]
 66%|██████▌   | 1953/2975 [32:22<14:42,  1.16it/s]
 66%|██████▌   | 1954/2975 [32:23<14:42,  1.16it/s]
 66%|██████▌   | 1955/2975 [32:24<14:41,  1.16it/s]
 66%|██████▌   | 1956/2975 [32:25<14:39,  1.16it/s]
 66%|██████▌   | 1957/2975 [32:25<14:40,  1.16it/s]
 66%|██████▌   | 1958/2975 [32:27<15:36,  1.09it/s]
 66%|██████▌   | 1959/2975 [32:27<15:19,  1.11it/s]
 66%|██████▌   | 1960/2975 [32:28<15:06,  1.12it/s]
                                                   

 66%|██████▌   | 1960/2975 [32:28<15:06,  1.12it/s]
 66%|██████▌   | 1961/2975 [32:29<14:57,  1.
0: {'loss': 0.5515, 'grad_norm': 0.40593854699947174, 'learning_rate': 7e-06, 'epoch': 0.66}
0: 13it/s]
 66%|██████▌   | 1962/2975 [32:30<14:51,  1.14it/s]
 66%|██████▌   | 1963/2975 [32:31<14:44,  1.14it/s]
 66%|██████▌   | 1964/2975 [32:32<14:39,  1.15it/s]
 66%|██████▌   | 1965/2975 [32:33<14:37,  1.15it/s]
 66%|██████▌   | 1966/2975 [32:33<14:32,  1.16it/s]
 66%|██████▌   | 1967/2975 [32:34<14:32,  1.16it/s]
 66%|██████▌   | 1968/2975 [32:35<14:29,  1.16it/s]
 66%|██████▌   | 1969/2975 [32:36<14:29,  1.16it/s]
 66%|██████▌   | 1970/2975 [32:37<14:28,  1.16it/s]
                                                   

 66%|██████▌   | 1970/2975 [32:37<14:28,  1.16it/s]
 66%|██████▋   | 1971/2975 [32:38<14:27,  1.16it/s]
 66%|██████▋   | 1972/2975 [32:39<14:33,  1.15it/s]
 66%|██████▋   | 1973/2975 [32:40<14:31,  1.15it/s]
 66%|██████▋   | 1974/2975 [32:40<14:27,  1.15it/s]
 66%|██████▋   | 1975/297
0: {'loss': 0.5524, 'grad_norm': 0.42373997498203675, 'learning_rate': 7e-06, 'epoch': 0.67}
0: 5 [32:41<14:26,  1.15it/s]
 66%|██████▋   | 1976/2975 [32:42<14:25,  1.15it/s]
 66%|██████▋   | 1977/2975 [32:43<15:01,  1.11it/s]
 66%|██████▋   | 1978/2975 [32:44<14:48,  1.12it/s]
 67%|██████▋   | 1979/2975 [32:45<14:40,  1.13it/s]
 67%|██████▋   | 1980/2975 [32:46<14:32,  1.14it/s]
                                                   

 67%|██████▋   | 1980/2975 [32:46<14:32,  1.14it/s]
 67%|██████▋   | 1981/2975 [32:47<14:29,  1.14it/s]
 67%|██████▋   | 1982/2975 [32:47<14:26,  1.15it/s]
 67%|██████▋   | 1983/2975 [32:48<14:21,  1.15it/s]
 67%|██████▋   | 1984/2975 [32:49<14:18,  1.15it/s]
 67%|██████▋   | 1985/2975 [32:50<14:16,  1.16it/s]
 67%|██████▋   | 1986/2975 [32:51<14:16,  1.16it/s]
 67%|██████▋   | 1987/2975 [32:52<14:14,  1.16it/s]
 67%|██████▋   | 1988/2975 [32:53<14:13,  1.16it/s]
 67%|█████
0: {'loss': 0.5574, 'grad_norm': 0.422261551019121, 'learning_rate': 7e-06, 'epoch': 0.67}
0: {'loss': 0.5578, 'grad_norm': 0.4117732299655185, 'learning_rate': 7e-06, 'epoch': 0.67}
0: █▋   | 1989/2975 [32:53<14:12,  1.16it/s]
 67%|██████▋   | 1990/2975 [32:54<14:11,  1.16it/s]
                                                   

 67%|██████▋   | 1990/2975 [32:54<14:11,  1.16it/s]
 67%|██████▋   | 1991/2975 [32:55<14:13,  1.15it/s]
 67%|██████▋   | 1992/2975 [32:56<14:11,  1.15it/s]
 67%|██████▋   | 1993/2975 [32:57<14:10,  1.15it/s]
 67%|██████▋   | 1994/2975 [32:58<14:08,  1.16it/s]
 67%|██████▋   | 1995/2975 [32:59<14:10,  1.15it/s]
 67%|██████▋   | 1996/2975 [33:00<14:08,  1.15it/s]
 67%|██████▋   | 1997/2975 [33:00<14:05,  1.16it/s]
 67%|██████▋   | 1998/2975 [33:01<14:26,  1.13it/s]
 67%|██████▋   | 1999/2975 [33:02<14:45,  1.10it/s]
 67%|██████▋   | 2000/2975 [33:03<14:32,  1.12it/s]
                                                   

 67%|██████▋   | 2000/2975 [33:03<14:32,  1.12it/s]
 67%|███
0: {'loss': 0.5695, 'grad_norm': 0.4311473069032605, 'learning_rate': 7e-06, 'epoch': 0.68}
0: ███▋   | 2001/2975 [33:04<14:23,  1.13it/s]
 67%|██████▋   | 2002/2975 [33:05<15:17,  1.06it/s]
 67%|██████▋   | 2003/2975 [33:06<15:59,  1.01it/s]
 67%|██████▋   | 2004/2975 [33:07<15:22,  1.05it/s]
 67%|██████▋   | 2005/2975 [33:08<14:56,  1.08it/s]
 67%|██████▋   | 2006/2975 [33:09<14:38,  1.10it/s]
 67%|██████▋   | 2007/2975 [33:10<14:24,  1.12it/s]
 67%|██████▋   | 2008/2975 [33:10<14:14,  1.13it/s]
 68%|██████▊   | 2009/2975 [33:12<15:14,  1.06it/s]
 68%|██████▊   | 2010/2975 [33:12<14:50,  1.08it/s]
                                                   

 68%|██████▊   | 2010/2975 [33:12<14:50,  1.08it/s]
 68%|██████▊   | 2011/2975 [33:13<14:32,  1.11it/s]
 68%|██████▊   | 2012/2975 [33:14<14:20,  1.12it/s]
 68%|██████▊   | 2013/2975 [33:15<14:12,  1.13it/s]
 68%|██████▊   | 2014/2975 [33:16<14:03,  1.14i
0: {'loss': 0.5576, 'grad_norm': 0.39207659768418185, 'learning_rate': 7e-06, 'epoch': 0.68}
0: t/s]
 68%|██████▊   | 2015/2975 [33:17<13:59,  1.14it/s]
 68%|██████▊   | 2016/2975 [33:18<13:54,  1.15it/s]
 68%|██████▊   | 2017/2975 [33:19<13:51,  1.15it/s]
 68%|██████▊   | 2018/2975 [33:19<13:50,  1.15it/s]
 68%|██████▊   | 2019/2975 [33:20<13:47,  1.15it/s]
 68%|██████▊   | 2020/2975 [33:21<13:47,  1.15it/s]
                                                   

 68%|██████▊   | 2020/2975 [33:21<13:47,  1.15it/s]
 68%|██████▊   | 2021/2975 [33:22<13:45,  1.16it/s]
 68%|██████▊   | 2022/2975 [33:23<13:44,  1.16it/s]
 68%|██████▊   | 2023/2975 [33:24<13:43,  1.16it/s]
 68%|██████▊   | 2024/2975 [33:25<13:40,  1.16it/s]
 68%|██████▊   | 2025/2975 [33:25<13:41,  1.16it/s]
 68%|██████▊   | 2026/2975 [33:26<13:39,  1.16it/s]
 68%|██████▊   | 2027/2975 [33:27<13:38,  1.16it/s]
 68%|██████▊   | 2028/2975 [
0: {'loss': 0.5572, 'grad_norm': 0.41223081193889455, 'learning_rate': 7e-06, 'epoch': 0.68}
0: {'loss': 0.5701, 'grad_norm': 0.40135196907178605, 'learning_rate': 7e-06, 'epoch': 0.69}
0: 33:28<13:38,  1.16it/s]
 68%|██████▊   | 2029/2975 [33:29<13:53,  1.14it/s]
 68%|██████▊   | 2030/2975 [33:30<13:47,  1.14it/s]
                                                   

 68%|██████▊   | 2030/2975 [33:30<13:47,  1.14it/s]
 68%|██████▊   | 2031/2975 [33:31<13:43,  1.15it/s]
 68%|██████▊   | 2032/2975 [33:32<13:40,  1.15it/s]
 68%|██████▊   | 2033/2975 [33:32<13:37,  1.15it/s]
 68%|██████▊   | 2034/2975 [33:33<13:36,  1.15it/s]
 68%|██████▊   | 2035/2975 [33:34<13:33,  1.16it/s]
 68%|██████▊   | 2036/2975 [33:35<13:32,  1.16it/s]
 68%|██████▊   | 2037/2975 [33:36<13:32,  1.15it/s]
 69%|██████▊   | 2038/2975 [33:37<13:29,  1.16it/s]
 69%|██████▊   | 2039/2975 [33:38<13:28,  1.16it/s]
 69%|██████▊   | 2040/2975 [33:38<13:27,  1.16it/s]
                                                   

 69%|██████▊   | 2040/
0: {'loss': 0.5608, 'grad_norm': 0.4028262533658366, 'learning_rate': 7e-06, 'epoch': 0.69}
0: 2975 [33:38<13:27,  1.16it/s]
 69%|██████▊   | 2041/2975 [33:39<13:27,  1.16it/s]
 69%|██████▊   | 2042/2975 [33:40<13:26,  1.16it/s]
 69%|██████▊   | 2043/2975 [33:41<13:25,  1.16it/s]
 69%|██████▊   | 2044/2975 [33:42<13:24,  1.16it/s]
 69%|██████▊   | 2045/2975 [33:43<13:24,  1.16it/s]
 69%|██████▉   | 2046/2975 [33:44<13:22,  1.16it/s]
 69%|██████▉   | 2047/2975 [33:44<13:22,  1.16it/s]
 69%|██████▉   | 2048/2975 [33:45<13:22,  1.16it/s]
 69%|██████▉   | 2049/2975 [33:46<13:21,  1.16it/s]
 69%|██████▉   | 2050/2975 [33:47<13:21,  1.15it/s]
                                                   

 69%|██████▉   | 2050/2975 [33:47<13:21,  1.15it/s]
 69%|██████▉   | 2051/2975 [33:48<13:19,  1.16it/s]
 69%|██████▉   | 2052/2975 [33:49<13:17,  1.16it/s]
 69%|██████▉   | 2053/2975 [33:50<13:17,  1.16it/s]
 69%|████
0: {'loss': 0.5643, 'grad_norm': 0.4167087098400813, 'learning_rate': 7e-06, 'epoch': 0.69}
0: ██▉   | 2054/2975 [33:51<13:15,  1.16it/s]
 69%|██████▉   | 2055/2975 [33:51<13:15,  1.16it/s]
 69%|██████▉   | 2056/2975 [33:52<13:14,  1.16it/s]
 69%|██████▉   | 2057/2975 [33:53<13:13,  1.16it/s]
 69%|██████▉   | 2058/2975 [33:54<13:13,  1.16it/s]
 69%|██████▉   | 2059/2975 [33:55<13:11,  1.16it/s]
 69%|██████▉   | 2060/2975 [33:56<13:10,  1.16it/s]
                                                   

 69%|██████▉   | 2060/2975 [33:56<13:10,  1.16it/s]
 69%|██████▉   | 2061/2975 [33:57<13:10,  1.16it/s]
 69%|██████▉   | 2062/2975 [33:57<13:10,  1.16it/s]
 69%|██████▉   | 2063/2975 [33:58<13:09,  1.15it/s]
 69%|██████▉   | 2064/2975 [33:59<13:07,  1.16it/s]
 69%|██████▉   | 2065/2975 [34:00<13:07,  1.16it/s]
 69%|██████▉   | 2066/2975 [34:01<13:07,  1.15it/s]
 69%|██████▉   | 2067/2975 [34:02<13:05,  1.16it/s
0: {'loss': 0.5434, 'grad_norm': 0.40135625895571303, 'learning_rate': 7e-06, 'epoch': 0.7}
0: ]
 70%|██████▉   | 2068/2975 [34:03<13:04,  1.16it/s]
 70%|██████▉   | 2069/2975 [34:04<13:03,  1.16it/s]
 70%|██████▉   | 2070/2975 [34:04<13:02,  1.16it/s]
                                                   

 70%|██████▉   | 2070/2975 [34:04<13:02,  1.16it/s]
 70%|██████▉   | 2071/2975 [34:05<13:02,  1.16it/s]
 70%|██████▉   | 2072/2975 [34:06<13:01,  1.16it/s]
 70%|██████▉   | 2073/2975 [34:07<13:01,  1.15it/s]
 70%|██████▉   | 2074/2975 [34:08<12:59,  1.16it/s]
 70%|██████▉   | 2075/2975 [34:09<13:16,  1.13it/s]
 70%|██████▉   | 2076/2975 [34:10<13:10,  1.14it/s]
 70%|██████▉   | 2077/2975 [34:11<13:05,  1.14it/s]
 70%|██████▉   | 2078/2975 [34:11<13:02,  1.15it/s]
 70%|██████▉   | 2079/2975 [34:12<12:58,  1.15it/s]
 70%|██████▉   | 2080/2975 [34:13<12:57,  1.15it/s]
                                             
0: {'loss': 0.561, 'grad_norm': 0.3999386144594851, 'learning_rate': 7e-06, 'epoch': 0.7}
0: {'loss': 0.5669, 'grad_norm': 0.39457082506419944, 'learning_rate': 7e-06, 'epoch': 0.7}
0:       

 70%|██████▉   | 2080/2975 [34:13<12:57,  1.15it/s]
 70%|██████▉   | 2081/2975 [34:14<12:56,  1.15it/s]
 70%|██████▉   | 2082/2975 [34:15<12:53,  1.15it/s]
 70%|███████   | 2083/2975 [34:16<12:52,  1.15it/s]
 70%|███████   | 2084/2975 [34:17<12:51,  1.15it/s]
 70%|███████   | 2085/2975 [34:18<13:15,  1.12it/s]
 70%|███████   | 2086/2975 [34:18<13:08,  1.13it/s]
 70%|███████   | 2087/2975 [34:19<13:01,  1.14it/s]
 70%|███████   | 2088/2975 [34:20<12:56,  1.14it/s]
 70%|███████   | 2089/2975 [34:21<12:52,  1.15it/s]
 70%|███████   | 2090/2975 [34:22<12:48,  1.15it/s]
                                                   

 70%|███████   | 2090/2975 [34:22<12:48,  1.15it/s]
 70%|███████   | 2091/2975 [34:23<12:50,  1.15it/s]
 70%|███████   | 2092/2975 [34:24<12:47,  1.15it/s]
 70%|███████   | 2093/297
0: {'loss': 0.5581, 'grad_norm': 0.41513431108818655, 'learning_rate': 7e-06, 'epoch': 0.71}
0: 5 [34:24<12:44,  1.15it/s]
 70%|███████   | 2094/2975 [34:25<12:44,  1.15it/s]
 70%|███████   | 2095/2975 [34:26<12:42,  1.15it/s]
 70%|███████   | 2096/2975 [34:27<12:41,  1.15it/s]
 70%|███████   | 2097/2975 [34:28<12:40,  1.15it/s]
 71%|███████   | 2098/2975 [34:29<12:37,  1.16it/s]
 71%|███████   | 2099/2975 [34:30<12:38,  1.16it/s]
 71%|███████   | 2100/2975 [34:31<12:54,  1.13it/s]
                                                   

 71%|███████   | 2100/2975 [34:31<12:54,  1.13it/s]
 71%|███████   | 2101/2975 [34:31<12:58,  1.12it/s]
 71%|███████   | 2102/2975 [34:32<12:51,  1.13it/s]
 71%|███████   | 2103/2975 [34:33<12:45,  1.14it/s]
 71%|███████   | 2104/2975 [34:34<12:41,  1.14it/s]
 71%|███████   | 2105/2975 [34:35<12:38,  1.15it/s]
 71%|███████   | 2106/2975 [34:36<12:42,  1.14it/s]
 71%|█████
0: {'loss': 0.5583, 'grad_norm': 0.4283157706662965, 'learning_rate': 7e-06, 'epoch': 0.71}
0: ██   | 2107/2975 [34:37<12:37,  1.15it/s]
 71%|███████   | 2108/2975 [34:38<12:34,  1.15it/s]
 71%|███████   | 2109/2975 [34:38<12:32,  1.15it/s]
 71%|███████   | 2110/2975 [34:39<12:49,  1.12it/s]
                                                   

 71%|███████   | 2110/2975 [34:39<12:49,  1.12it/s]
 71%|███████   | 2111/2975 [34:40<12:43,  1.13it/s]
 71%|███████   | 2112/2975 [34:41<13:31,  1.06it/s]
 71%|███████   | 2113/2975 [34:42<13:10,  1.09it/s]
 71%|███████   | 2114/2975 [34:43<12:54,  1.11it/s]
 71%|███████   | 2115/2975 [34:44<12:44,  1.13it/s]
 71%|███████   | 2116/2975 [34:45<12:35,  1.14it/s]
 71%|███████   | 2117/2975 [34:46<12:30,  1.14it/s]
 71%|███████   | 2118/2975 [34:47<12:37,  1.13it/s]
 71%|███████   | 2119/2975 [34:47<12:31,  1.14it/s]
 71%|███████▏  | 2120/2975 [34:48<12:25,  1.15it/s]
0: {'loss': 0.5459, 'grad_norm': 0.39670303604706025, 'learning_rate': 7e-06, 'epoch': 0.71}
0: {'loss': 0.55, 'grad_norm': 0.39636045931160196, 'learning_rate': 7e-06, 'epoch': 0.72}
0: 
                                                   

 71%|███████▏  | 2120/2975 [34:48<12:25,  1.15it/s]
 71%|███████▏  | 2121/2975 [34:49<12:22,  1.15it/s]
 71%|███████▏  | 2122/2975 [34:50<12:19,  1.15it/s]
 71%|███████▏  | 2123/2975 [34:51<12:17,  1.15it/s]
 71%|███████▏  | 2124/2975 [34:52<12:16,  1.16it/s]
 71%|███████▏  | 2125/2975 [34:53<12:14,  1.16it/s]
 71%|███████▏  | 2126/2975 [34:53<12:13,  1.16it/s]
 71%|███████▏  | 2127/2975 [34:54<12:13,  1.16it/s]
 72%|███████▏  | 2128/2975 [34:55<12:11,  1.16it/s]
 72%|███████▏  | 2129/2975 [34:56<12:11,  1.16it/s]
 72%|███████▏  | 2130/2975 [34:57<12:10,  1.16it/s]
                                                   

 72%|███████▏  | 2130/2975 [34:57<12:10,  1.16it/s]
 72%|███████▏  | 2131/2975 [34:58<12:09,  1.16it/s]
 72%|███████▏  | 
0: {'loss': 0.5404, 'grad_norm': 0.3968292409228454, 'learning_rate': 7e-06, 'epoch': 0.72}
0: 2132/2975 [34:59<12:08,  1.16it/s]
 72%|███████▏  | 2133/2975 [34:59<12:07,  1.16it/s]
 72%|███████▏  | 2134/2975 [35:00<12:06,  1.16it/s]
 72%|███████▏  | 2135/2975 [35:01<12:07,  1.15it/s]
 72%|███████▏  | 2136/2975 [35:02<13:06,  1.07it/s]
 72%|███████▏  | 2137/2975 [35:03<12:56,  1.08it/s]
 72%|███████▏  | 2138/2975 [35:04<12:38,  1.10it/s]
 72%|███████▏  | 2139/2975 [35:05<12:27,  1.12it/s]
 72%|███████▏  | 2140/2975 [35:06<12:18,  1.13it/s]
                                                   

 72%|███████▏  | 2140/2975 [35:06<12:18,  1.13it/s]
 72%|███████▏  | 2141/2975 [35:07<12:57,  1.07it/s]
 72%|███████▏  | 2142/2975 [35:08<12:38,  1.10it/s]
 72%|███████▏  | 2143/2975 [35:09<12:25,  1.12it/s]
 72%|███████▏  | 2144/2975 [35:10<12:51,  1.08it/s]
 72%|███████▏  | 2145/2975 [35:10<12
0: {'loss': 0.5447, 'grad_norm': 0.38452506333261516, 'learning_rate': 7e-06, 'epoch': 0.72}
0: :34,  1.10it/s]
 72%|███████▏  | 2146/2975 [35:12<13:19,  1.04it/s]
 72%|███████▏  | 2147/2975 [35:12<12:54,  1.07it/s]
 72%|███████▏  | 2148/2975 [35:13<12:37,  1.09it/s]
 72%|███████▏  | 2149/2975 [35:14<12:25,  1.11it/s]
 72%|███████▏  | 2150/2975 [35:15<12:14,  1.12it/s]
                                                   

 72%|███████▏  | 2150/2975 [35:15<12:14,  1.12it/s]
 72%|███████▏  | 2151/2975 [35:16<12:09,  1.13it/s]
 72%|███████▏  | 2152/2975 [35:17<12:03,  1.14it/s]
 72%|███████▏  | 2153/2975 [35:18<11:57,  1.15it/s]
 72%|███████▏  | 2154/2975 [35:18<11:54,  1.15it/s]
 72%|███████▏  | 2155/2975 [35:19<11:51,  1.15it/s]
 72%|███████▏  | 2156/2975 [35:20<11:49,  1.15it/s]
 73%|███████▎  | 2157/2975 [35:21<11:48,  1.15it/s]
 73%|███████▎  | 2158/2975 [35:22<11:47,  1.15it/s]
 73
0: {'loss': 0.545, 'grad_norm': 0.3978905995048389, 'learning_rate': 7e-06, 'epoch': 0.73}
0: {'loss': 0.569, 'grad_norm': 0.40812910264341107, 'learning_rate': 7e-06, 'epoch': 0.73}
0: %|███████▎  | 2159/2975 [35:23<11:47,  1.15it/s]
 73%|███████▎  | 2160/2975 [35:24<11:46,  1.15it/s]
                                                   

 73%|███████▎  | 2160/2975 [35:24<11:46,  1.15it/s]
 73%|███████▎  | 2161/2975 [35:25<11:44,  1.16it/s]
 73%|███████▎  | 2162/2975 [35:25<11:43,  1.16it/s]
 73%|███████▎  | 2163/2975 [35:26<11:40,  1.16it/s]
 73%|███████▎  | 2164/2975 [35:27<11:40,  1.16it/s]
 73%|███████▎  | 2165/2975 [35:28<12:09,  1.11it/s]
 73%|███████▎  | 2166/2975 [35:29<11:58,  1.13it/s]
 73%|███████▎  | 2167/2975 [35:30<11:51,  1.14it/s]
 73%|███████▎  | 2168/2975 [35:31<11:46,  1.14it/s]
 73%|███████▎  | 2169/2975 [35:32<11:41,  1.15it/s]
 73%|███████▎  | 2170/2975 [35:32<11:38,  1.15it/s]
                                                   

 73%|███████▎  | 2170
0: {'loss': 0.5491, 'grad_norm': 0.4102726571878277, 'learning_rate': 7e-06, 'epoch': 0.73}
0: /2975 [35:32<11:38,  1.15it/s]
 73%|███████▎  | 2171/2975 [35:33<11:36,  1.15it/s]
 73%|███████▎  | 2172/2975 [35:34<11:34,  1.16it/s]
 73%|███████▎  | 2173/2975 [35:35<11:32,  1.16it/s]
 73%|███████▎  | 2174/2975 [35:36<11:31,  1.16it/s]
 73%|███████▎  | 2175/2975 [35:37<11:30,  1.16it/s]
 73%|███████▎  | 2176/2975 [35:38<11:28,  1.16it/s]
 73%|███████▎  | 2177/2975 [35:38<11:31,  1.15it/s]
 73%|███████▎  | 2178/2975 [35:39<11:30,  1.15it/s]
 73%|███████▎  | 2179/2975 [35:40<11:28,  1.16it/s]
 73%|███████▎  | 2180/2975 [35:41<11:27,  1.16it/s]
                                                   

 73%|███████▎  | 2180/2975 [35:41<11:27,  1.16it/s]
 73%|███████▎  | 2181/2975 [35:42<11:27,  1.16it/s]
 73%|███████▎  | 2182/2975 [35:43<11:26,  1.16it/s]
 73%|███████▎  | 2183/2975 [35:44<11:25,
0: {'loss': 0.546, 'grad_norm': 0.40725964941871345, 'learning_rate': 7e-06, 'epoch': 0.74}
0:   1.16it/s]
 73%|███████▎  | 2184/2975 [35:45<12:13,  1.08it/s]
 73%|███████▎  | 2185/2975 [35:46<11:56,  1.10it/s]
 73%|███████▎  | 2186/2975 [35:46<11:45,  1.12it/s]
 74%|███████▎  | 2187/2975 [35:47<11:42,  1.12it/s]
 74%|███████▎  | 2188/2975 [35:48<11:34,  1.13it/s]
 74%|███████▎  | 2189/2975 [35:49<11:27,  1.14it/s]
 74%|███████▎  | 2190/2975 [35:50<11:28,  1.14it/s]
                                                   

 74%|███████▎  | 2190/2975 [35:50<11:28,  1.14it/s]
 74%|███████▎  | 2191/2975 [35:51<11:24,  1.15it/s]
 74%|███████▎  | 2192/2975 [35:52<11:20,  1.15it/s]
 74%|███████▎  | 2193/2975 [35:53<11:41,  1.11it/s]
 74%|███████▎  | 2194/2975 [35:53<11:32,  1.13it/s]
 74%|███████▍  | 2195/2975 [35:54<11:26,  1.14it/s]
 74%|███████▍  | 2196/2975 [35:55<11:21,  1.14it/s]
 74%|�
0: {'loss': 0.564, 'grad_norm': 0.4048584363663052, 'learning_rate': 7e-06, 'epoch': 0.74}
0: �██████▍  | 2197/2975 [35:56<11:17,  1.15it/s]
 74%|███████▍  | 2198/2975 [35:57<11:15,  1.15it/s]
 74%|███████▍  | 2199/2975 [35:58<11:12,  1.15it/s]
 74%|███████▍  | 2200/2975 [35:59<11:11,  1.15it/s]
                                                   

 74%|███████▍  | 2200/2975 [35:59<11:11,  1.15it/s]
 74%|███████▍  | 2201/2975 [35:59<11:10,  1.15it/s]
 74%|███████▍  | 2202/2975 [36:00<11:08,  1.16it/s]
 74%|███████▍  | 2203/2975 [36:01<11:07,  1.16it/s]
 74%|███████▍  | 2204/2975 [36:02<11:05,  1.16it/s]
 74%|███████▍  | 2205/2975 [36:03<11:04,  1.16it/s]
 74%|███████▍  | 2206/2975 [36:04<11:03,  1.16it/s]
 74%|███████▍  | 2207/2975 [36:05<11:02,  1.16it/s]
 74%|███████▍  | 2208/2975 [36:06<11:02,  1.16it/s]
 74%|███████▍  | 2209/2975 [36:06<11:01,  1.16it/s]
 74%|███████
0: {'loss': 0.5565, 'grad_norm': 0.38284311651830266, 'learning_rate': 7e-06, 'epoch': 0.74}
0: {'loss': 0.5664, 'grad_norm': 0.4124324148545162, 'learning_rate': 7e-06, 'epoch': 0.75}
0: ▍  | 2210/2975 [36:07<10:59,  1.16it/s]
                                                   

 74%|███████▍  | 2210/2975 [36:07<10:59,  1.16it/s]
 74%|███████▍  | 2211/2975 [36:08<10:59,  1.16it/s]
 74%|███████▍  | 2212/2975 [36:09<10:58,  1.16it/s]
 74%|███████▍  | 2213/2975 [36:10<10:58,  1.16it/s]
 74%|███████▍  | 2214/2975 [36:11<10:56,  1.16it/s]
 74%|███████▍  | 2215/2975 [36:12<10:56,  1.16it/s]
 74%|███████▍  | 2216/2975 [36:12<10:55,  1.16it/s]
 75%|███████▍  | 2217/2975 [36:13<10:54,  1.16it/s]
 75%|███████▍  | 2218/2975 [36:14<11:12,  1.13it/s]
 75%|███████▍  | 2219/2975 [36:15<11:11,  1.13it/s]
 75%|███████▍  | 2220/2975 [36:16<11:04,  1.14it/s]
                                                   

 75%|███████▍  | 2220/2975 [36:16<11:04,  1.14it/s]
 75%|███████▍  | 2221/2975 [36:17<11:56,  1.
0: {'loss': 0.5471, 'grad_norm': 0.4066181400242007, 'learning_rate': 7e-06, 'epoch': 0.75}
0: 05it/s]
 75%|███████▍  | 2222/2975 [36:18<11:36,  1.08it/s]
 75%|███████▍  | 2223/2975 [36:19<11:21,  1.10it/s]
 75%|███████▍  | 2224/2975 [36:20<11:10,  1.12it/s]
 75%|███████▍  | 2225/2975 [36:21<11:03,  1.13it/s]
 75%|███████▍  | 2226/2975 [36:21<10:58,  1.14it/s]
 75%|███████▍  | 2227/2975 [36:22<10:53,  1.14it/s]
 75%|███████▍  | 2228/2975 [36:23<10:50,  1.15it/s]
 75%|███████▍  | 2229/2975 [36:24<10:47,  1.15it/s]
 75%|███████▍  | 2230/2975 [36:25<10:45,  1.15it/s]
                                                   

 75%|███████▍  | 2230/2975 [36:25<10:45,  1.15it/s]
 75%|███████▍  | 2231/2975 [36:26<10:59,  1.13it/s]
 75%|███████▌  | 2232/2975 [36:27<10:53,  1.14it/s]
 75%|███████▌  | 2233/2975 [36:28<10:48,  1.14it/s]
 75%|███████▌  | 2234/2975 [36:28<10:45,  1.15it/s]
 75%|██
0: {'loss': 0.5553, 'grad_norm': 0.4022115239237684, 'learning_rate': 7e-06, 'epoch': 0.75}
0: █████▌  | 2235/2975 [36:29<10:42,  1.15it/s]
 75%|███████▌  | 2236/2975 [36:30<10:40,  1.15it/s]
 75%|███████▌  | 2237/2975 [36:31<10:42,  1.15it/s]
 75%|███████▌  | 2238/2975 [36:32<10:39,  1.15it/s]
 75%|███████▌  | 2239/2975 [36:33<10:39,  1.15it/s]
 75%|███████▌  | 2240/2975 [36:34<10:37,  1.15it/s]
                                                   

 75%|███████▌  | 2240/2975 [36:34<10:37,  1.15it/s]
 75%|███████▌  | 2241/2975 [36:34<10:36,  1.15it/s]
 75%|███████▌  | 2242/2975 [36:35<10:34,  1.15it/s]
 75%|███████▌  | 2243/2975 [36:36<10:33,  1.16it/s]
 75%|███████▌  | 2244/2975 [36:37<10:59,  1.11it/s]
 75%|███████▌  | 2245/2975 [36:38<10:50,  1.12it/s]
 75%|███████▌  | 2246/2975 [36:39<10:43,  1.13it/s]
 76%|███████▌  | 2247/2975 [36:40<10:38,  1.14it/s]
 76%|███████▌ 
0: {'loss': 0.5453, 'grad_norm': 0.3917071359906677, 'learning_rate': 7e-06, 'epoch': 0.76}
0:  | 2248/2975 [36:41<10:33,  1.15it/s]
 76%|███████▌  | 2249/2975 [36:41<10:29,  1.15it/s]
 76%|███████▌  | 2250/2975 [36:42<10:27,  1.15it/s]
                                                   

 76%|███████▌  | 2250/2975 [36:42<10:27,  1.15it/s]
 76%|███████▌  | 2251/2975 [36:43<10:26,  1.16it/s]
 76%|███████▌  | 2252/2975 [36:44<10:25,  1.16it/s]
 76%|███████▌  | 2253/2975 [36:45<10:24,  1.16it/s]
 76%|███████▌  | 2254/2975 [36:46<10:22,  1.16it/s]
 76%|███████▌  | 2255/2975 [36:47<10:23,  1.15it/s]
 76%|███████▌  | 2256/2975 [36:48<10:21,  1.16it/s]
 76%|███████▌  | 2257/2975 [36:49<11:06,  1.08it/s]
 76%|███████▌  | 2258/2975 [36:49<10:51,  1.10it/s]
 76%|███████▌  | 2259/2975 [36:50<10:40,  1.12it/s]
 76%|███████▌  | 2260/2975 [36:51<10:33,  1.13it/s]
                                                 
0: {'loss': 0.5565, 'grad_norm': 0.39282553840219026, 'learning_rate': 7e-06, 'epoch': 0.76}
0: {'loss': 0.5449, 'grad_norm': 0.4009481273646408, 'learning_rate': 7e-06, 'epoch': 0.76}
0:   

 76%|███████▌  | 2260/2975 [36:51<10:33,  1.13it/s]
 76%|███████▌  | 2261/2975 [36:52<10:34,  1.12it/s]
 76%|███████▌  | 2262/2975 [36:53<10:27,  1.14it/s]
 76%|███████▌  | 2263/2975 [36:54<10:23,  1.14it/s]
 76%|███████▌  | 2264/2975 [36:55<10:19,  1.15it/s]
 76%|███████▌  | 2265/2975 [36:56<10:22,  1.14it/s]
 76%|███████▌  | 2266/2975 [36:56<10:18,  1.15it/s]
 76%|███████▌  | 2267/2975 [36:57<10:16,  1.15it/s]
 76%|███████▌  | 2268/2975 [36:58<10:14,  1.15it/s]
 76%|███████▋  | 2269/2975 [36:59<10:12,  1.15it/s]
 76%|███████▋  | 2270/2975 [37:00<10:13,  1.15it/s]
                                                   

 76%|███████▋  | 2270/2975 [37:00<10:13,  1.15it/s]
 76%|███████▋  | 2271/2975 [37:01<10:10,  1.15it/s]
 76%|███████▋  | 2272/2975 [37:02<10:09,  1.15it/s]
 76%|███�
0: {'loss': 0.5446, 'grad_norm': 0.3882813819591711, 'learning_rate': 7e-06, 'epoch': 0.77}
0: ��███▋  | 2273/2975 [37:03<10:06,  1.16it/s]
 76%|███████▋  | 2274/2975 [37:03<10:05,  1.16it/s]
 76%|███████▋  | 2275/2975 [37:04<10:03,  1.16it/s]
 77%|███████▋  | 2276/2975 [37:05<10:02,  1.16it/s]
 77%|███████▋  | 2277/2975 [37:06<10:03,  1.16it/s]
 77%|███████▋  | 2278/2975 [37:07<10:01,  1.16it/s]
 77%|███████▋  | 2279/2975 [37:08<10:00,  1.16it/s]
 77%|███████▋  | 2280/2975 [37:09<10:41,  1.08it/s]
                                                   

 77%|███████▋  | 2280/2975 [37:09<10:41,  1.08it/s]
 77%|███████▋  | 2281/2975 [37:10<10:27,  1.11it/s]
 77%|███████▋  | 2282/2975 [37:10<10:18,  1.12it/s]
 77%|███████▋  | 2283/2975 [37:11<10:09,  1.13it/s]
 77%|███████▋  | 2284/2975 [37:12<10:04,  1.14it/s]
 77%|███████▋  | 2285/2975 [37:13<10:00,  1.15it/s]
 77%|███████▋  | 2
0: {'loss': 0.56, 'grad_norm': 0.40612363078585, 'learning_rate': 7e-06, 'epoch': 0.77}
0: 286/2975 [37:14<09:57,  1.15it/s]
 77%|███████▋  | 2287/2975 [37:15<09:55,  1.16it/s]
 77%|███████▋  | 2288/2975 [37:16<09:53,  1.16it/s]
 77%|███████▋  | 2289/2975 [37:16<09:52,  1.16it/s]
 77%|███████▋  | 2290/2975 [37:17<09:49,  1.16it/s]
                                                   

 77%|███████▋  | 2290/2975 [37:17<09:49,  1.16it/s]
 77%|███████▋  | 2291/2975 [37:18<10:33,  1.08it/s]
 77%|███████▋  | 2292/2975 [37:19<10:18,  1.10it/s]
 77%|███████▋  | 2293/2975 [37:20<10:08,  1.12it/s]
 77%|███████▋  | 2294/2975 [37:21<10:01,  1.13it/s]
 77%|███████▋  | 2295/2975 [37:22<09:55,  1.14it/s]
 77%|███████▋  | 2296/2975 [37:23<09:52,  1.15it/s]
 77%|███████▋  | 2297/2975 [37:24<09:48,  1.15it/s]
 77%|███████▋  | 2298/2975 [37:24<09:46,  1.15it/s]
 77%|███████▋  | 2299/2975 [37:25<09:
0: {'loss': 0.5491, 'grad_norm': 0.3904189323041483, 'learning_rate': 7e-06, 'epoch': 0.77}
0: {'loss': 0.5416, 'grad_norm': 0.4019065318486215, 'learning_rate': 7e-06, 'epoch': 0.78}
0: 45,  1.15it/s]
 77%|███████▋  | 2300/2975 [37:26<09:43,  1.16it/s]
                                                   

 77%|███████▋  | 2300/2975 [37:26<09:43,  1.16it/s]
 77%|███████▋  | 2301/2975 [37:27<09:42,  1.16it/s]
 77%|███████▋  | 2302/2975 [37:28<09:39,  1.16it/s]
 77%|███████▋  | 2303/2975 [37:29<09:38,  1.16it/s]
 77%|███████▋  | 2304/2975 [37:30<09:37,  1.16it/s]
 77%|███████▋  | 2305/2975 [37:30<09:35,  1.16it/s]
 78%|███████▊  | 2306/2975 [37:31<09:35,  1.16it/s]
 78%|███████▊  | 2307/2975 [37:32<09:34,  1.16it/s]
 78%|███████▊  | 2308/2975 [37:33<09:34,  1.16it/s]
 78%|███████▊  | 2309/2975 [37:34<09:34,  1.16it/s]
 78%|███████▊  | 2310/2975 [37:35<09:33,  1.16it/s]
                                                   

 78%|███████▊  | 2310/2975 [37:35<09:33,  1.16it/s]
 78%|████�
0: {'loss': 0.5439, 'grad_norm': 0.4032830606745122, 'learning_rate': 7e-06, 'epoch': 0.78}
0: �██▊  | 2311/2975 [37:36<09:35,  1.15it/s]
 78%|███████▊  | 2312/2975 [37:37<09:32,  1.16it/s]
 78%|███████▊  | 2313/2975 [37:37<09:31,  1.16it/s]
 78%|███████▊  | 2314/2975 [37:38<09:30,  1.16it/s]
 78%|███████▊  | 2315/2975 [37:39<09:28,  1.16it/s]
 78%|███████▊  | 2316/2975 [37:40<09:28,  1.16it/s]
 78%|███████▊  | 2317/2975 [37:41<09:26,  1.16it/s]
 78%|███████▊  | 2318/2975 [37:42<09:26,  1.16it/s]
 78%|███████▊  | 2319/2975 [37:43<09:25,  1.16it/s]
 78%|███████▊  | 2320/2975 [37:43<09:25,  1.16it/s]
                                                   

 78%|███████▊  | 2320/2975 [37:43<09:25,  1.16it/s]
 78%|███████▊  | 2321/2975 [37:44<09:25,  1.16it/s]
 78%|███████▊  | 2322/2975 [37:45<09:24,  1.16it/s]
 78%|███████▊  | 2323/2975 [37:46<09:22,  1.16it/s]
 78%|███████▊  | 2324/
0: {'loss': 0.5419, 'grad_norm': 0.3951807894319931, 'learning_rate': 7e-06, 'epoch': 0.78}
0: 2975 [37:47<09:21,  1.16it/s]
 78%|███████▊  | 2325/2975 [37:48<09:19,  1.16it/s]
 78%|███████▊  | 2326/2975 [37:49<09:18,  1.16it/s]
 78%|███████▊  | 2327/2975 [37:49<09:17,  1.16it/s]
 78%|███████▊  | 2328/2975 [37:50<09:16,  1.16it/s]
 78%|███████▊  | 2329/2975 [37:51<09:15,  1.16it/s]
 78%|███████▊  | 2330/2975 [37:52<09:15,  1.16it/s]
                                                   

 78%|███████▊  | 2330/2975 [37:52<09:15,  1.16it/s]
 78%|███████▊  | 2331/2975 [37:53<09:15,  1.16it/s]
 78%|███████▊  | 2332/2975 [37:54<09:14,  1.16it/s]
 78%|███████▊  | 2333/2975 [37:55<09:13,  1.16it/s]
 78%|███████▊  | 2334/2975 [37:55<09:12,  1.16it/s]
 78%|███████▊  | 2335/2975 [37:56<09:11,  1.16it/s]
 79%|███████▊  | 2336/2975 [37:57<09:11,  1.16it/s]
 79%|███████▊  | 2337/2975 [37:58<09:10, 
0: {'loss': 0.5538, 'grad_norm': 0.41919508705440267, 'learning_rate': 7e-06, 'epoch': 0.79}
0:  1.16it/s]
 79%|███████▊  | 2338/2975 [37:59<09:10,  1.16it/s]
 79%|███████▊  | 2339/2975 [38:00<09:10,  1.16it/s]
 79%|███████▊  | 2340/2975 [38:01<09:08,  1.16it/s]
                                                   

 79%|███████▊  | 2340/2975 [38:01<09:08,  1.16it/s]
 79%|███████▊  | 2341/2975 [38:02<09:07,  1.16it/s]
 79%|███████▊  | 2342/2975 [38:02<09:06,  1.16it/s]
 79%|███████▉  | 2343/2975 [38:03<09:15,  1.14it/s]
 79%|███████▉  | 2344/2975 [38:04<09:12,  1.14it/s]
 79%|███████▉  | 2345/2975 [38:05<09:08,  1.15it/s]
 79%|███████▉  | 2346/2975 [38:06<09:06,  1.15it/s]
 79%|███████▉  | 2347/2975 [38:07<09:04,  1.15it/s]
 79%|███████▉  | 2348/2975 [38:08<09:03,  1.15it/s]
 79%|███████▉  | 2349/2975 [38:08<09:02,  1.15it/s]
 79%|███████▉  | 2350/2975 [38:09<09:00,  1.16it/s]
        
0: {'loss': 0.5625, 'grad_norm': 0.4103610862555039, 'learning_rate': 7e-06, 'epoch': 0.79}
0: {'loss': 0.5448, 'grad_norm': 0.4028470420053187, 'learning_rate': 7e-06, 'epoch': 0.79}
0:                                            

 79%|███████▉  | 2350/2975 [38:09<09:00,  1.16it/s]
 79%|███████▉  | 2351/2975 [38:10<09:00,  1.15it/s]
 79%|███████▉  | 2352/2975 [38:11<08:58,  1.16it/s]
 79%|███████▉  | 2353/2975 [38:12<08:57,  1.16it/s]
 79%|███████▉  | 2354/2975 [38:13<08:56,  1.16it/s]
 79%|███████▉  | 2355/2975 [38:14<08:56,  1.16it/s]
 79%|███████▉  | 2356/2975 [38:15<08:55,  1.16it/s]
 79%|███████▉  | 2357/2975 [38:15<08:55,  1.16it/s]
 79%|███████▉  | 2358/2975 [38:16<08:53,  1.16it/s]
 79%|███████▉  | 2359/2975 [38:17<08:52,  1.16it/s]
 79%|███████▉  | 2360/2975 [38:18<08:51,  1.16it/s]
                                                   

 79%|███████▉  | 2360/2975 [38:18<08:51,  1.16it/s]
 79%|███████▉  | 2361/2975 [38:19<08:54,  1.15it/s]
 79%|███████▉  | 2362/2975
0: {'loss': 0.5429, 'grad_norm': 0.371874747303393, 'learning_rate': 7e-06, 'epoch': 0.8}
0:  [38:20<08:52,  1.15it/s]
 79%|███████▉  | 2363/2975 [38:21<08:53,  1.15it/s]
 79%|███████▉  | 2364/2975 [38:21<08:51,  1.15it/s]
 79%|███████▉  | 2365/2975 [38:22<08:50,  1.15it/s]
 80%|███████▉  | 2366/2975 [38:23<08:48,  1.15it/s]
 80%|███████▉  | 2367/2975 [38:24<08:48,  1.15it/s]
 80%|███████▉  | 2368/2975 [38:25<08:46,  1.15it/s]
 80%|███████▉  | 2369/2975 [38:26<08:44,  1.15it/s]
 80%|███████▉  | 2370/2975 [38:27<08:43,  1.16it/s]
                                                   

 80%|███████▉  | 2370/2975 [38:27<08:43,  1.16it/s]
 80%|███████▉  | 2371/2975 [38:28<08:43,  1.15it/s]
 80%|███████▉  | 2372/2975 [38:28<08:42,  1.16it/s]
 80%|███████▉  | 2373/2975 [38:29<08:41,  1.15it/s]
 80%|███████▉  | 2374/2975 [38:30<08:40,  1.16it/s]
 80%|███████▉  | 2375/2975 [38:31<08:38,  1.1
0: {'loss': 0.5594, 'grad_norm': 0.4004765783510905, 'learning_rate': 7e-06, 'epoch': 0.8}
0: 6it/s]
 80%|███████▉  | 2376/2975 [38:32<08:37,  1.16it/s]
 80%|███████▉  | 2377/2975 [38:33<08:35,  1.16it/s]
 80%|███████▉  | 2378/2975 [38:34<08:34,  1.16it/s]
 80%|███████▉  | 2379/2975 [38:34<08:38,  1.15it/s]
 80%|████████  | 2380/2975 [38:35<08:35,  1.15it/s]
                                                   

 80%|████████  | 2380/2975 [38:35<08:35,  1.15it/s]
 80%|████████  | 2381/2975 [38:41<21:36,  2.18s/it]
 80%|████████  | 2382/2975 [38:41<17:39,  1.79s/it]
 80%|████████  | 2383/2975 [38:42<14:54,  1.51s/it]
 80%|████████  | 2384/2975 [38:43<12:58,  1.32s/it]
 80%|████████  | 2385/2975 [38:44<11:36,  1.18s/it]
 80%|████████  | 2386/2975 [38:45<10:39,  1.08s/it]
 80%|████████  | 2387/2975 [38:46<09:58,  1.02s/it]
 80%|████████  | 2388/2975 [38:47<09:34,  1.02it/s]
 80%|██�
0: {'loss': 0.5646, 'grad_norm': 0.3945204345917697, 'learning_rate': 7e-06, 'epoch': 0.8}
0: {'loss': 0.5399, 'grad_norm': 0.42349086771020106, 'learning_rate': 7e-06, 'epoch': 0.81}
0: ��█████  | 2389/2975 [38:48<09:13,  1.06it/s]
 80%|████████  | 2390/2975 [38:48<08:57,  1.09it/s]
                                                   

 80%|████████  | 2390/2975 [38:48<08:57,  1.09it/s]
 80%|████████  | 2391/2975 [38:49<08:46,  1.11it/s]
 80%|████████  | 2392/2975 [38:50<08:39,  1.12it/s]
 80%|████████  | 2393/2975 [38:51<08:33,  1.13it/s]
 80%|████████  | 2394/2975 [38:52<08:29,  1.14it/s]
 81%|████████  | 2395/2975 [38:53<08:26,  1.15it/s]
 81%|████████  | 2396/2975 [38:54<08:23,  1.15it/s]
 81%|████████  | 2397/2975 [38:54<08:21,  1.15it/s]
 81%|████████  | 2398/2975 [38:55<08:23,  1.15it/s]
 81%|████████  | 2399/2975 [38:56<08:20,  1.15it/s]
 81%|████████  | 2400/2975 [38:57<08:18,  1.15it/s]
                                                   

 81%|████████  | 2400/2975 [38
0: {'loss': 0.5552, 'grad_norm': 0.4144051979042879, 'learning_rate': 7e-06, 'epoch': 0.81}
0: :57<08:18,  1.15it/s]
 81%|████████  | 2401/2975 [38:58<08:17,  1.15it/s]
 81%|████████  | 2402/2975 [38:59<08:16,  1.15it/s]
 81%|████████  | 2403/2975 [39:00<08:14,  1.16it/s]
 81%|████████  | 2404/2975 [39:01<08:36,  1.11it/s]
 81%|████████  | 2405/2975 [39:01<08:28,  1.12it/s]
 81%|████████  | 2406/2975 [39:02<08:26,  1.12it/s]
 81%|████████  | 2407/2975 [39:03<08:21,  1.13it/s]
 81%|████████  | 2408/2975 [39:04<08:16,  1.14it/s]
 81%|████████  | 2409/2975 [39:05<08:26,  1.12it/s]
 81%|████████  | 2410/2975 [39:06<08:19,  1.13it/s]
                                                   

 81%|████████  | 2410/2975 [39:06<08:19,  1.13it/s]
 81%|████████  | 2411/2975 [39:07<08:15,  1.14it/s]
 81%|████████  | 2412/2975 [39:08<08:11,  1.15it/s]
 81%|████████  | 2413/2975 [39:08<08:08,  1.15it/
0: {'loss': 0.5478, 'grad_norm': 0.41491974581236013, 'learning_rate': 7e-06, 'epoch': 0.81}
0: s]
 81%|████████  | 2414/2975 [39:09<08:05,  1.16it/s]
 81%|████████  | 2415/2975 [39:10<08:03,  1.16it/s]
 81%|████████  | 2416/2975 [39:11<08:04,  1.15it/s]
 81%|████████  | 2417/2975 [39:12<08:02,  1.16it/s]
 81%|████████▏ | 2418/2975 [39:13<08:00,  1.16it/s]
 81%|████████▏ | 2419/2975 [39:14<07:59,  1.16it/s]
 81%|████████▏ | 2420/2975 [39:15<07:58,  1.16it/s]
                                                   

 81%|████████▏ | 2420/2975 [39:15<07:58,  1.16it/s]
 81%|████████▏ | 2421/2975 [39:15<07:57,  1.16it/s]
 81%|████████▏ | 2422/2975 [39:16<07:56,  1.16it/s]
 81%|████████▏ | 2423/2975 [39:17<07:55,  1.16it/s]
 81%|████████▏ | 2424/2975 [39:18<07:53,  1.16it/s]
 82%|████████▏ | 2425/2975 [39:19<07:52,  1.16it/s]
 82%|████████▏ | 2426/2975 [39:20<07:51,  1.16it
0: {'loss': 0.5442, 'grad_norm': 0.4009410685025627, 'learning_rate': 7e-06, 'epoch': 0.82}
0: /s]
 82%|████████▏ | 2427/2975 [39:21<07:51,  1.16it/s]
 82%|████████▏ | 2428/2975 [39:21<07:51,  1.16it/s]
 82%|████████▏ | 2429/2975 [39:22<07:57,  1.14it/s]
 82%|████████▏ | 2430/2975 [39:23<07:54,  1.15it/s]
                                                   

 82%|████████▏ | 2430/2975 [39:23<07:54,  1.15it/s]
 82%|████████▏ | 2431/2975 [39:24<07:51,  1.15it/s]
 82%|████████▏ | 2432/2975 [39:25<07:49,  1.16it/s]
 82%|████████▏ | 2433/2975 [39:26<07:48,  1.16it/s]
 82%|████████▏ | 2434/2975 [39:27<07:46,  1.16it/s]
 82%|████████▏ | 2435/2975 [39:27<07:45,  1.16it/s]
 82%|████████▏ | 2436/2975 [39:28<07:45,  1.16it/s]
 82%|████████▏ | 2437/2975 [39:29<07:44,  1.16it/s]
 82%|████████▏ | 2438/2975 [39:30<07:42,  1.16it/s]
 82%|████████▏ | 2439/2975 [39:31<07:42
0: {'loss': 0.5533, 'grad_norm': 0.39124185547114904, 'learning_rate': 7e-06, 'epoch': 0.82}
0: {'loss': 0.5422, 'grad_norm': 0.4170815069697654, 'learning_rate': 7e-06, 'epoch': 0.82}
0: ,  1.16it/s]
 82%|████████▏ | 2440/2975 [39:32<07:40,  1.16it/s]
                                                   

 82%|████████▏ | 2440/2975 [39:32<07:40,  1.16it/s]
 82%|████████▏ | 2441/2975 [39:33<07:40,  1.16it/s]
 82%|████████▏ | 2442/2975 [39:34<07:41,  1.15it/s]
 82%|████████▏ | 2443/2975 [39:34<07:40,  1.15it/s]
 82%|████████▏ | 2444/2975 [39:35<07:39,  1.16it/s]
 82%|████████▏ | 2445/2975 [39:36<07:37,  1.16it/s]
 82%|████████▏ | 2446/2975 [39:37<07:35,  1.16it/s]
 82%|████████▏ | 2447/2975 [39:38<07:34,  1.16it/s]
 82%|████████▏ | 2448/2975 [39:39<07:35,  1.16it/s]
 82%|████████▏ | 2449/2975 [39:40<07:33,  1.16it/s]
 82%|████████▏ | 2450/2975 [39:40<07:32,  1.16it/s]
                                                   

 82%|████████▏ | 2450/2975 [39:40<07:32,  1.16i
0: {'loss': 0.5579, 'grad_norm': 0.38576096348171035, 'learning_rate': 7e-06, 'epoch': 0.83}
0: t/s]
 82%|████████▏ | 2451/2975 [39:41<07:32,  1.16it/s]
 82%|████████▏ | 2452/2975 [39:42<07:31,  1.16it/s]
 82%|████████▏ | 2453/2975 [39:43<07:29,  1.16it/s]
 82%|████████▏ | 2454/2975 [39:44<07:29,  1.16it/s]
 83%|████████▎ | 2455/2975 [39:45<08:03,  1.07it/s]
 83%|████████▎ | 2456/2975 [39:46<07:51,  1.10it/s]
 83%|████████▎ | 2457/2975 [39:47<07:43,  1.12it/s]
 83%|████████▎ | 2458/2975 [39:48<07:37,  1.13it/s]
 83%|████████▎ | 2459/2975 [39:48<07:33,  1.14it/s]
 83%|████████▎ | 2460/2975 [39:49<07:30,  1.14it/s]
                                                   

 83%|████████▎ | 2460/2975 [39:49<07:30,  1.14it/s]
 83%|████████▎ | 2461/2975 [39:50<07:27,  1.15it/s]
 83%|████████▎ | 2462/2975 [39:51<07:25,  1.15it/s]
 83%|████████▎ | 2463/2975 [39:52<07:2
0: {'loss': 0.5799, 'grad_norm': 0.4167891263869314, 'learning_rate': 7e-06, 'epoch': 0.83}
0: 3,  1.15it/s]
 83%|████████▎ | 2464/2975 [39:53<07:21,  1.16it/s]
 83%|████████▎ | 2465/2975 [39:54<07:21,  1.16it/s]
 83%|████████▎ | 2466/2975 [39:54<07:27,  1.14it/s]
 83%|████████▎ | 2467/2975 [39:55<07:23,  1.15it/s]
 83%|████████▎ | 2468/2975 [39:56<07:20,  1.15it/s]
 83%|████████▎ | 2469/2975 [39:57<07:18,  1.15it/s]
 83%|████████▎ | 2470/2975 [39:58<07:16,  1.16it/s]
                                                   

 83%|████████▎ | 2470/2975 [39:58<07:16,  1.16it/s]
 83%|████████▎ | 2471/2975 [39:59<07:16,  1.16it/s]
 83%|████████▎ | 2472/2975 [40:00<07:14,  1.16it/s]
 83%|████████▎ | 2473/2975 [40:01<07:13,  1.16it/s]
 83%|████████▎ | 2474/2975 [40:01<07:12,  1.16it/s]
 83%|████████▎ | 2475/2975 [40:02<07:10,  1.16it/s]
 83%|████████▎ | 2476/2975 [4
0: {'loss': 0.5552, 'grad_norm': 0.3924698679889801, 'learning_rate': 7e-06, 'epoch': 0.83}
0: 0:03<07:10,  1.16it/s]
 83%|████████▎ | 2477/2975 [40:04<07:08,  1.16it/s]
 83%|████████▎ | 2478/2975 [40:05<07:08,  1.16it/s]
 83%|████████▎ | 2479/2975 [40:06<07:07,  1.16it/s]
 83%|████████▎ | 2480/2975 [40:07<07:06,  1.16it/s]
                                                   

 83%|████████▎ | 2480/2975 [40:07<07:06,  1.16it/s]
 83%|████████▎ | 2481/2975 [40:08<07:24,  1.11it/s]
 83%|████████▎ | 2482/2975 [40:08<07:18,  1.12it/s]
 83%|████████▎ | 2483/2975 [40:09<07:13,  1.14it/s]
 83%|████████▎ | 2484/2975 [40:10<07:09,  1.14it/s]
 84%|████████▎ | 2485/2975 [40:11<07:06,  1.15it/s]
 84%|████████▎ | 2486/2975 [40:12<07:04,  1.15it/s]
 84%|████████▎ | 2487/2975 [40:13<07:07,  1.14it/s]
 84%|████████▎ | 2488/2975 [40:14<07:04,  1.15it/s]
 84%|████████▎ | 248
0: {'loss': 0.5632, 'grad_norm': 0.41364986835685064, 'learning_rate': 7e-06, 'epoch': 0.84}
0: {'loss': 0.5592, 'grad_norm': 0.4031734046358304, 'learning_rate': 7e-06, 'epoch': 0.84}
0: 9/2975 [40:14<07:02,  1.15it/s]
 84%|████████▎ | 2490/2975 [40:15<07:00,  1.15it/s]
                                                   

 84%|████████▎ | 2490/2975 [40:15<07:00,  1.15it/s]
 84%|████████▎ | 2491/2975 [40:16<06:59,  1.15it/s]
 84%|████████▍ | 2492/2975 [40:17<06:58,  1.15it/s]
 84%|████████▍ | 2493/2975 [40:18<06:56,  1.16it/s]
 84%|████████▍ | 2494/2975 [40:19<06:54,  1.16it/s]
 84%|████████▍ | 2495/2975 [40:20<06:54,  1.16it/s]
 84%|████████▍ | 2496/2975 [40:20<06:53,  1.16it/s]
 84%|████████▍ | 2497/2975 [40:21<06:52,  1.16it/s]
 84%|████████▍ | 2498/2975 [40:22<06:51,  1.16it/s]
 84%|████████▍ | 2499/2975 [40:23<06:50,  1.16it/s]
 84%|████████▍ | 2500/2975 [40:24<06:49,  1.16it/s]
                                                   

 84%|████████▍ | 2500/2975 [
0: {'loss': 0.54, 'grad_norm': 0.39318942416684366, 'learning_rate': 7e-06, 'epoch': 0.84}
0: 40:24<06:49,  1.16it/s]
 84%|████████▍ | 2501/2975 [40:25<06:48,  1.16it/s]
 84%|████████▍ | 2502/2975 [40:26<06:48,  1.16it/s]
 84%|████████▍ | 2503/2975 [40:27<06:51,  1.15it/s]
 84%|████████▍ | 2504/2975 [40:27<06:48,  1.15it/s]
 84%|████████▍ | 2505/2975 [40:28<06:46,  1.15it/s]
 84%|████████▍ | 2506/2975 [40:29<06:45,  1.16it/s]
 84%|████████▍ | 2507/2975 [40:30<06:43,  1.16it/s]
 84%|████████▍ | 2508/2975 [40:31<06:42,  1.16it/s]
 84%|████████▍ | 2509/2975 [40:32<06:40,  1.16it/s]
 84%|████████▍ | 2510/2975 [40:33<06:39,  1.16it/s]
                                                   

 84%|████████▍ | 2510/2975 [40:33<06:39,  1.16it/s]
 84%|████████▍ | 2511/2975 [40:33<06:39,  1.16it/s]
 84%|████████▍ | 2512/2975 [40:34<06:38,  1.16it/s]
 84%|████████▍ | 25
0: {'loss': 0.5587, 'grad_norm': 0.4065661507943617, 'learning_rate': 7e-06, 'epoch': 0.85}
0: 13/2975 [40:35<06:38,  1.16it/s]
 85%|████████▍ | 2514/2975 [40:36<06:36,  1.16it/s]
 85%|████████▍ | 2515/2975 [40:37<06:36,  1.16it/s]
 85%|████████▍ | 2516/2975 [40:38<06:35,  1.16it/s]
 85%|████████▍ | 2517/2975 [40:39<06:34,  1.16it/s]
 85%|████████▍ | 2518/2975 [40:39<06:34,  1.16it/s]
 85%|████████▍ | 2519/2975 [40:40<06:35,  1.15it/s]
 85%|████████▍ | 2520/2975 [40:41<06:33,  1.15it/s]
                                                   

 85%|████████▍ | 2520/2975 [40:41<06:33,  1.15it/s]
 85%|████████▍ | 2521/2975 [40:42<06:33,  1.16it/s]
 85%|████████▍ | 2522/2975 [40:43<06:31,  1.16it/s]
 85%|████████▍ | 2523/2975 [40:44<06:30,  1.16it/s]
 85%|████████▍ | 2524/2975 [40:45<06:29,  1.16it/s]
 85%|████████▍ | 2525/2975 [40:46<06:28,  1.16it/s]
 85%|███████�
0: {'loss': 0.5543, 'grad_norm': 0.38542545611695267, 'learning_rate': 7e-06, 'epoch': 0.85}
0: �▍ | 2526/2975 [40:46<06:27,  1.16it/s]
 85%|████████▍ | 2527/2975 [40:47<06:26,  1.16it/s]
 85%|████████▍ | 2528/2975 [40:48<06:26,  1.16it/s]
 85%|████████▌ | 2529/2975 [40:49<06:24,  1.16it/s]
 85%|████████▌ | 2530/2975 [40:50<06:23,  1.16it/s]
                                                   

 85%|████████▌ | 2530/2975 [40:50<06:23,  1.16it/s]
 85%|████████▌ | 2531/2975 [40:51<06:23,  1.16it/s]
 85%|████████▌ | 2532/2975 [40:52<06:22,  1.16it/s]
 85%|████████▌ | 2533/2975 [40:52<06:21,  1.16it/s]
 85%|████████▌ | 2534/2975 [40:53<06:20,  1.16it/s]
 85%|████████▌ | 2535/2975 [40:54<06:20,  1.16it/s]
 85%|████████▌ | 2536/2975 [40:55<06:19,  1.16it/s]
 85%|████████▌ | 2537/2975 [40:56<06:17,  1.16it/s]
 85%|████████▌ | 2538/2975 [40:57<06:16,  1.16it/s]
 85%|████�
0: {'loss': 0.5589, 'grad_norm': 0.4021866127023806, 'learning_rate': 7e-06, 'epoch': 0.85}
0: {'loss': 0.5462, 'grad_norm': 0.3992115256569203, 'learning_rate': 7e-06, 'epoch': 0.86}
0: �███▌ | 2539/2975 [40:58<06:16,  1.16it/s]
 85%|████████▌ | 2540/2975 [40:58<06:17,  1.15it/s]
                                                   

 85%|████████▌ | 2540/2975 [40:58<06:17,  1.15it/s]
 85%|████████▌ | 2541/2975 [40:59<06:16,  1.15it/s]
 85%|████████▌ | 2542/2975 [41:00<06:15,  1.15it/s]
 85%|████████▌ | 2543/2975 [41:01<06:14,  1.15it/s]
 86%|████████▌ | 2544/2975 [41:02<06:14,  1.15it/s]
 86%|████████▌ | 2545/2975 [41:03<06:12,  1.15it/s]
 86%|████████▌ | 2546/2975 [41:04<06:11,  1.15it/s]
 86%|████████▌ | 2547/2975 [41:05<06:10,  1.15it/s]
 86%|████████▌ | 2548/2975 [41:05<06:09,  1.16it/s]
 86%|████████▌ | 2549/2975 [41:06<06:08,  1.16it/s]
 86%|████████▌ | 2550/2975 [41:07<06:07,  1.16it/s]
                                                   

 86%|███████�
0: {'loss': 0.5536, 'grad_norm': 0.3976973150855805, 'learning_rate': 7e-06, 'epoch': 0.86}
0: ��▌ | 2550/2975 [41:07<06:07,  1.16it/s]
 86%|████████▌ | 2551/2975 [41:08<06:07,  1.16it/s]
 86%|████████▌ | 2552/2975 [41:09<06:19,  1.12it/s]
 86%|████████▌ | 2553/2975 [41:10<06:14,  1.13it/s]
 86%|████████▌ | 2554/2975 [41:11<06:10,  1.14it/s]
 86%|████████▌ | 2555/2975 [41:12<06:07,  1.14it/s]
 86%|████████▌ | 2556/2975 [41:12<06:04,  1.15it/s]
 86%|████████▌ | 2557/2975 [41:13<06:11,  1.13it/s]
 86%|████████▌ | 2558/2975 [41:14<06:07,  1.13it/s]
 86%|████████▌ | 2559/2975 [41:15<06:04,  1.14it/s]
 86%|████████▌ | 2560/2975 [41:16<06:02,  1.15it/s]
                                                   

 86%|████████▌ | 2560/2975 [41:16<06:02,  1.15it/s]
 86%|████████▌ | 2561/2975 [41:17<06:00,  1.15it/s]
 86%|████████▌ | 2562/2975 [41:18<05:59,  1.15it/s]
 86%|████�
0: {'loss': 0.556, 'grad_norm': 0.3919412746946118, 'learning_rate': 7e-06, 'epoch': 0.86}
0: ��███▌ | 2563/2975 [41:19<05:57,  1.15it/s]
 86%|████████▌ | 2564/2975 [41:19<05:56,  1.15it/s]
 86%|████████▌ | 2565/2975 [41:20<05:54,  1.15it/s]
 86%|████████▋ | 2566/2975 [41:21<05:53,  1.16it/s]
 86%|████████▋ | 2567/2975 [41:22<05:53,  1.16it/s]
 86%|████████▋ | 2568/2975 [41:23<05:51,  1.16it/s]
 86%|████████▋ | 2569/2975 [41:24<05:53,  1.15it/s]
 86%|████████▋ | 2570/2975 [41:25<05:51,  1.15it/s]
                                                   

 86%|████████▋ | 2570/2975 [41:25<05:51,  1.15it/s]
 86%|████████▋ | 2571/2975 [41:25<05:51,  1.15it/s]
 86%|████████▋ | 2572/2975 [41:26<05:49,  1.15it/s]
 86%|████████▋ | 2573/2975 [41:27<05:48,  1.15it/s]
 87%|████████▋ | 2574/2975 [41:28<05:47,  1.15it/s]
 87%|████████▋ | 2575/2975 [41:29<05:46,  1.15it/s]
 87%|█�
0: {'loss': 0.5458, 'grad_norm': 0.3851427762673055, 'learning_rate': 7e-06, 'epoch': 0.87}
0: ��██████▋ | 2576/2975 [41:30<05:45,  1.16it/s]
 87%|████████▋ | 2577/2975 [41:31<05:44,  1.16it/s]
 87%|████████▋ | 2578/2975 [41:32<05:43,  1.16it/s]
 87%|████████▋ | 2579/2975 [41:32<05:42,  1.16it/s]
 87%|████████▋ | 2580/2975 [41:33<05:41,  1.16it/s]
                                                   

 87%|████████▋ | 2580/2975 [41:33<05:41,  1.16it/s]
 87%|████████▋ | 2581/2975 [41:34<05:41,  1.15it/s]
 87%|████████▋ | 2582/2975 [41:35<05:46,  1.14it/s]
 87%|████████▋ | 2583/2975 [41:36<05:43,  1.14it/s]
 87%|████████▋ | 2584/2975 [41:37<05:40,  1.15it/s]
 87%|████████▋ | 2585/2975 [41:38<05:38,  1.15it/s]
 87%|████████▋ | 2586/2975 [41:39<05:37,  1.15it/s]
 87%|████████▋ | 2587/2975 [41:39<05:36,  1.15it/s]
 87%|████████▋ | 2588/2975 [41:40<05:34,  1.16it/s]
0: {'loss': 0.5288, 'grad_norm': 0.39125312875219587, 'learning_rate': 7e-06, 'epoch': 0.87}
0: {'loss': 0.5408, 'grad_norm': 0.40411645744144825, 'learning_rate': 7e-06, 'epoch': 0.87}
0:  87%|████████▋ | 2589/2975 [41:41<05:33,  1.16it/s]
 87%|████████▋ | 2590/2975 [41:42<05:32,  1.16it/s]
                                                   

 87%|████████▋ | 2590/2975 [41:42<05:32,  1.16it/s]
 87%|████████▋ | 2591/2975 [41:43<05:31,  1.16it/s]
 87%|████████▋ | 2592/2975 [41:44<05:30,  1.16it/s]
 87%|████████▋ | 2593/2975 [41:45<05:29,  1.16it/s]
 87%|████████▋ | 2594/2975 [41:45<05:28,  1.16it/s]
 87%|████████▋ | 2595/2975 [41:46<05:27,  1.16it/s]
 87%|████████▋ | 2596/2975 [41:47<05:27,  1.16it/s]
 87%|████████▋ | 2597/2975 [41:48<05:26,  1.16it/s]
 87%|████████▋ | 2598/2975 [41:49<05:25,  1.16it/s]
 87%|████████▋ | 2599/2975 [41:50<05:29,  1.14it/s]
 87%|████████▋ | 2600/2975 [41:51<05:26,  1.15it/s]
                                                   

 87%|█
0: {'loss': 0.5432, 'grad_norm': 0.3969212495213628, 'learning_rate': 7e-06, 'epoch': 0.88}
0: ███████▋ | 2600/2975 [41:51<05:26,  1.15it/s]
 87%|████████▋ | 2601/2975 [41:52<05:25,  1.15it/s]
 87%|████████▋ | 2602/2975 [41:52<05:31,  1.13it/s]
 87%|████████▋ | 2603/2975 [41:53<05:27,  1.14it/s]
 88%|████████▊ | 2604/2975 [41:54<05:24,  1.14it/s]
 88%|████████▊ | 2605/2975 [41:55<05:23,  1.14it/s]
 88%|████████▊ | 2606/2975 [41:56<05:21,  1.15it/s]
 88%|████████▊ | 2607/2975 [41:57<05:19,  1.15it/s]
 88%|████████▊ | 2608/2975 [41:58<05:17,  1.16it/s]
 88%|████████▊ | 2609/2975 [41:58<05:16,  1.16it/s]
 88%|████████▊ | 2610/2975 [41:59<05:15,  1.16it/s]
                                                   

 88%|████████▊ | 2610/2975 [41:59<05:15,  1.16it/s]
 88%|████████▊ | 2611/2975 [42:00<05:40,  1.07it/s]
 88%|████████▊ | 2612/2975 [42:01<05:31,  1.10it/s]
0: {'loss': 0.5526, 'grad_norm': 0.42241331221049827, 'learning_rate': 7e-06, 'epoch': 0.88}
0: 
 88%|████████▊ | 2613/2975 [42:02<05:24,  1.11it/s]
 88%|████████▊ | 2614/2975 [42:03<05:20,  1.13it/s]
 88%|████████▊ | 2615/2975 [42:04<05:16,  1.14it/s]
 88%|████████▊ | 2616/2975 [42:05<05:14,  1.14it/s]
 88%|████████▊ | 2617/2975 [42:06<05:12,  1.14it/s]
 88%|████████▊ | 2618/2975 [42:06<05:11,  1.15it/s]
 88%|████████▊ | 2619/2975 [42:07<05:10,  1.15it/s]
 88%|████████▊ | 2620/2975 [42:08<05:08,  1.15it/s]
                                                   

 88%|████████▊ | 2620/2975 [42:08<05:08,  1.15it/s]
 88%|████████▊ | 2621/2975 [42:09<05:07,  1.15it/s]
 88%|████████▊ | 2622/2975 [42:10<05:06,  1.15it/s]
 88%|████████▊ | 2623/2975 [42:11<05:04,  1.15it/s]
 88%|████████▊ | 2624/2975 [42:12<05:03,  1.16it/s]
 88%|████████▊ | 2625/2975 [42:13<05:02,  
0: {'loss': 0.5609, 'grad_norm': 0.41636269998733666, 'learning_rate': 7e-06, 'epoch': 0.88}
0: 1.16it/s]
 88%|████████▊ | 2626/2975 [42:13<05:01,  1.16it/s]
 88%|████████▊ | 2627/2975 [42:14<05:23,  1.08it/s]
 88%|████████▊ | 2628/2975 [42:15<05:15,  1.10it/s]
 88%|████████▊ | 2629/2975 [42:16<05:09,  1.12it/s]
 88%|████████▊ | 2630/2975 [42:17<05:06,  1.13it/s]
                                                   

 88%|████████▊ | 2630/2975 [42:17<05:06,  1.13it/s]
 88%|████████▊ | 2631/2975 [42:18<05:03,  1.13it/s]
 88%|████████▊ | 2632/2975 [42:19<05:00,  1.14it/s]
 89%|████████▊ | 2633/2975 [42:20<04:58,  1.15it/s]
 89%|████████▊ | 2634/2975 [42:21<04:56,  1.15it/s]
 89%|████████▊ | 2635/2975 [42:21<04:54,  1.15it/s]
 89%|████████▊ | 2636/2975 [42:22<04:53,  1.15it/s]
 89%|████████▊ | 2637/2975 [42:23<04:52,  1.15it/s]
 89%|████████▊ | 2638/2975 [42:24
0: {'loss': 0.5538, 'grad_norm': 0.4042302487808405, 'learning_rate': 7e-06, 'epoch': 0.89}
0: <04:51,  1.16it/s]
 89%|████████▊ | 2639/2975 [42:25<04:50,  1.16it/s]
 89%|████████▊ | 2640/2975 [42:26<04:50,  1.16it/s]
                                                   

 89%|████████▊ | 2640/2975 [42:26<04:50,  1.16it/s]
 89%|████████▉ | 2641/2975 [42:27<04:49,  1.15it/s]
 89%|████████▉ | 2642/2975 [42:27<04:48,  1.16it/s]
 89%|████████▉ | 2643/2975 [42:28<04:47,  1.15it/s]
 89%|████████▉ | 2644/2975 [42:29<04:46,  1.16it/s]
 89%|████████▉ | 2645/2975 [42:30<04:45,  1.16it/s]
 89%|████████▉ | 2646/2975 [42:31<04:44,  1.16it/s]
 89%|████████▉ | 2647/2975 [42:32<04:42,  1.16it/s]
 89%|████████▉ | 2648/2975 [42:33<04:46,  1.14it/s]
 89%|████████▉ | 2649/2975 [42:34<04:44,  1.15it/s]
 89%|████████▉ | 2650/2975 [42:34<04:42,  1.15it/s]
                                          
0: {'loss': 0.5573, 'grad_norm': 0.40824996767980704, 'learning_rate': 7e-06, 'epoch': 0.89}
0: {'loss': 0.5479, 'grad_norm': 0.4041150531736402, 'learning_rate': 7e-06, 'epoch': 0.89}
0:          

 89%|████████▉ | 2650/2975 [42:34<04:42,  1.15it/s]
 89%|████████▉ | 2651/2975 [42:35<04:41,  1.15it/s]
 89%|████████▉ | 2652/2975 [42:36<04:40,  1.15it/s]
 89%|████████▉ | 2653/2975 [42:37<04:38,  1.16it/s]
 89%|████████▉ | 2654/2975 [42:38<04:37,  1.16it/s]
 89%|████████▉ | 2655/2975 [42:39<04:36,  1.16it/s]
 89%|████████▉ | 2656/2975 [42:40<04:35,  1.16it/s]
 89%|████████▉ | 2657/2975 [42:40<04:35,  1.16it/s]
 89%|████████▉ | 2658/2975 [42:41<04:33,  1.16it/s]
 89%|████████▉ | 2659/2975 [42:42<04:32,  1.16it/s]
 89%|████████▉ | 2660/2975 [42:43<04:32,  1.16it/s]
                                                   

 89%|████████▉ | 2660/2975 [42:43<04:32,  1.16it/s]
 89%|████████▉ | 2661/2975 [42:44<04:31,  1.16it/s]
 89%|████████▉ | 2662/2975 [42:4
0: {'loss': 0.5372, 'grad_norm': 0.4022772848644423, 'learning_rate': 7e-06, 'epoch': 0.9}
0: 5<04:30,  1.16it/s]
 90%|████████▉ | 2663/2975 [42:46<04:45,  1.09it/s]
 90%|████████▉ | 2664/2975 [42:47<04:39,  1.11it/s]
 90%|████████▉ | 2665/2975 [42:48<04:35,  1.13it/s]
 90%|████████▉ | 2666/2975 [42:48<04:32,  1.14it/s]
 90%|████████▉ | 2667/2975 [42:49<04:29,  1.14it/s]
 90%|████████▉ | 2668/2975 [42:50<04:27,  1.15it/s]
 90%|████████▉ | 2669/2975 [42:51<04:25,  1.15it/s]
 90%|████████▉ | 2670/2975 [42:52<04:24,  1.15it/s]
                                                   

 90%|████████▉ | 2670/2975 [42:52<04:24,  1.15it/s]
 90%|████████▉ | 2671/2975 [42:53<04:23,  1.15it/s]
 90%|████████▉ | 2672/2975 [42:54<04:22,  1.16it/s]
 90%|████████▉ | 2673/2975 [42:54<04:21,  1.16it/s]
 90%|████████▉ | 2674/2975 [42:55<04:20,  1.16it/s]
 90%|████████▉ | 2675/2
0: {'loss': 0.5312, 'grad_norm': 0.4020664539255337, 'learning_rate': 6.997236914811404e-06, 'epoch': 0.9}
0: 975 [42:56<04:23,  1.14it/s]
 90%|████████▉ | 2676/2975 [42:57<04:20,  1.15it/s]
 90%|████████▉ | 2677/2975 [42:58<04:19,  1.15it/s]
 90%|█████████ | 2678/2975 [42:59<04:17,  1.15it/s]
 90%|█████████ | 2679/2975 [43:00<04:16,  1.15it/s]
 90%|█████████ | 2680/2975 [43:01<04:15,  1.16it/s]
                                                   

 90%|█████████ | 2680/2975 [43:01<04:15,  1.16it/s]
 90%|█████████ | 2681/2975 [43:01<04:14,  1.16it/s]
 90%|█████████ | 2682/2975 [43:02<04:13,  1.16it/s]
 90%|█████████ | 2683/2975 [43:03<04:11,  1.16it/s]
 90%|█████████ | 2684/2975 [43:04<04:11,  1.16it/s]
 90%|█████████ | 2685/2975 [43:05<04:10,  1.16it/s]
 90%|█████████ | 2686/2975 [43:06<04:09,  1.16it/s]
 90%|█████████ | 2687/2975 [43:07<04:08,  1.16it/s]
 90%|█████████
0: {'loss': 0.5515, 'grad_norm': 0.4003807641064491, 'learning_rate': 6.9662078488334135e-06, 'epoch': 0.9}
0:  | 2688/2975 [43:07<04:08,  1.16it/s]
 90%|█████████ | 2689/2975 [43:08<04:07,  1.16it/s]
 90%|█████████ | 2690/2975 [43:09<04:06,  1.16it/s]
                                                   

 90%|█████████ | 2690/2975 [43:09<04:06,  1.16it/s]
 90%|█████████ | 2691/2975 [43:10<04:05,  1.16it/s]
 90%|█████████ | 2692/2975 [43:11<04:04,  1.16it/s]
 91%|█████████ | 2693/2975 [43:12<04:03,  1.16it/s]
 91%|█████████ | 2694/2975 [43:13<04:02,  1.16it/s]
 91%|█████████ | 2695/2975 [43:13<04:01,  1.16it/s]
 91%|█████████ | 2696/2975 [43:14<04:00,  1.16it/s]
 91%|█████████ | 2697/2975 [43:15<03:59,  1.16it/s]
 91%|█████████ | 2698/2975 [43:16<03:58,  1.16it/s]
 91%|█████████ | 2699/2975 [43:17<03:57,  1.16it/s]
 91%|█████████ | 2700/2975 [43:18<03:57,  1.16it/s]
                       
0: {'loss': 0.5445, 'grad_norm': 0.40759855171933507, 'learning_rate': 6.901036957555188e-06, 'epoch': 0.91}
0: {'loss': 0.5635, 'grad_norm': 0.39504377427215553, 'learning_rate': 6.802438266899458e-06, 'epoch': 0.91}
0:                             

 91%|█████████ | 2700/2975 [43:18<03:57,  1.16it/s]
 91%|█████████ | 2701/2975 [43:19<03:56,  1.16it/s]
 91%|█████████ | 2702/2975 [43:20<03:55,  1.16it/s]
 91%|█████████ | 2703/2975 [43:20<03:54,  1.16it/s]
 91%|█████████ | 2704/2975 [43:21<03:53,  1.16it/s]
 91%|█████████ | 2705/2975 [43:22<03:52,  1.16it/s]
 91%|█████████ | 2706/2975 [43:23<03:51,  1.16it/s]
 91%|█████████ | 2707/2975 [43:24<03:50,  1.16it/s]
 91%|█████████ | 2708/2975 [43:25<04:06,  1.08it/s]
 91%|█████████ | 2709/2975 [43:26<04:00,  1.11it/s]
 91%|█████████ | 2710/2975 [43:27<03:55,  1.12it/s]
                                                   

 91%|█████████ | 2710/2975 [43:27<03:55,  1.12it/s]
 91%|█████████ | 2711/2975 [43:27<03:53,  1.13it/s]
 91%|████████�
0: {'loss': 0.5496, 'grad_norm': 0.3862329898186045, 'learning_rate': 6.67149204475415e-06, 'epoch': 0.91}
0: � | 2712/2975 [43:28<03:50,  1.14it/s]
 91%|█████████ | 2713/2975 [43:29<03:48,  1.15it/s]
 91%|█████████ | 2714/2975 [43:30<03:46,  1.15it/s]
 91%|█████████▏| 2715/2975 [43:31<03:45,  1.15it/s]
 91%|█████████▏| 2716/2975 [43:32<03:44,  1.16it/s]
 91%|█████████▏| 2717/2975 [43:33<03:43,  1.16it/s]
 91%|█████████▏| 2718/2975 [43:33<03:42,  1.16it/s]
 91%|█████████▏| 2719/2975 [43:34<03:41,  1.16it/s]
 91%|█████████▏| 2720/2975 [43:35<03:40,  1.16it/s]
                                                   

 91%|█████████▏| 2720/2975 [43:35<03:40,  1.16it/s]
 91%|█████████▏| 2721/2975 [43:36<03:39,  1.16it/s]
 91%|█████████▏| 2722/2975 [43:37<03:38,  1.16it/s]
 92%|█████████▏| 2723/2975 [43:38<03:42,  1.13it/s]
 92%|█████████▏| 2724/2975 [43:39<03:40,  1.14it/s]
0: {'loss': 0.5553, 'grad_norm': 0.39880256834136574, 'learning_rate': 6.509632965331348e-06, 'epoch': 0.92}
0:  92%|█████████▏| 2725/2975 [43:40<03:38,  1.15it/s]
 92%|█████████▏| 2726/2975 [43:40<03:36,  1.15it/s]
 92%|█████████▏| 2727/2975 [43:41<03:35,  1.15it/s]
 92%|█████████▏| 2728/2975 [43:42<03:33,  1.16it/s]
 92%|█████████▏| 2729/2975 [43:43<03:32,  1.16it/s]
 92%|█████████▏| 2730/2975 [43:44<03:31,  1.16it/s]
                                                   

 92%|█████████▏| 2730/2975 [43:44<03:31,  1.16it/s]
 92%|█████████▏| 2731/2975 [43:45<03:30,  1.16it/s]
 92%|█████████▏| 2732/2975 [43:46<03:29,  1.16it/s]
 92%|█████████▏| 2733/2975 [43:47<03:29,  1.16it/s]
 92%|█████████▏| 2734/2975 [43:47<03:27,  1.16it/s]
 92%|█████████▏| 2735/2975 [43:48<03:27,  1.16it/s]
 92%|█████████▏| 2736/2975 [43:49<03:26,  1.16it/s]
 92%|█████████▏
0: {'loss': 0.5378, 'grad_norm': 0.39326706836078557, 'learning_rate': 6.318634390576395e-06, 'epoch': 0.92}
0: | 2737/2975 [43:50<03:25,  1.16it/s]
 92%|█████████▏| 2738/2975 [43:51<03:24,  1.16it/s]
 92%|█████████▏| 2739/2975 [43:52<03:23,  1.16it/s]
 92%|█████████▏| 2740/2975 [43:53<03:23,  1.16it/s]
                                                   

 92%|█████████▏| 2740/2975 [43:53<03:23,  1.16it/s]
 92%|█████████▏| 2741/2975 [43:53<03:22,  1.15it/s]
 92%|█████████▏| 2742/2975 [43:54<03:21,  1.16it/s]
 92%|█████████▏| 2743/2975 [43:55<03:20,  1.16it/s]
 92%|█████████▏| 2744/2975 [43:56<03:19,  1.16it/s]
 92%|█████████▏| 2745/2975 [43:57<03:18,  1.16it/s]
 92%|█████████▏| 2746/2975 [43:58<03:17,  1.16it/s]
 92%|█████████▏| 2747/2975 [43:59<03:16,  1.16it/s]
 92%|█████████▏| 2748/2975 [43:59<03:15,  1.16it/s]
 92%|█████████▏| 2749/2975 [44:00<03:15,  1.16it/s
0: {'loss': 0.549, 'grad_norm': 0.3901651301280971, 'learning_rate': 6.10058894084333e-06, 'epoch': 0.92}
0: {'loss': 0.5395, 'grad_norm': 0.3773297783079588, 'learning_rate': 5.857885567708372e-06, 'epoch': 0.93}
0: ]
 92%|█████████▏| 2750/2975 [44:01<03:14,  1.16it/s]
                                                   

 92%|█████████▏| 2750/2975 [44:01<03:14,  1.16it/s]
 92%|█████████▏| 2751/2975 [44:02<03:13,  1.16it/s]
 93%|█████████▎| 2752/2975 [44:03<03:12,  1.16it/s]
 93%|█████████▎| 2753/2975 [44:04<03:17,  1.12it/s]
 93%|█████████▎| 2754/2975 [44:05<03:14,  1.14it/s]
 93%|█████████▎| 2755/2975 [44:06<03:14,  1.13it/s]
 93%|█████████▎| 2756/2975 [44:06<03:12,  1.14it/s]
 93%|█████████▎| 2757/2975 [44:07<03:10,  1.15it/s]
 93%|█████████▎| 2758/2975 [44:08<03:08,  1.15it/s]
 93%|█████████▎| 2759/2975 [44:09<03:06,  1.16it/s]
 93%|█████████▎| 2760/2975 [44:10<03:05,  1.16it/s]
                                                   

 93%|█████████▎| 2760/2975 [44:1
0: {'loss': 0.5281, 'grad_norm': 0.38697433164410866, 'learning_rate': 5.5931833801165335e-06, 'epoch': 0.93}
0: 0<03:05,  1.16it/s]
 93%|█████████▎| 2761/2975 [44:11<03:04,  1.16it/s]
 93%|█████████▎| 2762/2975 [44:12<03:03,  1.16it/s]
 93%|█████████▎| 2763/2975 [44:12<03:02,  1.16it/s]
 93%|█████████▎| 2764/2975 [44:13<03:01,  1.16it/s]
 93%|█████████▎| 2765/2975 [44:14<03:00,  1.16it/s]
 93%|█████████▎| 2766/2975 [44:15<03:12,  1.08it/s]
 93%|█████████▎| 2767/2975 [44:16<03:08,  1.10it/s]
 93%|█████████▎| 2768/2975 [44:17<03:04,  1.12it/s]
 93%|█████████▎| 2769/2975 [44:18<03:01,  1.13it/s]
 93%|█████████▎| 2770/2975 [44:19<02:59,  1.14it/s]
                                                   

 93%|█████████▎| 2770/2975 [44:19<02:59,  1.14it/s]
 93%|█████████▎| 2771/2975 [44:20<02:57,  1.15it/s]
 93%|█████████▎| 2772/2975 [44:20<02:56,  1.15it/s]
 93%|███�
0: {'loss': 0.547, 'grad_norm': 0.39596909838534416, 'learning_rate': 5.309382510627565e-06, 'epoch': 0.93}
0: ��█████▎| 2773/2975 [44:21<02:55,  1.15it/s]
 93%|█████████▎| 2774/2975 [44:22<02:54,  1.15it/s]
 93%|█████████▎| 2775/2975 [44:23<02:52,  1.16it/s]
 93%|█████████▎| 2776/2975 [44:24<02:52,  1.16it/s]
 93%|█████████▎| 2777/2975 [44:25<02:50,  1.16it/s]
 93%|█████████▎| 2778/2975 [44:26<02:58,  1.10it/s]
 93%|█████████▎| 2779/2975 [44:27<02:54,  1.12it/s]
 93%|█████████▎| 2780/2975 [44:27<02:52,  1.13it/s]
                                                   

 93%|█████████▎| 2780/2975 [44:27<02:52,  1.13it/s]
 93%|█████████▎| 2781/2975 [44:28<02:50,  1.14it/s]
 94%|█████████▎| 2782/2975 [44:29<02:49,  1.14it/s]
 94%|█████████▎| 2783/2975 [44:30<02:47,  1.15it/s]
 94%|█████████▎| 2784/2975 [44:31<02:45,  1.15it/s]
 94%|█████████▎| 2785/2975 [44
0: {'loss': 0.5346, 'grad_norm': 0.39071408962466864, 'learning_rate': 5.0095923409567364e-06, 'epoch': 0.94}
0: :32<02:44,  1.15it/s]
 94%|█████████▎| 2786/2975 [44:33<02:43,  1.16it/s]
 94%|█████████▎| 2787/2975 [44:34<02:42,  1.16it/s]
 94%|█████████▎| 2788/2975 [44:34<02:41,  1.16it/s]
 94%|█████████▎| 2789/2975 [44:35<02:40,  1.16it/s]
 94%|█████████▍| 2790/2975 [44:36<02:39,  1.16it/s]
                                                   

 94%|█████████▍| 2790/2975 [44:36<02:39,  1.16it/s]
 94%|█████████▍| 2791/2975 [44:37<02:38,  1.16it/s]
 94%|█████████▍| 2792/2975 [44:38<02:37,  1.16it/s]
 94%|█████████▍| 2793/2975 [44:39<02:50,  1.06it/s]
 94%|█████████▍| 2794/2975 [44:40<02:48,  1.07it/s]
 94%|█████████▍| 2795/2975 [44:41<02:43,  1.10it/s]
 94%|█████████▍| 2796/2975 [44:42<02:40,  1.12it/s]
 94%|█████████▍| 2797/2975 [44:42<02:37,  1.13it/s]
 94%|██�
0: {'loss': 0.5508, 'grad_norm': 0.39739096497620685, 'learning_rate': 4.6970974349380866e-06, 'epoch': 0.94}
0: �██████▍| 2798/2975 [44:43<02:35,  1.14it/s]
 94%|█████████▍| 2799/2975 [44:44<02:33,  1.14it/s]
 94%|█████████▍| 2800/2975 [44:45<02:32,  1.15it/s]
                                                   

 94%|█████████▍| 2800/2975 [44:45<02:32,  1.15it/s]
 94%|█████████▍| 2801/2975 [44:46<02:45,  1.05it/s]
 94%|█████████▍| 2802/2975 [44:47<02:42,  1.06it/s]
 94%|█████████▍| 2803/2975 [44:48<02:37,  1.09it/s]
 94%|█████████▍| 2804/2975 [44:49<02:46,  1.03it/s]
 94%|█████████▍| 2805/2975 [44:50<02:39,  1.06it/s]
 94%|█████████▍| 2806/2975 [44:51<02:35,  1.09it/s]
 94%|█████████▍| 2807/2975 [44:52<02:31,  1.11it/s]
 94%|█████████▍| 2808/2975 [44:53<02:30,  1.11it/s]
 94%|█████████▍| 2809/2975 [44:53<02:27,  1.12it/s]
 94%|█████████▍| 2810/2975 [
0: {'loss': 0.5642, 'grad_norm': 0.39916855441510035, 'learning_rate': 4.375321552155722e-06, 'epoch': 0.94}
0: {'loss': 0.5424, 'grad_norm': 0.38577476616043044, 'learning_rate': 4.0477901365173375e-06, 'epoch': 0.95}
0: 44:54<02:25,  1.13it/s]
                                                   

 94%|█████████▍| 2810/2975 [44:54<02:25,  1.13it/s]
 94%|█████████▍| 2811/2975 [44:55<02:23,  1.14it/s]
 95%|█████████▍| 2812/2975 [44:56<02:22,  1.14it/s]
 95%|█████████▍| 2813/2975 [44:57<02:22,  1.13it/s]
 95%|█████████▍| 2814/2975 [44:58<02:21,  1.14it/s]
 95%|█████████▍| 2815/2975 [44:59<02:19,  1.15it/s]
 95%|█████████▍| 2816/2975 [45:00<02:18,  1.15it/s]
 95%|█████████▍| 2817/2975 [45:00<02:17,  1.15it/s]
 95%|█████████▍| 2818/2975 [45:01<02:16,  1.15it/s]
 95%|█████████▍| 2819/2975 [45:02<02:14,  1.16it/s]
 95%|█████████▍| 2820/2975 [45:03<02:15,  1.15it/s]
                                                   

 95%|█████████▍| 2820/2975 [45:03<02:15,  1.15it/s]
 95%|████████�
0: {'loss': 0.5394, 'grad_norm': 0.3709249727987068, 'learning_rate': 3.7180916907530218e-06, 'epoch': 0.95}
0: ��▍| 2821/2975 [45:04<02:14,  1.15it/s]
 95%|█████████▍| 2822/2975 [45:05<02:15,  1.13it/s]
 95%|█████████▍| 2823/2975 [45:06<02:14,  1.13it/s]
 95%|█████████▍| 2824/2975 [45:07<02:12,  1.14it/s]
 95%|█████████▍| 2825/2975 [45:07<02:10,  1.15it/s]
 95%|█████████▍| 2826/2975 [45:08<02:09,  1.15it/s]
 95%|█████████▌| 2827/2975 [45:09<02:08,  1.15it/s]
 95%|█████████▌| 2828/2975 [45:10<02:07,  1.15it/s]
 95%|█████████▌| 2829/2975 [45:11<02:06,  1.15it/s]
 95%|█████████▌| 2830/2975 [45:12<02:05,  1.15it/s]
                                                   

 95%|█████████▌| 2830/2975 [45:12<02:05,  1.15it/s]
 95%|█████████▌| 2831/2975 [45:13<02:16,  1.05it/s]
 95%|█████████▌| 2832/2975 [45:14<02:12,  1.08it/s]
 95%|█████████▌| 2833/2975 [45:15<02:09,  1.1
0: {'loss': 0.5473, 'grad_norm': 0.38496935310007785, 'learning_rate': 3.389838460028404e-06, 'epoch': 0.95}
0: 0it/s]
 95%|█████████▌| 2834/2975 [45:16<02:15,  1.04it/s]
 95%|█████████▌| 2835/2975 [45:17<02:10,  1.07it/s]
 95%|█████████▌| 2836/2975 [45:17<02:06,  1.10it/s]
 95%|█████████▌| 2837/2975 [45:18<02:03,  1.12it/s]
 95%|█████████▌| 2838/2975 [45:19<02:14,  1.02it/s]
 95%|█████████▌| 2839/2975 [45:20<02:08,  1.06it/s]
 95%|█████████▌| 2840/2975 [45:21<02:04,  1.08it/s]
                                                   

 95%|█████████▌| 2840/2975 [45:21<02:04,  1.08it/s]
 95%|█████████▌| 2841/2975 [45:22<02:01,  1.10it/s]
 96%|█████████▌| 2842/2975 [45:23<01:59,  1.12it/s]
 96%|█████████▌| 2843/2975 [45:24<01:56,  1.13it/s]
 96%|█████████▌| 2844/2975 [45:25<01:55,  1.14it/s]
 96%|█████████▌| 2845/2975 [45:26<01:53,  1.14it/s]
 96%|███████�
0: {'loss': 0.5424, 'grad_norm': 0.38186198986510433, 'learning_rate': 3.0666268554307083e-06, 'epoch': 0.96}
0: �█▌| 2846/2975 [45:26<01:52,  1.15it/s]
 96%|█████████▌| 2847/2975 [45:27<01:51,  1.15it/s]
 96%|█████████▌| 2848/2975 [45:28<01:50,  1.15it/s]
 96%|█████████▌| 2849/2975 [45:29<01:49,  1.15it/s]
 96%|█████████▌| 2850/2975 [45:30<01:48,  1.15it/s]
                                                   

 96%|█████████▌| 2850/2975 [45:30<01:48,  1.15it/s]
 96%|█████████▌| 2851/2975 [45:31<01:58,  1.04it/s]
 96%|█████████▌| 2852/2975 [45:32<01:54,  1.08it/s]
 96%|█████████▌| 2853/2975 [45:33<01:51,  1.10it/s]
 96%|█████████▌| 2854/2975 [45:34<01:48,  1.12it/s]
 96%|█████████▌| 2855/2975 [45:34<01:46,  1.13it/s]
 96%|█████████▌| 2856/2975 [45:35<01:44,  1.14it/s]
 96%|█████████▌| 2857/2975 [45:36<01:43,  1.14it/s]
 96%|█████████▌| 2858/2975 [45:37<01:43,  1
0: {'loss': 0.5388, 'grad_norm': 0.3790685853269879, 'learning_rate': 2.7519980509362823e-06, 'epoch': 0.96}
0: .13it/s]
 96%|█████████▌| 2859/2975 [45:38<01:42,  1.14it/s]
 96%|█████████▌| 2860/2975 [45:39<01:40,  1.14it/s]
                                                   

 96%|█████████▌| 2860/2975 [45:39<01:40,  1.14it/s]
 96%|█████████▌| 2861/2975 [45:40<01:39,  1.15it/s]
 96%|█████████▌| 2862/2975 [45:41<01:51,  1.02it/s]
 96%|█████████▌| 2863/2975 [45:42<01:46,  1.05it/s]
 96%|█████████▋| 2864/2975 [45:43<01:42,  1.08it/s]
 96%|█████████▋| 2865/2975 [45:44<01:39,  1.10it/s]
 96%|█████████▋| 2866/2975 [45:44<01:37,  1.12it/s]
 96%|█████████▋| 2867/2975 [45:45<01:35,  1.13it/s]
 96%|█████████▋| 2868/2975 [45:46<01:34,  1.14it/s]
 96%|█████████▋| 2869/2975 [45:47<01:32,  1.14it/s]
 96%|█████████▋| 2870/2975 [45:48<01:31,  1.15it/s]
                          
0: {'loss': 0.5438, 'grad_norm': 0.37753517506431683, 'learning_rate': 2.449399185567479e-06, 'epoch': 0.96}
0: {'loss': 0.552, 'grad_norm': 0.464438664885083, 'learning_rate': 2.16214559581616e-06, 'epoch': 0.97}
0:                          

 96%|█████████▋| 2870/2975 [45:48<01:31,  1.15it/s]
 97%|█████████▋| 2871/2975 [45:49<01:30,  1.15it/s]
 97%|█████████▋| 2872/2975 [45:50<01:29,  1.15it/s]
 97%|█████████▋| 2873/2975 [45:50<01:28,  1.15it/s]
 97%|█████████▋| 2874/2975 [45:51<01:27,  1.15it/s]
 97%|█████████▋| 2875/2975 [45:52<01:26,  1.15it/s]
 97%|█████████▋| 2876/2975 [45:53<01:25,  1.16it/s]
 97%|█████████▋| 2877/2975 [45:54<01:24,  1.15it/s]
 97%|█████████▋| 2878/2975 [45:55<01:23,  1.16it/s]
 97%|█████████▋| 2879/2975 [45:56<01:23,  1.16it/s]
 97%|█████████▋| 2880/2975 [45:57<01:26,  1.10it/s]
                                                   

 97%|█████████▋| 2880/2975 [45:57<01:26,  1.10it/s]
 97%|█████████▋| 2881/2975 [45:58<01:24,  1.11it/s]
 97%|█
0: {'loss': 0.5446, 'grad_norm': 0.37930148013175713, 'learning_rate': 1.8933844921233225e-06, 'epoch': 0.97}
0: ████████▋| 2882/2975 [45:58<01:22,  1.13it/s]
 97%|█████████▋| 2883/2975 [45:59<01:20,  1.14it/s]
 97%|█████████▋| 2884/2975 [46:00<01:19,  1.14it/s]
 97%|█████████▋| 2885/2975 [46:01<01:18,  1.15it/s]
 97%|█████████▋| 2886/2975 [46:02<01:17,  1.15it/s]
 97%|█████████▋| 2887/2975 [46:03<01:16,  1.15it/s]
 97%|█████████▋| 2888/2975 [46:04<01:15,  1.16it/s]
 97%|█████████▋| 2889/2975 [46:04<01:14,  1.16it/s]
 97%|█████████▋| 2890/2975 [46:05<01:13,  1.16it/s]
                                                   

 97%|█████████▋| 2890/2975 [46:05<01:13,  1.16it/s]
 97%|█████████▋| 2891/2975 [46:06<01:12,  1.16it/s]
 97%|█████████▋| 2892/2975 [46:07<01:11,  1.16it/s]
 97%|█████████▋| 2893/2975 [46:08<01:10,  1.16it/s]
 97%|█████████▋| 2894/2
0: {'loss': 0.5678, 'grad_norm': 0.3824398820342379, 'learning_rate': 1.6460604773828986e-06, 'epoch': 0.97}
0: 975 [46:09<01:10,  1.16it/s]
 97%|█████████▋| 2895/2975 [46:10<01:09,  1.16it/s]
 97%|█████████▋| 2896/2975 [46:11<01:08,  1.16it/s]
 97%|█████████▋| 2897/2975 [46:11<01:07,  1.16it/s]
 97%|█████████▋| 2898/2975 [46:12<01:06,  1.16it/s]
 97%|█████████▋| 2899/2975 [46:13<01:05,  1.16it/s]
 97%|█████████▋| 2900/2975 [46:14<01:04,  1.16it/s]
                                                   

 97%|█████████▋| 2900/2975 [46:14<01:04,  1.16it/s]
 98%|█████████▊| 2901/2975 [46:15<01:04,  1.16it/s]
 98%|█████████▊| 2902/2975 [46:16<01:03,  1.16it/s]
 98%|█████████▊| 2903/2975 [46:17<01:06,  1.09it/s]
 98%|█████████▊| 2904/2975 [46:18<01:04,  1.11it/s]
 98%|█████████▊| 2905/2975 [46:18<01:02,  1.12it/s]
 98%|█████████▊| 2906/2975 [46:19<01:00,  1.13it/s]
 98%|�
0: {'loss': 0.5382, 'grad_norm': 0.37938783316358105, 'learning_rate': 1.4228832852562643e-06, 'epoch': 0.98}
0: ��████████▊| 2907/2975 [46:20<00:59,  1.14it/s]
 98%|█████████▊| 2908/2975 [46:21<00:58,  1.15it/s]
 98%|█████████▊| 2909/2975 [46:22<00:57,  1.15it/s]
 98%|█████████▊| 2910/2975 [46:23<00:56,  1.15it/s]
                                                   

 98%|█████████▊| 2910/2975 [46:23<00:56,  1.15it/s]
 98%|█████████▊| 2911/2975 [46:24<00:55,  1.15it/s]
 98%|█████████▊| 2912/2975 [46:25<00:54,  1.15it/s]
 98%|█████████▊| 2913/2975 [46:25<00:53,  1.16it/s]
 98%|█████████▊| 2914/2975 [46:26<00:52,  1.16it/s]
 98%|█████████▊| 2915/2975 [46:27<00:51,  1.16it/s]
 98%|█████████▊| 2916/2975 [46:28<00:50,  1.16it/s]
 98%|█████████▊| 2917/2975 [46:29<00:50,  1.16it/s]
 98%|█████████▊| 2918/2975 [46:30<00:49,  1.16it/s]
 98%|█████████▊| 2919
0: {'loss': 0.5529, 'grad_norm': 0.4113510249009852, 'learning_rate': 1.2262980917631866e-06, 'epoch': 0.98}
0: {'loss': 0.5513, 'grad_norm': 0.36745673129660866, 'learning_rate': 1.0584587254216735e-06, 'epoch': 0.98}
0: /2975 [46:31<00:50,  1.11it/s]
 98%|█████████▊| 2920/2975 [46:32<00:49,  1.12it/s]
                                                   

 98%|█████████▊| 2920/2975 [46:32<00:49,  1.12it/s]
 98%|█████████▊| 2921/2975 [46:32<00:47,  1.13it/s]
 98%|█████████▊| 2922/2975 [46:33<00:46,  1.14it/s]
 98%|█████████▊| 2923/2975 [46:34<00:45,  1.15it/s]
 98%|█████████▊| 2924/2975 [46:35<00:46,  1.10it/s]
 98%|█████████▊| 2925/2975 [46:36<00:44,  1.11it/s]
 98%|█████████▊| 2926/2975 [46:37<00:43,  1.13it/s]
 98%|█████████▊| 2927/2975 [46:38<00:42,  1.13it/s]
 98%|█████████▊| 2928/2975 [46:39<00:41,  1.14it/s]
 98%|█████████▊| 2929/2975 [46:39<00:40,  1.14it/s]
 98%|█████████▊| 2930/2975 [46:40<00:39,  1.15it/s]
                                                   

 98%|██████
0: {'loss': 0.5389, 'grad_norm': 0.3769599860550981, 'learning_rate': 9.212040694520083e-07, 'epoch': 0.99}
0: ███▊| 2930/2975 [46:40<00:39,  1.15it/s]
 99%|█████████▊| 2931/2975 [46:41<00:38,  1.15it/s]
 99%|█████████▊| 2932/2975 [46:42<00:37,  1.15it/s]
 99%|█████████▊| 2933/2975 [46:43<00:36,  1.15it/s]
 99%|█████████▊| 2934/2975 [46:44<00:35,  1.15it/s]
 99%|█████████▊| 2935/2975 [46:45<00:34,  1.15it/s]
 99%|█████████▊| 2936/2975 [46:46<00:33,  1.15it/s]
 99%|█████████▊| 2937/2975 [46:47<00:35,  1.08it/s]
 99%|█████████▉| 2938/2975 [46:47<00:33,  1.10it/s]
 99%|█████████▉| 2939/2975 [46:48<00:32,  1.12it/s]
 99%|█████████▉| 2940/2975 [46:49<00:30,  1.13it/s]
                                                   

 99%|█████████▉| 2940/2975 [46:49<00:30,  1.13it/s]
 99%|█████████▉| 2941/2975 [46:50<00:30,  1.13it/s]
 99%|█████████▉| 2942/2975 [46:51<00:2
0: {'loss': 0.555, 'grad_norm': 0.3882171238542473, 'learning_rate': 8.16037914587377e-07, 'epoch': 0.99}
0: 9,  1.14it/s]
 99%|█████████▉| 2943/2975 [46:52<00:27,  1.14it/s]
 99%|█████████▉| 2944/2975 [46:53<00:27,  1.15it/s]
 99%|█████████▉| 2945/2975 [46:54<00:26,  1.15it/s]
 99%|█████████▉| 2946/2975 [46:54<00:25,  1.15it/s]
 99%|█████████▉| 2947/2975 [46:55<00:24,  1.16it/s]
 99%|█████████▉| 2948/2975 [46:56<00:23,  1.16it/s]
 99%|█████████▉| 2949/2975 [46:57<00:22,  1.16it/s]
 99%|█████████▉| 2950/2975 [46:58<00:21,  1.16it/s]
                                                   

 99%|█████████▉| 2950/2975 [46:58<00:21,  1.16it/s]
 99%|█████████▉| 2951/2975 [46:59<00:20,  1.16it/s]
 99%|█████████▉| 2952/2975 [47:00<00:19,  1.16it/s]
 99%|█████████▉| 2953/2975 [47:00<00:19,  1.16it/s]
 99%|█████████▉| 2954/2975 [47:01<00:18,  1.16it/s]
 99%|█████�
0: {'loss': 0.54, 'grad_norm': 0.38993035693684813, 'learning_rate': 7.441124832279096e-07, 'epoch': 0.99}
0: ��███▉| 2955/2975 [47:02<00:17,  1.16it/s]
 99%|█████████▉| 2956/2975 [47:03<00:16,  1.16it/s]
 99%|█████████▉| 2957/2975 [47:04<00:15,  1.16it/s]
 99%|█████████▉| 2958/2975 [47:05<00:15,  1.13it/s]
 99%|█████████▉| 2959/2975 [47:06<00:14,  1.14it/s]
 99%|█████████▉| 2960/2975 [47:07<00:13,  1.14it/s]
                                                   

 99%|█████████▉| 2960/2975 [47:07<00:13,  1.14it/s]
100%|█████████▉| 2961/2975 [47:07<00:12,  1.14it/s]
100%|█████████▉| 2962/2975 [47:08<00:11,  1.15it/s]
100%|█████████▉| 2963/2975 [47:09<00:10,  1.15it/s]
100%|█████████▉| 2964/2975 [47:10<00:09,  1.15it/s]
100%|█████████▉| 2965/2975 [47:11<00:08,  1.15it/s]
100%|█████████▉| 2966/2975 [47:12<00:07,  1.15it/s]
100%|█████████▉| 2967/2975 [47:13<00
0: {'loss': 0.5537, 'grad_norm': 0.3854939392122983, 'learning_rate': 7.062158054509446e-07, 'epoch': 1.0}
0: {'train_runtime': 2844.5728, 'train_samples_per_second': 16.734, 'train_steps_per_second': 1.046, 'train_loss': 0.5776566073473762, 'epoch': 1.0}
0: :06,  1.15it/s]
100%|█████████▉| 2968/2975 [47:13<00:06,  1.16it/s]
100%|█████████▉| 2969/2975 [47:14<00:05,  1.15it/s]
100%|█████████▉| 2970/2975 [47:15<00:04,  1.15it/s]
                                                   

100%|█████████▉| 2970/2975 [47:15<00:04,  1.15it/s]
100%|█████████▉| 2971/2975 [47:16<00:03,  1.16it/s]
100%|█████████▉| 2972/2975 [47:17<00:02,  1.16it/s]
100%|█████████▉| 2973/2975 [47:18<00:01,  1.16it/s]
100%|█████████▉| 2974/2975 [47:19<00:00,  1.16it/s]
100%|██████████| 2975/2975 [47:20<00:00,  1.16it/s]
                                                   

100%|██████████| 2975/2975 [47:24<00:00,  1.16it/s]
100%|██████████| 2975/2975 [47:24<00:00,  1.05it/s]
0: [2025-08-20 17:51:49,560] [INFO] [axolotl.train.save_trained_model:246] [PID:195289] [RANK:0] Training completed! Saving trained model to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-1.5B/0.
0: [2025-08-20 17:51:53,190] [INFO] [axolotl.train.save_trained_model:331] [PID:195289] [RANK:0] Model successfully saved to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-1.5B/0