Model save

README.md

@@ -19,7 +19,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [bigcode/starcoder2-3b](https://huggingface.co/bigcode/starcoder2-3b) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4160
+- Loss: 0.4791
 
 ## Model description
 
@@ -38,22 +38,21 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-05
+- learning_rate: 3e-05
 - train_batch_size: 2
 - eval_batch_size: 2
 - seed: 42
 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
-- num_epochs: 3
+- num_epochs: 2
 - mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.3303        | 1.0   | 60   | 0.4534          |
-| 1.2079        | 2.0   | 120  | 0.4264          |
-| 1.2022        | 3.0   | 180  | 0.4160          |
+| 1.9751        | 1.0   | 60   | 0.4862          |
+| 1.4621        | 2.0   | 120  | 0.4791          |
 
 
 ### Framework versions

tokenizer_config.json

@@ -350,9 +350,16 @@
   "clean_up_tokenization_spaces": true,
   "eos_token": "<|endoftext|>",
   "extra_special_tokens": {},
+  "max_length": 1024,
   "model_max_length": 1000000000000000019884624838656,
+  "pad_to_multiple_of": null,
   "pad_token": "<|endoftext|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "stride": 0,
   "tokenizer_class": "GPT2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
   "unk_token": "<|endoftext|>",
   "vocab_size": 49152
 }