Update README.md
Browse files
README.md
CHANGED
|
@@ -20,16 +20,16 @@ datasets:
|
|
| 20 |
|
| 21 |
|
| 22 |
|
| 23 |
-
author:
|
| 24 |
|
| 25 |
|
| 26 |
-
experiment_name:
|
| 27 |
|
| 28 |
|
| 29 |
-
trackio_url:
|
| 30 |
|
| 31 |
|
| 32 |
-
dataset_repo:
|
| 33 |
|
| 34 |
|
| 35 |
hardware: "GPU (H100/A100)"
|
|
@@ -47,17 +47,17 @@ batch_size: 4
|
|
| 47 |
learning_rate: 2e-4
|
| 48 |
|
| 49 |
|
| 50 |
-
max_epochs:
|
| 51 |
|
| 52 |
|
| 53 |
max_seq_length: 2048
|
| 54 |
|
| 55 |
|
| 56 |
|
| 57 |
-
dataset_size: ~
|
| 58 |
|
| 59 |
|
| 60 |
-
dataset_format:
|
| 61 |
|
| 62 |
|
| 63 |
gradient_accumulation_steps: 16
|
|
@@ -66,15 +66,14 @@ gradient_accumulation_steps: 16
|
|
| 66 |
|
| 67 |
# med-gpt-oss-20b
|
| 68 |
|
| 69 |
-
A fine-tuned version of OpenAI's GPT-OSS-20B model for
|
| 70 |
|
| 71 |
## Model Details
|
| 72 |
|
| 73 |
-
- **Base Model**:
|
| 74 |
- **Model Type**: Causal Language Model
|
| 75 |
-
- **Languages**: English
|
| 76 |
- **License**: Apache 2.0
|
| 77 |
-
- **Fine-tuned**: Yes
|
| 78 |
|
| 79 |
|
| 80 |
## Usage
|
|
@@ -104,7 +103,7 @@ print(tokenizer.decode(output[0], skip_special_tokens=True))
|
|
| 104 |
|
| 105 |
### Training Configuration
|
| 106 |
- **Base Model**: openai/gpt-oss-20b
|
| 107 |
-
- **Dataset**:
|
| 108 |
- **Training Config**: GPT-OSS Configuration
|
| 109 |
- **Trainer Type**: SFTTrainer
|
| 110 |
|
|
@@ -134,9 +133,8 @@ This is a fine-tuned version of the SmolLM3-3B model with the following specific
|
|
| 134 |
## Performance
|
| 135 |
|
| 136 |
The model provides:
|
| 137 |
-
- **
|
| 138 |
-
- **Conversation**:
|
| 139 |
-
- **Multilingual**: Support for English and French
|
| 140 |
|
| 141 |
|
| 142 |
## Limitations
|
|
@@ -150,26 +148,23 @@ The model provides:
|
|
| 150 |
## Training Data
|
| 151 |
|
| 152 |
The model was fine-tuned on:
|
| 153 |
-
- **Dataset**:
|
| 154 |
-
- **Size**: ~
|
| 155 |
-
- **Format**:
|
| 156 |
-
- **Languages**: English
|
| 157 |
|
| 158 |
-
##
|
| 159 |
|
| 160 |
-
|
| 161 |
-
- **Metrics**: Loss, perplexity, and qualitative assessment
|
| 162 |
-
- **Monitoring**: Real-time tracking via Trackio
|
| 163 |
-
- **Validation**: Regular validation during training
|
| 164 |
|
| 165 |
## Citation
|
| 166 |
|
| 167 |
If you use this model in your research, please cite:
|
| 168 |
|
| 169 |
```bibtex
|
| 170 |
-
@misc{
|
| 171 |
title={{med-gpt-oss-20b}},
|
| 172 |
-
author={
|
| 173 |
year={2024},
|
| 174 |
url={https://huggingface.co/Tonic/med-gpt-oss-20b}
|
| 175 |
}
|
|
|
|
| 20 |
|
| 21 |
|
| 22 |
|
| 23 |
+
author: Joseph "Tonic" Pollack
|
| 24 |
|
| 25 |
|
| 26 |
+
experiment_name: exp_20250809_122413
|
| 27 |
|
| 28 |
|
| 29 |
+
trackio_url: https://huggingface.co/spaces/Tonic/track-tonic
|
| 30 |
|
| 31 |
|
| 32 |
+
dataset_repo: FreedomIntelligence/medical-o1-reasoning-SFT
|
| 33 |
|
| 34 |
|
| 35 |
hardware: "GPU (H100/A100)"
|
|
|
|
| 47 |
learning_rate: 2e-4
|
| 48 |
|
| 49 |
|
| 50 |
+
max_epochs: 2
|
| 51 |
|
| 52 |
|
| 53 |
max_seq_length: 2048
|
| 54 |
|
| 55 |
|
| 56 |
|
| 57 |
+
dataset_size: ~20K samples
|
| 58 |
|
| 59 |
|
| 60 |
+
dataset_format: Reasoning
|
| 61 |
|
| 62 |
|
| 63 |
gradient_accumulation_steps: 16
|
|
|
|
| 66 |
|
| 67 |
# med-gpt-oss-20b
|
| 68 |
|
| 69 |
+
A fine-tuned version of OpenAI's GPT-OSS-20B model for medical reasoning and instruction following.
|
| 70 |
|
| 71 |
## Model Details
|
| 72 |
|
| 73 |
+
- **Base Model**: openai/gpt-oss-20B
|
| 74 |
- **Model Type**: Causal Language Model
|
| 75 |
+
- **Languages**: English
|
| 76 |
- **License**: Apache 2.0
|
|
|
|
| 77 |
|
| 78 |
|
| 79 |
## Usage
|
|
|
|
| 103 |
|
| 104 |
### Training Configuration
|
| 105 |
- **Base Model**: openai/gpt-oss-20b
|
| 106 |
+
- **Dataset**: FreedomIntelligence/medical-o1-reasoning-SFT
|
| 107 |
- **Training Config**: GPT-OSS Configuration
|
| 108 |
- **Trainer Type**: SFTTrainer
|
| 109 |
|
|
|
|
| 133 |
## Performance
|
| 134 |
|
| 135 |
The model provides:
|
| 136 |
+
- **Medical Reasoning**: High-quality medical reasoning
|
| 137 |
+
- **Conversation**: Medical instruction following
|
|
|
|
| 138 |
|
| 139 |
|
| 140 |
## Limitations
|
|
|
|
| 148 |
## Training Data
|
| 149 |
|
| 150 |
The model was fine-tuned on:
|
| 151 |
+
- **Dataset**: FreedomIntelligence/medical-o1-reasoning-SFT
|
| 152 |
+
- **Size**: ~20K samples
|
| 153 |
+
- **Format**: reasoning
|
| 154 |
+
- **Languages**: English
|
| 155 |
|
| 156 |
+
## Monitoring
|
| 157 |
|
| 158 |
+

|
|
|
|
|
|
|
|
|
|
| 159 |
|
| 160 |
## Citation
|
| 161 |
|
| 162 |
If you use this model in your research, please cite:
|
| 163 |
|
| 164 |
```bibtex
|
| 165 |
+
@misc{med_gpt_oss_20B,
|
| 166 |
title={{med-gpt-oss-20b}},
|
| 167 |
+
author={Joseph "Tonic" Pollack},
|
| 168 |
year={2024},
|
| 169 |
url={https://huggingface.co/Tonic/med-gpt-oss-20b}
|
| 170 |
}
|