|
|
|
|
|
--- |
|
|
|
|
|
base_model: EvolutionaryScale/esmc-300m-2024-12 |
|
|
|
|
|
library_name: peft |
|
|
|
|
|
--- |
|
|
|
|
|
|
|
|
|
|
|
# Model Card for Model-Demo-ESMC-300M |
|
|
## Base model |
|
|
|
|
|
[EvolutionaryScale/esmc-300m-2024-12] |
|
|
|
|
|
## Task description |
|
|
|
|
|
This is a ESMC_300M model for the Protein-level Regression task and the dataset is the Dataset-RASH_HUMAN. |
|
|
|
|
|
## Label meaning |
|
|
|
|
|
Label means mutation effect score of each protein based on deep mutation scanning experiment, ranging from minus infinity to positive infinity where 0 is value of wildtype, larger means higher fitness. |
|
|
|
|
|
|
|
|
|
|
|
## Task type |
|
|
Protein-level Regression |
|
|
|
|
|
## Model input type |
|
|
AA Sequence |
|
|
|
|
|
## LoRA config |
|
|
|
|
|
- **r:** 8 |
|
|
- **lora_dropout:** 0.0 |
|
|
- **lora_alpha:** 16 |
|
|
- **target_modules:** ['layernorm_qkv.1', 'ffn.1', 'out_proj', 'ffn.3'] |
|
|
- **modules_to_save:** ['classifier'] |
|
|
|
|
|
## Training config |
|
|
|
|
|
- **optimizer:** |
|
|
- **class:** AdamW |
|
|
- **betas:** (0.9, 0.98) |
|
|
- **weight_decay:** 0.01 |
|
|
- **learning rate:** 0.0001 |
|
|
- **epoch:** 10 |
|
|
- **batch size:** 32 |
|
|
- **precision:** 16-mixed |
|
|
|
|
|
|
