metadata
library_name: keras-hub
This is a ESM model uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
This model is related to a MaskedPLM task.
Model config:
- name: esm_backbone
- trainable: True
- vocabulary_size: 33
- num_layers: 6
- num_heads: 20
- hidden_dim: 320
- intermediate_dim: 1280
- dropout: 0.0
- max_wavelength: 10000
- use_bias: True
- activation: gelu
- layer_norm_eps: 1e-05
- use_pre_layer_norm: False
- position_embedding_type: rotary
- max_sequence_length: 1026
- pad_token_id: 1
This model card has been generated automatically and should be completed by the model author. See Model Cards documentation for more information.