greedy-intersection / tokenizer_config.json
kibrq's picture
Update model
8f70f91
raw
history blame contribute delete
97 Bytes
{
"eos_token": "<s>",
"pad_token": "<pad>",
"tokenizer_class": "PreTrainedTokenizerFast"
}