Update config.json
#80
by
Akshay47
- opened
Add unk_token_id
(200019) to text_config to handle unknown tokens properly and prevent tokenization errors.
Add unk_token_id
(200019) to text_config to handle unknown tokens properly and prevent tokenization errors.