Rename tokenizer_config.json to tokenizer/tokenizer_config.json 2e94e57 verified Girinath11 commited on 11 days ago
Rename token_frequencies.pkl to tokenizer/token_frequencies.pkl f79d2a8 verified Girinath11 commited on 11 days ago