guillermoruiz
/

bilma_US

Model card Files Files and versions

guillermoruiz commited on Apr 2, 2024

Commit

fdca105

·

verified ·

1 Parent(s): 05c92b9

Upload tokenizer

Files changed (1) hide show

tokenizer.json +3 -0

tokenizer.json CHANGED Viewed

@@ -64,6 +64,9 @@
       {
         "type": "NFD"
       },
       {
         "type": "StripAccents"
       },

       {
         "type": "NFD"
       },
+      {
+        "type": "Lowercase"
+      },
       {
         "type": "StripAccents"
       },