Spaces:
Build error
Build error
File size: 272 Bytes
00137dc |
1 2 3 4 |
# For 8-bit loading (requires bitsandbytes and accelerate) model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_NAME, load_in_8bit=True, device_map="auto") # For 4-bit loading # model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_NAME, load_in_4bit=True, device_map="auto") |