Spaces:

Dovakiins
/

qwerrwe

Build error

App Files Files Community

qwerrwe / configs

Ctrl+K

Ctrl+K

100 contributors

History: 10 commits

winglian's picture

deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches

d1aed4c over 2 years ago

cerebras_1_3B_alpaca.yml
906 Bytes

deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches over 2 years ago
llama_65B_alpaca.yml
931 Bytes

deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches over 2 years ago
llama_7B_alpaca.yml
929 Bytes

deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches over 2 years ago
pythia_1_2B_alpaca.yml
974 Bytes

deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches over 2 years ago