Article
Ulysses Sequence Parallelism: Training with Million-Token Contexts
kashif, stas
• • 27Stas Bekman
stas
AI & ML interests
Toolmaker. Software creator, optimizer and harmonizer.
Makes things work and fly at Snowflake AI Research
Training LLM/RAG/Generative AI/Machine Learning/Scalability
Organizations
published an article 2 months ago
published an article almost 2 years ago
Article
From DeepSpeed to FSDP and Back Again with Hugging Face Accelerate
- +2
mirinflim, aldopareja, muellerzr, stas
• • 62 published an article over 2 years ago
Article
Introducing IDEFICS: An Open Reproduction of State-of-the-art Visual Langage Model
- +9
HugoLaurencon, davanstrien, stas, Leyo, SaulLu, TimeRobber, skaramcheti, aps, giadap, yjernite, VictorSanh
• • 37 published an article over 3 years ago
Article
Incredibly Fast BLOOM Inference with DeepSpeed and Accelerate
stas, sgugger
• • 1 published an article almost 4 years ago
Article
The Technology Behind BLOOM Training
stas
• • 45 published an article over 5 years ago
Article
Fit More and Train Faster With ZeRO via DeepSpeed and FairScale
stas
• • 5 published an article over 5 years ago
Article
Porting fairseq wmt19 translation system to transformers
stas
• • 2 published an article over 5 years ago
Article
Porting fairseq wmt19 translation system to transformers
stas
• • 2