DiLoCoX: A Low-Communication Large-Scale Training Framework for Decentralized Cluster
Paper
•
2506.21263
•
Published
•
4
•
1
Arthur Douillard
ArthurDouillard
AI & ML interests
Continual Learning, Computer Vision, Transformers
Organizations
None yet
commented
a
paper
6 months ago
commented
3
papers
10 months ago
Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
Paper
•
2503.09799
•
Published
•
15
•
2
Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch
Paper
•
2501.18512
•
Published
•
29
•
7
Eager Updates For Overlapped Communication and Computation in DiLoCo
Paper
•
2502.12996
•
Published
•
7
•
2