Spaces:

nanotron
/

ultrascale-playbook

Running

App Files Files Community

115

ultrascale-playbook / dist /assets /images

Ctrl+K

9 contributors

History: 32 commits

lvwerra HF Staff

last-edits (#110)

96abf36 verified 2 months ago

256px-PDF.png

1.25 kB
LFS

final final final typos (#72) 5 months ago
5Dparallelism_8Bmemoryusage.svg

122 kB

small_optims (#54) 5 months ago
5d_full.svg

76.9 kB

small_optims (#54) 5 months ago
5d_nutshell_cp.svg

37.3 kB

fix optims (#56) 5 months ago
5d_nutshell_ep.svg

41.9 kB

fix optims (#56) 5 months ago
5d_nutshell_tp_sp.svg

41.8 kB

Rename dist/assets/images/5D_nutshell_tp_sp.svg to dist/assets/images/5d_nutshell_tp_sp.svg 5 months ago
IMG_7537D08D7F41-1.jpeg

92.6 kB
LFS

small_optims (#54) 5 months ago
IMG_A95961668B3F-1.jpeg

210 kB
LFS

small_optims (#54) 5 months ago
IMG_C4260C5C58DC-1.jpeg

139 kB
LFS

small_optims (#54) 5 months ago
IMG_DA188FF29F45-1.jpeg

154 kB
LFS

small_optims (#54) 5 months ago
a0_all_gather.gif

172 kB

small_optims (#54) 5 months ago
a0_barrier.png

9.73 kB
LFS

small_optims (#54) 5 months ago
a0_broadcast.png

15 kB
LFS

small_optims (#54) 5 months ago
a0_gather_allgather.png

23.1 kB
LFS

small_optims (#54) 5 months ago
a0_general.png

12.9 kB
LFS

small_optims (#54) 5 months ago
a0_reduce_allreduce.png

25.9 kB
LFS

small_optims (#54) 5 months ago
a0_reduce_scatter.gif

131 kB

small_optims (#54) 5 months ago
a0_scatter_reducescatter.png

23.6 kB
LFS

small_optims (#54) 5 months ago
a1_kernels.png

23.6 kB
LFS

more stuff (#59) 5 months ago
a1_ncu.png

75.2 kB
LFS

more stuff (#59) 5 months ago
a1_profile_trace.png

24.2 kB
LFS

more stuff (#59) 5 months ago
activation_recomputation.js

5.62 kB

small_optims (#54) 5 months ago
activation_recomputation.png

13.7 kB
LFS

small_optims (#54) 5 months ago
activation_recomputation.svg

39.6 kB

small_optims (#54) 5 months ago
activationrecomputation_memory.svg

42.1 kB

small_optims (#54) 5 months ago
conclusion_llama3_parallelism.png

112 kB
LFS

small_optims (#54) 5 months ago
cp_70bmemoryusage.svg

51.5 kB

small_optims (#54) 5 months ago
cp_8Bmemoryusage.svg

48.4 kB

small_optims (#54) 5 months ago
cp_attnmask.svg

75.3 kB

small_optims (#54) 5 months ago
cp_memoryusage.svg

48.4 kB

small_optims (#54) 5 months ago
cp_overlap_all2all.svg

33.5 kB

small_optims (#54) 5 months ago
cp_overlap_allgather.svg

27.4 kB

small_optims (#54) 5 months ago
cp_zigzagmask.svg

75.3 kB

small_optims (#54) 5 months ago
diving_primergpu.svg

98.9 kB

small_optims (#54) 5 months ago
diving_primergpu2.svg

107 kB

small_optims (#54) 5 months ago
dp_diagram.png

25 kB
LFS

small_optims (#54) 5 months ago
dp_ourjourney_memoryusage.svg

47.6 kB

small_optims (#54) 5 months ago
dp_overlap1.svg

35 kB

small_optims (#54) 5 months ago
dp_overlap2.svg

46.7 kB

small_optims (#54) 5 months ago
dp_overlap3.svg

36.4 kB

small_optims (#54) 5 months ago
dp_scaling.svg

46.9 kB

small_optims (#54) 5 months ago
dp_zero1.gif

69.9 kB

small_optims (#54) 5 months ago
dp_zero1_overlap.svg

42.2 kB

small_optims (#54) 5 months ago
dp_zero2.gif

71.9 kB

small_optims (#54) 5 months ago
dp_zero2_overlap.svg

42.2 kB

small_optims (#54) 5 months ago
dp_zero3_bwd.svg

118 kB

small_optims (#54) 5 months ago
dp_zero3_fwd.svg

110 kB

small_optims (#54) 5 months ago
dp_zero3_overlap.svg

46 kB

small_optims (#54) 5 months ago
ep_moe.png

27.1 kB
LFS

small_optims (#54) 5 months ago
ep_schema.png

42.1 kB
LFS

small_optims (#54) 5 months ago
first_steps_memory_profile.js

5.69 kB

small_optims (#54) 5 months ago
first_steps_memory_profile.svg

541 kB

small_optims (#54) 5 months ago
first_steps_simple_training.js

5.63 kB

small_optims (#54) 5 months ago
first_steps_simple_training.svg

38.3 kB

small_optims (#54) 5 months ago
flashattn.png

19.4 kB
LFS

small_optims (#54) 5 months ago
flashattn2.png

107 kB
LFS

small_optims (#54) 5 months ago
fp8_diagram.png

29.9 kB
LFS

small_optims (#54) 5 months ago
fp8_divergence.png

65.7 kB
LFS

small_optims (#54) 5 months ago
fp8_training_loss_curves.svg

101 kB

small_optims (#54) 5 months ago
fused_kernels1.png

37.8 kB
LFS

small_optims (#54) 5 months ago
fused_kernels2.png

21.3 kB
LFS

small_optims (#54) 5 months ago
gradaccumulation_diag.png

23.3 kB
LFS

small_optims (#54) 5 months ago
image 1.png

15 kB
LFS

small_optims (#54) 5 months ago
image 10.png

227 kB
LFS

small_optims (#54) 5 months ago
image 11.png

180 kB
LFS

small_optims (#54) 5 months ago
image 12.png

282 kB
LFS

small_optims (#54) 5 months ago
image 13.png

255 kB
LFS

small_optims (#54) 5 months ago
image 14.png

230 kB
LFS

small_optims (#54) 5 months ago
image 15.png

177 kB
LFS

small_optims (#54) 5 months ago
image 16.png

166 kB
LFS

small_optims (#54) 5 months ago
image 17.png

482 kB
LFS

small_optims (#54) 5 months ago
image 2.png

25.9 kB
LFS

small_optims (#54) 5 months ago
image 3.png

23.1 kB
LFS

small_optims (#54) 5 months ago
image 4.png

23.6 kB
LFS

small_optims (#54) 5 months ago
image 5.png

9.73 kB
LFS

small_optims (#54) 5 months ago
image 6.png

6.71 kB
LFS

small_optims (#54) 5 months ago
image 7.png

6.9 kB
LFS

small_optims (#54) 5 months ago
image 8.png

11.2 kB
LFS

small_optims (#54) 5 months ago
image 9.png

60.3 kB
LFS

small_optims (#54) 5 months ago
image.png

12.9 kB
LFS

small_optims (#54) 5 months ago
memory_profile.svg

542 kB

small_optims (#54) 5 months ago
memorycoalescing.png

94.1 kB
LFS

last-edits (#110) 2 months ago
memorycoalescing2.png

17 kB
LFS

small_optims (#54) 5 months ago
memorycoalescing3.png

24.8 kB
LFS

small_optims (#54) 5 months ago
memorycoalescing4.png

18.9 kB
LFS

small_optims (#54) 5 months ago
memorycoalescing5.png

16.9 kB
LFS

small_optims (#54) 5 months ago
memusage_activations.svg

50.7 kB

small_optims (#54) 5 months ago
mixedprecision.png

50.6 kB
LFS

small_optims (#54) 5 months ago
mixedprecision_2.png

33 kB
LFS

small_optims (#54) 5 months ago
placeholder.png

6.05 kB
LFS

small_optims (#54) 5 months ago
pp_1f1b.svg

63.6 kB

small_optims (#54) 5 months ago
pp_1f1b_interleaved.svg

149 kB

small_optims (#54) 5 months ago
pp_1f1b_scaling.png

72.6 kB
LFS

small_optims (#54) 5 months ago
pp_afab.svg

31.3 kB

small_optims (#54) 5 months ago
pp_afab2.svg

66.4 kB

small_optims (#54) 5 months ago
pp_bubblesize.png

34.3 kB
LFS

small_optims (#54) 5 months ago
pp_comm_bandwidth.svg

32 kB

small_optims (#54) 5 months ago
pp_llama3.1_schedule.png

42.2 kB
LFS

small_optims (#54) 5 months ago
pp_memoryusage.svg

33.3 kB

small_optims (#54) 5 months ago
pp_zerobubble_compgraph.png

13.1 kB
LFS

small_optims (#54) 5 months ago
pp_zerobubble_dualpipe.png

52.5 kB
LFS

small_optims (#54) 5 months ago
pp_zerobubble_ppschedule.png

34.9 kB
LFS

small_optims (#54) 5 months ago
predict_memory_tool.png

26.5 kB
LFS

embed predict_memory (#47) 5 months ago
profile_trace_annotated.png

323 kB
LFS

small_optims (#54) 5 months ago
ring-attention.gif

200 kB

small_optims (#54) 5 months ago
rotation.jpeg

60.7 kB
LFS

small_optims (#54) 5 months ago
rotation_speed.jpeg

128 kB
LFS

small_optims (#54) 5 months ago
sign-mantissa-exponent.svg

10.7 kB

more fixes thom (#61) 5 months ago
threadcoarsening.png

18.7 kB
LFS

small_optims (#54) 5 months ago
tiling.png

4.42 kB
LFS

small_optims (#54) 5 months ago
torch-compile-triton-kernel.png

35.5 kB
LFS

more thom (#57) 5 months ago
torch-compile-triton.png

30.9 kB
LFS

more thom (#57) 5 months ago
tp_diagram.png

7.49 kB
LFS

small_optims (#54) 5 months ago
tp_diagram.svg

11.6 kB

fix-figures (#55) 5 months ago
tp_diagram2.png

89.8 kB
LFS

small_optims (#54) 5 months ago
tp_diagram3.png

86.6 kB
LFS

small_optims (#54) 5 months ago
tp_diagram4.png

129 kB
LFS

fix-figures (#55) 5 months ago
tp_full_diagram.png

34.6 kB
LFS

small_optims (#54) 5 months ago
tp_memoryusage.svg

40.7 kB

small_optims (#54) 5 months ago
tp_overlap.svg

28.5 kB

small_optims (#54) 5 months ago
tp_scaling.svg

43.1 kB

small_optims (#54) 5 months ago
tp_sp_diagram.png

39.9 kB
LFS

small_optims (#54) 5 months ago
tp_sp_diagram_zoomed.png

90 kB
LFS

small_optims (#54) 5 months ago
tp_sp_memoryusage.svg

44.6 kB

small_optims (#54) 5 months ago
tp_sp_overlap.svg

39.6 kB

small_optims (#54) 5 months ago
tp_sp_scaling.svg

41.3 kB

small_optims (#54) 5 months ago
ultra-cheatsheet.svg

1.2 MB

fixes (#68) 5 months ago
what_we_learnt_heatmap.svg

124 kB

small_optims (#54) 5 months ago
what_we_learnt_parallel_coordinates.html

4.65 MB

notes (#5) 5 months ago
what_we_learnt_parallel_coordinates.svg

5.1 MB

small_optims (#54) 5 months ago
zero3_memoryusage.svg

49.6 kB

small_optims (#54) 5 months ago
zero_memory.svg

267 kB

small_optims (#54) 5 months ago