hi!
Clem 🤗 PRO
clem
AI & ML interests
multi-modal, time-series, biology and chemistry
Recent Activity
liked
a model
about 1 hour ago
google/timesfm-2.5-200m-pytorch
liked
a Space
about 2 hours ago
facebook/fairchem_leaderboard
liked
a model
about 5 hours ago
Alibaba-NLP/Tongyi-DeepResearch-30B-A3B
Organizations
replied to
andywu-kby's
post
7 days ago
reacted to
meg's
post with 👍❤️
about 1 month ago
Post
2853
New work from my socially-minded colleagues at Hugging Face, creating some foundations for AI companionship behavior evaluation.
Evaluation Dataset: AI-companionship/INTIMA
Paper: AI-companionship/INTIMA
Work from @giadap , @frimelle , @yjernite .
Evaluation Dataset: AI-companionship/INTIMA
Paper: AI-companionship/INTIMA
Work from @giadap , @frimelle , @yjernite .
- 2 replies
replied to
their
post
about 1 month ago
🤗🤗🤗
replied to
their
post
about 1 month ago
Oliver Godement now on. So many 🇫🇷 people at OpenAI now. We want @romainhuet live next to speak about gpt-oss!
replied to
their
post
about 1 month ago
lots of demos, kind of tuning out
replied to
their
post
about 1 month ago
@gdb challenging Jensen with the leather jacket!
replied to
their
post
about 1 month ago
why do all OAI team members have impressive goals (running marathons, learning new languages,...) 😅😅😅
replied to
their
post
about 1 month ago
Is this a pun to http://hf.co/mistralai LeChat 😅?
replied to
their
post
about 1 month ago
Graph crime haha
replied to
their
post
about 1 month ago
let's see how smart these phd experts are haha
replied to
their
post
about 1 month ago
livestream production through the roof haha
replied to
their
post
about 1 month ago
Music is trippy! Feels like we're welcoming aliens
replied to
their
post
about 1 month ago
Are we expecting some open-source mentions?
replied to
their
post
about 1 month ago
What are the biggest leaks so far?
posted
an
update
about 1 month ago
Post
3603
Thread to gossip during the
openai
GPT-5 livestream: https://www.youtube.com/watch?v=0Uu_VJeVVfo. Feel free to post your impressions below!
openai
GPT-5 livestream: https://www.youtube.com/watch?v=0Uu_VJeVVfo. Feel free to post your impressions below!
-
- 29 replies
reacted to
eliebak's
post with 👀🔥
about 2 months ago
Post
4707
Kimi K2 tech report is full of gems as always. Here are my notes on it:
> MuonClip: Pretty crazy how after 70k the training stabilizes and the QK-clip is basically inactive. There is also no loss in perf with QK-clip which is not trivial at all (at small scale but with aggressive threshold). Also a cool explanation of why muon makes the logit explode in appendix E (tl;dr is that muon makes the singular value of the update matrix higher)
> Sparsity scaling laws to justify their ratio, they have a very solid training infra that allows the model to be trained at this sparsity level, they could have increased even more but as sparsity increases the training becomes less efficient.
> They diminish the number of attention heads to make it more efficient for long context since attention heads are a big bottleneck for long context. They also remove 2 of the 3 "first dense" layers in the dsv3 arch.
With the sparsity and attention heads (divided by 2) they achieve 83% increased flops compared to deepseek v3 arch at 128k.
> Data: Rephrasing is KEY. They do a lot more synthetic data generation and rephrase their corpus to have different styles, for longer documents they do it by chunk. I'm (half) surprised by the fact that ONLY 1 epoch (assuming same number of training tokens I think?) of data rephrased 10 times has better accuracy than 10 epochs of the same data rephrased once.
> They do rewriting for Math and Knowledge, for Math they apply the ShallowMath recipe and instruct the model to rephrase in a "learning note" style
> They talk about diversity and probably have some internal stuff/eval to test that, as always still a bit unclear for me how to properly measure that.
The infra is also very nice, quick summary:
> PP=16 (1F1B schedule, a bit custom), EP=16, zero1
> No FP8 computation but for storage of specific layers, selective recomputation for inexpensive block, activation offloading to CPU
> MuonClip: Pretty crazy how after 70k the training stabilizes and the QK-clip is basically inactive. There is also no loss in perf with QK-clip which is not trivial at all (at small scale but with aggressive threshold). Also a cool explanation of why muon makes the logit explode in appendix E (tl;dr is that muon makes the singular value of the update matrix higher)
> Sparsity scaling laws to justify their ratio, they have a very solid training infra that allows the model to be trained at this sparsity level, they could have increased even more but as sparsity increases the training becomes less efficient.
> They diminish the number of attention heads to make it more efficient for long context since attention heads are a big bottleneck for long context. They also remove 2 of the 3 "first dense" layers in the dsv3 arch.
With the sparsity and attention heads (divided by 2) they achieve 83% increased flops compared to deepseek v3 arch at 128k.
> Data: Rephrasing is KEY. They do a lot more synthetic data generation and rephrase their corpus to have different styles, for longer documents they do it by chunk. I'm (half) surprised by the fact that ONLY 1 epoch (assuming same number of training tokens I think?) of data rephrased 10 times has better accuracy than 10 epochs of the same data rephrased once.
> They do rewriting for Math and Knowledge, for Math they apply the ShallowMath recipe and instruct the model to rephrase in a "learning note" style
> They talk about diversity and probably have some internal stuff/eval to test that, as always still a bit unclear for me how to properly measure that.
The infra is also very nice, quick summary:
> PP=16 (1F1B schedule, a bit custom), EP=16, zero1
> No FP8 computation but for storage of specific layers, selective recomputation for inexpensive block, activation offloading to CPU
reacted to
cgeorgiaw's
post with 🚀
3 months ago
Post
2706
Huge new bio datasets just dropped!!!
Check out them out @
ginkgo-datapoints
Read the blog for more info: https://huggingface.co/blog/cgeorgiaw/gdp
Check out them out @
ginkgo-datapoints
Read the blog for more info: https://huggingface.co/blog/cgeorgiaw/gdp
- 1 reply
reacted to
MonsterMMORPG's
post with 🤗
3 months ago
Post
3691
WAN 2.1 FusionX + Self Forcing LoRA are the New Best of Local Video Generation with Only 8 Steps + FLUX Upscaling Guide : https://www.youtube.com/watch?v=Xbn93GRQKsQ
Tutorial : https://www.youtube.com/watch?v=Xbn93GRQKsQ
Video Chapters
0:00 Introduction to the New FusionX Video Model & FLUX Upscaling
0:30 One-Click Presets & The SwarmUI Model Downloader Explained
1:07 Achieving Hyper-Realism with the FLUX 2x Latent Upscale Preset
1:58 How to Download & Install the SwarmUI Model Downloader
2:49 Downloading Full Models vs. Downloading Just The LoRAs
3:48 Final Setup: Updating SwarmUI & Importing The New Presets
4:32 Generating a Video: Applying the FusionX Image-to-Video Preset
5:03 Critical Step: Correcting The Model's Native Resolution Metadata
5:55 Finalizing Image-to-Video Settings (Frame Count & RIFE Interpolation)
6:49 Troubleshooting Performance: Identifying Low GPU Usage & Shared VRAM Bug
8:35 The Solution: Disabling Sage Attention for Image-to-Video Models
10:02 Final Result: Showcasing The Amazing HD Quality Animation
10:40 How to Use the FusionX Text-to-Video Model with Presets
11:49 Text-to-Video Result & Quality Comparison
12:08 How to Use the FusionX LoRA with the Base Wan 2.1 Model
13:07 FLUX Tutorial: Downloading The Required Upscaler & Face Models
13:48 Generating a High-Quality Image with The Official FLUX Preset
14:50 Using Automatic Face Segmentation & Inpainting with FLUX
16:05 The Ultimate Upgrade: Applying The FLUX 2x Latent Upscaler Preset
16:32 Final Result: Comparing Standard vs. 2x Upscaled Image Quality
16:50 Outro & Sneak Peek of The New Ultimate Video Processing App
Tutorial : https://www.youtube.com/watch?v=Xbn93GRQKsQ
Video Chapters
0:00 Introduction to the New FusionX Video Model & FLUX Upscaling
0:30 One-Click Presets & The SwarmUI Model Downloader Explained
1:07 Achieving Hyper-Realism with the FLUX 2x Latent Upscale Preset
1:58 How to Download & Install the SwarmUI Model Downloader
2:49 Downloading Full Models vs. Downloading Just The LoRAs
3:48 Final Setup: Updating SwarmUI & Importing The New Presets
4:32 Generating a Video: Applying the FusionX Image-to-Video Preset
5:03 Critical Step: Correcting The Model's Native Resolution Metadata
5:55 Finalizing Image-to-Video Settings (Frame Count & RIFE Interpolation)
6:49 Troubleshooting Performance: Identifying Low GPU Usage & Shared VRAM Bug
8:35 The Solution: Disabling Sage Attention for Image-to-Video Models
10:02 Final Result: Showcasing The Amazing HD Quality Animation
10:40 How to Use the FusionX Text-to-Video Model with Presets
11:49 Text-to-Video Result & Quality Comparison
12:08 How to Use the FusionX LoRA with the Base Wan 2.1 Model
13:07 FLUX Tutorial: Downloading The Required Upscaler & Face Models
13:48 Generating a High-Quality Image with The Official FLUX Preset
14:50 Using Automatic Face Segmentation & Inpainting with FLUX
16:05 The Ultimate Upgrade: Applying The FLUX 2x Latent Upscaler Preset
16:32 Final Result: Comparing Standard vs. 2x Upscaled Image Quality
16:50 Outro & Sneak Peek of The New Ultimate Video Processing App
- 6 replies