ReSWD: ReSTIR'd, not shaken. Combining Reservoir Sampling and Sliced Wasserstein Distance for Variance Reduction
Mark Boss
Abstract
Reservoir SWD reduces variance in Sliced Wasserstein Distance, improving gradient stability and performance in vision and graphics tasks.
Distribution matching is central to many vision and graphics tasks, where the widely used Wasserstein distance is too costly to compute for high dimensional distributions. The Sliced Wasserstein Distance (SWD) offers a scalable alternative, yet its Monte Carlo estimator suffers from high variance, resulting in noisy gradients and slow convergence. We introduce Reservoir SWD (ReSWD), which integrates Weighted Reservoir Sampling into SWD to adaptively retain informative projection directions in optimization steps, resulting in stable gradients while remaining unbiased. Experiments on synthetic benchmarks and real-world tasks such as color correction and diffusion guidance show that ReSWD consistently outperforms standard SWD and other variance reduction baselines. Project page: https://reservoirswd.github.io/
Community
Distribution matching is central to many vision and graphics tasks, where the widely used Wasserstein distance is too costly to compute for high dimensional distributions. The Sliced Wasserstein Distance (SWD) offers a scalable alternative, yet its Monte Carlo estimator suffers from high variance, resulting in noisy gradients and slow convergence. We introduce Reservoir SWD (ReSWD), which integrates Weighted Reservoir Sampling into SWD to adaptively retain informative projection directions in optimization steps, resulting in stable gradients while remaining unbiased. Experiments on synthetic benchmarks and real-world tasks such as color correction and diffusion guidance show that ReSWD consistently outperforms standard SWD and other variance reduction baselines.
Project page: https://reservoirswd.github.io/
HF Demo: https://huggingface.co/spaces/stabilityai/ReSWD
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Efficient Sliced Wasserstein Distance Computation via Adaptive Bayesian Optimization (2025)
- Fast Estimation of Wasserstein Distances via Regression on Sliced Wasserstein Distances (2025)
- Aligning Visual Foundation Encoders to Tokenizers for Diffusion Models (2025)
- HiGS: History-Guided Sampling for Plug-and-Play Enhancement of Diffusion Models (2025)
- ReTrack: Data Unlearning in Diffusion Models through Redirecting the Denoising Trajectory (2025)
- CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Exploration (2025)
- SlimDiff: Training-Free, Activation-Guided Hands-free Slimming of Diffusion Models (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper