ScoreLiDAR: Distilling Diffusion Models to Efficient 3D LiDAR Scene Completion

Abstract

Diffusion models have been applied to 3D LiDAR scene completion due to their strong training stability and high completion quality. However, the slow sampling speed limits the practical application of diffusion-based scene completion models since autonomous vehicles require an efficient perception of surrounding environments. This paper proposes a novel distillation method tailored for 3D Li- DAR scene completion models, dubbed ScoreLiDAR, which achieves efficient yet high-quality scene completion. Score- LiDAR enables the distilled model to sample in significantly fewer steps after distillation. To improve completion quality, we also introduce a novel Structural Loss, which encourages the distilled model to capture the geometric structure of the 3D LiDAR scene. The loss contains a scene-wise term constraining the holistic structure and a point-wise term constraining the key landmark points and their relative configuration. Extensive experiments demonstrate that ScoreLiDAR significantly accelerates the completion time from 30.55 to 5.37 seconds per frame (>5x) on SemanticKITTI and achieves superior performance compared to state-of-the-art 3D LiDAR scene completion models. Our model and code are publicly available on https: //github.com/happyw1nd/ScoreLiDAR.

Repo Info

This repository stores checkpoints for the ScoreLiDAR paper. For more information and detailed usage of the models, please refer to the official GitHub repository.

Sample Usage

To get started with ScoreLiDAR, follow these steps to set up the environment, download the necessary weights, and run an inference example.

Clone the repository:

git clone https://github.com/happyw1nd/ScoreLiDAR
cd ScoreLiDAR

Install the environment:

sudo apt install build-essential python3-dev libopenblas-dev
pip3 install -r requirements.txt
pip3 install -U MinkowskiEngine==0.5.4 --install-option="--blas=openblas" -v --no-deps
pip3 install -U -e .

Download model checkpoints: Download the required checkpoint files from this Hugging Face repository and place them in the checkpoints/ directory within your cloned ScoreLiDAR repository.

huggingface-cli download happywind/ScoreLiDAR ScoreLiDAR_diff_net.ckpt --local-dir checkpoints
huggingface-cli download happywind/ScoreLiDAR refine_net.ckpt --local-dir checkpoints

Run inference: The inference script will read .ply files from the scorelidar/Datasets/test/ directory. A sample .ply file is typically included in the repository for testing. Results will be saved under scorelidar/results/.
```
python3 tools/diff_completion_pipeline.py --denoising_steps 8 --cond_weight 3.5
```
Visualize results (optional): You can visualize the generated point cloud files using the provided script:
```
python3 vis_pcd.py -p <path_to_.ply_file>
```

Citation

If you find our paper useful or relevant to your research, please kindly cite our papers:

@article{zhang2024distillingdiffusionmodels,
    title={Distilling Diffusion Models to Efficient 3D LiDAR Scene Completion}, 
    author={Shengyuan Zhang and An Zhao and Ling Yang and Zejian Li and Chenye Meng and Haoran Xu and Tianrun Chen and AnYang Wei and Perry Pengyun GU and Lingyun Sun,
    journal={arXiv:2412.03515},
    year={2024}
}