RT-MPINet

Real-Time View Synthesis with Multiplane Image Network using Multimodal Supervision (RT-MPINet)

We present a real-time multiplane image (MPI) network. Unlike existing MPI based approaches that often rely on a separate depth estimation network to guide the network for estimating MPI parameters, our method directly predicts these parameters from a single RGB image. To guide the network we present a multimodal training strategy utilizing joint supervision from view synthesis and depth estimation losses. More details can be found in the paper.

Please head to the Project Page to see supplementary materials and Full Code

Acknowledgements

We thank the authors of AdaMPI for their implementation of the homography renderer which has been used in this codebase under ./utils directory
We tank the author of Deepview renderer template, which was used in our project page.

Citation

If you use our work please use following citation:

@inproceedings{gond2025rtmpi,
  title={Real-Time View Synthesis with Multiplane Image Network using Multimodal Supervision},
  author={Gond, Manu and Shamshirgarha, Mohammadreza and Zerman, Emin and Knorr, Sebastian and Sj{\"o}str{\"o}m, M{\aa}rten},
  booktitle={2025 IEEE 27th International Workshop on Multimedia Signal Processing (MMSP)},
  pages={},
  year={2025},
  organization={IEEE}
}