thumbnail

VISPI: Virtual Staging Pipeline for Single Indoor Panoramic Images

Uzair Shah, Sara Jashari, Muhammad Tukur, Giovanni Pintore, Enrico Gobbetti, Jens Schneider, and Marco Agus

November 2024

Abstract

Taking a 360-degree image is the quickest and most cost-effective way to capture the entire environment around the viewer in a form that can be directly exploited for creating immersive content. In this work, we introduce novel solutions for the virtual staging of indoor environments, supporting automatic emptying, object insertion, and relighting. Our solution, dubbed VISPI (Virtual Staging Pipeline for Single Indoor Panoramic Images), integrates data-driven processing components, that take advantage of the analysis of knowledge learned from massive data collections, within a real-time rendering and editing system, allowing for interactive restaging of indoor scenes. Key components of VISPI include: i) a holistic architecture based on a multi-task vision transformer for extracting geometry, semantic, and material information from a single panoramic image, ii) a lighting model based on spherical Gaussians, iii) a method for lighting estimation from the geometric, semantic, and material signals, and iv) a real-time editing and rendering component. The proposed framework provides an interactive and user-friendly solution for creating immersive visualizations of indoor spaces. We present a preliminary assessment of VISPI using a synthetic dataset - Structured3D - and demonstrate its application in creating restaged indoor scenes.

Reference and download information

Uzair Shah, Sara Jashari, Muhammad Tukur, Giovanni Pintore, Enrico Gobbetti, Jens Schneider, and Marco Agus. VISPI: Virtual Staging Pipeline for Single Indoor Panoramic Images. In STAG: Smart Tools and Applications in Graphics, November 2024. DOI: 10.2312/stag.20241334.

Related multimedia productions

Bibtex citation record

@InProceedings{Shah:2024:VVS,
    author = {Uzair Shah and Sara Jashari and Muhammad Tukur and Giovanni Pintore and Enrico Gobbetti and Jens Schneider and Marco Agus},
    title = {{VISPI}: Virtual Staging Pipeline for Single Indoor Panoramic Images},
    booktitle = {STAG: Smart Tools and Applications in Graphics},
    month = {November},
    year = {2024},
    abstract = {Taking a 360-degree image is the quickest and most cost-effective way to capture the entire environment around the viewer in a form that can be directly exploited for creating immersive content. In this work, we introduce novel solutions for the virtual staging of indoor environments, supporting automatic emptying, object insertion, and relighting. Our solution, dubbed VISPI (Virtual Staging Pipeline for Single Indoor Panoramic Images), integrates data-driven processing components, that take advantage of the analysis of knowledge learned from massive data collections, within a real-time rendering and editing system, allowing for interactive restaging of indoor scenes. Key components of VISPI include: i) a holistic architecture based on a multi-task vision transformer for extracting geometry, semantic, and material information from a single panoramic image, ii) a lighting model based on spherical Gaussians, iii) a method for lighting estimation from the geometric, semantic, and material signals, and iv) a real-time editing and rendering component. The proposed framework provides an interactive and user-friendly solution for creating immersive visualizations of indoor spaces. We present a preliminary assessment of VISPI using a synthetic dataset -- Structured3D -- and demonstrate its application in creating restaged indoor scenes.},
    doi = {10.2312/stag.20241334},
    url = {http://vic.crs4.it/vic/cgi-bin/bib-page.cgi?id='Shah:2024:VVS'},
}