Projects per year
Abstract
While current style transfer models have achieved impressive results for the application of artistic style to generic images, they face challenges in achieving photorealistic performances on indoor scenes, especially the ones represented by panoramic images. Moreover, existing models overlook the unique characteristics of indoor panoramas, which possess particular geometry and semantic properties. To address these limitations, we propose the first geometry-aware and shading-independent, photorealistic and semantic style transfer method for indoor panoramic scenes. Our approach extends semantic-aware generative adversarial architecture capabilities by introducing two novel strategies to account the geometric characteristics of indoor scenes and to enhance performance. Firstly, we incorporate strong geometry losses that use layout and depth inference at the training stage to enforce shape consistency between generated and ground truth scenes. Secondly, we apply a shading decomposition scheme to extract the albedo and normalized shading signal from the original scenes, and we apply the style transfer on albedo instead of full RGB images, thereby preventing shading-related bleeding issues. On top of that, we apply super-resolution to the resulting scenes to improve image quality and yield fine details. We evaluate our model's performance on public domain synthetic data sets. Our proposed architecture outperforms state-of-the-art style transfer models in terms of perceptual and accuracy metrics, achieving a 26.76% lower ArtFID, a 6.95% higher PSNR, and a 25.23% higher SSIM. The visual results show that our method is effective in producing realistic and visually pleasing indoor scenes.
Original language | English |
---|---|
Title of host publication | Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 1545-1556 |
Number of pages | 12 |
ISBN (Electronic) | 9798350307443 |
DOIs | |
Publication status | Published - 2023 |
Event | 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 - Paris, France Duration: 2 Oct 2023 → 6 Oct 2023 |
Publication series
Name | Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 |
---|
Conference
Conference | 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 |
---|---|
Country/Territory | France |
City | Paris |
Period | 2/10/23 → 6/10/23 |
Keywords
- generative adversarial networks
- indoor panoramas
- style transfer
Fingerprint
Dive into the research topics of 'PanoStyle: Semantic, Geometry-Aware and Shading Independent Photorealistic Style Transfer for Indoor Panoramic Scenes'. Together they form a unique fingerprint.Projects
- 1 Active
-
EX-QNRF-NPRPS-16: AIN2: Artificial Intelligence for Indoor Digital Twins
Agus, M. (Lead Principal Investigator), Tukur, M. (Graduate Student), Engineer-1 (Engineer), Assistant-1, R. (Research Assistant), Gobbetti, D. E. (Principal Investigator), Fetais, D. N. (Principal Investigator), Tukur, M. M. (Research Assistant), Jashari, S. (Graduate Student) & Atiyah, M. N. (Consultant)
15/11/22 → 15/11/25
Project: Basic Research