Object-based illumination transferring and rendering for applications of mixed reality

Xu, D., Li, Z. and Cao, Q. (2021) Object-based illumination transferring and rendering for applications of mixed reality. Visual Computer, (doi: 10.1007/s00371-021-02292-2) (Early Online Publication)

[img] Text
249953.pdf - Published Version
Available under License Creative Commons Attribution.

6MB

Abstract

In applications of augmented reality or mixed reality, rendering virtual objects in real scenes with consistent illumination is crucial for realistic visualization experiences. Prior learning-based methods reported in the literature usually attempt to reconstruct complicated high dynamic range environment maps from limited input, and rely on a separate rendering pipeline to light up the virtual object. In this paper, an object-based illumination transferring and rendering algorithm is proposed to tackle this problem within a unified framework. Given a single low dynamic range image, instead of recovering lighting environment of the entire scene, the proposed algorithm directly infers the relit virtual object. It is achieved by transferring implicit illumination features which are extracted from its nearby planar surfaces. A generative adversarial network is adopted in the proposed algorithm for implicit illumination features extraction and transferring. Compared to previous works in the literature, the proposed algorithm is more robust, as it is able to efficiently recover spatially varying illumination in both indoor and outdoor scene environments. Experiments have been conducted. It is observed that notable experiment results and comparison outcomes have been obtained quantitatively and qualitatively by the proposed algorithm in different environments. It shows the effectiveness and robustness for realistic virtual object insertion and improved realism.

Item Type:Articles
Additional Information:This work was supported in part by the National Natural Science Foundation of China under Grant 61801391, in part by Open Project Program of the National Laboratory of Pattern Recognition under Grant 202000025, in part by China Postdoctoral Science Foundation under Grant 2018M631193.
Status:Early Online Publication
Refereed:Yes
Glasgow Author(s) Enlighten ID:Cao, Dr Qi
Authors: Xu, D., Li, Z., and Cao, Q.
College/School:College of Science and Engineering > School of Computing Science
Journal Name:Visual Computer
Publisher:Springer
ISSN:0178-2789
ISSN (Online):1432-2315
Published Online:07 October 2021
Copyright Holders:Copyright © 2021 The Authors
First Published:First published in Visual Computer 2021
Publisher Policy:Reproduced under a Creative Commons License

University Staff: Request a correction | Enlighten Editors: Update this record