Interactive exploration of augmented aerial scenes with free-viewpoint image generation from pre-rendered images

Exploration of the real world by flying in its virtual simulation is one of the most important applications of augmented virtuality (AV) due to the popularization of Virtual Globe applications [1] (e.g., Google Earth) as well as large-scale three-dimensional (3D) reconstruction approaches. To improve the appearance of the automatically constructed 3D models, free-viewpoint (sometimes referred to as novel- or arbitrary-viewpoint) image generation using a hybrid of image- and model-based rendering [2] provides promising solutions. In some applications that allow interactive virtual exploration of the real world, virtual objects can be superimposed for visualization of disaster areas or non-existent buildings [5]. In such applications, the virtual objects have to be rendered frame-by-frame. Real-time rendering approaches have been studied for photometric registration of augmented reality (AR) [3] and have become one of the most important research fields in MR. The goal of these research has always been to achieve the same quality as offline rendering using high-quality global illumination (GI), but it has not been achieved thus far.

In IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2013)