BiFuse: Monocular 360˚ Depth Estimation via Bi-Projection Fusion

Fu-En Wang*, Yu-Hsuan Yeh*, Min Sun, Wei-Chen Chiu, Yi-Hsuan Tsai (*indicate equal contribution)

Dept. of Electrical Engineering, National Tsing Hua University, Taiwan
Dept. of Computer Science, National Chiao Tung University, Taiwan
ASUS AICS Department
NEC Labs America

Abstract

Depth estimation from a monocular 360˚ image is an emerging problem that gains popularity due to the availability of consumer-level 360˚ cameras and the complete surrounding sensing capability. While the standard of 360˚ imaging is under rapid development, we propose to predict the depth map of a monocular 360˚ image by mimicking both peripheral and foveal vision of the human eye. To this end, we adopt a two-branch neural network leveraging two common projections: equirectangular and cubemap projections. In particular, equirectangular projection incorporates a complete field-of-view but introduces distortion, whereas cubemap projection avoids distortion but introduces discontinuity at the boundary of the cube. Thus we propose a bi-projection fusion scheme along with learnable masks to balance the feature map from the two projections. Moreover, for the cubemap projection, we propose a spherical padding procedure which mitigates discontinuity at the boundary of each face. We apply our method to four panorama datasets and show favorable results against the existing state-of-the-art methods.

Demo

CVPR 2020

BiFuse: Monocular 360˚ Depth Estimation via Bi-Projection Fusion

Fu-En Wang*, Yu-Hsuan Yeh*, Min Sun, Wei-Chen Chiu, Yi-Hsuan Tsai

Paper

Poster

Code

@InProceedings{Wang_2020_CVPR,
    author = {Wang, Fu-En and Yeh, Yu-Hsuan and Sun, Min and Chiu, Wei-Chen and Tsai, Yi-Hsuan},
    title = {BiFuse: Monocular 360 Depth Estimation via Bi-Projection Fusion},
    booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June},
    year = {2020}
}