Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB

Moving Object Reconstruction in Monocular Video Data Using Boundary Generation

Moving Object Reconstruction in Monocular Video Data Using Boundary Generation

Supplementary material to the paper "Moving Object Reconstruction in Monocular Video Data Using Boundary Generation"

Abstract

We present a method to reconstruct the three-dimensional shape of a moving instance of a known object category in video data. We exploit state-of-the-art semantic segmentation techniques to extract the object's two-dimensional shape in each frame. Therefore, our method is robust to occlusion, handles stationary objects and extends naturally to multiple video sequences. We apply Structure from Motion (SfM) to previously generated object images in order to compute a three-dimensional representation of the object. Our approach allows us to remove outliers in SfM reconstructions and to compute clean object meshes by leveraging previously computed semantic segmentations and virtual camera positions. We evaluate the accuracy of our method using a multi-view dataset of a moving vehicle. A laser scan serves as ground truth. We applied our algorithm on publicly available video data and on 25 sequences from our dataset. The algorithm achieves an average point distance of 3.3 cm evaluated on seven trajectories contained in the dataset.

 

Dataset

The dataset consists of 25 videos capturing a car moving on eight different trajectories. The video sequences cover a high variety of viewing angles. We created a laser scan of the vehicle present in the dataset serving as ground truth. The laser scans were acquired using a Zoller+Fröhlich scanner. The distance to the reflecting object is estimated based on the phase shift between received and emitted signal. The scanning head was operated on a rigid tripod which results in ranging accuracies of a few millimeters. The laser scans are automatically registered. Noise in the measurement is manually removed.

Trajectory Overview:

Laser Scan (Ground Truth):

 

Accessing the Dataset

In order to download the data use the form below or send an e-mail to Sebastian Bullinger. We will send you a download link to the original video sequences, the corresponding reconstruction results as well as the laser scan ground truth.

Name:

Institute:

E-mail:



We will grant you access to our FTP server, where the video original video sequences, the corresponding reconstruction results as well as the laser scan ground truth can be downloaded.

 

Example Reconstruction Results Using Boundary Generation

Sparse Reconstruction Result:

Boundary Generation Result:

Boundary Mesh Overlayed with Input Video:

 

References

S. Bullinger, C. Bodensteiner, S. Wuttke and M. Arens, "Moving Object Reconstruction in Monocular Video Data Using Boundary Generation", ICPR 2016.