Point-Plane SLAM for RGB-D Sensors and Its Applications
Real-time 3D reconstruction using an RGB-D sensor on a tablet.
MERL Researchers: (Computer Vision).
Search MERL publications by keyword: Computer Vision, SLAM, 3D reconstruction, registration, plane extraction,
We present a real-time 3D reconstruction system using an RGB-D sensor on a hand-held tablet. The main novelty of the system is a simultaneous localization and mapping (SLAM) algorithm that uses both point and plane features as primitives. Planes are the most common structures in man-made indoor and outdoor scenes.
In contrast to existing SLAM algorithms that use only point features, our algorithm that also exploits plane features has the following advantages:
- (1) it enables faster correspondence search and registration, since the number of planes is typically much smaller than the number of points;
- (2) planes generated by many points are more robust to noise than individual points, leading to more accurate registration; and
- (3) it produces plane-based 3D models that are more compact than point-based ones.
As the core of the algorithm, we show that it is possible to register 3D data in two different coordinate systems using any combination of three point/plane features (3 planes, 2 planes and 1 point, 1 plane and 2 points, and 3 points). We use the minimal set of features in a RANSAC framework to robustly compute correspondences and estimate the camera pose. We also use the point and plane features in bundle adjustment to jointly refine all the camera poses and point/plane parameters.
Our system enables large-scale indoor 3D reconstruction and generates plane-based and point-based 3D models. The 3D models can be used for several applications, including robot collision avoidance (demonstrated at iREX 2013 and MODEX 2014), 2D camera localization for augmented reality (demonstrated at 3DV 2014 and MVA 2015), and calibration of non-overlapping cameras (presented at 3DV 2014).
