Realtime 3D model construction with Microsoft Kinect and an NVIDIA Kepler laptop GPU

Paul Caheny

MSc in HPC 2011/2012 Project Preparation Presentation.

Today's Presentation

Project motivation.

Introduction to Microsoft Kinect and depth maps.

A taster on algorithms for constructing 3D models from depth maps.

Particular goals and constraints for this dissertation.

Paul Caheny 2012

Project Motivation

Proposal from Holoxica Ltd.

Most holograms produced from 3D models which are computer generated ab initio.

Some customers already posses highquality 3D models of realworld objects.

Commercial scanning solutionsprohibitively expensive forHoloxica's purposes.

A cheaper, good quality, portable 3D scanning & model construction system could make holograms a more viable and attractive proposition for customers.

Paul Caheny 2012

Microsoft Kinect & Depth Maps

Depth maps have been the subject of much research for 3D model construction over the past 20 years.

The Kinect provides unprecedented quality to cost ratio as a depth sensor.

Video Camera, IR Projector, IR Camera IR Projector/Camera constitute a depthsensor providing a 640 x 480 depth mapat 30Hz frame rate.

Paul Caheny 2012

3D Modelling from Depth MapsPut simply – fusing multiple viewpoints of the scene orobject into a single unified 3D model.

Early approaches involved highly calibratedmotion of the object plus simple surfaceconstruction techniques (e.g. directlymeshing the depth data points).

Current state of the art replaces calibratedmotion with software tracking & high quality3D model construction techniques.

Two techniques: Iterative Closest Point (ICP) algorithm for object tracking and Volumetric Integration for high quality model generation.

Paul Caheny 2012

Iterative Closest Point (ICP)

A method for the alignment of 3D surfaces. Introduced independently by Besl & McKay at GM and Chen

& Medioni at Uni. Southern California in early '90s.

If corresponding points on surface in two views are known, trivial to compute exact transform which aligns surfaces. But we don't know corresponding points on surfaces in distinct depth maps.

Make a heuristic guess of corresponding points, compute transform with selected points, achieving closer alignment. Reselect points heuristically on transformed surfaces, iterate until close enough. Paul Caheny 2012

Volumetric Integration

A technique for fusing data from multiple aligned depth maps. ICP alignment & depth sensors have inherent margin of error

/ measurement uncertainty. Results in noisy data points following ICP alignment.

Reduce noise & improve result by combining multiple views using heuristics which minimises noise.

Curless &Levoy, Stanford '96

Paul Caheny 2012

What's New?

ICP and Volumetric Integration introduced in the '90s. Both techniques have have been subject of much refinement

since.

In the 90s workflow looked like: Scan -> ICP Batch process -> Volumetric Integration Batch process -> Finished 3D Model.

Mid 2000s saw systems with realtime ICP phase plus batch Volumetric Integration phase to create finished model.

2011 saw publication of research by Augmented Reality Group at Microsoft Research Cambridge demonstrating a realtime, synchronous ICP and Volumetric Integration system called KinectFusion running at Kinect full frame rate of 30Hz.

Paul Caheny 2012

My Dissertation

Focus on small object scanning and Laptop GPU. Implement the algorithms from scratch using CUDA.

Plan to use NVIDIA's next generation Kepler architecture, recently launched for consumer desktop & laptop market.

Laptop GPU TDP ~35W versus Tesla 2090 TDP of 225W – Fewer Cores, Less Memory, Less Bandwidth, Lower Clock Speed.

Paul Caheny 2012

Wrap Up

References:

R. Newcombe et al. KinectFusion: Real-time dense surface mapping and tracking. In Proc. 10th IEEE Int. Symp. on Mixed and Augmented Reality, 2011.

B. Curless and M. Levoy. A volumetric method for building complex models from range images. ACM Trans. on Graphics, 1996.

P. Besl and N. McKay. A method for registration of 3D shapes. IEEE Trans. on Pattern Analysis & Machine Intelligence, 14:239–256, 1992.

Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing (IVC), 10(3):145–155, 1992.

Paul Caheny 2012