Overview of 3D ScannersOverview of 3D Scanners

Acknowledgement: some content and figures by Brian Curless

3D Data Types3D Data Types

• Point DataPoint Data

• Volumetric DataVolumetric Data

• Surface DataSurface Data

3D Data Types: Point Data3D Data Types: Point Data

• ““Point clouds”Point clouds”

• Advantage: simplest data typeAdvantage: simplest data type

• Disadvantage: no information onDisadvantage: no information onadjacency / connectivityadjacency / connectivity

3D Data Types: Volumetric Data3D Data Types: Volumetric Data

• Regularly-spaced grid in (x,y,z): “voxels”Regularly-spaced grid in (x,y,z): “voxels”

• For each grid cell, storeFor each grid cell, store– Occupancy (binary: occupied / empty)Occupancy (binary: occupied / empty)

– DensityDensity

– Other propertiesOther properties

• Popular in medical imagingPopular in medical imaging– CAT scansCAT scans

– MRIMRI

3D Data Types: Volumetric Data3D Data Types: Volumetric Data

• Advantages:Advantages:– Can “see inside” an objectCan “see inside” an object

– Uniform sampling: simpler algorithmsUniform sampling: simpler algorithms

• Disadvantages:Disadvantages:– Lots of dataLots of data

– Wastes space if only storing a surfaceWastes space if only storing a surface

– Most “vision” sensors / algorithms returnMost “vision” sensors / algorithms returnpoint or surface datapoint or surface data

3D Data Types: Surface Data3D Data Types: Surface Data

• PolyhedralPolyhedral– Piecewise planarPiecewise planar

– Polygons connected togetherPolygons connected together

– Most popular: “triangle meshes”Most popular: “triangle meshes”

• SmoothSmooth– Higher-order (quadratic, cubic, etc.) curvesHigher-order (quadratic, cubic, etc.) curves

– Bézier patches, splines, NURBS, subdivision Bézier patches, splines, NURBS, subdivision surfaces, etc.surfaces, etc.

3D Data Types: Surface Data3D Data Types: Surface Data

• Advantages:Advantages:– Usually corresponds to what we seeUsually corresponds to what we see

– Usually returned by vision sensors / Usually returned by vision sensors / algorithmsalgorithms

• Disadvantages:Disadvantages:– How to find “surface” for translucent How to find “surface” for translucent

objects?objects?

– Parameterization often non-uniformParameterization often non-uniform

– Non-topology-preserving algorithms difficultNon-topology-preserving algorithms difficult

3D Data Types: Surface Data3D Data Types: Surface Data

• Implicit surfaces (cf. parametric)Implicit surfaces (cf. parametric)– Zero set of a 3D functionZero set of a 3D function

– Usually regularly sampled (voxel grid)Usually regularly sampled (voxel grid)

• Advantage: easy to write algorithms Advantage: easy to write algorithms that change topologythat change topology

• Disadvantage: wasted space, timeDisadvantage: wasted space, time

2½-D Data2½-D Data

• Image: stores an intensity / color alongImage: stores an intensity / color alongeach of a set of regularly-spaced rays in each of a set of regularly-spaced rays in spacespace

• Range image: stores a Range image: stores a depthdepth along alongeach of a set of regularly-spaced rays in each of a set of regularly-spaced rays in spacespace

• Not a complete 3D description: does notNot a complete 3D description: does notstore objects occluded (from some viewpoint)store objects occluded (from some viewpoint)

• View-dependent scene descriptionView-dependent scene description

2½-D Data2½-D Data

• This is what most sensors / algorithmsThis is what most sensors / algorithmsreallyreally return return

• AdvantagesAdvantages– Uniform parameterizationUniform parameterization

– Adjacency / connectivity informationAdjacency / connectivity information

• DisadvantagesDisadvantages– Does not represent entire objectDoes not represent entire object

– View dependentView dependent

2½-D Data2½-D Data

• Range imagesRange images

• Range surfacesRange surfaces

• Depth imagesDepth images

• Depth mapsDepth maps

• Height fieldsHeight fields

• 2½-D images2½-D images

• Surface profilesSurface profiles

• xyzxyz maps maps

• ……

Related FieldsRelated Fields

• Computer VisionComputer Vision– Passive range sensingPassive range sensing

– Rarely construct complete, accurate modelsRarely construct complete, accurate models

– Application: recognitionApplication: recognition

• MetrologyMetrology– Main goal: absolute accuracyMain goal: absolute accuracy

– High precision, provable errors more High precision, provable errors more important than scanning speed, complete important than scanning speed, complete coveragecoverage

– Applications: industrial inspection, quality Applications: industrial inspection, quality control, as-built modelscontrol, as-built models

Related FieldsRelated Fields

• Computer GraphicsComputer Graphics– Often want complete modelOften want complete model

– Low noise, geometrically consistent model Low noise, geometrically consistent model more important than absolute accuracymore important than absolute accuracy

– Application: animated CG charactersApplication: animated CG characters

TerminologyTerminology

• Range acquisition, shape acquisition, Range acquisition, shape acquisition, rangefinding, range scanning, 3D rangefinding, range scanning, 3D scanningscanning

• Alignment, registrationAlignment, registration

• Surface reconstruction, 3D scan Surface reconstruction, 3D scan merging, scan integration, surface merging, scan integration, surface extractionextraction

• 3D model acquisition3D model acquisition

Range Acquisition TaxonomyRange Acquisition Taxonomy

RangeRangeacquisitionacquisition

ContactContact

TransmissiveTransmissive

ReflectiveReflectiveNon-opticalNon-optical

OpticalOptical

Industrial CTIndustrial CT

Mechanical Mechanical (CMM, jointed arm)(CMM, jointed arm)

RadarRadar

SonarSonar

UltrasoundUltrasound

MRIMRI

Ultrasonic trackersUltrasonic trackersMagnetic trackersMagnetic trackers

Inertial Inertial (gyroscope, accelerometer)(gyroscope, accelerometer)

Range Acquisition TaxonomyRange Acquisition Taxonomy

OpticalOpticalmethodsmethods

PassivePassive

ActiveActive

Shape from X:Shape from X:stereostereomotionmotionshadingshadingtexturetexturefocusfocusdefocusdefocus

Active variants of passive methodsActive variants of passive methodsStereo w. projected textureStereo w. projected textureActive depth from defocusActive depth from defocusPhotometric stereoPhotometric stereo

Time of flightTime of flight

TriangulationTriangulation

Touch ProbesTouch Probes

• Jointed arms with Jointed arms with angular encodersangular encoders

• Return position, Return position, orientation of tiporientation of tip

Faro Arm – Faro Technologies, Inc.Faro Arm – Faro Technologies, Inc.

Optical Range Acquisition Optical Range Acquisition MethodsMethods

• Advantages:Advantages:– Non-contactNon-contact

– SafeSafe

– Usually inexpensiveUsually inexpensive

– Usually fastUsually fast

• Disadvantages:Disadvantages:– Sensitive to transparencySensitive to transparency

– Confused by specularity and interreflectionConfused by specularity and interreflection

– Texture (helps some methods, hurts others)Texture (helps some methods, hurts others)

StereoStereo

• Find feature in one image, search along Find feature in one image, search along epipolar line in other image for epipolar line in other image for correspondencecorrespondence

StereoStereo

• Advantages:Advantages:– PassivePassive

– Cheap hardware (2 cameras)Cheap hardware (2 cameras)

– Easy to accommodate motionEasy to accommodate motion

– Intuitive analogue to human visionIntuitive analogue to human vision

• Disadvantages:Disadvantages:– Only acquire good data at “features”Only acquire good data at “features”

– Sparse, relatively noisy data (correspondence is hard)Sparse, relatively noisy data (correspondence is hard)

– Bad around silhouettesBad around silhouettes

– Confused by non-diffuse surfacesConfused by non-diffuse surfaces

• Variant: multibaseline stereo to reduce Variant: multibaseline stereo to reduce ambiguityambiguity

Why More Than 2 Views?Why More Than 2 Views?

• BaselineBaseline– Too short – low accuracyToo short – low accuracy

– Too long – matching becomes hardToo long – matching becomes hard

Why More Than 2 Views?Why More Than 2 Views?

• Ambiguity with 2 viewsAmbiguity with 2 views

Camera 1Camera 1 Camera 2Camera 2Camera 3Camera 3

Multibaseline StereoMultibaseline Stereo

[[OkutamiOkutami & Kanade] & Kanade]

Shape from MotionShape from Motion

• ““Limiting case” of multibaseline stereoLimiting case” of multibaseline stereo

• Track a feature in a video sequenceTrack a feature in a video sequence

• For For nn frames and frames and ff features, have features, have22nnff knowns, 6 knowns, 6nn+3+3ff unknowns unknowns

Shape from MotionShape from Motion

• Advantages:Advantages:– Feature tracking easier than Feature tracking easier than

correspondence in far-away viewscorrespondence in far-away views

– Mathematically more stable (large baseline)Mathematically more stable (large baseline)

• Disadvantages:Disadvantages:– Does not accommodate object motionDoes not accommodate object motion

– Still problems in areas of low texture, in Still problems in areas of low texture, in non-diffuse regions, and around silhouettesnon-diffuse regions, and around silhouettes

Shape from ShadingShape from Shading

• Given: image of surface with known, Given: image of surface with known, constant reflectance under known point constant reflectance under known point lightlight

• Estimate normals, integrate to find Estimate normals, integrate to find surfacesurface

• Problem: ambiguityProblem: ambiguity

Shape from ShadingShape from Shading

• Advantages:Advantages:– Single imageSingle image

– No correspondencesNo correspondences

– Analogue in human visionAnalogue in human vision

• Disadvantages:Disadvantages:– Mathematically unstableMathematically unstable

– Can’t have textureCan’t have texture

• ““Photometric stereo” (active method) Photometric stereo” (active method) more practical than passive versionmore practical than passive version

Shape from TextureShape from Texture

• Mathematically similar to shape from shading, Mathematically similar to shape from shading, but uses stretch and shrink of a (regular) but uses stretch and shrink of a (regular) texturetexture

Shape from TextureShape from Texture

• Analogue to human visionAnalogue to human vision

• Same disadvantages as shape from Same disadvantages as shape from shadingshading

Shape from Focus and DefocusShape from Focus and Defocus

• Shape from focus: at which focus Shape from focus: at which focus setting is a given image region setting is a given image region sharpest?sharpest?

• Shape from defocus: how out-of-focus is Shape from defocus: how out-of-focus is each image region?each image region?

• Passive versions rarely usedPassive versions rarely used

• Active depth from defocus can beActive depth from defocus can bemade practicalmade practical

Active Optical MethodsActive Optical Methods

• Advantages:Advantages:– Usually can get dense dataUsually can get dense data

– Usually much more robust and accurate Usually much more robust and accurate than passive techniquesthan passive techniques

• Disadvantages:Disadvantages:– Introduces light into scene (distracting, etc.)Introduces light into scene (distracting, etc.)

– Not motivated by human visionNot motivated by human vision

Active Variants of Passive Active Variants of Passive TechniquesTechniques

• Regular stereo with projected textureRegular stereo with projected texture– Provides features for correspondenceProvides features for correspondence

• Active depth from defocusActive depth from defocus– Known pattern helps to estimate defocusKnown pattern helps to estimate defocus

• Photometric stereoPhotometric stereo– Shape from shading with multiple known Shape from shading with multiple known

lightslights

Pulsed Time of FlightPulsed Time of Flight

• Basic idea: send out pulse of light Basic idea: send out pulse of light (usually laser), time how long it takes to (usually laser), time how long it takes to returnreturn tcd

2

1tcd

2

1

Pulsed Time of FlightPulsed Time of Flight

• Advantages:Advantages:– Large working volume (up to 100 m.)Large working volume (up to 100 m.)

• Disadvantages:Disadvantages:– Not-so-great accuracy (at best ~5 mm.)Not-so-great accuracy (at best ~5 mm.)

• Requires getting timing to ~30 picosecondsRequires getting timing to ~30 picoseconds

• Does not scale with working volumeDoes not scale with working volume

• Often used for scanning buildings, Often used for scanning buildings, rooms, archeological sites, etc.rooms, archeological sites, etc.

AM Modulation Time of FlightAM Modulation Time of Flight

• Modulate a laser at frequencyModulate a laser at frequencym m ,, it it

returns with a phase shift returns with a phase shift

• Note the ambiguity in the measured Note the ambiguity in the measured phase!phase! Range ambiguity of Range ambiguity of 11//22mmnn

2

2

2

1 n

ν

cd

m

2

2

2

1 n

ν

cd

m

AM Modulation Time of FlightAM Modulation Time of Flight

• Accuracy / working volume tradeoffAccuracy / working volume tradeoff(e.g., noise ~ (e.g., noise ~ 11//500 500 working volume)working volume)

• In practice, often used for room-sized In practice, often used for room-sized environments (cheaper, more accurate environments (cheaper, more accurate than pulsed time of flight)than pulsed time of flight)

TriangulationTriangulation

Triangulation: Moving theTriangulation: Moving theCamera and IlluminationCamera and Illumination

• Moving independently leads to Moving independently leads to problems with focus, resolutionproblems with focus, resolution

• Most scanners mount camera and light Most scanners mount camera and light source rigidly, move them as a unitsource rigidly, move them as a unit

Triangulation: Moving theTriangulation: Moving theCamera and IlluminationCamera and Illumination

Triangulation: Moving theTriangulation: Moving theCamera and IlluminationCamera and Illumination

Triangulation: Extending to 3DTriangulation: Extending to 3D

• Possibility #1: add another mirror (flying spot)Possibility #1: add another mirror (flying spot)

• Possibility #2: project a stripe, not a dotPossibility #2: project a stripe, not a dot

ObjectObject

LaserLaser

CameraCameraCameraCamera

Triangulation Scanner IssuesTriangulation Scanner Issues

• Accuracy proportional to working volumeAccuracy proportional to working volume(typical is ~1000:1)(typical is ~1000:1)

• Scales down to small working volumeScales down to small working volume(e.g. 5 cm. working volume, 50 (e.g. 5 cm. working volume, 50 m. accuracy)m. accuracy)

• Does not scale up (baseline too large…)Does not scale up (baseline too large…)

• Two-line-of-sight problem (shadowing from Two-line-of-sight problem (shadowing from either camera or laser)either camera or laser)

• Triangulation angle: non-uniform resolution if Triangulation angle: non-uniform resolution if too small, shadowing if too big (useful range: too small, shadowing if too big (useful range: 1515-30-30))

Triangulation Scanner IssuesTriangulation Scanner Issues

• Material properties (dark, specular)Material properties (dark, specular)

• Subsurface scatteringSubsurface scattering

• Laser speckleLaser speckle

• Edge curlEdge curl

• Texture embossingTexture embossing

Multi-Stripe TriangulationMulti-Stripe Triangulation

• To go faster, project multiple stripesTo go faster, project multiple stripes

• But which stripe is which?But which stripe is which?

• Answer #1: assume surface continuityAnswer #1: assume surface continuity

Multi-Stripe TriangulationMulti-Stripe Triangulation

• To go faster, project multiple stripesTo go faster, project multiple stripes

• But which stripe is which?But which stripe is which?

• Answer #2: colored stripes (or dots)Answer #2: colored stripes (or dots)

Multi-Stripe TriangulationMulti-Stripe Triangulation

• To go faster, project multiple stripesTo go faster, project multiple stripes

• But which stripe is which?But which stripe is which?

• Answer #3: time-coded stripesAnswer #3: time-coded stripes

Time-Coded Light PatternsTime-Coded Light Patterns

• Assign each stripe a unique illumination codeAssign each stripe a unique illumination codeover time [Posdamer 82]over time [Posdamer 82]

SpaceSpace

TimeTime