Combining Laser ScansCombining Laser Scans

Yong Joo KilYong Joo Kil11, Boris Mederos, Boris Mederos22, and Nina Amenta, and Nina Amenta11

1 1 Department of Computer Science, University of California at DavisDepartment of Computer Science, University of California at Davis22 Instituto Nacional de Matematica Pura e Aplicada - IMPA Instituto Nacional de Matematica Pura e Aplicada - IMPA

IDAV IDAV Institute for Data Analysis and VisualizationInstitute for Data Analysis and VisualizationVisualization and Graphics Research GroupVisualization and Graphics Research Group

2D Super Resolution2D Super Resolution

A Fast Super-Resolution Reconstruction Algorithm, [Michael Elad, Yacov Hel-Or]

Low Resolution Images Super Resolution Image

Surface Super ResolutionSurface Super Resolution

One Raw Scan Super resolved (100 scans) Photo

Improve 3D Acquisition MethodsImprove 3D Acquisition Methods

• Better hardware– Costly

• Multiple scans + software– Refine output of current hardware – Cost effective– Smaller devices

Physical SetupPhysical Setup

xy

z (viewing

direction)

Minolta Vivid 910

3D Super Resolution Pipeline3D Super Resolution Pipeline

Input Scans Global Registration

Super Resolution

Super Registration

Convergence

No

Yes

Smoothing Super Resolution Mesh

Viewing direction axisViewing direction axis

z

x

y

Sample PointsLow Resolution Sample SpacingSample PointsLow Resolution Sample Spacing

WidthOf one Scan

Super Resolution Sample SpacingSuper Resolution Sample Spacing

q

N(q)width/4

2.5D Super Resolution2.5D Super Resolution

First Super Resolution Mesh (S1)First Super Resolution Mesh (S1)

Super Resolution MethodSuper Resolution Method

Input Scans Global Registration

Super Resolution

Super Registration

Convergence

No

Yes

Smoothing Super Resolution Mesh

Bilateral FilterBilateral Filter

Super Resolution MethodSuper Resolution Method

Input Scans Global Registration

Super Resolution

Super Registration

Convergence

No

Yes

Smoothing Super Resolution Mesh

Super RegistrationSuper Registration

raw scan super resolution mesh

Second Super Resolution Mesh S2Second Super Resolution Mesh S2

Super Resolution MethodSuper Resolution Method

Input Scans Global Registration

Super Resolution

Super Registration

Convergence

No

Yes

Smoothing Super Resolution Mesh

Point Samples (1st Model)Point Samples (1st Model)

Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]

Nyquist Sampling Theorem:Sample signal finely enough, thenReconstruct original signal perfectly.

Band limited signal

Sampling at lower resolutionSampling at lower resolution

Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]

That’s it!

Linear Model with Blur (2nd Model)Linear Model with Blur (2nd Model)

Nkkkkkk EXY 1 FCD

High-ResolutionImage X

Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]

C

Blur

1 D1

Decimation

Low-Resolution

Images

Transformation

F1

Y1E1

Noise

+

CNFN DN

YNEN+

Nkkkkkk EX 1Y FCD

The Model as One Equation

NNNNN E

E

E

X

Y

Y

Y

2

1

222

111

2

1

FCD

FCD

FCD

EX HY

Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]

Model for 3D laser scan? Model for 3D laser scan?

Pipeline : Laser Scanner Pipeline : Laser Scanner

Derived from Better Optical Triangulation through Spacetime Analysis, Curless and Levoy, 1995

laser beam

SurfacePeak reconstructionCCD sensor

Video sequenceVideo sequencex

y

time

Non Linear functionsNon Linear functions

f ( ) =

g ( ) =

SimplificationSimplification

• Assume– Points from Surface– Gaussian Noise

Point Sampling ModelPoint Sampling Model

High-ResolutionImage X

C

Blur

k Dk

Decimation

Low-Resolution

ImagesTransformation

Fk x

[ ELAD M., HEL-OR Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur. IEEE Transactions on Image Processing 10,8 (2001) ]

Solution Average

YkEk

Gaussian Noise

+

SimplificationSimplification

• Solution– Register scans– Averaging

• Easy

• Inexpensive

• It works!

Close-up Scan of ParrotClose-up Scan of Parrot• 146 Scans• 4 times the original resolution.

Super resolve far & close objects?Super resolve far & close objects?

Derived from Better Optical Triangulation through Spacetime Analysis, Curless and Levoy, 1995

SurfaceCCD sensor

Super resolve small & large objects?Super resolve small & large objects?

One raw Scan Super resolution (117 scans)

Is it worth taking more than one scan? Is it worth taking more than one scan?

One raw scan Super resolution PhotographSubdivion of (a)

Is it worth shifting?Is it worth shifting?

With Shifts (117scans) Without Shifts (117scans)

How many scans are enough?How many scans are enough?

Point DistributionPoint Distribution

Tiling ArtifactTiling Artifact

Sampling PatternSampling Pattern

Random xy shift + Rotation

Mayan Tablet (One Scan)Mayan Tablet (One Scan)

39

Mayan Tablet (90 scans)Mayan Tablet (90 scans)

40

Before & AfterBefore & After

41

Systematic ErrorsSystematic ErrorsSuper resolved Photo

42

Parrot Model (6 views * 100 scans)Parrot Model (6 views * 100 scans)

Future workFuture work

• 2.5D to 3D

• Resolving Systematic Errors

• Other Devices

AcknowledgementsAcknowledgements

• Kelcey Chen

• Geomagic Studios

• NSF CCF-0331736

• Brazilian National Council of Technological and Scientific Development (CNPq)

45

ExtrasExtras

InterpolationsInterpolations

Nyquist frequencyNyquist frequency

48

DataData

g-1( ) =

50

N

k 1kk

tk

tk FDDFR

Solving this linear system is equivalent to an average. [ ELAD M., HEL-OR Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur. IEEE Transactions on Image Processing 10,8 (2001) ]

Solving this linear system is equivalent to an average. [ ELAD M., HEL-OR Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur. IEEE Transactions on Image Processing 10,8 (2001) ]

2

1k ||Y||)( XFDX k

N

kk

kF

PRX

N

k 1k

tk

tk YDFP

Mimize

Diagonal MatrixDiagonal Matrix

Can be a permutation or displacement matrixCan be a permutation or displacement matrix

Equivalent to Equivalent to

51

Error between low res and super res.Error between low res and super res.

52

Error between low res and super res.Error between low res and super res.

53

Registeration resultRegisteration result

54

Before and After RegistrationBefore and After Registration

55

Error between low res and super res.Error between low res and super res.

56

Least Squares Least Squares

)()(2

2XXXX T HYHYHY Minimize:

Solve by:

022

XX

TT HHYH

YHHH TT X , or

Steepest Descent Iteration:

N

kjkk

Tkjj XYXX

11 ]ˆ[ˆˆ HH

kkkk FCDH ,