2d optics 3d optics 3d optics - institute for mathematics ... · *caltech optical info. proc. group...

1

George BarbastathisMechanical Engineering Department

3D Optics

George Barbastathis

Mechanical Engineering Department

Massachusetts Institute of Technology


2D Optics 3D Optics

lenses gratings

Light interacts with medium on a sequence of discrete surfaces

GRadient INdex optics(GRIN)

VolumeHolograms

Light interacts with medium throughout a volumethroughout a volume



Imaging with traditional lenses

depthof

field

lens

A A imaged

B

B

blurred

image plane


Imaging with 3D lenses

depthof

field

3D lens

B

B

blocked

image plane

A

A imaged

Contrast Defocus ⇔


Shape recovery (“profilometry”) with 3D lenses

Line scan method: 2D scanningLine scan method: 2D scanninglongitudinal + one lateral dimensionlongitudinal + one lateral dimension

raw imageson CCD camera

2½D shape(“profile”)

in digital form

Working distance d= 50cmLongitudinal resolution ∆z FWHM< 1mm

2


z=0 µm z=50 µm

z=200 µm z=250 µm

originalobject

Microturbine provided by Chee Wei Wong, Alan Epstein, MIT

Object Distance = 46 cmObject Distance = 46 cm

Resolution accomplished Resolution accomplished ≤≤ 100 100 µµmmraw imagesraw images

profileprofilereconstructionreconstruction

Arnab Sinha, George BarbastathisMIT 3D Optical Systems groupOptics Express 11:3202, 2003

Shape recovery (“profilometry”) with 3D lenses


index ofrefraction

n0+n1

n0– n1

3D lens: plane-to-plane wave volume hologram


Making the (simplest) 3D lens

photosensitivematerial

referencepoint source

of

sθ

plane wavesignal beam

referenceplane

( ) ( )[ ]rkkr ⋅−+= rs10 cosnnnafter exposure


Operating the (simplest) 3D lens

3D objectobjective

lens 3D lens

collectorlens

digitalcamera

visiblecolumn

x∆ y∆

z∆ L

of

sθ

xy

z

mm1m100~ ÷µL


Operating a multiplex 3D lens

3D objectobjective

lens

collectorlens

digitalcamera

visiblecolumns

3D lens(multiplex)


Operating a hyperspectral multiplex 3D lens

3D objectobjective

lens

collectorlens

digitalcamera

visiblerainbow slices

3D lens(multiplex)

3


Rainbow volume holographic imaging

slice #1slice #1 slice #2slice #2

2½D shape2½D shape

Wenyang Sun, George BarbastathisMIT 3D Optical Systems group

Optics Letters 30:976, 2005


Multiplex imaging of 3D fluorescent object

3D object:fluorescent beads

in water

three three ““slicesslices”” throughthroughfluorescent (3D) object, stacked fluorescent (3D) object, stacked

along along longitudinallongitudinal direction direction ……

…… are viewed are viewed simultaneouslysimultaneously and and sideside--byby--sidesideon the digital cameraon the digital camera

Wenhai Liu,* Demetri Psaltis,* George Barbastathis***Caltech Optical Info. Proc. group **MIT 3D Optical Systems group

Optics Letters 27:854, 2002


Camera pixel layout for 4D (3D spatial + spectral) imaging#1

λ1 λ2 λw

slice index #2 #N

M pixels

W pixels

3D object

#1#2 #Nslice index

WNMWzyx

×××=××× λ

#1

#2

#W

slitindex

camera die


3D opticalimage

traditionalcamera

0 100 200 300 400 500 600 70040

60

80

100

120

140

160

180

200

220

240

irradiance acrossimage cross-section

Sun Light (completely passive) illumination

~10cm

~5m

objective

0 100 200 300 400 500 600 7060

80

100

120

140

160

180


Temporal Heterodyning

input signal

local oscillator

low pass filter

transducer


Temporal Heterodyning

input signal

local oscillator

low pass filter

transducer

( )φω +tA ii cos

( )tA LL cos ω

( )( )φωω −− tAA cos iLiL

4


1D Spatial Heterodyning

input signal

local oscillator

( ){ }φ+xkiA ii cos

( ){ }xkiA LL cos

?



input signal

local oscillator

( ){ }φ+xkiA ii exp

( ) ( ) 2SSSRR expexp φ++ xikAxikA

thintransparency



SRi kkk ++

SRi kkk −+

SRi kkk +−

SRi kkk −−

input signal

local oscillator





input signal

local oscillator





input signal

local oscillator





SRi kkk +−

input signal

local oscillator



5



SRi kkk +−

low pass filter

input signal

local oscillator



( )( ) ⎭

⎬⎫

⎩⎨⎧

⎥⎦

⎤⎢⎣

⎡++

′+−

S

SRiSRi exp

φφxkkk

iAAA


1D Spatial Heterodyning in the Fourier domain

input signal

local oscillator

( ){ }φ+′′xki i exp

( ) ( ) 2SSSRR expexp φ+′′+′′ xikAxikA

ℑ⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx



input signal

local oscillator

( ) ( ) 2SSSRR expexp φ+′′+′′ xikAxikA

ℑ ℑ

spatial filter

( )⎥⎦⎤

⎢⎣⎡ +−

−′π

λδ2

SRi kkkfx

⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx ( ){ }φ+′′xki i exp



input signal

local oscillator

( ){ }φ+′′⋅rk i exp i

( ) ( ) 2SSSRR expexp φ+′′⋅+′′⋅ rkrk iAiA

⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx

ℑ ?



input signal

local oscillator


⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx

ℑ ?

( )⎥⎦⎤

⎢⎣⎡ +−

−′π

λδ2

SRi kkkfx

( ){ }φ+′′⋅rk i exp i



input signal

local oscillator


⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx

ℑ ?

( )⎥⎦⎤

⎢⎣⎡ +−

−′π

λδ2

SRi kkkfx

( ){ }φ+′′⋅rk i exp i

6



input signal

local oscillator


⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx

ℑ ?

( )⎥⎦⎤

⎢⎣⎡ +−

−′π

λδ2

SRi kkkfx

( ){ }φ+′′⋅rk i exp i


Phase matching as a filter for 3D spatial heterodyning

Recording Bragg-matched readout

Diffracted beams from elemental thin slicesare all in phase in phase → strong diffraction


Recording Bragg-mismatched readout

Diffracted beams from elemental thin slicesare out of phase out of phase → no diffraction




Rθ∆

θλθsin2R LL

=Λ

=∆Angle Bragg selectivity

θ

Λ

L

θ




input signal

local oscillator


⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx

( )⎥⎦⎤

⎢⎣⎡ +−

−′π

λδ2

SRi kkkfx

propagation along z



input signal

local oscillator


⎟⎠⎞

⎜⎝⎛ −

πλδ2

ifkx

( )⎥⎦⎤

⎢⎣⎡ +−

−′π

λδ2

SRi kkkfx

propagation along z

7


3D Spatial Heterodyning in k-space


Rk

Sk

Kλπ2

radius

=k

RS kkK −=

pk

K



Rk

Sk

Kλπ2

radius

=k

RS kkK −=

pk

K

dk

Kkk += Rd




Rk

Sk

Kλπ2

radius

=k

RS kkK −=

pk

K

dk

( )[ ] zxxKkk ˆˆ ˆ dRd zk+⋅+=k=d k

⎟⎠⎞

⎜⎝⎛ ∆= LkII zd

2matched-Braggdiffracted 2

1sinc

zkd∆

x

z



Applications: 3D holographic data storage

⎟⎟⎠

⎞⎜⎜⎝

⎛−

πλ

δ2

i,mfkx ( )yxgm ′′,


Applications: 3D holographic data storage

⎟⎟⎠

⎞⎜⎜⎝

⎛−

πλ

δ2

i,nfkx ( )yxgn ′′,


Applications: 3D holographic optical correlators

( )yxgm , ⎟⎟⎠

⎞⎜⎜⎝

⎛−′

πλ

δ2

i,mfkx

strongest

8


Applications: 3D holographic optical correlators

( )yxgn , ⎟⎟⎠

⎞⎜⎜⎝

⎛−′

πλ

δ2

i,nfkx

strongest


Applications: 3D holographic optical imaging

( )zyxf ,,

⎟⎠⎞

⎜⎝⎛∆

×⎟⎠⎞

⎜⎝⎛

′∆′

×

×⎟⎠⎞

⎜⎝⎛ ′−′

zz

xx

zyfkxf

sliceslit

,,2

0

πλ



( )zyxf ,,

⎟⎠⎞

⎜⎝⎛∆

×⎟⎠⎞

⎜⎝⎛

′∆′

×

×⎟⎠⎞

⎜⎝⎛ ′−′

zz

xx

zyfkxf

sliceslit

,,2

0

πλ


Applications: 3D holographic hyperspectral imaging

( )λ,,, zyxf spectral components ofselected voxel are spread

out on detector plane


Applications: 3D multiplex hyperspectral imaging

( )λ,,, zyxf spectral components ofmultiplemultiple voxelss areare spread

out on nonnon--overlapping locationsoverlapping locationsinin detector plane


Applications: probing unknown 3D holograms

⎟⎠⎞

⎜⎝⎛ −

πλδ2

0fkx ( )yxI ′′,

?

( )zyx ′′′′′′ ,,ε

9


Optical imaging with 3D pupil

( )00 , yyxx −−δ ( )00 ,;, yxyxh ′′( )zyx ′′′′′′ ,,ε



( )00 , yyxx −−δ ( )00 ,;, yxyxh ′′

( ) { } ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+′−

+⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+⎟⎟

⎠

⎞⎜⎜⎝

⎛ ′+ℑ=′′

22

22

21

20

20

21

0

21

000 22

1,1,1,;,f

yxf

yxfy

fy

fx

fxyxyxh

λλλε

( )zyx ′′′′′′ ,,ε



( )00 , yyxx −−δ ( )00 ,;, yxyxh ′′

( ) { } ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+′−

+⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+⎟⎟

⎠

⎞⎜⎜⎝

⎛ ′+ℑ=′′

22

22

21

20

20

21

0

21

000 22

1,1,1,;,f

yxf

yxfy

fy

fx

fxyxyxh

λλλε

( )zyx ′′′′′′ ,,ε



( )0xx −δ ( )0; xxh ′

( ) { } ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′−⎟⎟

⎠

⎞⎜⎜⎝

⎛ ′+ℑ=′

22

2

21

20

21

00 22

1,1;f

xf

xfx

fxxxh

λλε

( )zx ′′′′ ,ε

simplify to 1D lateral dimension + longitudinal dimension


Optical imaging with 3D pupil: building the Hopkins matrix

( )xδ ( )0;xh ′( )zx ′′′′ ,ε

L

θs

( ) ( ) [ ]( ) 2ss cossinexpexp, θθε zxikikzzx ++=′′′′

Plane-to-plane wave hologramOn-axis reference



( )xδ ( )0;xh ′( )zx ′′′′ ,ε

L

θs

10



( )xδ ( )0;xh ′

L

θs



( )xx d−δ ( )xxh d;′

L

θs



( )xx d2−δ ( )xxh d2;′

L

θs



( )0; xxh ′

L

θs

( )0xx −δ

x′

0x



( )0; xxh ′

L

θs

( )0xx −δ

x′

0x

Hopkinsmatrix


Incoherent imaging with 3D pupil

L

θs

object:incoherent

superpositionof point sources

( )( )( )( )( )

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−M

M

xfxf

fxfxf

d2d-0dd2 ( )

( )( )

( )( )

?

d2d0

dd2

=

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

′−

′−

′′

M

M

xgxg

gxgxg

image

11



L

θs

( )( )( )( )( )

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−M

M

xfxf

fxfxf

d2d-0dd2

object:incoherent

superpositionof point sources

object



L

θs

object

( )( )( )( )( )

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−M

M

xfxf

fxfxf

d2d-0dd2

Hopkins matrix



L

θs

object

( )( )( )

( )( )

=

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

′−

′−

′′

M

M

xgxg

gxgxg

d2d0

dd2 ( )

( )( )( )( )

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−M

M

xfxf

fxfxf

d2d-0dd2

image Hopkins matrix


Incoherent imaging with clear (2D) pupil

object

( )( )( )

( )( )

=

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

′−

′−

′′

M

M

xgxg

gxgxg

d2d0

dd2 ( )

( )( )( )( )

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−M

M

xfxf

fxfxf

d2d-0dd2

image Hopkins matrix


The Hopkins matrix of 3D pupilsL

intra

page

cros

stal

k



interpage crosstalk(on-axis pixel)

12



interpage crosstalk

(pixel @ Gaussian image)



interpage crosstalk


pk

K

dk

zkd∆

x

z



interpage crosstalk


pk

K

dk

zkd∆

x

z

L1

∝

main lobe width



interpage crosstalk


intra

page

cros

stal

k

interpage crosstalk(on-axis pixel)



θs



θs

intra

page

cros

stal

k

interpage crosstalk

(pixel @ Gaussian image)interpage crosstalk

(on-axis pixel)

13


Defocused response of 3D pupils

( )00 ,, zzyxx −−δ ( )00 ,;, zxyxq ′′( )zyx ′′′′′′ ,,ε

?


Defocused response of 3D pupils

?

( )00 ,, zzyxx −−δ ( )00 ,;, zxyxq ′′( )zyx ′′′′′′ ,,ε

( ) ( ) ( )⎭⎬⎫

⎩⎨⎧ ′′+−×

⎭⎬⎫

⎩⎨⎧ ′′+′′

⎭⎬⎫

⎩⎨⎧=′′′′′′ ∫∫ 2

1

22

1

exp2exp,dd2exp,,f

zyxif

yyxxiyxypxzizyxPλ

πλ

πλ

π

( ) ( ) ( )zyxPzyxzyxg ′′′′′′×′′′′′′=′′′′′′ ,,,,,, ε

( ) ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+′−

′′=′′

22

22

22 211,,,

fyx

fy

fxGyxq

λλλ

( ) ( )0Fresnel, zyxp =

( )yxp ,


Blinking spot of 3D pupil



( )zyxf ,,

⎟⎠⎞

⎜⎝⎛∆

×⎟⎠⎞

⎜⎝⎛

′∆′

×

×⎟⎠⎞

⎜⎝⎛ ′−′

zz

xx

zyfkxf

sliceslit

,,2

0

πλ

3D pupil


3D holographic imaging: overview of experiments

1m1m 1cm1cm 100100μμmm 11μμmm 10nm10nm100m100mObject sizeObject size

Working distanceWorking distance

1mm1mm

10cm10cm

1m1m

1km1km

……

1cm1cm


3D optics for imaging: summary



1mm1mm

10cm10cm

1m1m

1km1km

Microscopy

……

1cm1cm

Ladar/Lidar

14





1mm1mm

10cm10cm

1m1m

1km1km

……

1cm1cm

Laser Line illumination

Laser Line illumination(inclined)

+ telephoto objective





1mm1mm

10cm10cm

1m1m

1km1km

……

1cm1cm

Rainbow illumination





1mm1mm

10cm10cm

1m1m

1km1km

……

1cm1cm

White Line illumination

Sun Light illumination+ Radon transform





1mm1mm

10cm10cm

1m1m

1km1km……

1cm1cm

fluorescent object+ multiplex imaging





1mm1mm

10cm10cm

1m1m

1km1km

……

1cm1cm

Laser Line illumination+ multiplex imaging

+ Viterbi post-processing



1m1m 1cm1cm 100100μμmm 11μμmm 10nm10nm100m100m

Plankton(individual)

Object sizeObject size


1mm1mm

10cm10cm

1m1m

1km1km

……

1cm1cm

GOAL

GOAL

15





1mm1mm

10cm10cm

1m1m

1km1km

Microscopy

……

1cm1cm

Imaging properties:Imaging properties:

• profilometric (2½D) or tomographic (3D) • no or minimal scanning

• laser or white light illumination• hyperspectral

GOAL

GOAL

Ladar/Lidar


3D holographic hyperspectral imaging

( )λ,,, zyxf spectral components ofselected voxel are spread

out on detector plane


4D imaging (3D spatial + spectral) ; White Line illumination

Tilted, flat objectSpectral plane

Grating

#1#1

#1

Volume hologram CCD cameraf1

f2f3 f4

invert

1.5o

0.5mm


4D imaging: data analysis

lateral x

lateral x (pixel #)

inte

nsity

I

heig

ht z

(mm

)

lateral x (pixel #)

invert



lateral x

lateral x (pixel #)

inte

nsity

I

heig

ht z

(mm

)

lateral x (pixel #)

dirt on mirrorsurface

invert



lateral x

lateral x (pixel #)

inte

nsity

I

heig

ht z

(mm

)

lateral x (pixel #)

accuracyaccuracy

Accuracy < 20µm

invert

16


4D imaging: depth discrimination in multiple spectral bands

Brightness ≡

Surface profile

(Depth)

( ) ( )( ) ⎟⎟

⎠

⎞⎜⎜⎝

⎛∝

λκ

2NAhump zzI

Color


Probing unknown 3D holograms

⎟⎠⎞

⎜⎝⎛ −

πλδ2

0fkx ( )yxI ′′,

?

( )zyx ′′′′′′ ,,ε


Diffraction from deformed 3D holograms


Experimental arrangement: transmission geometry

3D


Experiment: diffraction from indented 3D hologram


Fringe deformation due to indenter (transmission)indenter tip

17


Deformed transmission 3D hologram in k-space






Experimental arrangement: reflection geometry

3D


Experiment: diffraction from indented 3D hologram


Fringe deformation due to indenter (reflection)

z

indenter tip

18


Deformed reflection 3D hologram in k-space


The Conundrum: Making 3D lenses

photosensitivematerial

referencepoint source

of

sθ

plane wavesignal beam

referenceplane

• Exotic materials• Coherent 3D exposure

Need: • Established materials• Incoherent 3D exposure

3D lithography? 3D lithography?


Nanostructured Origami™ 3D Fabrication & Assembly


Fabrication Results Origami Theory

Precision Alignment using NanomagnetsEdge-to-Face Latching Demonstration

Corner Cube

100 µm

mµ2

NS

N

S

SN

SN

1 nN

Field Lines and Forces

(Provisional Patent filed 8/30/05)

Magnets

Membrane

Alignment Force

Animation of alignment


Nanostructured Origami™ Super Capacitor

dA

VQC ε==

++++++++++

-

Electrolyte

_________

ElectrodeElectrode

C C

Fabricated deviceSEM of carbon paint

Measured Capacitance: 1.0μFMeasured Capacitance density: 15F/g

─

++++

───

_


Summary

• 3D optics

– defocus ↔ contrast

– rainbow color management

– 3D spatial heterodyning

– applications in sensing & imaging

• 3D spatial heterodyning with subwavelength features?

19


Acknowledgments

Students

• Arnab Sinha

• Wenyang Sun

• Kehan Tian

• Se Baek Oh

• Tina Shih

Co-Principal Investigators

• Demetri Psaltis Caltech

• Christophe Moser Ondax

• Wehai Liu Ondax

• Mark Neifeld U Arizona

• Wojciech Matusik MERL

• Henry I. Smith MIT

• Yang Shao-Horn MIT

• Funding

• Will Arora

• Hyun Jin In

• Paul Stellman

• Tony Nichol

• Nader Shaar

2d optics 3d optics 3d optics - institute for mathematics ... · *caltech optical info. proc. group...

Documents