Survey of IR/TOF/3D Imaging

SystemsSystems

Manu Suryavansh

4/18/2012

Topics• History and Present –

– LIDAR

– Lock-In amplifier and pixel

– CCD

– CMOS Imagers– CMOS Imagers

– Infrared communication System – IRDA

– TOF and 3D Imaging

– SAPD

• Some major companies -

Depth/Range Sensor Applications

• 3D Imaging

• Gaming

• Auto Focus in Digital camera

• Object detection• Object detection

• Robotics

Distance/Depth Measurement

TechniquesContactless

distance measurement

Triangulation Interferometry Time of Flight

Pulse

Continuous Wave

Modulation

TOF Techniques

Depth/TOF Techniques - Triangulation

• Determine the depth of a single point in the scene.

• Used in sharp analog distance sensor (20 – 150 cm range)

Depth/TOF Techniques - Passive

Stereoscopy

• Single point depth estimation using two

cameras.

Living Images – Light Field

Camera/Plenoptic Camera

• Recently released to market by Lytro.

• It uses a microlensarray to capture 4D array to capture 4D light field.

• Depth information can also be extracted.

• Pictures can be focused after they are taken!!

Time of Flight - Wikipedia

• Applications –

– Calculate Electron mobility

– Mass spectrometry

– Near Infrared spectroscopy – property of material– Near Infrared spectroscopy – property of material

– Ultrasonic flow meter - velocity of fluids

– 3D Imaging/Depth Sensor

TOF -1

Complex and Big

Vs

Demodulating Pixel

Vs

Ref – Canesta Inc – 2004 IEEE Computer Society

Basic 3D TOF Imaging Principle

Phase Measurment

Simple Demodulating Pixel

Reference - Industrial Robot Journal

Demodulating IR Pixel

• Sending two pulses with 0 and 90 degree

phase.

• Performing DFT on the demodulated to get

phase.phase.

Noise Sources in photo detectors

CMOS Imager

CMOS Active Pixel Structure (APS)

• Color Image Sensor consists of a photo sensor and a color filter.

• There are three types of photo sensors –photodiode, bipolar phototransistor and photo FET.FET.

• There are several types of color filters like – Bayer sensor, Foveon X3 Sensor etc.

• Amplifier

• CMOS Imagers have relatively very small pixels (5um x 5um and even smaller).

SAPD – Single Photon Avalanche

Photodetector Camera

• An avalanche photodiode (APD) is a highly

sensitive semiconductor electronic device that

exploits the photoelectric effect to convert

light to electricity – Wikipedia.light to electricity – Wikipedia.

Lock-In Amplifier/System

• A lock-in amplifier system takes a periodic

reference signal and a noisy signal as input

signals to a phase sensitive detector.

• Reference Signal – r(t)=sin(2*pi*fo*t)• Reference Signal – r(t)=sin(2*pi*fo*t)

• Input Signal – i(t)=Asin(2*pi*fo*t + Q) + n(t)

• Output – P(t) = r(t)*i(t) = ½Acos(Q) - other

LOCK-In CCD/Photodiode

• The lock-in CCD is based on synchronous

photo charge detection and storage.

• Using 4 samples offset, phase and amplitude

can be determined. can be determined.

• Integration time – d1

Single Pixel TOF

Single Pixel + Demodulator

Gain ADC DFT/FFT

TOF Cont

Pixel ADC using SDM

Digital/Analog CDS can be used to cancel Ambient light/noise.

Example - A Range Image Sensor Based on 10-µm Lock-In

Pixels in 0.18µm CMOS Imaging Technology – ISSCC-2010

Ref – D. Stoppa – ISSCC-2010

Example -1, cont

Example2 – A 16x16 Pixel Distance sensor with In-Pixel circuitry

that tolerates 150klx of ambient light – JSSC-2010-Gerald Zach

• Range – 0 to 3.2m

• Standard deviation – few cm’s

• Linearity error -> +/-2cm

• Process – 0.6um BiCMOS• Process – 0.6um BiCMOS

• Pixel size – 16um x 16um

• Illumination source – 4 LED’s of 850nm

• Power Consumption – 10W

Example2- cont.

Some companies working on TOF/3D

– Canesta/Microsoft

– Primesense

– SoftKinetic

– PMDTech– PMDTech

– Mesa Imaging

PMDVision- CamBoard Nano

Infrared Remote Control Systems

Vishay/Sharp makes parts like these

Carrier Frequency – 30KHz to 56KHz

Wavelength – 940nm

Wireless Infrared Communications

IrDA supports 4Mb/s data rate.

Cmos Optical Preamplifier with ambient light cancellation- Ref – Phang and Johns – IEEE-1999

Ambient Light Cancellation - 1993

Wally did this app note almost 20 years ago!

Ultrasonic TOF Based depth sensor

• Time based as speed of sound travels much

slower.