Introduction to Computer Vision

CS223B, Winter 2005

1/25/2005 Introduction to Computer Vision 2

Richard Szeliski – Guest Lecturer

• Ph. D., Carnegie Mellon, 1988• Researcher, Cambridge Research

Lab at DEC, 1990-1995• Senior Researcher, Interactive

Visual Media Group, Microsoft, 1995-• Research interests:

• computer vision (stereo, motion),computer graphics (image-based rendering), parallel programming

What is Computer Vision?

1/25/2005 Introduction to Computer Vision 4

What is Computer Vision?

• Image Understanding (AI, behavior)• A sensor modality for robotics• Computer emulation of human vision• Inverse of Computer Graphics

Computervision

World model

Computergraphics

World model

1/25/2005 Introduction to Computer Vision 5

Intersection of Vision and Graphics

modeling- shape- light- motion- optics- images IP

animation

rendering

user-interfaces

surface design

Computer Graphics

shape estimation

motion estimation

recognition

2D modeling

modeling- shape- light- motion- optics- images IP

Computer Vision

1/25/2005 Introduction to Computer Vision 6

Computer Vision [Trucco&Verri’98]

1/25/2005 Introduction to Computer Vision 7

Image-Based Modeling

Images (2D)Geometry (3D)

shapePhotometryappearance+

graphics

vision

image processing

2.1 Geometric image formation

2.2 Photometric image formation

3 Image processing

4 Feature extraction

5 Camera calibration

6 Structurefrom motion

7 Image alignment

8 Mosaics

9 Stereo correspondence

11 Model-based reconstruction

12 Photometric recovery

14 Image-based rendering

1/25/2005 Introduction to Computer Vision 11

Syllabus

Image Transforms / Representations• filters, pyramids, steerable filters• warping and resampling• blending• image statistics, denoising/coding• edge and feature detection

1/25/2005 Introduction to Computer Vision 12

Image Pyramid

Bandpass Images

Lowpass Images

1/25/2005 Introduction to Computer Vision 13

Pyramid Blending

1/25/2005 Introduction to Computer Vision 14

Parametric (global) warping

Examples of parametric warps:

translation rotation aspect

affineperspective

cylindrical

1/25/2005 Introduction to Computer Vision 15

Syllabus

Optical Flow• least squares regression• iterative, coarse-to-fine• parametric• robust flow and mixture models• layers, EM

1/25/2005 Introduction to Computer Vision 16

Image Morphing

1/25/2005 Introduction to Computer Vision 17

Syllabus

Projective geometry• points, lines, planes, transforms

Camera calibration and pose• point matching and tracking• lens distortion

Image registration• mosaics

1/25/2005 Introduction to Computer Vision 18

Panoramic Mosaics

+ + … + =

1/25/2005 Introduction to Computer Vision 19

Syllabus

3D structure from motion• two frame techniques• factorization of shape and motion• bundle adjustment

1/25/2005 Introduction to Computer Vision 20

3D Shape Reconstruction

Debevec, Taylor, and Malik, SIGGRAPH 1996

1/25/2005 Introduction to Computer Vision 21

Face Modeling

1/25/2005 Introduction to Computer Vision 22

Syllabus

Stereo• correspondence• local methods• global optimization

1/25/2005 Introduction to Computer Vision 23

View Morphing

Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’96]

1/25/2005 Introduction to Computer Vision 24

Z-keying: mix live and synthetic

Takeo Kanade, CMU (Stereo Machine)

1/25/2005 Introduction to Computer Vision 25

Virtualized RealityTM

Takeo Kanade, CMU• collect video from 50+ stream

reconstruct 3D model sequences

http://www.cs.cmu.edu/afs/cs/project/VirtualizedR/www/VirtualizedR.html

1/25/2005 Introduction to Computer Vision 26

Virtualized RealityTM

Takeo Kanade, CMU• generate new video

• steerable version used for SuperBowl XXV“eye vision” system

1/25/2005 Introduction to Computer Vision 27

Syllabus

Tracking• eigen-tracking• on-line EM• Kalman filter• particle filtering• appearance models

1/25/2005 Introduction to Computer Vision 28

Syllabus

Recognition• subspaces and local invariance features• face recognition• color histograms• textures

Image editing• segmentation• curve tracking

1/25/2005 Introduction to Computer Vision 29

Edge detection and editing

Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain," Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374-381, June, 1998.

1/25/2005 Introduction to Computer Vision 30

Image Enhancement

High dynamic range photography[Debevec et al.’97; Mitsunaga & Nayar’99]• combine several different exposures together

1/25/2005 Introduction to Computer Vision 31

Syllabus

Image-based rendering• Lightfields and Lumigraphs• concentric mosaics• layered models• video-based rendering

1/25/2005 Introduction to Computer Vision 32

Concentric Mosaics

Interpolate between several panoramas to give a 3D depth effect

[Shum & He, SIGGRAPH’99]

1/25/2005 Introduction to Computer Vision 33

Applications

• Geometric reconstruction: modeling, forensics, special effects (ILM, RealVis,2D3)

• Image and video editing (Avid, Adobe)• Webcasting and Indexing Digital Video

(Virage)• Scientific / medical applications (GE)

1/25/2005 Introduction to Computer Vision 34

Applications

• Tracking and surveillance (Sarnoff)• Fingerprint recognition (Digital Persona)• Biometrics / iris scans (Iridian Technologies)• Vehicle safety (MobilEye)• Drowning people (VisionIQ Inc)• Optical motion capture (Vicon)

1/25/2005 Introduction to Computer Vision 35

Projects

Let’s look at what students have done in previous years …

Stanford  http://www.stanford.edu/class/cs223b/winter01-02/projects.html

CMU  http://www-2.cs.cmu.edu/~ph/869/www/869.html

UW http://www.cs.washington.edu/education/courses/cse590ss/01wi/

GA Tech  http://www.cc.gatech.edu/classes/AY2002/cs4480_spring/