04/18/23 CAM Talk G.Kamberova

Computer VisionIntroduction

Gerda Kamberova

Department of Computer Science

Hofstra University

04/18/23 CAM Talk G.Kamberova

What is Computer Vision?

• Trucco and Verri– computing properties of the 3D world from one or more digital

images

• Stockman and Shapiro– To make useful decisions about real physical objects and scenes

based on sensed images

• Ballard and Brown– The construction of explicit, meaningful description of physical

objects from images

• Forsyth and Ponce– ...extracting descriptions of the world from pictures or sequences of

pictures

04/18/23 CAM Talk G.Kamberova

From 2D images to 3D model

04/18/23 CAM Talk G.Kamberova

General Rule

If you can’t understand (i.e. model)the forward process, you will have

a hard time solving the inverse!

04/18/23 CAM Talk G.Kamberova

What Information is in Images?

04/18/23 CAM Talk G.Kamberova

What information is in the image?

04/18/23 CAM Talk G.Kamberova

Typical Sensor: A CCD Camera• Basic process:

– photons hit a detector

– the detector becomes charged

– the charge is read out as brightness

– high sensitivity

– Noise issues

                

           

04/18/23 CAM Talk G.Kamberova

Pixel

Binary1 bit

Grey1 byte

Color3 bytes

Image Representation

Each pixel is a measure of the brightness (intensity of light)that falls on an area of an sensor (typically a CCD chip)

04/18/23 CAM Talk G.Kamberova

Problems of Computer Vision: Radiometry

What are the physical and geometric processes thatgovern (digital) imaging?

04/18/23 CAM Talk G.Kamberova

Problems of Computer Vision: Imaging Geometry

What are the physical and geometric processes thatgovern (digital) imaging?

04/18/23 CAM Talk G.Kamberova

Some Related Terms

• Image Processing: the study of the properties of operators that produce images from other images

• Machine Vision: a somewhat outdated term which now tends to refer to industrial vision applications where (usually) a single camera is used to solve a structured inspection task

• Pattern Recognition: typically refers to the recognition of structures in 2D images (usually without reference to any underlying 3D information).

• Photogrammetry: the science of measurement though non-contact sensing, e.g. terrain maps from satellite images. Usually is more focused on accuracy issues than interpretation.

04/18/23 CAM Talk G.Kamberova

Computer Vision vs. Graphics

• Computer Graphics– Produce “plausible” images– You choose the models, conditions, imaging parameters, etc.

• Computer Vision– Given real images with noise, sampling artifacts …– Estimate physically quantities– Ill-posed ---- what is the minimum world knowledge we need?

Is Vision the “Inverse” of Graphics?

04/18/23 CAM Talk G.Kamberova

From 2D images to 3D model and back

04/18/23 CAM Talk G.Kamberova

Image Filter Result

Problems of Computer Vision: Feature Extraction

What are the “informative” areas of an image and how do we detect them?

04/18/23 CAM Talk G.Kamberova

Filter kernels that are larger see effects at coarserscales

Problems of Computer Vision: Feature Extraction

04/18/23 CAM Talk G.Kamberova

Problems of Computer Vision: Feature Extraction – edge detection

04/18/23 CAM Talk G.Kamberova

Computer Vision vs. Image Processing

• Image Processing– Mostly concerned with image-to-image transformations

• Filtering• Enhancement• Compression

• Computer Vision– Concerned with how images reflect the 3D world

• Filtering for feature extraction• Enhancement for recognition/detection• Compression that preserves geometric information in

images

04/18/23 CAM Talk G.Kamberova

Problems of Computer Vision: Segmentation and Grouping

What portionsof an image pertainto one another andto relevant physicalphenomena?

04/18/23 CAM Talk G.Kamberova

Computer Vision vs. Human Vision

What is the rightsegmentation?To us it seemsobvious …“common sense”

04/18/23 CAM Talk G.Kamberova

Grouping

04/18/23 CAM Talk G.Kamberova

Illusions

04/18/23 CAM Talk G.Kamberova

Illusions

04/18/23 CAM Talk G.Kamberova

Stereo(psis)

• Stereo • Recovers information about 3D structure• 2 images of the same scene (different viewpoints)

04/18/23 CAM Talk G.Kamberova

04/18/23 CAM Talk G.Kamberova

Autostereogram

04/18/23 CAM Talk G.Kamberova

Computer Vision Problems: Stereo

• Correspondence (matching):• From two (or more) images,

determine the geometry ofhe scene by matching corresponding areas ofthe images

•Reconstruction: from 2D matched elements to 3D pointshe.

RIGHT IMAGE PLANE

LEFT IMAGE PLANE

04/18/23 CAM Talk G.Kamberova

THE ORGANIZATION OF AN IMAGE SEQUENCE

Frames

04/18/23 CAM Talk G.Kamberova

THE MOTION FIELD

The “instantaneous” velocity of points in an image

The focus of expansion

1. Direction of motion 2. Time to collision

04/18/23 CAM Talk G.Kamberova

MOVING CAMERAS ARE LIKE STEREO: Structure from Motion

Locations ofpoints on the object(the “structure”)

The change in spatial locationbetween the two cameras (the “motion”)

04/18/23 CAM Talk G.Kamberova

Problems of Computer Vision: Recognition

Given a database ofobjects and an imagedetermine what, if anyof the objects are present in the image.

04/18/23 CAM Talk G.Kamberova

Problems of Computer Vision: Recognition

Given a database ofobjects and an imagedetermine what, if anyof the objects are present in the image.

04/18/23 CAM Talk G.Kamberova

Applications of Computer Vision: Biometrics

• Face recognition

• Iris scanning

• Fingerprint recognition

• Activity recognition

• 3D biometric

04/18/23 CAM Talk G.Kamberova

Applications of Computer Vision: Medical Imaging

04/18/23 CAM Talk G.Kamberova

Examples: Virtual colonoscopy

Patient: data, imaging, rather non-invasive

04/18/23 CAM Talk G.Kamberova

Applications of Computer Vision: HCI

calculator

GUI Face trackerGesture recognition

04/18/23 CAM Talk G.Kamberova

Applications of Computer Vision: Image Databases

(Courtesy D. Forsyth & J. Ponce)

From a searchfor horse pixin 100 horseimages and1086 non-horseimages

04/18/23 CAM Talk G.Kamberova

Applications of Computer Vision:

3D model aquisition(Jitendra Malik, Berkeley)

04/18/23 CAM Talk G.Kamberova

Applications of Computer Vision: Motion Control

04/18/23 CAM Talk G.Kamberova

CMU: Vitrualized Reality

04/18/23 CAM Talk G.Kamberova

Tele-Immersion: UPenn

is the technology that enables remotely located users to share the same real or virtual environments in real time.

04/18/23 CAM Talk G.Kamberova

Teleimmersion

• sensing of the real world

• interaction with the real world

• visualization of real and synthetic data

• networking

• Why 3D?

• Collaborative design

• Telemedicine

• Training in 3D

• Visualization of complex data

• Entertainment

04/18/23 CAM Talk G.Kamberova

Tele-immersion for the Common Citizen

04/18/23 CAM Talk G.Kamberova

Teleimmersion: Collaboration and Working in Virtual 3D Worlds(UPenn, UNC)

04/18/23 CAM Talk G.Kamberova

Computer Vision Problems: 3D reconstruction

A Reconstructed Depth Map

04/18/23 CAM Talk G.Kamberova

Polynocular Stereo

• From images to unorganized 3D point cloud

04/18/23 CAM Talk G.Kamberova

Surface Reconstruction from unorganized 3D point cloud

Orient Locally Reconstruction by polyhedral approximation

Compute 3D shape ifrom the polyhedral surface: mean and Gauss curvatures

04/18/23 CAM Talk G.Kamberova

3D Shape recovery: applications in registration and matching of

surfaces/objects

• G^2 Kamberov: Ongoing program to associate geometric descriptors and invariants directly to an unorganized oriented cloud of points

• Without polyhedral approximation, or model fitting

04/18/23 CAM Talk G.Kamberova

Another cloud

04/18/23 CAM Talk G.Kamberova

Example: Principal directions

04/18/23 CAM Talk G.Kamberova

3D shape: the mean curvature surface

04/18/23 CAM Talk G.Kamberova

3D shape from medical data

04/18/23 CAM Talk G.Kamberova

Medical Data 2