stockman cse/msu fall 2005 models and matching methods of modeling objects and their environments;...

Stockman CSE/MSU Fall 2005

Models and Matching

Methods of modeling objects and their environments;

Methods of matching models to sensed data for recogniton

Stockman CSE/MSU Fall 2005

Some methods to study Mesh models (surface) Vertex-edge-face models (surface) Functional forms: superquadrics (surface) Generalized cylinders (volume) Voxel sets and octrees (volume) View class models (image-based) Recognition by appearance (image-based) Functional models and the Theory of

affordances (object-oriented)

Stockman CSE/MSU Fall 2005

Models are what models do

Stockman CSE/MSU Fall 2005

What do models do?

Stockman CSE/MSU Fall 2005

Vertex-edge-face models

Polyhedra and extensions

Stockman CSE/MSU Fall 2005

Vertex-Edge-Face model

Stockman CSE/MSU Fall 2005

Sample object

All surfaces are planar or cylindrical

Stockman CSE/MSU Fall 2005

Matching methods Hypothesize point correspondences Filter on distances Compute 3D alignment of model to

data Verify positions of other model

points, edges, or faces LOTS of work in the literature on this!

Can work for many industrial objects (and human faces perhaps!)

Stockman CSE/MSU Fall 2005

Triangular meshes

Stockman CSE/MSU Fall 2005

Texture-mapped mesh dog

Courtesy of Kari PuliWith each triangle is a mapping of its vertices into pixels [r, c] of a color image. Thus any point of any triangle can be assigned a color [R, G, B]. There may be several images available to create these mappings.

Stockman CSE/MSU Fall 2005

Meshes are very general

They are usually verbose and often are too detailed for many operations, but are often used in CAD

Stockman CSE/MSU Fall 2005

Modeling the human body for clothing industry and …

Multiple Structured light scanners used: could this be a service industry such as Kinkos?

Actually cross sections of a generalized cylinder model.

Stockman CSE/MSU Fall 2005

Mesh characteristics

+ can be easy to generate from scanned data

Stockman CSE/MSU Fall 2005

Making mesh models

Stockman CSE/MSU Fall 2005

Physics-based models

Can be used to make meshes;Meshes retain perfect

topology;Can span spots of bad or no

data

Stockman CSE/MSU Fall 2005

Physics-based modeling

Stockman CSE/MSU Fall 2005

Forces move points on the model; halt at scanned data

Stockman CSE/MSU Fall 2005

Fitting an active contour to image data

Stockman CSE/MSU Fall 2005

Balloon model for closed object surface

Courtesy of Chen and Medioni

Stockman CSE/MSU Fall 2005

Balloon evolution• balloon stops at data points

• mesh forces constrain neighbors

• large triangles split into 4 triangles

• resulting mesh has correct topology

Stockman CSE/MSU Fall 2005

Physics-based models

Can also model dynamic behavior of solids (Finite Element Methods)

Stockman CSE/MSU Fall 2005

Algorithms from computer graphics make mesh models from blobs

Marching squares applied to some connected image region (blob)

Marching cubes applied to some connected set of voxels (blob)

See a CG text for algorithms: see the visualization toolkit for software

Stockman CSE/MSU Fall 2005

Volume model: voxels, octrees

Stockman CSE/MSU Fall 2005

Simple object and its octree

Stockman CSE/MSU Fall 2005

Generalized cylinders

Stockman CSE/MSU Fall 2005

Generalized cylinders

• component parts have axis

• cross section function describes variation along axis

• good for articulated objects, such as animals, tools

• can be extracted from intensity images with difficulty

Stockman CSE/MSU Fall 2005

Extracting a model from a segmented image region

Courtesy of Chen and Medioni

Stockman CSE/MSU Fall 2005

Interpreting frames from video

Can we match a frame region to a model?

What about a sequence of frames? Can we determine what actions

the body is doing?

Stockman CSE/MSU Fall 2005

Generalized cylinders

Stockman CSE/MSU Fall 2005

View class models

Objects modeled by the distinct views that they can

produce

Stockman CSE/MSU Fall 2005

“aspect model” of a cube

Stockman CSE/MSU Fall 2005

Recognition using an aspect model

Stockman CSE/MSU Fall 2005

View class model of chair

2D Graph-matching (as in Ch 11) used to evaluate match.

Stockman CSE/MSU Fall 2005

Side view classes of Ford Taurus (Chen and Stockman)

These were made in the PRIP Lab from a scale model.

Viewpoints in between can be generated from x and y curvature stored on boundary.

Viewpoints matched to real image boundaries via optimization.

Stockman CSE/MSU Fall 2005

Matching image edges to model limbs

Could recognize car model at stoplight or gate.

Stockman CSE/MSU Fall 2005

Appearance-based models

Using a basis of sub images;Using PCA to compress bases;

Eigenfaces (see older .pdf slides 14C)

Stockman CSE/MSU Fall 2005

Function-based modeling

Object-oriented;What parts does the object have;

What behaviors does it have;What can be done with it?

(See plastic slides of Louise Starks’s work.)

Stockman CSE/MSU Fall 2005

Theory of affordances: J.J. Gibson

An object can be “sittable”: a large number of chair types, a box of certain size, a trash can turned over, …

An object can be “walkable”: the floor, ground, thick ice, bridge, ...

An object can be a “container”: a cup, a hat, a barrel, a box, …

An object can be “throwable”: a ball, a book, a coin, an apple, a small chair, …