stanford hci group / cs376 research topics in human-computer interaction vision-based interaction...
Post on 19-Dec-2015
218 views
TRANSCRIPT
stanford hci group / cs376
research topics in human-computer interaction http://cs376.stanford.edu
Vision-based Interaction
Scott Klemmer17 November 2005
2
cs547: Blake Ross and Asa Dotzler
Mozilla: Creating simple software in a geek-driven culture
3
The first vision-based interface Myron Krueger used computer
vision to create Responsive Environments (1970s)
“Reaction is the Medium” http://www.artmuseum.net/w2vr/
timeline/videoplace_video.html
4
How it works
Video and background are separated in analog using chroma key techniques(think broadcast news)
The first and last points of each raster are stored in the computer, and represent the person’s outline
5
Vision-based UIs: “Verbs” Detecting and Tracking elements of a
certain type in a scene Capturing contents of detected objects Recognizing individual members in an
object class
6
Vision-based UIs: “Verbs” Detecting and Tracking elements of
a certain type in a scene
7
Vision-based UIs: “Verbs” Capturing contents of detected
objects
8
Vision-based UIs: “Verbs” Recognizing individual members in a
class
9
Vision-based UIs: “Nouns” People (one or multiple)
Bodies Faces Hands
Documents Objects
10
Vision-based UIs: “Nouns” People (one or multiple)
Bodies Faces Hands
Documents Objects
11
Vision-based UIs: “Nouns” People (one or multiple)
Bodies Faces Hands
Documents Objects
12
Background SubtractionI N F R A S T R U C T U R E
13
Image Moments (of Inertia) 0th moment is mass
(total number of pixels)
14
Image Moments (of Inertia) 1st moment is center
15
Image Moments (of Inertia) 2nd moment is orientation
16
Tools for Vision apps Intel’s OpenCV
C API to highly optimized image processing functions (threshold, dilate, optical flow, …)
http://www.intel.com/research/mrl/research/opencv Fast to run! Slow to develop Great for vision folks; too low-level for app folks
Papier-Mâché Java API (and to some extent visual UI) for vision
(and other physical input) http://guir.berkeley.edu/papier-mache Fast to develop! Slow to run Great for app folks; ~5 fps can sometimes be too
slow
17
Good Vision Books
Computer Vision: A Modern Approach David Forsyth and Jean Ponce (2003) Fantastic book; but goal is more
theoretical understanding than practical application
Robot Vision Berthold Horn (1987) More focused on apps and interactive
algorithms Somewhat out of date