multimodal interfaces
DESCRIPTION
Ignite Columbus presentation by Thomas Winningham 11/13/08 http://IgniteCbus.comTRANSCRIPT
Multimodal InterfacesNovember, 2008Thomas Winningham
Multi ah-wha?
๏ Computer vision, mostly
๏ The Wiimote 100hz camera
๏ Capacitive surfaces (or lots of buttons)- IPhone & Mac touchpad- HP TouchSmart
๏ Tin Foil & an Arduino
๏ Force transducers, the Wii Fit
๏ Teaching new dogs old tricks
Regions of interest
๏ At 30 fps or faster:- Find a finger smudge (blobs)- Dots of light- Triangulate based on closest data
- Color keying- LEDs as two-way- Use IR for spectrum separation- Isolate features with laser- Map in 3D using Lidar
Better Software & Hardware
๏ DARPA Challenge- Intel OpenCV
๏ Multiprocessor Machines
๏ Home clusters of machines
๏ GPU processors on videocards
๏ Better web cams
๏ Cheaper, more widely available materials
๏ Easier access to information
Fresh Ways to Interface with Computers
It is not a panacea, but at least everyone is mostly invited.
Leave it to the big boys?
๏ IPhone's - And knock-offs
๏ Microsoft Surface & Windows 7 & Tablets- Microsoft Research
๏ HP TouchSmart
๏ Perceptive Pixel
๏ NUI Group - step 1. cut a hole in a box
Gesturing
๏ Combos!
๏ Changes over time
๏ Inferring about the placement of input in orientation to the real world- Surface- Wii fit- You're holding a nail near tinfoil
Simple!
๏ Or at least, reasonably simple given the complexity of what is going on.
๏ Small blocks of code in repition
๏ Again, faster computers
Web Cam in a Box
๏ Step #2- Put your junk in that box
Double true!
๏ Stackable cubes
๏ Large walls
๏ Small hobby projects
๏ Multiple cameras
๏ GPUs
๏ Networked machines
๏ Store Windows, floors , buildings
Mostly flash
๏ (needs a lot of help from c)
๏ But also:- Python- Processing- Max- VVVV- Pure Data- .NET- Firefox / Javascript (Lily)
Rethinking a lot of things
๏ Gesture navigation
๏ User Interface
๏ Feedback
Guerrilla Arm
All kinds of things I tell ya!
๏ Sphere
๏ Gesturing with your hands above the home row
๏ Or just using your hands
๏ Or moving your head, or eyes, or thoughts
๏ This is an extra bullet point. Ha!
Meanwhile, back at the bat cave...๏ Home clustering
๏ Semistructured web and public data
๏ HDTV's, projectors, err... iphones
๏ Social connectivity- Except in the workplace where we'd generally
like everyone to politely sod off
๏ Advanced modeling, free software for fluid dynamics and physics simulation
Lack of standards?
lack of patent free ideas, maybe
TUIO / OSC for ROI or Fiducials
Romper room
๏ I see:- NUI Group- Open Frameworks- Processing- Apple Developer forums- IPhone jailbreakers- Multi-pointer X-Windows, Compiz, and the
whole open-source world.
Feedback loop
๏ Proliferation? Adoption?
๏ Social implications
๏ This will all spawn something else
๏ Everybody's coming to the net
๏ Voluntary mass hallucination (Gibson)
Come out and play!If yer under eighteen ya won't be doin any ty-ee-eye-ee-ime
There's a place for us!
๏ More people playing!- == better software
๏ Not as hard as it sounds, just steal code ;p
๏ Who's getting laid anyway?
๏ It is amazing what you can get done if you don't care who gets the credit. (jets ftw!)