![Page 1: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/1.jpg)
Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa
Ishikawa-Namiki-LaboratoryParallel Processing for Sensory information
University of Tokyo
![Page 2: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/2.jpg)
• remove the need for any physical I/O space at all
• remove the need of any additional input device
…Next step ?
UI : “WIMP” + alphanumeric inputI/O Device: touch sensitive
screen+stylus
Today’s approach
nice: input space is merged with viewing space
I. The problem:
physical interaction
Input and interaction with
“ever-shrinking” portable
electronic devices
(*) windows, icons, menus and pointer
”ergonomics-independent” miniaturization
![Page 3: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/3.jpg)
How to remove the input space?
In a word: passive-vision input versatile but not mature yet…
…in any case overkill for the problem at hand! (“inputspace-less” WIMP)
• May require intensive computation • Not robust enough• Sensitive to external illumination conditions/backgrounds...
drawbacks:
(non-invasive techniques!)
...By providing the PDA with intelligent sight:
- handwritten character recognition- gestures recognition, etc
Low, medium or high-level vision
- hand/finger position
![Page 4: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/4.jpg)
Closer look at the problem...
-Hand/finger closer than anything else-fingertip always visible (no occlusion)-fingertip “shape” simple and stable
animal antenna
proximity sensor...
+ ...angular information
should provide sufficient data for
tracking!
cane for the blind
what a short-sighted PDA would actually
see…
cost-efficient, ROBUST, FAST!
…but cumbersome, annoying?
![Page 5: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/5.jpg)
- laser can be used as output device (on any external surface, including the retina: VRD)
And...
The proposed “antenna”:
a directive beam of light
advantages:
- active lighting: provides and control illumination
- real-time, precise 3D measures (telemetry)
- measurement of surface roughness, speed, even biometric data.
Inconvenients: no haptic feedback (just like passive vision, but visual cues possible).
![Page 6: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/6.jpg)
... but how realistic is the integration of a LASER RADAR on a PDA?
VERY! Thanks to today’s “MOEMS” technology...
The realm of “MOEMS” (micro-opto-electro-mechanical systems):
Electronic circuit, photodetectors, laser diodes sources and mirrors integrated in the SAME chip.
![Page 7: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/7.jpg)
II. Smart Laser Scanner Prototype
•laser diode,
•a pair of steering micro-mirrors,
•single non-imaging photodetector.
Hardware (discrete components):
![Page 8: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/8.jpg)
circular laser “saccade”
Tracking Principle
smart laser scanner:
laser excursion is locked around the area of interest
Tracking sequence repeated every
millisecond.
kHz refreshing rate explains algorithmic extreme simplicity
![Page 9: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/9.jpg)
Prototype Smart Scanner in Action
![Page 10: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/10.jpg)
2 DOF (using fingertip)
(angular precision: better than tenth of a degree)
![Page 11: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/11.jpg)
3 DOF (using fingertip)
Depth precision: (intensity based) around 1cm up to 2 m
![Page 12: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/12.jpg)
a little farther away …
![Page 13: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/13.jpg)
Simultaneous tracking
(up to four targets without hardware duplication)
![Page 14: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/14.jpg)
Several users…
![Page 15: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/15.jpg)
… unique user, higher DOF
![Page 16: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/16.jpg)
Achieving 6 DOF (using three tracking points)
![Page 17: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/17.jpg)
Fast 3d tracking of a ping-pong ball
![Page 18: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/18.jpg)
![Page 19: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/19.jpg)
Interactive laser annotation
![Page 20: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/20.jpg)
UIs based on this 3D Input Device:Proof-of-principle examples
![Page 21: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/21.jpg)
X/Y/Z translation
RIGTH CLICK
LEFT CLICK
Gestural command:
3D locator
![Page 22: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/22.jpg)
![Page 23: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/23.jpg)
X/Y translation
ZOOM
(-)
Interface for visualizing 2D data
LEFT CLICK
RIGHT CLICK
Gestural command:
Positional command:
ZOOM
(+)
![Page 24: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/24.jpg)
![Page 25: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/25.jpg)
X/Y translation
Fram
e
positio
n (-)
Zoom x2 Zoom /2
Gestural command:
Positional command:
Fram
e
positio
n
(+)
Visualization interface for video
![Page 26: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/26.jpg)
![Page 27: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/27.jpg)
Conclusion
The problem: design a 3D input device that does not interfere with technological miniaturization
Proposed solution: use laser “light antennas” to track one (or more) fingers.
Advantages:
•fast (no image processing),
•precise 3D measurements,
•insensitive to external light conditions,
•no markers necessary,
•can be used as output device.
![Page 28: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/28.jpg)
Further Work
Study of adaptive saccade shapes (ex: follows the hand contour)
Prototype without mobile parts, using multiple laser beams...
MOEMS integration: PDA on a keychain, PDA on a chest-pin: touching the pin or activates tracking, a gesture can be used to ask for emails, and these are read on the hand...
![Page 29: Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo](https://reader035.vdocuments.site/reader035/viewer/2022062809/56649e6b5503460f94b69378/html5/thumbnails/29.jpg)
Precision (cm)
Speed (3m/s)
multi-tracking (up to four fingers)
Interactive laser annotation
see more at both #102
ACR - 5.4.2005