![Page 1: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/1.jpg)
Input and interaction
MICHAEL BERNSTEINCS 376
![Page 2: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/2.jpg)
Recall: Skinput
![Page 3: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/3.jpg)
Recall: Omnitouch
![Page 4: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/4.jpg)
Recall: SenseCam
![Page 5: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/5.jpg)
Input and interaction research� How can the user interact fluidly with the world around them?� New input modalities: e.g., depth cameras� New output modalities: e.g., pico projectors and fabrication� New user vocabulary: e.g., gestures
� This research is often driven by, or involves the creation of, new hardware
5
![Page 6: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/6.jpg)
Foundations
![Page 7: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/7.jpg)
Bolt. “Put-that-there”: voice and gesture at the graphics interface. SIGGRAPH ’80.
![Page 8: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/8.jpg)
Put That There� Contribution: combined gesture and voice input� In a closed world� With a toy goal� Using simple manipulation operations� Using a laser attached to the wrist
� In many ways, our goal since 1980 has been to relax those assumptions
8
![Page 9: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/9.jpg)
looks a bit like harry potter...
Wellner. Interacting with paper on the DigitalDesk. CACM ‘93.
![Page 10: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/10.jpg)
DigitalDesk� Contribution: fluid boundaries between digital and physical
objects� In a constrained space� On a small set of tasks� With predefined behaviors
� Again, we work to relax these assumptions
10
![Page 11: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/11.jpg)
Input technologies
![Page 12: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/12.jpg)
Tabletop
12
Dietz and Leigh. DiamondTouch: a multi-user touch technology. UIST’ 01.
![Page 13: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/13.jpg)
Tabletop
13
Han. Low-cost multi-touch sensing through frustrated total internal reflection. UIST ’05.
![Page 14: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/14.jpg)
Pen
14Hinckley et al. Pen + touch = new tools. UIST ’10.
![Page 15: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/15.jpg)
Depth sensing
15Izadi et al. KinectFusion. UIST ’11.
![Page 16: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/16.jpg)
Sensing biosignals
16
Saponas et al. Enabling Always-Available Input with Muscle-Computer Interfaces. UIST ’09.
![Page 17: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/17.jpg)
Machine learning model
17
30 millisecond sample
Root Mean Square (RMS) ratios between channels
Frequency Energy10 Hz bands
Phase Coherence ratios between channels
x6 Sensors
Support VectorMachine
user specific model
gesture classification
![Page 18: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/18.jpg)
Output technologies
![Page 19: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/19.jpg)
pico projector
19
Handheld projectors
Willis et al. SideBySide: ad-hoc multi-user interaction with handheld projectors. UIST ’11.
![Page 20: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/20.jpg)
Physically actuated material
20
Follmer et al. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. UIST ’12.
![Page 21: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/21.jpg)
jamming uis� \
21
Magnetic levitation
Lee, Post and Ishii. ZeroN: mid-air tangible interaction enabled by computer controlled magnetic levitation. UIST ’11.
![Page 22: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/22.jpg)
22
Projectors
Wilson et al. Steerable augmented reality with the Beamatron. UIST ’12.
![Page 23: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/23.jpg)
23
3D printing
Willis et al. Printed Optics: 3D Printing of Embedded Optical Elements for Interactive Devices. UIST ’12.
![Page 24: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/24.jpg)
Skill sets for input research� Learn “enough to get by” in...� Electrical engineering� Mechanical engineering� Computer graphics
� Known techniques for research in these domains often have direct mappings onto open questions in input
24
![Page 25: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/25.jpg)
Today’s papers
![Page 26: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/26.jpg)
Tangible Bits� What?� An extremely influential vision of making computing something
physically perceivable and manipulable� Why?� Ishii’s Tangible Bits projects laid the foundations
26
![Page 27: Input and interaction - Stanford University · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research! How can the user interact fluidly with the world](https://reader033.vdocuments.site/reader033/viewer/2022042620/5f3b198b1a9e70474a43eba9/html5/thumbnails/27.jpg)
Discussion rooms
27
Rotation Littlefield 107 Littlefield 103
a 12 34
b 24 13
c 14 23
d 34 12
e 13 24
f 23 14