![Page 1: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/1.jpg)
Tangible User Interfaces (TUI’s)
![Page 2: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/2.jpg)
What are Tangible User Interfaces?
Physical World Digital worldTUI’s
2
![Page 3: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/3.jpg)
GUI vs TUI
(Ishii 2008)
3
![Page 4: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/4.jpg)
Precursors
• E.g. Marble answering machine, Durrell Bishop, Royal College of Art, Interaction Design, 1992. http://vimeo.com/19930744
4
![Page 5: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/5.jpg)
Early Work
• Fitzmaurice et al. 1995 – Graspable user interfaces http://www.youtube.com/watch?v=V-TGEe-Imro
• Ishii et al. 1997 - Tangible bits
5
![Page 6: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/6.jpg)
Recent work
• E.g. Lumino, Baudish et al, 2010 http://www.youtube.com/watch?v=tyBbLqViX7g
• E.g. Portico, Avrahami et al, 2011 http://vimeo.com/29359319
6
![Page 7: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/7.jpg)
Benefits of TUI’s (Ishii 2008)
1. Double interaction loop -immediate tactile feedback
2. Persistency of tangibles
3. Coupled input/output space
4. Special vs generic purpose
5. Space-multiplexed vs time-multiplexed input
Also fun + engaging!
7
![Page 8: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/8.jpg)
• Sense position & orientation on touch technology
• Add intelligent drawing support
• How can they be best combined with multi-touch surfaces for enjoyable and productive interaction?
Can we connect these physical drawing tools to the digital space?
8
![Page 9: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/9.jpg)
Our Approach
• Design– Tangible hardware
• Implementation– Recognizer– Drawing application
• Usability evaluation
Capacitive Touch (CapTUI) Infrared Touch (TanGeo)– Ryan Tan & Bryan Chen
– Rachel Blagojevic
– Jacky Zhen
9
![Page 10: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/10.jpg)
CapTUI Technology – Capacitive
• Small touch screens e.g. iPad, smart phones etc…
• Touch detection via electrical pulse from fingers/conductive material
10
![Page 11: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/11.jpg)
Design: Tangible Hardware
11
![Page 12: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/12.jpg)
Final Design
12
![Page 13: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/13.jpg)
Implementation: Tangible Recognition
• Tangible ID
– 3 point (min) unique patterns
Valid patterns Invalid patterns13
![Page 14: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/14.jpg)
Implementation: Tangible Recognition
• Learning phase
• Recognition phase
– Touch point detection
– Match point distances to saved tangible ID’s
– No way of knowing which part of the touch point is in contact (+/- error)
14
![Page 15: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/15.jpg)
Implementation: Drawing Application
• Beautification
Ink-to-edge snapping Corner snapping
& Length visualization 15
![Page 16: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/16.jpg)
Implementation: Drawing Application
• Visual drawing guides
Tangible outline Angle visualization16
![Page 17: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/17.jpg)
Evaluation
• First iteration: usability
– Simple drawing tasks
• Second iteration: comparative study
– Recognizable vs non recognizable drawing tools on screen
17
![Page 18: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/18.jpg)
Usability Evaluation
• Can users construct simple drawings using the tangibles? Is the system usable?
• 10 participants
• 5 simple drawing tasks
18
![Page 19: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/19.jpg)
Usability Evaluation: Results
• First exploration– Technology works– Is usable for simple drawings
• Tangible detection problems– Stability– Consistent circuit– Finger to tangible contact– Friction with screen– Comfortable drawing
• Tangible outline helpful – recognition indicator• Drawing guides needed
19
![Page 20: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/20.jpg)
Comparative Study
• Does CapTUI assist users to easily draw precise geometric drawings
• Recognizable vs non recognizable drawing tools on screen
• 12 Participants
20
![Page 21: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/21.jpg)
Comparative Study: Results
• CapTUI rated significantly higher than Paint overall• Visual guides helpful for precise drawing
– significantly lower average angle error.
• Participants enjoyed using CapTUI significantly more • Participants believed that CapTUI produces significantly
more tidy drawings than Paint.• Making fine grained movements with the tangibles
difficult• Tangible design still needs work
– consistent detection– accurate positioning
21
![Page 22: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/22.jpg)
Tangeo Technology – Infrared
• Table tops e.g. Microsoft Surface 2.0
• PixelSense - Touch detection via infrared reflection for each pixel
• Image processing on detected pixels
• Identifies finger/blobs/tags
22
![Page 23: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/23.jpg)
Design: Tangible Hardware
23
![Page 24: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/24.jpg)
Final Design
24
![Page 25: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/25.jpg)
Implementation: Tangible Recognition
• Learning phase
• Recognition phase
– Detection via custom tags
– Use tag location to get tangible outline
– Use thresholds for blob sizes
25
![Page 26: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/26.jpg)
Implementation: Drawing Application
26
![Page 27: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/27.jpg)
Implementation: Drawing Application
• Visual Guides
– Tangible outline
– Angle visualisation
– Length visualisation
– Ink beautification
• corner snapping
• ink-to-edge snapping
27
![Page 28: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/28.jpg)
Usability Evaluation
• Can users construct simple geometric drawings using Tangeo? Is the system usable?
• 2 phase cycle
• 8 participants
• 4 drawing tasks
28
![Page 29: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/29.jpg)
Usability Evaluation: Results
• Enjoyable / easy to use the tangibles
• Good recognition
• Visual guides helpful and easy to understand
• Drawing accuracy – less positive perception
• Add stylus for drawing
29
![Page 30: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/30.jpg)
Oroo’ Tangibles
• Don’t need accurate sensing
• Extendable set
30
![Page 31: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/31.jpg)
Design Methodology
• Participatory design (ie users make decisions)
• First prototype
• 3 x elders review
– Need to be ‘real’
• Community meeting
– 10 core
31
![Page 32: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/32.jpg)
Oroo’ tangibles
• Web based html5 + Css
– Works on safari (iPad) and Chrome (everything we have tested)
• Video
32
![Page 33: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/33.jpg)
This year
• Tangrams – Nazish
– Spatial relationships between tangibles
33
![Page 34: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/34.jpg)
This year
• Jenga (Simon and Ben)
– 3D-2D-3D interaction
34
![Page 35: Tangible User Interfaces (TUI’s) · The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software](https://reader030.vdocuments.site/reader030/viewer/2022040402/5e7d8052602f10183d6a8ca7/html5/thumbnails/35.jpg)
References
• Marble answering machine - Crampton Smith, G. The Hand That Rocks the Cradle. I.D., May/June 1995, pp. 60-65.• Fitzmaurice G. W., H. Ishii, and W. Buxton. 1995. Bricks: laying the foundations for graspable user interfaces.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95), ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 442-449.
• Ullmer B. and H. Ishii. 1997. The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software and technology (UIST '97). ACM, New York, NY, USA, 223-232.
• Ishii H., B. Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States
• Ishii H., 2008. Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (TEI '08). ACM, New York, NY, USA, xv-xxv.
• Baudisch P., T. Becker, and F. Rudeck. 2010. Lumino: tangible building blocks based on glass fiber bundles. In ACM SIGGRAPH 2010 Emerging Technologies (SIGGRAPH '10). ACM, New York, NY, USA, Article 16 , 1 pages.
• Avrahami D., J. Wobbrock, and S. Izadi. 2011. Portico: tangible interaction on and around a tablet. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA, 347-356.
• Blagojevic R., X. Chen, R. Tan, R. Sheehan, and B. Plimmer. 2012. Using tangible drawing tools on a capacitive multi-touch display. In Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers (BCS-HCI '12). British Computer Society, Swinton, UK, UK, 315-320.
• Zhen, J. S., R. Blagojevic and B. Plimmer (2013). Tangeo: Geometric Drawing with Tangibles on an Interactive Table-Top. CHI 2013. Paris France, ACM. WIP: in press.
• Shaer O. and E. Hornecker (2010) "Tangible User Interfaces: Past, Present and Future Directions",Foundations and Trends® in Human-Computer Interaction: Vol. 3: No 1-2, pp 1-137.
• Plimmer, B., He, L., Zaman, T., Karunanayaka, K., Yeo, A.W., Jengan, G., Blagojevic, R., Do. E.Y-L., (2015). New Interaction Tools for Preserving an Old Language. Chi 2015. Seoul, ACM: pp 3493-3502
35