ubiquitous computing and augmented realities · 2004-10-12 · ubiquitous computing • any...

1

1/90

chapter 20

ubiquitous computing and augmented realities

2/90

ubiquitous computing and augmented realities

• ubiquitous computing– filling the physical world with computers

• virtual and augmented reality– making the physical world in a computer!

3/90

Challenging HCI Assumptions

• What do we imagine when we think of a computer?

“The most profound technologies are those that disappear.”

Weiser

• 1990’s: this was not our imagined computer!

4/90

Ubiquitous Computing

• Any computing technology that permits human interaction away from a single workstation

• Implications for– Technology defining the interactive

experience– Applications or uses– Underlying theories of interaction

5/90

Scales of devices

• Weiser proposed– Inch– Foot– Yard

• Implications for device size as well as relationship to people

6/90

Device scales

• Inch– PDAs– PARCTAB– Voice Recorders– smart phones

• Individuals own many of them and they can all communicate with each other and environment.

2

7/90

Device scales

• Foot– notebooks– tablets– digital paper

• Individual owns several but not assumed to be always with them.

8/90

Device scales

• Yard– electronic whiteboards– plasma displays– smart bulletin boards

• Buildings or institutions own them and lots of people share them.

9/90

Defining the Interaction Experience• Implicit input

– Sensor-based input– Extends traditional explicit input (e.g.,

keyboard and mouse)– Towards “awareness”– Use of recognition technologies– Introduces ambiguity because recognizers

are not perfect

10/90

Different Inputs

Capacitive sensing on a table Sensors on a PDA

11/90

Multi-scale and distributed output• Screens of many sizes

– (very) small

– (very) large

• Distributed in space, but coordinated

12/90

The output experience

• More than eye-grabbing raster displays– Ambient: use features of the physical

environment to signal information– Peripheral: designed to be in the

background

• Examples: – The Dangling String– The Water Lamp (shown)

3

13/90

Merging Physical and Digital Worlds• How can we remove

the barrier?– Actions on physical

objects have meaning electronically, and vice versa

– Output from electronic world superimposed on physical world

A “digital” desk

An augmented calendar14/90

Application Themes

• Context-aware computing– Sensed phenomena facilitate easier

interaction

• Automated capture and access– Live experiences stored for future access

• Toward continuous interaction– Everyday activities have no clear begin-end

conditions

15/90

Virtual and Augmented Reality(VR and AR)

VR - technology & experienceweb, desktop and simulatorsAR – mixing virtual and real

16/90

virtual reality technology

• headsets allow user to “see” the virtual world

• gesture recognition achieved with DataGlove(lycra glove with optical sensors that measure hand and finger positions)

• eyegaze allows users to indicate direction with eyes alone

• whole body position sensed, walking etc.

17/90

VR headsets

• small TV screen for each eye• slightly different angles• 3D effect

18/90

immersion

• VR– computer simulation of the real world

• mainly visual, but sound, haptic

– experience life-like situations• too dangerous, too expensive

– see unseen things:• too small, too large, hidden, invisible

– e.g. manipulating molecules

• the experience– aim is immersion, engagement, interaction

4

19/90

on the desktop

• headset VR– expensive, uncomfortbale

• desktop VR– use ordinary monitor and PC

• cheap and convenient

• in games …• and on the web

– VRML – virtual reality markup language

20/90

VRML … VR on the web

#VRML V1.0 ascii

Separator {Separator { # for sphere

Material {emmissiveColor 0 0 1 # blue

}Sphere { radius 1 }

}Transform { translation 4 2 0 }Separator { # for cone

Texture2 {filename "big_alan.jpg"

}Cone {

radius 1 # N.B. width=2*radiusheight 3

} } }

COSMO, CORONA, plug-in’s to view vrml models in web-browsers

21/90

command and control

• scenes projected on walls• realistic environment• hydraulic rams!• real controls• other people

• for:– flight simulators– ships– military

22/90

Augmented Reality (AR)

• Definition• Typical applications• Forms of Display• AR Examples• AR-Research Focus I and II

Research project overview:http://hci.rsc.rockwell.com/AugmentedReality/research.php

23/90


Definition of Augmented Reality

1. Combines real and virtual

2. Is interactive in real time

3. Is registered in three dimensions

(Azuma, 1997)24/90


Typical applications:

• Medical• Manufacturing and repair• Outdoor and Construction• Annotation and visualization• Robot path finding• Entertainment

5

25/90

Kinds of Displays for AR

• Head-up Display

• Head Mounted Display (HDM)

26/90

Head-up Display - Car

27/90

Head-up Display - Car

28/90

Head-up Display-Taxiway Navigation

29/90

Head-up Display-Taxiway Navigation

30/90

Head Mounted Display (HMDs)

6

31/90

Head Mounted Display (HDM) are either

• video display, or• see-through display, or• look-around display

32/90

Head Mounted Display (HDM)

video-based see-through HMD

33/90


video display34/90


look-around display

35/90

Augmented Reality Examples

Real World Wide Web (RWWW), HMD36/90


RWWW: Glare and gaze selection

7

37/90


RWWW: Glare and gaze selection38/90


Manufacturing and repair

39/90


Manufacturing and repair40/90


Design

41/90


Design42/90


Outdoor and Construction

8

43/90


Outdoor and Construction44/90


Outdoor and Construction

45/90


Stereoscopic telerobotic control46/90


Robotic application

47/90


Entertainment48/90


Entertainment

9

49/90


Museum50/90

High Degr.-of-Freedom Curve Input

Tovi Grossman, Ravin Balakrishnan, Karan Singh. (2003). An interface for creating and manipulating curves using a high degree-of-freedom input device. ACM CHI Conference on Human Factors in Computing Systems. ACM CHI Letters, 5(1). pp. 185-192.Paper: http://www.dgp.toronto.edu/~ravin/papers/chi2003_curvemanipulation.pdfVideo: http://www.dgp.toronto.edu/~ravin/videos/chi2003_curvemanipulation.mpg

51/90

AR-Research Focus I: BUILD-IT, Tangible Interface for Collaborative Design

Fjeld, Rauterberg, BixelETH Zurich

Video: http://www.fjeld.ch/pub/CHI_2000.mpg

52/90

Activity Theory: Complete Task Regulation -- 1

53/90

Activity Theory: Complete Task Regulation -- 2

54/90

10

55/90 56/90

57/90 57

AR-Research Focus II:Chemistry EducationA Tangible Interaction Approach

P. Juchli, B. Voegtli, M. Fjeld, HyperWerk FHBB, ETH Zurich

Video: http://www.fjeld.ch/pub/AC.mov58/90

Tangible User Interfaces (TUIs) are often seen as ...

• IntuitiveEasy to learn and to handle

• DirectNatural mapping of taskImmediate feedback

• Task specificRequire considerable developement ressourcesAre difficult to re-use in a different use context

• .. but , non-compliant with standard formatsCAD Interface is rareIntergration with Word/OSX etc. difficult

59/90

For educational use of TUIs, particular challenges are ...

• Combined spatial representation and interaction

• Adaptation to a standard curriculum,e.g. science

• Adaptation to personal learning strategies

• Collaborative learning and human communication

60/90

... but TUI for education also promise significant benefits

• Information pieces attached to physical objects help memorize content

• Limitations of time multiplexed interfaces may beovercome through bi-manual (and collaborative) use

• Enable non-linear, explorative learning strategies

11

61/90

Educational Context in Focus:Molecular models and modelling; intermediate level

62/90

Physical ball-stick kit Digital spacefill/wireframe GUI

Established tools for molecular modelling

63/90

• Offer direct manipulation of virtual molecular models

• Combine 3D representation and spatial interaction

• Give users audiovisual feedback

• Observe and keep track of user‘s interactive steps

Our strategy in interface design:Combine the strengths of physical and digital interface

64/90

The Augmented Chemistry (AC) System:

A virtual mirroras an augmentationof a tabletop learningenvironment

The mirroris a designmetaphor.An alternativewould be a user‘spoint-of-view.

65/90 66/90 66

CubeBooklet

Gripper Platform

Interactive Tools

12

67/90 67

Interaction Sequence1. Element selection 2. Load an element

with the Gripper

3. Place the atom onto the platform

4. Select bindingatom

5. Determine single-double or trible binding

6. User feedbackafter sucessfulcompletion

68/90 68

Specialized tools

69/90

Recent advances: Electronegativity; dipole-moment (1/4)

70/90


H

O H

H

O H

71/90


72/90


13

73/90

Work in progress: Learning Studies (1/2)

Same task, three different conditions

Measurement of:- Task completion- User satisfaction- Task load (NASA task load index)

74/90

Work in progress:Learning Studies(2/2)

75/90

information and data visualisation

VR, 3D and 2D displaysscientific and complex data

interactivity central

76/90

scientific and technical data

• number of virtual dimensions that are ‘real’• three dimensional space

– visualise invisible fields or values– e.g. virtual wind tunnel

• two dimensional space– can project data value up from plane– e.g. geographic data– N.B. viewing angle hard for static visualisation

• no ‘real’ dimensions– 2D/3D histograms, scatter plots, pie charts, etc.

77/90

virtual wind tunnel

• fluid dynamics to simulate air flow • virtual bubbles used to show movements

• ‘better’ than real wind tunnel …– no disruption of

air flow– cheaper and faster

78/90

structured informnation

• scientific data – just numbers• information systems … lots of kinds of data

• hierarchies– file trees, organisation charts

• networks– program flow charts, hypertext structure

• free text …– documents, web pages

14

79/90

visualising hiererchy

• 2D organisation chart– familiar representation– what happens when it gets wide?

managing director

salesmanager

F. Bloggs

J. Smith

F. Bloggs

marketingmanager

A. Jones R.Carter

productionmanager

K. West

P. Larkin

B. Firth

80/90

wide hierarchies … use 3D?

• cone trees (Xerox)• levels become rings• overlap ‘OK’ in 3D

managing director

salesmanager

F. Bloggs

J. Smith

F. Bloggs

marketingmanager

A. Jones R.Carter

productionmanager

K. West

P. Larkin

B. Firth

81/90

visualising knowledge and hiererchy

a) Walden's Path http://www.csdl.tamu.edu/walden/b) File System Navigator http://www.sgi.com/fun/freeware/3d_navigator.htmlc) Cheops http://www.crim.ca/hci/cheops/index1.htmld) VR-VIBE http://www.crg.cs.nott.ac.uk/research/technologies/visualisation/vrvibe/e) Information Pyramides http://www2.iicm.edu/keith/papers/vis97/ipyr.htmlf) The Web Book http://www.parc.xerox.com/istl/projects/uir/pubs/pdf/UIR-R-1996-14-Card-CHI96-WebForager.pdf (invalid)g) WWW3D http://www.crg.cs.nott.ac.uk/~dns/vr/www3d/webnet96-final.htmlh) The Task Gallery http://research.microsoft.com/users/marycz/tg2000.pdf 82/90

visualising and interacing with relations

http://www.knetmap.com

83/90

Scientific Visualisation: City O’Scope

http://www.macrofocus.com/cityoscope/index.html 84/90

» Test Details, Chapters from Textbook

Test questions which are based on the textbook(Dix et al., 3rd Edition.), will come from thesechapters and sub-chapters:

1. 1.1 – 1.52. Not relevant3. 3.1 – 3.64. Not relevant5. 5.1 – 5.66. 6.1 – 6.57. 7.1 – 7.58. Not relevant9. 9.1 – 9.510. 10.1 – 10.311. Not relevant12. 12.1 – 12.2, 12.513. 13.1 – 13.314. Not relevant15. 15.1 – 15.716. Not relevant17. Not relevant18. Not relevant19. 19.1 – 19.520. 20.1 – 20.421. Not relevant

15

85/90

» Test Details

The written test is compulsory ("obligatorisk") and will take place on Monday 18th October 2004, 08:30-12:30. The place is V-Huset, please follow indications in V-Huset. The written test consist of a compulsory ("obligatorisk") part and a voluntary ("frivillig") part.

The compulsory part: -presents questions which have a rather clear answer,-reflects the miminum requirement to have the test accepted ("godkänt"), and-must be accepted ("godkänt") for the voluntary part to be corrected at all.

86/90

» Test Details

The voluntary part is for participants who• have answered the compulsory part AND• belive they will have the compulsory part accepted

("godkänt") AND• aim for a higher score (4, 5, or VG)

The voluntary part consists of a few advanced questions/tasks.

Suggestion: Answer at least 7 of the 9 questions in Part I before proceeding to Part II.

87/90

» Test Details

Please answer in Swedish or English

Max points per question is indicated after each question, total max is 60.

PART I: Max 42 points, 25 points NECESSARY FOR PASS (GODKÄNT)

PART II: Max 18 points.This part is only corrected if 25 points or more were obtained in part I.

88/90

» Test-/tente-granskning

Oral feedback on written test, Q&A ("tentegranskning").

Monday 8th November 2004, 16:00-18:00,restaurant in Linsen.

89/90

» Assignment-granskning

Oral feedback on assignment, Q&A (“assignment granskning").

Tuesday 30th November and Thursday 2nd December 2004, 13:00-15:00.

Restaurant in Linsen

90/90

IMPORTANT Assignment file name IMPORTANT

The file submitted electronically for first draft and for final version MUST be named with the 40 first characters (including spaces) in the title of your paper, nothing else is accepted.

E.g. if the title of your paper is “Hand-held devices for whale hunting in Hudson Bay”, then the file name becomes:

hand_held_devices_for_whale_hunting_in_H.pdf

ubiquitous computing and augmented realities · 2004-10-12 · ubiquitous computing • any...

Documents