interaction in the virtual world: overview user interface navigation –wayfinding –travel...
TRANSCRIPT
![Page 1: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/1.jpg)
Interaction in the Virtual World: Overview• User interface• Navigation
– Wayfinding– Travel
• Interaction and communication with others– Sharing– Collaboration
• Manipulation– Methods– Properties– Selection– Operations
![Page 2: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/2.jpg)
Broad Classification of Interactions
• Interactions– Reality-based– Imagination-based (magic)
![Page 3: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/3.jpg)
Navigation
• Types of navigation goals: explore rate or position controlled), search (target) and maneuver (objects, environment)
• Methods:– Wayfinding– Travel
![Page 4: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/4.jpg)
Navigation: Wayfinding (knowing where you are and where you’re going)
• Need a mental or cognitive map
• Relationship to the world, to the destination
• Situational awareness
• eg. of GPS system maps
• Great differences in people, sometimes gender related
• Problem of becoming lost, metaphors
![Page 5: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/5.jpg)
Navigation: Styles
• Relative motion (steps)
• Absolute motion (point to a location and go)
• Teleport
• Move the world instead of the user
![Page 6: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/6.jpg)
Navigation: Wayfinding (con’t)
• Mental map:– Divide and conquer: small regions, then piece
together– Global network: use landmarks– Memorize a map of the space– Remember a story about the space
![Page 7: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/7.jpg)
Navigation: Wayfinding (con’t)
• Wayfinding aids– Path following: color, lines, labels– Maps (exo or ego): scale, icons, you-are-here– Landmarks: objects, audio– Placenames– Breadcrumbs (trail)– Compass– Instrument guidance: could be multimodal– Shift to exocentric view– Display of coordinates or grids, names– Constrain travel
![Page 8: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/8.jpg)
Navigation: Travel Properties
• Manipulation method: physical (steering wheels, flight, dashboard) and virtual controls
• Constraints: terrain following, no z-axis, no lateral
• Frame of reference: relative motion
• Movement: gain, velocity, acc; voice
![Page 9: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/9.jpg)
Navigation: Types of Travel
• Physical locomotion: direct tracking, issue of working volume
• Vehicle simulation
• Magic techniques, including steering, target-based travel
• Combination of natural and magic: manipulation-based travel
• Ride along or towrope: pre-set path
![Page 10: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/10.jpg)
Navigation: Types of Travel (con’t)
• Fly-through, walkthrough, pilot-through: pointer directed, gaze-directed, torso-directed, dual-handed
• Move the world• Scale the world• Put me there, jump to destination• Time travel
![Page 11: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/11.jpg)
Navigation and Travel
• Physical walking
• Vehicle simulation
• Magic techniques, including steering, target-based travel
• Combination of natural and magic: manipulation-based
![Page 12: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/12.jpg)
Interaction: Communication
• Shared experience: same world, may be competing or co-existing
• Collaborative experience: need to work together in some way
![Page 13: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/13.jpg)
Shared Experience
• Different ways of sharing in different mediums
• What can be shared: ideas, world, speech• Ways to share viewpoint
– Full multipresence: may have same equipment (such as HMDs), or be in same location (cockpit)
– Some viewers have partial presence: one immersed participant with onlookers, open display (possibly projection)
![Page 14: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/14.jpg)
Shared Experience (con’t)
• Purpose of sharing: marketing (onlookers become interested), inc. throughput (onlookers get some experience even if not the full experience)
![Page 15: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/15.jpg)
Collaborative Experience
• Communication: aural, visual, gestures, teleconferencing, marks or notes, physical proximity
• Synchronous communication: live conversations• Asynchronous communication: change a world
but then not be present, present in another part of the world – can leave marks or messages
• Can have both synchronous and asynchronous in the same world
![Page 16: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/16.jpg)
Collaborative Experience (con’t)
• Messages or annotations: recipient, time (when needed), purpose– How do you know it’s there: voice, text,
gestures, pictures– Location– Point of view– Time– placeholders
![Page 17: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/17.jpg)
Collaborative Experience (con’t)
• Who’s in charge (floor control and permissions): no one, moderated, permissions, formal, hierarchical, first come first served
![Page 18: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/18.jpg)
Metacommands
• Manipulation of environment by someone other than participant (eg. of VR therapy)
• User can cause different worlds to load
![Page 19: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/19.jpg)
Manipulation: methods
• Direct user control: gestures mimic real world
• Physical control devices: wands, buttons, joysticks, mouse
• Virtual controls: interfaces on the screen such as buttons, sliders, control panel, arrows, etc.
• Agent controls: intelligent intermediary
![Page 20: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/20.jpg)
Manipulation: properties
• Feedback: user needs to know when something has happened- haptic (can feel the contact), aural (sound), visual (change color)
• Ratcheting: repeat input – eg. pick up mouse
• Constraints: restrict a DOF, snap to grid, lock to surface, keep on floor
![Page 21: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/21.jpg)
Manipulation: properties (con’t)
• Distance: manipulate objects beyond reach
• Pointer beam scope: shape of beam; eg. laserbeam, spotlight, conelike
![Page 22: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/22.jpg)
Manipulation: properties (con’t)
• Frame of reference: world representation, local representation– inside or outside the world- telepresence and teleoperation-egocentric or exocentric (god’s eye)-– Coordinate systems: world, local, parent-
vocabulary of DOF: vertical, lateral, longitudinal; Euler angles; yaw, pitch, roll
![Page 23: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/23.jpg)
Manipulation: properties (con’t)
• Bimanual interface: sometimes with haptics, keypress with joystick (difficult), medical applications; must be well designed
• Control location: omnipresent or appear, placement (interfere with world or not): part of the world (door handle), attached to hand, front of the view (helmet, windshield), on the display, on a 2D panel of controls, physical devices (tablets)
![Page 24: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/24.jpg)
Manipulation: properties (con’t)
• Control visibility: visible and invisible; how made visible
• Movement formula: gain (how much movement in one action), derivatives (velocity, acceleration)
![Page 25: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/25.jpg)
Object Manipulation
• Manipulation of the environment: could also be considered viewpoint navigation
• Selection of objects: pointing, picking, grasping
• System control: usually through 2D interfaces, buttons
![Page 26: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/26.jpg)
Manipulation: selection• Can be direction, item or value
• Direction– Pointer-directed: gesture– Gaze or eye tracking– Crosshair: pointer and gaze– Torso for direction of travel– Joystick, mouse, trackball- what about angles– Coordinates: possibly by voice– Landmarks
![Page 27: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/27.jpg)
Manipulation: selection (con’t)• Item selection
– Contact between avatar and item – choose part of body; need feedback
– Point to select (prop, gesture)– 3D cursor select; may need button; question
of what’s in range and out of range; change in symbol
– Pinching or grasping– Naming– Menu; words or pictures– Select in a small world
![Page 28: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/28.jpg)
Manipulation: selection (con’t)• Value selection: pen devices, keyboard,
tablet, physical sliders or controls, agents
![Page 29: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/29.jpg)
Manipulation: operations
• Positioning and sizing objects
• Exerting force: pushing, hitting, supporting- does it move?
• Modifying attributes of objects: color, light, transparency, weight, shape, firmness, density, etc.
• Modifying global attributes: sounds, time of day, form of rendering
![Page 30: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/30.jpg)
Interactions• Reality-based interaction:
– Head tracking, locomotion, 3D pointing– Direct manipulation of objects, gloves, haptics– Indirect manipulation through input devices– Objects for manipulation, tools (eg. wands), tangible
input devices, sliders, joysticks, touch sensors• Imagination-based interaction:
– Suspension of physics– Scaling of geometry, zooming, no clear standards yet– Scaling of motion– Automation of tasks, motion, etc.– Magic spells: dynamically assign meaning and tasks– Mode changes: eg from selecting to manipulation
![Page 31: Interaction in the Virtual World: Overview User interface Navigation –Wayfinding –Travel Interaction and communication with others –Sharing –Collaboration](https://reader031.vdocuments.site/reader031/viewer/2022012922/56649e2a5503460f94b185a5/html5/thumbnails/31.jpg)
SourcesBuilding on Realism and Magic for Designing 3D Interaction
Techniques by Kulik, IEEE Computer Graphics and Applications, Nov/Dec 2009
Questioning Naturalism in 3D User Interfaces, Bowman, McMahan, and Ragan, Comm. Of the ACM, 2012
Understanding Virtual Reality by Sherman & Craig, Morgan Kaufman, 2003
Computer Graphics and Virtual Environments by Slater et al