virtual worlds: input

Download Virtual Worlds: Input

Post on 01-Jan-2016




0 download

Embed Size (px)


Virtual Worlds: Input. Virtual Environments. Importance of immersion, presence, interaction, engagement, multisensory Elements of the world: graphics, representation, visual and other senses, interface, navigation, manipulation, story Interface: input, output, computer interface. - PowerPoint PPT Presentation


  • Virtual Worlds: Input

  • Virtual EnvironmentsImportance of immersion, presence, interaction, engagement, multisensoryElements of the world: graphics, representation, visual and other senses, interface, navigation, manipulation, storyInterface: input, output, computer interface

  • Virtual Environments as a MediumThink of the following mediums and how they communicate: medium, kind of virtual world, final composition, how experiencedPainting, dance, music, written word, play, movie, video game, virtual environments

  • Interaction in the Virtual World: OverviewUser interfaceManipulationMethodsPropertiesSelectionOperationsNavigationWayfindingTravelInteraction and communication with othersSharingCollaboration

  • VR Worlds: Input OverviewUser input:3DMouse, keyboardPosition tracking Body trackingOther physical input, including speechWorld input

  • User Interface MetaphorsFamiliar user interfaces: mobile phones, TV controls, remote controls, cars, video games, microwaves, doorways, cursor icons, desktop, touchscreens, notions of right, left and double clicking, mouseover, tweeting, textingDo you use familiar ones in a VR world? Has limits and shortcuts

  • User Interface InteractionsNeed to map user input to interactionsUsers must already know how to use a tool or must be able to learnUsers bring experience and cultural knowledgeUsers need familiarity but also want to extend what is normally possible in the real world (has limitations)

  • User Interface Interactions (cont)Viewpoint navigation (or travel) and object manipulation (and selection) are the main types of interactionDifferences between the two:Users conceptual model (what moves)Extent of interaction space (navigation can be large, object manipulation often small)Perceptual cues (navigation generally visual, object manipulation uses haptics)

  • 3D User InterfacesOperate in 3D spaceNatural: measure of how closely the actions in the virtual environment correspond to the actions in the real worldExamples: Wii Remote, Sony Move, Microsoft Kinect

  • Bowman, et all Questions about 3D User Interfaces and NaturalismAre 3D inherently more natural than traditional?Is naturalism the ultimate goal for designers?Does naturalism lead to better performance, increased engagement, or better learning?When naturalism isnt possible should the designer go for traditional user interfaces or interfaces with some degree of naturalism?

  • Issues Involving Natural User InterfacesFor the tasks of navigation, selection and manipulation of object, how well do 3D natural user interfaces perform vs. magic techniques or some combination of them?Issues involving turning, tracking, selecting, manipulating, steeringNatural interfaces sometimes out-perform and are sometimes inferior- depends on task and context; they need high level of accuracy and must be familiar

  • Definition of natural?Lack of guidelines and standards for gestural interfacesProblems raised by Norman and Nielsen of visibility, feedback, discoverability, consistency, scalability, reliability

  • Bowman et al table from Comm. of the ACM, 2012

  • Problems with Natural User InterfacesHow natural are they?Who decides what the gestures or other actions will be?Cultural differences

  • Input: Position TrackersPosition tracking gives location and orientation of user and/or parts of user and feeds it to the computer - concerned with 6 degrees of freedom: x,y,z position and orientation, generally given by pitch, roll and yaw (angles with orthogonal axes)General issues of accuracy (walking in a landscape vs surgery), latency (conducting an orchestra vs walking), interference from surrounding objects (noise, occlusion, monitors, metal), encumbrance such as cables (dancing, large-scale movement), cost

  • Position Trackers: Main TypesOpticalElectromagneticMechanicalTangibleNeural brainOthers: 3D, Videometric, Ultrasonic, Inertial

  • Position Trackers: OpticalVisual information is used for trackingGenerally from video camera(s)Generally need more than one unless dont want 6 DOF; for example, the KinectNeed good gesture recognition or visual recognition techniques (algorithms)Sometimes use body sensors (motion capture)Main disadvantage is line of sight issues

  • Electromagnetic Position TrackersVery commonNeed a transmitter (sends low-level magnetic fields from 3 orthogonal coils) and a receiver- can be in both configurations: transmitter on user or receiver on userStrength of receiving signal varies with orientation and positionSome with cables and some wireless

  • Electromagnetic (cont)Advantages are that line of sight does not have to be clearDisadvantages: can be interference from metal and monitors, must be within several feet of the transmitterEg. Ascension Flock of Birds, Polhemus

  • Position Trackers: MechanicalMechanical arms and booms that physically track movementEg. of BOOM by FakespaceAdvantages: accurate, no interference problemsDisadvantages: Cant move very far

  • Position Trackers: TangibleUses hands in a more natural way: always been the case for tools from an axe to a toothbrush; relationship between design and interaction Physical models (for architecture, urban planning)ClaySandBlocksInteractive lighting devicesUse of effectors, LEDs, motors, sensors, react to heatBoxes for manipulating music

  • Position Trackers: NeuralField of brain-machine interface (BMI)Attempts to read brain signals to direct the computerEfforts to use EEGs, VEP (visually evoked potential), motor imageryNeuroprosthetic devices, brain-implantable chips

    Interfaces also benefit from research in neuroscience

  • Position Trackers: OthersVideometric: camera on person and tracks surroundings via landmarks such as infrared light sourcesUltrasonic: high-pitched sounds emitted at definite intervals length of time need multiple transmitters and receivers subject to noise in the environmentInertial: gyrocsopes, accelerometers, tracking device attached to user with cable- generally only orientation, so used with other devices- inexpensive, good accuracyNeural: muscular (monitors electrical impulses in skin) some experiments in reading brain wave

  • User Input: Body TrackingPosition of joints: KinectHead tracking- usually through orientation tracking- used to know what to project visuallyHand and fingers; position trackers, finger sensors (gloves), virtual scalpelsTorso and feetOther body tracking such as heart rate, temperatureIndirect tracking using props and platforms

  • Body Input: Eye TrackingTracks movement of pupils pretty accurate, requires that the head be stilleg. of Jeanne Sterns project Used for people with disabilities Problem of Midas Touch

  • User Input: Other Physical DevicesPhysical controls: buttons, switches, joysticks, mouse, steering wheelsArduino devicesProps such as wands, 3D mouse, scalpel, drill, realistic devices for particular application (balls, bodies, spiders), pressure devices, mobile phonesPlatforms: treadmills, locomotion, rings, kiosks, wheelchairs, cockpits, submarine control rooms, cars, workbenches, CAVEs, DiVE system at Duke

  • Other Physical Input: SpeechSpeech recognition: system will recognize commands, natural language interaction, SiriMicrophoneProblem of accents and training: speaker independence or dependenceHow activated: button, command (talk), vision (need eye tracking)Sometimes increases bandwidth by using separate processor

  • World Input to Virtual WorldPersistent virtual worlds: exist over many experiences, multiple users, Web communication, databasesImporting data that changes (weather systems)Importing real world data such as objects and obstacles, vision, knowledge, digital images, satellite infoSometimes the info is brought in with transducers: translates data into electrical signals for the computer- devices such as microphones, weather stations, video, sensors, medical devices (for AR)

  • SourcesQuestioning Naturalism in 3D User Interfaces, Bowman, McMahan, and Ragan, Comm. Of the ACM, 2012The Artificiality of Natural User Interfaces, Malizia and Bellucci, Comm of the ACM, 2012Gestural Interfaces: A Step Backward In Usability, Norman and Nielsen, Interactions, 2010Understanding Virtual Reality, Sherman & Craig, Morgan Kaufman, 2003The Tangible User Interface and its Evolution, Ishi, Comm. Of the ACM, 2008Neuroscience and the Future of Human-Computer Interface, Minnery & Fine, Interactions, Mar-Apr 2009Building on Realism and Magic for Designing 3D Interaction Techniques by Kulik, IEEE Computer Graphics and Applications, Nov/Dec 2009

  • Sources contTangible Interaction=Form+Computing, Baskinger & Gross, ACM Interactions, Jan-Feb 20103D input devices, Frohlich et al, CG&A, Mar-Apr 20063D User Interfaces: New Perspectives and Directions, Bowman et al, Comp Graphics and Apps, Nov-Dec 2008Usability of Multiple Degree-of-Freedom Input Devices and Virtual Reality: Displays for Interactive Visual Data Analysis, Moritz and Wischgoll, VRST (Virtual Reality Software and Technology) 2007