modeling perceptual attention in virtual humans
DESCRIPTION
Modeling Perceptual Attention in Virtual Humans. Randall W. Hill, Jr. University of Southern California Information Sciences Institute. Introduction. Virtual humans are proliferating Military simulations Games Animated characters for entertainment Immersive training environments. - PowerPoint PPT PresentationTRANSCRIPT
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Modeling Perceptual Attention
in Virtual Humans
Randall W. Hill, Jr.University of Southern California
Information Sciences Institute
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Introduction
⢠Virtual humans are proliferatingâ Military simulationsâ Gamesâ Animated characters for entertainmentâ Immersive training environments
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
What We Want From Virtual Humans
⢠Realistic behaviorâ Perform within range of human capability
⢠Superman need not apply âŚ.⢠No drones, either!
⢠Believabilityâ Does the agentâs behavior enable me to
believe that the agent could be a human?
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Challenges for Realistic Behavior
⢠Synthetic battlespace is information richâ 100âs of other entitiesâ Vehicle instrumentsâ Terrain, weather, buildings, etc.â Communications (messages)â Amount of information will continue to increase âŚ.
⢠Perceive, understand, decide and actâ Comprehend dynamic, complex situationsâ Decide what to do nextâ Do it!
Tale of the Apache Pilot
Pilot Behavior Modeled with Soar
ModSAF Simulator
Soar
HelicopterFlight dynamics, weapons, visual sensor
Simulation Network
Entity State
Motor Commands Percepts
Soar-ModSAF Interface
Rules for Pilot Behavior
Working Memory
Engagement Area
Battle Position
HoldingArea Phase Line
LD
~5 km
~70 km
Army Deep Attack Mission
Assembly
Area
Engage Targets
The Pilot Who Saw Too Much
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Roots of the Problem
⢠NaĂŻve vision modelâ Entity-level resolution only
â Unrealistic field of view (360o, 7 km radius)
⢠Perceptual-Cognitive imbalance â Too much perceptual processing
â Cognitive system needs inputs, but âŚ
â It also needs time to respond to world events
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
How Do Humans Do It?
⢠Employ perceptual attentionâ Discard excess informationâ Focus perceptual processing on limited regions / objectsâ Zoom lens metaphor
⢠Process percepts in stagesâ Preattentive stage: segment, group, filterâ Attentive stage: search, fixate, track
⢠Control the focus of attentionâ Goal-directed versus stimulus-driven
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Virtual Humans Need Attention, Too!
⢠ATTENTION is at the nexus between cognition and perceptionâ Consists of a set of mechanisms â Spans perceptual and cognitive systemsâ Cognition orients and controls perceptual processingâ Perception influences cognition
⢠Operationalize attention in pilotâ Use human attention model to guide virtual human attention
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Approach
⢠Create a focus of attentionâ Entity perception initiallyâ Zoom lens model (covert attention)
⢠Provide controls over processingâ Stages of attention: preattentive & attentiveâ Filtering
⢠Control the focus of attentionâ Goal-drivenâ Stimulus-driven
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Zoom Lens Model of Attention(Eriksen & Yeh, 1985)
⢠Attention limited in scopeâ Multi-resolution focusâ Magnification inversely proportional to field of view
⢠Low resolutionâ Large region, encompassing more objects, fewer detailsâ Perceive groups of entities as a coherent whole
⢠High resolutionâ Small region, fewer objects, more detailsâ Perceive individual entities (e.g., tank, truck, soldier)
Low Resolution
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Perceptual Grouping
⢠Preattentive⢠Gestalt grouping
â Involuntaryâ Proximity-basedâ Other features
⢠Dynamic⢠Voluntary grouping
K
K
K
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Group Features
⢠Quantity and composition⢠Activity
â Movingâ Shooting
⢠Locationâ Center-of-massâ Bounding-box
⢠Geometric relationships wrt pilotâ Slant-range, azimuth, etc.
High Resolution
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Entity Features
⢠Location (GCS)⢠Speed⢠Velocity⢠Orientation⢠Slant Range⢠Force⢠Object, Object Type⢠Vehicle Class⢠Function⢠Sense Name
⢠Altitude⢠Angle Off⢠Target Aspect⢠Magnetic bearing⢠Heading⢠Status⢠Lateral Range⢠Lateral Separation⢠Closing Velocity⢠Vertical Separation
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Stages of Attention
⢠Preattentiveâ Sense entities with ModSAF visual sensorâ Update location, range, and speed features of entitiesâ Cluster newly sensed entities into groupsâ Update group features
⢠Attentiveâ Apply selective filter to entitiesâ Compute and assert features to Working Memoryâ Match Soar productions on entity/group features
Soar Decision Cycle
Perception Cognition Motor
Input Phase
Elaboration Phase
Output Phase
Decision Phase
⢠Fire rules
⢠Generate preferences
⢠Update working memory
⢠Evaluate operator preferences
⢠Select new operator OR
⢠Create new state
⢠Sense & update entities
⢠Cluster & update groups
⢠Apply entity filters
⢠Compute entity attributes
⢠Assert to WM
⢠Command effectors
⢠Adjust entity filters
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Control of Attention
⢠Goal-driven controlâ Agent controls the focus / resolution of attention
⢠Low resolution: Scouting groups of enemy; escorting group⢠High resolution: Search for air-defense entities; engage target
â Sets filters that select entities for WM
⢠Stimulus-driven controlâ Attention can be captured involuntarily by a visual event
⢠Muzzle flash (luminance contrast, abrupt onset)⢠Sudden motion (abrupt onset)
Goal-driven Attention
Sea
LandOverwatchPosition
OverwatchPosition Transport Carrier
Escort Carrier
Rendezvous Point
Escort task⢠Orient on group
⢠Voluntary grouping
Low Resolution
Stimulus-driven Control
High Resolution
No Attention versus Attention
525
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Summary
⢠Focus of attentionâ Zoom lens modelâ Perceptual grouping
⢠Controls over perceptual processingâ Stages of attentionâ Filtering
⢠Controlling attentionâ Goal-drivenâ Stimulus-driven
8th CGF & BR Conference11 - 13 May 1999
Copyright 1999 Institute for Simulation & Training
Current Work
⢠More realistic model of visionâ Foveal, parafoveal, and peripheral visual fields
⢠Visual search strategiesâ Tracking and projectionâ Situation awareness
⢠Perceptual learningâ Where should I look?â How long do I need to look?â What do I see? Learning by experience.