Virtual CharactersVirtual Characters

OverviewOverview

What is a digital character?What is a digital character? Why do would we want digital characters?Why do would we want digital characters? Scope of the problemScope of the problem What we want to tackleWhat we want to tackle How we want to tackle itHow we want to tackle it

What is a Virtual Character?What is a Virtual Character? Virtual character - Virtual character - a character who represents an executing a character who represents an executing

systemsystem In TRON (1982), In TRON (1982), humans humans and and humans that represents software humans that represents software

interacted within a interacted within a world that represented the hardwareworld that represented the hardware..

What is a Virtual CharacterWhat is a Virtual Character We look to to have We look to to have humans humans and and human the represent human the represent

software software interact in the real world.interact in the real world.

http://movies.yahoo.com/shop?d=http://movies.yahoo.com/shop?d=hv&idhv&id=1807432839&cf=trailer=1807432839&cf=trailer

Digital CharactersDigital Characters

A new medium to interact with system informationA new medium to interact with system information Spectrum from the paper clip -> AIBOSpectrum from the paper clip -> AIBO Examples:Examples:

Agents (AI, NLP)Agents (AI, NLP) Robots (Robotics)Robots (Robotics)

Life-Sized Virtual CharactersLife-Sized Virtual Characters Virtual Characters as a way to interact with informationVirtual Characters as a way to interact with information

If virtual characters are presented with an adequate level of If virtual characters are presented with an adequate level of realism, would people respond to them as they would to other realism, would people respond to them as they would to other people?people?

Effective Effective InteractionInteraction Natural (> than keyboard and mouse)Natural (> than keyboard and mouse) 3D, Dynamic (augmentable)3D, Dynamic (augmentable)

Effective Effective CollaborationCollaboration Non-verbal communication (60%)Non-verbal communication (60%) High impact?High impact?

Comparing Human-Human Interaction ModalitiesComparing Human-Human Interaction ModalitiesIM Phon

eVideo Phone

DigitalCharacte

r

3D Tele-Immsn

Face toFace

Input Verbal L H H H H H

Non-Verbal L N H ? (M) H H

Privacy H H L H L L

Output Verbal L H H H H H

Non-Verbal L N H ? (M) H H

Dynamic? M N N H L/? L

Efficacy Impact L M M ? (H) H H

Interaction L L L ? (M) H H

Logistics

Bandwidth L L M L H N/A

Cost L L M M H [ N – H ]

Availability H H L N/A L [ N – H ]

NN- None- None, L , L – Low– Low, M , M – Medium– Medium, H , H – High– High

Digital CharactersDigital Characters

Advantages:Advantages: Input: Non-Verbal, privacyInput: Non-Verbal, privacy Output: Non-Verbal, dynamicOutput: Non-Verbal, dynamic Efficacy: Impact, interactionEfficacy: Impact, interaction Logistics: Bandwidth, CostLogistics: Bandwidth, Cost

Disadvantages:Disadvantages: Input: Non-verbal fidelityInput: Non-verbal fidelity Output: Non-verbal fidelityOutput: Non-verbal fidelity Efficacy: Interaction fidelityEfficacy: Interaction fidelity

InteractionInteraction

Each participant in a communication has three stages: Each participant in a communication has three stages: perceptionperception, , cognitioncognition, and , and responseresponse

Investigate: Display, perception, efficacyInvestigate: Display, perception, efficacy

ThinkingParticipant

ThinkingThinking

Perceiving

Responding

Virtual Character

Responding

Perceiving

Responding

Perceiving

Interaction

What we plan on studyingWhat we plan on studying

There are many components in digital charactersThere are many components in digital characters We will focus only on interactionWe will focus only on interaction

Input (perception)Input (perception) How can we have the system capture non-verbal information? How can we have the system capture non-verbal information?

(emotion, gestures, eye-gaze, etc.)(emotion, gestures, eye-gaze, etc.) Proposed solution: Initially straps -> advanced computer vision Proposed solution: Initially straps -> advanced computer vision

techniquestechniques

Output (response)Output (response) How do we display the digital character?How do we display the digital character? Proposed solution: Evaluate different display technologiesProposed solution: Evaluate different display technologies

Eventually we will look at efficacy (future research)Eventually we will look at efficacy (future research)

How we plan on studying itHow we plan on studying it

Perception:Perception: STRAPS: Track head, arms, upper body, and objects (telephone).STRAPS: Track head, arms, upper body, and objects (telephone). Develop techniques to transmit eye-gaze and expressionsDevelop techniques to transmit eye-gaze and expressions User Studies: Evaluate effectiveness of each componentUser Studies: Evaluate effectiveness of each component

Response:Response: Evaluate the display modality. User Studies: Evaluate the display modality. User Studies:

Immersive vs Non-ImmersiveImmersive vs Non-Immersive Life-sized vs. Non-life sizedLife-sized vs. Non-life sized 2D vs 3D2D vs 3D

Goal: TV + webcams + laptopGoal: TV + webcams + laptop

Projects underwayProjects underway

Interpersonal communicationInterpersonal communication Distributed acting rehearsal (DAS Distributed acting rehearsal (DAS

student)student)

TeachingTeaching Deaf studentsDeaf students

Future work: Universal AccessFuture work: Universal Access DisabledDisabled MinoritiesMinorities Rural communitiesRural communities

RecruitingRecruiting

Looking forLooking for MS and PhD studentsMS and PhD students

EquipmentEquipment Wide area 12’x12’ trackerWide area 12’x12’ tracker V8 HMDV8 HMD 42” plasma TV42” plasma TV Cyberware ScannerCyberware Scanner 6 data projectors (stereo)6 data projectors (stereo) Firewire camerasFirewire cameras High-end PCs, videoHigh-end PCs, video

Work withWork with Cognitive PsychologistsCognitive Psychologists Computer ScientistsComputer Scientists Computational Computational

GeometryGeometry Aesthetic computingAesthetic computing Simulation/modelingSimulation/modeling Digital Arts and ScienceDigital Arts and Science