audio for telepresence and virtual reality colin s. harrison

Audio for Telepresence and Virtual Audio for Telepresence and Virtual RealityReality

Colin S. HarrisonColin S. Harrison

Objective is to enhance sense of hearing for Telepresence and integrate with Objective is to enhance sense of hearing for Telepresence and integrate with Virtual Environment LaboratoryVirtual Environment Laboratory

• Replicate feeling of being thereReplicate feeling of being there

• Hear a sound – turn to look at it Hear a sound – turn to look at it

• Replicate human hearingReplicate human hearing

• Localise azimuth, elevation, range, properly externalisedLocalise azimuth, elevation, range, properly externalised

• Use audio to augment video for improved sensation of presenceUse audio to augment video for improved sensation of presence

Human 3D Spatial HearingHuman 3D Spatial Hearing

• Differences in Time (ITD)Differences in Time (ITD)

• Differences in Intensity (IID)Differences in Intensity (IID)

• Movable Head Movable Head

• Pinnae Filtering (Head Related Transfer Function)Pinnae Filtering (Head Related Transfer Function)

ImplementationImplementation

• Binaural sound / Simulated head Binaural sound / Simulated head (HRTF) for Headphones(HRTF) for Headphones

• Integrate with Telepresence ProjectsIntegrate with Telepresence Projects

• B-Format for loudspeakers B-Format for loudspeakers

• Integrate with VE aspects: ABACUS suiteIntegrate with VE aspects: ABACUS suite

• Lake DSP for Headtracked Binaural Lake DSP for Headtracked Binaural Simulation and Loudspeaker reproductionSimulation and Loudspeaker reproduction

Originality Originality

• Binaural -Two eared technique Binaural -Two eared technique • Ear/sensor assembly Ear/sensor assembly • Mechatronic headMechatronic head

•World first system World first system • with physical binaural sensorswith physical binaural sensors

• B-Format approach for Telepresence ( new Application)B-Format approach for Telepresence ( new Application)

• No other UK VE site has installed realistic sound No other UK VE site has installed realistic sound

Virtual Environment IntegrationVirtual Environment Integration

• Add realistic sound to VE presentations of architectural Add realistic sound to VE presentations of architectural projectsprojects

• Change sound as position and room type changesChange sound as position and room type changes

• Link to existing Virtual Environment LaboratoryLink to existing Virtual Environment Laboratory

PublicationsPublications

•Conference Paper: Conference Paper: (Mechatronics 98)(Mechatronics 98)

•Journal Paper:Journal Paper: (by invitation Mechatronics)(by invitation Mechatronics)

Industrial RelevanceIndustrial Relevance

• Hostile human environmentsHostile human environments

• Nuclear Chemical -ChernobylNuclear Chemical -Chernobyl

• Surveillance (Big Brother hears you too!)Surveillance (Big Brother hears you too!)

• EntertainmentEntertainment

• Architectural walkthroughsArchitectural walkthroughs

• Remote Military InterventionRemote Military Intervention

And FinallyAnd Finally

• Telepresence: seamlessly represent senses: consider a Turing Telepresence: seamlessly represent senses: consider a Turing TestTest

• Accurate sound triggers sense memoriesAccurate sound triggers sense memories