audio-only augmented reality system for social interaction
DESCRIPTION
Presentation of Tom Gurion's master thesis project from the Music and Technology program, Bar-Ilan University.TRANSCRIPT
Audio-Only Augmented Reality System for Social InteractionTOM GURION1 AND NORI JACOBY1,2
1BAR-ILAN UNIVERSITY, RAMAT-GAN2HEBREW UNIVERSITY, JERUSALEM
Augmented Reality
Google Glasseswww.google.com/glass/
Decolabshttp://www.decolabs.com/
Wikitudehttp://www.wikitude.com/
Augmented reality in the audio domain?
Introduction and framework:Interactive music systems
ADA: The Intelligent Space (2002)http://specs.upf.edu/installation/547
Reactable (2007)http://www.reactable.com/
Introduction and framework:Interactive music systems
AutoRap by Smule (2012)http://www.smule.com/autorap
RjDj – Sonic Experiences (2008)http://www.smule.com/autorap
Introduction and framework:Social context“A flash mob is a group of people who assemble suddenly in a public place, perform an unusual and seemingly pointless act for a brief time, then quickly disperse, often for the purposes of entertainment, satire, and artistic expression. Flash mobs are organized via telecommunications, social media, or viral emails.”
(Wikipedia)
Introduction and framework:Social context“Silent disco has become a common name for a disco where people dance to music listened to on wireless headphones. Rather than using a speaker system, music is broadcast via an FM-transmitter with the signal being picked up by wireless headphone receivers worn by the participants.”
(Wikipedia)
Research targetsPropose and implement an audio-only augmented reality system for social interaction.
Evaluate the system in the context of silent disco party.
More generally, explore the potential of new ways of music consumption through mobile devices.
System Description
Overview
The six balloons bundles Bluetooth beacon
Overview
App Screenshot The partyApplication page in Google Play
Play / Pause
Button
Our positioning requirementsWhat we need:
Accuracy of about one meter
Mobile
No infrastructure
What we don’t need:
Exact positioning
The solution:
Bluetooth Based Relative Positioning system
Bluetooth discovery routineRepeatedly search for Bluetooth beacons.
Extract received signal strength indicator (RSSI) from the Bluetooth device discovery and use it as an estimation of the distance between the user and the beacon.
Send the device address and RSSI value to Pure Data patch using libpd API (Brinkmann, 2011).
Player patchEach one of the six beacons correspond to pre defined sound zone player.
The patch receives RSSI values from the app and route them to sound zone player according to Bluetooth address.
Each sound zone uses the RSSI value differently to manipulate the music in real-time (E.g. volume, music filtration, etc.)
Evaluation: preliminary results
Experiment designParticipants were randomly assigned to two groups.
Both groups started the experiment together.
In the control blocks participants heard pre composed music based on the music of the interactive system.
They were informed that the experiment consists of interactive and control segments.
Group A consists of 8 participants (4 females and 4 males) with mean age of 36.7 (s.d = 12.3); group B consists of 10 participants (3 females and 7 males) with mean age of 29.6 (s.d = 10.2). Participants had a diverse musical background with 4.7 mean years of musical training (s.d = 5.2).
MeasurementsSurveys
Each participant filled pre/post party surveys that included questions regarding their musical background and preferences as well as system evaluation feedback.
Bluetooth discoveries countCounting the number of Bluetooth device discoveries made by the participants’ phones during the interactive and the control blocks. In order to eliminate edge effects, we analyzed only the two middle blocks of the experiment.
Positioning trackingVideo capturing of the party from above the dance floor. Analyzed offline.
Results: SurveysWe found that using the interactive system participants tends to dance less with people they knew in advance.
Participants self-reported significantly higher levels of movement using the system.
We didn’t find supportive results to show that participants dance more with strangers.
Dancing with known people: paired t-test, t(14) = -2.5, p_value = 0.01Changing location in space: paired t-test, t(15) = 3.9, p_value < 0.01
Results: Bluetooth discoveries countWe found higher counts during the interactive blocks of the party compared with the control blocks.
paired t-test, t(16) = 1.7, p_value = 0.06, non-significant
Video trackingTracking participants manually in Pure Data to generate positioning tables.
Results:Positioning trackingWe didn’t find significant difference between the position in the interactive and control blocks. However position was indicative to the social relations between participants!
We asked participants for their social relationships to other participants.
We compared the resulted map with video correlation index.
The results shows that the two matrices are unusually similar.
p_value < 0.001
ConclusionsOur system demonstrate a simple way to use Bluetooth technology for relative indoor positioning.
Our preliminary results demonstrate the potential for audio-only augmented reality to significantly enrich the experience of music consumption and social interaction.
We show that the social implications of the system can be validated in a controlled experiment.
We show that positioning tracking can be used to obtain social relationships within large group of people.
Future researchWe will use the devices accelerometers to asses entrainment.
Larger and more comprehensive experiments with a different design.
Thank you!