AV Music for Mobile Phones

Affordances of Mobile Phones

● Ubiquitous● Many modes of interaction

– GPS

– Accelerometer/Gyroscope

– Camera

– Microphone

– Multitouch

● Networking capabilities

Categorization of Mobile Music Projects

● Location● Collaborative composition● Interaction using wearable devices● Phone as performance tool

Often projects focus on social, technological, and geographic aspects

(John)

Example Mobile Music Activities

● Pushing music to people nearby● Sonifying local wifi coverage● Remixing music tracks with remote friends

(Gaye et. al)

Brief History of Telephone Art

● Laszlo Maholy Nagy (1922)

● Max Neuhaus

– Public Supply 1 (1966) -> Audio● Disembodied Art Gallery

– Babble (1993) -> Soundcloud

– Temporary Line (1993/4)● Thomson and Craighead

– Telephony (2000) -> Video

Dialtones (A Telesymphony) (2001)

Vimeo● By Golan Levin, Gregory Shakar, and Scott

Gibbons● Each audience member contributes his/her

phone number and seat number before the concert.

● Socially oriented composition

Dialtones (A Telesymphony) (2001)

Interface as visual music● A mirror is placed above the audience● The computer interface is projected

“Visual-musical software instrument”

“...this structure also allows for the exploration of a broad range of musical interaction-models: from the deeply practiced (e.g. Gibbons' solo performance), to the entirely visual (e.g. the graphical interface controls used by Levin and Shakar to interactively perform the audience phones), to a lightweight model of consumer participation (e.g. through one's selection/purchase of a phone model, negotiation of its ringtone, and manner of displaying it during the performance).”

SimpleTEXT (2003)

Video● By Jonah Brucker-Cohen, Duncan Murphy, and

Tim Redfern

● Audience members send texts to the system● “...[text] messages are first parsed according to a code that dictates

how the music is created, and then rhythmically drive a speech synthesizer and a picture synthesizer in order to create a compelling, collaborative audiovisual performance”

● Similar project: The exchange by Giles Perring -> exchangeart.co.uk

Augmented Reality

● Masayuki Akamatsu – Seeing Objects Moving -> video

● Project Paperclip -> video

● Alejandro Schianchi – Sin Titulo 2011 -> schianchi.com

● S/N Coalition – Handy Video -> video

Project Paperclip

Video

● Augmented reality photo exhibit

● Each photo has an associated soundscape which is accessed using the phone's camera

● Real-time processing using...

– Time of day

– Location

– Background noise

– Movement

Handy-Video

Video● By the S/N Coalition● QR codes are posted around Berlin● Each code accesses an AV “remix” of the

local soundscape and street art

Interactive Imaginary Environments

● Radiohead – Polyfauna -> video● Bjork – Biophilia -> video● Rob Hamilton – nous sommes tous

Fernando -> ccrma.stanford.edu – (not actually for mobile but so similar that I'm including it anyway)

Simple Interactive Apps

● Brian Eno – Bloom – Interface as visual music

● Thicket – Interface as visual music

● Phonopaper – Maps image to audio

● Ge Wang – Ocarina – Performance tool with social aspect

Games

● Audio surf● Optic Echo● Sound Swallow

Visual Music Interfaces

● George Essl, Michigan Mobile Phone Ensemble, urMus

– Space Pong -> video

– Color Organ -> video

– Glow Music -> video

● Vox Tactum/Vuzik– Aura Pon – Intertwine -> video