Fundamentals of New MediaMUMT 202

Cory McKay

Jason Hockman

course info

Fundamentals of New Media (MUMT 202)

• lectures: monday 5:35 PM - 8:25 PM in E-106

• labs: thursday 5:35 PM - 7:35 PM in E-106

– additional lab TBA if required

• course materials: on blackboard system

• office: 550 Sherbrook St. West, suite 500 (dial x0300 on hallway phone)

• office hours by appointment

course info

• course will be in 2 sections

• section 1: – instructor: Jason Hockman (jason.hockman@mail.mcgill.ca)– TA: Jung-Suk Lee (jungsuk.lee@mail.mcgill.ca)

• section 2: – instructor: Cory McKay (cory.mckay@mail.mcgill.ca)– TA: Bertrand Scherrer (bertrand.scherrer@gmail.ca)– TA: John Ashley Burgoyne (ashley@music.mcgill.ca)

main goals

to provide:

• an overview of new technologies related to music and music production

• an understanding of the basics of digital audio and digital signal

processing

• an exposure to the present state of artistic work in music technology

and new media

lecture schedule

Part 1

• Introduction to New Media

• Basic Concepts of Acoustics and Psychoacoustics

• Digital Signal Processing Introduction

• Music and the Internet

• MIDI, OSC and Symbolic Music

• Apple Logic

lecture schedule

Part 2

• Editing and Mixing Audio

• Audio Effects

• Score Editing

• Sound Recording

• Artistic Examples

• Overview of Music Technology Research Areas (if time permits)

• Optional Topics (if time permits)

lecture schedule

• 9/14: class 1• 9/21: class 2• 9/28: class 3• 10/5: class 4• 10/12: no class - Thanksgiving• 10/19: class 5• 10/22: class 6

– special Thursday class

• 10/26: class 7– test 1

• 11/2: no class• 11/9: class 8• 11/16: class 9• 11/23: class 10• 11/30: class 11• 12/3: class 12

– special Thursday class

– test 2

– final project presentations

part 1 part 2

deadlines and attendance

• attendance to all courses is required

• collaboration and mutual aid is encouraged

• cheating and plagiarism will not be tolerated

• students are expected to be familiar with McGill’s policy on

academic integrity

• part 1 will have 4 assignments, and one test

• part 2 will have 2 or 3 assignments and a final project

• all assignments are due at the beginning of class on the due date

• late assignment policy (20% off total grade each day late)

deadlines and attendance

• Part 1:

– Assignment 1 (5%) September 21

– Assignment 2 (5%) September 28

– Assignment 3 (5%) October 5

– Assignment 4 (5%) October 19

– Test 1 (30%) October 26

deadlines and attendance

• Part 2:

– Assignment 5/6 (10%) November 23 (Editing and Mixing Audio)

– Assignment 7 (5%) November 30 (Score Editing or Sound Recording)

– Test 2 (10%) December 3

– Final Project (25%) December 3

MTCL entry

• registered students should already have access to the lab

• any problems with entry: please see Building Director (room E234)

• network accounts: max 5 GB, 100 page (per 120 days) printing limit

• new user accounts email Darryl:

– login will be: lastnamefirstinitial (e.g., hockmanj)

– password: login name + 1234 (e.g., hockmanj1234)

• network account issues: Darryl Cameron (darryl.cameron@mcgill.ca)

introductions

. . .

what is new media?

• media: tools to store and deliver information– advertising

– communications technology

– print media

– can also be related to materials and techniques used by an artist

• integration of media into our lives expands with technology

• new media: tools developed relatively recently that relate to computerized communication and information distribution

• audio, video, internet and computer technologies

what is new media?

• new media development has led to:

– globalization

– increased communal knowledge across nation boundaries

• e.g., online higher education, wikipedia

– social and political change

• response to this year’s Iran election

• influence in determining recent US election

what is new media?

Audio

• production and performance technologies:– sequencers: Apple Logic, Steinberg Cubase, Abelton Live

– synthesis: Access Virus Snow

– auto-effects: ARTMA, BBCUT

– control and performance: Lemur, Reactable, Tenori-On, T-Stick, Serato Scratch, Mixxx

what is new media?

Video

• video and visual technologies:– video editing software: Apple Final Cut

what is new media?

Video

• video and visual technologies:– video editing software: Apple Final Cut

– image editing software: Photoshop, GIMP

what is new media?

Computational technologies

• internet– engines: Google, Yahoo

– social networks: Facebook, Myspace

– streaming audio: lastFM, Pandora, Grooveshark

– streaming video: Youtube, Vimeo

– media stores: itunes store, Beatport, Digital-tunes, Bleep, Boomkat

– torrent: pirate bay, mininova

outline of today’s material

• Basic Concepts of Acoustics and Psychoacoustics– source-medium-receptor chain

– acoustic and musical parameters

– simple harmonic motion and sinusoids

– complex tones

– superposition of waves / Fourier analysis

break

Psychoacoustics

• definition: the study of subjective human perception of sounds

• the study of psychoacoustics seeks to explain how we experience the phenomena of sound through an understanding of the intricate relationships between sound waves and our organs for perception

• why its important:– we need to know how sound propagates and how we perceive it

– basis for space-saving compression techniques

– used for distribution of sound within space - a.k.a. spatialization

Psychoacoustics

• area includes a broad range of topics: the ear, brain, and vocal organs, as wel;l as a neurological and psychological approach to concepts such as pitch, loudness, timbre, and spatialization

• also includes memory (important for timing measurements)

Psychoacoustics

• what is sound?

– traveling wave(s) that are oscillating through a medium with parameters that are in accordance with the range of the heard frequencies (20 Hz - 20 kHz)

– involves 3 systems: source-medium-receptor model• performer singing on a stage - source• air in the concert venue - medium• audience members - receptors

– source creates the energy, while the medium transmits the sound, and the receptor receives the signal via senses, and then perceives the sound through higher level cognition.

Psychoacoustics

• source is comprised of 3 components:

– primary excitation (e.g., blowing, bowing, striking)• energy source for system

– vibrating element (e.g., string on a guitar, head on a drum)• excitation elicits oscillation modes that generate musical pitch• upper harmonics influence timbre

– resonator (e.g., body on guitar or violin)• to transfer the oscillation from the vibrating element to the external

environment with greater amplification (and richer timbre)

Psychoacoustics

• medium (e.g., air, water)– when the traveling oscillations reach boundaries such as walls,

ceilings, floors, etc., the are either reflected or absorbed• various materials have different absorption characteristics• this determines the quality of room acoustics and reverberation

Psychoacoustics• receptor

– process of audition begins with our ears

• sound first hits our pinna (collects sound and filters it). pinna is also used for propagation analysis (2 ears)

• sound then into auditory canal• eardrum at end of canal -

sound waves cause vibration in eardrum, causing 3 bones on opposite side to vibrate (malleus, incus, stapes)

• nerve impulses are then created in the cochlea in a region called the basilar membrane (lined with ~3500 hairs that sense vibration)

• nerve impulses are then interpreted within frequency groupings, that ranging from 20 Hz - 20 kHz

Psychoacoustics

• Cochlea

Psychoacoustics

• Cochlea lower frequencies

higher frequencies

video

Psychoacoustics

• Pitch, Loudness and Timbre

– sound stimuli broken down into these 3 subjective measures

– pitch represents the fundamental frequency of a sound

• sine tones contain a single frequency

• complex tones have several frequencies (more on this later)

– loudness is a sensation of the intensity of a sound

– timbre is everything that is not pitch and loudness that can be used to

differentiate between two sounds

– tone burst must be tens of periods before it is recognized as pitch

– noise is without pitch, but contains both loudness and a timbre

• e.g., waterfall, fricatives

Psychoacoustics

• Pitch, Loudness and Timbre

– as mentioned these are subjective, and measuring these is difficult

– there are 3 objective measures sound identification

• fundamental frequency (oscillation rate of lowest freq. in harmonic series)

• amplitude (amplitude of the oscillation - equivalent to pressure on eardrum)

• spectrum (arrangement of partials)

Auditory Perception

• Fletcher: pitch perception varies with frequency– high frequency perception is result of cochlea excitation– low frequency perception involves a “time mechanism”

• Fletcher-Munson (1933): sound intensity level varies with frequency– i.e., given equally loud sinusoids of different frequencies, we hear

them as being different levels.– most audible = 2000 Hz - 3000 Hz– least = below 100 Hz and above 10000 Hz

• phon represents the perceived loudness of a sound wiht a given frequency

Equal-Loudness Contours

• Fletcher-Munson curves (1933): created by asking large number of subjects to rate loudness of different frequency sinusoids

Periodic Motion

• motion that is repeated in equal intervals of time– e.g., rocking chair, tuning fork, orbit around sun

– period: time it takes for one cycle

– frequency: inverse of period• e.g., tuning fork may have a frequency of 1000 Hz (repeats motion

1000 times per second, but its period is then 1/1000 or 1 millisecond)

• simple harmonic motion– point comes back repeatedly to the same position at exactly equal

intervals of time and exactly repeating the same type of motion in between

– can be represented as the projection of uniform circular motion on the diameter of a circle

– aka sinusoidal motion

Periodic Motion

• think of the displacement of a spring over time

Periodic Motion

Periodic Motion

• Sinusoid is then fully described by its– Frequency (f)

– Amplitude (A)

– Phase (φ)• measured in radians• 2π = 360 degrees = full cycle

• s

Periodic Motion

Complex Tones

• what happens when 2 or more sinusoidal waves with same

frequency are heard simultaneously?

– constructive interference: if the 2 waves share a common

frequency and phase, the result will be a sum of the amplitudes of

each of the original sinusoids

– destructive interference: if the 2 waves share only a common

frequency, and is 1/2 cycle out of phase (φ = π), then the amplitude

will be the difference between the 2 amplitudes

Complex Tones

• what happens when 2 waves with different frequencies are

added?

– resulting signal is no longer a sinusoid since it doesn't follow pattern

of simple harmonic motion

– resulting signal freq. = lower of the 2 original frequencies

– amplitude of the resulting signal = sum of the originals

– if two sinusoids are `in phase' then peaks and troughs coincide

– If sinusoids are out of phase then peaks and troughs oppose each

other and they will cancel each other out

Complex Tones

tone A

tone B

combined

Complex Tones

• this leads us to Fourier Theory, which implies that any complex periodic waveform can be decomposed into a set of sinusoids with different amplitudes, frequencies and phases

• following this model then, all sounds that we hear may be reduced into the linear combination of simple sinusoids