Slide 1

Slide 2

COMPUTER AUDIO CGDD 4003

Slide 3

What is Sound? Compressions of air or other media (such as water or metal) from something vibrating Sounds are made up of high frequency and low frequency sounds Dont confuse pitch (frequency) with volume! Volume is measured in decibels (dB) Frequency in Hertz (Hz) = cycles per second Humans only hear from 20Hz to 20KHz Frequency Strange Fact! Speed of sound (Air: 340m/s; Water:1,230 m/s; Gold: 3,240m/s):

Slide 4

Spatial Sound 1 Channel mono. Can be split to several speakers; still no direction 2 Channels stereo. Fades from left to right. Can determine direction 5.1 Audio Common for home theaters 3D Sound? Video games (PC). Still has time to develop

Slide 5

The Human Side (20Hz-20KHz)

Slide 6

The Equal Loudness Contour Killa Hurts Ouch!

Slide 7

A Note about decibels A decibel is 1/10 th of a bel Abbreviated dB This is the perceived loudness, which increases linearly as power increases exponentially Something sounds twice as loud? 10log 10 (2) = 3.01dB In gaming, volume usually ranges 0.0f-1.0f

Slide 8

Human Perception (InterAural Time Difference) Sound hits both ears Difference in time Hits Right Ear First Hasnt gotten to left yet

Slide 9

How Computers Perceive Sound Digitization (DAC and ADC) Computers listen to the amplitude a certain number of times per second (sample rate) 44K is CD 22K is good 8K is lame Computers have to approximate what they heard and assign it a number 4 bits = 16 level to approximate to 16 bits = 2 million levels to approximate to

Slide 10

Original Sound Amplitude (in dB)Frequency

Slide 11

Low Sampling Rate TIME

Slide 12

Low Sampling Rate What the computer hears TIME

Slide 13

High Sampling Rate TIME

Slide 14

High Sampling Rate TIME

Slide 15

2 bits per sample 4 Approximations TIME

Slide 16

StairStep Effect TIME Called quantization errors

Slide 17

3 bits per sample 8 Approximations TIME

Slide 18

Less StairStep TIME

Slide 19

Signal to Noise Ratio (SNR) Represents the quanitization error 8-bits = 128 discrete values (upper-half only) Sample is rounded up or down SNR is 256:1 256:1 translates to 48dB (difference in average noise to max signal) 16-bit = 32K discrete values (upper-half) SNR = 65,536:1, or 96dB

Slide 20

In General Sampling rate affects range of frequencies you can capture (Nyquist) Bits per sample affects noise level as well as volume range What about recording: Rock? Mozart (or anything on NPR for that matter)? Voice/dialog?

Slide 21

Capturing Sounds Usually done with: a microphone (such as voice) Line in CD Hollywood Edge Computer has sound card Input types (RCA, MIDI, mini, , XLR) Card has quality (plays 16-bit sound) Need some kind of software SoundForge/Audacity Windows SoundRecorder (gag)

Slide 22

Typical Pipeline Permanent Storage Decoding (from mp3, ogg, etc) Memory Buffer Sound Channel Processing (2D/3D effects) Hardware mixing and DAC Individual Channel

Slide 23

Sample Playback Playback Loaded entirely into memory (called sample as well) Streamed (pre-buffer data using a circular buffer) Channel properties Pan left/right Pitch frequency Volume

Slide 24

Compressed Audio Requires a codec (compress/decompress) Lossless (e.g..zip files) Lossy Bit-reduction (ADPCM, reduces bps from 16 to 4) Simple Used on Sony PSP, Wii and Nintendo DS Physcho-acoustics (.mp3,.ogg,.wma) Discard sound we dont normally hear anyway Hard to implement CPU intensive PS3, Xbox 360, PCs Note: mp3 format requires licensing fees to Franhofer-Thompson!

Slide 25

ADSR Envelopes Used for defining the volume of a sound Attack Decay Sustain Release Time Volume

Slide 26

3D Sound Dont have 5.1? Panning is one option Psycho-acoustic options Head-Relative Transfer Function (HRTF) Tweak the frequencies to match your ears Sounds have position and velocity There is a listener component (like a camera) Relationship between the two Attenuation (with distance) Occlusion (low-pass filter) Doppler (relative velocities)

Slide 27

3D Sound Environmental effects Reverb (depends on materials in room) Echo (depends on size of room) Occlusion (a wall blocking part of the sound) Obstruction (no direct path to the listener Competing reverb technologies I3DL2 (Interactive 3D Audio Rendering Level 2) EAX (Creative Labs) Almost identical

Slide 28

MIDI (Musical Instrument Digital Interface) MIDI a method for representing sounds electronically Became popular in the 80s Send 16 different channels (tracks) at one time Have a total of 128 possible instruments

Slide 29

The Keyboard The MIDI Keyboard No audible sounds Generates a series of 1s a 0s (on/off) Signals represent Note, loudness Length, type of instrument Signals come out of the keyboard and usually go into a sequencer

Slide 30

The Sequencer Can be a PC Responsible for recording individual tracks of music Responsible for playback Receives input from keyboard Sends output to synthesizer

Slide 31

The Synthesizer Receives 1s and 0s from the sequencer Interprets the 1s and 0s to produce audible sounds Piano Drums Saxophone Sounds are sent to speakers

Slide 32

Speakers Like you havent seen these before

Slide 33

MIDI 01101101000110 010010

Slide 34

MIDI vs Digital Recording MIDI: Smaller file size (like 10-20K) Change keys/tempo/looping on the fly! Song sounds different on every sound card No singing allowed! Also a DLS format (DownLoadable Sound) Digital Recording: Larger file size (like 5M) Sound is close approximation to real thing

Slide 35

Sampling There are two main approaches to synthesis: Sampling FM Synthesis Sampling A sample is a recording of actual instrument/sound Samples are taken at certain intervals Samples are then shifted up or down depending on the note

Slide 36

Sampling

Slide 37

FM Synthesis Basic waves: Sine Square Saw Triangle Noise

Slide 38

FM Synthesis Start with basic waveform, and have one wave modulate the other Heres volume modulation 440 sine wave, control 2Hz: 440 sine wave, control 880Hz: 440 sine wave, control 3KHz:

Slide 39

Interactive Music Music adapts based on current state of game Music broken into chunks Called segments (or cues) Can be played back to back Can be smoothly cross-faded Segments are combined into themes fmods Sound Designer can do this

Slide 40

Themes in fmod

Slide 41

Sound Variations Sounds can be triggered by events Theres no reason to play the same sound the same way Pick a random sample Change pitch Change attenuation

Slide 42

Other technology Lip-synch Use the amplitude of the wave to control mouth Analyze phonemes of sample (language neutral)

Slide 43

Common Audio Technology XAudio (free) cross-platform OpenAL (free) cross-platform XACT (free) Xbox/Windows fmod (commercial) cross-platform