COM 597Streaming Media

Class 2

June 22, 2006

Today’s topics:

• The nuts and bolts of Audio• The encoding and delivery process (sampling

& resampling)• Aliasing• Audio Workshop• Codecs and Players• Compression demonstration• Compression workshop

Audio

• How do you deal with legacy material?

Audio sources:RecordsOpen reel tapeCDDVDOff-air recordingDownloaded filesOptical sound from a filmSurround soundEmbedded audio

Analog v. digital audio

What is the difference?

Analog material must be played in real timeYou will need to use a software tool or hardware device to convert these different physical representations of sound (grooves in a record for example) into a digital representation of the sound. This process can happen in a number of different ways during your production path.A-to-D converters (scanners)

Analog v. digital audio

What is the difference?

Sample rate will determine the fidelity of your final product

Remember you cannot add detail to a sampled media file so it is best to start with a file that is as large as is reasonable. (RAW v. JPEG for example)

Raw Image .jpg Image

Resolution Comparison

Source v. Delivery

Raw Image .jpg Image

Resolution Comparison

Source v. Delivery

• Sample rate is not bits-per-second, although they are related

• Sample rate is the number of “slices” of sound per second where bps is about how much data is begin delivered in the signal. Bps can be 22KHz or 96KHz or even 192KHz. For this class most of our work will be either in 44.1KHz or 48KHz.

• When you sample a sound you need to record both the wave peaks and troughs

• Human ear is limited to detecting frequencies

no higher than 22KHz. • It was believed that simply doubling this

number would give a sample rate that would reproduce all the sounds we can hear. Therefore, CDs run at 44.1KHz

Resampling a fileAnd the problems inherent in the

process

The difficulty with 44.1 as a standard is that it does not integrate easily into a

standard production path.• Most audio files are converted using integer-

based averaging. • This becomes apparent when you see that the

common rates used for mastering are 48, 96 and 192 Khz. Moving down in quality involves simple averaging and not a complex resampling of the file.

Aliasing Artifacts

Aliasing Artifacts

Aliasing Artifacts

Aliasing Artifacts

Aliasing Artifacts

Aliasing Artifacts

• These quantization errors become evident when you convert between CD rates and film/television production rates. It is important to know your delivery before you start the process of sampling your sound.

• This is also why it is important to make certain all the tools in your production path are in agreement. (set to the same sample rate)

Audio formats you are likely to encounter

• Mono one track of audio• Dual Mono (this is not stereo) identical audio on

both tracks• Stereo multi-channel audio on two tracks• Mix Minus production audio with two mono tracks

containing different sources (dialogue on one and effects on the other is one

common example)• 5.1 surround Home theater is the most common

application with three front speakers (left, right and center), two rear speakers and once low-frequency speaker (sub-woofer)

• AC3 An alternative to Dolby Labs 5.1 surround format.

Mixing and blending of tracks needs to be carefully managed – phase and resampling errors

Flash example

• Workshop

Codecs

• Which is best? There really is no one-stop solution.

• All of the major standards do a very good job of compressing most media

• The differences are subtle

There are real advantages to choosing one standard as your default distribution method

• Users will only need to rely on one playback solution

• Hardware will be optimized to that solution (servers and production tools)

• Licensing of technology is easier• Staff is trained with emphasis on only one

platform• Support is easier

There are real disadvantages to choosing one standard as your default distribution method

• Not a large user base for the technology• Format of files is not competitive with

changes in the marketplace• High licensing fees• Flaky software• Challenges with using the files on all types of

hardware (WM on a Mac)• You are tied to the fortunes of the company

that created your codec

Format compatibilities and incompatibilities

• QuickTime Mac and PC are OK, no Linux support

• Windows Media PC is great, Mac support is limited to older versions, no Linux

• Real Networks Supports most every operating system

• Flash Supports most every operating system because it runs in a browser

• MPEG will play in many of the proprietary systems H.264 shows promise as a common standard.

How might you examine a codec to see if it will work for your needs?

You might choose one compression solution for your production and another for delivery to your users.

Additional degradation of quality is likely

General rule is that as compression schemas improve the files get smaller and the images either improve or stay the same.

Let’s look at how we compress a file to get an understanding of all the variables we can manage

Prior to compression you need to get your media in the best shape possible. Often it is easier in production to prep the media than it is to heal bad video during compression.

Video Preparation

• Deinterlacing the footage

• Frame rate agreement (2-3 cadence, 15fps, PAL v. NTSC, HD)

• Noise filters (dubious benefits at best)

• Grain removal

Video Preparation

• Color correction (gamma adjustments)

• Scaling (size your canvas can speed up compression significantly)

• Aspect ratio adjustment including anamorphic adjustments

Compression Example

• What the heck do those letters mean? And why do I care?

• I-frame Intra frame or key frame• P-frame predicted difference frame• B-frame bi-directional difference frame

• Compression Demo