Digital Media

Dr. Jim Rowan

Chapter 2

Today’s Question!

• Phenomenon in the real world can be described as having two different modalities. Discrete phenomenon are easy to convert to digital because you can simply count them. Continuous phenomenon are a bit more complicated because you can’t simply count them, instead you must do what?

Before we start…other software you will be using

(all found in the application folder)• Grab - used to do a screen capture• TextEdit - used as a simple word processor• PhotoBooth - used to take your picture with

the built in camera and take video with the built in camera

• Quicktime Pro - to convert videos to a number of different formats and check the sample rate and sample size

The Question:

• How do you put stuff in a computer – so that you can manipulate it– so that you can send it– so that someone else can see and use it?

• How do you represent the real world in a digital world?

The answer:

• Represent the real world as numbers

• Store the numbers

• Transmit the numbers

• Retrieve the numbers

• Display them in a form humans understand

Today:

• Chapter 2 is a “first cut” of nearly all the material that will be covered in greater detail this semester

• About the real world

• About digital representation

File formats and extensions(like .au, .doc, .ppt, .mov)

• Indication to us (the humans) what kind of file this is

• Some software looks at the extension– so... some software will try to open files

with improper extensions– results in “file corrupted” error message– try it... change the extension from .doc

to .jpg

File formats and extensions

• Some software looks at the data in the file for more definitive answer (the header)

• important file-related information is encoded in the data of the file

• for example: some image formats have color tables to reduce the size of the file

• some video just saves the changes from one frame to the next

• we’ve seen the header before when we used hexFiend to look at images… image size is stored in the header

But it’s all just numbers, and binary numbers at that!

Note on paper

Picture

Song: fieldsOfGold.mp3

Video

Numbering systems:

DecimalBinary

Hexadecimal

Numbering systems

• Humans: decimal– Humans: 10 fingers, 10 digits– 0, 1, 2, 3, 4, 5, 6, 7, 8 & 9

• Computers: binary– Computers: 1 finger, 2 digits– 0 & 1

Hexadecimal

• Humans and Computers: hexadecimal– Hexadecimal: 16 fingers, 16 digits– 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F– Each of these can be represented as binary by

using 4 binary digits for each hex digit• while this seems complicated it is actually easier for

humans to deal with 16 different digits than 4 0s and 1s

• with 4 binary digits you can represent 16 different things

• [number of digits in the numbering system]**[number of digits used]…

• 4 binary digits ===> 2**4 = 16• 4 hexadecimal digits ===> 16**4 = 65,536

How to count using a different number of fingers

(it’s the same process but different number sets)

• 10 fingers: Counting in decimal – 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,– start over with 0 and increment the digit to the left

• 1 finger: Counting in binary– 0, 1– start over with 0 but increment the digit to the left

• 16 fingers: Counting in hexadecimal – 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F– start over with 0 but increment the digit to the left

Binary Coding

• Data for a computer... binary– zeros and ones, – off and on– false and true

• Data for humans... ASCII, Hex... (others)– Two of the coding schemes used are:

• Hexadecimal represented by 1 Hex => 4 bits• ASCII represented by 2 Hex codes => 8 bits

• BUT… – they all end up as 0’s and 1’s

Example: ASCII Code

• Humans and Computers: ASCII– ASCII code for C

• decimal: 67• hexadecimal: 43• binary: 0100 0011

From the Real Worldto

Stuff on a computer• A note

– Paper and pen -> bits (0s and 1s)

• A picture– Reflected light -> bits (0s and 1s)

• A song– Pressure waves in air -> bits (0s and 1s)

• A video– Pressure waves in air and Reflected light -> bits (0s and 1s)

First, the real world:

Discrete&

Continuous

Phenomena in the Real world: discrete vs continuous

• Things in the real world can be discrete• They either ARE or ARE NOT there• These things can be counted• Examples:

– The number of cars in the parking lot– The number of beans in a jar

Phenomena in the Real world: discrete vs continuous

• Things in the real world can be continuous• Continuous can’t be counted, it must be

measured• Examples:

– Atmospheric pressure– Height of an ocean wave– Frequency of a sound wave

But... computers can only count

• Discrete data is easy for a computer– count it and store it as a number

• Continuous data... easy? not so much– music:

• measure the frequency & amplitude• encode as a collection of numbers

– pictures: • measure the amount (intensity) and frequency of light (color)

in a number of regular places (pixels)• encode the frequency and the intensity as a collection of

numbers

Question...

• If computers only store 0s and 1s...

• How does all this continuous stuff end up in a computer so that we can save it and play it back?

• Answer– Continuous data must be converted to

discrete data

Continuous phenomenon to digital data:-SAMPLE!Sampling consists of two processes

1) stop to take a measurementthe number per time period is called the sample

rate

2) take the measurementthe number of different values each sample can

take on is called the quantization level

Digital data back to continuous phenomenon:– Display samples using “sample and hold”

• Play the sample for the duration of the sample time

From the Real World …and Back!

But... How many samples do you

need?

It depends…

single sample

single sample

single sample(sample and hold)

two samples

two samples

two samples (sample and hold)

three samples

three samples

three samples (sample and hold)

four samples

four samples

four samples (sample and hold)

five samples

five samples

five samples(sample and hold)

How frequently should I sample?

• too few– small file size (good)– not a faithful representation when replayed

• too many– large file size (bad)– excellent representation when replayed

• The Nyquist rate – twice as many samples as the frequency– Results in an ok file size– faithful representation when replayed

CD quality is44,100 samples per second

• Why?– Human hearing response is in the range of 20

to 22,000 cycles per second

• Nyquist sample rate = highest frequency to be captured = 22,500 CPS

2 x 22,500 = 44,100 samples per second

Looking at FieldsOfGold.mp3…

• 4 minutes and 59 seconds long• 1,201,173 bytes in lengthCan this be right?• CD quality

– 44,100 samples per second (sample rate)– 16 bit samples (quantization level is 16 bit)– 16 bits can store 65,536 different levels

• (2**16 = 65,536 individual levels)

FieldsOfGold.mp3

• 4’59 = 299 seconds long• 299 x 44,100 samples per second

= 13,156,000 samples

• 13,185,900 x 2 bytes/sample (2 bytes = 16 bits)– 26,371,800 bytes

• Stereo: 2 channels => 52,743,600 bytes• Should be 52+ megabytes!• Why does it show only 1.2 megabytes?• HMMMmmm...

FieldsOfGold.mp3

• Why 52+ megabytes not 1.2 megabytes?– wait for it…

FieldsOfGold.mp3

• Why 52+ megabytes not 1.2 megabytes?

• This is an MP3!

• The data is COMPRESSED!

• If you had the song on a CD it would be 52+ megabytes long and in .aiff format

Two types of compression

• Lossy

• Lossless

.mp3 audio

.jpeg imagesrun length encodingtable compression

Further reading

• http://en.wikipedia.org/wiki/Nyquist_rate

• http://en.wikipedia.org/wiki/Sampling_%28signal_processing%29

• http://en.wikipedia.org/wiki/Mp3

The side effects of sampling:sampling artifacts

Sampling Artifacts are the negative side effects caused by having to sample

continuous data

Sampling Artifacts

• Under-sampling: not enough samples being taken of continuous data can produce undesired artifacts

• Examples might be:– Moire’ patterns on images– retrograde motion on video

Sampling Artifacts (cont.)

• Not enough quantization levels when sampling continuous data can produce undesired artifacts

• Examples might be:– too few grey levels: gradients become steps– too few brightness levels: posterization

Sampling ArtifactsRetrograde Motion

4 samples/cycle, 2 cycles

2 samples/cycle, 2 cycles

Sampling Artifacts (cont.)

• Audio – too few amplitude levels, quantization

noise• 8 bits (256 amplitude levels) produces

discernable noise• 16 bits (65,536 amplitude levels) CD quality,

no discernable hiss

– general sound “fuzziness” or a “flat” sound

Data RepresentationsHow stuff is stored in a computer

• Images• Bitmapped• Vector

• Audio• Animation• Video• Text

Images, bitmapped

• Are stored as arrays of pixels

• Can be stored directly – TIFF for example

• Can have an associated color map– JPEG for example

• Generating these pixels from the stored model is called rendering

Images, vector graphic

• Are stored as mathematical descriptions

• Often smaller than bitmapped

• Size is independent of resolution or image size

• Not suitable for some type of images

Example & Comparison• Bitmapped graphics

– Defined as spots (pixels) of color– Problems scaling– File size unaffected by image complexity– File size affected by the image size

• Vector graphics– Defined by their parts– File size affected by image complexity– File size unaffected by the image size

(scaling is easy)

Moving images

• Captured live with camera– iMovie– Stored as video

• Generated from animation– Blender– Similar to 2D vector graphics… but in 3D and

with a means of creating motion

Network communication

But first…what this stuff means:

Bit: binary digit

Byte: 8 Bits

KB: kilo byte (1000 bytes)

MB: mega byte (1,000,000 bytes)

GB: giga byte (1,000,000,000 bytes)

KBPS: kilo (1,000) bits per second

MBPS: mega (1,000,000) bits per second

Network access...

• dial up connection– phone modem– limited to 56,000 bps (bits, not bytes) max

downstream (internet to modem)– 33.6 kbps upstream (modem to internet)– rarely get these speeds

Network access...

• ADSL – asymmetric digital subscriber line– over copper phone wires– limited to short distance from phone switch– 6.1 mbps downstream– 640 kbps upstream

Network access...

• Other options– Cable modem (also asynchronous)– satellite with phone (also asynchronous)– satellite alone (expensive but available in the boonies)– local wireless networks– high altitude tethered balloons– transmission over power lines

Network access...Commercial internet users

• T1 connection 1.544 mbps

• T3 connection 44.7 mbps

• Provide web servers for others to put websites on

• Large commercial enterprises will have their own web server

Time-to-download/upload calculations

• The Speeds:– Dial-Up

• 56,000 bps internet to modem (downstream)• 33,600 bps modem to internet (upstream)

– ADSL• 6.1 mbps (million bps) downstream• 640 kbps (thousand bps) upstream

– T1 • 1.544 mbps

– T3 • 44.7 mbps

NOTE!

bps is bits per second while filesize

is stated in bytes

Time-To-Load calculations

For a 1.2 megabyte video:

How long would it take to load it to youTube over-fastest dialup-adsl-T1-T3

How long would it take to download it from youTube over-fastest dialup-adsl-T1-T3

Note this!

• Communications are usually stated in bps (bits per second)

• Filesize is usually stated in bytes

• AND: 8 bits per byte– you will have to convert from one to the

other when you do download/upload calculations

Servers and Clients

Servers & Clients...

• Clients consume and display internet content• Your browser is a client• Clients request content from servers

– by sending a server an HTTP://URL message which is a request for a web page

• Servers respond to requests for internet content– send requested web pages to Clients

• The content is sent in HTML code– HTML sent by the server is interpreted by the client (browser) and

displayed on your display

• Look at http://www.pondliner.com/ and view source

URL (uniform resource locator)… a human-readable name

• URL takes the form:

http://www.amazon.com/newStuff/index.html

• URL has 3 parts– the protocol that you are using (http://)– The domain name: (www.amazon.com)– The directory and file you want to see:

(newStuff.index.html)– the URL maps to a number called an IP address

Servers & Clients...

• servers have fixed IPs so they are easy to find• your computer probably uses DHCP which is a

dynamic (changing from connection to connection) IP

• An example: my IP right now (assigned through dhcp) is: (look it up in system preferences)

• IPv4 vs IPv6

The Internet

your computer

DHCP:

your browser(Safari)(client)

yahoo.com(server)

235.01.30.164

walmart.com(server)

100.43.153.07

Domain Name System (DNS)

requestedwebpages

yahoo.com(server)

235.01.30.164

The Internet

walmart.com(server)

100.43.153.07

ISP

DHCP: 135.10.34.143

your computer

your browser(Safari)(client)

Domain Name System (DNS)

requestedwebpages

yahoo.com(server)

235.01.30.164

The Internet

walmart.com(server)

100.43.153.07

DHCP: 135.10.34.143

your computer

your browser(Safari)(client) http://www.yahoo.com

Domain Name System (DNS)http://www.yahoo.com =235.01.30.164

requestedwebpages

yahoo.com(server)

235.01.30.164

The Internet

walmart.com(server)

100.43.153.07

/index.html

DHCP: 135.10.34.143

your computer

your browser(Safari)(client) http://www.yahoo.com

Domain Name System (DNS)http://www.yahoo.com =235.01.30.164

requestedwebpages

Questions?