lect8-compress2

CS 414 - Spring 2014CS 414 Multimedia Systems Design Lecture 8 Basics of Compression (Part 2)

Klara NahrstedtSpring 2014

CS 414 - Spring 2014

CS 414 - Spring 2014Administrative MP1 is postedLeasing Web cameras + check camerasCheck group directoriesReading: Media Coding and Content Processing, for Compression - Chapter 7Basic Coding Chapter 7.1-7.4 JPEG Chapter 7.5MPEG Chapter 7.7JPEG 2000 and H.264 see lectures and links in the lectures


Comments on Reading for Audio, Video, Images ContentReading: Media Coding and Content Processing Audio Technology: Chapter 3Chapters 3.1-3.2 + LecturesImages: Chapter 4Chapters 4.1 + Lectures and links in the lecturesVideo: Chapter 5 (all + lectures) CS 414 - Spring 2014


Statistical Entropy Coding Huffman codingWho patented Arithmetic Coding? IBM had most of the patents on arithmetic coding Ricoh had patent on one of the methods and apparatus for entropy coding. CS 414 - Spring 2014Today Discussed Topics


Huffman Encoding Statistical encodingTo determine Huffman code, it is useful to construct a binary treeLeaves are characters to be encodedNodes carry occurrence probabilities of the characters belonging to the sub-tree

CS 414 - Spring 2014Source: images.search.yahoo.com(D. Huffman)


Huffman Encoding (Example)P(C) = 0.09P(E) = 0.11P(D) = 0.13P(A)=0.16P(B) = 0.51Step 1 : Sort all Symbols according to their probabilities (left to right) from Smallest to largest these are the leaves of the Huffman treeCS 414 - Spring 2014


Huffman Encoding (Example)P(C) = 0.09P(E) = 0.11P(D) = 0.13P(A)=0.16P(B) = 0.51P(CE) = 0.20P(DA) = 0.29P(CEDA) = 0.49P(CEDAB) = 1Step 2: Build a binary tree from left toRight Policy: always connect two smaller nodes together (e.g., P(CE) and P(DA) had both Probabilities that were smaller than P(B),Hence those two did connect firstCS 414 - Spring 2014


Huffman Encoding (Example)P(C) = 0.09P(E) = 0.11P(D) = 0.13P(A)=0.16P(B) = 0.51P(CE) = 0.20P(DA) = 0.29P(CEDA) = 0.49P(CEDAB) = 1010101Step 3: label left branches of the treeWith 0 and right branches of the treeWith 101CS 414 - Spring 2014


Huffman Encoding (Example)P(C) = 0.09P(E) = 0.11P(D) = 0.13P(A)=0.16P(B) = 0.51P(CE) = 0.20P(DA) = 0.29P(CEDA) = 0.49P(CEDAB) = 1010101Step 4: Create Huffman CodeSymbol A = 011Symbol B = 1Symbol C = 000Symbol D = 010Symbol E = 00101CS 414 - Spring 2014


Huffman DecodingAssume Huffman TableSymbol Code X 0 Y 10 Z 11Consider encoded bitstream: 000101011001110CS 414 - Spring 2014What is the decoded string?


Huffman ExampleConstruct the Huffman coding tree (in class)

Symbol (S)P(S)A0.25B0.30C0.12D0.15E0.18


Characteristics of Solution CS 414 - Spring 2014

Symbol (S)CodeA01B11C100D101E00


Example Encoding/DecodingEncode BEAD110001101

Decode 0101100CS 414 - Spring 2014

Symbol (S)CodeA01B11C100D101E00


Entropy (Theoretical Limit)

= -.25 * log2 .25 + -.30 * log2 .30 + -.12 * log2 .12 + -.15 * log2 .15 + -.18 * log2 .18

H = 2.24 bits

SymbolP(S)CodeA0.2501B0.3011C0.12100D0.15101E0.1800


Average Codeword Length

= .25(2) + .30(2) + .12(3) + .15(3) + .18(2)

L = 2.27 bits



Code Length Relative to Entropy

Huffman reaches entropy limit when all probabilities are negative powers of 2i.e., 1/2; 1/4; 1/8; 1/16; etc.

H Code Length H + 1CS 414 - Spring 2014


ExampleH = -.01*log2.01 + -.99*log2.99 = .08

L = .01(1) + .99(1) = 1


SymbolP(S)CodeA0.011B0.990


Group ExerciseCompute Entropy (H)

Build Huffman tree

Compute average code length

Code BCCADECS 414 - Spring 2014

Symbol (S)P(S)A0.1B0.2C0.4D0.2E0.1


LimitationsDiverges from lower limit when probability of a particular symbol becomes highalways uses an integral number of bits

Must send code book with the datalowers overall efficiency

Must determine frequency distributionmust remain stable over the data setCS 414 - Spring 2014


SummaryImportant Lossless (Entropy) Coders Huffman Coding and Arithmetic Coding(Also LZW)CS 414 - Spring 2014


Huffman Encoding (Exercise) Example: How does a Huffman code look like for symbols with statistical symbol occurrence probabilities:P(A) = 8/20, P(B) = 3/20, P(C ) = 7/20, P(D) = 2/20?CS 414 - Spring 2014


Solution to Group ExerciseCompute Entropy (H)H = 2.1 bits

Build Huffman tree

Compute code length L = 2.2 bits

Code BCCADE => 11000100111101




**Depends on occurrence frequency of single characters or sequences of data bytesCharacters are stored with their probabilitiesLength (number of bits) of the coded characters differsThe shortest code is assigned to the most frequently occurred character

******ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign**Depends on occurrence frequency of single characters or sequences of data bytesCharacters are stored with their probabilitiesLength (number of bits) of the coded characters differsThe shortest code is assigned to the most frequently occurred character

*ORCHID Research GroupDepartment of Computer Science, University of Illinois at Urbana-Champaign*

lect8-compress2

Documents