data compression 2011 - jan sloot
TRANSCRIPT
Animation 1
Data extraction compression and recovery tutorial
February 20111
(right click/ open hyperlink)
Data compression
February 20112
Original Image --- 24 bits per pixel JPEG --- 0.5 bits per pixel
http://www.data-compression.com/image.shtml
‘De broncode’ (claims)
• “alleen essentiële onderdelen van beeld en geluid opslaan”
• “bijv. tekst niet letter voor letter opslaan, maar door middel van één getal dat de volgorde van alle letters
February 20115
in dat boek in zich draagt”• “de opslagcapaciteit kan met deze code 1 miljoen keer
worden vergroot”• “op een chipkaartje (16 kbyte) kunnen 16 films staan”• “alle speelfilms ooit gemaakt passen op één CD”
meer info op EN- en NL-wikipedia (Jan Sloot)
introductory bookDavid Salomon "A guide to data compression
methods“Springer-Verlag New York, 2002 [EL e91 s13]
See also:- Waser, Introduction to part VI Data
February 20116
- Waser, Introduction to part VI Datatransmission and interfaces
- Report Broadband applications on limited ...- http://www.data-compression.com/
Links to scanned book sections etc., and also this powerpoint,via Blackboard (Course Materials & Course Information)
general remarks (1)• compressing data = removing redundancy
data without redundancy (most files!) cannot be compressed
• ASCII-coded text is redundant (always 8 bits per character)
February 20117
per character)� variable-size codes
(short codes for common data)• digital images
� assign short codes to dominant colors� consider correlation of neighboring pixels
general remarks (2)
• redundancies depend on data type (text, image, sound)each data type has its own compression algorithm
• adaptive (example: adaptive Huffman)
February 20118
• adaptive (example: adaptive Huffman)non-adaptive (example: facsimile compression)
• lossy compression (loss of data: movie/sound)lossless compression (for text, computer data)
• (non)symmetric compression (“real-time” divX)
subject overview
• statistical methods Huffman coding• dictionary compression (LZW)• image compression (JPEG)
February 20119
• image compression (JPEG)• wavelet transform (SPIHT)• video compression (MPEG)• audio compression (MP3)
statistical methods (digrams)
digrams � ta, he, ca (common)xa, hz, qu (rare)
trigrams � and
February 201110
trigrams � and
assign variable-size codesto the many digrams (and trigrams)
statistical methods (entropy 1)
alphabet of symbols a1 - an, probabilities P1 - Pn
self-information of ai : -log2Pi → unit: “bits”-log20 = ∞ -log20.5 = 1 -log21 = 0
February 201111
entropy is “the average self-information”
entropy of entire alphabet: �=
−n
iii PP
12log
statistical methods (entropy 2)alphabet with 5 symbols
February 201112
23 = 8, so without compression we need10 (symbols) * 3 bits = 30 bits
statistical methods (variable-size codes)1. assign short codes to frequent symbols2. obey the prefix property
February 201113
ACBAcode 1 1 01 001 1 � 1 010 01 1
A B D A � A C B A
code 2 1 000 01 1
A C B A
Huffman for Dutch text
http://nl.wikipedia.org/wiki/Huffmancodering
Letter frequencies in Dutch texts
February 201117
Letter percent Letter percent Letter percent Letter percent Letter percent Letter percent Letter percent
E 18,91% N 10,03% A 7,49% T 6,79% I 6,50% R 6,41% O 6,06%
D 5,93% S 3,73% L 3,57% G 3,40% V 2,85% H 2,38% K 2,25%
M 2,21% U 1,99% B 1,58% P 1,57% W 1,52% J 1,46% Z 1,39%
C 1,24% F 0,81% X 0,04% Y 0,03% Q 0,01%
�=
−n
iii PP
12log (entropy) is 4.1
Huffman tree and letter codes
Optimal
February 201118
Optimalcodesfor theDutch
language( capitals
&small letters)
statistical methods (fax)
fax machine
• scans document line by line
RLE = Run-Length Encoding
February 201119
RLE = Run-Length Encodingrun length = “combination of black/white pixels”
• most common run lengths assigned to the shortest codes (2, or 3, or 4 black pixels)
• modified Huffman algorithm, non-adaptive• compression 4-10
subject overview
• statistical methods Huffman coding• dictionary compression (LZW)• image compression (JPEG)
February 201120
• image compression (JPEG)• wavelet transform (SPIHT)• video compression (MPEG)• audio compression (MP3)
dictionary compression (1)
- example -
English text, with average word size 5 characters
output file = indexes and/or raw words
February 201121
output file = indexes and/or raw words
• index = flag bit 1, 19-bit index (219 = 524288)
• raw word = flag bit 0, 7-bit field (=size), 5 bytes
under conditions � 48 bits compressed to 20 bits
dictionary compression (2)this exampleprobability of a word found in dictionary: P� size of input file (N words) = 40N bits
size of output file = N[20P + 48(1-P)] bits
February 201122
size of output file = N[20P + 48(1-P)] bitscompression if P > 0.29 (matching rate > 29%)--------------more sophisticated:adaptive dictionary-based method
(building a dictionary from scratch)
ZIP code
applies 'deflate', a combination of• L777 dictionary method• Huffman
February 201123
• Huffman
subject overview
• statistical methods Huffman coding• dictionary compression (LZW)• image compression (JPEG)
February 201124
• image compression (JPEG)• wavelet transform (SPIHT)• video compression (MPEG)• audio compression (MP3)
image compression (1)
- book contains about 106 characters 1 Mbyteimage (512 x 512 x (8+8+8))/8 = 0.7 Mbyte
- lossy compression accepted(= removing “irrelevance”)
February 201125
(= removing “irrelevance”)
- redundancy & irrelevance properties dependon type of image(bi-level, grayscale, color, continuous tone,discrete tone, cartoon-like)
lossy image compression
February 201126
Original Image --- 24 bits per pixel JPEG --- 0.5 bits per pixel
image compression (2)
• large number of elementary ‘symbols’224 = 16.78 106 (3 colors, each 28 intensities)(text: 256 ASCII characters, � 256 bytes)
February 201127
(text: 256 ASCII characters, � 256 bytes)• correlation between near-neighbour pixels
(= spatial redundancy)• methods exist for automatically deleting
irrelevant image information
image compression (3)
“How to represent pixel properties?”RGB YCbCr
red green blue luminance Y B-Y R-Y
February 201128
• eye is more sensitive to brightness changesthan to color changes
• compression should focus on removing irrelevant color changes� represent pixels in YCbCr color space
image compression (image transforms)
1 transform (correlated) pixels to representation which is decorrelated e.g.discrete cosine transform- compression if new values are smaller than
original ones
February 201129
original ones- transformation to representation withdecorrelated pixels results in fully compressedimage
2 lossy compression by quantizing the transformed values
image compression (JPEG)
main steps• transform from RGB to YCbCr• color components are organized in groups of 8x8 pixels• discrete cosine transform (DCT) for pixel groups
February 201130
• discrete cosine transform (DCT) for pixel groups• quantization of frequency components (= loss of information)• encoding of freq. components using RLE and Huffman coding• add headers, values of JPEG parameters, output the results
image compression (DCT 1)
DCT (discrete cosine transform)
for pixels pxy:, the 2-dimensional DCT is
February 201131
for pixels pxy:, the 2-dimensional DCT is
( ) ( )��
= =��
���
� +��
���
� +=7
0
7
041
16
12cos
16
12cos
x yxyjiij
jyixpCCG
ππ
0for 1and ,0for 2
1 >=== f CfC ff
image compression (DCT 2)
February 201132
calculating aone-dimensionaldiscrete cosine transform (DTC)
imagecompression
(DCT 3)
the 64 basis
February 201133
the 64 basis imagesof thetwo-dimensional discrete cosine transform (DCT)
subject overview
• statistical methods Huffman coding• dictionary compression (LZW)• image compression (JPEG)
February 201138
• image compression (JPEG)• wavelet transform (SPIHT)• video compression (MPEG)• audio compression (MP3)
toepassing wavelets in jpeg2000-standaard
February 201139
Uit: Natuurwetenschap & Techniek, juli/aug 2006
subject overview
• statistical methods Huffman coding• dictionary compression (LZW)• image compression (JPEG)
February 201142
• image compression (JPEG)• wavelet transform (SPIHT)• video compression (MPEG)• audio compression (MP3)
capacity of "tv-cable" at home
"2008"analog:35 tv channels35 radio channels
"1997"analog35 tv channels35 radio channels
February 201143
35 radio channels
digital100 tv channels
incl. HD channels60 audio channelstelephoneinternet
35 radio channels"kabel is vol"
tv cable (2)
see also Waser,pages 709-713 about encoding:
1 source encoding
February 201144
1 source encoding (data compression) +
2 channel encoding (error correction) +
3 line encoding and modulation(adapting to physical channel)
video compression (1)
book contains ~106 characters 1 Mbyteimage (512 x 512 x (8+8+8))/8 = 0.7 Mbytevideo / studio camera ‘raw’ 35 Mbyte/s after compressionvideo / studio quality 720 x 576 1.25 Mbyte/s
February 201146
video / studio quality 720 x 576 1.25 Mbyte/svideo / HDTV 1920 x 1080 2.5 Mbyte/s
spacial redundancy (pixels in one frame)temporal redundancy (neighbor frames)
video compression (2)
1 encode 1th frame � intra frame I
2a consider differences between I and next
February 201148
2a consider differences between I and nextframe(s),using past frame (I or P) � interframe P,or encode independently � I
2b encoding, based on a past frame (I or P)and a future frame (I or P) � B
subject overview
• statistical methods Huffman coding• dictionary compression (LZW)• image compression (JPEG)
February 201151
• image compression (JPEG)• wavelet transform (SPIHT)• video compression (MPEG)• audio compression (MP3)
Sampling andquantization ofa signal (red)for 4-bit
audio compression
February 201152
for 4-bitpulse-codemodulation(PCM )
CD: 44100 Hz sampling rate (each channel)16-bit PCM (65536 levels) � 1411.2 kbit/s
audio compression (3)• lossless compression of 16-bit PCM audio is
possible to factor 2.6 (using LA, WMA, ..)
• MP3 represents PCM audio, discarding portions that are less important for human ear and brain
February 201153
with respect to CD bit rate:• mp3 compression factor 4.3 to 43• for best encoder
“good” 128 kbit/s 11 “excellent” 192 kbit/s 7.4
audio compression (mp3 encoding)
• filtering, 32 bands
• Modified Discrete Cosine Transform (MDCT)
February 201154
• Modified Discrete Cosine Transform (MDCT)36 or 12 tap
• quantizing, using a psychoacoustic model �(frequency & temporal masking)
final conclusions ondata compression
• achievable compression factors depend on redundancy and irrelevance
• compression method must match the type of
February 201155
• compression method must match the type of raw data (text, image, video, ...)
• “assuming that current compression factors will essentially improve in future does not appear realistic”
some compression factors
• statistical methods Huffman coding fax 4-10• dictionary compression ‘english dictionary’ < 2.4• image compression JPEG 10-20, ‘monkey’ 48
February 201156
• image compression JPEG 10-20, ‘monkey’ 48• wavelet transform (SPIHT)• video compression DV 5, MPEG-2 dvd 3hr 40
iPod video player (if from 18 MB/s to 0.11 MB/s) 160 audio compression WMA (lossless) 2.5
MP3 (lossy) 4-40
and finally...
compression costs processing power !
February 201157
compression costs processing power !
consider this in your 'case studies' and'design projects'