cm 05 image compression - comlab · image compression • • in digital image compression there...
TRANSCRIPT
1
M. Carli - 2005
Image Compression
Image Compression
But life is short and information endless...Abbreviation is a necessary evil and the abbreviator’s business is to make the best of a job which, although intrinsically bad, is still better than nothing.
Aldous Huxley
M. Carli - 2005
Image Compression
• Reduction of the amount of data ⇒ removal removal of redundant dataof redundant data
• Math.: Transforming a 2-D pixel array into a statistically uncorrelated data set
What’s Image Compression?
2
M. Carli - 2005
Image Compression
• Important in data storagedata storage and data transmissiondata transmission• Examples:
– Progressive transmission of images/videos (Internet)– Video coding (HDTV, teleconferencing)– Digital libraries and image databases– Remote sensing– Medical imaging
Why Compression?
M. Carli - 2005
Image Compression
Digital image
• A digital image is a two-dimensional sequence of samples
3
M. Carli - 2005
Image Compression
Discrete image intensities
• Unsigned B-bits: x[n1, n2]∈{0,1,…,2B-1}• Signed B-bits: x[n1, n2]∈{- 2B-1, - 2B-1+1,-
1,0,1,…,2B-1-1}• Usually B=8• Interpretation:
M. Carli - 2005
Image Compression
Color
• Color images: three values per sample location, (red, green and blue)
xr [n1, n2], xg[n1, n2] xb[n1, n2]
4
M. Carli - 2005
Image Compression
• A color image represented with 8 bits/channel/pixel requires
• With a compression standard like JPEG, it is possible to represent it with a much smaller number of bits!
Example: JPEG Standard
500 500×
500 500 8 3 6M bits× × × =
M. Carli - 2005
Image Compression
• Compression techniques
•• Lossless (or informationLossless (or information--preserving) compressionpreserving) compression: Images can be compressed and restored without any loss of information (e.g., medical imaging, satellite imaging)
•• LossyLossy compressioncompression: Perfect recovery not possible;larger data compression (e.g., TV signals, teleconferencing)
Lossless vs. Lossy Compression
{losslesslossless
lossylossy
5
M. Carli - 2005
Image Compression
• Data are the means by which information is conveyed
• Various amounts of data may be used to represent the same amount of information
• Data redundancy: if n1 and n2 denote the number of information-carrying units in two data sets that represent the same information, the relative data redundancyrelative data redundancy of the first data set with respect to the second is
Data and Information
11DR
RC
= − 1
2R
nCn
=where compression compression ratioratio
M. Carli - 2005
Image Compression
• Case 1:
• Case 2:
• Case 3:
11DR
RC
= −11DR
RC
= −Examining
2 1n n=
2 1n n
2 1n n
1and 0R DC R= =
0 andR DC R→ →−∞
and 1R DC R→∞ →
the first data set contains no redundant information
highly redundant data significant compression
(undesirable) data expansion
6
M. Carli - 2005
Image Compression
•
• In digital image compression there exist three basic data redundancies:
1.1. Coding redundancyCoding redundancy2.2. InterpixelInterpixel redundancyredundancy3.3. PsychovisualPsychovisual redundancyredundancy
Redundancy
(0, ) and ( ,1)R DC R∈ ∞ ∈ −∞
M. Carli - 2005
Image Compression
Suppose that:
• This means that the first dataset needs 10 information-carrying units (e.g., bits) whereas the second dataset just needs 1!
• The corresponding redundancy is which says that 90% of the data in the first set is redundant
Example
10 (or 10 :1)RC =
0.9DR =
7
M. Carli - 2005
Image Compression
• Let be a discrete random variable representing the gray levels (L) in an image
• Its probability is represented by
• Let be the total number of bits used to represent each value of
• The average number of bits required to represent each pixel is
Coding Redundancy
( ) , 0,1, , 1kr k
np r k Ln
= = −…
[0,1]kr ∈
kr( )kl r
1
0( ) ( )
L
avg k r kk
L l r p r−
== ∑
M. Carli - 2005
Image Compression
Coding
• The average lenght of the code words assigned tothe various gray-level values: the sum of the product of the number of bits used to representeach gray level and the probability that the graylevel occurs.
• Total number of bits to code an MxN image isMNLavg
• Usually l(rk) = m bits (constant). ⇒ Lavg = ∑k mpr (rk)=m
8
M. Carli - 2005
Image Compression
Coding
• It makes sense to assign fewer bits to those rk for which pr(rk) are large in order to reduce the sum.
• ⇒ achieves data compression and results in a variable length code.
• More probable gray levels will have fewer number of bits.
1
0( ) ( )
L
avg k r kk
L l r p r−
== ∑
M. Carli - 2005
Image Compression
Example: Variable-Length Coding
Code 1:
Code 2:
3avgL bits=
2.7avgL bits=
3 1.112.7RC = =
11 0.0991.11DR = − =
natural binary code
variable-lengthcode
Gray-level distribution
2(0.19)+2(0.25)+...
9
M. Carli - 2005
Image Compression
Example: Rationale Behind Variable-Length Coding
• Assign shortest codewords to the most probable gray levels and longest codewords to the least probable
probability codeword length
M. Carli - 2005
Image Compression
Interpixel Redundancy
( )( )(0)
A nnA
γ ∆∆ =
autocorrelation autocorrelation coefficients along one coefficients along one line of each imageline of each image
1
0
1( ) ( , ) ( , )N n
yA n f x y f x y n
N n
− −∆
=∆ = + ∆
−∆ ∑
x
y
“Same2 histograms
trimodal
10
M. Carli - 2005
Image Compression
• Second image shows high correlation between pixels 45 and 90 samples apart
• Adjacent pixels of both images are highly correlated•• InterpixelInterpixel (or spatial) redundancy(or spatial) redundancy: the value of any
given pixel can be reasonably predicted from the values of its neighbors; as a consequence, any pixel carries a small amount of information
• Interpixel redundancy can be reduced through mappings mappings (e.g., differences between adjacent pixels)(e.g., differences between adjacent pixels)
Interpixel Redundancy
M. Carli - 2005
Image Compression
Example: Run-Length Coding
1024 343 1 2.6312,166 11RC ⋅ ⋅
= =⋅
11 0.622.63DR = − =
11 bits are necessary to represent each run-length pair
number of runs
11
M. Carli - 2005
Image Compression
• The eye does not respond with equal sensitivity to all visual information
• Certain information has less relative importance than other information in normal visual processing (psychovisually redundant)
• It can be eliminated without significantly impairing the quality of image perception
Psychovisual Redundancy
M. Carli - 2005
Image Compression
Irrelevance in Color Imagery
• Human visual system has much lower acuity for color hue and saturation than for brightness
• Use color transform to facilitate exploiting that property
12
M. Carli - 2005
Image CompressionCompression by Quantization
8 bits 4 bits (2:1) Improved Gray Scale (IGS) Improved Gray Scale (IGS) Quantization Quantization (2:1)
M. Carli - 2005
Image Compression
• is the input image• is the estimate or approximation of
resulting from compression and decompression
• Error between the two images
•• RootRoot--mean square errormean square error:
Objective Fidelity Criteria
ˆ( , ) ( , ) ( , )e x y f x y f x y= −
( , )f x yˆ ( , )f x y( , )f x y
1/ 21 1 2
0 0
1 ˆ ( , ) ( , )M N
rmsx y
e f x y f x yMN
− −
= =
⎡ ⎤⎡ ⎤= −⎢ ⎥⎣ ⎦⎢ ⎥⎣ ⎦
∑ ∑
13
M. Carli - 2005
Image Compression
•• MeanMean--square SNR of the output imagesquare SNR of the output image
Signal-to-Noise Ratio (SNR)
1 12
0 01 1 2
0 0
( , )
ˆ ( , ) ( , )
M N
x yms M N
x y
f x y
SNRf x y f x y
− −
= =− −
= =
=⎡ ⎤−⎣ ⎦
∑ ∑
∑ ∑
M. Carli - 2005
Image Compression
Subjective Fidelity Criteria
14
M. Carli - 2005
Image Compression
Image Compression Models
• Source encoderSource encoder: removes input redundancies
• Channel encoder: Channel encoder: increases the noise immunity of the source encoder’s output
M. Carli - 2005
Image Compression
Typical structured compression system
15
M. Carli - 2005
Image Compression
Source Encoder and Source Decoder Model
M. Carli - 2005
Image Compression
•• MapperMapper: designed to reduce interpixelinterpixel redundancyredundancy; e.g.:– Run-length encoding– Transform encoding (e.g., DCT in JPEG standard)
•• QuantizerQuantizer: reduces psychovisualpsychovisual redundanciesredundancies (it cannot be used in lossless compression)
•• Symbol EncoderSymbol Encoder: creates a fixed/variable length code;it reduces coding redundanciescoding redundancies
Source Encoder
16
M. Carli - 2005
Image Compression
Quantizer
• Goal: reduce the number of possible amplitude values for coding
• Simple scalar quantizerwith four output indices
M. Carli - 2005
Image Compression
• The channel encoder adds ““controlled controlled redundancyredundancy”” to the data to protect it from channel noise
•• Hamming encoding:Hamming encoding: based on appending enough bits to the data to ensure that some minimum number of bits must change between valid code words thereby providing resiliency against transmission errors
The Channel Encoder and Decoder
17
M. Carli - 2005
Image Compression
• Information can be modeled as a probabilistic process that can be measured in a manner which agrees with intuition
• A random event E that occurs with probability P(E) is said to contain
units of information• I(E) is often called selfself--informationinformation
Measuring Information
1( ) log log ( )( )
I E P EP E
= = −
M. Carli - 2005
Image Compression
• If , (no information)
• If the base of the logarithm is 2, the unit of information is called a “bit”
• If ,Example: flipping a coin and communicating the result requires one bit of information
• 1 bit is the amount of information conveyed
Self-Information
( ) 1P E = ( ) 0I E =
( ) 1/ 2P E = 2( ) log (1/ 2) 1 bitI E = − =
18
M. Carli - 2005
Image Compression
Entropy
{ }1 2, , , La a a…information information sourcesource
• Let be the probability of each symbol
• Then the entropy (or uncertainty) of the sourceentropy (or uncertainty) of the sourceis given by
( ), 1,2, ,lp a l L= …
1( ) log( ( )) bits/symbol
L
l ll
H p a p a=
= −∑
Source alphabet
M. Carli - 2005
Image Compression
• Consider the 8-bit image:
Computing the Entropy of an Image
21 21 21 95 169 243 243 24321 21 21 95 169 243 243 24321 21 21 95 169 243 243 24321 21 21 95 169 243 243 243
21 1Gra
2 3/y Level
895 4 1/
Counts Probab
8169 4 1/8243 1
ility
2 3/8
1.81 bits/pixelH =(first-order estimate)
8-bit gray level source- statistically independent pixels emission
19
M. Carli - 2005
Image Compression
• Keep first column and replace following with the arithmetic difference between adjacent columns
Using Mappings to Reduce Entropy
21 0 0 74 74 74 0 021 0 0 74 74 74 0 021 0 0 74 74 74 0 021 0 0 74 74 74 0 0
Gray LevelCounts Probabilit
0 12 1/ 2
yor Dif
21 4 1/87
f
4
erence
12 3/8
1.41 bits/pixelH =(first-order estimate)
M. Carli - 2005
Image Compression
• In some applications, error-free (lossless)compression is the only acceptable means of data reduction (e.g., medical and business documents, satellite data)
• Applicable to both binary and gray-scale images
• Consist of two operations:1. Alternative image representation to reduce
interpixel redundancies2. Coding the representation to eliminate coding
redundancies
Error-Free (or Lossless) Compression
20
M. Carli - 2005
Image Compression
• Devised by Huffman in 1952 for removing coding redundancy
• Property: If the symbols of an information source are coded individually, the Huffman coding yields the smallest possible number of code symbols per source symbol
Huffman (Variable-Length) Coding
M. Carli - 2005
Image Compression
• Method: create a series of source reductions by ordering the probabilities of the symbols under consideration and combining the lowest probability symbols into a single symbol that replaces them in the next source reduction
• Rationale: To assign the shortest possible codewords to the most probable symbols
Huffman Coding
21
M. Carli - 2005
Image Compression
Example: Huffman Source Reductions
symbols are ordered according to decreasing probability
M. Carli - 2005
Image Compression
Example: Huffman Code Assignment Procedure
0.4 1 0.3 2 0.1 3 0.1 4 0.06 5 0.04 5 2.2 bits/symbolavgL = ⋅ + ⋅ + ⋅ + ⋅ + ⋅ + ⋅ =
2log ( ) 2.14 bits/symbolH p p= − =∑
22
M. Carli - 2005
Image Compression
• Coding/decoding is accomplished with a lookup table
• It is a block codeblock code: each source symbol is mapped into a fixed sequence of code symbols
• It is instantaneousinstantaneous: each code word in a string of code symbols can be decoded without looking at succeeding symbols
• It is uniquely decodableuniquely decodable: any string of code symbols can be decoded only in one way; exampleexample:
Huffman Coding
2 23 1 6
010100111 1 00a aa a a
M. Carli - 2005
Image Compression
• Unlike the Huffman coding, arithmetic coding generates nonblock codes
• A one-to-one correspondence between source symbols and code words does not exist
• An entire sequence of source symbols (or message) is assigned a single arithmetic code word
Arithmetic Coding
23
M. Carli - 2005
Image Compression
• Property: The code word itself defines an interval of real numbers between 0 and 1.
• As the number of symbols increases, the interval becomes smaller and the number of bits necessary to represent it becomes larger
• Each symbol of the message reduces the size of the interval in accordance with its probability of occurrence
Arithmetic Coding
M. Carli - 2005
Image Compression
Example: Arithmetic Coding
Suppose that a 4-symbol source generates the sequence (or message) 1 2 3 3 4a a a a a
any number in this subinterval, e.g. 0.068, can be used to represent the message
24
M. Carli - 2005
Image Compression
• Uses a dictionary• Dictionary is adapted to the data• It assigns fixed-length codewords to
variable-length sequences of source symbols• Decoder builds the matching dictionary
based on the codewords received• Used in GIF, TIFF, and PDF formats
Lempel-Ziv-Welch (LZW) Coding
M. Carli - 2005
Image Compression
• Consider the 4x4 8-bit image
Example: LZW Coding
39 39 126 12639 39 126 12639 39 126 12639 39 126 126
25
M. Carli - 2005
Image Compression
Example: LZW Coding
M. Carli - 2005
Image Compression
Lossless Predictive Coding
EncoderEncoder
DecoderDecoder
ˆn n ne f f= −
ˆn n nf e f= +
variable-length code
prediction error
26
M. Carli - 2005
Image Compression
• Generates an estimate of the value of a given pixel based on the values – of some past input pixels (temporal predictiontemporal prediction)
or– of some neighboring pixels (spatial predictionspatial prediction)
•• ExampleExample:
•• ExampleExample:
The Predictor
1
ˆm
n i n ii
f round fα −=
⎡ ⎤= ⎢ ⎥
⎣ ⎦∑ m=order of predictor
1
ˆ ( , ) ( , )m
ii
f x y round f x y iα=
⎡ ⎤= −⎢ ⎥
⎣ ⎦∑
x
y
ˆ ( , )f x y
M. Carli - 2005
Image Compression
Example: Predictive Coding
[ ]ˆ ( , ) ( , 1)f x y round f x yα= −
1α =
( , )f x y ˆ( , ) ( , ) ( , )e x y f x y f x y= −
histograms of original histograms of original and error imageand error image LaplacianLaplacian PDFPDF
previous pixel predictor
27
M. Carli - 2005
Image Compression
Lossy Predictive Coding
EncoderEncoder
DecoderDecoder
Feedback loop prevents error buildup at the decoder’s output
ˆn n nf e f= +
ˆn n nf e f= +
M. Carli - 2005
Image Compression
Example: Delta Modulation
1ˆ (0 1)n nf fα α−= < <
for 0otherwise
nn
ee
ξξ+ >⎧
= ⎨−⎩
28
M. Carli - 2005
Image Compression
• Minimize the encoder’s mean-square prediction error
subject to the constrain:
and to
Optimal Predictors
{ } 22 ˆn n nE e E f f⎧ ⎫⎡ ⎤= −⎨ ⎬⎣ ⎦⎩ ⎭
ˆ ˆn n n n n nf e f e f f= + ≈ + =
1
ˆm
n i n ii
f fα −=
=∑
(quantization error is assumed to be negligible)
(prediction is a linear combination of m previous pixels)
M. Carli - 2005
Image Compression
Optimal predictor
• The optimization criterion is chosen toMINIMIZE the MSPrediction Error
• The quantization error is assumed to benegligible
• The prediction is constrained to a linearcombination of m previous pixels.
29
M. Carli - 2005
Image Compression
• The solution of
is where
Differential Pulse Code Modulation (DPCM)
2
1min
i
m
n i n ii
E f fα
α −=
⎧ ⎫⎡ ⎤⎪ ⎪−⎨ ⎬⎢ ⎥⎣ ⎦⎪ ⎪⎩ ⎭
∑
1−=α R r{ } { }
{ } { }
1 1 1
1
n n n n m
n m n n m n m
E f f E f f
E f f E f f
− − − −
− − − −
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
R
{ }
{ }
1n n
n n m
E f f
E f f
−
−
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
r1
m
α
α
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
α
autocorrelation autocorrelation matrixmatrix
M. Carli - 2005
Image Compression
Example: Comparison of Four Linear Predictors
ˆ ( , ) 0.97 ( , 1)f x y f x y= −
ˆ ( , ) 0.5 ( , 1) 0.5 ( 1, )f x y f x y f x y= − + −
ˆ ( , ) 0.75 ( , 1) 0.75 ( 1, ) 0.5 ( 1, 1)f x y f x y f x y f x y= − + − − − −
0.97 ( , 1) fˆ ( , )0.97 ( 1, ) otherwise
f x y i h vf x y
f x y− ∆ ≤ ∆⎧
= ⎨ −⎩
( 1, ) ( 1, 1)h f x y f x y∆ = − − − −
( , 1) ( 1, 1)v f x y f x y∆ = − − − −
where:
30
M. Carli - 2005
Image Compression
Example: Comparison of Four Linear Predictors
original image
M. Carli - 2005
Image Compression
Transform Coding
EncoderEncoder
DecoderDecoder
31
M. Carli - 2005
Image Compression
• Consider an image of size• The forward discrete transformforward discrete transform is defined as
• The inverse discrete transforminverse discrete transform is defined as
Transform Selection
( , )f x y N N×
1 1
0 0( , ) ( , ) ( , , , )
N N
x yT u v f x y g x y u v
− −
= == ∑ ∑, 0,1, , 1u v N= −…
1 1
0 0( , ) ( , ) ( , , , )
N N
u vf x y T u v h x y u v
− −
= == ∑ ∑, 0,1, , 1x y N= −…
M. Carli - 2005
Image Compression
• Forward transform kernel
• Inverse transform kernel
Example: Discrete Fourier Transform
2 ( )( , , , )
j ux vyNg x y u v eπ
− +=
2 ( )
21( , , , )
j ux vyNh x y u v e
N
π+
=
32
M. Carli - 2005
Image Compression
Discrete Cosine Transform (DCT)
( , , , ) ( , , , )(2 1) (2 1)( ) ( )cos cos
2 2
g x y u v h x y u vx u y vu v
N Nπ πα α
=
+ +⎡ ⎤ ⎡ ⎤= ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
1 for 0( )
2 for 1,2, , 1
uNu
u NN
α
⎧=⎪⎪= ⎨
⎪ = −⎪⎩…
where
and similarly for ( )vα
M. Carli - 2005
Image Compression
DCT Basis Functions For N=4
33
M. Carli - 2005
Image Compression
Example: DFT vs. DCT
DCTDCT
DFTDFT
(8x8 subimages and truncation of 50% of transform coefficients)
1.28rms =
0.68rms =
M. Carli - 2005
Image Compression
• Blocking artifacts are less pronounced in the DCT than in the DFT
• The DCT is a good approximation of the Karhunen-Loeve Transform (KLT) which is optimal in terms of energy compaction
• However, unlike the KLT, the DCT has image-independent basis functions
• The DCT is used in JPEG compression standards
Why DCT?
34
M. Carli - 2005
Image Compression
Implicit Periodicity: DFT vs. DCT
M. Carli - 2005
Image Compression
Subimage Size Selection
35
M. Carli - 2005
Image Compression
Example: Different Subimage Sizes
2 2×
4 4× 8 8×
Approximation of original image (left) and relative error (right) with 8x8 blocks and keeping 25% of DCT coefficients
original
M. Carli - 2005
Image Compression
• In most transform coding systems, the coefficients to retain are selected– on the basis of maximum variance (zonal zonal
codingcoding)– or on the basis of maximum magnitude
(threshold codingthreshold coding)
•• Bit allocationBit allocation is the overall process of truncating, quantizing, and coding the coefficients of a transformed subimage
Bit Allocation
36
M. Carli - 2005
Image Compression
1. For each subimage, arrange the transform coefficients in decreasing order of magnitude
2. Keep only the top X% of the coefficients and discard the rest
3. Code the retained coefficients using a variable length code
Threshold Coding
M. Carli - 2005
Image Compression
1. Compute the variance of each of the transform coefficients (this is done by using the subimages)
2. Keep X% of the coefficients which have maximum variance
3. Code the retained coefficients using a variable length code proportional to the variance
Zonal Coding
37
M. Carli - 2005
Image Compression
Typical Masks
threshold maskthreshold mask
zonal mask zonal mask
zonal bit zonal bit allocation allocation
thresholded thresholded coefficients coefficients orderingordering
threshold coding is adaptive: the locations of the coefficients to keep depend on subimages
1:= keep
0:=discard
M. Carli - 2005
Image CompressionThreshold Coding vs. Zonal Coding
Threshold Threshold CodingCoding
Zonal Zonal CodingCoding
Approximations obtained by using 12.5% of the 8x8 DCT coefficients
38
M. Carli - 2005
Image Compression
• Is a compression standard for still images• It defines three different coding systems:
1. A lossylossy baseline coding systembaseline coding system based on DCT (adequate for most compression applications)
2. An extended coding system extended coding system for greater compression, higher precision, or progressive reconstruction applications
3. A lossless independent coding systemlossless independent coding system for reversible compression
JPEG (Joint Photographic Experts Group)
M. Carli - 2005
Image Compression
39
M. Carli - 2005
Image Compression
M. Carli - 2005
Image Compression
1.1. Computing the DCTComputing the DCT: the image is divided into 8X8 blocks; each pixel is level shifted by subtracting the quantity 2n-1 where 2n is the maximum number of gray levels in the image. Then, the DCT of each block is computed. The precision of input and output data is restricted to 8 bits whereas the DCT values are represented with 11 bits
2.2. QuantizationQuantization: The DCT coefficients are thresholded and coded using a quantization matrix, and the recorded using zig-zag scanning to form a 1-D sequence
3.3. CodingCoding: The non-zero AC coefficients are Huffman coded. The DC coefficients of each block are DPCM coded relative to the DC coefficient of the previous block
JPEG – Baseline Coding System
40
M. Carli - 2005
Image Compression
Example: Implementing the JPEG Baseline Coding System with Matlab
183 160 94 153 194 163 132 165 183 153 116 176 187 166 130 169 179 168 171 182 179 170 131 167 177 177 179 177 179 165 131 167 17
f =8 178 179 176 182 164 130 171
179 180 180 179 183 164 130 171 179 179 180 182 183 170 129 173 180 179 181 179 181 170 130 169
⎡ ⎤⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣
⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦
8x8 subimage
8x8 block taken from an image with N=256 gray levels
M. Carli - 2005
Image Compression
Example: Level Shifting
55 32 -34 25 66 35 4 37 55 25 -12 48 59 38 2 41 51 40 43 54 51 42 3 39 49 49 51 49 51 37 3 3
128fs f= − =9
50 50 51 48 54 36 2 43 51 52 52 51 55 36 2 43 51 51 52 54 55 42 1 45 52 51 53 51 53 42 2 41
⎡⎢⎢⎢⎢⎢
⎣
⎤⎥⎥⎥⎥⎥
⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥
⎦
41
M. Carli - 2005
Image Compression
Example: Computing the DCT
312 56 -27 17 79 -60 26 -26 -38 -28 13 45 31 -1 -24 -10 -20 -18 10 33 21 -6 -16 -9 -11 -7
( ( )round dct2 )dcts fs= = 9 15 10 -11 -13 1
-6 1 6 5 -4 -7 -5 5 3 3 0 -2 -7 -4 1 2 3 5 0 -4 -8 -1 2 4 3 1 -1 -2 -3 -1 4 1
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
M. Carli - 2005
Image Compression
Example: The Quantization Matrix
16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87
qmat = 80 62
18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 72 92 95 98 112 100 103 99
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
42
M. Carli - 2005
Image Compression
M. Carli - 2005
Image Compression
Example: Thresholding
20 5 -3 1 3 -2 1 0 -3 -2 1 2 1 0 0 0 -1 -1 1 1 1 0 0 0 -1 0 0 1 0 0
( ./ )roundt dcts qmat= = 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
43
M. Carli - 2005
Image Compression
Example: Zig-Zag Scanning of the Coefficients
20 5 -3 1 3 -2 1 0 -3 -2 1 2 1 0 0 0 -1 -1 1 1 1 0 0 0 -1 0 0 1 0 0
( ./ )roundt dcts qmat= = 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
[ ]20,5, 3, 1, 2, 3,1,1, 1, 1, ,1,20,0 0,0,0,0,,3 0,, 2,1,1, ,1,1,0 B0 ,1,EO− − − − − − −
end-of-block
DC coefficient
M. Carli - 2005
Image Compression
Example: Coding the Coefficients• The DC coefficient is DPCM coded
(difference between the DC coefficient of the previous block and current block)
• The AC coefficients are mapped to run-length pairs:
• These are then Huffman coded (codes are specified in the JPEG scheme)
(0,5),(0, 3),(0, 1),(0, 2),(0, 3),(0,1),(0,1),(0, 1),(0, 1), ,(0,2),(0,3),(0, 2),(0,1),(0,1),
(2,1)(6,1),(0,1 (), ,E, ) B1 O1
− − − −− − −
44
M. Carli - 2005
Image Compression
Example: Decoding the Coefficients
320 55 -30 16 72 -80 51 0 -36 -24 14 38 26 0 0 0 -14 -13 16 24 40 0 0 0 -14 0 0 29 0
.*ds hat t qmat− = = 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
M. Carli - 2005
Image Compression
Example: Computing the IDCT
67 12 -9 20 69 43 -8 42 58 25 15 30 65 40 -4 47 46 41 44 40 59 38 0 49
roun 4
d1 52 59 43
idct2
( ( ))fs hat ds hat− −= =57 42 3 42
44 54 58 40 58 47 3 33 49 52 53 40 61 47 1 33 53 50 53 46 63 41 0 45 55 50 56 53 64 34 -1 57
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
45
M. Carli - 2005
Image Compression
Example: Shifting Back the Coefficients
195 140 119 148 197 171 120 170 186 153 143 158 193 168 124 175 174 169 172 168 187 166 128 177 169 180 187 171 185 170
128f hat fs hat− −= + =131 170
172 182 186 168 186 175 131 161 177 180 181 168 189 175 129 161 181 178 181 174 191 169 128 173 183 178 184 181 192 162 127 185
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
183 160 94 153 194 163 132 165 183 153 116 176 187 166 130 169 179 168 171 182 179 170 131 167 177 177 179 177 179 165 131 167 17
f =8 178 179 176 182 164 130 171
179 180 180 179 183 164 130 171 179 179 180 182 183 170 129 173 180 179 181 179 181 170 130 169
⎡ ⎤⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣
⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦
to be compared to …
M. Carli - 2005
Image Compression
Example: Original Image vs. Decompressed Image
Original Original Decompressed Decompressed
46
M. Carli - 2005
Image Compression
Example: JPEG Compression
34 :13.42
RCrms
==
67 :16.33
RCrms
==
M. Carli - 2005
Image Compression
47
M. Carli - 2005
Image Compression