quantization - itunesdeimos.apple.com/webobjects/core.woa/downloadtrackpreview/uno.… · quantizer...

09: Scalar Quantization CSCI 6990 Data Compression

Vassil Roussev 1

CSCI 6990.002: Data Compression

09: Scalar Quantization

Vassil Roussev<vassil@ cs.uno.edu>

UNIVERSITY of NEW ORLEANSDEPARTMENT OF COMPUTER SCIENCE

2

Quantization

Definition:The process of representing a large—possibly infinite—set of values with a much smaller set.Example:

Source: Real numbers in the [-10.0, 10.0]

QuantizationQ(x) = ⎣x+0.5⎦[-10.0, -10.0] {-10, -9, …, -1, 0, 1, 2, …, 9, 10}

Scalar vs. vector quantizationScalar: applied to scalarsVector: applied to vectors


Vassil Roussev 2

3

The Quantization Process

Two aspectsEncoder mapping

Map a range of values to a codewordIf source is analog A/D converterKnowledge of the source can help pick more appropriate ranges

Decoder mappingMap the codeword to a value in the rangeIf output is analog D/A converterKnowledge of the source distribution can help pick better approximations

Quantizer = encoder + decoder

4

Quantization Example


Vassil Roussev 3

5

Quantizer Input-Output Map

6

Quantization Problem Formulation

Input:X – random variablefX(x) – probability density function (pdf)

Output:{bi}i=0..M decision boundaries

{yi}i=1..M reconstruction levels

Discrete processes are often approximated by continuous distributions

E.g.: Laplacian model of pixel differenceIf source is unbounded, then first/last decision boundaries = ±∞


Vassil Roussev 4

7

Quantization Error

Mean squared quantization error

( )

( ) dxfyx

dxfxQx

X

M

i

b

b i

Xq

i

i∑∫

∫

=

∞

∞−

−

−=

−=

1

2

22

1

)(σ

iii bxbyxQ ≤<= −1iff)(

Quantization error is a.k.a.Quantization noiseQuantizer distortion

8

Quantization Problem Formulation (2)

Bit rates w/ fixed-length codewordsR = ⎡log2M⎤E.g.: M = 8 R = 3

Quantizer design problemGiven:

input pdf fX(x) & MFind:

decision boundaries {bi} and

Reconstruction levels {yi}Such that:

MSQE is minimized


Vassil Roussev 5

9


Bit rates w/ variable-length codewordsR depends on boundary selectionExample

∑ ==

M

i ii yPlR1

)(

∫−

=i

i

b

b Xi dxxfyP1

)()(

∑ ∫=−

=M

i

b

b Xii

i

dxxflR1

1

)(

10


Rate-optimization formulation

Given:

Distortion constraint σq2 ≤ D*

Find:{bi},{yi}, binary codes

Such that:R is minimized

Distortion-optimization formulation

Given:Rate constraint R ≤ R*

Find:{bi},{yi}, binary codes

Such that:

σq2 is minimized


Vassil Roussev 6

11

Uniform Quantizer

All intervals of the same sizeI.e., boundaries are evenly spaced (∆)Outer intervals may be an exception

ReconstructionUsually the midpoint is selected

Midrise quantizerZero is not an output level

Midtread quantizer Zero is an output level

12

Midrise vs. Midtread Quantizer

Midrise Midtread


Vassil Roussev 7

13

Uniform Quantization of Uniform Source

Input:Uniform [‐Xmax, Xmax]

OutputM‐level uniform quantizer

MXmax2

=∆

1221

2122

22

1)1(

max

22 ∆

=⎟⎠⎞

⎜⎝⎛ ∆

−−= ∑∫

=

∆

∆−

M

i

i

iq dxX

ixσ

14

Alternative MSQE Derivation

Consider quantization error instead:q = x – Q(x)q ∈ [‐∆/2, ∆/2]

∫∆

∆−

∆=

∆=

2

2

222

121 dqqqσ

( ) ( )

dB02.6)2(log20)(log10

212

122log1012

122log10log10)dB(SNR

102

10

2max

2max

102

2max

102

2

10

nM

MX

XX

n

q

s

===

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎟⎠⎞

⎜⎝⎛

=⎟⎟⎠

⎞⎜⎜⎝

⎛∆

=⎟⎟⎠

⎞⎜⎜⎝

⎛=

σσ


Vassil Roussev 8

15

Examples (8 1, 2, 3 bits/pixel)

Darkening, contouring & dithering

16

Uniform Quantization of Nonuniform Sources

Example nonuniform source:x ∈ [‐100, 100], P(x ∈ [‐1, 1]) = 0.95

ProblemDesign an 8‐level quantizer

The naïve approach leads to95% of samples values represented by two numbers:

-12.5 and 12.5

Max quantization error (QE)= 12.5Min QE = 11.5 (!)

Consider an alternativeStep = 0.3Max QE = 98.5, however 95% of the time QE < 0.15


Vassil Roussev 9

17

Optimizing MSQE

Numerically solvable for specific PDF

18

Example Optimum Step Sizes


Vassil Roussev 10

19

QE for 3-bit Midrise Quantizer

20

Overload/Granular Regions

"4"4,stdev

valuegranularmax11 loadingff σ→==


Vassil Roussev 11

21

Variance Mismatch Effects

22

Variance Mismatch Effects (2)


Vassil Roussev 12

23

Distribution Mismatch Effects

8-level quantizers, SNR

24

Adaptive Quantization

IdeaInstead of a static scheme, adapt to the actual data:

Mean, variance, pdf

Forward adaptive (off-line)Divide source in blocksAnalyze block statisticsSet quantization scheme

Side channel

Backward adaptive (on-line)Adaptation based on quantizer outputNo side channel necessary


Vassil Roussev 13

25

Forward Adaptive Quantization (FAQ)

Choosing block sizeToo large

Not enough resolutionIncreased latency

Too smallMore side channel information

Assuming a mean of zeroVariance estimate:

∑ −

= +=1

022 1ˆ N

i inq xN

σ

26

Speech Quantization Example

16-bit speech 3-bit fixed


Vassil Roussev 14

27

Speech Quantization Example (2)

16-bit speech 3-bit FAQBlock = 128 samples8‐bit variance quantization

28

FAQ Refinement

So far we assumed uniform pdfRefinement

Assume uniform pdf butRecord min/max values for each block

Example:Sena image8x8 blocks3‐bit quantizationOverhead = 16/8x8 = 0.25 bits/pixel


Vassil Roussev 15

29

FAQ Refinement Example

Original: 8 bits/pixel Quantized: 3.25 bits/pixel

30

Backward Adaptive Quantization (BAQ)

ObservationOnly encoder sees inputAdaptation can only be based on quantized output

ProblemHow do we deduce mismatch information from output only?It is possible, if we know the pdf …… and we are very patient


Vassil Roussev 16

31

Jayant Quantizer

IdeaIf input falls in the outer levels

Expand step size

If input falls in the inner levelsContract step size

The product of expansions & contraction should be 1

Multipliers: MkIf input Sn‐1 falls in the kth interval, thenstep is multiplied by Mk

Inner Mk < 1, outer Mk > 1 1)1( −− ∆=∆ nnln M

32

3-bit Jayant Quantizer Output Levels


Vassil Roussev 17

33

Jayant Example

M0 = M4 = 0.8, M1 = M5 = 0.9M2 = M6 = 1.0, M3 = M7 = 1.2, ∆0 = 0.5Input: 0.1 -0.2 0.2 0.1 -0.3 0.1 0.2 0.5 0.9 1.5

34

Picking Jayant Multipliers

∆min / ∆max to prevent under/overflow

⎣ ⎦

∑∏

∏

∏

∏

==

=

=

=

=⇒=

=>=

==

=

=

M

kkk

M

k

Pl

kkll

k

M

k

kk

P

M

k

Nn

M

k

n

Pl

llMNnPM

M

M

kk

kk

k

k

k

k

k

k

00

0

0

0

01

,1where,

where,1

1

1

γ

γγ


Vassil Roussev 18

35

Jayant Example

“Ringing”

36

Jayant Performance


Vassil Roussev 19

37

Non-uniform Quantization

Idea: Pick the boundaries such that error is minimized

I.e., smaller/bigger step for smaller/bigger values

E.g.:

38

pdf-optimized Quantization

Problem: Given fX, minimize MSQE:

( ) dxfyx X

M

i

b

b iqi

i∑∫

= −

−=1

22

1

σ

Set derivative w.r.t. yj to zero and solve for yj:

( )

( )dxxf

dxxfxy

j

j

j

j

b

b X

b

b X

j

∫∫

−

−=

1

1

Set derivative w.r.t. bj to zero and solve for bj:( ) 21 jjj yyb += +


Vassil Roussev 20

39

Lloyd-Max Algorithm

Observation:Circular dependency b/w bj and yj

Lloyd/Max/Lukaszewics/Steinhaus approach:Solve the two iteratively until an acceptable solution is found

Example:

40

Lloyd-Max Algorithm (2)

Boundaries: { b1, b2, …, bM/2-1}b0 = 0, bM/2‐1 = MAX_INPUT

Reconstruction levels: { y1, y2, …, yM/2-1}

( ) ( )dxxfdxxfxyb

b X

b

b X ∫∫=1

0

1

01

One equation, two unknowns: b1, y1Pick a value for b1 (e.g. b1 = 1), solve for y1 and continue:

112 2 yby += ( ) ( )dxxfdxxfxyb

b X

b

b X ∫∫=2

1

2

12

… and so on until all { bn} and { ym} are found


Vassil Roussev 21

41

Lloyd-Max Algorithm (3)

Terminating condition:

12122 2ˆ −− += MMM yby

( ) ( )dxxfdxxfxy M

M

M

M

b

b X

b

b XM ∫∫−−

=2/

12/

2/

12/2

ε≤− 22 ˆMM yy

Else: pick a different b1 & repeat

42

Example: pdf-Optimized Quantizers


Vassil Roussev 22

43

Lloyd-Max Quantizer Properties

1. MeanOUTPUT = MeanINPUT

2. VarianceOUTPUT ≤ VarianceINPUT

3. MSQE:

[ ]jj

M

jjxq bxbPy ≤≤−= −

=∑ 1

1

222 σσ

4. If N is a random variable representing QE:

[ ] 2qXNE σ−=

5. Quantizer output and QE are orthogonal:[ ] 0,,,|)( 10 =MbbbNXQE K

44

Mismatch Effects

4-bit Laplacian pdf-optimized quantizer


Vassil Roussev 23

45

Companded Quantization (CQ)

Compressor Uniform quantizer Expander

46

CQ Example: Compressor


Vassil Roussev 24

47

CQ Example: Uniform Quantizer

48

CQ Example: Expander


Vassil Roussev 25

49

CQ Example: Equivalent Non-uniform Quantizer

50

Problems & Extra

Preparation problems (Sayood 3rd, pp. 270-271)

Minimum: 3, 6Recommended: 5, 7

ExtraBennett integral derivation (pp. 261‐263)Entropy‐encoded quantization /9.7/

quantization - itunesdeimos.apple.com/webobjects/core.woa/downloadtrackpreview/uno.… · quantizer...

Documents