h. r. sheikh, a. c. bovik, “image information and visual quality,” ieee trans. image process.,...
TRANSCRIPT
IMAGE QUALITY ASSESSMENT
H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image
Process., vol. 15, no. 2, pp. 430-444, Feb. 2006
Lab for Image and Video Engi., Dept. of ECE
Univ. of Texas at Austin
Outline
Introduction of Image Quality Assessment
Visual Information Fidelity Experiments and Results Conclusion
Quality Assessment (QA)
For testing, optimizing, bench-marking, and monitoring applications.
Quality?
Three Broad QA Categories
Full-Reference (FR) QA Methods Non-Reference (NR) QA Methods Reduced-Reference (RR) QA Methods
Reference Image
Distorted ImageFR QA Quality
PSNR Simple but not close to human visual
quality
Contrast enhancement
Blurred
JPEG compressed
VIF = 1.10
VIF = 0.07VIF = 0.10
Prior Arts
Image Quality Assessment based on Error Sensitivity
CSF: Contrast Sensitivity FunctionChannel Decomposition: DCT or Wavelet TransformError Normalization: Convert the Error into Units of Just Noticeable Difference (JND)Error Pooling:
1
,,
l kklkl eeE
Problems of Error-Sensitivity Approaches
The Quality Definition Problem The Suprathreshold Problem The Natural Image Complexity Problem The Decorrelation Problem The Cognitive Interaction Problem
Visual Information Fidelity
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
ReferenceImage
TestImage
Human Visual
System
Reference Image Information
Human Visual
System
Test Image Information
Definition of VIF
subbandsj
jNjNjN
subbandsj
jNjNjN
sECI
sFCI
VIF,,,
,,,
;
;
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
Source Model
The natural images are modeled in the wavelet domain using Gaussian scale mixtures (GSMs).
tscorfficien bubband toingcorrespond
vectorsldimensiona- are and
: , : and
indices spatial ofset thedenodes where
)C covariance andmean -(zero
:
U2
M
MUC
IiUUIiSS
I
S
IiUSUSC
ii
ii
i
ii
The subband coefficients are partitioned into nonoverlapping blocks of M coefficients each
Gaussian scale mixture (GSM)
Wavelet coefficient => non-Gaussian
The variance is proportional to the squared magnitudes of coefficients at spatial positions.
UzX
V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.
Implementation Issues
Assumption about the source model:
N
i
TiiU
iUTi
i
CCN
C
M
CCCs
1
12
1ˆ
ˆ
V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a localGaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol.4119, pp. 363–371, 2000.
Distortion Model
IC
IiVV
IigG
IiVCgVGCD
vV
i
i
iii
2 ance with variRF noise
Gaussian mean -zreo additive stationary a is :
fieldgain scalar icdeterminst a is : where
:
Distorted Images
Distorted Images Synthesized versions
Distorted Images
Human Visual System (HVS)Model
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
ICC
C
IiNNIiNN
NDF
NCE
nNN
i
ii
2'
aslity dimensiona same the
ithGaussian w temultivaria eduncorrelatmean -zreo are
:'' and : where
image)(test '
image) (reference
Visual Information Fidelity Criterion (IFC)
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
C E
Mutual InformationI(C;E)
Mutual Information
Assuming that G, and are known2v 2
n
N
i n
nUi
N
iiiiii
N
iiji
Njiji
N
j
N
i
NNN
I
ICs
sNhsNCh
sECI
sECECIsECI
12
22
2
1
1
11
1 1
log2
1
;
,,;;
C E
Mutual InformationI(C;E)
Mutual Information
N
i
M
k nv
kiiNNN
N
i
M
k n
ki
N
i n
nUiNNN
k
TUU
sgsFCI
s
I
ICssECI
QQCC
1 122
22
2
1 12
2
2
12
22
2
1log2
1;
1log2
1
log2
1;
seigenvalue ofmatrix diagonal a is
symmetric, is Since
Implementation Issues
Assumption about the source model:
N
i
TiiU
iUTi
i
CCN
C
M
CCCs
1
12
1ˆ
ˆ
V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a localGaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol.4119, pp. 363–371, 2000.
Implementation Issues
Assumption about the distortion model:
use B x B window centered at coefficient i to estimate and at i
Assumption about the HVS model:
Hand-optimize the value of
),(ˆ),(ˆ
),(),(ˆ2,
1
DCCovgDDCov
CCCovDCCovg
iiv
i
ig2ˆv
2ˆn
(by linear regression)
Definition of VIF
subbandsj
jNjNjN
subbandsj
jNjNjN
sECI
sFCI
VIF,,,
,,,
;
;
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
Experiments Twenty-nine high-resolution(768x512) 24-bits/pixel
RGB color images Five distortion types: JPEG 2000, JPEG, white
noise in RGB components, Gaussian blur, and transmission errors
20-25 human observers Perception of quality: “Bad,” “Poor,” “Fair,” “Good,”
and “Excellent” Scale to 1-100 range and obtain the difference
mean opinion score (DMOS) for each distorted image
Data base: http://live.ece.utexas.edu/research/quality/
Scatter Plots for Four Objective Quality Criteria
(x) JPEG2000, (+) JPEG,(o) white noise in RGB space, (box) Gaussian blur, and (diamond) transmission Errors in JPEG2000 stream over fast-fading Rayleigh channel
Scatter Plots for the Quality Prediction
Validation Scores
THE VALIDATION CRITERIA ARE: CORRELATION COEFFICIENT (CC), MEAN ABSOLUTE ERROR (MAE), ROOT MEAN-SQUARED ERROR (RMS), OUTLIER RATIO(OR), AND SPEARMAN RANK-ORDER CORRELATION COEFFICIENT (SROCC)
Two version of VIF:VIF using the finest resolution at all orientations andUsing the horizontal and vertical orientations only
Cross-Distortion Performance
Cross-Distortion Performance
(dark solid) JPEG2000, (dashed) JPEG, (dotted) white noise, (dash-dot) Gaussian blur, and (light solid) transmission errors in JPEG2000 stream over fast-fading Rayleigh channel
Dependence on the HVS Parameter
Dependence of VIF performance on the parameter.(solid) VIF, (dashed) PSNR,(dash-dot) Sarnoff JNDMetrix 8.0, and (dotted) MSSIM.
2ˆn
Conclusion
A VIF criterion for full-reference image QA is presented.
The VIF was demonstrated to be better than a state-of-the-art HVS-based method, the Sarnoff’s JND-Metrix, as well as a state-of-the-art structural fidelity criterion, the SSIM index
The VIF provides the ability to predict the enhanced image quality by contrast enhancement operation.
Reference1. H. R. Sheikh, A. C. Bovik, “Image Information and Visual
Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006.
2. H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117–2128, Dec. 2005.
3. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error measurement to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
4. V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.