h. r. sheikh, a. c. bovik, “image information and visual quality,” ieee trans. image process.,...
TRANSCRIPT
![Page 1: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/1.jpg)
IMAGE QUALITY ASSESSMENT
H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image
Process., vol. 15, no. 2, pp. 430-444, Feb. 2006
Lab for Image and Video Engi., Dept. of ECE
Univ. of Texas at Austin
![Page 2: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/2.jpg)
Outline
Introduction of Image Quality Assessment
Visual Information Fidelity Experiments and Results Conclusion
![Page 3: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/3.jpg)
Quality Assessment (QA)
For testing, optimizing, bench-marking, and monitoring applications.
Quality?
![Page 4: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/4.jpg)
Three Broad QA Categories
Full-Reference (FR) QA Methods Non-Reference (NR) QA Methods Reduced-Reference (RR) QA Methods
Reference Image
Distorted ImageFR QA Quality
![Page 5: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/5.jpg)
PSNR Simple but not close to human visual
quality
Contrast enhancement
Blurred
JPEG compressed
VIF = 1.10
VIF = 0.07VIF = 0.10
![Page 6: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/6.jpg)
Prior Arts
Image Quality Assessment based on Error Sensitivity
CSF: Contrast Sensitivity FunctionChannel Decomposition: DCT or Wavelet TransformError Normalization: Convert the Error into Units of Just Noticeable Difference (JND)Error Pooling:
1
,,
l kklkl eeE
![Page 7: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/7.jpg)
Problems of Error-Sensitivity Approaches
The Quality Definition Problem The Suprathreshold Problem The Natural Image Complexity Problem The Decorrelation Problem The Cognitive Interaction Problem
![Page 8: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/8.jpg)
Visual Information Fidelity
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
ReferenceImage
TestImage
Human Visual
System
Reference Image Information
Human Visual
System
Test Image Information
![Page 9: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/9.jpg)
Definition of VIF
subbandsj
jNjNjN
subbandsj
jNjNjN
sECI
sFCI
VIF,,,
,,,
;
;
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
![Page 10: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/10.jpg)
Source Model
The natural images are modeled in the wavelet domain using Gaussian scale mixtures (GSMs).
tscorfficien bubband toingcorrespond
vectorsldimensiona- are and
: , : and
indices spatial ofset thedenodes where
)C covariance andmean -(zero
:
U2
M
MUC
IiUUIiSS
I
S
IiUSUSC
ii
ii
i
ii
The subband coefficients are partitioned into nonoverlapping blocks of M coefficients each
![Page 11: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/11.jpg)
Gaussian scale mixture (GSM)
Wavelet coefficient => non-Gaussian
The variance is proportional to the squared magnitudes of coefficients at spatial positions.
UzX
V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.
![Page 12: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/12.jpg)
Implementation Issues
Assumption about the source model:
N
i
TiiU
iUTi
i
CCN
C
M
CCCs
1
12
1ˆ
ˆ
V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a localGaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol.4119, pp. 363–371, 2000.
![Page 13: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/13.jpg)
Distortion Model
IC
IiVV
IigG
IiVCgVGCD
vV
i
i
iii
2 ance with variRF noise
Gaussian mean -zreo additive stationary a is :
fieldgain scalar icdeterminst a is : where
:
![Page 14: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/14.jpg)
Distorted Images
Distorted Images Synthesized versions
![Page 15: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/15.jpg)
Distorted Images
![Page 16: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/16.jpg)
Human Visual System (HVS)Model
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
ICC
C
IiNNIiNN
NDF
NCE
nNN
i
ii
2'
aslity dimensiona same the
ithGaussian w temultivaria eduncorrelatmean -zreo are
:'' and : where
image)(test '
image) (reference
![Page 17: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/17.jpg)
Visual Information Fidelity Criterion (IFC)
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
C E
Mutual InformationI(C;E)
![Page 18: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/18.jpg)
Mutual Information
Assuming that G, and are known2v 2
n
N
i n
nUi
N
iiiiii
N
iiji
Njiji
N
j
N
i
NNN
I
ICs
sNhsNCh
sECI
sECECIsECI
12
22
2
1
1
11
1 1
log2
1
;
,,;;
C E
Mutual InformationI(C;E)
![Page 19: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/19.jpg)
Mutual Information
N
i
M
k nv
kiiNNN
N
i
M
k n
ki
N
i n
nUiNNN
k
TUU
sgsFCI
s
I
ICssECI
QQCC
1 122
22
2
1 12
2
2
12
22
2
1log2
1;
1log2
1
log2
1;
seigenvalue ofmatrix diagonal a is
symmetric, is Since
![Page 20: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/20.jpg)
Implementation Issues
Assumption about the source model:
N
i
TiiU
iUTi
i
CCN
C
M
CCCs
1
12
1ˆ
ˆ
V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a localGaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol.4119, pp. 363–371, 2000.
![Page 21: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/21.jpg)
Implementation Issues
Assumption about the distortion model:
use B x B window centered at coefficient i to estimate and at i
Assumption about the HVS model:
Hand-optimize the value of
),(ˆ),(ˆ
),(),(ˆ2,
1
DCCovgDDCov
CCCovDCCovg
iiv
i
ig2ˆv
2ˆn
(by linear regression)
![Page 22: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/22.jpg)
Definition of VIF
subbandsj
jNjNjN
subbandsj
jNjNjN
sECI
sFCI
VIF,,,
,,,
;
;
Natural Image Source
Channel(Distortion)
HVS
HVS
C DF
E
![Page 23: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/23.jpg)
Experiments Twenty-nine high-resolution(768x512) 24-bits/pixel
RGB color images Five distortion types: JPEG 2000, JPEG, white
noise in RGB components, Gaussian blur, and transmission errors
20-25 human observers Perception of quality: “Bad,” “Poor,” “Fair,” “Good,”
and “Excellent” Scale to 1-100 range and obtain the difference
mean opinion score (DMOS) for each distorted image
Data base: http://live.ece.utexas.edu/research/quality/
![Page 24: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/24.jpg)
Scatter Plots for Four Objective Quality Criteria
(x) JPEG2000, (+) JPEG,(o) white noise in RGB space, (box) Gaussian blur, and (diamond) transmission Errors in JPEG2000 stream over fast-fading Rayleigh channel
![Page 25: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/25.jpg)
Scatter Plots for the Quality Prediction
![Page 26: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/26.jpg)
Validation Scores
THE VALIDATION CRITERIA ARE: CORRELATION COEFFICIENT (CC), MEAN ABSOLUTE ERROR (MAE), ROOT MEAN-SQUARED ERROR (RMS), OUTLIER RATIO(OR), AND SPEARMAN RANK-ORDER CORRELATION COEFFICIENT (SROCC)
Two version of VIF:VIF using the finest resolution at all orientations andUsing the horizontal and vertical orientations only
![Page 27: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/27.jpg)
Cross-Distortion Performance
![Page 28: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/28.jpg)
Cross-Distortion Performance
(dark solid) JPEG2000, (dashed) JPEG, (dotted) white noise, (dash-dot) Gaussian blur, and (light solid) transmission errors in JPEG2000 stream over fast-fading Rayleigh channel
![Page 29: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/29.jpg)
Dependence on the HVS Parameter
Dependence of VIF performance on the parameter.(solid) VIF, (dashed) PSNR,(dash-dot) Sarnoff JNDMetrix 8.0, and (dotted) MSSIM.
2ˆn
![Page 30: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/30.jpg)
Conclusion
A VIF criterion for full-reference image QA is presented.
The VIF was demonstrated to be better than a state-of-the-art HVS-based method, the Sarnoff’s JND-Metrix, as well as a state-of-the-art structural fidelity criterion, the SSIM index
The VIF provides the ability to predict the enhanced image quality by contrast enhancement operation.
![Page 31: H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and](https://reader038.vdocuments.site/reader038/viewer/2022102818/56649cfa5503460f949cb716/html5/thumbnails/31.jpg)
Reference1. H. R. Sheikh, A. C. Bovik, “Image Information and Visual
Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006.
2. H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117–2128, Dec. 2005.
3. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error measurement to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.
4. V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.