unique: unsupervised image quality estimation 2017...unique: unsupervised image quality estimation...
Post on 11-Mar-2020
5 Views
Preview:
TRANSCRIPT
UNIQUE: Unsupervised Image Quality Estimation
D.Temel, M. Prabhushankar, and G. AlRegib
Center for Signal and Information ProcessingSchool of Electrical and Computer Engineering
Georgia Institute of TechnologyAtlanta, GA
1
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
2
Application Average daily shared photos
390 Million
700 Million
70 Million
760 Million
[1]
[1] Adweek, http://www.adweek.com/socialtimes/how-many-photos-are-uploaded-to-snapchat-every-second/621488, Jun 2015[2]LG, “Ultra Clarity, Ultra Scale,” http://www.lg.com/levant_en/Mini-page-ultra/index[3] PetPixel, http://petapixel.com/2015/07/08/instagram-resolution-increase-heres-how-it-affects-image-quality-and-file-size/, July 8, 2015
Smart Capturing
Remote Assistance
[2]
[3]
I. IntroductionImage Quality Assessment : Why?
3
Test setup Reference images [1] Distorted images [1] Subjective Scores
Bad1 veryannoying
Poor2 annoying
Fair3 slightlyannoying
Good4 distortion but not
annoying
VeryGood5 no
perceiveddistortion
[1] Sheikh, H.R., Wang, Z., Cormack, L. and Bovik, A.C., "LIVE Image Quality Assessment Database Release 2", http://live.ece.utexas.edu/research/quality.
Mean opinion scores
I. IntroductionImage Quality Assessment : In Practice
4
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
5
II. Literature Review Data-driven Image Quality Estimators
YEAR 2011 2012 2013 2014 2015 2016
QUALITYESTIMATORS LB
IQ
DIIV
INE
CO
RN
IA
BRIS
QU
E
MLI
QM
CB/
SF
QAC
SPAR
Q
Tang
QAF
Kang
IQA-
CN
N++
Li
DLI
QA
Gao
CN
N-S
VR
UN
IQU
E
Visual system
Color
Do not require
Distortion specific data in the training
Labels in the training
Handcrafting features
6
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
7
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationOverview of UNIQUE
8
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationPreprocessing
9
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationUnsupervised Learning Mechanism
10
11
III. UNIQUE: Unsupervised Image Quality EstimationVisualization of Learnt Filters
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationVisualization of Processed Images
12
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
13
LIVE MULTI TID Total
Compression 460 180 375 1015
Image Noise 174 180 1375 1729
Communication 174 250 424
Blur 174 315 250 739
Color - 375 375
Global - 250 250
Local - 250 250
LIVE database
TID 2013 database
𝐸𝐸[(𝑋𝑋 − 𝜇𝜇𝑋𝑋)(𝑌𝑌 − 𝜇𝜇𝑌𝑌)]𝜎𝜎𝑋𝑋𝜎𝜎𝑌𝑌 1 −
6∑𝑖𝑖=1𝑁𝑁 𝑥𝑥𝑖𝑖 − 𝑦𝑦𝑖𝑖 2
𝑁𝑁(𝑁𝑁2 − 1)
𝑋𝑋𝑖𝑖 ,𝑌𝑌𝑖𝑖 𝑥𝑥𝑖𝑖 ,𝑦𝑦𝑖𝑖𝐸𝐸 𝑋𝑋 − 𝑌𝑌 2
Pearson Linear Correlation Coefficient (PLCC)
Linearity
Root mean square error (RMSE)
Accuracy
Spearman Rank Correlation Coefficient (SRCC)
RankingPerformanceMetrics
Databases
𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑖𝑖𝑜𝑜𝑜𝑜𝑜𝑜𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜𝑡𝑡𝑜𝑜
Outlier Ratio (OR)
Consistency
V. Image Quality Estimators• Validation
14
15
V. Image Quality Estimators• Validation
16
V. Image Quality Estimators• Validation
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
17
YEAR 2011 2012 2013 2014 2015 2016
QUALITYESTIMATORS LB
IQD
IIVIN
EC
OR
NIA
BRIS
QU
EM
LIQ
MC
B/SF
QAC
SPAR
QTa
ngQ
AFKa
ng
IQA-
CN
N++
Li
DLI
QA
Gao
CN
N-S
VRU
NIQ
UE
Visual systemColor
Do not require
Distortion specific data in the trainingLabels in the training
Handcrafting
VI. ConclusionContributions and Observations
To measure perceived quality Hand-crafting is not sufficient, we should also learn from the data.
Labels are not easy to find, we need to focus more on unsupervised approaches.
Color perception must be included in a comprehensive visual system model.
The best example is our visual system, we should model it as much as we can.18
19
Thank You!
20
Backup Slides
81 121 169 400 625
IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUEMulti-model
Varying the number of neurons to learn global and local features
21
Sharpness Threshold
IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUESharpness-weighted Multi-model
22
IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUEVisualization
Images UNIQUE MS-UNIQUE
23
V. Image Quality Estimators
0.60
0.65
0.70
0.75
0.806.55
7.05
7.55
8.05
8.55
0.55
0.75
0.95
1.15
TID13
OR
RM
SER
MSE
LIVE
TID13
• Validation
24
V. Image Quality Estimators
0.90
0.92
0.94
0.96
SRC
C
LIVE
0.70
0.75
0.80
0.85
SRC
C
TID13
0.87
0.89
0.91
0.93
0.95
PLC
C
LIVE
0.70
0.75
0.80
0.85
PLC
C
TID13
• Validation
25
top related