structure and aesthetics in non- photorealistic images hua li, david mould, and jim davies carleton...
TRANSCRIPT
Structure and Aesthetics in Non-Photorealistic Images
Hua Li, David Mould, and Jim DaviesCarleton University
Artistic or Messed
2/35
Related Work on Evaluating Non-Photorealistic Algorithms
• Performance based on processing speed– ill-suited for stylization
• Side-by-side comparisons – not fully convinced by audience
3/35
Perceptual Evaluation on Non-Photorealistic Algorithms
• Quantitative evaluation– rating scores [Schumann et al. 96, Gooch and
Willemsen 02, Mandryk et al. 2011, Mould et al. 2012]
– response time [Gooch et al. 04]– eye-tracking data [Mandryk et al. 2011, Mould et
al. 2012]• Qualitative evaluation– questionnaire-based
4/35
Motivation of Our StudyTone Structure
Tone-based Structure-based
Halftoning
[Pang et al. 08] [Chang et al. 09] [Ours 10]
Stippling [Secord 02]
[Mould 07] [Martin et al. 11] [Ours 11]
Screening [Ulichney 98]
Abstraction [Kyprianidis 11] [Mould 12, 13]
[Qu et al. 08] [Ours 11]
[Floyd and Steinberg 76]
[Ostromoukhov 01]
5/35
Questions to Answer
• Are structural and aesthetic quality related? • Do images matter for side-by-side
comparisons?
6/35
Participants
• 30 participants– 15 female and 15 male– 11 artists– aged 18 to 33
7/35
Study Overview
• 1 ~ 1.5 hours to complete the experiment• Using the keyboard or the mouse to enter
their responses• Tasks:– rating structural and aesthetic quality– collecting response times for rendered images
8/35
Image Stimuli
• Seven categories – include cars, cats, persons, flowers, buildings,
mugs, and birds.• Each category contains 13 different images
including– one unprocessed image– and 12 rendered images using 12 algorithms.
• Images are black and white, or greyscale to remove the influence of color.
9/36
Procedure
• Step 1: verbal introduction• Step 2: training• Step 3: formal study• Step 4: questionnaire• Step 5: ranking
10/35
Interfaces Used
Interface for collecting the response time 11/35
Interfaces Used
Aesthetic rating Structural rating12/35
Experimental Images -Bird Category
Unprocessed13/35
Experimental Images - Bird Category
Structure-Preserving Stippling (SPS)
Structure-Aware
14/35
Experimental Images - Bird Category
Content-Sensitive Screening (CSS)
Structure-Aware
15/35
Experimental Images - Bird Category
SPS with Exclusion Masks (SPH)
Structure-Aware
16/35
Experimental Images - Bird Category
Line Art using edge tangent field (ETF)
Structure-Aware
17/35
Experimental Images - Bird Category
Artistic Tessellation (AT)
Structure-Aware
18/35
Experimental Images - Bird Category
Line Art from SPS (Drawing)
Structure-Aware
19/35
Experimental Images - Bird Category
Secord’s Stippling Method (Secord)
Tone-based
20/35
Experimental Images - Bird Category
Line Art using edge tangent field (Mmosaics)
Tone-based
21/35
Experimental Images - Bird Category
Contrast-Aware Halftoning (CAH)22/35
Experimental Images - Bird Category
Black and White (BW)23/35
Experimental Images - Bird Category
Adding 50% salt and pepper noise (Noisy)
Reduced Information
24/35
Experimental Images - Bird Category
Gaussian filter (Blurring)
Reduced Information
25/35
Positive Correlation Between Structural and Aesthetic Ratings
26/35
Dot-based Methods (Stippling)
Aesthetic Structure1
1.5
2
2.5
3
3.5
4
4.5
SecordSPS
Mea
n A
esth
etic\
Stru
ctur
e Ra
tings
Structure-Aware
Tone-basedStructure-
Aware
Tone-based
27/35
Region-based Methods (Mosaics)
Aesthetic Structure1
1.5
2
2.5
3
3.5
MmosaicsAT
Mea
n A
esth
etic\
Stru
ctur
e Ra
tings
Structure-Aware
Tone-basedStructure-
Aware
Tone-based
28/35
Effect of Category on Ratings
Aesthetic Structure 1
1.5
2
2.5
3
3.5
4
4.5CarCatBirdPersonMugFlowerBuilding
Mea
n A
esth
etic\
Stru
ctur
e Ra
tings
29/35
Effect of Category on Response Time
Series1
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
CarCatBirdPersonMugFlowerBuilding
Z va
lues
of R
espo
nse
Tim
e
30/35Building < Flower < Bird < Cat < Person < Mug < Car
Artists and Non-Artists
Aesthetic Structure1
1.5
2
2.5
3
3.5
4
4.5
5
ArtistNon-Artist
Mea
n A
esth
etic\
Stru
ctur
e Ra
tings
31/35
Overall Ranks after Study
• Participants preferred the AT images (7/30 responses) the most, CAH second (6/30).
• Participants’ least favorite – blurred images most often (20/30 responses), and
with AT second (5/30). • Controversial ranking for stylized images
rendered by the AT method.
32/35
1st Ran
k
2nd Rank
3rd Ran
k
4th Ran
k
5th Ran
k
6th R
ank
7th R
ank
8th R
ank
9th R
ank
10th R
ank
11th R
ank
12th R
ank
0
5
10
15
20
25
CAHATNoisyBlurring
From Most Favourite to Least Favourite
Num
ber o
f Par
ticip
ants
Conclusions
• Considering structure as a possible way to increase aesthetic appeal.
• Considering the choice of the images used.– Generally, bird images were the easiest images to
abstract, while Person images were the most difficult.
33/35
Future Work
• More Participants• More Categories• More NPR Algorithms• Eye tracker
34/35
Thanks for Your Attention.
35/35
Questions?