Aesthetic appeal versus usability:Implications for user satisfaction

Gitte Lindgaard & Cathy Dudek

Carleton HOTLab

Ottawa, Canada

2

3

Satisfaction

• …is the poor cousin of usability• Satisfaction defined as attitudinal

– Avoid negative feelings– Measured in rating scales– Outcomes, summaries

• We are interested in the experiential – Process– Construct

4

• One site tested was very high in appeal and very low in usability

• Appeal = reliable ‘wow’ effect• Usability:

• Heuristic evaluation found 157 unique problems

• 121 of these were severe• A subset of these were exposed in the 8

usability tasks

5

Research question• Does the first impression persist

after completing usability tasks?• Or do users change their mind

after encountering serious usability problems?

6

Measures• Satisfaction: proportion of positive

statements in – Perceived usability– Perceived aesthetics– emotion– likeability– expectation

(Lindgaard & Dudek, 2001)

7

Issues raised here

• Task demands– Do users who anticipate a usability test pay

more attention to usability problems when first browsing a site than users who do not expect a test?

– If so, perceived usability scores will be lower for the former than for the latter

8

Experimental design

9

Findings

• Subjects completed, on average, 3.8 of 8 tasks successfully

• no subject completed all the tasks• no task was completed by all

subjects

So, it is safe to conclude that usability levels were very low

10

Findings: perceived usability

33%

31% 13%

Site was not perceived as usable by either groupSite was seen to be less usable after than before the test

11

Findings: perceived aesthetics

91%

87% 91%

Site was perceived as beautiful by both groupsSite remained beautiful after the usability test

12

Findings: emotion

87%

66% 20%

The ratio of positive:negative emotion statements was lowerfor the test group both before and after the test.

13

Findings: likeability

79%

49% 25%

The ratio of positive:negative likeability statements was Lower for the test group both before and after the test

14

Findoings: expectation

64%

11% 5%

The ratio of positive:negative expectation statements wasVery much lower for the test group before and after The test

15

Findings: satisfaction

66%

51% 25%

Satisfaction appears to be determined by several factors

16

• The first impression apparently rests on aesthetics

• The perception of beauty persists, but

• Perceived usability, likeability and expectation change after facing serious usability problems

First Impressions

17

First impression

• Formed in an instant (3-5 msec)• Based on changes in arousal levels

(Berlyne, 1971; 1972)

• Evoked via the amygdala, not via the hypothalamus (Damasio, 2000; LeDoux, 1994;1996; Goleman, 1996)

• Can be overridden by pre-exposure decision (Epstein, 1997)

• Is difficult to change – confirmation bias

(Doherty, Mynatt & Tweney, 1977)

18

One question is…

• Does emotion precede cognition…

(Zajonc, 1980; Bornstein, 1992)

• …or is it the other way around? (Epstein, 1997)

• I.e. are first impressions

‘what my body tells me to feel’, or are they

‘what my brain tells me to think’?

19

Issues raised here

• Confirmation bias – If the first impression drives satisfaction, it

should not change after usability test– If usability drives satisfaction, it should change

after usability test– and it should vary between high- and low-

usability sites

20

Experimental design

• Group 1 (n = 40):Browse interview/ratings

• Group 2 (n = 40): Browse interview/ratings usability test interview/ratings

• 2 e-commerce sites tested

• Scores: • (a) proportion of positive statements• (b) WAMMI (Kirakowski et al. 1998)

21

Results: Satisfaction

0

0.5

1

LOW HIGH

Web Site

BUT

AUT

Mean satisfaction scores before & after test

Main effect, before/after (p < .001); main effect for web site (p ,< .05)

22

Perceived usability

1

2

3

4

5

LOW HIGH

Web Site

BUT

AUT

Mean perceived usability before/after test

Main effect before/after (p < .001); main effect for web site (p < .001)

23

Aesthetics

0

0.5

1

LOW HIGH

Web Site

BUT

AUT

Mean aesthetics score before/after test

No significant effects

24

Satisfaction, before test only

0

0.5

1

LOW HIGH

Web Site

BRO

BUT

Mean satisfaction scores, first interview

Main effect for web site (p < .05); main effect for subject-group (p < .05)

25

Perceived usability before test

1

2

3

4

5

LOW HIGH

Web Site

BRO

BUT

Mean perceived usability scores before test only

Main effect for web site (p < .001)

26

Conclusion

• Confirmation bias– Aesthetics scores taken on their own did not differ

before the test– ..and they did not change after test – Confirmation bias on the aesthetics dimension

• But– Satisfaction scores decreased after the test– Perceived usability scores decreased– No confirmation bias on overall satisfaction or on

usability

27

Conclusion

• Task demands– Lower satisfaction scores for subjects expecting

a usability test than for browsing-only subjects suggest that task demands do affect attention to usability

– Subjects are sensitive to actual usability levels– As evidenced both in satisfaction scores and in

perceived usability scores

28

Conclusion

• So, satisfaction appears to be driven partly by actual usability

• Aesthetics judgments appear to be independent of perceived usability

29

Next steps

• Currently developing satisfaction scales that enable developers to pinpoint where to improve their sites to increase user satisfaction

30

So, now to aesthetics

• Gary Fernandes MA thesis:– 125 sites collected, all of unknown companies– Preliminary study, n = 22– Selected 25 best and 25 worst sites– N = 30– Viewed sites for 500 msec, then rated visual

appeal in two rounds

31

Measurement scale

VeryUnattractive

VeryAttractive

33

34

35

36

37

38

Appeal ratings, study 1

Median Visual Appeal, Phase 2

1009080706050403020100

Med

ian

Vis

ual A

ppea

l, P

hase

1100

90

80

70

60

50

40

30

20

10

0

39

Appeal ratings, study 2

Median Visual Appeal, Phase 2

1009080706050403020100

Med

ian

Vis

ual A

ppea

l, Ph

ase

1

100

90

80

70

60

50

40

30

20

10

0

40

Visual appeal ratings, study 1 vs study 2

Median Visual Appeal, Study 2

1009080706050403020100

Med

ian

Vis

ual A

ppea

l, St

udy

1

100

90

80

70

60

50

40

30

20

10

0

41

Conclusion

• Aesthetics judgments are made very quickly

• They are highly robust

• New results show that they persist even when subjects are able to inspect the home page for an unlimited period of time

42

Next steps

• Expose stimuli for 40 msec

• Collect genres of sites

• Evolve tool enabling companies to test their own web site against others

43