iphone vs. samsung galaxy s ii - user experience case study (english)

iPhone vs. Samsung Galaxy A User Experience Case Study

The Study

User Experience Evaluation using Eye Tracking, i² BrandREACT and an online survey.

Sample: N=20; Age: 25 – 45

• 10 ordinary mobile phone users (non smartphone users)

• 5 iOS (iPhone) users

• 5 Android users

Test Design: Within Subject Design - each respondent evaluates both smart phones.

Simulation of a daily situation to compare the user experience of the iPhone 4 and the Samsung Galaxy S2.

Eye Tracking

You have received a missed call from an unknown number.

Please save this number as a new contact,

„Peter Schneider“.

Apple iPhone 4 Samsung Galaxy S2

Apple iPhone 4

Samsung Galaxy S2

Task Scenario

14/11/12

2

Set Up & Test phones

Key Takeaways

14/11/12 3

Strong brand image of Apple distorts real product experience. Samsung Galaxy definitely catches up with Apple.

Consumer awareness of the iPhone’s success and its reputation for usability makes users explicitly state that the iPhone is easier to use and more attractive than the Samsung Galaxy S2.

On an emotional, implicit level users are stimulated by a straight forward and powerful brand image. They associate positive items faster and more often with the iPhone than with Galaxy S2.

But: Eye tracking data reveals that the real user experience is not always like users say or think it is!

Eye tracking and interaction tracking show scattered attention on the display. Users do not focus on the correct icons/buttons to solve the task and perform more false interactions than with the Samsung Galaxy S2.

Performance data pronounces a lower success rate for the iPhone.

14/11/12 4

The Study

14/11/12 5

14/11/12 6

Toolset

Signal Implicit

Explicit

ACTION

Methods:

Eye Tracking

Behavioral Monitoring

EEG

EDA

Methods:

Ratings

Interviews

Methods:

BrandREACT

Explicit: Evaluating user experience through Interviews, Thinking Aloud and System Acceptance Ratings

Interview = Opinions, problems and needs of the target group are analyzed. Users are addressed on an individual level.

Thinking Aloud = Users directly or retrospectively (retrospective thinking aloud) comment on their own usage behavior. This way the user’s thoughts become more comprehensible and usage behavior can be reflected and understood easily.

System Acceptance Ratings = Evaluation of the acceptance of product features using semantic differentials and rating scales.

14/11/12 7

Implicit: Measuring the unconscious using the i² BrandREACT. Stimulation / Power / Balance

BALANCE

entertaining playful young modern exciting

dominant powerful superior convincing strong

cautious, robust, tidy, correct, formal

The need for STIMULATION

The need for joy & curiosity.

Willing to discover and to learn.

The need for BALANCE

The need for stability & safety.

Avoiding risks.

The need for POWER

The need for distinction & superiority.

Striving for status and autonomy.

i² BrandREACT = Reaction time based tool Respondents are asked to associate certain adjectives (based on 3 emotional need dimensions) with a product/brand. Compliance rate and reaction time are measured.

14/11/12 8

Signal: Assessing attention and interaction behavior with Eye Tracking

Gaze heatmap & interactions To find out… • if attention is focused

on the correct areas

• To find out where users interacted and how (tap, slide, hold, etc.)

Interaction shares To find out… • How many interactions

were there with a certain area/icon/button

• Which interaction type is mostly expected

Reach attention To find out… • What percentage of the

target group have actually seen the area/button/icon

• What are the dominant mental models in the users minds

Time to first contact To find out… • How early a certain

area/icon/button is perceived

• If an icon is intuitively designed and easily accessible 14/11/12 9

Results

14/11/12 10

45%

48%

54%

48%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Re-use

Recommendation

Re-use & Recommendation

iPhone Samsung Galaxy S2

3,3

3,5

3,6

3,4

3,6

3,8

3,8

4,2

1,0 1,5 2,0 2,5 3,0 3,5 4,0 4,5 5,0

Acceptance

Comprehensibility

Utility

Aesthetical quality

SystemAcceptance

iPhone Samsung Galaxy S2

Explicit: iPhone strong in all categories, especially aesthetics

The iPhone is considered to be especially...

• Aesthetically attractive

• Comprehensible

Users show high acceptance levels and state their willingness to re-use the iPhone.

N=20

Please rate the smart phone you just used for the following dimensions on a scale from 1 to 5. „1“ means „not at all“ and „5“ means „very much“.

„How likely is it that you would recommend/reuse this smart phone on a scale from 0 – 100 %?“ 14/11/12 11

12,71

2,66

14,12

3,09

9,24

5,00

iPhone Samsung Galaxy

Implicit: Faster & more positive associations with the iPhone. Samsung not convincing.

The iPhone performs better than the Galaxy in all 3 categories.

Respondents associate more positive adjectives faster with the iPhone.

• The iPhone is especially strong in the power dimension: It is convincing, strong and secures social status.

• The Galaxy S2 mostly fulfills the need for balance: associated adjectives are: cautious, robust and correct.

N=20 The higher the value, the more positive associations have been selected and the faster the reaction time. Negative values show negative associations.

ST

IMU

LA

TIO

N

PO

WE

R

BA

LA

NC

E

ST

IMU

-L

AT

ION

PO

WE

R

BA

LA

NC

E

14/11/12 12

Signal: iPhone not intuitive

• Users mostly accidentally call the number.

• Only 18 correct interactions

• Correct Icon is perceived rather late (after 12.6s) N=20

Correct interaction: Tap small arrow

• Attention is rather scattered No clear focus on arrow..

• Many users fail/struggle because of tapping the number

14/11/12 13

Signal: Samsung Galaxy convinces with user-friendly interface

• Users intuitively perform the adequate interaction. They choose the phone number.

• Correct screen element (phone number) spotted instantaneously (after 0.6s)

N=20

Correct interaction: Tap the number of missed call

• User attention is concentrated on the phone number.

14/11/12 14

Signal: Greater overall success rate with Galaxy

iPhone:

More than the half of the users struggle or fail.

40% succeed.

Galaxy:

Android seems to be easier to use:

65% succeed, nobody fails totally.

N=20

40%

65%

50%

35%

10%

0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Task 1 - iPhone Task 1 - Galaxy

Success Rate

Fail/Timeout

Struggle

Success

14/11/12 15

Signal: Galaxy S2’s Android interface easier to use for all user groups

iPhone:

• Common cell phone users often struggle.

• iPhone users succeed.

• Android users tend to face problems as well.

Galaxy:

• Feature phone users show greater success rate. • iPhone users mainly succeed. • All Android users directly succeed.

All smartphone users succeed with the type of interface they already know! iPhone users are better in handling the Android phone than Android users are with iOS.

N=20

14/11/12 16

eye square GmbH

Schlesische Straße 29 – 30 (Aufgang F)

D – 10997 Berlin

Fon +49 - 30 - 698144-0 | Fax +49 - 30 - 698144-10

E-Mail eye@eye-square.com | www.eye-square.com

Your contact:

Carina Lehne

UNIT LEAD | Senior Research Consultant

+49 30 69 81 44 28

lehne@eye-square.com