Skype Heuristics

Evaluation Report

Killian Vigna (10129758),

Killian Stone (09005157),

Michelle Kennedy (09005334)

Contents:

1) Executive Summary

2) Introduction

3) Product description

4) A Detailed Heuristic List

5) Cooperative Evaluation of Prototype (Task List)

6) Setting up the Task Environment & Equipment

7) Performing the Evaluation

8) Results from evaluation

9) Design recommendations

10) Conclusions

Executive Summary

This is a heuristics evaluation report, the aim of this report is to identify a system, and explore its

interactive qualities. All evaluations are based on heuristics, which are a set of guidelines set out by

experts to evaluate an Interactive System’s Usability. This report was undertaken to summarize and

evaluate User Interface ability of the program Skype and better help the design of usable Interactive

Systems.

“The goal of heuristic evaluation is to find the usability problems in the design so that they can be

attended to as part of an iterative design process. Heuristic evaluation involves having a small set of

evaluators examine the interface and judge its compliance with recognized usability principles”

(http://www.useit.com/papers/heuristic/)

The heuristics used in this report, we’re that of Usability expert, Jakob Nielsen, and his 10 heuristics.

1. Visibility of system status.

2. Match between system and the real world.

3. User control and freedom.

4. Consistency and standards.

5. Error prevention.

6. Recognition rather than recall.

7. Flexibility and efficiency of use.

8. Aesthetic and minimalist design.

9. Help users recognize, diagnose, and recover from errors.

10. Help and documentation.

These 10 recommended heuristics by Jakob Nielsen will be used to explore the Interactive usability of

the Skype interface and allow us to better understand its use for online Calling, Video Calling, and

Instant Messaging amongst other user’s.

Introduction

This report discuss’s the heuristic evaluation process for Skype, a free internet based video calling

service. It begins by describing the product and discussing the target population for the program. It

then gives a look at the heuristics used to review the program and what major issuses, if any, were

found in this review of the program. We plan to outline these problems and suggest possible measures

which could be taken to fix or avoid them in the future therefore creating a greater overall experience

for skype users everywhere..

Product Description

Skype, which is now part of Microsoft, is software which allows a community of people from all over

the world interact for free via Phone, Live stream Video Calling, or even through Instant Messenging.

It allows its users to free call anyone, anywhere, at any time of the day once downloaded.

You can even obtain Skype for your business through paying by Pay as You Go, or committing to a

small fee Bill Pay package, allowing its customers to do more. Paying users have more access to

colleagues, customers, whoever, with their low cost Skype fee’s to Landlines and Mobile phones,

which may not have the Skype tool. The extra fee benefits can mean better corporate calls, by being

able to interact by Live Stream Video conference calls to up to four Skype accounts at a go.

Skype can be used anywhere, at any time now thanks to the use of Smartphones. Anyone who has

access to a PC, MAC, Laptop, or even smartphone can now download Skype for free and avail of its

free interaction services. Although its recognised for its Live stream Video Calling, users without a

web cam can still engage in online communications through Instant Messaging, or by connecting a

Skype phone up to their computer.

A Detailed Heuristic List

1. Visibility of system status

To sign into Skype it is clear to the user where to enter the username and password.

While the program is “signing in”, a loading circle appears indicating it is in the process of

doing so.

On start up of the Skype program, there is an audio cue to let the user know it is ready for use.

A pop up interface appears when a user is receiving an incoming call or video call. The

options to answer, answer with video or decline are given in a clear manner.

When an instant message is received a small orange dot appears beside the other user’s name

in the contacts. Also an audio cue can be heard at the same time. The conversation also

appears in the recent tab which is highlighted orange. The Skype icon flashes on the task bar.

2. Match between system and the real world

The Skype program avoids using technical jargon, instead using familiar language where any

user can comprehend. It presents information in such a way that the user understands easily.

Information is presented in appropriate order.

There is a slight repetitiveness with regard to the Skype options. Many of the sub categories

from the drop down menus lead to the same screen. For example, if a user selects privacy

from the Skype menu, or audio settings from the call menu, or options from the tool menu,

they are led to the same window.

3. User control and freedom

If the user wishes to go back to the log in page while the loading circle is on screen, there is no

option to do so. The user must wait for the program to finish loading.

The grouping together of the end call button, adding participants to the conversation, the

microphone and video buttons is of poor design. The user could easily make the mistake of

ending the call rather than choosing the other options.

In the options window, the user is given the choice of either saving the changes made or

returning by clicking cancel.

There is a clearly visible “close” button in the conversations window to exit.

4. Consistency and standards

Skype uses typical language accepted by numerous video calling and instant messaging

services.

Regular and novice users can easily understand what action leads to what result as no jargon is

used in the program.

Skype adheres to these standards and obeys them. Making it an easy program to learn for

anyone, novice or expert.

5. Error prevention

Skype is a well built program which rarely crashes

The program can crash however due to poor Internet connection, typically during a video call.

Skype automatically tries to reconnect the call without any actions being taken by the user.

If connection cannot be re-established then Skype shows a pop up message explaining the

issue in plain English to the user.

After every call Skype asks the user to rate the quality of the call to help further development

of the platform.

6. Recognition rather than recall

Typically Skype’s layout is quite simple and easy to use.

During video calls however a menu bar appears along the bottom of the received video. This

menu bar contains images referencing certain actions that can be made during the call.

However, it is not clearly defined what these buttons do and can only really be discovered by

trial and error. After a few use’s it becomes second nature but the user should never have to

do this. The actions the buttons take should be clear for the users.

7. Flexibility and efficiency of use

Brightly coloured, simple layout.

Few buttons leaving for simplified efficiency.

Live stream calling at the click of a button.

Visually aesthetic, simple on screen navigation and easily able to tell who’s online, currently

busy, or offline.

8. Aesthetic and minimalist design

6 basic toolbars, Skype, Contacts, Conversation, Call, View, Tools and Help, allowing the

user simple access.

Stored contacts are easily displayed, relieving the hassle trying to find friends contact details.

Miniature green Skype logo show’s online friends activity, a yellow one to state the user is

online, but currently busy, and a red logo for user’s who are not to be interrupted.

The dial function displays a visual of a mobile key pay, with the 0-9 digits, *, Hash, Save,

Call and SMS contact features for users who want to contact unsaved user’s.

9. Help users recognize, diagnose, and recover from errors

A pop up window is displayed to show error.

Plain simple text, no coding references apart from connection port statuses.

Direct links to Skype’s online support website.

10. Help and documentation

Help feature in toolbar re-directs the user to https://support.skype.com/en/ where the users can

brows a selection of Frequently Asked Questions, and various How-to Guides.

FAQ’s and How-to Guide are designed with the beginner Skype user in mind and are

presented with a mixture of basic short and sweet sentences mixed with screenshots to aid

their users both mentally and aesthetically.

The use of these visuals aid the users in a more direct manner, by showing what each action

taken should look like on their own screen.

Cooperative Evaluation of Prototype (Task List)

The Task List will be broken down into three stages;

1) Setting up the Equipment and Environment.

2) Performing the Task’s.

3) Results and Evaluation of the completed task.

1) The set up of our task will be a very basic, but effective one. All that is needed of our user is a

laptop, microphone, a Screen Capture tool (ViewletCam), a USB for backing up our project,

and the Skype software with an already set up account.

2) The only approach the user need’s to keep in mind is the “Think Aloud” method we have

decided to use. With this method, all the user is asked to do is give a running commentary on

any actions they are required to do, and how they go about pursuing it, therefore allowing us

to have an audio recording of their evaluation.

3) Once the user has finished up their experience using the Skype program, we will then be able

to take their audio and visual recordings away for review, and see how easily (or confusing)

the user found being able to navigate their way through the program.

Setting up the Task Environment & Equipment

In order to properly record and monitor our user during this evaluation report, we decided upon using

the “Think Aloud” process, which would aid our ability to read into what the users are thinking

throughout the process. With the use of the “Think aloud” process, and a microphone to record, we will

be able to achieve a crystal clear recording of our user’s thoughts on navigating through Skype.

As a pre-task trial, we firstly set up a laptop with a plug in microphone to record ourselves participating

in the task to make sure all goes well before our subject enter’s the room to clear up any possible flaws

he might encounter subjects speech throughout the process as well as recording our actions on the

monitor by using the “ViewletCam” screen capture software.

Eventually we could bring in our user ands aloud him work his way though the program. The screen

capture and audio recording files will then be backed up onto a USB for further evaluation. This set up

would leave us with a step-by-step guide though the mind of the user as his speech and on screen

mouse movements will all have been recorded and stored to allow any time reviewing of our user.

Performing the Evaluation

In order to properly monitor our chosen user we decided to video record him as he attempted to use the

Skype interface. We began by explaining the consent form to our user; we reassured him that he had

full access to all the data and how it would be used. We also informed him that he could leave any time

he liked. After agreeing to the consent form with our user we set up a laptop, turned on the Screen

Capture software, and began recording through the microphone. We also asked him to think aloud as

he worked his way through the program.

As we wished to get a full cooperative evaluation of Skype we decided to begin by getting our user to

open the program and try to call a friend. We watched as he clicked on the Skype symbol on the

desktop and the program opened. Our user was confronted with a log in screen where he entered the

username and password we had already set up in order to gain access to the Skype account. Once this

information was entered the program started up. Upon opening the program the user was presented

with tips from the program. For proper evaluation we asked him not to use these tips and so he closed

the window and continued with the evaluation.

We watched and listened as our user told us how he was scrolling down through the friends list looking

for the one we had asked him to call. Upon arriving on the chosen friend our user double clicked the

name opening the friends Skype page. From this window our user could see all the options available to

him, which he made clear by listing out what he could see on screen.

Our user then clicked on the green phone or ‘call’ button on the chosen friends page. Skype then took

over and began trying to connect the call. Our user told use he knew the call was in action as Skype

gave him a dial tone as he awaited an answer from the other end. The call was then connected and the

chosen friend appeared on screen.

However, our users video feed was not being sent to the friend and so we asked him to try to figure this

problem out, aloud. He scanned over the options available on the call screen until he came upon a

video camera symbol. Upon clicking it his video feed appeared on screen. He mentioned he could tell

the feed was being sent by the reaction of his friend on screen when the video appeared.

We then asked him if he could send an instant message to the friend whilst in the call. This posed

slightly more difficulty than expected as our user decided to scroll through his friends list again and

attempted to send a message this way. However he quickly discovered his mistake and returned to the

video call screen, scanning the options there for a message icon. He announced he found what he called

a ‘speech bubble’ icon. Upon clicking this a small chat window opened below the video feed and he

began writing a message, which was sent immediately once the ‘send’ button was clicked. Our user had

found the instant messaging option.

Now the time had come to end the call. We asked our user to say goodbye and hang up. Our user

quickly spotted the red phone symbol and clicked it ending the call. Video feed disappeared and he was

assured that the call was over via an on screen message. Once the call had ended Skype gave our user a

option to review the call for quality purposes. He went to ignore it but for the purposes of this

evaluation we asked him to do it. He was given about 5 questions about call clarity and video quality,

which he could rate from 1-5 depending on the standard he felt. Upon completing this he sent the

feedback to Skype and the window closed returning him to the same friends page he had been on prior

to starting the video call. Our user then logged out of his profile and closed the application, completing

his part in our evaluation. We thanked him for his time and patience and he left us to continue our

report.

Results from Evaluation

Once the tasks set out had been completed by our user we were left with a lot of information regarding

this use of the Skype program. We felt using “ViewletCam” to record the users mouse movements

along with recording the “out-loud thoughts” gave a greater understanding of how the user was

working his way through the program. Therefore, we could easily notice any mistakes made, or wrong

paths taken and fully understand why or how the user ended up in any particular situation. From this

evaluation we have found that the basic uses’ of Skype are quite simple to understand, however, certain

aspects of the program need re-designing in order to avoid potential problems such as sending an SMS

rather than instant messages and making some options more easily visible from the video call screen as

to avoid confusion for the novice user.

Design recommendations

There are many possible re-design recommendations that can be made to improve the Skype program.

User Interface design is an iterative process that never stops, so therefore the program can undergo

numerous re-designs.

Skype could introduce a form of parental control within the program. There are options to block people

from contacting the user, but there are no controls in place as to whom a child could contact. This

would increase the safety and security of the user greatly.

There are possible issues with the program when it is integrated with the toolbar of the operating

system. An option to turn off the flashing notifications during an active conversation could be

provided. As it is, it is only possible to turn them off when initial contact is made.

Another issue found with the toolbar and Skype, is that conversations are shown as different windows.

This could be integrated so that only one icon for Skype appears rather than the multiplying effect that

can be seen in the following image,

The location of the icon that changes text conversation into SMS in the conversation window should be

changed to a different location. This is because it is far too easy, when using emoticons, to scroll over

the SMS icon on the way back to the text box, and end up sending an SMS.

Skype could create an option whereby it is possible to see who is viewing your profile. This would help

with the user’s privacy and they may then be able to block contacts. Or perhaps create an option where

only people in a certain country or area of the world can add you to their contacts.

The group chat function is very popular amongst the Skype community. However, if one member of

the group is not online they are not updated with the rest of the conversation. This means the user may

miss an important part to the discussion. If the messages would still be sent to the offline user, and they

could receive them the next time they log in, nothing will be missed out on.

Conclusions

Product Analysis

Upon evaluating Skype. We found the software quite simple to use and user friendly. The aesthetics of

the user interface are very basic and direct, as we discovered throughout our evaluation, our users had

no major difficulties navigating themselves around the layout. However certain buttons and options

were not made easily visible for the user. This small problem could be easily fixed with the addition of

text underneath the symbols so as to avoid confusion.

Overall Summary

Overall, the main flaws we found whilst evaluating Skype were, the layout of buttons during user

interaction, and the representation of icons which confused our user as to which icon meant Instant

Message or send an SMS, as well as smaller issues such as video feed lagging during calls, and voice

clarity being lost due to poor connection.

However outside of the video call window the general layout prior to calling someone is very simplistic

and understandable for both the novice and advanced users.

“Think-Aloud” Evaluation Consent Form

I hereby acknowledge my free and willing participation in a “think-aloud” project designed to uncover

flaws on the User Interface of Skype .

I understand that _______________________________________ will take notes and may record my

speech as I “think aloud” while navigating through the program I have agreed to review, and that

she/he will use this recording only to evaluate and improve the material I review.

I also understand (1) that no one except __________________________________________ will listen

to my recording, and (2) that any use of the information contained in the recording will not reveal my

name or other information that could identify me.

Name of Participant: _______________________________________________

Participant’s Signature: ______________________________________________

Date: ________________

(https://sites.google.com/a/ceunh.unh.edu/writing-editing-for-the-living-web/Do-you-need-and-editor-

/Think-aloud-testing/Think-alouds/think-aloud-consent-form)