usability main document

8/3/2019 Usability Main Document

http://slidepdf.com/reader/full/usability-main-document 1/12

What is Usability?

Usability is an assessment of how easy to use an interface for a user. This is a quality attribute which

improves the ease of use during the design process. Usability can be defined 5 quality components:

· Learnability - How easy to complete the tasks when users interact for the first time with the design.

· Efficiency - Once the user gets to know the design, how fast he can complete the task.

· Memorability - How fast a user can re-establish his skills after not playing the game for a period of

time.

· Errors - What kind of errors user make, how severe they are, how easy to recover form it

· Satisfaction - How satisfying is it to use the design (Nielsen,2011)

Usability Testing

Usability testing is a method use to evaluate/asses on a product done by using users. During the test

observers will collect data based on users performance (time takes to complete the task, how theyreact, how many errors they make), watch users and listen. Observer’s main goal is to collect data,

look for usability problems in the design and measure how satisfied the users are. (U.S. Department

of Health & Human Services, 2011)

What gives a product good or bad usability?

When creating a product one of the most important things you need to consider is usability. Having

good usability can make your product a success and bad usability can lead to a bad reputation for

your product. ‘If you give careful consideration to things such as navigation and menus, feature set,

content layout, and come up with a consistent strategy and design for your products, you will end up

with a better end product, no doubt about it.’ Creech, J. (2010).

All of these things will lead to a good product as everything will be clear and easy for your customer

to understand and use, meaning good usability.

To ensure good usability, keep the user in mind. You are designing the product for them, not

yourself.

While the users and designers are similar in some aspects there is still a big difference between the

two. ‘Unfortunately, designers are different from users in several respects, including their general

computer experience (and enthusiasm) and their knowledge of the conceptual foundation of the

design. When you have a deep understanding of the structure of a system, it is normally easy to fit a

small extra piece of detail into the picture and interpret it correctly.’ Nielsen, J (1993) Whilesomething may make sense to the designer, the user does not have the same understanding so put

things as simply as possible without being patronizing.

Some things that give a product bad usability are: too hard to find what you are looking for, bad

layout, confusing menus and even too many options. The content of your product is the most

important thing you have, ‘we need to ensure that content takes pride of place on its’ throne, that

users can find all the information that they are looking for. We shouldn’t have to go in and out of a

million different menus before we find the correct content.’ Creech, J. (2010). To make your

information easy to access keep things in relevant sections/catagories, have a search function and

make sure the most important points stand out.

A poor layout leads to bad usability as something may be distracting away from the focus of the



product or the focus may not stand out against everything else. A study into the effects of poor

layout concluded: ‘This study examined the effects of enhanced layout (headers, indentation, and

figure placement) on reading performance, comprehension, and satisfaction. Participants read text

passages with and without enhanced layout. Results showed that reading speed and comprehension

were not affected by layout, however, participants were more satisfied with the enhanced layout

and reported it to be less fatiguing to read.’ Chaparro, Barbara, Shaikh, A. & Baker, J (2005).

Menus that are included in your product need to be clear and to the point, ‘Navigation must be

simple and consistent. All web navigation must answer these questions: Where am I? Where have I

been? Where can I go next? Where's the Home Page? Common mistakes include different types of

navigation on the same site, a link to the current page on the current page (home page link on home

page), and poorly worded links so the visitor doesn't know where he'll go if he clicks, no links back to

the home page and confusing links to the home page.’ Flanders, V (2010). By using a clear layout

that is consistent people will be able to find what they are looking for quickly and easily. Another

issue that would give a product bad usability is having too many options to choose from. ‘You know

your users/customers/readers best. Figure out what they want, and just give them that. They

honestly don’t need millions of different options.’ Creech, J (2010). Separate information intocategories that are relevant to the point you are trying to make instead of a separate category for

every option you have, group information together that has the same main point and then use

subheading to separate these. This will make it a lot easier to navigate around your product and give

good usability.

What are the outcomes of good/bad usability?

Attractive usability factors meet the unspoken or unexpected needs of the customer. They lead to

high levels of satisfaction. (Timo, 2004)

People form opinions on products after their usability experiences. Those products with goodusability feel natural and intuitive, aiding in the speed in which a user will pick up on the ways they

are required interact and use the product.

Users may form impressions of usability by briefly examining the product. A complex looking user

interface, such as a remote control with 80 or more buttons and tiny text, may give the shopper an

impression that the device is hard to use. Similarly, a simple-looking user interface on a cellular

phone does not mean good usability. (Timo, 2004)

The best control schemes allow fast navigation, usually with easy to understand, memorable icons;

or the fewest and most logical clicks to move through menus. Having good usability allows people to

get things done in a short time span, and reduces the stress that arises from bad, confusing, illogicalor actions that are not obvious.

Bad usability decreases the likelihood of experiencing task based flow because the player has to

sacrifice attention and other cognitive resources to inappropriate activity. (Kiili, 2004)

Sometimes, bad usability can make a product practically unusable, such as a lack of coherent

instructions or goals that aren’t obvious, possibly stopping a user from carrying on their usage of the

product, perhaps being unable to complete the story mode or a level of a game.

For an example of bad usability, In Tony Hawks: Underground, you are required to “skitch” a ride on

a car, which proved to be one of the biggest halt points in the game, one of the places where lots of

people get stuck and stop playing entirely. (Isbister and Schaffer, 2009)



According to player X, “Bad usability breaks the flow experience down because one has to

concentrate on using the system not for example, solving a problem”. Player O compared bad

usability to bad playability. “I have not experienced flow in games whereas playability has been bad.

In order to be able to experience flow, the game should be easy to control, so that the controls are

apparent to me.” These results are consistent with Pilke's (2004) findings indicating that userinterface that requires less cognitive processing facilitates flow experience. (Kiili, 2005)

The products with good usability give users an enjoyable experience that they don’t mind coming

back to since they understand what needs to be done without needing to concentrate too hard.

Products with bad usability create stress and frustration, ultimately ending with the loss of

customers, and at the very least, an overall experience that is less satisfactory than one with a

product with good usability.

Usability of other products and gamesAny product that requires human interaction is a product that will require usability testing. For a

product to have good usability it must be extensively tested. For a video game usability is

particularly important (Laitinen, S. (2005). Better Games Through Usability Evaluation and Testing.)

“For one, playing games is voluntary. If the player has to struggle with problems that make playing

less fun than doing something else, then there is nothing to stop the player from switching off the

console.”. Evaluating The Usability of a game could be split into 4 heuristics:

( Fraunhofer IPSI, (n.d). Exploring the Usability of Video Game Heuristics

for Pervasive Game Development in Smart Home Environments [pdf] Available at

www.ipsi.fraunhofer.de/.../pergames2006/.../PG_Roecker_Usability.p...Similar )

Game Play (a set of problems and challenges a user must face to win a game),

Game Story (collection of all plots and characters),

Game Mechanics (structure by which units interact with the environment), and

Game Usability (interface and interaction elements, e.g., mouse, keyboard).

In terms of game play, the player must be able to start and complete tasks not necessarily with ease

but it must be clear that the challenge is possible to be solved. Game mechanics can involve the

player easily being able to identify their goals in a game, The game must react appropriately to the

players actions in a game, Artificial intelligence must be consistent to the players expectations i.e.

Not too simple or seemingly impossible to beat ect. And Game usability meaning the actual

interaction with the game like The graphical interface of the game being clear and accessible, thecontrol system , in for example playing a first person shooter game. Any person with experience

playing first person shooter games would instinctively use the buttons that are used in other First

person shooter game. But say if there was a player that has never played a first person shooter game

then the game control system must be easy enough for the player to grasp in a short amount of time.

Usability is not just limited to video games, any product that requires any human interaction has the

potential to require heuristics into the product. (Kalkkonen, A. Kalio, T. Kekäläinen, A.

Kankainen, A. Cankar, M. (2005). Usability Testing of Mobile Applications: A. Journal Of

Usability Studies. 1 (1), p4-16.) If there are any problems with the usability it can prevent the

product from being completed depending on the severity of the problem. In (Kalkkonen, A. Kalio,

T. Kekäläinen, A. Kankainen, A. Cankar, M. (2005). Usability Testing of Mobile Applications:

A. Journal Of Usability Studies. 1 (1), p4-16.) they tested the usability of a mobile application by

http://www.ipsi.fraunhofer.de/.../pergames2006/.../PG_Roecker_Usability.p...Similar






















giving tasks to the subject to complete within a laboratory and in the field. The test gave 23 different

issues with the completion of the different task with only one being severe. From these findings,

rectification can be done to the product to hopefully eradicate these issues which could arise when

the consumer uses the product.

In comparison to Video Games, general products can require different heuristics to that of a Videogame but for the majority of the time they are very similar. For example, being able to find a

particular menu in the video game is the same as for example finding a buy ticket menu in a ticket

machine but testing whether a person can easily find the money slot in which to pay for the ticket

has no relevance to the usability of a video game.

Dark Souls Usability & Pre-Test

The game we have chosen to test is called Dark Souls. It can be found on Xbox 360 or PS3, though for

our test we shall be using an Xbox 360.

We chose Dark Souls because so many elements of the game have usability issues. I would class

myself as a hardcore gamer with the amount of hours I play games per week, and I even found DarkSouls very confusing and difficult to play.

Tutorial Level:

The first usability issue is in the tutorial section. The tutorial consisted of you interacting with scrolls

on the floor, they were hard to find and they looked like part of the environment. As a consequence

I found that I was consistently walking past important tutorial scrolls, so when I reached an enemy I

was just pressing any button trying to kill them. This usually ended up in me dying and having to

start from a checkpoint further back in the tutorial level.

Dark Souls doesn’t give you an opportunity to understand your basic actions and controls before

challenging you in combat. When the game starts you navigate through a tutorial dungeon withinstructions on how to play the game through use of the scrolls. The game also challenges you

before you have time to learn what you can and can’t do. For example, when you reach the area

where you are supposed to learn to parry and counter attack by pressing ‘LT’ and ‘RT’, the enemy

will kill you if you miss. This is not ideal for the player to early on in the game as you want the player

to gradually become more viable in the game. I also found that by being challenged too early on in

the tutorial, players will be less inclined to use these techniques in more challenging moments in the

game. As a game designer you want players to understand your game mechanics and exercise them.

(Solis [Gamertalk], 2011).

At the end of the tutorial you also come across a boss enemy that you have to kill. This is the point

that I couldn’t get past without cheating because if you take one or two hits from him then you are

dead. Dying here sends you back to a halfway point in the tutorial instead of right back to the startof the boss level, which is also very frustrating.

Open World:

The open world environment is impressive, though not knowing what way to go and being able to

run to a section near the end of the game straight away isn’t ideal. This definitely is another usability

issue. In other similar games such as ‘Elder Scrolls: Skyrim’ you have places pinpointed on a radar

map on your HUD. Why they couldn’t do this in Dark Souls I do not know, as it would be so much

easier to know where you have to go. Another example is any ‘Call of Duty’ game, were if you kill all

the enemies in a certain area you can continue, and you know you’re going the right way if you

come across more enemies. If you go back and see that theres nothing appearing then you know



you’reas in Dark Souls going the wrong way. This is a typical basic usability interaction that is used in

most games and that is effective, where the enemies respawn if you die and they are scattered

everywhere. This just adds to the confusion and countless number of usability issues that this game

has.

Rather than structured missions or waypoints dictating where to go, you're forced to feel your way

through the proverbial dark with no map and only a vague goal to guide you. Instead of instigating agentle learning curve, Dark Souls's outlandish difficulty punishes you right from the start. This is a

game that wants you to fail. (Matulef [G4TV], 2011)

There's no central hub, no safe haven for you to run back to and recuperate. Instead, there are

bonfires strategically placed around the world. Bonfires are your checkpoints, the place where you

can hunker down to replenish your health flasks, spend the souls of vanquished enemies on leveling

up, repair your equipment, and meditate on your doomed existence. Resting at a bonfire ensures

that you'll spawn there the next time you're dispatched, but resting also respawns all the enemies in

an area. (MacDonald [IGN], 2011)

Controls & Navigation

I found that the control system has its ups and downs. There is a lot of complex moves for you to

learn that can help you a great deal when taking out enemies quickly and efficiently, but I can

imagine an average/casual gamer to struggle with these controls.

I would also say the same about the navigation. I have previously played ‘Elder Scrolls: Skyrim’ so I

guessed the layout would be pretty much similar to navigate through the menus and to equip/wield

weapons and armour. Anyone who hasn’t played either game, may take time to get to grips with this

menu and the items within it.

Overall

This game was obviously made to test hardcore gamers abilities. By doing this they have limited theaudience that the game will sell to. People will get frustrated and die multiple times, even within the

tutorial level. Some may argue that it is due to bad game design, though this debate could tilt either

way.

Evaluation based on your analysis and usability research

In conclusion Dark Souls, after analysing the game and researching the usability of the game for a

new user, i have found as stated before this game is defiantly more suited to a advanced gamer or

hardcore. Due to the generally difficulty of the game and the gaming experience and ability to be

able to progress and complete the game fully. As i found in the HUD (heads up display) there is no

min map or markers which creates a condrum as the player has no indication where to go or any

objectives to complete as there is no information, and its very easy to end up at the end of the game

when the player hasnt got any equipment to even progress in that area.

The game in general from the start can be compared to learning to run before you can walk, or

getting thrown into the deep end before you can swim. The tutorial throws you straight in, no

chance for the player to get a chance to practice the controls, they just get thrown into fight afterfight, and then a boss which for even experienced gamers have a lot of difficulties and deaths before



eventually killing him. However to also add to this with every death, all the enemies respawn.

Dark Souls is a game for very advanced users, even though they will have quite a lot of difficulty

completing the game. This broadens the audience, as new users will find it impossible and very

frustrating to complete.

What alternative methods of testing are there?

Time

As a testing method is reliable and is very useful and suited for usability testing, as to gauge how

good the usability of the game is, we can first record the time it takes for a experienced or hardcore

gamer to complete the test we have set out. Then we can use the time we have recorded anddouble it, to be a reliable time for a new users playing the game to measure the usability of the

game in the tutorial part of the game.

However in some instances some users may easily go over the time limit due to game difficultly, not

understanding the controls, or lastly not fully reading the tutorial scrolls properly and understand

their meaning of which the scroll is telling the user to carry out a certain order or enlighten the user

with information.

By recording these results using time as the testing method for all the users, the information gained

would be very reliable as it is easy to compare against the other users.

So by timing the user playing through the test, by taking notes in a time based diary, indicating at

specific times what the using was doing, were they stuck, having problems. By Time dating these

would give a great results and information on each user and give a wide range of information

orientated for usability.

What They Liked/Disliked/Found Confusing

This could be more of an experimental testing form, as a whole it does not really give reliable resuls

for usability as people who are testing the game Dark Souls, might not be there favourite genre of

game already disrupting results. Although if the tester finds the game confusing, this can point to

bad usability in the tutorial, where the user has no clue what to do, maybe vague instructions on thetutorial scrolls which to new users would not understand.

Within the tutorial stage of the game you have to defeat a boss, which for even some experienced

gamers is difficult. So for new users they might dislike the general difficultly of the game.

As a whole using these types of testing would not be useful data as stated before many of our

testers might not like the game. This would not really give an indication of the usability, more on the

side of the game visually or difficultly. As recording this data correctly would have to be implication

into using time another testing method, to get reliable and fair results. To record at which points

they liked, dislike etc, to get fair results.

Functionality



We can test usability of Dark Souls on functionality in regards to, menu navigation, world navigation

and the general controls of the game. As a lack of information within the tutorial would indicate bad

functionality and usability, as it doesn't clearly clarify what the user has to do or how to navigate in

the game, interact and main point play the game. The tutorial scrolls within the game do provide the

user with the buttons to play the game and also the more advanced controls for example to parryor block and strike at the same time, by having the shield up. The scrolls however are very vague

in regards to the whole menu setup and actually how to equip a weapon, which some users found

confusing and took some time to figure out, where as with the correct information from the scrolls

in regards to navigating the menu and then how to equip the weapon and then how to draw it

would be better.

Conclusion

In Conclusion of the alternate types of testing we can use, i believe Time as a testing method is the

more appropriate and generate very reliable results in comparison with the other types of testing.

By timing the users and taking notes at specific times, we can easily compare to the other users to

see if there is a similarity or difference which can be easily noted. Overall gives very reliable andfair results and can identify where in the game or how far in on Dark Souls Tutorial where there are

usability issues.

Participants

Andrew Collins (#1)Paul Jones(#2)

Lee Wilson (#3)

Kylie Williams (#4)

Matt Downes (#5)

Steven Brown (#6)

George Pepper (#7)

Robert Wells (#8)

James Spencer (#9)

Lincoln Chadwick (#10)

Equipment & Environment



Xbox 360

Controller

Dark Souls

Camera/Webcam

TV MonitorPC

Speakers

Questionnaires

Pens

Chair

The test will be conducted in the student halls within a student’s room. In a relaxed environment

so our results will not be skewed due to any pressure the participant may feel. We will also have a

waiting room that the participants can play other games in beforehand so they don’t get bored and

give the test a very lame attempt.

Process

● First the participant will be informed about the test, what type of data will be collected

then the participant will sign the consent form giving the permission to video record the

Participant and collect all the necessary data.

● Then the profile and the entry questionnaire will be given to the participant.

● Then the participant will get to play the given level.

● Participant will be recorded on a camera for late use of the observers mean while any

significant incidents or comments made by the participant.

● After the participant complete the game an exit questionnaire will be given to the

participant.

Data Collection Method

● Questionnaires before and after the game play.

● A camera to record visual and audio data while participants playing the game.

● Written data from observations will be taken during the game play.



As a result of our testing we found a

numerous amount of usability issues



within the game.

We found that most people didn't have a problems with the basic control system at the start of the

demo. Everyone knew how too navigate and most people extended their knowledge of the control

system by reading the majority of the scrolls. What was interesting is how

each person had their own way of learning the control system. Some of the subjects relied

completely on the scrolls to figure out the control system. But some subjects analyzed the buttonsand experimented with them to figure out the control system on their own and some used a blend

of both. We found that the subjects who had figured out the controls from the start progressed

quicker and with more ease than the ones who just relied on the scrolls.

There was very few people who read all the scrolls, a lot of the subjects just plain missed them as

they are not easy to see against the environment, the only way to locate them easily is stepping over

them and seeing the prompt button to read the scroll.

A big issue that every subject had difficulty with was facing the boss for the first time. Every person

attempted to fight the boss when the player isn’t meant to fight the boss this time round at all. In

the first boss sequence you have to run to a door that opens as soon as the boss appears. Due to the

lack of weapons it would be very hard to fight the boss at this point. It is evident through our resultsthat this is not an obvious goal as it took our subjects an average of 3-4 attempts to finally realise

that there was a open door you could go through. There is a scroll that you can read in the room that

tells you to run but not everyone found and read it and even the subjects that had found it still had

no idea about the door. The way the game portrayed this less than obvious solution to fighting the

boss is by showing two poorly lit torches in the distance and a well hidden scroll saying ‘get away!’.

This is definitely a usability issue, possibly to do with lighting or just bad design in where they have

made a simple game-play puzzle seem very hard to figure out.

The next part of the tutorial level is aimed at teaching you how to use your menu system to equip

other weapons and equipment which again is taught to you by using scrolls. The obstacle is an

enemy with a bow that you cant defend yourself against without a shield. A lot of the subjectsfigured out that you couldn't get past this enemy at the current state. Most of the participants found

the required equipment for the next step and the scrolls instructing you how to use them. The issue

here was the menu system. A majority of the participants had difficulty using the menu system in all

aspects such as navigation and button pressing etc. With most of them it took a minute to figure out

how to equip weapons and such, which is a vital part to completing the tutorial level. One of our

subjects didn't figure out how to equip the weapons and progressed through the level with a lot of

difficulty until she then figured out how to equip the weapons and then progressed through the

level with a lot more ease. At this point you also receive a bow. Nearly all of our subjects clearly

shown that they were having difficulty with using the bow in game and we found that no one used

the bow extensively, if not at all in the game. In the recordings a lot of the subjects looked at their

controller during the use of the bow with confusion and frustration.

On the final boss level every subject had difficulty with defeating it. Out of all of the subjects only

one succeeded in defeating the boss and actually completed the tutorial level. We think this is due

to the difficulty of the boss characters AI being too difficult and is simply not suited for someone

who has just started playing the game

Conclusion

As we expected from our original analysis of the game is that Dark Souls has a mixture of; usability

issues, bad game design and elements that aim only for hardcore gamers. From our results we can

see that the more ‘casual’ gamers struggled a lot with understanding key elements of the game such

as; the in game menus, control system, puzzle solving and being able to kill enemies. These elements



are key to progressing in the game, and should of been resolved during any large scale beta testing

that the game went through before being released.

By keeping the game this way, Dark Souls has a very limited target audience. No one wants to

spends hours upon hours dying in the same spot in a game. The games designer needs to give the

player the feeling that they are progressing or will progress, with Dark Souls this is not the case.

How do your results compare to your initial evaluation?

In our initial evaluation we stated that we believed the game was more suited for advanced/

hardcore gamers due to the difficulty of the game and how quickly it advances for a beginner of

the game. Our results prove this statement to be true . In our graphs it shows that only 3 out of 10

people actually completed the tutorial level with an average rate of 7 deaths between each person.

We also found that the subjects gave an average of 6 out of 10 for ease of use of the controls, which

kind of challenges our view of the control system slightly from very confusing to being able to cope

with which we found surprising. Again the results were similar to our evaluation with the menusystem, in the questionnaire the subjects gave an average of 3 out of 10 of the ease of use of the

menu system. Overall our predictions were very similar to our results.

What worked well in your test? What could you have improved?

During our test we think there were a few things that worked well. We conducted the test within

Orlando Village Halls where we live as we thought this would be a more relaxed environment and

would add stress to the player which could cause them to panic about how they’re doing and make

mistakes that they wouldn’t if they were calm and in the normal environment that they are used to

playing in.

Another thing we think worked well was that we kept each test the same, so there was nothing that

could have compromised the results that we collected, by keeping everything the same we kept the

test fair and kept the challenges the same for everyone.

Another thing we felt we did well was the test plan; we knew who was being tested, what they

were being tested on and how to record what was happening. We recorded the person playing the

game to see reactions and could also see how they were handling the controls at the same time, we

kept other participants separate from the test so that they wouldn’t see the game before hand and

have an advantage over anybody else and we also wrote notes on what happened throughout each

person’s test so we had a visual and a written entry for how people coped during the test so our

results would be very clear.

Although our work was always completed for the time needed, our organisation and time

management could have used some improvement. Our questionnaires were created with very

little time to spare before the actual testing began. At the beginning of the assignment we assigned

tasks to everybody, we could have been arranged who was doing what for all of the tasks instead

of completing one task then looking for another. By doing this it would’ve made things easier but it

didn’t necessarily hinder our work.

Overall we think our test was a success as we managed to complete everything and got the results

we wanted that proved our theory about the games usability.

usability main document

Documents