Web E’s goal is for you to understand how to create an initial interaction design and how to evaluate that design by studying a sample. Web F’s goal is for you to create screens for three specific tasks your user will do from your GUI Project – then start to prepare the material for a usability evaluation of your initial design.

Web F ( for class 10/16). Remember:

You are also working in Web F towards creating a usability evaluation for the your paper prototyped designed screens.

In Web E you saw a usability report sample. It had many of the items you need to prepare now.

Both Web E report & Web F report are due Sunday, 10/19 at midnight in D2L. All paper prototypes are due in class on Tuesday, 10/21

The Challenge of creating a strong ULAB evaluationEvaluate a prototype of an early design (the system is at the beginning of the design = conceptual). The scenarios for the tasks are not fully developer nor fully functional – so this is a challenge for the user who wants to “play” with every thing on the screen. Get the user’s cooperation

Part X =the plan and preparation materials for an evaluation

Set an usability objective and goals. Deciding on the user tasks. Create the Pre-Questionnaire

1. Assurance that you have representative users.2. Obtaining the specific User Profile = characteristics.

Decide on recorded Events & Baseline Criteria

1. What events will you log from the users experience.2. What formula (baseline) will decide that you redo the interface.

Create the Post-Questionnaire - Subjective Data - Perceptions

Part Y= Evaluation with Users - observation Preparing the Ulab & performing the evaluations Part Z =Evaluating the results of the orientation Use the results and give recommendations

Let’s think of the evaluation in 3 parts =X, Y, Z.

You will learn and understand all parts of X, Y, and Z.

But you will only prepare your usability evaluation – you will not conduct your evaluation till next week….

so you will do only part X to hand in.

In this WEB F:

Before you do any testing, you should take time to figure out what you're testing and what you're not. In other words, determine an objective for your test that focuses on a specific aspect of the product. By limiting the scope of the test, you're more likely to get information that helps you solve a specific problem.

Give the target user group and the usability objective for this evaluation and the goal for the prototype. Remember slide 4 of the sample.

Part X -Set an objective and goals.

Your test participant will work through specific tasks. These tasks should be real tasks that you expect most users will do when they use your product. The entire user observation should not run over 20 minutes, so you should design tasks that focus on the part of the product you're studying. For example, if you want to know whether your menus are useful, you could design a task that requires the participant to access the menus frequently. After you determine which tasks to use, write them out as short, simple instructions.

The tasks the user will attempt to do need to be printed on separate pages.

Important: Your instructions must be clear and complete, but they should not explain how to do things you're trying to test. For example, if you want to find out whether users can navigate through your program easily, don't give them instructions for navigation. Or, if you want to know whether your interface is self-explanatory, don't describe how it works. This concept is extremely important to remember. If you teach your participants about something you're trying to test, your data will not be useful.

Part X- Deciding on the tasks.

Create a questionnaire that will verify that you have a true user. Also include small profile questions of your user. The Briefer/Debriefer will do this at the start of the evaluation next week.

Part X -Pre-Questionnaire -Assurance that you have representative users. 

a. Decide on the events (metrics) that you will log.

List the events to be recorded during the evaluation. Give which are quantified and which are qualitative.

b. Decide on the Baseline criteria for the evaluation. –

Decide on a formula that contains the events (metrics).

Give this formula on the Report for Web F. A sample of this is in this presentation.

Part X -What events will you log for the users experience. –

Prepare the computer to the “Opening scene” for each user by having the same opening scene

Test the recording video and audio to insure you will have hard data that you can review later.

Practice your conversation with the user to obtain their initial attitude and their pre-questionnaire data.

Explain and let user rehearse the “thinking out loud”.

Give the task handouts to the user and let them do the tasks alone.

Part Y –Evaluation of Users (To do next week)

List the data logged and compile it into lists of pros & cons.

State whether the Baseline criteria was met.

Give recommendations- which are the areas of concerns – not the solutions!.

Part Z -Use the results and give recommendations (to be done after next week)

Additional Sample of a small usability evaluation I’ve chosen the Barnes and Noble

example for my sample for you. www.barnesandnoble.com

Here are my evaluation Objectives

Scenario 1 – Evaluate the user’s perception of the home page. Ask the user: About their first impressions of the site Which link they would use to find specific information What kind of information they would be able to find by clicking the

linksScenario 2 – Want to see the consistency of the site by the

screens visited when the user searches for a cookbook. Ask the user: To search for a cookbook To determine whether the cookbook meets your requirements Are they having difficulty determining which cookbooks meet their

requirementsScenario 3- Want to evaluate the affordance of the site with the

purchasing metaphors. Task: Purchase the cookbook. Ask the user:

To add the cookbook that they have chosen to their cartGo through the purchase process Is there anything they dislike about the purchasing process

Here is my simple Pre-Evaluation Questionnaire

◦ Do you use the web? If yes, how often? If no, thanks so much for coming…..

◦ Do you shop for records, books, gifts, etc. on the web? If yes, what items have you purchased? If no, would you consider ever doing it in the future?

◦ Have you ever purchased a book from a website? If yes, which website? If no, would you consider ever doing it in the future?

◦ Have you ever been to the Barnes and Noble website?

Usability Evaluation ScenariosYour mother’s birthday is coming soon and she has

started to be very interested in food and the nutritional information about the meals she cooks. You have decided to look at different cookbooks to see if they have that kind of information and then buy one for your mother.

You have decided that the Barnes and Noble website would be able to help you in finding a cookbook for your mother that has nutritional information including calorie and protein counts.

Speak out loud and describe all that you see and do.

Task 1 – Go to the Barnes and Noble website at www.barnesandnoble.com. Without doing any mouse travel or mouse clicks:

What are your first impressions about the site? Answer out loud.

What links are available? Answer out loud. What information would you find in those links? Answer out

loud.Speak out loud and describe all that you see and do.

When you are finish with Task 1, please call the help desk.

Task 2 – Now you will look for the cookbook for your mother.Speak out loud and describe all that you see

and do. Search for the cookbook.When you see a possible cookbook - explain if the

cookbook meets your criteria.How do you know what cookbooks meet you

criteria?

Are you having trouble? Explain Are you successful? ExplainSpeak out loud and describe all that you see

and do.

When you are found the cookbook for your mother, please call the help desk.

Task 3 –You decide to purchase the cookbook for your mother.Speak out loud and describe all that

you see and do. Go though the purchase process until it

asked you to provide a credit card number. Was it difficult to purchase the book after

you found it? What items were easy in this task?Speak out loud and describe all that

you see and do.

When you are at the credit card screen, please call the help desk.

Here are my simple Metrics All the team’s usual metrics Smiles, Frowns,

etc. How long it takes the user to figure out how

to search for a book. How long it takes the user to find a book

that meets the requirements. How many different books the user looks at. How long does it take to add the chosen

book to the shopping cart. How long does it take to go through the

checkout process.

Here is a simple Baseline Criteria for 5 users Task 1 Impressions about the site

◦ 4 out of the 5 users Look interested; ◦ All < 3 frowns; ◦ All > 1 smile; ◦ All read screens correctly; ◦ 4 out of the 5 users state that they see something that would take

them where they need to go Task 2 Looking for the book

◦ 4 out of the 5 user’s total Time <3 minutes;◦ all < 5 frowns;◦ All < 15 clicks; ◦ 4 out of the 5 book look at > 3 books◦ 3 out of the 5 user’s finds a book according to their criteria

Task 3 Purchase of book◦ All user’s total Time<4 minutes;◦ 4 out of the 5 users < 8 frowns;◦ 3 out of the 5 users state they understand all of the screens; ◦ 4 out of the 5 users fill out screen successfully;◦ All users state identifiers to >7 screen items associated to purchasing.

Usability Evaluation Checklist Have you prepared the following for your GUI Project:

Identify the intended user GROUP for your evaluation. Identify the User GOALs for the three specific task of your GUI

project. Give the Hierarchical Task Diagram with your three User Tasks. Make

sure these tasks are prototyped completely in your paper system, i.e. don’t ask them to register for class if you don’t have the screen that would do that!

Performing a User Analysis for your three specific tasks - Give the storyboard of interaction

Give usability goals and concerns for your evaluation Developing two screening questionnaire for the user=PRE ? and

POST ? (make sure the Post obtains subjective data from the user). Creating the 3 task scenarios (make sure you have a setting to your

tasks) that you will hand out to your user. Give all the data to be collected. Determine the quantitative and

qualitative measures. Develop Usability Baseline Criteria – you will have two users for your

evalaution.