"Think-Aloud" Usability TestNetSmartz.org

Inquiry Topic: Child Online Safety & Parental Measures

Presented by:

Dixie Davis

Laurie Miller

Mark Pella

Francis Tan

EME 6635: Inquiry & Measurement

Dr. Vanessa Dennen

Spring 2008

Increasing growth of children (ages 5-17) using the

Internet for school and entertainment.

Problem: Greater access to the Internet has exposed

children to objectionable & dangerous content

Inquiry Background

(pornography, predators, violence, identity theft).

Question: What are parents doing to keep their children

safe when they are using the Internet at home? How

effective are these measures?

Usability Testing Focus

• 70% of parents confident in

ability to gather Internet

safety information

• Assess parental ability to

I know where to go to get information regarding

Internet safety

St rongly A g ree3 2 %

St rongly D isag ree0 %

D isag ree12 %

U ndecid ed8 %

• Assess parental ability to

locate materials concerning

online safety for children

using the Internet.

• Resource Website Focus:

www.netsmartz.org

Ag ree4 8 %

Protocol ScriptWelcome:"Thank you for participating in today’s usability test. This is an exciting study that we are conducting to

research child safety online. This usability test helps assess parental ability in locating online resources that

contain online safety materials. It should not take long to complete. We are testing the site, not you. We are

conducting this as a “think aloud” protocol. You will speak your thoughts aloud while completing the task. I

will observe and take notes."

Agenda:"There are a few steps to today’s usability test. First, I would like to gather your demographic background

(used survey questions) Second, you will be asked to complete three separate, but related, tasks using a web

browser. You will discuss aloud your thought process as you work in completing the task objective."

Entrance Questions:"Do you have any questions that need clarification from me? I want to ensure your comfort and

understanding of your role in the study. I want to make this a rewarding experience for you, so do not hesitate

to ask if you need anything."

Exit Questions:"Do you have any questions for me? Any general comments regarding the site? Regarding this test? Thank

you for your time & willingness to participate. You have made a valuable contribution to our efforts."

Protocol TasksTask 1: Using a search engine

Protocol TasksTask 2: Locate & Register for NetSmartz Bulletin

Protocol TasksTask 3: Locate & Download Safety Pledge

Test Findings

“Are you kidding me? HaHa.

Where do I even begin?”

“This looks like a crayon box

threw-up.”threw-up.”

– Test Subject A

Test Subjects

• Three females, one male.

• Ages ranged from 34 (youngest) to 41 (oldest).

• Three with post graduate degrees, one a high school • Three with post graduate degrees, one a high school

graduate.

• Two with IT backgrounds.

Success Rate: 100%

• All four subjects able to complete the three tasks.

• On average, testers spent nearly 6 minutes completing

tasks. They spent the most time on Task 2: Finding

Bulletin.Bulletin.

• All had initial difficulty finding either the bulletin or the

safety pledges.

• All initially had difficulty

navigating the site.

• Quick Links – List of

Popular Destinations only

Ah found it! (Sigh) It is so well hidden!”

– Test Subject B

Popular Destinations only

available from main page.

• Left-hand menu bar not

intuitive.

Conclusions• Test showed parents could use site to sign up for bulletin

and download safety pledges.

• Website could include search box to allow items to be

more easily found. “I don’t know if you’re looking for

suggestions, but I wish they had a key word box.” –suggestions, but I wish they had a key word box.” –

Subject C.

• Quick Links list could be included on all topic pages, not

just main page.

Reflections•Overall the Usability test was more fruitful than many

of us thought.

•Maybe we should have done a hybrid of Retrospective

and Concurrent Think Aloud Protocols instead of solely

Concurrent Think Aloud Protocol.Concurrent Think Aloud Protocol.

•The preparedness level for the administrator of a

Usability Test is similar to that of an Interview.

Lessons Learned•Reiterate the fact the participants should use the

NetSmartz website in the directions for each tasks.

•Be more specific about what pledge we want them to

downloaddownload

•It is difficult to not give a hint when the participant is

frustrated

•Following Think -Aloud protocol is difficult for some

participants.