"Think-Aloud" Usability TestNetSmartz.org

Inquiry Topic: Child Online Safety & Parental Measures

Presented by:Dixie Davis

Laurie MillerMark PellaFrancis Tan

EME 6635: Inquiry & MeasurementDr. Vanessa Dennen

Spring 2008

Increasing growth of children (ages 5-17) using the Internet for school and entertainment.

Problem: Greater access to the Internet has exposed children to objectionable & dangerous content (pornography, predators, violence, identity theft).

Question: What are parents doing to keep their children safe when they are using the Internet at home? How effective are these measures?

Inquiry Background

Usability Testing Focus• 70% of parents confident in

ability to gather Internet safety information

• Assess parental ability to locate materials concerning online safety for children using the Internet.

• Resource Website Focus: www.netsmartz.org

I know where to go to get information regarding Internet safety

Strongly Agree32%

Strongly Disagree0%

Disagree12%

Undecided8%

Agree48%

Protocol ScriptWelcome:"Thank you for participating in today’s usability test. This is an exciting study that we are conducting to research child safety online. This usability test helps assess parental ability in locating online resources that contain online safety materials. It should not take long to complete. We are testing the site, not you. We are conducting this as a “think aloud” protocol. You will speak your thoughts aloud while completing the task. I will observe and take notes."

Agenda:"There are a few steps to today’s usability test. First, I would like to gather your demographic background (used survey questions) Second, you will be asked to complete three separate, but related, tasks using a web browser. You will discuss aloud your thought process as you work in completing the task objective."

Entrance Questions:"Do you have any questions that need clarification from me? I want to ensure your comfort and understanding of your role in the study. I want to make this a rewarding experience for you, so do not hesitate to ask if you need anything."

Exit Questions:"Do you have any questions for me? Any general comments regarding the site? Regarding this test? Thank you for your time & willingness to participate. You have made a valuable contribution to our efforts."

Protocol TasksTask 1: Using a search engine

Protocol TasksTask 2: Locate & Register for NetSmartz Bulletin

Protocol TasksTask 3: Locate & Download Safety Pledge

Test Findings

“Are you kidding me? HaHa. Where do I even begin?”

“This looks like a crayon box threw-up.”

– Test Subject A

Test Subjects• Three females, one male.

• Ages ranged from 34 (youngest) to 41 (oldest).

• Three with post graduate degrees, one a high school graduate.

• Two with IT backgrounds.

Success Rate: 100%• All four subjects able to complete the three tasks.

• On average, testers spent nearly 6 minutes completing tasks. They spent the most time on Task 2: Finding Bulletin.

• All had initial difficulty finding either the bulletin or the safety pledges.

• All initially had difficulty navigating the site.

• Quick Links – List of Popular Destinations only available from main page.

• Left-hand menu bar not intuitive.

Ah found it! (Sigh) It is so well hidden!” – Test Subject B

Conclusions• Test showed parents could use site to sign up for bulletin

and download safety pledges.

• Website could include search box to allow items to be more easily found. “I don’t know if you’re looking for suggestions, but I wish they had a key word box.” – Subject C.

• Quick Links list could be included on all topic pages, not just main page.

Reflections• Overall the Usability test was more fruitful than many

of us thought.

• Maybe we should have done a hybrid of Retrospective and Concurrent Think Aloud Protocols instead of solely Concurrent Think Aloud Protocol.

• The preparedness level for the administrator of a Usability Test is similar to that of an Interview.

Lessons Learned

• Reiterate the fact the participants should use the NetSmartz website in the directions for each tasks.

• Be more specific about what pledge we want them to download

• It is difficult to not give a hint when the participant is frustrated

• Following Think -Aloud protocol is difficult for some participants.