usability report finalsga72/docs/iscreen-usability... · 2009-03-09 · info 610 george abraham...
TRANSCRIPT
INFO 610 George Abraham & Christine Wania
1
I-Screen: USABILITY REPORT
Introduction
Context
The College of IST recently installed an interactive kiosk called iScreen, designed to
serve as an information resource for student/visitors to the College of IST. The
content is designed to answer some of the common questions students/visitors might
have, for example: faculty office locations, building hours etc.
The hardware comprises of a touch-screen capable 42” Plasma monitor, featuring a
walk-up-and-use interface. The interface needs to support novice as well as repeat
users in terms of content and accessibility.
Problem statement
Ethnographic study of prospective and actual users revealed system/hardware-
specific, system environment-specific and interface-specific problems that hinder
usability of the iScreen (refer Portfolio Pt.1). In this report, we focus on the study of
interface-specific problems.
Goal
Our goal is to research the information seeking behavior and the information needs
of students and visitors, so as to customize content on the iScreen. We try and
determine a way to present this information to aid effective use.
Approach
Based on the ethnographic findings related to the interface, we conducted semi-
structured interviews with prospective and current users. In parallel, we designed
and performed usability tests followed by a debriefing to understand and target the
usability issues faced by the users. The following figure presents an overview of our
study design.
Fig.1 Study design overview.
The study results were analyzed quantitatively and interpreted qualitatively to
generate recommendations for improving the usability of the iScreen. The following
sections describe in detail the product under study, research method utilized, and
the implications of our findings for (re)design.
Ethnographic observations
Interview design
Usability tasks Questionnaire
Findings and design
implications
INFO 610 George Abraham & Christine Wania
2
Product description
Description of the environment
The iScreen is located in the lobby of the College of IST, Rush Building. The
environment is the lobby of the Rush Building. The lobby has several conversation
spaces and seating where students meet to socialize, eat and/or work. The lobby
area is the first space any building visitor enters. There is little on the walls except
for a bulletin board and the interactive iScreen.
User population its intended for
The current content is targeted at common queries students and visitors at the
College of IST might have, like for e.g. Directory assistance, faculty office location,
classroom location etc. The interface and content needs to support both novice and
repeat users. It has been designed as a walk-up-and-use kind of interface and
should require minimal or no training. The information resource on the iScreen is
presently organized in the following manner:
Main (Touch screen)
• Directory
o Faculty
o Staff
o Main
• Building info
o Hours of operation
o Telephone numbers
o Main
• IST calendar
o IST events
o Academic calendar
� Graduate Fall schedule
o Main
• Course schedule
o Fall courses
o Main
• Academic FAQ
o Undergraduate
o Graduate
� More questions
o Main
Hardware and behavior
The hardware comprises of a 42” Panasonic plasma screen mounted 4 ft. above floor
level on the wall. In its inert state the iScreen displays a number of different screens
in a cyclical manner. One screen simply displays the Drexel University logo, followed
by two screens displaying pictures of past events at the College of IST. The
remaining few screens are advertisements for the College of IST degree programs.
Each screen image is displayed for a set amount of time (approximately around 5
seconds, this was not measured) and then the next screen is displayed. In the
bottom left-hand corner of each of these screens is text that reads “Touch Screen to
Begin.” This text is relatively small and very difficult to read unless you are standing
right in front of the iScreen within a short distance (5 feet).
INFO 610 George Abraham & Christine Wania
3
Study design
Participant profile
Number of participants
The aim of our study was not to achieve statistical significance in the quantitative
sense. Rather, we aimed at ensuring that we had at least 4-6 representative users
for our interview and representative 4-6 users for our usability tests (Mayhew,
1999). The participants were selected based on the target population for the
system/product, primarily students and visitors.
Segmentation
We identified three types of student users and one visitor category. The three types
of students belonged to the Bachelors, Masters, and Doctoral program. We did not
differentiate between students within each level of study. It can be argued that a
senior would have different information requirements as compared to a freshman or
a sophomore. In an ideal condition this would be considered, but our study focus was
at a higher level. During the interview and debriefing, we recorded responses
regarding information requirements specific to the user-group to account for these
finer grained differences.
Key characteristics and capabilities
In total there were eight participants for our study, four for the interview and four for
the usability test. Each of them fit the generic profile of a target user. This was
important especially for the usability test. We did not want to make participants
perform tasks that would not be part of their routine. The participants needed to be
familiar with abbreviations like INFO 435 etc. that were used commonly to represent
a course name. It was not necessary for the participants to have previous experience
with the interactive screen. If they had previous experience with the iScreen, we
accounted for this during our interview and debriefing by asking a few open-ended
questions to get further insight regarding usage and experience level.
Selection process
In an ideal situation we would have advertised on bulletin boards to recruit
participants for this study after receiving IRB approval. Because of the lack of time to
complete this study we needed to improvise. We would have liked to include one
undergraduate student, one graduate student, and two visitors as participants for
both the interview and the usability study. We did not have much luck finding such
participants, therefore we settled for whoever agreed to volunteer. An overview of
the participants in our study can be seen in Table 1. The users that participated in
the interview were not the same as those that participated in the usability testing.
Participant Type Interview Usability Testing
Undergraduate Student 1
Graduate Student 3 3
Visitor 1
Table 1: Participant details for interview and usability testing
INFO 610 George Abraham & Christine Wania
4
Interviews
Purpose
The purpose of the interview was to learn about the general information seeking
behaviors of students in the College of IST. In addition we were looking to find out
what students know about the interactive iScreen located in the lobby of the College
of IST.
Questions posed
The interview contained a number of questions related to students’ information
seeking behaviors and their knowledge of the interactive iScreen. We also asked
questions based on the information presented in the iScreen. A copy of the interview
questions can be seen in Appendix A. As we saw fit throughout the interviews we
asked related questions that were not included in the original question list. The
interview protocol was to elicit responses regarding:
• Profile of the user- What year of study and program enrolled for?
• Previous experience- Have you used the iScreen before?
• Information resources known to them- Like for e.g. College website etc.
• Information seeking behavior- Who to ask/ where to go.
• Information needs- Common information requirements
• Preferred information format- How should content be presented?
• Suggested improvements
Question selection
The interview was used as mini-requirements gathering process to facilitate better
usability test design, and also to inform us regarding information that should be
represented in the content but not there yet. This also helps us in deciding what
information needs to be prioritized on the interactive screen.
Test environment
It was not necessary to conduct a contextual interview because the questions were
more abstract in nature, and the primary focus was on information behavior. Test
We audio taped the interviews and transcribed the audio. The participants were
informed that their identities would be kept anonymous unless specified otherwise.
Also, no permanent audio record was maintained once it was transcribed.
Key findings
Interview data
As mentioned earlier, the focus of our interviews with participants was to determine
the users’ information seeking behavior, information requirements, iScreen
functionality awareness and improvements to the interface. In this section, we
present our findings based on qualitative interpretation of the participant responses.
We have considered the fact there might be some bias in the responses based on the
fact that the participant might have used the iScreen before. Among the four
participants we interviewed, two of them had used it in the past, and two of them
had no idea that the iScreen was interactive.
1. Information seeking behavior
INFO 610 George Abraham & Christine Wania
5
All the participants tended to differentiate between information related to their
program of study and general inquiries. In all the cases, they confirmed that they
knew whom to talk to regarding program related questions, in this case, advisors.
For general inquiries participants preferred to consult the IST website or call upon
the information desk at the College of IST.
2. Information requirements/format
Three of the four participants said they would be unable to dial a department phone
based on the 4-digit extension number alone. They did not know the how to call from
a campus phone, or from an external line based on the information provided.
Almost all of them expressed that the room numbers alone were not sufficient to
determine the physical locations. This is because the building is in V-shape with two
wings. Providing room numbers alone was not sufficient for determining which wing
the room belonged to.
3. iScreen functionality awareness
The purpose was to determine if the iScreen in its current form attracted the
attention of the student and visitors. Since it is located in a prominent and well-
lighted area there was a high chance that the participants had seen it even if they
had not used it. Two of the participants had not idea that it was interactive, and two
of them used it because they paid enough attention to notice a “Touch screen to
begin” icon. When asked if they knew if the screen was interactive one of the
participants replied, “Was it a touch screen? No way! I had no idea”.
Based on the interviews it became apparent that the users came to know about the
content only if they went up to the screen and touched it. In its inert phase, the
iScreen did not communicate its purpose or content. Another aspect of the
awareness problem that emerged from the interviews was that the name iScreen did
not communicate much to the users. Most of the participants suggested a more
semantically appropriate information screen or information kiosk instead.
4. Suggested improvements
The participants suggested improvements to enhance usability on two aspects,
aesthetics/interface and content. Aesthetic improvements included having icons in
addition to text that communicate the content. They emphasized the role of colors
and size of the icons that would attract the users’ attention without having to come
close to the screen. This information should be displayed during the inert phase of
the information screen.
Some of the participants gave suggestions to include faculty profiles, student
profiles, and videos to improve interactivity. One of them even suggested sound to
enhance the user experience. We feel that having sound to confirm clicks would
serve as an adequate feedback to the user.
Other suggestions were to have a list of classes in session or classes that are
scheduled for that particular day displayed during the inert phase. Train schedules,
and traffic information were also suggested, which we feel are excellent inclusions.
INFO 610 George Abraham & Christine Wania
6
Usability test
Purpose
The purpose of the usability test is to collect and analyze information about the
effectiveness, efficiency, and user satisfaction of the College of IST interactive
iScreen.
The goals of the usability testing include:
• Determining how easy it is to use the iScreen
• Determining how easy it is to learn to use the iScreen
• Determining how easy it is to find information using the iScreen
• Determining if the information available in the iScreen suits the needs of the
users
• Determining what information is not provided that may be useful
Task list
We developed a task list based on the information available in the iScreen. We also
verified that all the tasks could be completed before conducting any usability tests.
The task list can be seen in Appendix B. The tasks were determined after analyzing
the structure of information in the iScreen. (The menu structure was discussed in the
background section in the beginning of this paper.) In an effort to be thorough in our
testing we constructed tasks that would force users to access every menu option.
The task list embedded in the menu structure can be seen in table 2.
Table 2. Task list embedded in Menu Structure
Main (Touch screen)
• Directory
o Faculty> Scott Silverstein office location and phone extension (Task 1)
o Staff> Colin Saunders office location and extension (Task 11)
o Main
• Building info
o Hours of operation> Hours of operation of CRC (Task 3)
> CRC helpdesk phone # (Task 8)
o Telephone numbers> Teaching assistants office number (Task 6)
> CRC helpdesk phone# (Task 8)
o Main
• IST calendar
o IST events> Events scheduled this week (Task 7)
o Academic calendar
� Graduate Fall schedule > Last day to withdraw (Task 5)
> Fall Quarter exam begins (Task 9)
o Main
• Course schedule
o Fall courses > professor for INFO 435 (Task 2)
> Location date and time for INFO 646 (Task 10)
o Main
• Academic FAQ
o Undergraduate > Advising walk-in hours (Task 4)
> Who to contact regarding advisors (Task 12)
o Graduate
� More questions> Who to contact regarding advisors (Task 12)
o Main
INFO 610 George Abraham & Christine Wania
7
Test environment
The iScreen is in the lobby of the College of IST, therefore we assumed that the main
users would be students and visitors of the College of IST. The iScreen is intended to
be a walk-up-and-use interface therefore we are interested in studying novice users.
Based on the intended user population we included three students of the College of
IST and one visitor in our study. (We tried to find more visitors but did not have any
luck.)
Test set-up
The test was conducted in the actual environment as we felt that the system
environment was important to the study. We wanted to study the system in use, and
interpret the impact it might have on task performance or effectives qualitatively.
Conducting the test in the real environment would help bring out realistic concerns a
participant might have with the information presented.
Investigator tools
Based on the task list we determined that we wanted to measure the time, number
of clicks, and whether or not the task was completed. In addition we decided to
record the path a user takes to find the information in order to allow us to record
errors. We also agreed to record anything that seemed interesting including anything
the user commented on while performing the tasks.
Based on the task list and what we were interested in measuring, we created a form
to record data during the usability test. After conducting the first usability test we
slightly refined the documentation form. We realized it was not necessary to record
the number of clicks if we were recording the path the user took to find the
information. Since we had the site-map, we knew beforehand the actual number of
nodes that need to be traversed to find the task related information. We used this as
measure to determine if the information was difficult to find. We also had difficulty
recording the time because users found the information very quickly and we did not
have great equipment for recording the time, therefore we decided not to record the
time in subsequent tests. The documentation form can be seen in Appendix C.
We created the following steps to guide us through the usability test.
• Introduce ourselves to the user
• Describe the purpose of the test
• Inform the user that we are testing the system, not the user
• Describe what the user will do (12 tasks)
• Inform the user of how long it will take
• Inform the user that they may stop at any time
• Explain to the user that we would like them to think aloud
• Give the user the task list
• Record data
• Give the user the questionnaire
• Debrief the user
• Thank the user
INFO 610 George Abraham & Christine Wania
8
Key findings
Usability test
All four users were able to complete all twelve tasks successfully. Table 3 displays
the tasks in which users committed an error while completing the specified task. In
table 3 the task number appears in the first column of the table. For each task if the
user committed an error the word “ERROR” appears in the corresponding cell in the
table. If the user completed the task without error the corresponding cell is left
blank. For example subject 3 completed tasks 1, 2, 3, 7, 8, 9, 10, and 12
successfully without error, while she committed an error while completing tasks 4, 5,
6, and 11. Examining this table by looking across the rows to focus on which tasks
most of the participants had difficulty completing was enlightening.
Task
Number Subject 1 Subject 2 Subject 3 Subject 4
1 ERROR ERROR 2 ERROR 3 4 ERROR ERROR ERROR ERROR 5 ERROR ERROR 6 ERROR ERROR ERROR ERROR 7 8 9 10 11 ERROR ERROR ERROR 12 ERROR
Table 3. Participant errors while completing tasks
All four participants committed an error while completing tasks 4 and 6. Three users
encountered error while trying to complete task 11. Two users encountered error
while trying to complete tasks 1 and 5. One user encountered error while completing
task 2 and another user encountered error while completing task 12. We analyzed
the paths that each user traversed while trying to complete these tasks.
The correct path and the user paths for these tasks can be seen below. Examining
the user traversal paths can give us insight into were the users expected to find the
information they were looking for. Some users also expressed their thoughts during
the usability testing.
Task 4. Find advising walk-in hours
Correct Path: Academic FAQ – Undergrad FAQ
S1: Building Info – Hours – Main – Academic FAQ – Undergrad FAQ
S2: Academic FAQ – graduate FAQ – undergrad FAQ
S3: Academic FAQ – graduate FAQ – undergrad FAQ
S4: IST Calendar – Academic Calendar - Academic FAQ– undergrad FAQ
INFO 610 George Abraham & Christine Wania
9
For task 4 S1 looked first in building information, possibly assuming that all listed
hours of operation would be found in building information. S2 and S3 are both
graduate students therefore it is not surprising that they both looked first at
graduate FAQs. S4 looked first at the IST calendar to see if the hours would be
displayed there. The errors that both S1 and S4 made are very understandable.
Task 6. Find the phone number for the Teaching Assistant’s office
Correct Path: Building Info – Phone Numbers
S1: Directory – Staff – Faculty – Building Info – Phone Numbers
S2: Directory – Staff – Faculty – Building Info - Phone
S3: Directory – Staff – Academic FAQ – Grad FAQ – Building Info - Phone
S4: Directory – Faculty - Building Info - Phone
For task 6 all the subjects looked in the directory first, assuming that all phone
numbers would be listed in the directory. This is an understandable error. Two
subjects also commented during the usability test that they did not understand why
the phone numbers were separated. They expressed that they assumed all phone
number would be in the directory.
Task 11. Find Colin Saunders’ office location and phone extension
Correct Path: Directory – Staff
S1: Directory – Faculty – Staff
S2: Directory – Faculty - Staff
S3: Directory – Faculty - Staff
S4: Directory –Staff
For task 11 three of the four subjects looked in the directory and then looked in the
faculty section before looking in the staff directory. This is an understandable error.
Users should not be expected to know what position a person holds in order to find
out their office location and phone number.
Task 1. Find Scott Silverstein’s office location and phone extension
Correct Path: Directory - Faculty
S1: Directory – Staff - Faculty
S2: Directory – Faculty
S3: Directory – Faculty
S4: Directory – Staff – Directory – Faculty
The errors made in task 1 are the same as those in task 11. This is an
understandable error. Users should not be expected to know what position a person
holds in order to find out their office location and phone number.
INFO 610 George Abraham & Christine Wania
10
Task 5. Find the last day to withdraw from a course
Correct Path: IST Calendar – Academic Calendar - Fall
S1: Academic FAQ – Main – Course Schedule – IST Calendar – Academic Calendar
S2: IST Calendar - Fall
S3: Academic FAQ – Undergrad FAQ - IST Calendar - Events
S4: IST Calendar – Fall
For task 5 S1 and S3 first looked to the FAQ list, assuming that this would be a FAQ.
The other two subjects went directly to the calendar. This may be an indication that
this type of information should also be displayed as a FAQ.
Examining the errors made by users provided information that the users may not
have expressed while completing the tasks or after completing the tasks. This data
was evidence of where users had the most difficulty and helped us determine what
may need to be redesigned. Examining this data also revealed that not all users are
the same. There were differences in how visitors, undergraduate students, and
graduate students used the system.
Questionnaire
We designed a questionnaire to be administered after the user completed the tasks.
The purpose of the questionnaire was to collect the users’ opinions about
effectiveness, efficiency, and their satisfaction after actually using the system.
Research has shown that efficiency, effectiveness and satisfaction are independent
variables, and hence, have to be measured independently (Frokjar, 2000). We also
took into consideration issues like learnability and navigability in the design of the
post-test questionnaire. The questionnaire can be seen in Appendix D. The
questionnaire was designed with closed-ended and open-ended questions. The close-
ended question responses were collected using a 5-point Likert scale because these
were subjective measures.
The questionnaire was administered after the users completed the twelve tasks. All
four users responded that they knew the iScreen was interactive before the study.
Two of the users had used the iScreen before this study and the other two had not. A
summary of the user responses can be seen in figure 1. We grouped these
statements based on which usability aspect we thought they measured. The raw
counts of the user responses can be seen in Appendix E.
Overall users expressed that it was easy to learn to use the system. This may be
because you only need to be able to touch the screen to use it. The majority of users
expressed that they found it easy to navigate through the system even though most
of them did not like where the icons/buttons were located.
Overall the users were not very satisfied with the efficiency of the system. A few
users also expressed aloud while performing the tasks that “it’s taking a long time to
find these things.” Although it was difficult for us to time these tasks because we
thought users were finding things quickly the users felt that it was taking too long.
Only one user expressed that it was easy to find the information they needed.
INFO 610 George Abraham & Christine Wania
11
0%
10%
20%
30%
40%
50%
60%
70%
80%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Statement
Pe
rce
nta
ge
of
us
ers
ag
ree
ing
wit
h s
tate
me
nt
Effectiveness
Efficiency
Satisfaction
Ease of use
Learnability
Navigability
Table 3. Questionnaire Response
Statement
1. The interface communicated the content found in the system
2. The organization of information on the system screens is clear
3. This system has all the functions and capabilities I expect it to have
4. Using the system for answering some common questions is a good idea
5. I would use the system in the future
6. It was easy to read the information on the screen
7. It is easy to find the information I needed
8. I felt it took too long to find the information I was looking for
9. Overall, I am satisfied with how easy it is to use this system
10. I feel comfortable using this system
11. The interface of this system is pleasant
12. I like using the interface of this system
13. Overall, I am satisfied with this system
14. It was simple to use this system
15. It was easy to learn to use this system
16. Whenever I make a mistake using the system, I recover easily and quickly
17. It is easy to navigate
18. The icons/buttons were conveniently located
The main dislike expressed by the users in the open-ended questions was that the
screen they were currently on disappeared too quickly. In one user’s words “it comes
out of the current screen very quickly, by the time I decide to press something, it
goes back to the flashing screens.” The main thing users expressed the iScreen being
useful for was for finding faculty phone numbers and office locations. Finding the
faculty and staff phone numbers was one of the more difficult tasks for the users to
perform based on the number of errors made in these tasks. Therefore this is one
INFO 610 George Abraham & Christine Wania
12
aspect that should be redesigned. Some users suggested improving the design by
adding a search feature for faculty. Another thing that users expressed was that
there needs to be something more to draw one’s attention to the screen.
Implications for design
Based on the interview responses, usability test and post-test questionnaire we
suggest some changes that could be implemented to improve usability of the
information screen.
1. Improve the phone directory interface to facilitate browsing capabilities
a. Facilitate last name based searching
During the usability test it became apparent that both students and visitors were
finding it difficult to find people because the current content layout implicitly
implies that the user would know who is a faculty and who is a staff (refer Task 1
& 11, Appendix B). The site map of current layout is as follows:
Main-Directory-Faculty or Staff
We suggest that this interface should be improved so that one could search/sort
list of names based on last name alone.
An alternative interface could be:
Main-Directory- A|B|C|D….|Z
In the results it could be indicated if the person listed is a faculty or staff like
Doe, John (Staff).
b. List important phone numbers as default page of the phone directory
Some important phone numbers and contact information could be presented on
the default page of the phone directory, like Teaching assistants (TA) office
number, Computing resource center (CRC) phone number, IST information desk,
Campus security, Bursars office without using abbreviations for the above. This
could be done in addition to listing it under important phone numbers in Building
Info (Refer Fig.2, Pg XX).
c. Provide help in dialing the extension and finding the office.
The current directory listing provides only the extension number and room
number of the contact person. Only two among the eight people who participated
knew how to use a campus phone or call based on the extension number alone. A
one line help should be displayed that provides instruction to the user on how to
place a call using a campus phone or an external line. Providing some help to the
user for finding the faculty office would be beneficial. The Rush Building is a V-
shaped building with two wings that house offices and classrooms. Letting the
user know which wing to proceed to when they exit out of the stairway, or
elevator could speed up the process of locating it.
Note: Another optional improvement could be including faculty profiles if
someone clicks on the faculty name in the directory search result. This is based
INFO 610 George Abraham & Christine Wania
13
on the observation that during the usability test the participants continued to
click on the faculty name hoping to get some more information.
2. Improve visibility and layout of F.A.Q section
Tasks 4 & 12 were related to finding information on advising hours, and advisors.
All four usability test participants made an error in getting to this information.
One of the reasons for this was that the current FAQ section followed the
questions-followed-by-answers format. This seemed to be too much information
too quick worsened by the fact the screen returned to main screen before one
could finish reading. In case of graduate FAQ, this information required two
pages, and the participants tended to miss seeing the link to the next page.
We suggest that the FAQ section default page lists just the questions categorized
as follows:
Q.No. Undergraduate FAQ Graduate FAQ
1 What are the advising walk-in hours? What is the minimum GPA
requirement?
2 Who is my advisor? Who to contact to find out
graduation requirements?
This listing can be ordered based on what the students ask most frequently.
3. Using icons and colors to communicate content/attract user attention
Seven of our eight participants were aware that the information screen was
interactive, but only four of them had actually used it. Among the four users only
one of them had actually used it to look for something. The other three users had
touched it but did not have an idea of what it actually contained.
The fact that they knew that the information screen is interactive is influenced by
the fact that the College sent out an email to the student and faculty to inform
them that it was interactive.
Our interview and usability test elicited suggestions that the information screen
could use customized icons and colors to communicate content quickly even if
viewed from a distance, in this case the entrance. Some of the suggestions were
for motion graphics to attract attention. May be having white/light grey text on
black background could improve readability of the interface.
4. Maximize the use of the information screen during the inert phase (when not
touched)
Interviews and questionnaires with prospective and current users communicated
the need for having some meaningful information displayed on the screen.
Currently the screen shows guest lectures scheduled, and pictures of past events,
which is good but does not exploit the full potential. Displaying list of classes
scheduled for that day, room changes, or even SEPTA train schedules for
commuter students would be good to increase the chances of students/visitors to
use it as an information resource.
INFO 610 George Abraham & Christine Wania
14
5. Increase the amount of time a user-selected screen is displayed.
Most of the participants expressed dissatisfaction when the screens kept
returning to the main screen while they were reading the information. This forced
them to search for information all over again. In addition to this, the touch
sensitivity of the touch screen caused the screen to behave unpredictably at
times. The screen touch-sensitivity needs to be adjusted and tested after
studying the pattern of use. Also it would be beneficial if the user-selected
screens are visible for a longer period of time.
6. Change the name iScreen to Information Kiosk
Although the name IST-iScreen may seem catchy, it does not have a significant
impact on the participants. Most of them when asked, what came to their mind
when the word iScreen was used said, “a nice cold treat”. We tested words like
“Information Kiosk” and “Information Screen” which met with significantly higher
approval. All eight of our participants preferred a semantically appropriate word
to describe the screen and its contents.
INFO 610 George Abraham & Christine Wania
15
Appendix A
Interview Questions
What program are you in?
How many years have you been in the program?
When you have questions about the College of IST who do you usually ask?
How often do you use the College of IST website to look for information?
Are you familiar with the room numbers in this building?
How would you make a call to one of the faculty offices based on the extension
number?
Did you know the iScreen was interactive? YES NO
Have you used the iScreen before? YES NO
If Yes, what were you looking for? And did you find it?
INFO 610 George Abraham & Christine Wania
16
Appendix B
Task List
Kindly complete these tasks using the iScreen
1. Find Scott Silverstein’s office location and phone extension.
2. Find the professor for INFO 435 001 Info Services.
3. Find the hours of operation for the Computing Resource Center.
4. Find advising walk in hours.
5. Find the last day to withdraw from a course.
6. Find the phone number for the Teaching Assistant’s office.
7. Find the events scheduled for this week.
8. Find the Computing Resource Center help desk phone number.
9. Find when the Fall Quarter exams begin.
10. Find the location, days and time the INFO 646 510 IS Management course is
offered.
11. Find Colin Saunders’ office location and phone extension.
12. Find out who to contact for details about faculty advisors.
INFO 610 George Abraham & Christine Wania
17
Appendix C
Documentation Form for Usability Testing
1. Find Scott Silverstein’s office location and phone extension.
Success
Path
Notes
2. Find the professor for INFO 435 001 Info Services.
Success
Path
Notes
3. Find the hours of operation for the Computing Resource Center.
Success
Path
Notes
4. Find advising walk in hours.
Success
Path
Notes
5. Find the last day to withdraw from a course.
Success
Path
Notes
6. Find the phone number for the Teaching Assistant’s office.
Success
Path
INFO 610 George Abraham & Christine Wania
18
Notes
7. Find the events scheduled for this week.
Success
Path
Notes
8. Find the Computing Resource Center help desk phone number.
Success
Path
Notes
9. Find when the Fall Quarter exams begin.
Success
Path
Notes
10. Find the location, days and time the INFO 646 510 IS Management course is
offered.
Success
Path
Notes
11. Find Colin Saunders’ office location and phone extension.
Success
Path
Notes
12. Find out who to contact for details about faculty advisors.
Success
Path
Notes
INFO 610 George Abraham & Christine Wania
19
Appendix D
Questionnaire
Please answer the following questions.
Did you know the iScreen was interactive? YES NO
Have you used the iScreen before today? YES NO
If Yes, what were you looking for? And did you find it?
Please answer the following questions about the iScreen by placing a check
in the appropriate box.
STRONGLY
AGREE
AGREE NEITHER AGREE OR DISAGREE
DISAGREE
STRONGLY DISAGREE
The interface communicated the content found in the system
□
□
□
□
□
Overall, I am satisfied with how easy it is to use this system
□
□
□
□
□
It was simple to use this system □
□
□
□
□
I feel comfortable using this system □
□
□
□
□
It was easy to learn to use this system □
□
□
□
□
Whenever I make a mistake using the system, I recover easily and quickly
□
□
□
□
□
It is easy to find the information I needed □
□
□
□
□
The organization of information on the system screens is clear
□
□
□
□
□
The interface of this system is pleasant □
□
□
□
□
I like using the interface of this system
□
□
□
□
□
This system has all the functions and capabilities I expect it to have
□
□
□
□
□
Overall, I am satisfied with this system
□
□
□
□
□
Using the system for answering some common questions is a good idea
□
□
□
□
□
I felt it took too long to find the information I was looking for
□
□
□
□
□
I would use the system in the future □
□
□
□
□
It is easy to navigate
□ □ □ □ □
The icons/buttons were conveniently located
□ □ □ □ □
It was easy to read the information on the screen
□ □ □ □ □
INFO 610 George Abraham & Christine Wania
20
What were things you liked or disliked about the iScreen?
Why or why not would you use the iScreen in the future?
What information did you find useful?
Was there information not included in the iScreen that you may find useful?
What could we have on the iScreen that would catch your attention when you walked
by?
What improvements would you suggest?
INFO 610 George Abraham & Christine Wania
21
Appendix E
Questionnaire Responses
ST
RO
NG
LY
A
GR
EE
AG
RE
E
NE
ITH
ER
A
GR
EE
OR
D
ISA
GR
EE
DIS
AG
RE
E
ST
RO
NG
LY
D
ISA
GR
EE
The interface communicated the content found in the system 2 2
Overall, I am satisfied with how easy it is to use this system 1 3
It was simple to use this system 1 1 2
I feel comfortable using this system 1 2 1
It was easy to learn to use this system 3 1
Whenever I make a mistake using the system, I recover easily and quickly 2 2
It is easy to find the information I needed 1 2 1
The organization of information on the system screens is clear 2 2
The interface of this system is pleasant 2 2
I like using the interface of this system 1 2 1
This system has all the functions and capabilities I expect it to have 4
Overall, I am satisfied with this system 1 1 2
Using the system for answering some common questions is a good idea 1 1 2
I felt it took too long to find the information I was looking for 1 2 1
I would use the system in the future 1 1 2
It is easy to navigate 3 1
The icons/buttons were conveniently located 1 2 1
It was easy to read the information on the screen 2 2