umbc blackboard usability testing spring 2002. overview usability testing basics umbc blackboard...

55
UMBC UMBC Blackboard Blackboard Usability Testing Usability Testing Spring 2002 Spring 2002

Upload: clemence-sparks

Post on 22-Dec-2015

221 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

UMBCUMBCBlackboard Usability Blackboard Usability

TestingTestingSpring 2002Spring 2002

Page 2: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

OverviewOverview

Usability Testing BasicsUsability Testing Basics UMBC Blackboard Usability TestingUMBC Blackboard Usability Testing Highlights VideoHighlights Video Sneak Peek: Blackboard Version 6Sneak Peek: Blackboard Version 6 Next Web Usability Study: myUMBCNext Web Usability Study: myUMBC

Page 3: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Usability DefinedUsability Defined

““Making whatever you’re working on easier Making whatever you’re working on easier to use for who ever is going to use it.”to use for who ever is going to use it.”

““An element of design, focusing on ‘can this An element of design, focusing on ‘can this be used when it’s done?’ rather than just be used when it’s done?’ rather than just making it look good.”making it look good.”

Steve Krug, 2001 WebTalkGuys interviewSteve Krug, 2001 WebTalkGuys interview

Page 4: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Must ReadMust Read

One of Amazon’s top One of Amazon’s top 200 books.200 books.

Companion site Companion site www.sensible.comwww.sensible.com Sample chapterSample chapter Sample scriptSample script Workshops & updatesWorkshops & updates

Page 5: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Usability BasicsUsability Basics

Instead of imagining what users want, you observe, Instead of imagining what users want, you observe, interview them and prototype their needs.interview them and prototype their needs.

Develop content, design and programming strategy by Develop content, design and programming strategy by reconciling observed user needs with sponsor’s desired reconciling observed user needs with sponsor’s desired objectives.objectives.

Test user tasks and expectations with low-cost Test user tasks and expectations with low-cost prototypes (e.g., paper, PhotoShop) prototypes (e.g., paper, PhotoShop)

User’s don’t make mistakes; websites doUser’s don’t make mistakes; websites do

Page 6: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Pros & ConsPros & Cons

The alternative of using designers and The alternative of using designers and programmers as “proof of concept” tools is programmers as “proof of concept” tools is expensive and demoralizing.expensive and demoralizing.

May seem to add time (and cost) to a production May seem to add time (and cost) to a production schedule, but constantly revising is expensive, schedule, but constantly revising is expensive, too.too.

Page 7: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

BlackboardBlackboard

Page 8: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Blackboard @ UMBCBlackboard @ UMBC

UMBCUMBC Faculty = 616 full-time, 318 part-timeFaculty = 616 full-time, 318 part-time Students = 11,237Students = 11,237 Staff = 415Staff = 415

BlackboardBlackboard Courses & organizations = 699 (238 Sp02)Courses & organizations = 699 (238 Sp02) Number of users = 12,141Number of users = 12,141 Number of Instructors & managers = 610Number of Instructors & managers = 610 Average hits per day = 35,585Average hits per day = 35,585

Page 9: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User/Task AnalysisUser/Task Analysis

Page 10: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

QuestionnaireQuestionnaire

Page 11: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User : Instructor 1User : Instructor 1

Interview Date: Interview Date: 2/18/022/18/02

Name: RobinName: Robin Sex: FemaleSex: Female Role: InstructorRole: Instructor

Page 12: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical Usage & Findings Typical Usage & Findings

Post syllabus, Post syllabus, handouts, handouts, announcementsannouncements

Collect and return Collect and return papers via Digital papers via Digital DropBoxDropBox

Discussion boardDiscussion board

Would like to be able to Would like to be able to do bulk deletesdo bulk deletes

Likes email discussions, Likes email discussions, students don’t use students don’t use Blackboard forumsBlackboard forums

Grade book inflexibleGrade book inflexible Likes Word editing and Likes Word editing and

feels like she could feels like she could duplicate many BB duplicate many BB features with other tools.features with other tools.

Usage Findings

Page 13: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User : Instructor 2User : Instructor 2

Interview Date: Interview Date: 2/18/022/18/02

Name: KristieName: Kristie Sex: FemaleSex: Female Role: InstructorRole: Instructor

Page 14: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical Usage & FindingsTypical Usage & Findings

Posting to and usage of Posting to and usage of course syllabuscourse syllabus

Posting to and usage of Posting to and usage of course schedulescourse schedules

Posting of assignmentsPosting of assignments Use of Grade BookUse of Grade Book Accessing documents in Accessing documents in

Drop BoxDrop Box Usage of discussion Usage of discussion

boardboard

Frequently uses 2 or more Frequently uses 2 or more browser windows along with browser windows along with other apps (such as Word)other apps (such as Word)

Makes heavy use of her own Makes heavy use of her own external siteexternal site More reliableMore reliable Easier to back upEasier to back up Always accessibleAlways accessible

Uses BB to post copyrighted Uses BB to post copyrighted materialmaterial

Uses MS Word for spell Uses MS Word for spell checkingchecking

Wants fewer buttonsWants fewer buttons Sees BB as an enhancement Sees BB as an enhancement

to course.to course.

Usage Findings

Page 15: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User : Instructor 3User : Instructor 3

Interview Date: Interview Date: 2/18/022/18/02

Name: KarinName: Karin Sex: FemaleSex: Female Role: InstructorRole: Instructor

Page 16: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical Usage & FindingsTypical Usage & Findings

Posting of assignmentsPosting of assignments Retrieval of completed Retrieval of completed

workwork Assistance in assigning Assistance in assigning

and collecting workand collecting work Support of classroom Support of classroom

instructioninstruction CommunicationCommunication Homework assignment Homework assignment

hand-inhand-in

Wants:Wants: More flexible search More flexible search Check boxes with Check boxes with

Digital Drop Box to Digital Drop Box to indicate document indicate document statusstatus

Replication from Replication from computer-to-computercomputer-to-computer

Batch processingBatch processing Additional levels of Additional levels of

admin for TA’sadmin for TA’s

Usage Findings

Page 17: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User : Student 1User : Student 1

Interview Date: Interview Date: 2/12/022/12/02

Name: RogerName: Roger Sex: MaleSex: Male Role: StudentRole: Student

Page 18: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical Usage and FindingsTypical Usage and Findings

Checking:Checking: AssignmentsAssignments SchedulesSchedules AnnouncementsAnnouncements

Discussion board for Discussion board for assignmentsassignments

Downloads course Downloads course docs.docs.

Checks gradesChecks grades

Interface didn’t match Interface didn’t match mental modelmental model

Primary functions:Primary functions: Discussion boardsDiscussion boards Digital drop boxDigital drop box

Not familiar with many Not familiar with many featuresfeatures

Uses Word for spell Uses Word for spell checkingchecking

Backs up replies in Word.Backs up replies in Word. Wants to save docs,not Wants to save docs,not

look at them in browser.look at them in browser. Easy to use, convenientEasy to use, convenient

Usage Findings

Page 19: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User : Student 2User : Student 2

Interview Date: Interview Date: 2/13/022/13/02

Name: ChrisName: Chris Sex: MaleSex: Male Role: StudentRole: Student

Page 20: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical Usage/FindingsTypical Usage/Findings

Mandatory useMandatory use CommunicationCommunication Drop BoxDrop Box Retrieving and Retrieving and

submitting submitting assignmentsassignments

Likes interface: “User Likes interface: “User friendly”friendly”

Easy to navigateEasy to navigate Quick to learnQuick to learn Got several errorsGot several errors

Usage Findings

Page 21: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User : Student 3User : Student 3

Interview Date: Interview Date: 2/13/022/13/02

Name: AnitaName: Anita Sex: FemaleSex: Female Role: StudentRole: Student

Page 22: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical Usage/FindingsTypical Usage/Findings

Checks course Checks course materialsmaterials

Completes homeworkCompletes homework Posts questions to Posts questions to

discussion board discussion board about homeworkabout homework

Encountered server Encountered server errorserrors

Confusion about Confusion about multiple Discussion multiple Discussion BoardsBoards

Clicks on all buttons Clicks on all buttons to find what she’s to find what she’s looking forlooking for

Does not check site Does not check site frequently/dailyfrequently/daily

Usage Findings

Page 23: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User : Student 4User : Student 4

Interview Date: Interview Date: 2/13/022/13/02

Name: SaraName: Sara Sex: FemaleSex: Female Role: StudentRole: Student

Page 24: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical usage/findingsTypical usage/findings

Checks Checks announcements and announcements and course materialcourse material

Do homeworkDo homework Use discussion boardUse discussion board Maximum use: 5X per Maximum use: 5X per

weekweek EmailEmail

Works from own Works from own computercomputer

Uses multiple Uses multiple windows to avoid windows to avoid going back and forthgoing back and forth

Confused by multiple Confused by multiple discussion boardsdiscussion boards

Missed Missed announcements announcements burried in lots of text.burried in lots of text.

Usage Findings

Page 25: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User/Task MatrixUser/Task Matrix

Page 26: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Typical EnvironmentsTypical Environments

University public computer lab workstation University public computer lab workstation (Students)(Students)

Home computer (students & instructors)Home computer (students & instructors) ModemModem CableCable

Office computer (instructors)Office computer (instructors)

Page 27: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User GoalsUser Goals

Earn a good gradeEarn a good grade Learn about course Learn about course

topictopic Complete Complete

requirementrequirement Complete courseComplete course Research topicsResearch topics

Use the Web to enhance Use the Web to enhance coursecourse

Enable learningEnable learning Design coursesDesign courses Facilitate administrationFacilitate administration Foster interactionFoster interaction Provide supplementary Provide supplementary

researchresearch Manage large classes with Manage large classes with

multiple sectionsmultiple sections Create online archiveCreate online archive Organize materialsOrganize materials Personal enrichmentPersonal enrichment

Students Instructors

Page 28: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Task FlowTask Flow

Page 29: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User TasksUser Tasks

View course workView course work Read notesRead notes Ask questionsAsk questions Share informationShare information Turn in assignmentsTurn in assignments Do researchDo research View SchedulesView Schedules View syllabusView syllabus Find required readingsFind required readings Get helpGet help Download documentsDownload documents

Deliver course workDeliver course work Track student progressTrack student progress Monitor participationMonitor participation Track attendanceTrack attendance Track gradesTrack grades Archive course infoArchive course info Communicate with studentsCommunicate with students Conduct discussions and lecturesConduct discussions and lectures Review workReview work Grade workGrade work Edit assignments, course work, Edit assignments, course work,

schedules, etc.schedules, etc. Make announcementsMake announcements Post schedules, syllabus, etc.Post schedules, syllabus, etc. Manage accessManage access

Students Instructors

Page 30: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User TestsUser Tests

Page 31: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

MethodologyMethodology

Conducted at UMBCConducted at UMBC Volunteer participantsVolunteer participants

5 students5 students 4 instructors4 instructors Varying experienceVarying experience

Test team:Test team: Moderator- lead subject through tasksModerator- lead subject through tasks Observer- noted subject responses/actionsObserver- noted subject responses/actions

Protocol:Protocol: Subjects in front of computer, asked to complete tasksSubjects in front of computer, asked to complete tasks Subjects encouraged to verbalize thoughtsSubjects encouraged to verbalize thoughts Responses recorded by ObserverResponses recorded by Observer

Page 32: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Usability Test ScriptsUsability Test Scripts

Page 33: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

SummarySummary

Easy to learn and useEasy to learn and use Consistent navigation helpsConsistent navigation helps Operational scalability issuesOperational scalability issues Features for “power users”Features for “power users” EfficienciesEfficiencies Menu wording/functionsMenu wording/functions

Page 34: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Summary: Task EfficienciesSummary: Task Efficiencies

Faculty spend more time than students, Faculty spend more time than students, complained about repetitive and boring complained about repetitive and boring tasks:tasks: Enrolling more than one student in course or Enrolling more than one student in course or

groupgroup Managing documents via the Digital Drop BoxManaging documents via the Digital Drop Box Managing own contentManaging own content Inflexibility of gradebookInflexibility of gradebook

Page 35: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Summary: Interface labelingSummary: Interface labeling

Students main issue: misunderstanding Students main issue: misunderstanding important menus and functions:important menus and functions: ““Tools” a vague termTools” a vague term ““Digital Drop Box” and Grades sections use Digital Drop Box” and Grades sections use

unclearunclear Send vs. Drop metaphor problematicSend vs. Drop metaphor problematic Confusion over Assessment & Pool managerConfusion over Assessment & Pool manager Editable navigationEditable navigation

Page 36: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

SubjectsSubjects

5 students5 students Graduate, PsychologyGraduate, Psychology Undergraduate, Computer ScienceUndergraduate, Computer Science Undergraduate, AnthropologyUndergraduate, Anthropology Graduate Student, Information SystemsGraduate Student, Information Systems Undergraduate, Computer ScienceUndergraduate, Computer Science

4 Instructors4 Instructors Geography, expertGeography, expert Modern Languages, advanced beginnerModern Languages, advanced beginner English, advanced beginnerEnglish, advanced beginner Chemical Engineering, beginnerChemical Engineering, beginner

Page 37: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User Testing ResultsUser Testing ResultsStudentsStudents

Page 38: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Digital Drop BoxDigital Drop Box Observation:Observation:

Difficult for studentsDifficult for students Confused with “submit” and Confused with “submit” and

“send”“send” Confirmation/warning Confirmation/warning

ignored.ignored.

Recommendation:Recommendation: Language should be Language should be

reconsidered to clear reconsidered to clear up confusionup confusion

Submit should send Submit should send with first clickwith first clickRating: UNUSABLE

Page 39: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Tools LanguageTools Language Observation:Observation:

Confusion over “Tools” Confusion over “Tools” buttonbutton

Frequently ignoredFrequently ignored When used, not able to When used, not able to

locate informationlocate information

Recommendation:Recommendation: Use more descriptive Use more descriptive

language in labelslanguage in labels Allow instructors to move Allow instructors to move

frequently accessed frequently accessed functionalityfunctionality

Create more meaningful Create more meaningful name based on user insight.name based on user insight.Rating: SEVERE

Page 40: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Discussion BoardDiscussion Board Observation:Observation:

Confusing to studentsConfusing to students Unclear about role of Unclear about role of

discussion boarddiscussion board Overwhelmed with active Overwhelmed with active

boardsboards

Recommendation:Recommendation: Work on hierarchyWork on hierarchy Provide multiple levels of Provide multiple levels of

configurability.configurability. Clearly differentiate Group Clearly differentiate Group

Discussions from general Discussions from general class discussionsclass discussions

Be able to rename “Group” as Be able to rename “Group” as “Lab”“Lab”Rating: MODERATE to SEVERE

Page 41: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

NavigationNavigation Observation:Observation:

Students go to “staff Students go to “staff information” instead of information” instead of “communications” for email“communications” for email

Recommendation:Recommendation: Provide clearly-labeled “email” Provide clearly-labeled “email”

or “contact instructor” buttonsor “contact instructor” buttons

Rating: MODERATE

Page 42: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

EmailEmail Observation:Observation:

Students used external email Students used external email appapp

Did not use Blackboard email Did not use Blackboard email functionsfunctions

Recommendation:Recommendation: Provide more familiar e-mail Provide more familiar e-mail

interfaceinterface

Rating: IRRITANT

Page 43: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

User Testing ResultsUser Testing ResultsInstructorInstructor

Page 44: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Adding Multiple UsersAdding Multiple Users Observation:Observation:

Batch functions not evidentBatch functions not evident Added one at a timeAdded one at a time Large amount of search returnsLarge amount of search returns Used “guessing” strategy to locate Used “guessing” strategy to locate

studentsstudents One used user IDOne used user ID Instructors have batch enroll rightsInstructors have batch enroll rights

Recommendation:Recommendation: Provide more evident labelsProvide more evident labels Separate enrollment from Separate enrollment from

creation of useridcreation of userid Make the function obvious and Make the function obvious and

easy to useeasy to use

Rating:SEVERE

Page 45: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Editable NavigationEditable Navigation Observation:Observation:

Mandatory use of pre-made labels Mandatory use of pre-made labels limiting, especially with foreign-limiting, especially with foreign-language instructorslanguage instructors

Labels often confusing to studentsLabels often confusing to students

Recommendation:Recommendation: Provide multiple language Provide multiple language

supportsupport Provide configuration options Provide configuration options

for interface elementsfor interface elements

Rating:MODERATE

Page 46: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Virtual ClassroomVirtual Classroom Observation:Observation:

Instructors did not find function to be Instructors did not find function to be usefuluseful

Unproductive with large classesUnproductive with large classes Concerned about student access& Concerned about student access&

browser compatibilitybrowser compatibility Limited space for typingLimited space for typing Little space to read messagesLittle space to read messages Loads slowly in some configurationsLoads slowly in some configurations

Recommendation:Recommendation: Provide greater stability Provide greater stability

across platforms/browsersacross platforms/browsers Larger in-chat text areaLarger in-chat text area

Rating:MODERATE

Page 47: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Assessment ManagerAssessment Manager Observation:Observation:

Frequently misinterpreted and Frequently misinterpreted and ignoredignored

Could not identify function from Could not identify function from labellabel

Few instructors could create Few instructors could create online quizonline quiz

Recommendation:Recommendation: Clearly defined labels based Clearly defined labels based

on user insighton user insight Clearly define function based Clearly define function based

on academic-- not technical--on academic-- not technical--languagelanguage

Rating:MODERATE

Page 48: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

GradebookGradebook Observation:Observation:

Did not find usefulDid not find useful InflexibleInflexible Grades confused studentsGrades confused students Wanted ability to import gradesWanted ability to import grades Wanted ability to export gradesWanted ability to export grades

Recommendation:Recommendation: Provide “weighting” functionalityProvide “weighting” functionality Provide ability to factor in letter Provide ability to factor in letter

gradesgrades Provide import/export capability.Provide import/export capability.

Rating:MODERATE

Page 49: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Advanced OptionsAdvanced Options Observation:Observation:

Ignored option settingsIgnored option settings Expressed curiosity about Expressed curiosity about

infrequently-used functionsinfrequently-used functions Amount of choices overwhelmingAmount of choices overwhelming Often assumed red reminders Often assumed red reminders

were warnings of errorswere warnings of errors

Recommendation:Recommendation: Improve trainingImprove training Improve help functionsImprove help functions Provide multiple user levels of Provide multiple user levels of

complexity: “Power User” vs. complexity: “Power User” vs. default simple interfacedefault simple interface

Interactive “Wizards” to guide Interactive “Wizards” to guide setupsetup

Use color other than red in text for Use color other than red in text for simple confirmationssimple confirmationsRating:MODERATE

Page 50: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

GroupsGroups Observation:Observation:

Instructors added each student Instructors added each student individuallyindividually

One instructor hired grad assistant One instructor hired grad assistant just for this functionjust for this function

Recommendation:Recommendation: Allow batch updatingAllow batch updating Create groups before adding Create groups before adding

students to themstudents to them

Rating:MODERATE

Page 51: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Highlights: VideoHighlights: Video

Page 52: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

OverallOverall

Effective and easy to use for basic tasksEffective and easy to use for basic tasks Terminology/wording confusingTerminology/wording confusing Instructors crave batch functionsInstructors crave batch functions Interface inflexibleInterface inflexible Metaphors can get in the way of functionality Metaphors can get in the way of functionality

(eg. Digital DropBox)(eg. Digital DropBox) Often too much information/optionsOften too much information/options

Page 53: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Sneak Peek: Blackboard 6Sneak Peek: Blackboard 6

UMBC migrating to version 6 for Spring 2003UMBC migrating to version 6 for Spring 2003 LoginLogin

http://preview9.blackboard.com/http://preview9.blackboard.com/ Userid = umbctestUserid = umbctest Psswd = umbctestPsswd = umbctest

Available for 4-6 weeksAvailable for 4-6 weeks Release notes in “UMBC Test Course”Release notes in “UMBC Test Course” Send comments to Send comments to [email protected]@umbc.eduedu

Page 54: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

Next Usability Study: myUMBCNext Usability Study: myUMBC

myUMBC has not changed much in 4 yearsmyUMBC has not changed much in 4 years PeopleSoft HR & Finance launch July ’03PeopleSoft HR & Finance launch July ’03 Need for more roles & affinitiesNeed for more roles & affinities Questions: Questions:

How do current students, faculty & staff currently use How do current students, faculty & staff currently use myUMBC vs. UMBC website?myUMBC vs. UMBC website?

What would users like to do now but can’t?What would users like to do now but can’t? How could campus make better use of its portal?How could campus make better use of its portal?

Likely redesign launch date: April ‘03Likely redesign launch date: April ‘03

Page 55: UMBC Blackboard Usability Testing Spring 2002. Overview  Usability Testing Basics  UMBC Blackboard Usability Testing  Highlights Video  Sneak Peek:

LinksLinks

www.umbc.edu/oit/newmedia/blackboard/usabilitywww.umbc.edu/oit/newmedia/blackboard/usability::

PresentationPresentation Final ReportFinal Report Highlights VideoHighlights Video