(almost) free usability testing
DESCRIPTION
My presentation for Reboot 10 (http://reboot.dk) on the subject of usability testing of software. The presentation addresses a particular set of solutions for the most important questions in usability testing and in particular in how do lower the budget needs for each situation.TRANSCRIPT
(almost) FREE USABILITY TESTING
http://flickr.com/photos/74676300@N00/509491063/
http://flickr.com/photos/dulciedulcie/2413926822/
NOTES / DISCLAIMERS
Why Make Usability Tests?
BETTER EXPERIENCE
SATISFACTION LOYALTY NUMBERS ...
Business Reasons
REDUCE COSTS: Development Support Maintenance
‘software engineering, a practitioners approach’ McGraw-Hill
COST OF CHANGES
1x During Design
6x DuringDevelopment
100x Afterdeployed
Users are all those persons thatis affected by our products/ services
Have you ever done a usability test, lost hours writing the usability report
and in the end didn’t seen any results applied?
How many times have you read someone else usability report?
Lost time?
Usability Reportsnormally don’t get to those
who need them most!
And few are willing to read them!
Traditional Tests
Number of Users
Recruiting Effort
Where to make them
Who makes them
Planning
Preparation
What? And When to test?
Costs
next?
10/12+
Selected according to the audience
Usability Lab
Usability Professional
Weeks planning
Write and Discuss a test protocol
Normally everything and close to the end of the project
€5000 - €15000+
A ‘n’ page report a week later / Development meeting
Traditional Tests
Number of Users
Recruiting Effort
Where to make them
Who makes them
Planning
Preparation
What? And When to test?
Costs
next?
10/12+
Selected according to the audience
Usability Lab
Usability Professional
Weeks planning
Write and Discuss a test protocol
Normally everything and close to the end of the project
€5000 - €15000+
A ‘n’ page report a week later / Development meeting
DIFFICULTIES
Test as often as we need
Recruit Testers
Recording and analyzing all the information
Processing all the recorded materials that came out of the tests
Gather and process all observer’s notes
Evaluate the overall satisfaction for each service/site
Discuss observations
Implementing conclusions
Traditional Tests Fast Usability Tests
Number of Users
Recruiting Effort
Where to make them
Who makes them
Planning
Preparation
What? And When to test?
Costs
next?
8+ 3/4
Selected according to the audience
Any person with web experience
Usability Lab Any Room
Usability Professional Someone with patience
Weeks planning Anytime
Write and Discuss a test protocol
Select the features we want to test
Normally everything and close to the end of the project
Small tests at any time of the development process
€5000 - €15000+ 0 - €300 (€0 - €100 per user)
A ‘n’ page report a week later / Development meeting
Meeting Immediately after the tests
AGILE TESTING
RECORD INFORMATION
Usage stats
Customer support reports
Eye tracking studies
Usage Heat-Maps
...
RECORD INFORMATION
Possible Setup
VNC
Sala de Testes
Observadores
RECORD INFORMATION
Morae
http://www.techsmith.com/morae.asp
ScreenFlow + VNC
https://www.varasoftware.com/products/screenflow/
SilverBack (soon!)
http://www.silverbackapp.com/
RECORD INFORMATION
GATHER AND PROCESS
Assemble a standard observation report form for all observers, to use during the tests
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
(...)2
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
TAREFA 11
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
(...)3
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
(...)4
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
(...)5
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
(...)6
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
(...)7
Terminou com Facilidade Terminou com Dificuldade Não conseguiu terminar
TAREFA 88
Estudo:
Utilizador:
Avaliador:
Data:
Observações:
GATHER AND PROCESS
0
4
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8
Finished without problems Finished with problems Didn’t finish
GATHER AND PROCESS
EVALUATE SATISFACTION
Escala de Usabilidade de Serviços Discordo Muito
Concordo Muito
1. Gostaria de usar este serviço frequentemente
2. Achei que o serviço era desnecessariamente complexo
3. Achei o serviço simples de usar
4. Penso que iria precisar de apoio técnico para usar o serviço
5. Achei as várias funcionalidades do serviço bem integradas
6. Penso que havia demasiadas inconsistências no serviço
7. Imagino que a maioria das pessoas aprenda rapidamente a usar o serviço
8. Achei que o serviço não era trivial de usar
9. Senti-me muito confiante a usar o serviço
10. Preciso de aprender muito antes de poder usar este serviço
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
‘SUS - A quick and dirty usability scale’ - John Brooke
EVALUATE SATISFACTION
Gostaria de usar este serviço frequentemente
Achei que o serviço era desnecessariamente complexo
Achei o serviço simples de usar
Penso que iria precisar de apoio técnico para usar o serviço
Achei as várias funcionalidades do serviço bem integradas
Penso que havia demasiadas inconsistências no serviço
Imagino que a maioria das pessoas aprenda rapidamente a usar o serviço
Achei que o serviço não era trivial de usar
Senti-me muito confiante a usar o serviço
Preciso de aprender muito antes de poder usar este serviço
0 4
Discordo Muito Concordo Muito
EVALUATE SATISFACTION
DISCUSS AND IMPLEMENT
NO FORMAL REPORTS!!!
DISCUSS AND IMPLEMENT
User Test 1
User Test 2
User Test 3
User Test 4
Debriefing Session
DISCUSS AND IMPLEMENT
CONCLUSIONShttp://flickr.com/photos/rayvensmoon/1837103777/
LISTEN USERSRECRUIT WISELYAGILE APPROACH
TEAM WORKDESIGN FOR INTERACTION
CONCLUSIONS
Thoughts
http://flickr.com/photos/jazz_dalek/1845705965/
Thank you!
http://flickr.com/photos/42304632@N00/232004616/
Pedro Custódio
http://blog.centopeia.com