Evaluating and Improving Software Usability
Post on 28-Oct-2014
Embed Size (px)
DESCRIPTIONPresented at Software Test Professionals, New Orleans 2012;Todays web-based applications (WebApps) containing complex business logic and which are sometimes critical to operating the business, now must have an increased focus on usability as well as the newer and broader term, user experience. Especially with SaaS based business models where users can switch applications at a heartbeat and pay by the month (or even a free trial), both usability and user experience become paramount as there is no up-front investment forcing a user to stay. The ISO 25010 standard describes a model for general usage in specifying and evaluating software quality requirements, one of which is usability, but there are no models or formal definitions for user experience, nor its relation to usability. For usability, ISO 25010 is intended as a general guideline to be adapted based on a specific context and lacks implementation specifics. UX, on the other hand, does not have any formal standard definition, although some models have been developed regarding its elements. Models and research have been used mostly for the purpose of understanding, rather than evaluating improvements. In this session, we draw relationships between usability and user experience and explore measurement and evaluation methods that can be used as the first step toward improvement.
<ul><li> 1. Evaluating andImprovingSoftware Usability 902 : Thursday, 9:30am - 10:45amPhilip Lewwww.xbosoft.com</li></ul> <p> 2. Understand, Evaluate and Improve 2 3. Agenda Introduc7on Importance of usability What is usability? What is UX? Usability Modeling and Measurements Case studies on measuring and improving Summary 4. What is Quality? Pertaining to soEware J Requirements 4 5. Importance of Usability 6. Why is Usability Important? Usability is important especially for web applica7ons And now especially mobile applica7ons Saas, mobile are in alignment with and soon becoming equivalent to cloud Most prevalent implementa7on method in the next 5 years Without good usability: Users will leave the applica7ons For mobile, if they cant learn in 30 seconds, they wont come back 7. Web and Mobile have Changed the Terrain Business models have changed Instead of paying upfront and owning the soEware Pay as you go, pay by subscrip7on Behavior and expecta7ons have changed 8. What is Usability In rela7on to quality In rela7on to user experience 9. When you think Usability Naviga7on Eciency Responsiveness-performance Learnability AZrac7veness Anything else come to Understandability mind? 10. Current Research Usability User Quality Experience Satisfaction UsabilityISO 9241-11 ISO9126-1UsabilityDesign ISO ExpertsProduct 25010Owners 11. Usability - Key Characteris7c of Product Quality Source: ISO 25010 12. Quality in Use ISO 25010 13. Notes on Sa7sfac7on and Usability dont havewhat I wantIm unsatisfied Sa7sfac7on is a subjec7ve feeling Highly usabledependent on many software things other than usability: My password doesnt work A user can be highly sa7sed but the applica7on with low Nice weatherIm usability. todaysatisfied! An applica7on can be ! highly usable (high low usabilitysoftwareusability) but the user is not sa7sed! Finishedmy work today 14. Usability-Major Component of Quality Quality and Usability in the so1ware development lifecycle Usability Usability Can be measured from the design point of view or of the product Can be measured in-use with real users 15. Evolu7on of SoEware quality ISO 9000ISO 9126 ISO 9241ISO 25010Software Software Software Type of ProcessQuality QualityQuality qualityquality(internal) (external) (in use)What ismeasuredSoftwareprocessesCodeProduct ?CMMIHowmeasured? assessment model white boxtestingblack box testing? CMMIWhoAssessment Programmer Tester End Usermeasures?Company 16. What is Usability-Eect of the SoEware Product Degree to which specifiedusers can achieve specifiedgoals with effectiveness,efficiency and satisfaction in aspecified context of use.Source: ISO 25010 17. Usability in Actual Usage User role specied users Objec7ve specied goals Task Environment specied context of use Domain What else can you think of? 18. Usability and User ExperienceUX, Usability and Quality in Use Difficult to understand their relationships Lack of consensus in meaning in literature Not totally clear how related to quality Standard definition for user experience is still not available 19. Some UX Deni7ons UX DESCRIPTIOND1 a continuous process of user engagement with the product D2 entire set of affects that results in user-product interaction D3 the evolution of usability D4 elaboration of the satisfaction component of usability D5 a categorization of do-goals (pragmatics) and be-goals(hedonics) , D6 infinite small experiences relating to people, products and contexts D7 consequence of users and products characteristics when interacted in a specific environment D8 degree to which specified users can achieve actual usability, safety, and satisfaction in use in a specified context of use D9 A persons perceptions and responses that result from the use or anticipated use of a product, system or service D10the degree of positive or negative emotions that can be experienced by a specific user in a specific context during and after product use and that motivates for further usage  20. User Experience (UX) Pragmatic (do-goals) : refers to the products perceived ability to support the task achievement and focuses on the products utility and usability in completing tasks, the do-goals of the user. Hedonic (be-goals) : products ability to support the users achievement of be-goals, such as being satisfied, pleasure. Real goal of the end user is fulfilling be-goals such as being autonomous, competent, related to others, stimulated, etc. Be-goals are the driver of UXHassenzahl 21. Rela7ng Usability and User Experience Quality 508 User Interface Func7onality Learnability characteris7c compliancy Usability Product Text/Sound Appropriate Easy to use Easy to learn aZributes aZributes Func7ons UX Pragma7c Eec7ve and Ecient Do Goals Usability Eec7veness in use Eciency in use Learnability in use Measurement UX Hedonic Sa7sfac7on (pleasure, sense of community) Be Goals Sa7sfac7on Sense of Pleasure Comfort Trust Measurement Community Context USER EXPERIENCE 21 22. Now that we understand it ALL We Need to Dene Usability For Our Own Purposes 22 23. Lets Dene Usability/UX From the Product Viewpoint Usability Characteris7c 1 Naviga7on Understandable Characteris7c 2 Characteris7c n Subcharacteris7c 1 Control Stability Subcharacteris7c 2 Previous-Next Subcharacteris7c n AZribute 1 Home Loca7on AZribute 2 Save Loca7on Ease of nding AZribute n 24. Dening Usability from an Eect-Real usage Point of View Usability Quality Eec7veness Eciency Sa7sfac7on Characteris7c n Accuracy Completeness Subcharacteris7c n Errors AZribute 2 AZribute 3 AZribute 1 25. Some Poten7al AZributes Measurements for in use Usability Eec9veness Comple7on rates Can you think Error rate of any others, Help usage par9cular to Eciency your applica9on? Task 7me Backtracking Learnability Learning rate Task 7me devia7on 26. Adding in UX Sa9sfac9on in use Degree to which users are sa7sed in a specied context of use. Sa7sfac7on is the users response to interac7on with the product, including amtudes towards use of the product. Subdivided into sub- characteris7cs: Likability (cogni7ve sa7sfac7on) UX Component Pleasure (emo7onal sa7sfac7on) Comfort (physical sa7sfac7on) Trust 26 27. Example of Product Quality Model with AZributes (Operability in ISO 25010) External Quality Requirements (for Shopping Cart Entity) 1 Usability 1.1 Understandability 1.1.1 Icon/label ease to be recognized 1.1.2 Information grouping cohesiveness 1.2 Learnability 1.2.1 .. 1.3 Ease of Use 1.3.1 Control permanence 1.3.2 Helpfulness 28. Learnability Degree to which the soEware product enables users to learn its applica7on - 7 principles (ISO 9241-10) Suitability for the task - should be suitable for the users task and skill level Self-descrip7veness- should be clear what the user should do next Controllability - the user should be able to control the pace and sequence of the interac7on Conformity with user expecta7ons - should be consistent Error tolerant - forgiving Suitability for individualiza7on - should be able to be customized to suit the user Suitability for learning - should support learning 28 29. Ease of use Degree to which the soEware product makes it easy for users to operate and control it. controllability error tolerance (by operator) conformity with user expecta7ons 29 30. Ease of Use - Helpfulness Degree to which the soEware product provides help when users need assistance including help that is: easy to nd comprehensive eec7ve 30 31. Example: Usability Measurement ACribute Scale HowCalcula9on Goal CurrentHelp Percent of Compare %90%40%completeness Menu items menus and with help items helpEase of Keystrokes Sample 50 average3.512access to items nd/use a feature/ func7on/ informa7onConsistencyNumber Examine integer15 loca7ons for menus and same buZon doc. What types of usability measures are these?31 32. Using a measurement model Evalua7on Usability composed of Indicator Metrics Evalua7on Evalua7on Characteris7c Measurement composed ofFunc7on Evalua7on Evalua7on Measurements Sub-characteris7cs 32 33. Usability-Review usability usability in use UX Can be measured from the design point of view or of the product Can be measured in-use with real users 34. Measuring Usability Methods and models Now that we know what it is, and what to measure, then HOW? 34 35. Measurable AZributes Usability characteris7c Descrip7on and purpose Once you have a model (what you How to measure are going to What is measured measure), then Measurement/Calcula7on you start doing IT! Range (min, max) Metric Objec7ve (goal) versus Current (indicator) 36. UX/Usability Measurement Methods Focus groups Ques9on naires Observa9on Heuris9c Labs Evalua9on Logging Lets get started 37. Measurement methods Expert Evalua9on (Heuris9c evalua9on) Dene a set of rules or criteria with measurements and evaluate against them Web-based logs Collect user ac:vity data Mistakes and errors How long it takes to complete tasks Comple:on rates Ques9onnaires Quan7ta7ve subjec7ve measurement of UX characteris7cs 37 38. Heuris7c Evalua7on Using a checklist But a liZle more complicated than yes/no 38 39. Heuristic Evaluation of a PharmacyapplicationEach usability attribute has a quantifying metric. For example, for Predic9ve textual aCribute, users should be able to understand a buCons results prior to pressing it. Direct metrics need to be designed, i.e.: (0) No support at all (1) Par9al (2) Complete Need a mapping from 0, 1, 2 to something more understandable, i.e. 2 = 100, and 1 = 60 Need an indicator to interpret the level of sa9sfac9on met with decision criteria with acceptability ranges in a percentage scale: 0-40 (unsa9sfactory red) means changes must take place with high priority; 40-70 (marginal yellow) indicates a need for improvement ac9ons; 70-100 indicates a sa9sfactory level green- for the analyzed aCribute. 39 40. 2.2.2 Error Recovery Support" In the current state, users filling a new prescription aresupported well in error recovery (2.2.2) in automatic cursorpositioning 40 41. Heuristic Evaluation Pharmacy SoftwareThe purpose of the evaluation Understand the external quality level of the learnability sub-characteristic for filling a new prescriptionTable 2 - excerpt of the whole current evaluation 41 42. Heuris7c Evalua7on of Usability-Shopping Cart External Quality Requirements Measure EI value P/GI value Global Quality Indicator61.97% 1 Usability 60.88% 1.1 Understandability83% 1.1.1Icon/label ease to be recognized100% 1.1.2Information grouping cohesiveness66% 1.2 Learnability 51.97% 1.2.1 1.3 Operability 49.50% 1.3.1Control permanence100% 1.3.2Expected behaviour 50% 2 Content Quality 63.05% 2.1 Content Suitability 63.05% 2.1.1Basic Information Coverage50% 184.108.40.206 Line item information completeness250% 220.127.116.11 Product description appropriateness50% 2.1.2Coverage of other Contextual Information 76.89% 18.104.22.168 .. 22.214.171.124 Return policy information completeness 33% 43. JIRA Heuristic Usability Evaluation 44. Usability Logging Collec7ng user ac7vity and behavior Tradi7onally used for other things like 44 45. Usability Logging Iden7fy users by using session ID to iden7fy a unique user. Itera7vely insert code into the applica7on Collect data Analyze the data for each aZribute in dierent dimensions and aggrega7ons Determine the need for further calcula7ons and what aZributes to measure further Revise the data we are collec7ng, adding or decreasing granularity 46. Some easy and some hard 46 47. Jira Evaluation Summary47 48. Using Both Product Perspec7ve and the User Perspec7ve If we nd some aZribute with low performance from the user perspec7ve Then we can go back to the product perspec7ve, gure out why, and improve 48 49. JIRA Evaluation by Attribute Excerpt 49 50. For example:Login BuZon Visibility (product usability aZribute) 50 51. Using Ques7onnaires The process: 1.Use your model of what you want to measure and improve 2.Design the ques7onnaire according to the model 3.Execute the ques7onnaire 4.Analysis 52. One UX Model 53. Ques7onnaires Used SUMI (So1ware Usability Measurement Inventory ) QUIS (Ques:onnaire for User Interface Sa:sfac:on ) PSSUQ (Post-Study System Usability Ques:onnaire ) ASQ (A1er-Scenario Ques:onnaire ) PUTQ (Purdue Usability Tes:ng Ques:onnaire ) PHUE (Prac:cal Heuris:cs for Usability Evalua:on ) SUS (System Usability Scale ) IUI (Isometrics Usability Inventory) 54. 54 55. 55 56. What do you do with all this? 56 57. Itera7ve Improvement Measure external quality usability with heuris7cs Measure in-use usability with logging or ques7onnaires 58. Goal is to Understand, Evaluate and Improve 58 59. Next Steps Produce an ac7on plan What usability aZributes are important to your organiza7on? Develop a model What data can you collect/Which technique can you use Maybe some elements of the model drop out-cant be measured that easily Start collec7ng and developing benchmarks 60. Conclusion Usability and UX are both abstract concepts Cri7cal component of quality Dening is dierent for each organiza7on Need a model for your organiza7on The model is the founda7on of what to measure Once you can measure, then you can evaluate and improve 61. Resources These are all journal publications that canbe accessed via these links61 62. Thanks Ques7ons and Answers Please ll out an evalua:on form and drop it in the collec:on basket located at the back of the room. Philip Lew @xbosoE philip.lew@xbosoE.com phone: 408-350-0508 www.xbosoE.com </p>
View more >