9/13/13

1

Improving ERP Usability Through User-System Collaboration

Tamara Babaian, Wendy Lucas,

Heikki Topi, and Jennifer Xu CIS Department, Bentley University

Presentation by Heikki Topi

University of Eastern Finland September 12, 2013

Background

¢  Enterprise Resource Planning (ERP) systems integrate business processes across the entire enterprise

¢  ERP tasks automate business processes that typically: l  Have well-defined hierarchical sub-task

structures l  Involve multiple users and system modules l  Can extend over days/weeks

¢  Many other types of large-scale enterprise systems share characteristics with ERPs

9/13/13

2

Background

¢  Anecdotal evidence of poor usability of ERP systems

¢  Poorly designed user interfaces have negative impact on bottom line due to: l  Increases in new user training time l  Decreases in end-user productivity l  Poor user adoption rates (from Forrester Research studies of enterprise

and business application usability - Ragsdale 2004; Herbert 2006)

Motivation

¢  Research initiative driven by lack of attention paid to addressing significant usability shortcomings of ERP systems by both manufacturers and the usability community

¢  Focus on understanding ways in which users interact with ERP software and degree to which interaction model supports tasks being performed

¢  Address gap between capabilities of ERP systems and how they meet each user’s objectives

9/13/13

3

Motivation

¢  User’s spend weeks (months) in training, yet often characterize ERP systems as ‘brutal,’ ‘unforgiving,’ and ‘unintuitive,’ because l  System’s logic (process and data flow) opaque

to users l  Relationships and interdependencies between

tasks difficult to determine l  Semantics of fields, forms, and interfaces hard

to understand l  Little if any support for error resolution

¢  Users rely on in-house made documentation, notes, and support by IT departments

Categorization of Usability Problems from a Field Study* ¢  Difficulty in identifying and accessing the

correct functionality ¢  Lack of transaction execution support ¢  System output limitations ¢  Inadequate support in error situations ¢  Incompatibility between the users’ and

the system’s terminology ¢  Usage-related problems arising from the

overall complexity of the system *Topi, Lucas, Babaian. Identifying Usability Issues with an ERP Implementation" in Proc. ICEIS’2005.

9/13/13

4

Broader Problematic System Characteristics ¢  Users lack an understanding of how their

narrowly defined tasks are related to the broader process in which they are taking part

¢  Underlying business processes and their relationships are hidden from users

¢  Data-to-process and process-to-process relationships too complex for a user to understand

¢  Local requirements lead to manual tailoring efforts that are very expensive, particularly over time

Typical ERP Screens

“Easy” Access Menus

Data entry form

9/13/13

5

User notes – example 1

User notes – example 2 –process description

and 11 more pages

9/13/13

6

General Approach

¢ Apply Collaboration Theory to the design and evaluation of ERP user interfaces l Collaboration between user and

the system, with system acting as a collaborative partner helping the user achieve system-related goals •  Note that this is not CSCW

Human-Computer Collaboration Theory*

¢  Commitment to Joint Activity (CJA) l  Each party in the interaction commits to the joint

activity and is aware of the context surrounding their collaboration

¢  Mutual Responsiveness (MR) l  Each party responds to the intentions and

actions of the other by adjusting his own behavior based on the behavior of the other’s

¢  Commitment to Mutual Support (CMS) l  Each party is committed to supporting the

efforts of the other when that party needs help and will recognize the need and provide assistance

*Bratman, 1992; Grosz & Kraus, 1996

9/13/13

7

Conceptual Model of System-User Collaboration

Current Research Program

¢ Three-year NSF grant (2008-2011) with a one-year no-cost extension (2011-12)

¢ Multiple research perspectives and methods

¢ Four faculty members (two with a CS background; two with an IS background) and two IS doctoral students actively involved

9/13/13

8

Three Broad Principles

¢ Greater collaborative strength of software yields greater usability

¢ Embedding knowledge about the users, tasks, processes, domains, prior usage, and characteristics of interface components into an ERP systems will strengthen its collaborative capabilities

¢ Usage logs are a true reflection of reality

Four Primary Methods

¢  Field studies (both interview and survey) to identify usability issues with existing ERP systems from the end users’ perspective

¢  Formulation of design principles based on user input and collaboration theory to guide interface design

¢  Applying design principles to the development of illustrative prototypes and evaluating them in laboratory studies

¢  Specifying new usability evaluation methodologies and evaluating them in the lab and in the field

9/13/13

9

Research Goal ¢  Improve the usability of ERP systems by

increasing the collaborative capabilities of their interfaces through: l  Development of design guidelines and

interface evaluation techniques based on collaboration theory

l  Implementation of prototype interfaces to validate design and evaluation methodologies

Related Research

¢  Human-Computer Collaboration l  Terveen (1995)

¢  Human factors issues and usability characteristics of ERP systems l  Bishu et al. (1999) l  Calisir and Calisir (2004)

¢  User-centered/human-centered design l  Maguire (2001)

¢  Activity modeling l  Constantine (2006): Activity theory & usage-centered design

¢  TAM, UTAUT l  Davis (1989), Venkatesh & al. (2003)

9/13/13

10

Sample Publications and Working Papers ¢  Xu, J. & Topi, H. “A Conceptual Model for Collaborative ERP

Systems,” Bentley University Working Paper. ¢  Lucas, W., Xu, J., and Babaian, T. Visualizing ERP Usage Logs

in Real Time. In ICEIS ’13. ¢  Lucas, W. and Babaian, T. “Implementing Design Principles for

Collaborative ERP Systems,” In DESRIST ‘12 ¢  Babaian, Lucas, Xu, and Topi, 2010,” Usability through System-

User Collaboration: Deriving Design Principles for Greater ERP Usability,” In DESRIST ’10

¢  Cooprider, Topi, Xu, Dias, Babaian, and Lucas, 2010, “A Collaboration Model for ERP User-System Interaction,” In HICSS ‘10

¢  Lucas and Babaian, 2009, “Reasoning for Intelligent System-User Interactions with Enterprise Resource Planning Systems,” IJCAI 2009 Workshop on Intelligence and Interaction

Connections with UEF Projects ¢ We believe that the key principles of

the human-computer collaboration model can be successfully applied to all types of enterprise systems

¢ We are interested in testing the ideas developed in the ERP context in other environments

9/13/13

11

Connections with UEF Projects ¢  “The main aim of DAVID is to enhance

corporate leadership by combining leadership technologies, findings of information systems research, visualization and language technologies.” (Dai, 2013, p. 77)

¢  These goals are, at the general level, not far from the goals of enterprise systems à could we apply HCC principles to evaluate and potentially improve DAVID?

Connections with UEF Projects ¢ Expansion of the usability concept

beyond user interface issues – our research encourages all designers to consider the collaborative relationship between the users and the system from very early stages of the design l Moving from ease of use to usability

and user experience

9/13/13

12

Example: Illustrations of CJA, MR, and CMS

CJA-User

¢  How do you know that this is your debit piece and this is your credit piece? You wouldn't know that until you walked through it all and then you understood conceptually. (need for commitment to learn)

¢  So, that's the one drawback I would say versus some other systems that's not very intuitive. So, you need to know the screens. And, you need to be trained in what the process is. (commitment to get training)

¢  No. No. I think it's just (up to) you to understand what you're trying to do and say 'Oh, yeah. Your (journal actions are going to have to be) balanced, of course. (commitment to certain subtasks)

9/13/13

13

CJA-User – Survey Items

¢  I try to stay aware of the progress of my tasks

¢  In general, I know how to complete tasks in <Enterprise System>

¢  I know how to perform my tasks with <Enterprise System>

¢  I know the business goals I am trying to achieve using <Enterprise System>

CJA-System

¢  And so, you can drill in here and see why you short-paid someone, or why you paid the amount you did. It makes sense. It's like a story when you go through them. (+)

¢  I think that the system is actually very helpful. This little help button right here, which will launch into either the definitions for where you are, if you're on a particular field, but then it will also launch into help with the overall applications, so I actually think it's very helpful. (+)

9/13/13

14

CJA-System

¢  User: Depending on what I'm going to try to do here. I forget if I'm in account payable or accounts receivable here. Interviewer: How can you tell if you need to know? User: I actually don't know. I'd probably just go back and then I'd look at the menu path that I was in. So here I'm in accounts receivable. (-)

¢  When you first see this layout stuff, it doesn't make sense. But then once you know what it is it does make sense. (-)

¢  Honestly, for me personally, every time I've gone and tried to drill in through the folders, I can never get to where I think I need to be. (-)

CJA-System – Survey Items

¢  <Enterprise System> always informs me of the progress of my tasks

¢  In general, I feel that <Enterprise System> knows how the tasks should be completed

¢  <Enterprise System> knows how business tasks should be completed

9/13/13

15

CJA-Total – Survey Items

¢  <Enterprise System> and I have a shared understanding about how to complete business tasks while we work on them together

¢  <Enterprise System> and I share goals for tasks that need to be completed when we work together

¢  <Enterprise System> and I share understanding about the progress of our tasks while we work on them together

MR-User

¢  If you're used to doing an upload of the same thing, you might have to manually type that in, if the upload doesn't work. You always have to adapt to the system.

¢  Interviewer: Are there any examples of situations where the system forces you to do things in a certain way that is different from your preferred way of doing it, accomplishing a certain goal? User: Geez, I don’t know. I think after all these years I think the system has just beaten me down.

9/13/13

16

MR-User – Survey Items

¢  I have learned how to use <Enterprise System> in order to adapt to its way of working

¢  I take into consideration how <Enterprise System> does things

¢  I understand how <Enterprise System> typically gets things done and adjust my work accordingly

MR-System

¢  The system just doesn't know what the problem is. It doesn't know how to diagnose those. So this one says, no suitable documents found. Message [XXXXX]. (-)

¢  I would say just getting rid of some of the fields that you don’t need to look at [for this task] instead of having to go through them all. It’s just a waste of time. (-)

9/13/13

17

MR-Systems – Survey Items

¢ <Enterprise System> is able to learn from how I work with it

¢ <Enterprise System> adjusts to the way I perform tasks

¢ <Enterprise System> takes into consideration my previous actions

CMS-System

¢  Yes, it can give you error signs, saying something is not right, or this field is incorrect, or this has already been inputted. (+)

¢  No, it doesn't give you like error messages that say hey, you need to do x, y, z. (-)

¢  The thing is that in this case [of getting lost], the system is not reaching out to you saying that you obviously need help. It’s me having to go find it. (-)

9/13/13

18

CMS-System – Survey Items

¢  In case of an error, <Enterprise System> provides mechanisms for fixing the problem

¢  <Enterprise System>'s error messages provide detailed guidance and instructions to help me solve problems

¢  <Enterprise System>'s error messages use terms that I can understand

CMS-User – Survey Items

¢  I recognize the error situations in which <Enterprise System> needs additional input from me

¢  I do my best to understand <Enterprise System>'s error messages in order to solve the problem

¢  I do my best to diagnose and solve problems in error situations

9/13/13

19

Example: Proposed Design Principles (Babaian, Lucas, Xu, and Topi, DESRIST 2010)

Design Principle 1

¢  DP1. The user interface should provide a mechanism for customizing the vocabulary of terms used by the system in its communication to the user, the composition of business transactions, and the content of the system's informational output to match the practices of the organization. There should be a mechanism for incorporating the customizations from an earlier version of the system to a later one.

9/13/13

20

Design Principle 2

¢  DP2. The system should provide navigational and progress guidance to a user performing a transaction, indicating the broader context of each interaction in terms of the related business process components and specifying the completed and remaining parts. A sufficiently competent user should be able to turn off this guidance if it becomes a distraction.

Design Principle 3

¢  DP3. When the system detects a problem, it should identify the possible causes and ways of resolving it. If the fix is obvious, the system should inform the user and perform it. If it isn't obvious, the possible causes and resolution scenarios should be presented to the user and be readily executable. If the system is unable to identify resolution strategies, it should present the user with the relevant data and transactions.

9/13/13

21

Design Principle 4

¢  DP4. In presenting selection choices, the system should utilize what it knows about the user, the organization, the task, and the context, and provide faster access to the more likely choices than the less likely ones. Where the choice of data or action is obvious, the system should have an option of not waiting for the user to enact it. The user should have an option to replace/cancel the system's provided choice of data/action.

Example: Moving towards a Collaborative Walkthrough / Collaborative Critique

9/13/13

22

Collaborative Critique

¢ A new usability walkthrough method ¢ Questions that the evaluators answer

were formulated from the human-computer collaboration paradigm

¢ The first version of the method has been developed and gone through the first round of testing

¢ CHI’12 Extended Abstract

Interface Evaluation: Cognitive Walkthrough* ¢  Will the user try to achieve the right

effect? ¢  Will the user notice that the correct action

is available? ¢  Will the user associate the correct action

with the effect trying to be achieved? ¢  If the correct action is performed, will the

user see that progress is being made toward solution of the task?

*[Wharton et al, 1994]

9/13/13

23

Viewing Cognitive Walkthrough Through a Collaborative Lens

¢  Based on the user’s overall goal, will the system recognize the next step in the process and either act to perform that step or, if the user’s input is necessary, present a set of alternative actions from which the user may make a selection?

¢  Does the system help the user identify the next action and present it in a highly visible manner?

¢  Does the system present a meaningful set of alternative actions based on the user’s overall goal?

¢  Will the system keep the user informed about the consequences of actions taken by either the user or the system, as they relate to progress made toward the achievement of the task?

Key Questions

¢  Does the allocation of responsibilities between the system and its user within the task align with their respective natural strengths?

¢  Is the system-user communication mechanism effective for both novice and experienced users?

¢  Does the system make effective use of its resources in the process of working on the task and communicating with the user?

¢  What are the system capabilities and information that could be added to improve interaction?

9/13/13

24

Example: Use of Process Graphs to Improve System-User Communication (Babaian, Lucas, and Topi, 2007)

Typical ERP task interface

9/13/13

25

49

Improving System-User Communication Via Process Graphs

Add Material

Edit Material

Add Vendor

Process Graphs Form Hierarchies

9/13/13

26

Benefits

¢  Improves system-user communication of system-prescribed processes and progress through task

¢ Simplifies navigation to related tasks ¢ Enables preview of future steps and

revisiting of previously completed steps

Example: Improving System-User Communication based on Usage Logs (Babaian and Lucas, 2011)

9/13/13

27

Improving System-User Communication via Usage Logs

¢  Embed model capturing relationships between users, actions, tasks, and processes into the system so that it can reason about itself and its users l  Capture usage data via input-aware

components à supporting users with basic system usage and in error situations

l  Ongoing usability evaluations l  Automated planning for diagnosing and

resolving errors

Usage Logs + Task-Interface Model

¢  Enables l  playback of past interactions l  usage log mining l  estimation of user’s experience level

¢  Makes relevant usage histories available to all users: l  Show how Jack posted a goods

receipt.

¢  Enables automated usage-based support mechanisms l  Show all account numbers used by

inspectors l  Which fields in the GR are always

filled?

Business Processes, Tasks

Interface components

Usage log Users

ERP database

9/13/13

28

Usage Log Examples

¢  Identify required fields based on prior usage ¢  Reconstruct completed processes

l  Remind user how process was completed before via a “playable” reconstruction process

l  Show user how others completed the same process

¢  Adapt level of instructional/error support based on user experience

Data Model

Represents and links: o  Domain data o  User data o  Task data o  UI data o  User input log

9/13/13

29

Automated Usability Evaluation ¢  Keystroke analysis

l  Components that are prone to user entry errors

¢  Time to complete a task l  Subtasks that are more difficult to complete

¢  Time between tasks l  Navigation difficulties

¢  Number of times a task is revisited l  Tasks requiring additional support

Concluding Observations ¢  Underlying theoretical framework as a strong guide for a

research program that approaches the same underlying research question from multiple perspectives

¢  A planned multi-method approach will build a much stronger evidentiary foundation than any single method would

¢  The empirical evidence gathered so far provides support for our key propositions, but getting to a point where we can conclusively reject alternative hypotheses will still take time

¢  Design principles and guidance for design of future enterprise systems will be in the center of the research program outcomes

¢  The benefits of a multidisciplinary research team are significant

9/13/13

30

Work that Continues in the Project ¢  More in-depth analysis of the interview data ¢  Further formalization of collaboration theory

applied to the ERP usability context ¢  Finalizing the survey instrument to enable

positivist theory testing ¢  Further development of collaborative critique for

evaluating interface usability ¢  Continuation of lab studies with experimental

collaborative interfaces ¢  Continued development and testing of the

prototypes ¢  Further articulation of the design principles

Acknowledgements

¢  This material is based in part upon work supported by the National Science Foundation under Grant No. 0819333. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.