international journal of human-computer interaction ... · human-machine interfaces for life...

PLEASE SCROLL DOWN FOR ARTICLE

This article was downloaded by: [Kaber, David B.]On: 24 May 2011Access details: Access Details: [subscription number 937423582]Publisher Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Human-Computer InteractionPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t775653655

Assessing Usability of Human-Machine Interfaces for Life ScienceAutomation Using Computational Cognitive ModelsDavid B. Kabera; Rebecca S. Greena; Sang-Hwan Kima; Noa Segallb

a North Carolina State University, Raleigh b Duke University Medical Center, Durham, North Carolina

Accepted uncorrected manuscript posted online: 10 February 2011

Online publication date: 09 May 2011

To cite this Article Kaber, David B. , Green, Rebecca S. , Kim, Sang-Hwan and Segall, Noa(2011) 'Assessing Usability ofHuman-Machine Interfaces for Life Science Automation Using Computational Cognitive Models', International Journalof Human-Computer Interaction, 27: 6, 481 — 504, doi: 10.1080/10447318.2011.555295, First posted on: 10 February 2011(iFirst)To link to this Article: DOI: 10.1080/10447318.2011.555295URL: http://dx.doi.org/10.1080/10447318.2011.555295

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.

http://www.informaworld.com/smpp/title~content=t775653655

http://dx.doi.org/10.1080/10447318.2011.555295

http://www.informaworld.com/terms-and-conditions-of-access.pdf

INTL. JOURNAL OF HUMAN–COMPUTER INTERACTION, 27 (6), 481–504, 2011Copyright © Taylor & Francis Group, LLCISSN: 1044-7318 print / 1532-7590 onlineDOI: 10.1080/10447318.2011.555295

Assessing Usability of Human–Machine Interfacesfor Life Science Automation Using Computational

Cognitive Models

David B. Kaber1, Rebecca S. Green1, Sang-Hwan Kim1,and Noa Segall2

1North Carolina State University, Raleigh2Duke University Medical Center, Durham, North Carolina

The objective of this study was to assess the plausibility of using computationalcognitive models for evaluating the usability of human–machine interfaces in super-visory control of high-throughput (biological) screening (HTS) operations. Usabilityevaluations of new interface prototypes were conducted by comparisons with exist-ing technologies. Model assessment occurred through comparison with human testresults. Task completion times and the number of errors were recorded during humanperformance trials, and task time was predicted in cognitive model trials in testswith two HTS interfaces. Computational GOMSL (Goals, Operators, Methods, andSelection rules Language) models were constructed based on a combination of cog-nitive task analyses (abstraction hierarchy modeling and goal-directed task analysis).The usability tests revealed improvements in task performance with the new proto-types. The cognitive model outputs were correlated with actual human performance,and the approach was considered useful for evaluating the usability of new interfacesin life sciences automation in the future.

1. INTRODUCTION

The life sciences domain presents many opportunities for human–automationinteraction research. Since the early 1990s, high-throughput screening (HTS)

This research was supported by a National Science Foundation (NSF) Information TechnologyResearch Grant (no. 046852). Ephraim Glinert was the technical monitor for the NSF. The viewsand opinions expressed in this paper are those of the authors and do not necessarily reflect theviews of the NSF. We thank the University of Rostock and CELISCA for providing us with accessto high-throughput biological screening systems and supporting the research through allocation ofbiopharmacologist and process engineer time to the effort.

Correspondence should be addressed to David B. Kaber, North Carolina State University,Department of Industrial & Systems Engineering, 400 Daniels Hall, 111 Lampe Drive, Raleigh, NC27695-7906. E-mail: [email protected]

Downloaded By: [Kaber, David B.] At: 05:42 24 May 2011

482 Kaber et al.

processes identifying biological and/or chemical compounds for drug-derivativedevelopment have become increasingly automated. In such processes, lab oper-ators typically interact with multiple robotic systems for materials handling,liquid transfer, and so on, as well as automated analytical measurement systems.Operator requirements are twofold: (a) plan and program HTS experiments basedon the design of previous manual assays, and (b) monitor automated systems forerrors, such as robot positioning and material handling errors. The latter require-ment is critical because system failures are very costly to test facilities in time andresources. In general, these tasks pose high workload for workers, and their inter-action with screening systems needs to be effectively supported through controlinterfaces.

One technique for evaluating the detailed procedural knowledge necessaryto operate an interactive system and the implications on human performance isGOMS (Goals, Operators, Methods, and Selection rules) modeling. GOMS modelswere first proposed by Card, Moran, and Newell (1983) to describe how opera-tors carry out tasks within a certain system. Card et al.’s (1983) Model HumanProcessor is the cognitive architecture underlying GOMS models. It describeshuman information processing in terms of basic perceptual, cognitive, and motorabilities (Kieras, 1997; Olson & Olson, 1990). The processor is also used to predicthow long it would take an experienced user to complete a task, based on estimatesof execution times of plan retrieval from long-term memory, method selection (asa function of task features), working memory access, and motor movement exe-cution (Olson & Olson, 1990). Time estimates are based on previous psychologyresearch (Card et al., 1983) on, for example, working memory search as well aslaws of human motor control (e.g., Fitts, 1954). Prior research has also shown thatGOMS models can be used to predict the amount of learning required to performa task (Kieras, Meyer, Ballas, & Lauber, 1999) and, in general, task complexity (Dix,Finlay, Abowd, & Bealle, 2004), and imprecision in human judgment (Karwowski,Kosiba, Benabdallah, & Salvendy, 1990). It has also been suggested that GOMSmodels can be used to predict some aspects of usability of designs and may effec-tively complement user testing in early design stages, reducing the amount oftesting that may be necessary for usability evaluation as well as development costs(John & Kieras, 1996; Kieras, 1999). GOMS has been successfully used to modelhuman interaction with many real-world applications, from a television on-screenmenu interface to a command and control database system for space operations(John & Kieras, 1996).

In this research we sought to assess the plausibility of using computationalGOMS Language (GOMSL; Kieras, 1999) models for evaluating various aspectsof usability of new interface design for HTS applications, as compared to usingdesigned experiments and human performance test results. Historical GOMSmodels are written in pseudo code or natural language. GOMSL models are actualcomputer code that can be compiled and run to provide a simulation of humanbehavior. The overarching goal was to determine whether GOMSL modeling is atool that could be used for future interface design evaluation in the life sciencesdomain.


Human Interaction with Life Science Automation 483

1.1. Target Domain

Contemporary HTS processes involve chemical-based assays of organic and inor-ganic compounds for effects on human cellular functions, or enzyme reactions thatare common in cells (Entzian et al., 2005). Among the uses of HTS is the testing ofcompounds that may have the potential to serve as bases for drug derivatives usedin future medications for treating cancer, viruses, and so on. Specialized biologicalscreening tests, such as Trypsin (enzyme) screening, are developed by scientistsand published in the biotechnology literature (Thurow, Göde, Dingerdissen, &Stoll, 2004). “Bench-top” versions describe how to manually perform a particularscreening assay. Biopharmacologists then adapt the manual protocol to automatedHTS lines (Thurow et al., 2004).

Automation of a screening assay typically includes integration of an opti-mized robot for chemical analysis (ORCA) with analytical measurement deviceson a process line (see Figure 1). The ORCA, which is programmed, con-trolled, and monitored from a central process control system (PCS), transportsmicroplates to and from the various workstations on the screening line. Theautomated devices on the line include a barcoder for labeling and tracking themicroplates, a pipetting or liquid transfer robot (e.g., Biomek FX®; Beckman-Coulter Corporation, Brea, CA) for filling microplates with liquids (enzymesubstrates, compound extracts, and other reagents), an incubator for bringingthe temperature of reactions in the microplates to normal human body tempera-ture, and an automated microplate reader (FLUOstar®; BMG Labtech Corporation,Offenburg, Germany) for analyzing cellular or enzyme reactions to compoundsin each plate well (e.g., luminance or fluorescence tests of light reflected offsamples).

FIGURE 1 An ORCA® traversing a rail to convey a microtest plate between theincubator station and Biomek® FX (pipetting robot) (color figure available online).


484 Kaber et al.

1.2. Current Interfaces and Hypotheses

The process by which devices, such as the barcoder and incubator, are integratedfor specific HTS methods and operations are sequenced occurs through the PCSusing a screening line “method editor.” For example, Beckman-Coulter currentlypublishes the SAMI® (SagianTM Automated Method Development Interface) soft-ware, which allows a biochemist to develop a graphical process flow chart fora screening assay to be conducted on an HTS line. Screening methods are con-structed at the user interface by dragging icons representing available HTS devices(which will be used in the assay) to a central “scratch pad” and connecting themwith arrows to define transportation of microplates among the device worksta-tions by the ORCA (see Figure 2). Icons representing different types of microplatesare also available to further specify a method. Lab operators may also use pro-prietary control software to program certain devices on the screening line (e.g.,the pippetting robot, plate reader). These devices can then be integrated in thescreening process with the method editor and the PCS. Furthermore, the SAMIsystem includes standard Windows action/configuration dialogs for all deviceson the HTS line, which are used to set device parameters and to send data to

FIGURE 2 SAMI® method editor interface by Beckman-Coulter (color figureavailable online).



FIGURE 3 Barcoder action/configuration dialog as part of SAMI® interface (colorfigure available online).

device software modules via an executive controller. The main screen of the PCSaction/configuration dialog for a barcoder device can be seen in Figure 3.

For biopharmacologists, who may prefer to have samples (or microplates thatcontain samples), versus HTS devices, at the “center” of the method-developmentprocess, this approach to programming the system may be less intuitive. For thisreason, we prototyped enhanced human–machine interfaces to support HTS oper-ator planning, control, and analysis (Kaber, Segall, & Green, 2007). To addressthe larger study objective, we then assessed the usability of the prototypesthrough human performance trials and with computational GOMSL models.We hypothesized that a “process-centered” design approach to the new inter-face prototypes (focusing on the microplates and what is to happen to them)would lead to increased usability, as indicated by human performance during


486 Kaber et al.

test trials, involving target tasks as compared to performance with the existinginterface designs focusing on device configuration. Because GOMSL models rep-resent expert, error-free performance, usability of an interface is assessed based onthe level of task complexity and performance efficiency (time-to-task completion).The accuracy of model predictions may be limited for novice user performance. Inits current form, the modeling approach does not permit usability assessment interms of potential user errors.

We developed GOMSL models for representing lab operator interface behav-iors with the existing and new HTS control interfaces and made comparisons ofthe human data with model output, when the models were applied to the sametasks the humans performed. Kieras and Meyer (1995) previously demonstratedthat GOMSL models could be applied to highly complex and dynamic tasks (e.g.,a simulation of a tactical military task) that lend themselves to decompositionin terms of procedural (planning-based) methods, which are applied at specifictimes during performance. The task requirements of HTS lab operators initiallyappear highly complex and intense but can be considered to involve proceduralknowledge (common sequences of action) that can be dynamically constructedin memory and applied in real-time to an ongoing process. Consequently, weexpected our comparisons of the model output and human performance data toserve as a valid basis for evaluating the GOMSL models and their potential use as atool for supporting design changes in new prototype interfaces in the HTS domain.

2. METHODS

2.1. Interface Prototyping

To address the potential for costly errors in HTS processes, we previously eval-uated an automated HTS line with the goal of identifying functional limitationsand usability problems in existing software control interfaces (Kaber, Segall,Green, Entzian, & Junginger, 2006). We employed Cognitive Task Analysis (CTA)techniques, including Goal-Directed Task Analysis (GDTA; Endsley, 1993) alongwith Abstraction Hierarchy (AH) modeling (Rasmussen, 1985), to characterizethe knowledge structure of biopharmacologists in HTS operations as well asthe lab automation and equipment used. An AH model is the representationof a work system along multiple levels of abstraction (Rasmussen, 1985), rang-ing from physical component properties to system purposes, which are linkedin the model based on means–end relationships. AH models have been used forsystems analysis and design and as a framework for describing control tasks nec-essary to maintain adequate system performance. We developed AH models forall automated devices integrated in the HTS line under study.

GDTA is a methodology that focuses on identifying operator critical decisionsand situation awareness requirements relevant to performing complex systemscontrol (Endsley, 1993). GDTA uses a diagram format similar to hierarchical taskanalysis but concentrates on the organization of an operator’s goal set and noton low-level operations with system interfaces. The outcomes of GDTA includeperceptual requirements for task performance and operator needs in terms of



system-state comprehension and projection. The results can be used as a basis fordisplay design, training program development, and operator selection (Usher &Kaber, 2000). We developed a GDTA describing biopharmacologist goals, tasks,decisions, and information requirements relevant to supervisory control of theHTS line.

The combination of AH models describing screening line devices and automa-tion with the results of the GDTA identifying HTS operator tasks and decisionrequirements provided a meta-method for determining the extent to which HTSline automation supported operator task performance. Comparisons of the GDTAand AH model results were used to formulate interface design and automationfunctionality recommendations for enhancing the existing software applicationsused in the HTS process we studied (Kaber, et al., 2006). Operator informationrequirements for task decisions, as identified through the GDTA, were comparedwith specification of current interface action sequences and display content basedon the AH models to identify potential usability issues with the existing software.That is, we were able to determine whether particular interface displays and actionsequences led to the information operators needed for certain process decisions.The recommendations we made were targeted at usability goals based on Nielsen(1993) and Norman’s (1995) work, including providing a good match betweenthe system and the real world, promoting operator recognition of system statesrather than recall, enhancing the visibility of system status, preventing errors, anddeveloping a parsimonious interface design.

We used the recommendations from our previous work to develop new pro-totype designs for both the screening method editor software used with the HTSline and a control dialog for a barcoder device. We used an adaptation of Nealeand Carroll’s (1997) approach to metaphor-based design, integrating the CTA tech-niques just described, as a framework for developing the new screening methodeditor prototype with a format analogous to the presentation of a recipe in acookbook. The strong similarity in the structure of the two concepts (cookingand bench-top assaying) led to our use of the cookbook metaphor for redesign-ing the interfaces of the method editing software originally created for the HTSdomain (Kaber et al., 2007). Semifunctional prototypes were developed in Javato facilitate usability testing with actual operators and GOMSL cognitive models.For comparison purposes, we also developed Java prototypes of both the origi-nal SAMI method editor and the barcoder action/configuration dialog developedby Beckman-Coulter. These prototypes were exact visual replicas of the actual soft-ware and were used as surrogates to the Beckman-Coulter applications in our testsbecause of access limitations to the software integrated with the actual screeningline. The prototypes were also programmed to allow us to automatically collectuser time-to-subtask completion and the number of interface errors committed insimulated HTS task performance.

The new process-oriented method editor prototype integrated a menu bar andtoolbar, an assay name field, and four windows: an Assay Procedure Window,Process Flow Chart Window, Tools Window, and “Ingredients” Window (seeFigure 4). The prototype interface requires a user to initially describe an assayprocedure rather than identify the devices used to conduct screening. This isdone by entering text into the Assay Procedure Window, like writing a cooking


488 Kaber et al.

FIGURE 4 “Cookbook” method editor prototype (color figure available online).

recipe. The numbers of devices and amounts of materials to be used in the pro-cess are updated in the Tools and Ingredients windows of the interface on thebasis of the assay description entered by the user. Certain keywords in the text(recognizable by the system) that represent objects, such as devices, equipment,or materials, become hyperlinks that provide access to additional dialogs allow-ing for detailed levels of process specification. For example, “yellow” hyperlinksrepresent partially defined objects for which the system needs some additionalinformation from the user; however, even if this information is not provided, themethod will run on the HTS line using default values. Clicking on any hyperlink(in the Assay Procedure, Process Flow Chart, or Tools windows) will open a deviceor material configuration dialog. After a device (or ingredient) is configured, theassociated hyperlink (in the Assay Procedure Window) turns “green.” The proto-type configuration dialog for the barcoder can be seen in Figure 5. Tables 1 and 2provide a summary of the design changes implemented in the new method editorand barcoder dialog prototypes, based on the recommendations form our earlierstudy.



FIGURE 5 New barcoder interface (color figure available online).

Table 1: Summary of Design Changes for the Barcoder Dialog

1. Used terminology that is familiar to users (e.g., “128 character label” vs. “Code 128”).2. Reduced Action Selection (Manual, Automated, and Read) options.3. Provided explanations of different options in dialogs (e.g., by right-clicking on the option) for

inexperienced users.4. Dialog suggests (default) label content and position.5. Reduced dialog manipulations for actions that are not part of the user’s task.6. Graphic-oriented manipulation of dialog components (e.g., types of labels, plate orientation for

labeling).

Table 2: Summary of Design Changes for the Method Editor

1. Allows for direct description of assay procedure rather than configuration of devices to defineprocess.

2. Interface indicates different levels of configuration specification required during stages of assayprogramming.

3. Provided explanations of different options in dialogs for inexperienced users.4. Automatically updated numbers of devices and amounts of materials to be used in process (as

described by user).5. Provided overall view of assay tasks during programming process (e.g., flow chart).6. Graphic-oriented configuration of tools and ingredients.

2.2. Empirical Study Procedures

Five domain experts (biopharmacologists and process engineers) were recruitedfrom the Center for Life Science Automation (CELISCA) at the University of


490 Kaber et al.

Rostock, Germany, to evaluate the new prototype interfaces and the prototypesof the existing SAMI interfaces. (A small sample size was used because access toexperts in this domain is very limited.) Three male and two female participantsranged in age from 31 to 41. They completed scenarios for configuring and pro-gramming a new assay procedure for an HTS line. They were all experts in usingthe existing SAMI interfaces but were novices in using the new prototype inter-faces. In addition to our work with the experts at CELISCA, five graduate studentsfrom North Carolina State University, between 23 and 40 years old, were recruitedto evaluate only the SAMI method editor interface. Four of the students weremale. They were all novices at using the software, but some had previously seenthe application interface. We made a comparison of the new prototype interfacedesign, as evaluated by the CELISCA personnel, with the original SAMI methodeditor interface evaluated by the students in terms of errors committed with theapplications. Therefore, the two user groups were inexperienced with at least oneof the interfaces they evaluated. However, the CELISCA personnel were highlyexpert in the domain.

At the beginning of the study, each participant was provided with an intro-duction to the interfaces they were to evaluate and their purpose. They were alsogiven brief user manuals for the prototypes. Subsequent to familiarization withthe interfaces, the participants were given step-by-step instructions for followingthe task scenarios, including brief explanations of the purpose of each step. Duringtheir completion of the scenarios, times to subtask completion and the number andtypes of interface errors were recorded by the Java software.

We specified two objective usability goals for evaluating the new interfaceprototypes and for making comparison of the GOMSL models with human per-formance data. These included (a) increased efficiency with the new interfaces(i.e., a shorter task completion time), and (b) enhanced effectiveness (i.e., a smallernumber of errors). The goal of improved efficiency was expected to be influencedby the development of system components that support the user’s tasks (e.g.,Recommendations 1 and 6 in Table 1). The goal of increased effectiveness wasexpected to be related to development of system components that support bothnovice and expert user understanding of tasks (e.g., Recommendation 5 in Table2). This usability goal could be evaluated only through the empirical study, as theGOMSL models were only capable of producing task execution times.

Two scenarios were completed by the participants: (a) selecting and configuringa barcode label for application to microplates, and (b) HTS method programmingin which the operator defined an assay procedure, configured a barcoder, config-ured an incubator, and programmed microplate use. Both of these scenarios werebased on actual programming tasks performed by operators of HTS systems. Thescenario-based evaluations allowed us to assess whether users of the new pro-totype made fewer errors and were able to complete the scenarios in less timethan users of the existing interface. Erickson (1990) encouraged the use of scenar-ios for evaluating different interface alternatives involving different interactionmethods when performing similar tasks. Although the scenarios used with eachof the interfaces were designed to be similar in steps, task performance with thenew prototype and the SAMI editors was not identical (from a functional perspec-tive) because of certain interface features. For example, to configure the incubator,



with both interfaces, users were able to set the incubation time. However, in theredesigned prototype interface, the incubator dialog also allowed users to setadditional variables, such as temperature and incubator gas mixture, which theSAMI interface does not include. (In the existing HTS system at CELISCA, set-ting these variables is done at the physical incubator control panel.) Thus, morescenario steps were required for interacting with the incubator dialog in usingthe new interface than in using the existing SAMI interface and did not allowfor an exact efficiency comparison of these specific tasks in the two interfaces.Therefore, we only made a comparison of performance results for the tasks thatwere functionally similar across both the new prototype and SAMI interfaces.These included (a) assay development, which involved dragging and droppingdevices in SAMI and text entry in the new prototype method editor interface, and(b) barcoder configuration, which involved selecting a label location and contentin both interfaces.

2.3. GOMSL Approach to Cognitive Modeling

To assess the potential utility of cognitive models for describing HTS operatorbehaviors and for conducting HTS interface evaluations, two GOMSL modelswere developed for application to the SAMI and new prototype interfaces in eachof the test scenarios. The same tasks presented to expert biologists in the empiricalstudy were used as a basis for developing the cognitive models for the two typesof interfaces. There are four common code structures used in GOMS modeling:

● Goals. A state of affairs to be achieved.● Operators. Perceptual, motor, or cognitive actions that the user executes.● Methods. A list of steps necessary to accomplish a goal; a conditional

sequence of goals and operators.● Selection rules. Route control to the appropriate method using if-then state-

ments.

GOMSL is a variant of GOMS, which uses a structured notation (computer code) inwhich methods contain both external operators (physical actions at interfaces) andinternal operators (e.g., used to add or remove content from working memory).“Fully-specified” GOMSL models can also include definitions of visual objects (toappear on screen) as well as tasks that a user is to perform.

Because the process of programming the method editor or configuration ofthe barcoder device requires perceptuo-motor control, knowledge-based judg-ments, and decision making, many different types of GOMSL operators wereused to model the test scenarios. Operator interface behaviors were identified toinclude visual search and action with a mouse or keyboard. Cognitive behaviorsincluded memory storage and retrieval, comparison judgments, and method selec-tion. Table 3 presents the general human information-processing components ofthe HTS operations and some of the GOMSL operators used to represent them.In general, the modeling approach provides descriptive and predictive analysiscapability for a range of physical and cognitive behaviors.


492 Kaber et al.

Table 3: GOMSL Operators Available for Different Human Information-ProcessingComponents

Human Information-Processing Components GOMSL Operators

Perception Visual Look_for_object_whose Label is <object>and_store_under <WM>.

Cognition Memory storage and retrieval Store <value> under <WM>

Get_task_item_whose <property> is<value> and_store_under <WM>.

Recall_LTM_item_whose <property> is<value>..and_store_under <WM>

Comparisons is, is_equal_to, is_not, is_greater_than, etc.Decision making Decision Decide: If <predicates> then <operators>

Selection Rules Selection_rules_for_goal: <rule name>

If <condition> of <predicate> is<value>, then Accomplish_goal:<sub-method name>

Action implementation Manual Keystroke <key_name>

Type_in <String of characters>Click <mouse_button>

Point_to <target_object>

Note. GOMSL = Goals, Operators, Methods, and Selection rules Language.

In modeling the HTS scenarios, all sequential subtasks were coded as“Task_items” in the GOMSL models, and each item included an identificationnumber, type of action, target object, and pointers to the next task item. The mainmethod of the GOMSL models was designed to read the properties of the taskitems and to send them to appropriate submethods for processing. The submeth-ods consisted of sequential unit operators, such as pointing, clicking (pressing amouse button), or typing, to address the associated tasks (clicking on an interfacebutton, dragging an icon, typing a string, etc.).

Based on our observations of expert HTS operator performance at CELISCA,we assumed this strategy to task processing for the model development. Any dif-ferences between the cognitive models for representing SAMI interface behaviorsand user behaviors with the new prototype interfaces (for either the barcoder con-figuration task or the method editing task) can be found in the unit task operationsin each model. Figure 6 shows an example of the GOMSL model code for the SAMImethod editor interface containing two of the user tasks to be performed, the mainmethod, a selection rule for submethods, and a submethod for clicking on objects.

To compile and execute GOMSL models of human behaviors and to conductstatic and dynamic evaluations of task interfaces, such as learning assessments andtime-to-task completion, Kieras et al. (1999) developed GLEAN (GOMS LanguageEvaluation and Analysis tool). GLEAN runs on a PC and accepts GOMSL code asinput human behavior for simulation purposes. Like other classic programminglanguages, a model can be created and edited in any text editor and submit-ted to the GLEAN compiler. In compiling GOMSL models, GLEAN supports aset of psychological constraints based on Meyer and Kieras’s (1997) theory of



Define_model: "Integrating SAMI Interface"Starting_goal is Configuring SAMI.

Task_item: T38 Number is "38". Type is Drag_into_Center. Label is "Home". Next is "39". Target is "Center". Task_item: T39 Number is "39". Type is Draw_Line. Next is "End". Target1 is "Biomek_Center". Target2 is "Home2_Center".

Method_for_goal: Configuring SAMI Step 1. Store "1" under <current_task_number>. Step 2. Decide: If <current_task_number> is "End", Then Return_with_goal_accomplished. Step 3. Accomplish_goal: Read Scenario. Step 4. Get_task_item_whose Number is <current_task_number> and_store_under

<current_task>. Step 5. Accomplish_goal: Perform Unit_Task. Step 6. Store Next of <current_task> under <current_task_number>. Step 7. Goto 2.

Selection_rules_for_goal: Perform Unit_Task If Type of <current_task> is Drag_into_Center, then Accomplish_goal: Drag Icon using Label of <current_task>. If Type of <current_task> is Draw_Line, then Accomplish_goal: Draw Line using Target1 of <current_task>. If Type of <current_task> is Click_for_Popup, then Accomplish_goal: Configure Select usingLabel of <current_task>. If Type of <current_task> is Click_Object, then Accomplish_goal: Clicking Object usingTarget of <current_task>. If Type of <current_task> is Typing_String, then Accomplish_goal: Typing String using Target of <current_task>. Return_with_goal_accomplished.

Method_for_goal: Clicking Object using <object>Step 1. Look_for_object_whose Label is <object>and_store_under <target>. Step 2. Point_to <object>. Step 3. Click <object>. Step 4. Return_with_goal_accomplished.

FIGURE 6 Selected code from GOMSL model for SAMI® method editor interface.

Executive Process Interactive Control. For example, values of task variables mustbe retrieved from either long-term memory or a Task definition, as part of aGOMSL model, before related processing can occur. Otherwise, a model compila-tion error will be generated. When a compiled model is executed with GLEAN, thetool assigns a default execution time of 50 ms for every GOMSL statement. Extratime is added for certain operators; for example, a mouse click requires 200 ms.Again, these estimates are primarily based on prior research by Card et al. (1983).GOMSL also uses Fitts’ (1954) law to estimate discrete movement times. Statement


494 Kaber et al.

and operator times are totaled to determine overall task completion times. GLEANalso provides output on the percentage of task time for which each goal in a modelis active. With respect to learning evaluations, GLEAN counts the number of state-ments included in the methods of a GOMSL model. Similar methods do not needto be relearned and, thus, are counted only once. This type of evaluation providesan indication of the complexity of the interactive task/interface being modeled(Kieras & Meyer, 1995).

The static and dynamic evaluation capabilities of GLEAN provide a basis forinterface design using GOMS (Kieras, Wood, Abotel, & Hornof, 1995). One lim-itation of GLEAN is that to simulate human use of a software interface, thecomplier must be integrated with additional C++ programs to implement anotional representation of an interface (i.e., code identifying interface objects withlabels, and display locations using numerical coordinates) for presentation to aGOMSL model. Furthermore, scenario/script files must be developed to populatethe notional interfaces with data for model processing in run-time. Consequently,the cognitive model is not actually applied to a dynamic graphical userinterface.

Recently, new tools have been developed for computational GOMSL mod-eling, including EGLEAN (Error-extended GLEAN; Soar Technology, 2005).EGLEAN runs as a plug-in to the ECLIPSE IDE (Integrated [Java] DevelopmentEnvironment). The plug-in provides a graphical user interface that supports writ-ing, debugging, and running GOMSL models. Because EGLEAN was developedbased on the framework of GLEAN, it compiles GOMSL files with all the psycho-logical constraints and rules used in GLEAN, supporting the cognitive plausibilityof models. ECLIPSE also allows for integration of GOMSL models compiled inEGLEAN with Java devices acting as surrogates to task interfaces by using XMLparsing. Through ECLIPSE, a modeler can observe a GOMSL model controlling avirtual task interface in real-time. With this technique, the integration between themodel and the interface provides a highly realistic simulation of human interactivetasks extending beyond that achievable with the historical GLEAN tool and a fullyspecified GOMSL model. Consequently, EGLEAN can be used to facilitate cogni-tive modeling analysis of complex, real-world user interfaces to identify superiorprototypes and methods of task processing. Cognitive model performance can bedirectly compared with empirical data on human performance with identical taskinterfaces, including display dynamics.

We have previously successfully implemented this technique to simulatehuman behavior in a real-time control task (Kaber, Wang, & Kim, 2006). In thepresent study, the GOMSL models controlled the Java-based versions of the exist-ing interfaces (SAMI) and new prototype interfaces, acting as human operators.Figure 7 shows the EGLEAN system architecture. Because EGLEAN is still underdevelopment (Soar Technology, 2005), some functions of GLEAN are not currentlyavailable through its graphical user interface. For example, the learning analysisresults and method execution profiles for a GOMSL model that can be obtainedwith the command line version of GLEAN are not supported in EGLEAN. In thisresearch, we used EGLEAN to facilitate GOMSL modeling and representation ofthe HTS control interfaces (using Java). We also used GLEAN for model analysispurposes. We compared the GLEAN outputs with the human performance data,



FIGURE 7 System architecture for integration of EGLEAN and Java using theECLIPSE platform (color figure available online).

specifically the task execution times, to assess the validity of our approach and todetermine if the model output could be used as a basis for HTS interface usabilityevaluation.

3. RESULTS AND DISCUSSION

3.1. Usability Tests

Two usability evaluations were conducted with the first focusing on the newbarcoder action/configuration dialog and the second dealing with the prototypemethod editor interface. Related to the specified usability goals, here we presentthe results on time-to-task completion and number of errors from the empiricalstudies.

Scenario 1. The subtask performance times for the new barcoder proto-type and SAMI barcoder dialog for the CELISCA participants are presented inTable 4. The new interface appeared to allow for quicker performance overalland on certain tasks. However, a one-sided Wilcoxon rank-sum test revealed nosignificant differences in interface use times for the barcoder configuration pro-cedure at the p = .05 level. (The nonparametric alternative to the t test was usedhere because the assumption of normality of the distribution of the differencesbetween the two samples was not supported by the data based on residual diag-nostics. This was also the case for the other statistical tests presented.) The lack


496 Kaber et al.

Table 4: Performance Times for Prototype and SAMI Barcoders (in Seconds) forExpert Biochemists

SAMI Barcoder Interface New Barcoder Interface

Task M Mdn Range M Mdn Range

Select side 3.5 2.8 4.0 10.8 5.7 32.4Select label 3.8 3.7 5.7 3.7 1.9 5.6Content 12.9 14.4 15.7 9.8 4.0 26.0Total scenario 24.8 32.2 19.5 24.3 11.3 63.6

Note. SAMI = SagianTM Automated Method Development Interface.

of significant differences in the subtask performance times with the new pro-totype was likely due to the high variability attributable to participant limitedfamiliarity with the interface. A post hoc power analysis on all interface timesfor the barcoder configuration procedures revealed power levels ranging from0.28 to slightly lower than 0.10. It is also likely that the small study sample sizecontributed to a lack of sensitivity of the statistical tests to the differences amongthe interface conditions. The CELISCA operators committed no errors while usingboth barcoder interfaces. This is attributable to their prior experience with lifescience applications and method editing software.

Scenario 2. We subsequently evaluated the complex interaction scenarioinvolving the method editor interface and programming of an HTS assay. Table 5presents the subtask performance times for the new prototype and SAMI methodeditors for the expert biochemists at CELISCA. Again, the new interface appearedto allow for quicker performance overall and in a particular subtask. A one-sidedWilcoxon rank-sum test revealed input of the assay procedure to be significantlyshorter using the new interface (p = .004), and there was a marginally significantdifference (p = .0754) between the total method development times for the twointerfaces. A post hoc power analysis of the interface times for the method edi-tor procedures revealed power for the assay development test to be 0.77. Powerfor the test on overall scenario time was 0.24 and other levels ranged from 0.10

Table 5: Performance Times for Prototype and SAMI Method Editors (in Seconds)for Expert Biochemists

SAMI Method Editor New Method Editor

Task M Mdn Range M Mdn Range

Barcoder configuration 34.2 28.8 38.9 40.1 16.6 96.6Incubator configuration 13.2 10.9 22.6 29.4 14.1 68.8Assay development 254.2 244.9∗∗ 187.9 46.7 16.0∗∗ 140.2Total scenario 373.5 354.0∗ 286.3 236.2 120.0∗ 655.5

Note. SAMI = SagianTM Automated Method Development Interface.∗Marginally significant difference among interfaces, p < .10.∗∗Significant difference among interfaces, p < .05.



to 0.28. Again, the small pool of expert biochemists available for the study likelylimited the sensitivity of some of our analyses for revealing differences among theinterfaces. As previously mentioned, some aspects of the incubator configurationtask differed across the interfaces, and therefore direct statistical comparison ofperformance times was not made.

With respect to the number of errors committed during the method program-ming scenario, we compared novice user performance with the test interfaces (newprototype users at CELISCA compared to student users of SAMI at North CarolinaState University). Users evaluating the new prototype committed one error ofcommission (taking an incorrect step) and no errors of omission (failing to take thecorrect step). Novice users evaluating the SAMI method editor made 16 errors ofcommission and 11 errors of omission. Although users made more errors using theSAMI interface than when using the new interface, it is important to note that forsome of the steps in the two operating scenarios there were functional differencesdue to the interface features. With this in mind, and the fact that our evaluationgroups were not equivalent in terms of experience in life sciences, we did not makeformal statistical comparison of these results. However, such a difference in errorcounts across interfaces for the method programming task would be of practicalsignificance for an actual HTS operation. That is, there were mean differences inthe error counts that provide insight into overall effectiveness of the interfaces.These differences may also be attributable to learnability issues (this is confirmedthrough the GOMSL output and analysis in next section).

Based on these performance results, we inferred that the new method editorinterface design may provide some performance advantage over the existing inter-face design based on usability for assay development. This is important becauseassay entry is the primary task performed by biochemists with the interface, andimprovements here may lead to shorter editing times, overall. Furthermore, thecomparison of absolute error counts with the interfaces indicated that the new pro-totype generally led to better support for novices in their completion of the HTStask. On this basis, we also inferred that our use of the metaphor-based approachto redesign of the HTS control software, incorporating the CTA results, might havesome benefit from a usability perspective.

Based on the post hoc power analyses, it was evident that a larger sample ofexpert biochemists would have been helpful for detecting other potential sig-nificant differences among the method editors. Unfortunately, the population ofexperts in HTS operations is very limited, and volunteers for control softwareanalysis are even fewer. With this in mind, our descriptive statistics alone may beuseful at this stage for careful design purposes and as a basis for initial modelingefforts. With respect to using the GOMSL modeling approach for usability evalu-ation in this domain, the statistically significant differences we did detect amonginterface conditions were considered sufficient for testing model sensitivity.

3.2. Model Validation and Evaluations

By using EGLEAN to compile and run the GOMSL models for both the bar-coder configuration dialogs and the method editor interfaces, we were able to


498 Kaber et al.

compare the model output to the human usability test results. Because GOMSLmodels are intended to represent expert, error-free performance, the methods ina model represent the established memory structures an expert is expected topossess. Therefore, we made comparisons between the GOMSL model outputfor the new prototype interfaces for both scenarios with the expert biochemistcompletion times for each subtask. Again, such models are not capable of pre-dicting task errors; therefore, we could not make comparison of the models withthe experiment results in terms of errors. If a GOMSL model for an existinginterface design is determined to represent expert behavior, then the modelingapproach may be applied to predict comparable expert performance with othervariations on the interface design. GOMSL model predictions do not suffer fromthe variability that may exist in task performance estimations based on empiricalstudies.

Scenario 1. The subtask and total times for expert interaction with the SAMIbarcoder dialog (Table 6) and for the new prototype barcoder dialog (Table 7), ascompared to the GOMSL model output for each interface, suggested that modelperformance trended with actual expert performance. Prior GOMS research hasestablished model performance criteria for assessing validity and predictive util-ity. John and Newell (1989) offered that models yielding output with less than20% deviation from human performance could be considered valid. Gray, John,and Atwood (1993) considered correlation coefficients on model and actual per-formance times as low as r = .69 to be sufficient evidence for using GOMSmodels for system alternative selection purposes. For the existing barcoder inter-face, model times followed the same pattern as user times across subtasks (r =.99, p = .0017). (We provide Pearson correlation coefficients, as Spearman’s non-parametric correlation analysis did not provide sufficient resolution to reveal acorrespondence among model and human times through ranks on a limited num-ber of task observations; n = 4). The actual deviations among model and usertimes were, on average, 49%. All subtask time predictions for the new barcorderinterface also appeared to be highly (but not significantly) correlated with actualuser times (r = .82, p = .18). Because the GOMSL model represents expert behav-iors and the subjects were highly experienced with the new interface, it was

Table 6: Interface Use Times for SAMI Barcoder (in Seconds) Model versusCenter for Life Science Automation Experts

Expert Users

Task GOMSL Model M Mdn%Deviation

(Model vs. User)

Select side 2.0 3.5 2.8 29Select label 1.6 3.8 3.7 57Content 6.7 12.9 14.4 54Total scenario 14.0 24.8 32.2 57

Note. SAMI = SagianTM Automated Method Development Interface; GOMSL= Goals, Operators, Methods, and Selection rules Language.



Table 7: Interface Use Times for New Barcoder Interface (in Seconds)—Modelversus Center for Life Science Automation Experts

Expert Users


(Model vs. User)

Select side 1.8 10.8 5.7 69Select label 1.6 3.7 1.9 16Content 6.9 9.8 4.0 73Total scenario 12.1 24.3 11.3 7


not surprising that the model output for the new interface did not significantlyaccount for the human data. However, the model for the new barcoder interfacedid yield a total time estimate that was within 7% of actual user times, and theprediction of the time to select a label was within 16% of the observed humantime.

Scenario 2. The interface use times for the SAMI method editor (Table 8) andfor the new prototype method editor (Table 9), as compared to the GOMSL modeloutput for each interface, indicated that model performance approached actualexpert behavior for the majority of operations. High correlations were found forthe existing editor (r = .99, p < .0001) and the new editor (r = .90, p = .1033).Furthermore, the majority of model predicted subtask times for the existing inter-face (three of four) were well within 20% of actual user times (see last columnsof Table 8). Due to some functional differences across the interface designs, somemethods used in the scenario that contributed to the total scenario time could notbe included in this analysis.

Regarding the substantial deviation of the model assay entry time from usertime for the new interface (see Table 9), this was attributable to the fact that par-ticipants were not highly experienced with use of the interface for this purpose.Beyond this, the limitations of GOMSL for accurately modeling typing perfor-mance, as required by the assay entry task, have been noted in the literature (Wu &

Table 8: Interface Use Times for SAMI Method Editor (in Seconds)—Modelversus Center for Life Science Automation Experts

Expert Users


(Model vs. User)

Barcoder 28.7 34.2 28.8 0Incubator 16.7 13.2 10.9 53Enter assay 224.1 254.2 244.9 9Total scenario 318.1 373.5 354.0 10

Note. SAMI = SagianTM Automated Method Development Interface; GOMSL= Goals, Operators, Methods, and Selection rules Language.


500 Kaber et al.

Table 9: Interface Use Times for New Method Editor Interface (in Seconds)—Modelversus Center for Life Science Automation Experts

Expert Users


(Model vs. User)

Barcoder 47.2 40.1 16.6 184Incubator 31.8 29.4 14.1 125Enter assay 192.7 46.7 16.0 1, 104Total scenario 379.1 236.2 120.0 216


Liu, 2004). (The total scenario time was, consequently, impacted by this.) Wu andLiu (2004) said that the mechanisms involved in human typing are more detailedthan current GOMS modeling supports. Therefore, the difference between theGOMSL model and the expert performance for the new prototype method edi-tor (and not the barcoder interface) could be due to the model execution timesnot accurately representing human word construction and typing speed. In gen-eral, the high correlations of the GOMSL model times with the user task times, aswell as the limited deviations of time estimates from actual performance with themethod editors for the existing interfaces, provided support for validation of ourmodel construction.

Finally, using the learning analysis capability of GLEAN, we examined themethod execution profiles and “task units” of the two models. We found that thenew method editor prototype required far fewer steps in methods to be learned(35 steps) compared to the SAMI method editor, with 55 steps to be learned. Thisdifference in the learning requirements of the two interfaces supported our earlierfinding on the effectiveness of the new interface for producing fewer errors thanthe existing interface. Related to this, based on the GOMSL model task time esti-mates, the new barcoder dialog required less time than the SAMI barcoder dialog.Entering an assay in the new method editor also required less time than doing soin the SAMI method editor and led to a shorter overall scenario time.

In general, there are two caveats that should be emphasized here regarding ourresults on expert performance versus the GOMSL model output. As in the com-parisons of user performance with the existing and new interfaces, the user taskcompletion time data were based on a small sample from CELISCA. As previ-ously mentioned, it is very difficult to recruit expert HTS biologists, chemists, andprocess engineers for such experiments. These persons are typically engaged incontract work for large drug companies with a single HTS process over a 2- to3-day period costing several hundred thousand dollars. The small expert samplesize may have limited the reliability of our results. Second, the GOMSL modeloutput for the time-to-task completion of the individual methods results in a sin-gle point estimate regardless of the random seed used during model simulationof human behavior. Consequently, the output does not uphold the distributionassumptions of statistical tests. Therefore, we used the correlation and time devia-tion analyses to make our model and human performance comparisons, which isaccepted practice in the literature (Gray et al., 1993; John & Newell, 1989).



4. CONCLUSIONS

In this research, we sought to assess the validity of using computational GOMSLmodels for human–automation interface usability evaluations in the life sciencesdomain. We prototyped enhanced interfaces for human supervisory control ofHTS processes based on usability design recommendations from prior workinvolving several CTA techniques. We tested the prototypes through humanusability trials and with cognitive model simulations. The usability tests revealedsome linkage between the usability-based interface design enhancements andoperator performance in terms of task execution times, when compared to timeswith existing HTS control interfaces. This, however, was limited to assay entrywith a method editor, which is one of the most important tasks in HTS operations,and overall editing performance.

Based on prior research using GOMS models to represent dynamic, interac-tive task behavior, we expected GOMSL models of HTS task performance to haveaccuracy for predicting expert performance and to reflect the impact of the usabil-ity design changes we made in the HTS control interfaces on human task time. Aswith the usability tests results, the GOMSL model outputs and method executionprofiles revealed improvements in performance with the new interface prototypes,as compared to existing HTS control interface designs for assay planning tasks.The predicted time for assay programming was lower for the new interface (192.7s vs. 224.1 s), but statistical tests could not be conducted on the point estimates.The learning analysis for the two interfaces using GOMS also revealed a substan-tial reduction in the number of steps that operators needed to learn with the newinterface. In addition, results on comparison of GOMSL model output with actualhuman performance data supported the contention that the computational cogni-tive models can be used to represent expert task performance in the HTS domain,and they may also be used as tools for usability evaluation. However, additionalstudies are needed to confirm these findings with larger expert populations inHTS, if at all possible.

The usability evaluation results (human and model) indicated experts wereable to take advantage of new prototype HTS interface features to achievehigher performance for a specific type of control task. However, our mod-eling experience revealed certain limitations of GOMSL relevant to keyboardtyping performance, which may account for some of the identified differencesamong human performance and model output. Differences between humanand model word construction and typing speeds may contribute to some inac-curacy of the GOMSL model compared to expert performance data (Wu &Liu, 2004).

GOMS modeling has previously been criticized as being difficult to learn andapply and posing a high workload for system designers (Byrne, Wood, Sukaviriya,Foley, & Kieras, 1994). Although new tools have been developed to make GOMSLmore accessible for interface evaluation (e.g., EGLEAN), the process of predic-tive model development remains somewhat laborious. This is, in part, due to theneed for a detailed CTA as a basis for model development. In addition, significantobservational study may be necessary to identify operator task strategies. Multiplestrategies may be used for performance and, consequently, testing each strategy


502 Kaber et al.

may be necessary before a model yields results comparable to human performancedata. In this research, it took us approximately 3 months to conduct the targetHTS task analysis, code the GOMSL models, develop the Java interface proto-types, and link them to the GOMSL models using EGLEAN. That said, we believea computational GOMSL modeling approach to usability evaluation has severaluses, including description of HTS task complexity and learning requirements interms of the number of methods and steps executed at an interface, quantificationof the speed and efficiency with which biochemists can execute HTS tasks, andidentification of the potential for operators to make errors in performance.

Beyond this caveat, it is important to note that the experts at CELISCA may alsohave been influenced in performance by their prior knowledge of the SAMI soft-ware. This may have had an impact on the times to task completion with the SAMIsoftware compared to the new prototype interfaces. An alternative approach to theusability evaluation presented here could involve training operators to a predeter-mined level of proficiency with the software and prototypes before assessing theirperformance.

Finally, future research should investigate the application of the overallmethodology presented here to other work domains to establish robustness. Itwould be interesting to compare the types of recommendations generated indomains similar to HTS in terms of performance and cognitive task demands onoperators as estimated through formal usability testing and cognitive model-basedevaluation due to unstructured environments, dynamic workloads, performanceof multiple and simultaneous tasks, and so on. Example domains might includeaviation, air traffic control, and patient care in hospitals. These domains might alsopermit larger user sample sizes and greater reliability in statistical comparisons ofinterface designs.

REFERENCES

Byrne, M. D., Wood, S. D., Sukaviriya, P., Foley, J. D., & Kieras, D. E. (1994). Automatinginterface evaluation. In Proceedings of CHI ’94, Human Factors in Computing Systems (pp.232–237). New York, NY: ACM.

Card, S., Moran, T., & Newell, A. (1983). The psychology of human–computer interaction.Hillsdale, NJ: Erlbaum.

Dix, A., Finlay, J., Abowd, G., & Bealle, R. (2004). Human–computer interaction (3rd ed.). NewYork, NY: Prentice Hall.

Endsley, M. R. (1993). A survey of SA requirements in air-to-air combat fighters.International Journal of Aviation Psychology, 3, 157–168.

Entzian, K., Allwardt, A., Holzmüller-Laue, S., Junginger, S., Roddelkopf, T., Stoll, N.,& Thurow, K. (2005, June). Automationslösungen für biologische und chemis-che Screeningverfahren. Proceedings GMA-Kongress “Automation als InterdisziplinäreHerausforderung,” 235–242.

Erickson, T. D. (1990). Working with interface metaphors. In B. Laurel (Ed.), The art ofhuman–computer interface design (pp. 65–73). Reading, MA: Addison-Wesley.

Fitts, P. M. (1954). The information capacity of the human motor system in controlling theamplitude of movement. Journal of Experimental Psychology, 47, 381–391.



Gray, W. D., John, B. E. & Atwood, M. E. (1993). Project Ernestine: A validation of GOMS forprediction and explanation of real-world task performance. Human–Computer Interaction,8, 237–309.

John, B. E., & Kieras, D. E. (1996). Using GOMS for user interface design andevaluation: Which technique? ACM Transactions on Computer–Human Interaction, 3,287–319.

John, B. E., & Newell, A. (1989). Cumulating the science of HCI: From S-R compatibility totranscription typing, In Proceedings of CHI (pp. 109–114). New York, NY: ACM.

Kaber, D. B., Segall, N., & Green, R.S. (2007). Metaphor-based design of high throughputscreening process interfaces. Journal of Usability Studies, 2, 190–210.

Kaber, D. B., Segall, N., Green, R. S., Entzian, K., & Junginger, S. (2006). Using multiplecognitive task analysis methods for supervisory control interface design in high-throughput biological screening processes. Cognition, Technology & Work, 8, 237–252.doi:10.1007/s10111-006-0029-9.

Kaber, D. B., Wang, X., & Kim, S-H. (2006, October). Computational cognitive modeling ofoperator behavior in telerover navigation. In Proceedings of the 2006 IEEE InternationalConference on System, Man, and Cybernetics, Taipei, Taiwan.

Karwowski, W., Kosiba, E., Benabdallah, S., & Salvendy, G., (1990). A framework for devel-opment of fuzzy GOMS model for human–computer interaction. International Journal ofHuman–Computer Interaction, 2, 287–305.

Kieras, D. (1997). Task analysis and the design of functionality. In A. Tucker (Ed.), Thecomputer science and engineering handbook (pp. 1401–1423). Boca Raton, FL: CRC Press.

Kieras, D. (1999). A guide to GOMS model usability evaluation using GOMSL and GLEAN3.Ann Arbor: University of Michigan, Ann Arbor.

Kieras, D. E., & Meyer, D. E. (1995). Predicting human performance in dual-task track-ing and decision making with computational models using the EPIC architecture. InProceedings of the 1995 International Symposium on Command and Control Research andTechnology (pp. 314–325). Washington, DC: National Defense University.

Kieras, D. E., Meyer, D. E., Ballas, J. A., & Lauber, E. J. (1999). Modern computational perspec-tives on executive mental processes and cognitive control: Where to from here? (EPIC Report:No. 12, RT-99/ONR-EPIC-12). Washington, DC: Office of Naval Research.

Kieras, D.E., Wood, S.D., Abotel, K., & Hornof, A. (1995). GLEAN: A computer-based toolfor rapid GOMS model usability evaluation of user interface designs. Proceedings of the1995 UIST Symposium (pp. 91–100). Pittsburgh, PA.

Meyer, D. E., & Kieras, D. E. (1997). A computational theory of executive control processesand human multiple-task performance: Part 1. Basic mechanisms. Psychological Review,104, 3–65.

Neale, D. C., & Carroll, J. M. (1997). The role of metaphors in user interface design. InM. Helander, T. K. Landauer, & P. Prabhu (Eds.), Handbook of human–computer interaction(pp. 441–462). Amsterdam, the Netherlands: Elsevier Science.

Nielsen, J. (1993). Usability engineering. Boston, MA: Academic Press.Norman, D. A. (1995). The psychopathology of everyday things. In R. M. Baeker, J. Grudin,

W. A. S. Buxton, & S. Greenberg (Eds.), Readings in human–computer interaction: Towardthe year 2000 (pp. 5–22). San Francisco, CA: Morgan Kaufmann.

Olson, J. R., & Olson, G. M. (1990). The growth of cognitive modeling in human–computerinteraction since GOMS. Human–Computer Interaction, 5, 221–265.

Rasmussen, J. (1985). The role of hierarchical knowledge representation in decision-makingand system management. IEEE Transactions on Systems Man and Cybernetics, 15, 234–243.

Soar Technology. (2005). EGLEAN Science and Technology Report: The first six months. AnnArbor, MI: Author.


504 Kaber et al.

Thurow, K., Göde, B., Dingerdissen, U., & Stoll, N. (2004). Laboratory information manage-ment systems for life science applications. Organic Process Research & Development, 12.8,A–M.

Usher, J. M. and Kaber, D. B. (2000). Establishing information requirements for supervisorycontrollers in a flexible manufacturing system using goal-directed task analysis. HumanFactors & Ergonomics in Manufacturing, 10(4), 431–452.

Wu, C., & Liu, Y. (2004). Modeling human transcription typing with QN-MHP (QueueingNetwork - Model Human Processor). In Proceedings of the 48th Annual Conference ofthe Human Factors and Ergonomics Society (pp. 381–385). Santa Monica, California,CA: HFES.


international journal of human-computer interaction ... · human-machine interfaces for life...

Documents