7/28/2019 p85-grudin

1/9

WHY CSCW APPLICATIONS FAIL: PROBLEMS IN THE DESIGNAND EVALUATION OF ORGANIZATIONAL INTERFACESJonathan Grudin

MCC3500 West Balcones Center DriveAustin, Texas 78759

Abstract.Many systems, applications, and features that supportcooperative work share two characteristics: A significantinvestment has been made in their development, and theirsuccesses have consistently fallen far short of expecta-tions. Examination of several application areas reveals acommon dynamic: 1) A factor contributing to theapplications failure is the disparity between those whowill benefit from an application and those who must doadditional work to support it. 2) A factor contributing tothe decision-making failure that leads to ill-fateddevelopment efforts is the unique lack of managementintuition for CSCW applications. 3) A factor contributingto the failure to learn from experience is the extremedifficulty of evaluating these applications. These threeproblem areas escape adequate notice due to two naturalbut ultimately misleading analogies: the analogy betweenmulti-user application programs and multi-user computersystems, and the analogy between multi-user applicationsand single-user app lications. These analogies influencethe way we think about cooperative work applications anddesigners and decision-makers fail to recognize theirlimits. Several CSCW application areas are examined insome detail.Introduc tion. An illustrative example: automatic meetingscheduling.Where electronic calendars are in use on a large ornetworked system, an automatic meeting schedulingfeature is often provided (e.g., Ehrlich, 1987a, 1987b).The concept that underlies automatic meeting schedulingis simple: The person scheduling the meeting specifies adistribution list and the system checks the calendar foreach person, finding a time convenient for all. The systemthen notifies all involved of the tentative schedule.

Permission to copy without fee all or part Of this material is granted provided thatthe copies are not made or distributed for direct commercial advantage. the ACMcopyright notice and the title of the publication and its date appear, and notice isgiven that copying is by permission of the Association for Computing Machinery.TO copy otherwise. or to republish, requires a fee and/or specific permission.0 1988 ACM O-89791-282-9/88/0085 $1.50

For automatic meeting scheduling to work efficiently,everyone involved must maintain a personal calendar andbe willing to let the computer schedule their free timemore often than not. Data reported by Ehrlich (1987a,1987b) suggest that neither of these requirements isgenerally satisfied.Electronic calendars are not electronic versions of papercalendars. They serve communication functions, primarilyfor managers and executives with personal secretarieswho maintain the calendars. An electronic calendar maybe used simultaneously by the secretary for scheduling,the manager for reviewing, and other group members forlocating or planning. Ehrlich describes the successful useof the electronic calendar in detail; a key point is that thesecretary s role is critical; those who do not have a secre-tary are much less likely to maintain an electronic calen-dar. Another relevant finding is that for managers, freetime is never really free. Unauthorized scheduling of amanagers apparently open time can be sufficient moti-vation for total rejection of the system by the manager.Thus, electronic calendars are voluntarily maintainedprimarily by managers and executives (or theirsecretar ies). This has dire consequences for au tomaticmeeting scheduling. If a manager wants to meet withnon-management subordinates, few of the latter are likelyto maintain electronic calendars. The scheduling programwill find all times open, schedule a meeting, and conflictswill ensue. In order to take full advantage of anelectronic calendar, all members of a group must committo using this medium, (Ehrlich, 1987b) . If managers orexecutives keeping on-line calendars wish to meet amongthemselves, automatic scheduling could work. But asnoted above, free time is often not truly free for suchmanagers; it would be wise to consult with theirsecretaries anyway. Thus, automatic meeting schedulingmay rarely be used in this situation, either.The simple meeting scheduling feature previews thepattern that emerges from the major applicationsdiscussed later. Who would benefit from automaticmeeting scheduling? The person who calls the meeting: ingeneral, a manager would benefit. But who would have todo additional work to m ake the application succeed? Thesubordinates, who would have to maintain electroniccalendars that they would not otherwise use. The

85

7/28/2019 p85-grudin

2/9

application might be madr to work through persuading orordering employees to maincain calendars, and replacingpeople who wont, but automatic meeting scheduling isnot perceived to be of great. enough collective benefit towarrant such measures.Why design and implement a feature that is unlikely to beused? The managers who make the final design decisionsmay see the personal benefit of automatic meeting sched-uling to managers such as themselves, without noticingthat those users who would be forced to do extra work tosupport the feature would not benefit from it. (Otherreasons for adding this feature might be a potential mar-keting benefit or simply the ease of implementing it.)As with the more complex cases described later, theconclusion is not entirely negative. Automatic meetingscheduling could be targeted to environments or groupsmaking the most uniform use of electronic calendars.Their value can be enhanced by adding conference roomand equipment scheduling. Individual use of calendars tosupport the feature might increase if the perceivedcollective benefit were higher; that is, if organizationsrecognized how much they may be losing throughinefficient meeting scheduling (Ehrlich, 1987b).Problems in the design and evaluation of organizationalinterfaces.Several major CSCW application areas have attractedsignificant investments of capital and labor over manyyears, with results that have uniformly fallen far short ofexpectations. These include the areas of digitized voice,group decision support, natural language interface toshared databases, and project management. The preced-ing example, automatic meeting scheduling, is a simpleand relatively inexpensive feature. Its problems are easilyidentified, however, and can then be seen as common tomany CSCW applications and as key factors in the theirdisappointing performance. The problems are:

The application fails because it requires that somepeople do additional work, while those people are notthe ones who perceive a direct benefit from the use ofthe application.The design process fails because our intuitions arepoor for multi-user applications -- decis ion-makerssee the potential benefits for people similar to them-selves, but dont see the implications of the fact thatextra work will be required of others.We fail to learn from experience because these com-plex applications introduce almost insurmountableobstacles to meaningful, generalizable analysis andevaluation.

These problems have received altogether inadequateattention given their impact on CSCW applications. Thismay be due to two powerful, natural analogues to CSCWapplications for which these problems are in general lesssevere: multi-user systems such as management informa-tion systems, computer-integrated manufacturing, order-and-inventory-control systems), and single-user applica-tions. Our possible unconscious use of the analogies and

the danger of faiiing to identify where they break downare discussed below and explored in more detail in twoappendices.The paper concludes with detailed case studies of fourCSCW application areas. While the focus is on theirproblems, all of these areas are important and may leadto significant advances. It sometimes appears that in striv-ing toward very ambitious goals we are taking turns beat-ing our heads against the same wall, but pointing to thewall is not intended to devalue the goals. There are waysto get over the wall -- to build successful CSCW applica-tions. Investing resources adequate to the solution of theproblems, developing the appropriate research and devel-opment methodologies, finding niches where the problemsdont arise or where applications will succeed in spite ofthem, and adequately preparing users for the introductionof the applications are all approaches that may lead tosuccess. The firs t step is to see the problems c learly.Problem 1. The disparity between who does the workand who gets the benefit.The immediate beneficiary of the automatic meetingscheduler is the manager (or secretary) who initiates atypical meeting. Successful use of the feature in a typicalenvironment would require additional work for othergroup members, most of whom would have to maintainelectronic calendars when they would not otherwise do so.Not all CSCW applications introduce such a disparity --with electronic mail, for example, everyone generallyshares the benefits and burdens equally. But electronicmail may turn out to be more the exception than the rule,unless greater care is taken to distribute the benefit inother CSCW applications.Can a CSCW application succeed if doing the extra workis left to individual discretion? Unfortunately, probablynot. Communication, at the heart of m ost CSCWapplications, will break down without relatively uniformuse. If a substantial number of people do not maintaintheir calendars, the meeting scheduler is pointless. In thisrespect, the single-user application is a misleading model.In many environments, there is no harm if different userschoose to use different editors, for example; but anapplication designed to support the entire group must beused by everyone in the group. A critical mass of usersis essential for the success of any communication system(Ehrlich, 1987b).Can a CSCW application be made to succeed bymandating that those who need to do the extra work doso? Even setting aside the implications of coercion (whichare discussed in the final appendix), this is complicated.This traditional approach, changing existing job descrip-tions or inventing new ones, is widely used when CSCWsystemsare introduced (Chems, 1980; Rowe, 1985). Wordprocessing skills become a job requirement forsecretaries, the new job of database administrator iscreated, and so forth. However, the multi-user system --multi-user application analogy breaks down. An organiza-tion invests a large amount in introducing a systemand isusually willing to reorganize to make it succeed,retraining, transferring, or dismissing people as deemedI zessary. This will not be done for each CSCW

86

7/28/2019 p85-grudin

3/9

application that arrives -- the cost will outweigh thebenefit. Maintaining a personal calendar in order tosupport automatic meeting scheduling is unlikely tobecome a job requirement. In general, the orgunizutionmay adapt to the computer system, but an applicationprogram must adapt to the orgcmization.What if the application really might provide a collectivebenefit to the group or organization? Of course, measur-ing a collective benefit may be hard. If maintaining theapplication requires an hour per week from each groupmember, and its benefit is to save ust one person an hourper week, is it worth it? What if the one person is thegroup manager? What if it is the Vice President of Re-search & Development? But assuming that a collectivebenefit has been determined, education and leadershipmay be critical. If it is demonstrated that inefficiency inscheduling meetings is costly to the group and that main-taining calendars to support the scheduling feature is thebest solution, people may be willing to do the extra work.However, the best solution is to try to insure that everyonebenefits directly from using the application. This maymean building in additional features. It certainly meanseliminating or minimizing the extra work required ofanyone, or rewarding them for doing it. (This includesminimizing the training needed; Carasik and Grantham,1988, attribute a rejection of The Coordinator, a CSCWapplication, in large part to the effort required to learn it.)User interfaces must be provided that vary appropriatelywith a users background, job, and preferences. This is asubstantial undertaking, but there may be no other option.Problem 2. The breakdown of intuitivedecision-making.Why was the problem with the automatic meeting sched-uler not anticipated? More generally, we need to under-stand why thousands of developer-years and hundreds ofmillions of dollars have been committed to various CSCWapplication areas despite little or no return. In mostinstances of failure, a substantial and timely return oninvestment was certainly anticipated; the decision-makerswere not in business to throw away money.Decision-makers in a position to com mit the resources toapplication development projects rely heavily on intuition(see e.g. Butler, Bennett, & Whiteside, 1987). The experi-ence, and very likely the track record, of a developmentmanager considering a CSCW application is generallybased on single-user applications. Intuition may be a farmore reliable. guide to single-user applications -- amanager with good intuition may quickly get a feel for theusers experience with a word processor, spreadsheet, orso forth. But a typical CS CW application will be used by arange of user types -- people with different backgroundsand job descriptions, ull of whom may have to participatein one way or another for the application to succeed. Thedecision-makers intuition will fail when an appreciationof the intricate dynamics of such a situation is missing.Not surprisingly, the decision-maker is drawn to applica-tions that selectively benefit one subset of the user popu-lation: managers. Intuitions about what will be useful toPeople similar to ourselves are generally good. Managers

tend to overlook or underestimate the down side, the extrawork that might be required of other users to maintain theapplication; extra work that might not be forthcoming inmost environments, subjecting the application to neglector sabotage. The decision-makers may also fail to appre-ciate the difficulty of producing and evaluating this newtype of application, as described in the next section. Theconverse possibility also exists: the decision-maker maynot see the value in applications or features that willprimarily benefit other categories of user, even wherethey would provide a collective benefit to the organiza-tion. This would be particularly true for features thatmight undercut or create additional work for the manager.Intuition may be less unreliable for applications directedat smaller or more homogenous groups. In particular,there may be less bias when only peer-peer communica-tion is involved than when the communication movesvertically through the organizational hierarchy. Beyondthat, education seems to be called for -- generaleducation about groupware, the risks involved, and theresources and approaches required to minimize the risk,specific research on the application area at hand.Education is needed, and vigilance.Problem 3. The underestimated difficulty of evaluatingCSCW applications.Task analysis, design, and evaluation are never easy, butthey are considerably more difficult for CSCWapplications than for single-user applications. Anindividuals success with a particular spreadsheet or wordprocessor is not likely to be affected by the backgroundsof other group members or by administrative orpersonality dynamics within the group. These factors are,however, quite likely to affect applications intended tosupport an entire group, where motivational, economic,and political factors come to the fore (Malone, 1985).Evaluation of CSCW applications requires a very differentapproach, based on the methodologies of socialpsychology and anthropology. This may not be news tothose who have been monitoring the field of CSCW veryclosely, but the skills are largely absent in developmentand many research environments, where human factorsengineers and cognitive psychologists are only starting tobe accepted. And the required methods are generallymore expensive, more time-consuming, and less precise.It is relatively easy to bring a single user into a lab to betested on the perceptual, cognitive, and motor variablesthat have been the focus for single-user applications. Butit is difficult or impossible to create a group in the labthat will reflect the social, motivational, economic, andpolitical factors that are central to group performance(Malone, 1985). In addition, group observation mustextend over a longer period of time. Much of a personsuse of a spreadsheet might be observed in a single hour,for example, but group interactions typically unfold overdays or weeks.Evaluation of groupware in the field is remarkablycomplex due to the number of people to observe at eachsite, the wide variability that may be found in groupcomposition, and the range of environmental factors that

87

7/28/2019 p85-grudin

4/9

play a role in determining acceptance, such as usertraining, management buy-in, and vendor follow-through(e.g., Lucas, 1976; Gaffney, 1985; White, 1985; Ehrlich,1987b). Establishing success or failure will be easier thanestablishing the underlying factors that brought it about.Finally, the difficulty of evaluation is increaseddramatically by the importance of providing features andinterfaces that vary according to a users job, background,and preferences, as mentioned above. A single-userapplication may get away with appealing to a kind oflowest common denominator. CSCW applications willoften have to appeal to every possible denominator.As with the other problems, evaluation may be less diffi-cult if the application supports a smaller or more homoge-neous group than if the target user population involvesindividuals distributed across an organization. But it willstill be substantial, and management must be aware fromthe start of the skills and the time that will be required.Case 1. Digitized voice applications.At a conference panel titled Voice: Technology searchingfor communication needs it was noted that after 25 yearsof research, no company specializing in voice technologyhas become profitable, and that projected sales of voiceproducts have recently been revised downward sharply(Aucella, 1987). Eventual success of voice technologymay require an understanding of the exaggerated fore-casts and the relative failures to date. Here only the useof voice in computer-mediated communication is consid-ered, as in computer-based voice messaging or voiceannotation to documents. (Voice is also used for inputonly -- speech recognition -- and for output only -- e.g.,speech synthesis.)The advantages of digitized voice as a computer-basedcommunication medium. Almost everyone can speak, whilemany people cannot type fluently. Moderately pacedspeech is much faster than even the fastest typing. Speechcan readily convey emotion and subtle nuance. Voicemessagescan be sent or received by telephone when awayfrom the computer. Voice annotation can be addedwithout cluttering or overloading a visual display.The disadvantages of digitized voice. It can be more difficultto understand than live speech because stereophonyand lip movements are absent and the speaker cannot beasked to clarify inaudible or unclear passages. Speakingmay be faster than typing, but reading is faster thanlistening to speech. A digitized voice message cannot bescanned (by computer or human) as written text can --the only way to be sure there is nothing of interest in amessage is to listen to the whole thing. Similarly, thereceiver cannot review a voice message ater as easily as awritten message. If suggested document changes or addi-tions are contained in a voice annotation, the receivermust type those changes into the document. For thespeaker, reviewing and correcting a spoken message ismore difficult; hence, voice messages may be more likelyto contain errors. A recipient cannot edit a voice messageand must forward the entire messageor none at all, whichcan be inconvenient (or even embarrassing to the origina-tor; Ehrlich, 1987a, 1987b). Voice mail with no accompa-

:ying visual display has only the transient auditory chan-nel for presenting and explaining options, leading toserious user interface challenges (see Aucella andEhrlich, 1986). Finally, digitized voice requires a lot ofdisk space. (See Newell, 1984, for a broader discussion.)72e pattern. The advantages of digitized voice over typedinput are almost all advantages for the speaker: Speech isfaster to produce, conveys emotion and nuance easily,and may be available without access o a computer termi-nal. The disadvantages to digitized voice, however, areoverwhelmingly problems for the listener. It is harder tounderstand, slower to take in, not easily scanned or re-viewed, more likely to contain errors, and more difficultto manipulate.To succeed, voice systems require that everyone in anenvironment use them. If some people do not use voicemail, time is wasted trying to reach them and groupdistribution lists wont work. If some people do not listento their voice mail frequently enough, calls wont bereturned promptly and use of the system may dwindle anddie.The speaker benefits from voice applications, and thelistener does additional work. When will it be acceptablefor speakers to thus burden listeners? One such time iswhen all users are both speakers and listeners in equalmeasure, thus sharing the benefits and costs, which isgenerally true of voice mail systems. In som e cases, theremay be no alternative -- a sales force on the road mayhave no electronic mail option, and for such users voicemail has proven particularly successful (Ehrlich, 1987a).Similarly, voice may be the best recourse for a user whosehands are necessarily busy. A disparity may also beacceptable when the speaker is of higher status than thelistener, although this may be unpredictable.Some past failures of voice technology are no doubt dueto technical problems and storage requirements. Digitizedvoice messaging has proven successful given the rightenvironment and implementation (Ehrlich, 1987a, 1987b).Voice may succeed more generally where its potentialcollective benefit is conveyed through high-level supportand action (Ehrlich, 1987a, 1987b). In one case, a voicemessaging system that failed initially succeeded when thealternative, telephone receptionists, was removed.But, in general, the disparity between who is inconven-ienced and who gets direct benefit from digitized voicemay work against its adoption in situations where senderand receiver are of comparable status -- the imposition itmakes upon the receiver may be unacceptable. Voiceannotation may be unacceptable in most environments,where the authorial and editorial roles are rarely evenlyshared. Voice annotation particularly benefits those whodont type or who are more likely to act in an editorialthan in an authorial role. These are characteristics ofmanagers and executives. Because of their status, theymay not be concerned by the inconvenience of voice forothers -- dictaphones are used. But the manager-secre-tary gulf is a particularly wide one; the danger is thatdecision-makers will support the development of voiceapplications that appeal to them but that will fail becausetheir use will be onerous to other categories of user.

88

7/28/2019 p85-grudin

5/9

Case 2. Project management applications.A project management application running on adistributed system might be the best demonstration of thepotential of computer-supported cooperative work -- amajor advance over the currently available single-userwork managem ent applications. A distributed projectmanagement application would cover the scheduling andchronicling of activities, the creation and evaluation ofplans and schedules, the management of product versionsand changes, and the monitoring of resources andresponsibilities (Sathi, Morton, and Roth, 1986).Milestone completions might be signaled, documentsrouted to appropriate recipients, problems identified earlyand communicated to those who can help solve them,delays in critical path activities flagged, costs calculated,and so forth. Some people have felt that such anapplication will be the next major commercial success,the next spreadsheet.Cooperative work management applications are beingdeveloped. Callisto: an intelligent project managementsystem (Sathi et al., 1986; reprinted in Greif, 1988), is athorough description of a project begun in 1981. It is clearthat in this area, it is crucial to ask who is the immediatebeneficiary, who will be asked to take on additional workto make the application succeed, and what will be theincentive to do this extra work. The principal beneficiariesare clear: project managers. It is also clear that thesuccess of such an application will be contingent on allgroup members keeping the information base current.This includes updating significant developments thatoccur around, rather than through, the system: inmeetings, telephone conversations, and so forth. Theproject management application may also require thatcritical information that is usually unstated be madeelectronically accessible, such as a secretarys awarenessof a managers priorities (Ehrlich, 1987b).The greatest user interface challenges will be on the sideof information input -- reducing the additional effort to abare minimum, allowing information to be entered in amanner comfortable to each worker, providing compensa-tory benefits to those who must take on the additionaleffort of maintaining the on-line database or knowledgebase. But that is not where attention is being directed. It isbeing directed toward information display, toward the us-er interface for the principle beneficiary, the manager.Managers must know what information is needed, whereto locate it, and how to interpret and use it. Equally im-portant is that they be able to do so without great effort(Sathi et al., 1986). This is not unimportant, but exclusivefocus on improving the system for the person already itsprincipal beneficiary seems ill-advised, although it mightappeal to the manager sponsoring such a project.This is reflected in experience with managementinformation systems. In one example, a ten yeardevelopment project culminated in a computer-assistedmanagement system installed on an aircraft carrier, itsprimary purpose to help the Commanding Officer and hisdepartment heads administer the ship (McCracken andAkscyn, 1984). While numerous factors contributed to itseventual replacement by a system that lacked manage.

ment features, one reported reason for the failure of themanagement system was the difficulty of getting everyoneto use it (Kling, 1987).Worse fates than neglect may confront a project manage-ment application if monitoring and reporting are notcarefully handled. In one implemented system, an em-ployee who reported identifying a priority problem beganreceiving system-generated requests for progress reportsto be forwarded to the Chief Executive O fficer! Thisquickly led to the end of priority problem reporting. Thevigilant system noted that employees had stopped usingthe system, and alerted the administrator. The employeesdealt with the resulting complaints by writing programsthat periodically opened files and changed dates, whichsatisfied the watchful, automatic monitor. Thus sabo-taged, the work management application was of littleuse, and was eventually quietly withdrawn. (Carroll Hall,personal communication.)Case 3. Natural language interfaces to shareddatabases.Natural language is not usually included in treatments ofCSCW , but it is typically described as an interface stylethat by virtue of familiarity will appeal to subsets of users-- novice and casual or intermittent users -- with otherinterfaces available for heavy users (Rich, 1984;Shneiderman, 1987). Thus, it is in fact portrayed asuseful in group work settings, and will seem moreattractive as we address the problems of designinginterfaces that must appeal to almost all users. AScomputer systems offer more capability, casual use of agiven feature will increase, perhaps become the norm.Within a group, frequency of use of a CSCW applicationwill inevitably be uneven; natural language could make iteasier for some users to enter and retrieve information.Database access seems a logical target application: thedomain is circumscribed, much of the necessaryvocabulary is explicitly set down in the database field andrecord labels, and the interaction -- user query followedby system response -- is predictable and limited. Naturallanguage interfaces to databases have been available forseveral years.Over the last ten years, most major developers of officesystems have undertaken natural language projects andover fifty software companies have entered the field(Foley, 1986). While absorbing 1000 developers andhundreds of millions of dollars, none of these ventureshad been profitable by 1985 (Johnson, 1985). (While oneor two companies marketing databases with naturallanguage interface options have since reported profitabil-ity, surveys have shown that the natural language featurewas not responsible; Paul Martin, personal communica-tion, 1988.) A survey of the natural language industryconcluded its story is not the stuff of which venturecapitalists dreams are made, (Johnson, 1985).We need to understand two things: why has naturallanguage failed to meet expectations and how has itattracted such high levels of support?

89

7/28/2019 p85-grudin

6/9

Problenls of tutural language interfaces to databases. Natu-ral language understanding is incredibly complex: there isnot yet a complete theory of syntax and semantics andpragmatics may be even more difficult. While a databaseinterface based even on primitive linguistic approachescan correctly respond to a high percentage of the limitedrange of queries it encounters, it is not clear how anoccasional error will affect the users overall confidencein the system. If you ask for the average secretarys salaryat the U.N. and are told $SOE; ecause it has averaged inthe General Secretary of the U.N., you may cease to tru.;tthe system (Paul Martin, persona! communication).Another potential problem is coverage. People rely on ahuge, structured knowledge acquired over years in orderto understand simple things, more than existing systemscan hope to incorporate (see e.g., Bobrow et al., 1977). Arelated problem is that users may expect an applicationthat handles English to exhibit broad human intelligenceand be disappointed when it does not (Rich, 1984). Richalso notes that the natural language user must do a lot oftyping, although users can and do develop truncatedpidgin languages that may end up more concise thancomplex queries in a formal query language. And onemust also consider the conservatism of the databasemarket and the need to develop appropriate marketingstrategies as contributing factors to the poor reception fornatural language interfaces.Finally and more speculatively, natural language may notbe matched to the tasks for which computers are used. Inhuman interactions, we gravitate toward more formallanguage when we are uncertain of our audience, whenwe will get minimal feedback and opportunity to correctmisinterpretations, and when we desire precise responsesby the listener. Ail of these are characteristic o f hurnan-computer interaction. Perhaps if neural net or connec-tionist systems succeed in giving computers a morefuzzy, human-like intelligence, natural language will bea good match.The attrwtion of a natwal language interface to databases.Perhaps more important than the circumscribed domainof a database and the explicit, built-in terminology arethe problems outlined in this paper. The casual databaseuser is the beneficiary of the natural language interface.The heavy user pays the price of additional keystrokesand reduced precision. The truly heavy user may workprimarily by creating and modifying command files forfrequently-issued complex queries in the formal querylanguage. The heavy user always retains the option ofusing the formal query language that accompanies thenatural language interface, but if that query language isnot the best available formal interface, the heavy userwould pay a price to use the system.The manager and executive can envision themselves ascasual users of a shared database, with others delegatedto enter the data and carry out routine queries. Thus,natural language interfaces may appeal to decision-mak-ers. But once again, decision-making intuition has failedif frequent users, the principal users of databases, prefernot to do the extra work that choosing such a system mayentail.

Case 4. Group decision support systems.The many efforts to develop computer support for groupdecision-making have generally produced systems, but itis clear that group decision support will benefit consider-ably by integrating with the systems people use for otheraspects of their work and will thus become a CSCWapplication area. Such applications are already underdevelopment. At CSCW86, Kraemer and King reviewed alarge number of group decision support systems andconcluded that their current reality is far less than mightbe expected given their need and promise, and thatalthough some for-profit companies have undertaken tobuild (group decision support system s), they are not yetmaking much money, (Kraemer and King, 1986).While they vary considerably in character, group decisionsupport systems are highly susceptible to the problemsoutlined in this paper. They are expressly designed to beof principal benefit to decision-makers, insofar as oneperson is primarily responsible for the outcome of a meet-ing or a group decision process (undoubtedly the norm inOUT culture). The amount of work required of others tolearn and use the system may vary. If use of the systemrequires significant learning, requires putting informationon-line to make it publicly available, records informationthat a participant would prefer not to leave the meeting,blocks other means to influence decision-making (such asprivate lobbying), or undermines management authority,then the system may encounter resistance.Conclusion.Computer support for the activities of individuals in theirgroup and organizational contexts will unquestionablychange the way people live in significant ways. It isdifficult to imagine anything more important or fascinat-ing than trying to understand and guide that change. Theanalyses in this paper suggest that we are just beginning.Progress has been technology-driven to a surprising de-gree -- technologies searching for needs, as one panelorganizer described it (Aucella, 1987). Many of us areaware of this general problem, pointed out by Engelbart(1982, 1985) and others, but its specific manifestationsmay continue to elude analysis, much less solution.We need to have a better understanding of how groupsand organizations function and evolve than is reflected inmost of the systems that have been developed. At thesame time, we also need to know more about individualdifferences in responding to technology if we are todevelop systems that can support entire groups. Oneapproach may be the contextual research of JohnWhiteside and his colleagues (Whiteside, Bennett, andHoltzblatt, 1987). Another is that used at AarhusUniversity in Denmark: The Aarhus people start out witha problem situation defined by workers, and work besidethem a long time in order to develop a new system that isowned by the workers... This is very different fromtraditional systems development, as you can imagine, andyou cant simply package a set of techniques to do thejob...see Ehn and King (1987). (Liam Bannon, personalc:ommunication).

90

7/28/2019 p85-grudin

7/9

We must also develop a better behavioral understandingof our own decision-mak ing processes as researchers anddevelopers. The intuitions that have guided us in the pastare breaking down. If we are going to support groups thatinclude any diversity at all, we will have to learn muchmore about how different kinds of people work. Veryfrequently we see researchers studying other researchers,developers building systems because the technologyexists, and managers supporting the development ofsystems that will appeal to other managers. We mustmake a strong effort to broaden our intuitions becauseexperiments in the cooperative work area are soexpensive and time-consuming.

Appendices.

Analogy 1. Multi-user systems and multi-userapplications.Most of our experience with computer support for groupactivity is based on the introduction of entire systems ntoan organization. I do not intend multi-user system toinclude a central, timesharing computer that essentiallysupports several individuals using individual applications,but rather a system that includes hardware and softwaredeveloped to support group activity, such as a management information system, a computer-integratedmanufacturing system, or an inventory control system.Multi-user application refers to software (and possiblyminor hardware) acquired with the intent of integrating itinto an existing computer system, such as a co-authoringprogram.Computer support of group activity has typically requiredthe acquisition or development of an entire systembecause the prerequisites -- multi-tasking, networking,interactive interfaces, and computer literacy -- were notin place. But as more advanced environments becomewidespread and people are comfortable with the terminal,PC, or workstation on their desk, systems will have to giveway to applications. Today, an entire work managementsystemmight be installed, replacing or absorbing existingtechnology, but tomorrow a work or project managementupplication will be sought, to run on an existing system.Cooperative applications that are appearing includeco-authoring aids, sophisticated mail and time manage-ment, voice applications, shared databases, sharedfinancial analysis packages, etc.Our experience with multi-user systems, whether director through the literature, may influence our approach tomulti-user applications. They have similarities -- theymay serve the same purpose and behave much the same.But there are critical differences, particularly at the timeof introduction: the system has a much higher cost,greater visibility, and stronger commitment o f uppermanagement. As a result, a new system brings with it theexpectation of organizational change. While an organiza-tion will also adapt or evolve following the introduction ofa CSCW application, the far less expensive application

will not carry the same visibility, commitment, andexpectation of change. From the perspective of the user,the introduction of the application must be smoother. Thismakes the job of the designer and implementer moredifficult (see Pew, 1986).The strong management commitment to ensuring thesuccess of a new system means that a) the collectivebenefit of the system is recognized to be high; b) theorganization may create new jobs to achieve success, ifnecessary; c) if a few important individuals will not orcannot use the system (the manager who wont use aterminal, for example), ways to work around them maybe found; d) pressure from management to try the systemmay be high (whether through leadership and positiveexample or through more coercive approaches). Evenwith these forces working to the advantage of the system,we know that successful implementation is difficult.Introducing CSCW applications without this backing, allelse being equal, will be more difficult. Better design andimplementation are ways to ensure that all else isntequal. (The application may have the advantage of findinga higher level of computer literacy, since a system isalready in place.)The much less expensive application program is likely toprovide a smaller or uncertain collective benefit and wonthave the same degree of management commitment. Theorganization cannot restructure itself around each newapplication, nor will management be likely to work ashard to ensure full participation. To a greater degree, theapplication must fit into existing work patterns and appealto all the people needed to support it. For many of thesecommunication-centered applications, this may be every-one: The application program may require full groupparticipation without the advantage that a system oftenhas of choosing or defining its users.

Analogy 2. Single-user applications and multi-userapplications.Whether we are researchers, designers, implementers,users, evaluators, or managers, most of our computerexperience has almost certainly been with single-userapplications. This experience has inevitably influencedthe skills we have acquired, the intuitions we havedeveloped, and the way we view our work. When we findourselves thinking about or working with a CSCWapplication, it is useful to examine our approach carefullywith this in mind, as many of our skills, intuitions, andoutlooks will not help us in this different domain.One effect of working with single-user applications is thatwe do not train ourselves to think extensively in terms ofthe disparity between the benefit obtained by and thework required of different user categories. We do ofcourse give some consideration to the novice, casual user,heavy user distinctions, but in general, we can rely onfeedback from a few typical users. This experience maylead us to be unaware that we are only viewing a CSCWapplication from the perspective of the primary intendeduser, the user who obtains the most direct benefit. For

91

7/28/2019 p85-grudin

8/9

managers, this may have the effect of biasing their judg-ment regarding the CSCW application. A manager withgood intuition might look at the design of an editor andcorrectly surmise I would like these features and I thinkmost users would. Looking at a CSCW application, themanager might surmise I would like these features and Ithink most users would, but only be correct insofar asthe other users are also managers. The single-user appli-cation does not train us to consider users of the sameproduct who have a crucial but entirely different engage-ment with it.Another effect of working with single-user applications isthat we do not acquire the very different evaluative skillsthat CSCW applications will require. Most human factorsengineers and other user interface specialists are versedprimarily in applying techniques from perceptual, cogni-tive, and motor psychology to study phenomena of rela-tively brief duration. The one-hour experiment is stilltypical, and a study involving even a few sessions overseveral days is rare. But the group processes that willinfluence and be influenced by the use of CSCW applica-tions bring soc ial, motivational, economic, and politicalfactors into prominence, and the temporal granularityrequired to understand such dynamics is much larger.

When is job redesign justifiable?Central to this paper is the point that many CSCWapplications wil! directly benefit certain users, oftenmanagers, while requiring additional work from others. Atraditional method of coping with such a problem *is tocreate new jobs or redesign existing jobs -- in short, torequire people to do the additional work. Technology andorganizational change is covered in depth elsewhere (e.8.,Kraut, 1987a, 1987b; Crowston and Malone, 1987). Thispaper comments more on how things are than on howthey might be, so I will limit myself to a few observations.First,as noted in the paper, CSCW applications will nothave recourse to changing job requirements to the degreethar often occurs when entire systems are installed. Theinvestment and commitment are smaller and theorganization wont tolerate significant disruption for eachnew application acquired. CSCW applications will have tobe more group-friendly than systems have been. Theywill change the organization, but more gradually. For thisreason. the focus of CSCW will shift to user interfaceissues to minimize the disruption and additional workrequired of uny user of the application.Second, there may be a shift toward greater egalitarian-ism in the workplace (see e.g. Chems, 1980), some of itsurface and some of it perhaps a deeper emphasis onmanaging by building consensus. Therefore, it may bemore difficult for management to mandate participationin new applications unless the collective benefit is veryevident.

Third, when the collective benefit of using an applicationdoes appear great enough to warrant requiring somepeople to accept new or different tasks -- andmeasurements of collective benefit are of course difficult-- management has several options. Educating all usersto the collective benefit may create a willingness to do thework. Inspiring through example or positive leadership isanother approach. And, of course, improving the userinterface to minimize the work or providing compensatorybenefits in another area will help.Finally, in som e cases the work will be made part of thejob. Setting aside tasks that mos t people would agree noone should be asked to do, the discomfort from job rede-sign is often transitory: Those hired with an understandingof the new requirements will be less uncomfortable withthem than those living through the change.Consider the example of programmer documentation ofsoftware code. Twenty years ago programmers writingentirely undocumented code might have been unhappy ifforced to change for the collective benefit to the companyof having maintainable software. But today moreprogrammers are educated and socialized to accept this aspart of their work; it is written into job descriptions, thosetaking the job are reasonably content to do it.This is a cursory treatment of a difficult ethical topic, butanything more is, as they say, beyond the scope of thispaper.

Acknowledgment.This paper owes a lot to published and personalcommunications of Susan F. Ehrlich and to LiamBannons insightful comments. Clarence Ellis, DonGentner, Donald A. Norman, Gail Rein, and Elaine Richalso contributed Useful comments and encouragement. Iam especially grateful for conversations on specific issueswith Carroll Hall, Paul Martin, and Steven Roth. ClarenceEllis, Simon Gibbs, Bill Kuhlman, Steve Poltrock, GailRein, and I explored the significance of a groups positionon the continuum from a small, homogeneous team to alarge, heterogeneous organization using GROVE, aCSCW application developed by the MCC SoftwareTechnology Program to support brainstorming, leading tomy greater appreciation for the importance of this factor.Many of the ideas in this paper were developed from apaper delivered at Interact87 (Ggdin, 1987).

92

7/28/2019 p85-grudin

9/9

References.Aucella, A:F. (moderator), 1987. Voice: Technologysearching for communication needs. In Proc. CHZ+GI87 Human Factors in Compuring Systems (Toronto, April5-9, 1987), pp. 41-44.Aucella, A.F . and Ehrlich, S.F., 1986. Voice messaging:Enhancing the user interface based on field perform-ance. In Proc. CHI 86 Human Factors in Computing

Systems (Boston, April 13-l 7, 1986), pp. 156-161.Bobrow, D.G., Kaplan, R.M., Kay, M., Norman, D.A.,Thompson, H., and W inograd, T., 1987. Gus, aframe-driven dialog system, Artificial Intelligence, 8,pp. 155-173.Butler, K., Bennett, J., and Whiteside, J., 1987. Engineer-ing objectives for usability. Tutorial presented atCHI+GI 87 Human Factors in Computing Systems(Toronto, April 5-9, 1987).Carasik, R.P. and Grantham, C.E., 1988. A case study ofcomputer-supported cooperative work in a dispersedorganization. In Proc. CHI 88 Human Factors in Com-puting Systems (Washington D.C., May 15-19, 1988),pp. 61-66.Chems, A.B., 1980. Speculations on the social effects ofnew microelectronics technology. Innternational LubourReview, 119, 6, pp. 705-721.Crowston, K. and Malone, T.W., 1987. Information tech-nology and work organization. CISR WP No. 165.Cambridge, MA: MIT Sloan School of Management.Ehn, P., and Kyng, M., 1987. The collective resourceapproach to systems design. In B jet-knes, G., Ehn, P.,and Kyng, M. (Eds.) Computers and democracy - aScandinavian challenge. Aldershot, UK: Cower.Ehrlich, SF., 1987a. Social and psychological factorsinfluencing the design of office communication sys-tems. In Proc. CHI+GI 87 Human Factors in ComputingSystems (Toronto, April S-9. 1987), pp. 323-329.Ehrlich, S.F., 1987b. Strategies for encouraging success-ful adoption of office communication systems. ACMTOOIS, 5, pp. 340-357.Engelbart, D.C., 1982. Towards high-performance knowl-edge workers. OAC 82. Reprinted in Greif, 1988.Engelbart, D.C., 1985. Plenary address, CHl 85 HumanFactors in Computing Systems (San Francjsco, April18, 1985).Foley, M.J., 1986. Teaching computers plain English.High Technology, May, 1986.Gaffney, C.T., 1985. Avoiding the seven deadly sins ofOA implementation. In Proc. Syntopican X111MakingBusiness Systems Effective (Washington, D.C., June17-20, 1985), pp. 241-254.Greif, I. (Ed.), 1988. Computer-supported cooperative work:a book of readings. San Mateo: Morgan Kaufmann.Grudin, J., 1986. Designing in the dark: Logics thatcompete with the user. In Proc. CHI 86 Human Factorsin Compvting Systems (Boston, April 13-l 7, 1986), pp.281-284.

Grudin, J., 1987. Social evaluation of the user interface:Who does the work and who gets the benefit? In Proc.INTERACT87 (Stuttgart, September I-4, 1987), rj;p.805-811.Johnson, T., 1985. Natural language computing: the com-mercial applications. London: Ovum Ltd.Kling, R., 1987. The social dimensions of computeriza-tion. Plenary address given at CHI+GI 87 HumanFactors in Computing Systems (Toronto, April 5-9,1987).Kraemer, K. and King, J., 1986. Computer-based systemsfor group decision support: Status of use and prob-lems of development. In Proc. CSCW Conference onComputer-Supported Cooperative Work, (Austin, De-cember 3-5, 1986), pp. 353-375.Kraut, R.E. (Ed.), 1987a. Technology arid the transforma-tion of white-collar work. Hillsdale: Lawrence ErlbaumAssociates.Kraut, R.E., 1987b. Social issues and white-collar tech-nology: an overview. In Kraut (1987a), pp. 1-21.Lucas, H.C., Jr., 1976. The analysis, design and implemen-tation of information systems.New York: McGraw-Hill.Malone, T.W., 1985. Designing organizational interfaces.In Proc. CHI 85 Human Factors in Computing Systems[San Francisco, April 14-18, 1985), pp. 66-71.McCracken, D.L. and Akscyn, R.M., 1984. Experiencewith the ZOG human-computer interface system. Int..I. Man-Machine Studies, 21, pp. 293-310.Newell, A.F., 1984. Speech -- the natural modality forman-machine interaction? In Proc. INTERACT 84IFIP Conference on Human-Computer Interaction, (Lon-don, September 4-7, 1984), pp. 231-235.Pew, R. (moderator), 1986. Socio-tech: What is it (andwhy should we care)? In Proc. CHI 86 Human Factors

in Computing Systems (Boston, April 13-I 7, 1986), pp,129-130.Rich, E., 1984. Natural-language interfaces. Computer,September, 1984, pp. 39-47.Rowe, C.J., 1985. Identifying causes of failure: a casestudy in computerized stock control. Behaviour andInformatipn Technology, 4, pp. 63-72.Sathi, A., Morton, T.E., and Ro\h, S.F., 1986. Callisto:An intelligent project management system. AI Maga-zine, Winter, 1986, pp. 34-52. Reprinted in Greif(1988), pp. 269-309.Shneiderman, B., 1987. Designing the user interface. Read-ing: Addison-Wesley.White, K .B., 1985. Socio-technical task team design. InProc. Syn topican XIII Making Business Systems Effective(Washington, D.C., June 17-20, 1985), pp. 32-35.Whiteside, J., Bennett, J., and Holtzblatt, K., 1988. Us-ability engineering: our experience and evolution. InM. Helander (Ed.), Handbook of human-computerinteraction. Amsterdam: North-Holland, in press.

93