complexity theory and the organization: beyond the metaphor

5
36 C O M P L E X I T Y Complexity Theory and the Organization: Beyond the Metaphor ROGER LEWIN, TERESA PARKER, AND BIRUTE REGINE Birute Regine is a visiting scholar at Wellesley College, MA, where she is researching a relational approach to the workplace. Roger Lewin, an Associate of the Peabody Museum, Harvard University, is author of Complexity: Life at the Edge of Chaos. Together, Regine and Lewin are writing a book titled The Soul at Work: Complexity Theory and Business, as if People Mat- ter, to be published by Simon and Schuster, January 1999. Teresa Parker leads a research team studying business applications of complexity at the Ernst & Young Center for Business Innova- tion. Noting that many advances of CAS in business have been made in manufacturing, she has a special interest in identify- ing implications for the overriding concerns of management – growth and change. © 1998 John Wiley & Sons, Inc., Vol. 3, No. 4 CCC 1076-2787/98/04036-05 C omplexity theory offers an appealing metaphor for ana- lyzing organizational behavior. Fundamental to com- plex adaptive systems (CAS) is the emergence of high- level order from low-level interactions among heteroge- neous, autono- mous agents, each guided by a few simple rules. To look at business organizations as complex adaptive systems, then, is to see their prop- erties as emerg- ing from the in- teractions among people in the workplace and with the environ- ment they share. The metaphor accords with ex- perience because it suggests that the essence of business organi- zation is what in- dividuals do, not what executives plan. We examine two investiga- tions, both based on this CAS view, but very different in approach, and describe how the two ap- proaches might be combined to develop a management analy- sis tool. The first uses an agent-based computer model of so- cial behavior known as Sugarscape, developed by Robert Axtell and Joshua Epstein of the Brookings Institution[1]. Axtell adapted this simulation approach to show how the interac- tion of individuals, following simple rules defining their pref- erences for work and leisure, relates to the growth, life span, and death of firms[2]. The second is a qualitative study, by Birute Regine and Roger Lewin, of ten business organizations in four coun- tries—all firms pursuing a complexity-oriented manage- ment practice. Like the work of Ralph Stacey, who intro- duced the idea of the “shadow organization[3],” their analysis has identified informal relationships as powerful in shaping organizations. It argues, too, that these infor- mal relationships are the source of the emergent cultural features and performance of companies. Their analysis has led to development of a framework that categorizes these relationships among the members of an organization and among organizations. One of the hopes for CAS theory is that it will open social science to the analytical approaches of physical science. This article considers the state of the art in modeling organizational behavior. Inroads using CAS techniques would be of particular value to business, because leaders often wish their organizations were other than what they are. A lot of effort goes into changing organizations—and by and large managers and their staff feel they get a very low return on the effort. This article looks at organizations as collections of relationships among agents, and at work being done to model this point of view. It ends with speculation about where this research may take us: toward simulation capability that can teach managers about better leadership interventions, especially those that foster adaptive organization and more connected, integrated individual experience of organization. Building simulations that can teach managers about better leadership

Upload: roger-lewin

Post on 06-Jun-2016

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Complexity theory and the organization: beyond the metaphor

36 C O M P L E X I T Y © 1998 John Wiley & Sons, Inc.

Complexity Theory and the Organization:Beyond the Metaphor

ROGER LEWIN, TERESA PARKER, AND BIRUTE REGINE

Birute Regine is a visiting scholar at Wellesley College, MA,where she is researching a relational approach to the workplace.Roger Lewin, an Associate of the Peabody Museum, HarvardUniversity, is author of Complexity: Life at the Edge of Chaos.Together, Regine and Lewin are writing a book titled The Soulat Work: Complexity Theory and Business, as if People Mat-ter, to be published by Simon and Schuster, January 1999. TeresaParker leads a research team studying business applications ofcomplexity at the Ernst & Young Center for Business Innova-tion. Noting that many advances of CAS in business have beenmade in manufacturing, she has a special interest in identify-ing implications for the overriding concerns of management –growth and change.

© 1998 John Wiley & Sons, Inc., Vol. 3, No. 4CCC 1076-2787/98/04036-05

Complexity theory offers an appealing metaphor for ana-lyzing organizational behavior. Fundamental to com-plex adaptive systems (CAS) is the emergence of high-

level order from low-level interactions among heteroge-neous, autono-mous agents, eachguided by a fewsimple rules. Tolook at businessorganizations ascomplex adaptivesystems, then, isto see their prop-erties as emerg-ing from the in-teractions amongpeople in theworkplace andwith the environ-ment they share.The metaphoraccords with ex-perience becauseit suggests thatthe essence ofbusiness organi-zation is what in-dividuals do, notwhat executivesplan.

We examinetwo investiga-tions, both basedon this CAS view,

but very different in approach, and describe how the two ap-

proaches might be combined to develop a management analy-sis tool. The first uses an agent-based computer model of so-cial behavior known as Sugarscape, developed by Robert Axtelland Joshua Epstein of the Brookings Institution[1]. Axtelladapted this simulation approach to show how the interac-tion of individuals, following simple rules defining their pref-erences for work and leisure, relates to the growth, life span,and death of firms[2].

The second is a qualitative study, by Birute Regine andRoger Lewin, of ten business organizations in four coun-tries—all firms pursuing a complexity-oriented manage-ment practice. Like the work of Ralph Stacey, who intro-duced the idea of the “shadow organization[3],” theiranalysis has identified informal relationships as powerfulin shaping organizations. It argues, too, that these infor-mal relationships are the source of the emergent culturalfeatures and performance of companies. Their analysis hasled to development of a framework that categorizes theserelationships among the members of an organization andamong organizations.

One of the hopes for CAS theory is thatit will open social science to theanalytical approaches of physicalscience. This article considers thestate of the art in modelingorganizational behavior. Inroads usingCAS techniques would be of particularvalue to business, because leadersoften wish their organizations wereother than what they are. A lot ofeffort goes into changingorganizations—and by and largemanagers and their staff feel they geta very low return on the effort.

This article looks at organizationsas collections of relationships amongagents, and at work being done tomodel this point of view. It ends withspeculation about where this researchmay take us: toward simulationcapability that can teach managersabout better leadership interventions,especially those that foster adaptiveorganization and more connected,integrated individual experience oforganization.

Building simulations that can teach managers about better leadership

Page 2: Complexity theory and the organization: beyond the metaphor

C O M P L E X I T Y 37© 1998 John Wiley & Sons, Inc.

THE PROBLEM OF ORGANIZATIONAL INTERVENTION

Much of the work of management—its organizationaldesign, restructuring, and human resources efforts—is aimed at getting people to shift priorities or improve

performance. In doing this work, managers have learned thatchange doesn’t happen simply because they plan or man-date it.

Managers are now aware of communication networks,trust, and an array of lessons from psychology confirming thatthe ways people change are more complicated thanbehavioralists’ stimulus-response models supposed[4]. Man-agers have begun to apply this better understanding of peopleas they replace traditional illustrations of relationships, likeorganizational charts, with greater acknowledgment of infor-mal organization.

The problem with efforts that aim to motivate change isthat their impact is difficult to evaluate. Robert Eccles andNitin Nohria decry the proliferation of massive organizationalchange programs with themes like “empowerment” and “cul-ture”[5]. The fact is, these programs have not produced provenresults, although management consultants eagerly promotethem. In Eccles and Nohria’s assessment, these programs con-tribute to a stream of activities that make management feel itis doing something right. However, as the activities build onone another, they are rarely subjected to careful evaluationthat might reveal the important relationship between inter-vention and outcome.

J effrey Pfeffer makes a different argument about theproblem of evaluating the impact of programs aimed atempowering people and nurturing culture. Studies of

such programs have been made, he says, and do show a strongcorrelation with enhanced economic performance. However,the problem he points to is that the causal link between man-agement program and outcome is not well understood[6].

We propose that CAS-based simulations will help makesuch correlations more than metaphorical—that they canbecome powerful tools for managers if they incorporate thekinds of interpersonal exchanges that drive organizationalperformance. Moreover, in such simulations, the relationshipsbetween parameter characteristics and system-wide behav-ior can be tested. Simulation, then, shows promise as a prov-

ing ground for what have so far been management’s least wellunderstood efforts.

THE FIRM-BUILDING MODELTraditional economics has produced models of social systemsthat are often simplistic and unrealistic. Most are built on as-sumptions such as populations of rational individuals en-dowed with perfect information. There is no heterogeneity ofagents in such models; they look instead at averages. And eachindividual in these models makes decisions based on an un-constrained ability to maximize fixed appetites, such as pref-erences for work and leisure. ( The degree to which eachachieves its preference determines its “welfare.”)

Conventional modeling of the formation of firms—theirbirth, growth, and death—has focused principally on expla-nations that involve macroeconomic phenomena, such aseconomies of scale and the terms of trade[7]. By contrast, thecomplexity theory perspective identifies the micro-domain asthe primary explanatory level, with firm building an emergenteffect at the macro level.

Axtell’s agent-based model tests this latter hypothesis byallowing agents to decide whether to join a firm based on howwell it satisfies its work-leisure preferences. In the model, com-panies grow because cooperation maximizes productivity. Butbecause the fruits of productivity are shared equally withinfirms, as companies increase in size, they begin to attract someagents who are less productive (“free riders”). When overallproductivity falls below levels that satisfy the more ambitiousagents, these leave the firm to join more productive ones. Thesystem never reaches equilibrium: Firms are created and dis-integrate continually.

A xtell’s model is clearly based on some of the same sim-plistic assumptions that underlie traditional economicformulas. As he observed of the work-leisure calcula-

tion, “This is a very stylized picture of how humans behave”[8].But it is different in that the agent population is heteroge-neous, and agents interact with each other to produce thephenomenon of the firm.

Axtell discovered that the firm size distribution the modelproduced resembled the real world; plotted on a log/log scale,the distribution of company size, he found, was a power lawwith a negative exponent close to 2[9]. The existence of a powerlaw distribution in a complex system is often taken to implythat the system is operating in the zone between stability andchaos, or at the edge of chaos[10, 11], although some ques-tion this assumption[12]. In any case, as in the real world, themodel’s economy was made up of many small companies andonly a few large ones; none of them lived forever.

It is remarkable that Axtell’s simple agents create even anarrow kind of verisimilitude. These dynamics appear aschanging patterns on a computer screen and encouragespeculation about the kind of culture that might be repre-sented by them. For instance, as individuals leave one orga-

The real potential of this modeling approach,however, is in identifying which parameters are

important in establishing the emergent culture ofthe workplace and creating simulations that helpfirms to discover the reliable interventions—in

compensation systems, hiring approaches,insourcing and outsourcing, organization

structure, etc.—that management should make.

Page 3: Complexity theory and the organization: beyond the metaphor

38 C O M P L E X I T Y © 1998 John Wiley & Sons, Inc.

nization to join another, what overall effect does that have onthe firm? And on the larger economy of firms?

The real potential of this modeling approach, however, isin identifying which parameters are important in establish-ing the emergent culture of the workplace and creating simu-lations that help firms to discover the reliable interventions—in compensation systems, hiring approaches, insourcing andoutsourcing, organization structure, etc.—that managementshould make. The welfare utility function is obviously limited.The work of Regine and Lewin could be one source of candi-date parameters. A simple example of how field work like theirscould contribute to different measures: their subjects havesaid that they join firms for other reasons. And their workdocuments how people often stay with a company, even if theycould easily join another where they would be better paid,because of loyalty. Loyalty is created by many factors, such aspersonal relationships, commitment, and alignment with thepurpose of the organization. The challenge, therefore, is todiscover what behaviors should be used to define parametersoutside the traditional economic utility function for model-ing organizations.

FIVE DOMAINS OF CONNECTIONIn their ongoing study, Regine and Lewin interviewed a broadselection of people—fromCEOs to secretaries—in tenorganizations. Their investi-gation covered companiesdiverse in size and businessactivity, from DuPont, theglobal chemical company, toa small family-owned chainof paint stores, and fromVeriFone, Inc., the high-techtransaction security firm toBroken Hill Proprietary, Ltd.an industrial mining com-pany. The companies theystudied were based in theUnited States, the UnitedKingdom, Germany, andAustralia[13].

Most companies were se-lected because their leadersclaimed to be managing ac-cording to complexitytheory–based principles.Previous interpretations ofCAS in management assertthat these approaches in-clude an emphasis on self-organization rather thanmanagement control; atten-tion to potentially exponen-

tial impact of small interventions; and encouragement ofchurn, redundancy, and conflict, which are seen to move theorganization away from a high degree of order and toward thehealthy edge of chaos [14-17]. A few were chosen for study be-cause their leaders (or outside business observers) said theypracticed these kinds of management approaches, eventhough they did not profess to be employing CAS theory.

The data for the analysis are linguistic; that is, Regine andLewin sought to identify patterns in words, phrases, andexpressions that people use, and the way people talk

about their interactions within and between organizations.Regine and Lewin listened to how people described day-to-day life in these workplaces, paying special attention to whatinterview subjects said they valued most about their organi-zations. Consistently, they noted, people at all levels of thesecompanies valued relationships highly. Regine and Lewin hy-pothesize that this confirms a focus on relationships as morethan metaphorical interpretation of CAS, for it is in fact thecentral experience of the complexity-oriented company.

From preliminary analysis, Regine and Lewin identified fivedomains of relationship: to oneself, to others, to the organi-zation (or CEO), to other organizations and the community,and ultimately to the global ecosystem, or environmental

FIGURE 1

Relational Model of Business Organizations.

Culture of Domain of Culture ofCommand and Control Connection Care and Connection

language of disconnection (parameter of connection) language of connection

Complacent ECOSYSTEM Accountable(responsibility)

“It’s not my problem.” “It’s not business or the environment.”

Competitive OTHER ORGS/COMMUNITY Adaptive(interdependence)

“We don’t need anyone.” “How can we be better together?”

Controlling ORG/CEO Allowing(purpose)

“It’s my way or no way.” “Go ahead, just try it.”

Complaining OTHER Acknowledging(mutuality)

“Who authorized that?” “You did a great job. Thanks!”

Calculating SELF Authentic(presence)

“I just do what I’m told.” “I strive to keep my word.”

Page 4: Complexity theory and the organization: beyond the metaphor

C O M P L E X I T Y 39© 1998 John Wiley & Sons, Inc.

awareness. The framework theydeveloped illustrates a nestedhierarchy of interactions withinand between organizations:they observed that each do-main of relationship is depen-dent on the quality of relation-ship in the domain below it(see Figure 1).

Regine and Lewin used thisframework in assessing the relative quality of life and eco-nomic success of the organizations they studied. Theyobserved that in organizations that pay attention to rela-tionships, CEOs and managers typically operate by a fewsimple rules. These include acknowledging workers’ efforts,demonstrating respect, creating opportunities for others,and valuing time spend talking with and listening to co-workers.

And although few of the organizations studied achievedpositive relationships at all five levels (most frequently absentwas relationship to the global ecosystem), those in which atleast three or four domains of relationship were healthy werehighly effective workplaces. That is, in Regine and Lewin’s analy-sis, relationship-oriented rules resulted in what they call anemergent culture of “care and connection.” They found thatsuch a culture produced more than just a “feel good” socialenvironment; it positively influenced the creativity, effi-ciency, adaptability, productivity and financial results in thesecompanies.

One immediate application of this qualitative work mightbe for managers to listen for the language of relationship intheir organizations to diagnose the status of relationships intheir organizations in terms of Regine and Lewin’s framework:presence, mutuality, purpose, interdependence, and respon-sibility (see Figure 1 for elaboration).

W e believe this kind of work will also produce manycandidate parameters that, brought into to an agent-based model like Axtell’s, could be tested against well-

defined measures of performance. This approach has alreadyshown some success in simulations of military units basedon a six-parameter representation of each soldier’s priori-ties[18]. Eventually, we believe, such parameters will replacethe crude work-leisure utility function. Regine and Lewin’sframework provides a useful starting point for developing inmanagement simulations what Robert Axelrod calls, “a thirdresearch methodology,” which combines the inductive skillsof students of business organization with the deductive mod-els of hard science[19].

NEW MODELING STRATEGIES: PRELIMINARY EXPLORATIONWe believe that modeling may offer a way for managers totest their interventions—something that rarely happens inbusiness life. This kind of modeling involves identifying agent

characteristics, relationshipdimensions, and figures ofmerit for organizations. Wehave suggested that qualita-tive, field-based work canproduce candidate param-eters for these. Without yetbeginning a simulation ex-periment, we have tried toimagine how insights from

qualitative field work might be married with agent-basedmodeling work.

A manager might want to understand whether one ofRegine and Lewin’s domains is dramatically more importantthan the other four in influencing an organization’s emergentculture and performance. Each parameter could be scaledfrom 0 to 1, and individual agents in a heterogeneous popula-tion could be endowed with a fixed amount of each param-eter, say 0.5 for “responsibility” and so on. A representation ofthe overall performance of the organization would have to bedefined so as to analyze the output of the model (e.g., speedof adaptation, rate of innovation, or efficiencies of resource).The relationship of “responsibility” (and other parameters) toperformance could then be tested.

Amanager might also want to know how, armed with thatinformation, to improve organizational perfor-mance. The model could be run to reveal how the cul-

ture changes as “responsibility” and other parameters change.By analogy with the Schelling tipping model[20], we wouldexpect the change in overall performance to be nonlinear,shifting via a phase transition, for example, when a certainthreshold of collective “responsibility” is passed.

Alternatively, the parameters for individual agents couldbe nonfixed, perhaps having a different potential range ineach of them. In this model, we could study how acompany’s performance might evolve as individuals inter-act and influence one another at the local level. Agentsrepresenting CEOs could be endowed with extra power toinfluence others.

Such a model could also be set up to investigateinterorganizational dynamics. For instance, in an economythat rewards those who form alliances with other companies—as in the computer industry—one might compare companyA, which scores high on “interdependence” (external connec-tivity) but low on “mutuality” (internal connectivity), withcompany B, which scores the reverse.

CHALLENGES AND BENEFITSCreating truly useful models of human dynamics may still bea long way off. The greatest challenge lies in translating in-sights into parameters, and finally into simple behavioralrules. It is not clear how to translate Regine and Lewin’s frame-work, with parameters like “presence,” and a corresponding

We see two valuable possibilities in bringingmodeling into organization consulting. The first is

its usefulness in testing interventions, asdiscussed above. The second is its potential toilluminate the relationship between individual

behaviors and system dynamics.

Page 5: Complexity theory and the organization: beyond the metaphor

40 C O M P L E X I T Y © 1998 John Wiley & Sons, Inc.

rule such as “authenticity” into the kind of welfare functionsthat are compatible with the current state of agent-basedmodeling. (Although people intuitively know what is meantby authenticity, it encompasses a myriad of behaviors thatmight be judged differently from one person to the next). Norare we certain how to create measures of organizational per-formance that are both broad enough and discriminatingenough to be useful.

Another problem may emerge from the success of thesemodels. Because simulations provide powerful visual expla-nations, they can make it easy to forget that models are stillvastly simpler than the real human context in the workplace.

REFERENCES1. J. M. Epstein and R. Axtell: Growing artificial societies: social science from the bottom up. The MIT Press, Cambridge, MA, 1996.2. R. Axtell: On the emergence of firms: an agent-based model. Forthcoming as a paper of the Santa Fe Institute.3. R. D. Stacey: Complexity and creativity in organizations. Berrett-Koehler, San Francisco, CA, 1996.4. J. Bruner: Acts of meaning. Harvard University Press, Cambridge, MA, 1990.5. R. G. Eccles and Nitin Nohria: Beyond the hype: rediscovering the essence of management. Harvard Business School Press, Cambridge, MA, 1992.6. J. Pfeffer: The human equation. Harvard Business School Press, Cambridge, MA, 1998.7. Y. Ijiri and H. A. Simon: Skew distributions and the sizes of business firms. Elsevier Science Ltd., New York, 1977.8. R. Axtell: personal communication.9. M. H. R. Stanley et al: Scaling behaviour in the growth of companies. Nature, 379: pp. 804-806, 1996.10. P. Bak and K. Chen: Self-organized criticality. Scientific American, January 1991.11. S. A. Kauffman and S. Johnsen: Coevolution to the edge of chaos: coupled fitness landscapes, poised states, and coevolutionary avalanches. Journal

of Theoretical Biology, at: 149: pp. 467-473, 1991.12. D. M. Raup: How nature works: the science of self-organized criticality. Complexity, 2(6): pp. 30-33, 1997.13. R. Lewin and B. Regine: The soul at work. Simon and Schuster, New York, in press.14. R. D. Stacey: Managing the unknowable: strategic boundaries between order and chaos. Jossey-Bass, San Francisco, CA, 1992.15. M. J. Wheatley and M. Kellner-Rogers: A simpler way. Berrett-Koehler, San Francisco, CA, 1996.16. K. Kelly: Out of control: the new biology of machines, social systems, and the economic world. Addison-Wesley, Reading, MA, 1994.17. D. W. Hock: The chaordic organization: out of control and into order. World Business Academy Perspectives, 9(1): 1995.18. A. Ilachinsky: Irreducible semi-autonomous adaptive combat (ISAAC): an artificial life approach to land warfare. a presentation of work developed

at the Center for Naval Analyses at the Ernst & Young Embracing Complexity Conference, Cambridge, MA, August 1997.19. R. Axelrod: Advancing the art of simulation in the social sciences. Complexity, 3; p. 17, 1997.20. T. C. Schelling: Micromotives and macrobehavior. Norton, New York, 1978.

Models, as Axtell and Epstein urge, should serve as laborato-ries[1]. Our findings there may teach us a lot about manage-ment, but they will not produce behavioral formulas. It is stillup to those of us who come together in organizations, or leadthem, to put what we learn into practice and estimate its im-pact in the real world.

W e see two valuable possibilities in bringing modelinginto organization consulting. The first is its useful-ness in testing interventions, as discussed above. The

second is its potential to illuminate the relationship betweenindividual behaviors and system dynamics.

HAVE YOURENEWED YOURSUBSCRIPTION

TO COMPLEXITY?FOR FAST SERVICE CALL

USA(800) 825-7550

OUTSIDE USA(212) 850-6347

E-MAIL

[email protected] Wiley & Sons, Inc. • Subscription Department • 605 Third Avenue • New York, New York 10158-0012

“Its academic pedigreeis impeccable.

Production qualityis terrific; it looks like a

newsstand magazine.Its contents

are organizedlike a magazine too;editorial material,

tutorial articles andsurveys outweighspecialist papers”

–Nature