10.16.03 learning organization complexity

8/8/2019 10.16.03 Learning Organization Complexity

1/75


2/75

Copyright (c) 2003 Russ Linden

About Russ LindenAbout Russ Linden

Russ Linden is a management consultant who specializes in organizational change methods. Since 1980, he

has helped government, non-profit and private-sector organizations develop leadership, foster innovation, andimprove organizational performance. He is an adjunct faculty member at the University of Virginia, Universityof Maryland, and the Federal Executive Institute. He writes a column on management innovations forTheVirginia Review, where he serves as Contributing Management Editor, and hasproduced nationalvideoconferences on re-engineering and on the human side of change.

Russ current teaching and consulting interests include the human side of change, systems thinking andlearning organization principles, re-engineering forseamlessservice, customer-focused organizations,strategic thinking and planning. He haspublished numerous articles, and his first book, From Vision to Reality:

Strategies of Successful Innovators in Government, came out in 1990. His next book, Seamless Government:A Practical Guide to Re-engineering in the Public Sector(Jossey-Bass, 1994), was excerpted in the May, 1995issue ofGoverning Magazine. His most recent book, Working Across Boundaries: Making Collaboration Workin Government and Nonprofit Organizations, waspublished in 2002. It was a finalist for the best book onnonprofit management in 2002. His clients have included the National Geographic Society, Departments of theNavy and Army, Health Data Services, Inc., U.S. Customs Service, U.S. Dept. of State, Metro InformationServices, Inc., Government of the Cayman Islands, U.S. Information Agency, FBI Academy, U.S. Dept. OfEducation, two state attorneys general and over four dozen state, local government, and non-profit agencies.Several local governments are redesigning their organizationsusing the principles in his book, Seamless

Government.

Before beginning his full-time practice, Russ was a Senior Faculty Member at the Federal Executive Institute.He served as the Director of Executive Programs at the University of Virginia's Center for Public Service,taught at the UVa McIntire School of Commerce, and managed in the human services field forseven years.

Russ Linden's bachelor's and master's degrees are from the University of Michigan. His Ph.D. is from theUniversity of Virginia. He lives in Charlottesville, VA with his wife and two children. He is on the web at:www.russlinden.com.


3/75


Organizational Learning: Why?Organizational Learning: Why? 66

Organizational Learning: What?Organizational Learning: What? 88

Workshop ObjectivesWorkshop Objectives 1010

Example of Organizational Learning: FEMAExample of Organizational Learning: FEMA 1111

Practices of Organizations That Learn and Adapt WellPractices of Organizations That Learn and Adapt Well 1616

Practice 1Practice 1: Understand, communicate the Main Thing: Understand, communicate the Main Thing 1717

Practice 2Practice 2: Create alignment by giving everyone a line of sight: Create alignment by giving everyone a line of sight 2626

Table of ContentsTable of Contents


4/75


Practice 3Practice 3: Utilize multiple perspectives to gain a systems understanding: Utilize multiple perspectives to gain a systems understanding 3333

PracticePractice 4: Defer to expertise4: Defer to expertise 4444

Practice 5:Practice 5: Maintain fluid communications across permeable boundariesMaintain fluid communications across permeable boundaries 5151

Practice 6:Practice 6: Support continual training, learning and practiceSupport continual training, learning and practice 6262

AppendixA: Five Leadership Skills That Foster LearningAppendixA: Five Leadership Skills That Foster Learning 7070

Appendix B: 9 Requirements for Organizational ChangeAppendix B: 9 Requirements for Organizational Change 7171

Appendix C: Partnering WorkshopsAppendix C: Partnering Workshops 7272

References and ResourcesReferences and Resources 7474

Table of ContentsTable of Contents


5/75


A Challenging EnvironmentA Challenging Environment ---- Some ExamplesSome Examples

The Marines new challenge: fighting in urban settings.The Marines new challenge: fighting in urban settings.

Internal Revenue ServiceInternal Revenue Service ---- caught in the financial andcaught in the financial andpolitical cross fire.political cross fire.


6/75


Why Organizational Learning? The C WordsWhy Organizational Learning? The C Words

Complexity:Complexity: Situations that are difficult to understand, haveSituations that are difficult to understand, haveconsiderable ambiguity and uncertainty, and often have noconsiderable ambiguity and uncertainty, and often have nosolutions, only options and tradeoffssolutions, only options and tradeoffs

Chaos:Chaos: Seemingly random events that have an underlyingSeemingly random events that have an underlyingpattern (which is difficult to discern)pattern (which is difficult to discern)

Change:Change: turbulent environments in which the future isturbulent environments in which the future isdifficult to predict or controldifficult to predict or control

The illiterate of the 21st century will not be the individual whocannot read and write, but the one who cannot learn, unlearn,

and relearn. -- Futurist Alvin Toffler


7/75


The C WordsThe C Words ---- Some examplesSome examples

Examples of complexity, chaos and change facing federalExamples of complexity, chaos and change facing federalagencies today: a starter list ...agencies today: a starter list ...

worldworld--wide epidemicswide epidemics ---- AIDS, SARSAIDS, SARS

urban sprawlurban sprawl ---- balancing demands for growth with environmental protectionbalancing demands for growth with environmental protection

fighting terrorism (while protecting civil liberties)fighting terrorism (while protecting civil liberties)

gaining boarder security (without harming economic activity)gaining boarder security (without harming economic activity)

information security (while dealing with privacy concerns)information security (while dealing with privacy concerns)

attracting/retaining quality employees (in an age of competitive sourcing)attracting/retaining quality employees (in an age of competitive sourcing)

dealing with rising (and often competing) customer expectations (with stabledealing with rising (and often competing) customer expectations (with stableor declining resources)or declining resources)

continually changing senior leadership prioritiescontinually changing senior leadership priorities


8/75


Organizational LearningOrganizational Learning ---- What?What?

In learning organizations, people are encouraged toIn learning organizations, people are encouraged to

reflect as they act, share lessons learned and other keyreflect as they act, share lessons learned and other keyinformation, so that they (and their customers) caninformation, so that they (and their customers) caneasily access the best thinking of the entireeasily access the best thinking of the entireorganization.organization.

Put more simply: In a learning organization,Put more simply: In a learning organization,when one of us gets smarter, we all can getwhen one of us gets smarter, we all can getsmarter.smarter.


9/75


Examples of Organizations That Manage TheseExamples of Organizations That Manage These

Challenges WellChallenges Well

FEMAFEMA

U.S. MarinesU.S. Marines

NAVAIR Manned Flight SimulatorNAVAIR Manned Flight Simulator

High Reliability OrganizationsHigh Reliability Organizations

Basketball, soccer, hockey teamsBasketball, soccer, hockey teams

FEIFEI


10/75


Workshop ObjectivesWorkshop Objectives

Understand why organizational learning can be a powerfulUnderstand why organizational learning can be a powerfulmethod for dealing with complexity, chaos and change,method for dealing with complexity, chaos and change,

Learn six management practices that will help you deal withLearn six management practices that will help you deal withcomplex and chaotic environments,complex and chaotic environments,

Apply the above to your organization, andApply the above to your organization, and

Leave with specific actions you/your associates can takeLeave with specific actions you/your associates can take


11/75


12/75


FEMA (cont.)FEMA (cont.)

Key Actions:Key Actions:

1.1. Witt worked hard on Congressional relations with both parties. He cut a deal with Sen.Witt worked hard on Congressional relations with both parties. He cut a deal with Sen.Mikulski: if he made fundamental changes in FEMA, she wouldnt try to close it down. HeMikulski: if he made fundamental changes in FEMA, she wouldnt try to close it down. Healso maintained his good relationship with the President.also maintained his good relationship with the President.

2.2. Witt: instituted a new business model of all hazardsWitt: instituted a new business model of all hazards ---- everyone would be deployed to anyeveryone would be deployed to anyhazard (nobody would work only on earthquakes, tornadoes, floods or hurricanes). Athazard (nobody would work only on earthquakes, tornadoes, floods or hurricanes). Atmultiple meetings he explained reasons for the change.multiple meetings he explained reasons for the change.

3.3. To support the new model, he abolished the old stovepipe structure (offices of flood,To support the new model, he abolished the old stovepipe structure (offices of flood,hurricanes, tornadoes, etc.), and replaced them with operational directorates that wouldhurricanes, tornadoes, etc.), and replaced them with operational directorates that wouldrespond to any emergency. This was done with significant input from employees.respond to any emergency. This was done with significant input from employees.

4.4. To emphasize the reality that FEMA had to change, and to reinforce his all hazards model,To emphasize the reality that FEMA had to change, and to reinforce his all hazards model,Witt rotated all FEMA SESers to different offices one Friday afternoon in 1993.Witt rotated all FEMA SESers to different offices one Friday afternoon in 1993.


13/75


FEMA (cont.)FEMA (cont.)

Some of the SESers were given offices they had publicly bad mouthed in the past (!)Some of the SESers were given offices they had publicly bad mouthed in the past (!)

Some (who showed little interest or talent for the new approach) were givenSome (who showed little interest or talent for the new approach) were givenassignmentsusually reserved for managers of lower grades (GM 14assignmentsusually reserved for managers of lower grades (GM 14--15)15)

Some 14sSome 14s--15s who had demonstrated competence and a desire to collaborate were15s who had demonstrated competence and a desire to collaborate weregiven offices that SESers had been runninggiven offices that SESers had been running

Witt made it clear he would rotate people again in the future, and he didWitt made it clear he would rotate people again in the future, and he did

5.5. Witt provided extensive cross training to many employees, which reinforced theWitt provided extensive cross training to many employees, which reinforced theallall--hazards model and gave them skills to handle their broader responsibilitieshazards model and gave them skills to handle their broader responsibilities

6.6. Witt also emphasized customerservice. He did this by:Witt also emphasized customerservice. He did this by:

talking about it at alltalking about it at all--hands meetingshands meetings

doing indoing in--depth research on what customers most needed from FEMA during andepth research on what customers most needed from FEMA during anemergency (their #1 need was to know that someone at FEMA cares about me)emergency (their #1 need was to know that someone at FEMA cares about me)

communicating customer feedback and expectations to the staffcommunicating customer feedback and expectations to the staff

creating three telecommunications centers that back up each other, so that callerscreating three telecommunications centers that back up each other, so that callersdont have long waits or busy signals during emergenciesdont have long waits or busy signals during emergencies


14/75


15/75


16/75


17/75


Practice 1:Practice 1: Understand, communicate the mainUnderstand, communicate the mainthingthing

Organizations that thrive on complexity are willing toOrganizations that thrive on complexity are willing tochange virtually everything, except whats at their core:change virtually everything, except whats at their core:

3M: Thou shalt not kill a new product idea3M: Thou shalt not kill a new product idea

Nordstrom: Service to the customer above all elseNordstrom: Service to the customer above all else

The principals mantra at an inner city public school: teach them to read!The principals mantra at an inner city public school: teach them to read!

High reliability organizations (aircraft carriers, hospital ERs, nuclearHigh reliability organizations (aircraft carriers, hospital ERs, nuclear

power plants): Safe operationspower plants): Safe operations

Marines: have an end state, commanders intent, main effortMarines: have an end state, commanders intent, main effort

FEMA: We all work on all hazards.FEMA: We all work on all hazards.


18/75


Practice 1: Understand, communicate the mainPractice 1: Understand, communicate the main

thingthing

As the authors ofAs the authors ofBuilt to LastBuilt to Lastpointed out, organizationspointed out, organizationsthat clearly identify their main thingthat clearly identify their main thing ---- a core value,a core value,purpose, or operating principlepurpose, or operating principle ---- have done very wellhave done very well

over time, even in chaotic, complex and changingover time, even in chaotic, complex and changingenvironments. They are willing to change almostenvironments. They are willing to change almostanything, but they protect whats at their core.anything, but they protect whats at their core.

The main thing is to keep the main thing, the mainthing!

-- former Netscape CEO Jim Barksdale


19/75


20/75


21/75


Understand, communicate the main thing: TheUnderstand, communicate the main thing: TheNAVAIR Manned Flight SimulatorNAVAIR Manned Flight Simulator

There are many ways toThere are many ways to

focuspeople on the main thing.focuspeople on the main thing.

Heres one model to try:Heres one model to try:


22/75


Understand, communicate the main thing: TheUnderstand, communicate the main thing: The

NAVAIR Manned Flight SimulatorNAVAIR Manned Flight Simulator

This model can help you manage and lead using a systemspoint ofThis model can help you manage and lead using a systemspoint ofview. The factors are interrelated; a change in any one of them willview. The factors are interrelated; a change in any one of them willripple through the organization. This model can help youpredict, andripple through the organization. This model can help youpredict, and

manage the ripples.manage the ripples.

Using this model, managers and staff can also keep the organizationUsing this model, managers and staff can also keep the organizationaligned around its main thing.aligned around its main thing.

They use a variation of this model at the Manned Flight Simulator;They use a variation of this model at the Manned Flight Simulator;employees there report that it helps them focus on their main thing, andemployees there report that it helps them focus on their main thing, andthat it reminds them of the need to keep all factors in mind whenthat it reminds them of the need to keep all factors in mind whenchanging any one of them.changing any one of them.


23/75


The NAVAIR Manned Flight SimulatorbusinessThe NAVAIR Manned Flight Simulatorbusiness

model (as shown on theirbusiness card)model (as shown on theirbusiness card)


24/75


The Manned Flight Simulatorbusiness model (asThe Manned Flight Simulatorbusiness model (as

shown on theirbusiness card)shown on theirbusiness card)


25/75


Practice 1:Practice 1: Understand,Understand,communicate the main thingcommunicate the main thing

Exercise:Exercise:

1. Whats your organizations main thing? What is at its core --purpose, function, value, operating principle -- that is essential to its

success and cannot be compromised?

2. How do you communicate the main thing to your direct reportsand associates? What else could you do to communicate it?


26/75


Practice 2:Practice 2: Create alignment around the main thingCreate alignment around the main thing

by giving everyone a line of sightby giving everyone a line of sight

Line ofsight means the employee:Line ofsight means the employee:

A. Sees how the organization works as a system, andA. Sees how the organization works as a system, and

B. Knows how her work contributes to it.B. Knows how her work contributes to it.


27/75


Practice 2:Practice 2: Create alignment around the main thingCreate alignment around the main thing

by giving everyone a line of sightby giving everyone a line of sight

Line of sightLine of sight mattersmatters. Without it . Without it

Employees operate in a vacuumEmployees operate in a vacuum

They see no connection between their work and the endThey see no connection between their work and the endproduct/end userproduct/end user

They often grow apathetic, play bureaucratic gamesThey often grow apathetic, play bureaucratic games

Theres no sense of meaning orpurpose in their workTheres no sense of meaning orpurpose in their work


28/75


Practice 2:Practice 2: Create alignment around the main thingCreate alignment around the main thingby giving everyone a line of sightby giving everyone a line of sight

Methods:Methods:

Use process mappingUse process mapping

Track and post performance, to makeTrack and post performance, to make

it visibleit visible

Use short and longUse short and long--term job rotationsterm job rotations

Experience the work process from the customerspoint of view/getExperience the work process from the customerspoint of view/get

prompt customer feedbackprompt customer feedback

Observe the product or end result the organization is in business toObserve the product or end result the organization is in business toachieve (e.g., at NASA, staff at all levels, and their families, areachieve (e.g., at NASA, staff at all levels, and their families, areinvited to witness the launches)invited to witness the launches)


29/75


Some other action items to create a line of sight:Some other action items to create a line of sight:

1. Bring some staff to your agencys senior management meetings, so they see the1. Bring some staff to your agencys senior management meetings, so they see thebigger picture.bigger picture.

2. Invite end users to meet directly with your staff. Ask them to describe for the staff:2. Invite end users to meet directly with your staff. Ask them to describe for the staff:

A. What they expect from your unit,A. What they expect from your unit,

B. What theyB. What they dodo with what you provide them, andwith what you provide them, andC. Any gaps that they perceive, between what they expect and what they get from you.C. Any gaps that they perceive, between what they expect and what they get from you.

Help the staff and the end users discuss these perceptions, so that they learn moreHelp the staff and the end users discuss these perceptions, so that they learn moreabout the others reality. If appropriate, negotiate a Service Level Agreement (SLA)about the others reality. If appropriate, negotiate a Service Level Agreement (SLA)between your unit and specific customer groups.between your unit and specific customer groups.

3. Co3. Co--locate certain units whose work is closely interrelated.locate certain units whose work is closely interrelated.

4. Rotate the convening role for your staff meetings: enable others to take on broader4. Rotate the convening role for your staff meetings: enable others to take on broaderresponsibilities.responsibilities.

5. Help staff get quick feedback on the results of their work, so they can manage with5. Help staff get quick feedback on the results of their work, so they can manage withnear realnear real--time information.time information.


30/75


Be sure everyone understands the purposeBe sure everyone understands the purpose

Some rotational assignments are shortSome rotational assignments are short--term (e.g., 6 months),term (e.g., 6 months),others are longer (2others are longer (2--3 years in the intelligence community)3 years in the intelligence community)

Start with volunteers (so its not seen aspunitive)Start with volunteers (so its not seen aspunitive) To raise the importance, invite people to bid for the opportunityTo raise the importance, invite people to bid for the opportunity

to have rotational assignmentsto have rotational assignments

Make it an expectation that people share what they learned fromMake it an expectation that people share what they learned fromtheir rotationstheir rotations

Make it in peoples career interest to seek rotations; giveMake it in peoples career interest to seek rotations; givepreference in promotions to those with several rotationspreference in promotions to those with several rotations

Method to create a line of sight: Job rotationsMethod to create a line of sight: Job rotations


31/75


32/75


Practice 2:Practice 2: Create alignment around the main thingCreate alignment around the main thingby giving everyone a line of sightby giving everyone a line of sight

1. What has helped you gain a line of sight when you worked at alower organizational level?

2. What are 2-3 things you and your associates could start doing, inthe near term, to help subordinates gain a greater line of sight?

Exercise:Exercise:


33/75


34/75


Practice 3Practice 3: Utilize multiple perspectives: Utilize multiple perspectives

At FEMA, they ensure a variety of perspectives through theAt FEMA, they ensure a variety of perspectives through theuse of matrix teams that are deployed to disasters. Theuse of matrix teams that are deployed to disasters. Theteams include people with different technical backgrounds.teams include people with different technical backgrounds.

FEMA also calls on several thousand reservists,FEMA also calls on several thousand reservists,individuals with expertise who train with FEMA staff and areindividuals with expertise who train with FEMA staff and arecalled upon for specific emergencies. They bring a freshcalled upon for specific emergencies. They bring a freshperspective because they have some distance from FEMA.perspective because they have some distance from FEMA.


35/75



Utilizing multiple perspectives can be threatening toUtilizing multiple perspectives can be threatening tomanagers. It helps to usemanagers. It helps to use structuredmethodsstructuredmethods that elicitthat elicitdifferent points of view (e.g., AfterAction Reviews), withdifferent points of view (e.g., AfterAction Reviews), withclear ground rules about no retribution.clear ground rules about no retribution.

Its also important to foster aIts also important to foster a cultureculture that supportsthat supportsdivergent perspectives. Marines are trained to lead, anddivergent perspectives. Marines are trained to lead, andpart of their training is to insist that subordinates are free topart of their training is to insist that subordinates are free todisagree (until the decision is made).disagree (until the decision is made).

Another way to gain multiple perspectives: bring inAnother way to gain multiple perspectives: bring inoutsiders with no vested interest who review and critiqueoutsiders with no vested interest who review and critiqueyour plans and operations.your plans and operations.


36/75


Practice 3Practice 3: Utilize multiple perspectives:: Utilize multiple perspectives:

Sometimes, you have to insist on itSometimes, you have to insist on it

Management theorist Peter Drucker tells the story of a seniorManagement theorist Peter Drucker tells the story of a seniorfinancial committee meeting at General Motors. Alfred Sloan, GMsfinancial committee meeting at General Motors. Alfred Sloan, GMsCEO, asked each of the executives on the committee for theirCEO, asked each of the executives on the committee for their

comments on a proposal they had heard. Each of the 10 memberscomments on a proposal they had heard. Each of the 10 membersgave a strong positive reaction. Sloan, too, said he approved of thegave a strong positive reaction. Sloan, too, said he approved of theproposal.proposal. Then he surprised the group:Then he surprised the group:

Since there are 11 votes in favor, and none against well table thisSince there are 11 votes in favor, and none against well table thisproposal until next months meeting. If we only see the positives inproposal until next months meeting. If we only see the positives in

the idea,and nobody sees any potential downside, we haventthe idea,and nobody sees any potential downside, we haventthought it through clearly.thought it through clearly.

Sometimes, leaders have to insist on counter views, in order to hearSometimes, leaders have to insist on counter views, in order to hearmultiple perspectives.multiple perspectives.


37/75



+

-

?

Ask your associates to identify the potential strengths and weaknesses ofa given proposal; ask them to list some questions they have about theproposal. This helps avoid groupthink, and getspeoples views on thetable.

Asimple toolfor elicitingmultiple perspectives:Asimple toolfor elicitingmultiple perspectives:


38/75



Whatever approach you use to gain multiple perspectives, this is aleadership task. Most subordinates wont offer differing views until

theyre convinced their leaders want to hear them.

How do you/can you ensure that your team provides a diverse set ofperspectives on complex, chaotic problems?

How can you get candid views from those outside your organization,when need be?

Exercise:Exercise:


39/75



A tool for using multiple perspectives: The Four FramesA tool for using multiple perspectives: The Four Frames

Structural frameStructural frame

Human resource frameHuman resource frame

Political framePolitical frame

Symbolic frameSymbolic frame


40/75


Exercise for the Four FramesExercise for the Four Frames

Think back to the last time you took a new job.

A. What were the key steps you took in your first 2-3 months?

B. Did those steps reflect one or two of the four frames?

C. In general: which frames do you use least? Which do you want touse more orbetter?


41/75


Another way to utilize multiple perspectives:Another way to utilize multiple perspectives:Partner with your customers and key stakeholdersPartner with your customers and key stakeholders

Thats what FEMA does when it works with communities to do risk/vulnerabilityThats what FEMA does when it works with communities to do risk/vulnerabilityassessments, and take risk reduction actions.assessments, and take risk reduction actions.

Thats how the MFS office designs flight simulators: in partnership with pilotsThats how the MFS office designs flight simulators: in partnership with pilotsand engineers who will use them.and engineers who will use them.

Thats what some agencies do when theyre getting publicly beaten up byThats what some agencies do when theyre getting publicly beaten up byspecial interest groups. Today, the Army Corps of Engineers has an activespecial interest groups. Today, the Army Corps of Engineers has an activepartnership with the Nature Conservancy; which is charting river flows, restoringpartnership with the Nature Conservancy; which is charting river flows, restoringwildlife habitats, and conducting experiments around 13 of the nations damswildlife habitats, and conducting experiments around 13 of the nations dams

managed by the Corps, in an active partnership between the two.managed by the Corps, in an active partnership between the two.


42/75


To utilize multiple perspectives, partner with yourTo utilize multiple perspectives, partner with your

customers and key stakeholderscustomers and key stakeholders

Three ways to partner with customers and stakeholders onThree ways to partner with customers and stakeholders onyour products, programs and services:your products, programs and services:

1.1. PlanningPlanning---- getting their active involvement in determininggetting their active involvement in determiningwhat and how the products and services are delivered,what and how the products and services are delivered,

2.2. ImplementationImplementation ---- helping them do some of the work you usedhelping them do some of the work you usedto do for them, for themselves (e.g., self service), andto do for them, for themselves (e.g., self service), and

3.3. EvaluationEvaluation ---- working with you to assess the effectiveness andworking with you to assess the effectiveness andquality of your work, helping you find opportunities for changequality of your work, helping you find opportunities for change


43/75


Partnering Workshops: A method for partneringPartnering Workshops: A method for partnering

with your customers and key stakeholderswith your customers and key stakeholders

These are 1These are 1--3 day sessions that include leaders of the3 day sessions that include leaders of theorganizations trying to form a partnership.An outsideorganizations trying to form a partnership.An outsideconsultant helps the participants exchange information onconsultant helps the participants exchange information on

their expectations for the project: goals, time table, roles, howtheir expectations for the project: goals, time table, roles, howto deal with conflicts, etc.to deal with conflicts, etc.

The workshop helps create common objectives for the project.The workshop helps create common objectives for the project.Equally important, it fosters open and trustingEqually important, it fosters open and trusting relationshipsrelationships,,

and an ability to find winand an ability to find win--win solutions.win solutions.

See Appendix C for more on Partnering Workshops.See Appendix C for more on Partnering Workshops.


44/75


Practice 4Practice 4: Defer to expertise: Defer to expertise

In high reliability organizations, there are often three distinctIn high reliability organizations, there are often three distinctcultures.cultures.

1. During normal operations, the culture is hierarchical.1. During normal operations, the culture is hierarchical.

2. When the operational tempo moves into high gear, it2. When the operational tempo moves into high gear, itbecomes very flat: ad hoc teams form, those closest to thebecomes very flat: ad hoc teams form, those closest to theissue are expected to diagnose and take action on it, and ifissue are expected to diagnose and take action on it, and ifmistakes are made the emphasis is on learning, not blame.mistakes are made the emphasis is on learning, not blame.

3. During crises, a third culture appears: if the problem/crisis3. During crises, a third culture appears: if the problem/crisishas occurred before, the organization has prepared for ithas occurred before, the organization has prepared for itand goes into its crisis action mode (with defined roles,and goes into its crisis action mode (with defined roles,clear SOPs, etc.).clear SOPs, etc.).


45/75


Practice 4Practice 4: One example of deference to expertise:: One example of deference to expertise:

Aircraft CarriersAircraft Carriers

During daily operations, the roles are clear and communications areDuring daily operations, the roles are clear and communications arelargely hierarchical.largely hierarchical.

When planes start taking off and landing, theres no time forWhen planes start taking off and landing, theres no time forhierarchy. A rich level ofhierarchy. A rich level oflateralcommunicationslateralcommunications occurs (radio,occurs (radio,phone, hand signals), and this info helps them spot mistakes beforephone, hand signals), and this info helps them spot mistakes beforethey cause damage.they cause damage.

When crises occur, the crew followsWhen crises occur, the crew follows

carefullycarefully--rehearsed procedures; eachrehearsed procedures; each

member has a premember has a pre--assigned role.assigned role.


46/75


The Marine Corps has a long tradition of distributing battlefieldauthority to its lowest level management for a simple reason: high-risk, high-speed assaults tend to be unforgiving on bureaucratic ...management styles. Freedman, CorpsBusiness, p. 35.

Practice 4Practice 4: The importance of deferring to expertise: The importance of deferring to expertise


47/75


Defer to expertise: How to deal with the question ofDefer to expertise: How to deal with the question ofaccountability?accountability?

A key issue: how can managers defer to expertise duringA key issue: how can managers defer to expertise duringproblems and crises, given that they are still responsible andproblems and crises, given that they are still responsible andaccountable for the outcomes? There are several answers:accountable for the outcomes? There are several answers:

1. The Marines handle this by giving local commanders and1. The Marines handle this by giving local commanders andtheir troops this guidance related to their missions:their troops this guidance related to their missions:

The end state:The end state: how they want the situation to end uphow they want the situation to end up

The commanders intent:The commanders intent: the broader goals that the units actions willthe broader goals that the units actions will

supportsupportThe main effort:The main effort: if the mission ispart ofseveral related missions, theif the mission ispart ofseveral related missions, themain effort is the most critical onemain effort is the most critical one

The rules of engagement:The rules of engagement: the boundaries within which they can operatethe boundaries within which they can operate


48/75


Defer to expertise: How to deal with the question ofDefer to expertise: How to deal with the question of

accountability? (cont.)accountability? (cont.)

2. Another approach: continually train and practice.2. Another approach: continually train and practice.

In general, the higher the stakes, the greater emphasis anIn general, the higher the stakes, the greater emphasis an

organization places on continual training and practiceorganization places on continual training and practice ----TheTheMarines, who operate in very highMarines, who operate in very high--stakes environment, excelstakes environment, excelin the area of continual training/practice. See Section 6 forin the area of continual training/practice. See Section 6 formore.more.

3. Share your thinking about decisions with staff. Theyll3. Share your thinking about decisions with staff. Theyllunderstand the organization as you do, and know how youunderstand the organization as you do, and know how youwould respond in most situations.would respond in most situations.


49/75


Defer to expertise: How to deal with the question ofDefer to expertise: How to deal with the question of

accountability? (cont.)accountability? (cont.)

4. Practice shared leadership: give everyone an opportunity to4. Practice shared leadership: give everyone an opportunity tolead the group in certain situations. A good example: thelead the group in certain situations. A good example: theOrpheus Chamber OrchestraOrpheus Chamber Orchestra -- it plays with no conductor, andit plays with no conductor, and

the members take turns providing leadership.the members take turns providing leadership.

Shared leadership reduces the gap between leader and associates,Shared leadership reduces the gap between leader and associates,because everyone has some of the responsibility for leading on anbecause everyone has some of the responsibility for leading on anongoing basis. When chaos or crises hit, the team is prepared.ongoing basis. When chaos or crises hit, the team is prepared.


50/75


51/75


52/75


Practice 5Practice 5: Maintain fluid communications across: Maintain fluid communications acrosspermeable boundariespermeable boundaries

For groups to learn together and deal withFor groups to learn together and deal withchaos and complexity, fluid communicationschaos and complexity, fluid communicationsare imperative:are imperative:

theres not usually enough time to rely on thetheres not usually enough time to rely on thehierarchyhierarchy

complex and chaotic situations require the bestcomplex and chaotic situations require the bestthinking of the group, not the power of one seniorthinking of the group, not the power of one seniorleaderleader

organizations that maintain fluid communicationsorganizations that maintain fluid communicationslook like good basketball teams: most of theirlook like good basketball teams: most of theirimportant communications occur during play, asimportant communications occur during play, asthey constantly adjust to the changingthey constantly adjust to the changingenvironmentenvironment


53/75


Practice 5Practice 5: When the stakes are high, fluid: When the stakes are high, fluidcommunications across permeable boundaries arecommunications across permeable boundaries are

especially importantespecially important

The night before the Challenger disaster of Jan. 28, 1986, a teleconference took place among 34 peopleThe night before the Challenger disaster of Jan. 28, 1986, a teleconference took place among 34 peopleat three locations. The purpose: decide whether to launch the next morning. This was unusual; in theat three locations. The purpose: decide whether to launch the next morning. This was unusual; in thepast, launch decisions were discussed at facepast, launch decisions were discussed at face--face meetings of NASA managers and its contractors.face meetings of NASA managers and its contractors.

Many engineers at Morton Thiokol recommended against launching. NASA managers werent happy toMany engineers at Morton Thiokol recommended against launching. NASA managers werent happy tohear that, and werent persuaded by by the data and analysis Thiokol put forward. Some angrilyhear that, and werent persuaded by by the data and analysis Thiokol put forward. Some angrilychallenged Thiokols conclusions. One manager, Larry Mulloy said hotly, When do you want me tochallenged Thiokols conclusions. One manager, Larry Mulloy said hotly, When do you want me tolaunch, Thiokol, next April?launch, Thiokol, next April?

Midway through the teleconference, senior people at Thiokol held an offMidway through the teleconference, senior people at Thiokol held an off--line discussion. They cameline discussion. They cameback to the call, said they had reback to the call, said they had re--examined their data, and reversed the engineers noexamined their data, and reversed the engineers no--launchlaunchrecommendation. A NASA manager then asked, does anybody have anything more to say? Nobodyrecommendation. A NASA manager then asked, does anybody have anything more to say? Nobodyspoke up.spoke up. The Challenger was launched the next morning, and the crew perished.The Challenger was launched the next morning, and the crew perished.

With only voice cues, NASA did not have visual data such as facial expressions that might have givenWith only voice cues, NASA did not have visual data such as facial expressions that might have giventhem fuller information aboutjust how worried Thiokol engineers were Weick and Sutcliffe,them fuller information aboutjust how worried Thiokol engineers were Weick and Sutcliffe,Managing the UnexpectedManaging the Unexpected, p. 168., p. 168.


54/75


Practice 5Practice 5: Maintain fluid communications across: Maintain fluid communications acrosspermeable boundariespermeable boundaries ---- Some methods:Some methods:

Use the rumor millUse the rumor mill ---- check in with associates at regular staffcheck in with associates at regular staffmeetings and ask what rumors are going around. This givesmeetings and ask what rumors are going around. This givesmanagers a quick check on the pulse of the organization, allowsmanagers a quick check on the pulse of the organization, allowsthem to correct misunderstandings, and offers a lowthem to correct misunderstandings, and offers a low--risk way forrisk way foremployees to check out concerns.employees to check out concerns.

A good way to open up the discussion is to ask, What rumors areA good way to open up the discussion is to ask, What rumors areyou hearing lately?you hearing lately?


55/75


Practice 5Practice 5: Maintain fluid communications across: Maintain fluid communications acrosspermeable boundariespermeable boundaries ---- Some methods:Some methods:

Hold issueHold issue--oriented staff meetings; one person presents a currentoriented staff meetings; one person presents a currentor approaching challenge, and peers ask questions, offer insights.or approaching challenge, and peers ask questions, offer insights.The objective is peerThe objective is peer--peer learning.peer learning.

Employee a classic (but still underused) management method:Employee a classic (but still underused) management method:Managing by wandering around (MBWA). Take one hour/week,Managing by wandering around (MBWA). Take one hour/week,drop in on 3drop in on 3--4 units, ask about their current projects, whats going4 units, ask about their current projects, whats goingwell, where they might want outside assistance. The first fewwell, where they might want outside assistance. The first fewtimes people may be skeptical and defensive; once they knowtimes people may be skeptical and defensive; once they know

youre there to learn, they start to open up and everyone gains.youre there to learn, they start to open up and everyone gains.


56/75


Method: AfterAction ReviewsMethod: AfterAction Reviews

Purpose:Purpose:

To help employees think systematically about their important initiatives; to ensure that everyone can learn fromTo help employees think systematically about their important initiatives; to ensure that everyone can learn fromothers learningothers learning

U.S Army agencies try to capture and export knowledge after important activities and projects. The methodU.S Army agencies try to capture and export knowledge after important activities and projects. The methodused for thispurpose is called an After Action Review (check the Armys lessons learned home page, atused for thispurpose is called an After Action Review (check the Armys lessons learned home page, athttp://call.army.mil/call.htm, for more).http://call.army.mil/call.htm, for more).

Method:Method:

During or after an activityDuring or after an activity -- a conference, a project team developing a new system, a military exercise or actuala conference, a project team developing a new system, a military exercise or actualinterventionintervention -- those responsible for it write an After Action Review. The report should be kept simple. Thesethose responsible for it write an After Action Review. The report should be kept simple. Thesecategories are sometimesused:categories are sometimesused:

Key Steps in the AfterAction Review ProcessKey Steps in the AfterAction Review Process

1. What was the objective of the activity?1. What was the objective of the activity?

2. What actually happened? (describe the events as they occurred)2. What actually happened? (describe the events as they occurred)

3. If the objective wasnt achieved, why not?3. If the objective wasnt achieved, why not?

4. What have we learned? (What are the implications, and lessons learned?)4. What have we learned? (What are the implications, and lessons learned?)


57/75


TheArmy has perfected a remarkably efficient process (AfterAction

Reviews) for correcting its mistakes and sustaining its successes.

- David Garvin, Harvard Business Schooland author of a case study on the AAR

5. Take actions. (Learning doesnt take place without action. What will be different now?)5. Take actions. (Learning doesnt take place without action. What will be different now?)

6. Share this with others who could benefit.6. Share this with others who could benefit.

AfterAction Reviews can also be put into a database, which allows others quick access toAfterAction Reviews can also be put into a database, which allows others quick access tothe knowledge and lessons learned.the knowledge and lessons learned.

Benefits from AfterAction ReviewsBenefits from AfterAction Reviews

1. Focus is on learning, not blame.1. Focus is on learning, not blame.

2. The structure issimple.2. The structure issimple.

3. It breaks the hierarchical barriers, gets everyone on the same page, with no fear.3. It breaks the hierarchical barriers, gets everyone on the same page, with no fear.

4. It encourages reflection in the process of action, so that both learning and improved performance take place.4. It encourages reflection in the process of action, so that both learning and improved performance take place.

5. It helpspeople push back against the tyranny of the immediate.5. It helpspeople push back against the tyranny of the immediate.

6. It provides a record that others can analyze, add to, and use.6. It provides a record that others can analyze, add to, and use.


58/75


Who has a worldWho has a world--classsystem for training employees to beclasssystem for training employees to becustomer focused?customer focused?

What organizations have superior 1What organizations have superior 1--800 service?800 service?

Whos done an exceptional job at integrating their informationWhos done an exceptional job at integrating their informationsystems?systems?

And And howhow do they do it? And whom do we call for moredo they do it? And whom do we call for moreinformation?information?

Best practices databases answersuch questions.Best practices databases answersuch questions.

Method: Best Practices DatabasesMethod: Best Practices Databases


59/75


Best Practices DatabasesBest Practices Databases -- Natural ResourcesNatural ResourcesExampleExample

Forest Service and Bureau of Land Management staff collaborateclosely in many parts of the country, sharing staff, resources,

training, space, even managers.

They have documented theirsuccesses in their Effective PracticesDirectory, listing specific practices, how they were achieved,obstacles encountered and how they were overcome, and contacts

for more information.


60/75


Best Practices DatabasesBest Practices Databases -- Ford Motor ExampleFord Motor Example

Ford Motor requires each manufacturing manager to improveproductivity 5% annually. To help them do that, Ford created an

Internet-based database called Best PracticesReplication System.

When plants make productivity improvements, they document howthey did it on the database (in text, pictures, video, etc.), andinclude information on vendors and toolsused in the process

improvement. Every manager is expected to use any informationthats applicable.


61/75


Practice 5Practice 5: Maintain fluid communications across: Maintain fluid communications acrosspermeable boundariespermeable boundaries

Exercise:Exercise:

What can you and your associates doWhat can you and your associates do to create more permeablecreate more permeable

boundaries, and increase information flow in your organization:boundaries, and increase information flow in your organization:

A. During normal periods of operations?A. During normal periods of operations?

B: When dealing with chaotic situations, and with the unexpected?B: When dealing with chaotic situations, and with the unexpected?


62/75


Practice 6Practice 6: Support continual training, learning and: Support continual training, learning andpracticepractice

Were filled with knowledge about management:Were filled with knowledge about management:

1,700 business books published/year1,700 business books published/year

over $60 billion spent/year on corporate trainingover $60 billion spent/year on corporate training

80,000 MBAs graduate each year80,000 MBAs graduate each year

If we know so much, why are there so many poorlyIf we know so much, why are there so many poorly--runrunorganizations?organizations?

One reason: too much training isOne reason: too much training is separatedseparated from doing.from doing.Studies show that up to 70% of workplace learning is informalStudies show that up to 70% of workplace learning is informal(observing others, reflecting on our own experiences), but most(observing others, reflecting on our own experiences), but mosttraining ignores that; it doesnt integrate knowing with doing.*training ignores that; it doesnt integrate knowing with doing.*

* Source:* Source: The KnowingThe Knowing--Doing GapDoing Gap, by Pfeffer and Sutton., by Pfeffer and Sutton.

Practice 6Practice 6: How to support continual training: How to support continual training


63/75


Practice 6Practice 6: How to support continual training,: How to support continual training,learning and practice:learning and practice: Integrate learning with theIntegrate learning with the

work.work.

Most workplace learning takes place in the context of action. ItsMost workplace learning takes place in the context of action. Itsnot that experience is necessarily the best teacher;not that experience is necessarily the best teacher; reflectingreflectingononexperience is the best teacher.experience is the best teacher.

Most of the highly effective training and learning activities have aMost of the highly effective training and learning activities have aheavy dose of action, in a setting that helps people learn fromheavy dose of action, in a setting that helps people learn fromtheir experiences, such astheir experiences, such as

being given stretch objectivesbeing given stretch objectives

having rotational assignmentshaving rotational assignments

serving on crossserving on cross--functional teamsfunctional teams

One Lesson:One Lesson: Integrate learning with work.Integrate learning with work.

If you do it, then you will know. - David Sun, Kingston Technology

Practice 6Practice 6: How to support continual training: How to support continual training


64/75


Practice 6Practice 6: How to support continual training,: How to support continual training,learning and practice: Have people teach what theylearning and practice: Have people teach what they

know.know.

A second lesson:A second lesson: have people teach what they know.have people teach what they know.

One of the most powerful ways to learn is to put managers andOne of the most powerful ways to learn is to put managers and

others intoothers into teachingteachingroles. Most professionals learn bestroles. Most professionals learn bestwhen they must teach others.when they must teach others.

Teaching can be done in many ways:Teaching can be done in many ways:

formal classroom teachingformal classroom teaching

informal teaching through mentorprogramsinformal teaching through mentorprograms

writing up and publishing lessons learned paperswriting up and publishing lessons learned papers

leading orientation sessions for new employeesleading orientation sessions for new employees

Practice 6Practice 6: How to support continual training,: How to support continual training,


65/75


Practice 6Practice 6: How to support continual training,: How to support continual training,learning and practice: Provide knowledgelearning and practice: Provide knowledge

templatestemplates

A third lesson:A third lesson: helppeople develop knowledge templates.helppeople develop knowledge templates.

There are no SOPs for many of the situations encountered inThere are no SOPs for many of the situations encountered incomplex environments. Thus,complex environments. Thus, the best training includes exposingthe best training includes exposingpeople to a large numberofpeople to a large numberofscenariosscenarios, so that the brain learns to, so that the brain learns torecognize patterns and apply them as analogies in new situations.recognize patterns and apply them as analogies in new situations.

The Marines excel at this. Their training schools continually putThe Marines excel at this. Their training schools continually putMarines into situations requiring leadership and decision making. AsMarines into situations requiring leadership and decision making. As

they debrief their actions, they developthey debrief their actions, they develop knowledge templatesknowledge templatesthatthatcan be adapted and applied to future challenges.can be adapted and applied to future challenges.

The way you get a shared vision is through a shared experience.- Col. Robert E. Lee (no relation), director of one of theMarine Corps officer training programs


66/75


Practice 6Practice 6: How to support continual training,: How to support continual training,learning and practice: Discuss near misseslearning and practice: Discuss near misses

A fourth lesson:A fourth lesson: regularlydiscuss nearmisses.regularlydiscuss nearmisses.

Organizations that deal with complexity well often discuss nearOrganizations that deal with complexity well often discuss near

missesmisses ---- opportunities forjunior and senior people to talk openlyopportunities forjunior and senior people to talk openlyabout activities that almost resulted in failure or catastrophes. Theseabout activities that almost resulted in failure or catastrophes. Thesesituations are ripe for learning, but are usually ignored. It often helpssituations are ripe for learning, but are usually ignored. It often helpsto hold the discussions without managers in the room, to ensureto hold the discussions without managers in the room, to ensurecandor. Then have someone write up and disseminate the key points.candor. Then have someone write up and disseminate the key points.

Humans are notoriously bad monitors of systems that rarely fail Near misses should sound organizational alarms.

Scott A. Snook, author ofFriendly Fire


67/75


68/75


Practice 6Practice 6: Support continual training, learning and: Support continual training, learning andpracticepractice

1. What have been the most powerful learning moments in your career?1. What have been the most powerful learning moments in your career?

2. Which of the lessons just listed would you like to use (more) in your2. Which of the lessons just listed would you like to use (more) in yourorganization:organization:

- Integrate learning with action

- Helppeople learn by having them teach- Give people experience in different scenarios, so they develop knowledge templates- Organize candid, safe discussions of near misses- Use stories to convey the key principles and messages you want associates to learn

Exercise:Exercise:


69/75


Assessing Potential ChangeAssessing Potential Change

LOW HIGH

EA

SY

HA

RD

**FEA

SIBILI

TY

* IMPACT

*Impact:: On external customers, on staff, on performance, on costs, etc.

**Feasibility: The ease and/orspeed of implementation


70/75


Appendix A. FiveAppendix A. Five Leadership Skills That FosterLeadership Skills That FosterLearningLearning

1. Developing intellectual curiosity1. Developing intellectual curiosity -- What do you find intriguing? What can you learnWhat do you find intriguing? What can you learnfrom others views?from others views?

2. Asking open questions. Can you ask others truly open (not leading) questions,2. Asking open questions. Can you ask others truly open (not leading) questions,

without trying to make a point?without trying to make a point?

3. Maintaining non3. Maintaining non--defensive reactions. When someone disagrees or criticizes, dodefensive reactions. When someone disagrees or criticizes, doyouseek more data and reflect on it, rather than defend yourposition oryouseek more data and reflect on it, rather than defend yourposition oryourself, or attack?yourself, or attack?

4. Examining assumptions. Do you make your assumptions explicit, examine them4. Examining assumptions. Do you make your assumptions explicit, examine themto see if theyre valid, and invite others to do the same?to see if theyre valid, and invite others to do the same?

5. Slow down the game. High performing athletes do this. It helps them see the5. Slow down the game. High performing athletes do this. It helps them see theball, the playing field, the competition. When youre feeling pressed with tooball, the playing field, the competition. When youre feeling pressed with toomany demands, do you know how to focus on the vital few that reallymany demands, do you know how to focus on the vital few that really

matter? Can youslow down your game?matter? Can youslow down your game?


71/75


Appendix B:Appendix B: Nine Requirements for Starting, andNine Requirements for Starting, and

Maintaining Major Organizational ChangeMaintaining Major Organizational Change

1. There is an important business1. There is an important business--related problem or need driving the changerelated problem or need driving the change2. Organizational leaders are communicating the need for change in a simple,2. Organizational leaders are communicating the need for change in a simple,

clear wayclear way

3. There is an emphasis on continual, two3. There is an emphasis on continual, two--way communications throughout theway communications throughout thechange.change.

4. A senior manager with creditability and clout is designated the executive sponsor4. A senior manager with creditability and clout is designated the executive sponsor

for the change, and has a strong team whose time is dedicated to the changefor the change, and has a strong team whose time is dedicated to the changeeffort.effort.

5. Key internal and external stakeholders are involved.5. Key internal and external stakeholders are involved.

6. There is a game plan that addresses the need for change, the goals, guiding6. There is a game plan that addresses the need for change, the goals, guidingprinciples, key elements in the change, timetable and responsibilities,principles, key elements in the change, timetable and responsibilities,communicationsstrategy, methods forshaping realistic expectations,communicationsstrategy, methods forshaping realistic expectations,involvement of key stakeholders.involvement of key stakeholders.

7. The change team generatessome early wins.7. The change team generatessome early wins.

8. There are many informal organizational leaders who are enthusiastic about the8. There are many informal organizational leaders who are enthusiastic about thechange.change.

9. The change is consolidated in the culture to create a sense of inevitability.9. The change is consolidated in the culture to create a sense of inevitability.


72/75


Appendix C: Partnering WorkshopsAppendix C: Partnering Workshops

Partnering workshops have been used by a variety of government and privatePartnering workshops have been used by a variety of government and private--sector organizationssincesector organizationssince

the 1980s. This method helps organizations that are frequently in conflict, to develop common goals andthe 1980s. This method helps organizations that are frequently in conflict, to develop common goals andwinwin--win relationships. Its most frequent use is in the construction industry.win relationships. Its most frequent use is in the construction industry.

The Partnering ProcessThe Partnering Process

Prepare for itPrepare for it educate your organizationeducate your organization Educate the other organization(s)Educate the other organization(s) Gain commitment from senior managementGain commitment from senior management Identify partnering champions (or leaders)Identify partnering champions (or leaders) Prepare for the partnering workshopPrepare for the partnering workshop Conduct the partnering workshop.Conduct the partnering workshop. This may last from one to three days. The deliverables from theThis may last from one to three days. The deliverables from the

workshop include:workshop include: Partnering charter.Partnering charter. This is a formal agreement among the partners, in which they spell out theirThis is a formal agreement among the partners, in which they spell out their

mutual goals, objectives, time lines, roles, etc. Some common partnering goals for constructionmutual goals, objectives, time lines, roles, etc. Some common partnering goals for constructionprojects, for instance, are:projects, for instance, are:

Complete the project so that it meets the design intentComplete the project so that it meets the design intent Complete the project without litigationComplete the project without litigation Achieve savings of $_________Achieve savings of $_________ Finish the project ___ days ahead ofscheduleFinish the project ___ days ahead ofschedule Suffer no lost time by promoting a safe job siteSuffer no lost time by promoting a safe job site Ensure fair treatment for all partiesEnsure fair treatment for all parties Solve problems at the lowest possible management levelSolve problems at the lowest possible management level


73/75


Appendix C: Partnering WorkshopsAppendix C: Partnering Workshops

Day 1:

a.m.

Introductions

Overview to partnering

Overview to the Project

Experiential exercise: Win as much as you can.

p.m.

Debrief exercise

Discussion of win-win possibilities for the project

Identification of each organization's previousexperience in partnerships

Discussion of project goals

Day 2:

a.m.

Learning about individual differences: Myers-BriggsType Indicator

Profile of each teams Myers-Briggs styles

Conflict management exercise

p.m.

Develop project charter, including conflict-management norms and project goals, timeline,roles, communications channels, evaluationmethods, etc.

Each participant signs the charter

Review of the session, prepare for first steps

Adjourn

Sample Partnering WorkshopAgendaSample Partnering WorkshopAgenda

Conflict resolution process to be used.Conflict resolution process to be used. Evaluation processEvaluation process -- when project will be evaluated, criteria to be used.when project will be evaluated, criteria to be used.

Periodic evaluationPeriodic evaluation

Celebrate successesCelebrate successes


74/75


75/75

10.16.03 learning organization complexity

Documents