ISSN 1744-1986

T e c h n i c a l R e p o r t N O 2007/ 17

An assessment of un-structured knowledgemanagement techniques in the projectmanagement of software development

E. Aitken

28 June, 2007

Department of ComputingFaculty of Mathematics, Computing and TechnologyThe Open University

Walton Hall, Milton Keynes, MK7 6AAUnited Kingdom

http://computing.open.ac.uk

An assessment of un-structured knowledge management techniques in the project management of

software development

A dissertation submitted in partial fulfilment of the requirements for the Open University‘s

Master of Science Degree in Computing for Commerce and Industry

Elaine Aitken (T9915646)

5 March 2008

Word Count: 16100

Preface

I wish to thank the employers who have supported me throughout the courses that

have led up to this dissertation: ts.com, OUP and RM.

The support of my supervisor, Dr David Butts, was essential in helping me sort out

what I was trying to achieve. His responses were always timely and helpful.

The contributions from the participants who responded to my questionnaires were

invaluable. I am grateful to them all, and particularly to those who hung in there, right

to the end.

Finally, I would like to thank Helen Charlesworth for being such a fabulous sounding

board and for her patience and support.

ii

Table of Contents

Preface ............................................................................................................. i

List of Figures ................................................................................................. v

List of Tables .................................................................................................. vi

Chapter 1 Introduction ....................................................................................... 8

1.1 Background to the research ..................................................................... 9

1.1.1 Technology ...................................................................................... 10

1.1.2 Human factors ................................................................................. 11

1.2 Knowledge management in software development ............................... 11

1.3 Knowledge management and project management ............................... 12

1.4 Aims and objectives of the research ...................................................... 13

1.5 Overview of the dissertation .................................................................. 13

Chapter 2 Literature Review ........................................................................... 15

2.1 The historical perspective ...................................................................... 15

2.2 Data, information, knowledge – the degrees of knowing ....................... 16

2.3 Organisational knowledge ..................................................................... 17

2.4 Knowledge capture ................................................................................ 20

2.5 Knowledge transfer ................................................................................ 21

2.6 Making the most of tacit knowledge ....................................................... 23

2.7 Finding the gaps in current processes ................................................... 26

2.8 Research question ................................................................................. 27

2.9 Summary ............................................................................................... 28

Chapter 3 Research Methods ......................................................................... 29

3.1 Other techniques considered ................................................................. 29

3.1.1 Questionnaire .................................................................................. 29

3.1.2 Case study ...................................................................................... 30

3.1.3 Interviews ........................................................................................ 30

iii

3.2 The Delphi technique ............................................................................. 30

3.2.1 Participants ...................................................................................... 31

3.2.2 Stage 1: Gathering initial contributions ............................................ 32

3.2.3 Stage 2: Collation ............................................................................ 34

3.2.4 Stage 3: Refinement 1 ..................................................................... 34

3.2.5 Stage 4: Refinement 2 ..................................................................... 35

3.2.6 Stage 5: final data collection ............................................................ 35

3.3 Analysis of existing solutions ................................................................. 36

Chapter 4 Data Collection ............................................................................... 37

4.1 Data Sources ......................................................................................... 37

4.2 Process .................................................................................................. 37

4.3 Preliminary Analysis .............................................................................. 37

4.4 Analysing the ranking ............................................................................ 38

Chapter 5 Results ........................................................................................... 41

5.1 Background information ......................................................................... 41

5.2 Delphi method data ................................................................................ 42

5.2.1 What are the benefits of knowledge management? ........................ 42

5.2.2 What techniques are currently used? .............................................. 44

5.2.3 What other techniques are you aware of but have not personally used?............................................................................................... 47

5.2.4 What types of knowledge have been hardest to capture and share effectively? ...................................................................................... 47

5.2.5 Wildcards ......................................................................................... 50

5.3 Results of Delphi process ...................................................................... 50

5.3.1 The connection between ambition and practice .............................. 52

5.3.2 Gaps in current practice .................................................................. 53

5.4 Comparison with the literature ............................................................... 54

5.5 Analysis of practice vs literature ............................................................ 58

5.5.1 Patterns ........................................................................................... 59

iv

5.5.2 Document management and collaboration software ....................... 62

5.5.3 Storytelling ....................................................................................... 64

5.5.4 Semi-structured interviews .............................................................. 66

5.5.5 Rich personal interaction ................................................................. 68

5.5.6 Estimates for future work ................................................................. 70

5.5.7 Summary of proposed techniques ................................................... 71

5.6 Validation ............................................................................................... 72

5.6.1 Analysis ........................................................................................... 73

Chapter 6 Conclusions .................................................................................... 75

6.1 Project review ........................................................................................ 76

6.2 Future research ..................................................................................... 76

References 78

Bibliography ..................................................................................................... 82

Index ............................................................................................................ 83

Appendix A: Delphi method initial questionnaire results ................................... 84

Appendix B: First refinement ............................................................................ 93

Appendix C. Second Refinement ................................................................... 102

v

List of Figures

Figure 1. Process of knowledge conversion. Nonaka & Takeuchi (1995). ....... 15

Figure 2. Questionnaire sent to participants in the Delphi method. .................. 33

Figure 3. Connections between intended benefits and practice. ...................... 53

Figure 4. Connections between problem areas and practice. .......................... 53

Figure 5. Current knowledge management techniques categorised in terms of the scope of their relevance across projects and the level of detail they capture. ........................................................................................... 55

Figure 6. The gaps in current practice categorised in terms of the scope of their relevance across projects and the level of detail they require. ........ 57

vi

List of Tables

Table 2.1. From Crossan et al. (1999). Processes involved in knowledge transfer at different levels of an organisation. .................................. 19

Table 2.2. Knowledge transfer frictions (from Davenport & Prusak, 1998). ...... 24

Table 5.1. Level of maturity of organisational knowledge management approach. ....................................................................................... 41

Table 5.2. Technical sophistication of organisational knowledge management approach. ........................................................................................ 41

Table 5.3. Personnel structure with respect to knowledge management. ........ 42

Table 5.4. Responses for list 1 ranked by total marks received, with comments added by participants. Benefits of knowledge management............ 43

Table 5.5. Responses for list 2 ranked by total marks received, with comments added by participants. Commonly used knowledge management methods. .......................................................................................... 45

Table 5.6. Collated responses to Question 6 of questionnaire. ........................ 47

Table 5.7. Responses for list 3 ranked by total marks received, with comments added by participants. Hard to share knowledge. ............................ 48

Table 5.8. Literature sources of research into knowledge capture. .................. 58

Table 5.9. Matrix showing which gaps in knowledge capture identified by the Delphi method are addressed by which techniques from the literature. ......................................................................................... 59

Table 5.10. Matrix showing which of the current techniques are extended or complemented by the additional techniques found in the literature. 59

Table B.1. Reponses to the first refinement of List 1. ....................................... 99

Table B.2. Reponses to the first refinement of List 2. ..................................... 100

Table B.3. Reponses to the first refinement of List 3. ..................................... 101

vii

Abstract

Those involved in managing projects are acutely aware of the benefits that could be

achieved if knowledge could be transferred between projects and teams. There are

few organisations, however, that support structured organisation-level management

of knowledge. This study looks at the knowledge management methods employed

by individuals in organisations that do not have well-defined or well-embedded

knowledge management processes. The study finds that the techniques used most

often are those that are recommended by Project Management methodologies, such

as post-project reviews and lessons learned reports. It was also found that there is

an interest in using some of the recently available collaborative, de-centralised

technologies such as blogs that can be implemented on a small scale without

necessarily having to be run at a corporate level.

Through responses to a questionnaire and the application of the Delphi method, the

participants in this research agreed on a number of areas where their own current

practice does not meet their needs for knowledge transfer. The techniques currently

being employed were limited in their ability to share knowledge effectively and

promote re-use. An analysis of existing research shows that the following techniques

may improve current practice:

The use of patterns, analogous to those used in software development

Blogging and other forms of collaboration software

Using metrics to get quantifiable information

A more narrative approach to knowledge capture, using storytelling to convey

key points

The use of oral history capture by semi-structured interview

All of these are found to provide the possibility of improved knowledge transfer

across project teams.

8

Chapter 1 Introduction

The field of knowledge management is broad. It covers the creation, capture,

transfer and use of knowledge within an organisation and can be approached from

both a technology-centric and people-centric point of view. Regardless of which

approach an organisation favours, the value of good knowledge management is

summed up by Lew Platt, former CEO of Hewlett-Packard (quoted in Davenport &

Prusak, 1998):

―If HP knew what HP knows, we would be three times as profitable.‖

Nonaka (1991) cites a number of companies who have embraced formalised

knowledge management, including Canon, Honda, NEC and Sharp. As evidence

that knowledge management is not restricted to successful Japanese companies,

Brand (1998) states that ―3M‘s objective is to become the most innovative company

in the world‖. A key to this strategy is to create an environment that encourages a

willingness to share knowledge between individuals.

Despite an acceptance from organisations that we are now in an era when

knowledge is a valuable resource, knowledge management is often not formally

implemented at an organisation level. In many organisations, individual project or

team managers are often responsible for making sure that the knowledge available

in an organisation is used most effectively on their projects. One of the fields that

makes extensive use of project managers in this way is that of software

development. Companies who have traditionally not been in the software business,

such as publishers, find that they are now a part of the software market and have to

develop the skills required to compete. Innovation is critical to success in this market

and knowledge management has been shown to contribute towards successful

9

innovation (Faniel & Majchrzak, 2006) and maintainability of software (Anquetila et

al., 2006). Given this, it is valuable to find what techniques are being used by project

and team managers in organisations such as these – where software development is

key to their success but the organisation as a whole does not implement formal

knowledge management processes.

1.1 Background to the research

In order to understand the ways in which knowledge can be managed and

transferred, it is beneficial to understand more about the nature of knowledge itself.

In the work of Nonaka (1991) and Nonaka & Takeuchi (1995), the foundation of the

field of knowledge management was laid. They highlighted the difference between

tacit and explicit knowledge, and the benefits that come from moving knowledge

between those two states. Tacit knowledge is held in people‘s heads, and is often

taken for granted by the holder. Explicit knowledge is what we know we know, and

can be easily described, shared or learned. A particular focus of the work of Nonaka

& Takeuchi is the innovation that can arise when efforts are made to share tacit

knowledge and explicit knowledge is created. Case studies in their work illustrate the

techniques used in Japanese companies to share understanding and mental models

through the use of metaphorical and visual language. The point is made that this has

not historically been a mode of communication that has been understood or valued

in the West.

Those interested in developing the practice of knowledge management have

approached the field from two sides:

1. The study of how technological systems can help with capturing, codifying

and delivering knowledge.

10

2. The study of the human factors that affect the creation and absorption of

knowledge.

1.1.1 Technology

A wide range of technological solutions have been proposed and implemented that

have attempted to improve the knowledge management process.

Some examples are:

1. Video-nets, that allow real time and asynchronous access to video and voice

contact between members of a community, with the aim of building up the

body of knowledge held by veterinarians in towns and villages across Japan.

(Kodama, 2002)

2. Bespoke artificial intelligence systems, with context-specific ontologies written

to provide a structure for the knowledge bases that are the foundation of the

means of knowledge-retrieval (O‘Leary, 1998).

3. Web-based groupware to encourage collaboration and communication and

thus gain competitive advantage (Sipcic & Makonnen, 1998).

The investment in these technological systems has been considerable and their

scale is such that historically they could only be implemented once an organisation

was prepared to make the investment and implement a system across an entire

organisation. They were corporate solutions.

11

1.1.2 Human factors

For all the investment in them, these systems cannot produce results by themselves.

They require knowledge to be gathered, shared, adopted and absorbed. For this to

happen, those who need the information also need to know that it exists, they have

to go and find it, they have to trust it when they find it and they have to be able to

extract some relevant learning points from it. Whether or not users adopt information

once they have found it has been shown to depend on the extent to which they were

able to see it as a starting point, ready for adaptation, rather than a proscriptive

blueprint (Faniel & Majchrzak, 2006).

Where knowledge is explicit and easily codified it might be quite straightforward to

capture it, and it might be obvious to users what they need to do to find it again.

Where knowledge is tacit, the person with that knowledge might not even know that

it is significant and the best way to share it might be by a demonstration, or an

apprenticeship or an anecdote. There is therefore a very human aspect to tacit

knowledge sharing.

1.2 Knowledge management in software development

Software development is one of the areas that have sought to make use of

knowledge management techniques. It has been shown to bring improvements in

innovation (Faniel & Majchrzak, 2006), quality (Robillard, 1999; Tiwana, 2004) and

maintainability (Anquetila et al., 2006). These papers show that the gathering and

use of knowledge is important at all stages of software application development.

Faniel & Majchrzak (2006) found that innovation was more likely if developers are

exposed to knowledge that is new to them and different to what they already know.

Robillard (1999) emphasises the need for a good design if the output of the software

12

development process is to be of a high quality. Both innovation and good design are

facilitated if the available body of knowledge is consulted at the start of the

development process.

The field of software development already contains many ways of specifying

knowledge – design documents, requirements documents and code documentation

for example. These are all ways of capturing and sharing what people know but they

all tend to refer to a particular problem or project and are not often generalised. Code

libraries contain snippets of code designed to solve a very specific problem that can

be reused across many projects but this is an example of sharing and reusing the

easily codified knowledge.

1.3 Knowledge management and project management

Knowledge management has a special relationship with project management in

terms of both knowledge creation and knowledge use. There is much research that

deals with the importance of communities of knowledge, networks (Bresnen et al.,

2003) and trust (Ji-Hong Park, 2006) in the ease with which knowledge can be

shared and the value associated with it. This work recognises the human aspects of

knowledge transfer - that people feel that lifting solutions from other people stifles

their creativity, that even if information is available it takes a particular environment

for people to be open to accepting it. Particularly in new product development, where

innovation is especially important, then personal interaction, information sharing,

redundancy and the importance of shared mental models have been found to be

very valuable (Madhavan & Grover, 1998).

There are many outcomes of projects that can be hard to make tangible – the

lessons learned that those involved will take with them into future projects. Project

13

management methodologies advocate the sharing of information through end of

project reports and lessons learned reports, but these might not be sufficient to

capture all types of knowledge needed in projects. This study will look at what those

involved in the field are currently doing to capture lessons learned and will focus in

particular on where the gaps are in the types of knowledge that are captured.

1.4 Aims and objectives of the research

The aims of this study were to establish how project managers use knowledge

management techniques and to suggest how this could be improved. The study used

a panel of project managers who were experienced in software development. The

objectives were:

1. To identify what techniques project managers use to capture knowledge in

software projects.

2. To establish the perceived benefits of knowledge management to the

participants‘ organisations.

3. To identify where project managers feel these techniques are lacking.

4. To identify techniques that could be used at a project-level to improve

knowledge capture and transfer in the areas where it is shown to be lacking.

1.5 Overview of the dissertation

Chapter 2 is a review of the existing literature on knowledge management. It

provides an overview of the field of knowledge management, but focuses on what

existing literature offers to those who have a need to share knowledge.

14

The research approach is described in Chapter 3. This includes the details about the

Delphi method participants, the questions asked of them and how the results were

refined. Chapter 3 also discusses the iterative way in which the findings from the

Delphi method were used to inform the analysis of the literature. Finally it explains

how the results from the Delphi method and those from the literature analysis were

combined to find the techniques that looked most promising in addressing the needs

of the project managers.

Chapter 4 explains the process of data gathering and analysis but the detail of the

results found and the analysis that was carried out is expanded in Chapter 5.

Chapter 5 also contains the details of the findings from the analysis of the literature

and the comparison of these findings with the conclusions from the Delphi method.

Conclusions are summarised in Chapter 6 and the success of the project is

reviewed.

15

Chapter 2 Literature Review

2.1 The historical perspective

The foundation for the specialism of knowledge management is in the work of

Nonaka & Takeuchi (1995). This work considers the difference between explicit and

tacit knowledge, and the ways in which knowledge can move between those two

states and between individuals. This distinction is crucial when considering

knowledge management and knowledge transfer techniques. Figure 1 shows these

processes as described by Nonaka & Takeuchi.

Tacit to Tacit

SOCIALIZATION

Tacit to Explicit

EXTERNALIZATION

Explicit to Tacit

INTERNALIZATION

Explicit to Explicit

COMBINATION

Figure 1. Process of knowledge conversion. Nonaka & Takeuchi (1995).

Although the 1990s saw the development of knowledge management as a discipline,

the study of knowledge dates back to the earliest days of philosophy, and the

approaches to knowledge developed then continue to shape the field today.

Plato drew a distinction between the intelligible (something that can be understood

and defined) and the sensible (something that can be experienced by our interaction

with it). Cartesian dualism separated the mind and matter: introspection and

experience. In time, the historical Western philosophical split of mind and body in the

understanding of knowledge was challenged by philosophers such as Immanuel

Kant. In Critic of Pure Reason, Kant (1787) defines knowledge as being born from

experience. Kant states:

16

―Without consciousness, that which we think is the very same as what we

thought a moment before.‖ p.230.

The Japanese view of knowledge has long adopted this view of experience providing

something more than just the intelligible. It has embraced a ―oneness of humanity

and nature‖ (Nonaka & Takeuchi, 1995): the oneness of body and mind. Nonaka &

Takeuchi argue that this is intrinsic even in the Japanese language, where the lack

of verb conjugation makes it difficult for a speaker to personalise their thoughts and

feelings.

The practical outcome of this difference between historical Western and Japanese

theories of knowledge is that the Japanese are traditionally more comfortable with

tacit knowledge gained and shared through experience whereas the West gives

more respect to codified knowledge learned by knowing and reason.

2.2 Data, information, knowledge – the degrees of knowing

Another perspective on the debate about what constitutes knowledge comes from

the distinction between data, information and knowledge. Clarke & Rollo (2001)

define each term as:

―Data

Data are sets of discrete objective facts, presented without judgement or

context. Data become information when they are categorised, analysed,

summarised and placed in context, becoming intelligible to the recipient.

Data relate to the actual bits and characters in an information system or in the

other physical manifestations of communication such as sound and

temperature.

Information

Information is data endowed with relevance and purpose. Information

develops into knowledge when it is used to make comparisons, assess

consequences, establish connections and engage in dialogue.

17

Information is data in context that can be used for decision making. Data are

usually arranged to provide meaning to the observer. Typically it is arranged

as text but could be an image, a film clip, a conversation with another person

or even a busy signal on a phone line.

Knowledge

Knowledge can be seen as information that comes with insights, framed

experience, intuition, judgement, and values. In some sense knowledge

represents truth and therefore offers a reliable basis for action.

Knowledge is the body of understanding and skills that is mentally constructed

by people. Knowledge is increased through interaction with information

(typically from other people).‖

These descriptions are an elaboration on the work of Ackoff (1989) who also

defines:

―Understanding: appreciation of ‗why‘

Wisdom: evaluated understanding. ―

Ackoff does not just make a linguistic distinction between these concepts. He also

applies a hierarchy to them. Each concept builds on the one before it, and wisdom is

the more evolved, valuable and permanent form of knowledge. This distinction is

repeated throughout the literature, with a consensus that knowledge is a much

richer, more valuable resource than simple data. It requires a context, some insight

and other such attributes that can only be achieved when information is processed

by humans.

2.3 Organisational knowledge

The belief that the real value of knowledge comes when it is exposed and transferred

was crucial to understanding the role of the organisation as a facilitator of that

transfer (Grant, 1996). Grant recognises the individual as the knowledge generator

and believes that it is the role of the organisation to facilitate the use of that

knowledge more widely.

18

Re-using the experience of others could be made easier if that experience was

sufficiently generalised to widen its relevance and left open enough to allow the

developers to apply their own interpretation and use their creativity in shaping it to

best fit their project. Faniel & Majchrzak (2006) looked at some of the issues around

knowledge management systems. These systems were intended to provide

engineers with access to knowledge from other departments. Faniel & Majchrzak

found that engineers valued the knowledge the most when they saw it as a stepping

stone towards their own solutions, and when they could trace it back to an expert in

the field. An organisation can share knowledge more effectively if they can provide

an overview in the first instance and access to more specific detail if needed. Access

to the expert who could be the source of the detail was highly valued. This approach

provides the organisation with the role of sharing what knowledge is available, but

still values the role of the individual in sharing what they know and not just sharing

what can be codified in the knowledge management system.

Crossan et al. (1999) describe the three learning levels (individual, group and

organisation) and four processes that move knowledge up the chain towards

organisational learning.

Table 2.1 shows how the processes relate to the levels.

19

Table 2.1. From Crossan et al. (1999). Processes involved in knowledge transfer at different

levels of an organisation.

Level Process Inputs/Outcomes

Individual

Intuiting

Experiences

Images

Metaphors

Interpreting

Language

Cognitive map

Conversations/dialogue

Group Integrating

Shared understandings

Mutual adjustment

Interactive systems

Organisation Institutionalizing

Routines

Diagnostic systems

Rules and procedures

This table shows clearly that the features of knowledge at an individual level are

referred to in far more creative terms than those at the organisational level. As

knowledge is shared between more and more people, it seems to become more

codified, less flexible, less innovative, less personal. Referring back to the

terminology of Ackoff, the organisational knowledge is more similar to what Ackoff

would describe as data or information than it is to understanding or wisdom. In the

terminology of Nonaka & Takeuchi, individual knowledge can be held tacitly and

organisational knowledge is more explicit.

It would seem to be self-evident that having knowledgeable individuals is immensely

valuable. Research has shown that getting the right people on a project is one of the

best things you can do to improve its chance of success (Graham, 2000). There is

some knowledge that simply cannot be held at an organisational level but the

20

organisation is still important in facilitating the sharing of knowledge, and

encouraging an atmosphere of collaboration and data gathering. Staff retention is a

key way of getting a body of knowledge in an organisation, but it is limited and does

not in itself bring any of the innovation that can result when knowledge is shared.

Tsoukas & Vladimirou (2001) look at what organisational knowledge is, and conclude

that it can be said to exist when "individuals draw and act upon a corpus of

generalizations in the form of generic rules, produced by the organization". One

would struggle to argue that the organisation could be said to have wisdom, or

understanding. Organisational knowledge requires individuals to make sense of it

and adopt it as their own.

2.4 Knowledge capture

There has been a great deal of work in assessing methods of knowledge capture,

often based around the technology employed to do this such as databases,

knowledge management systems and portals.

Henninger (2001) concludes that there is a strong tie between the technology used

to capture knowledge and the processes of knowledge management within an

organisation. Faniel & Majchrzak (2006) found that individuals are more likely to

innovate and generate new ideas if they are exposed to new knowledge. They state

that it is not sufficient to expect individuals to seek out the knowledge they need –

there must also be an element of pushing new concepts out to people who did not

realise they were relevant. They see technology as serving a purpose here, but

advocate a system that associates the personal with the concrete.

Liebowitz & Megbolugbe (2003) provide a table that lists seven knowledge

management solutions. The first of these, ―Frequent get-togethers to exchange tacit

21

knowledge (i.e. knowledge fairs, brown bag lunches, bird of a feather tables, inter-

departmental seminars, etc.)‖ is rated as being low in complexity of use and low in

difficulty of development. This would suggest that any organisation that values

knowledge management should have this process well-established.

Each of the other six suggestions involve a substantial technology overhead, ranging

from chat rooms and online communities to ―Using intelligent agents to actively build

user profiles and push appropriate lessons learned and material to the respective

user‖. In the past, a system such as this would require a substantial organisation-

wide commitment to its development, implementation and on-going use. With the

development of technology such as blogs and wikis, there is now a much lower entry

point into some of these applications. New freely-available collaborative solutions

would not have all of the features of fully customised knowledge management

groupware but they can be owned and implemented at a group level and may be

more successful in encouraging the sharing of knowledge for that very reason.

2.5 Knowledge transfer

Despite improvements in technological options, researchers have found that

interactions with colleagues are still one of the most effective methods of sharing

knowledge (Kasvi et al., 2003). Bresnen et al. (2003) confirm the importance of

strong social networks and the role of the individual as the key holder of tacit

knowledge. They raise two key questions:

―First, how is the organisation able to capture learning and deploy it over the

long term, when it is so embodied in the individual and manifested in their

particular expertise and range of contacts? Second, what happens when the

individual leaves and takes their knowledge and contacts with them?‖

22

Information redundancy is the solution to the second of those issues – making sure

that no one person holds a unique body of knowledge – and this problem is

addressed in many different ways by those who have detailed a variety of methods

for knowledge capture.

Davenport & Prusak (1998) cite the example of organisations who:

―record the stories and experiences of its senior practitioners on video or CD-

ROM before they leave the company.‖

Capturing this kind of anecdotal information adds it to an organisational memory.

Technology can then be used to codify and publish it, and make it available to those

who need to use it. Davenport & Prusak make the distinction between using

technology to share the information, and using technology to represent the

information itself. This is a particular issue if the knowledge is tacit and has a

richness that the technological approach can not do justice to.

Reflection (Henninger, 2001), debriefing, surveys, (Collier et al., 1996), Ishikawa

(fishbone) diagrams (Birk et al., 2002) are a few of the more creative methods

mentioned. These can contribute to the sharing of knowledge, but as Davenport &

Prusak point out in Working Knowledge (1998):

―Knowledge that isn‘t absorbed hasn‘t really been transferred‖

This draws a clear distinction between knowledge management and knowledge

transfer. Whilst technology can be used to codify and manage knowledge, and make

it easier for users to retrieve the information they are looking for, this does not

actually spread knowledge. Only once the end user has adopted that information and

preferably used and personalised it, can the benefits of that knowledge management

system be realised, i.e. once knowledge (not just information) has been transferred

(not just captured).

23

Knowledge management can also support reuse in software development, which in

turn leads to improved productivity, performance and quality (Tautz & Althoff, 1997).

Basili & Rombach (1991) state that in addition to the traditional emphasis on reusing

source code, organisations should see all experience as being reusable. They

emphasise the need to capture feedback of experiences for use in future software

development projects, and describe feedback as occurring where an organisation

has a way of recording and packaging learning, and identifying, evaluating and

modifying that learning through reuse.

Desouza & Awazu (2005) list the five capabilities that an organisation must have in

order to claim success in knowledge management: create, transfer, store, retrieve

and apply. Again, there is a focus on what an organisation can achieve. This study

will review what issues practising project managers have with knowledge

management, and whether there are any of these 5 capabilities that prove

particularly difficult.

2.6 Making the most of tacit knowledge

As Marwick (2001) finds, current solutions are strongest when they deal with explicit

knowledge. The creation and sharing of tacit knowledge is weaker. As Marwick

points out, tacit knowledge by its nature is difficult to make explicit therefore

externalization is the hardest to achieve of the four defined forms of knowledge

conversion.

Ambrosini & Bowman (2001) also find that one of the reasons why there have been

very few attempts to empirically research tacit skills is that it is problematic. They find

a need for a method that would allow us to capture the constructed reality of

individuals which allows them to make sense of the world around them.

24

Davenport & Prusak (1998) highlight the factors that encourage knowledge

exchange – reciprocity, repute and altruism. They also state that trust is an essential

factor and that a culture of rewarding sharing and discouraging knowledge hoarding

is also needed. This is also supported by Ji-Hong Park (2006).

The factors that work against knowledge transfer (frictions), as described by

Davenport & Prusak (1998) are given in Table 2.2 below.

Table 2.2. Knowledge transfer frictions (from Davenport & Prusak, 1998).

Frictions Possible solutions

Lack of trust

Build relationships and trust through

face-to-face meetings

Different cultures, vocabularies, frames

of reference

Create common ground through

education, discussion, publications,

teaming, job rotation

Lack of time and meeting places; narrow

idea of productive work

Establish times and places for knowledge

transfer: fairs, talk rooms, conference

reports

Status and rewards go to knowledge

owners

Evaluate performance and provide

incentives based on sharing

Lack of absorptive capacity in recipients

Educate employees for flexibility: provide

time for learning; hire for openness to

ideas

Belief that knowledge is prerogative of

particular groups, not-invented-here

syndrome

Encourage non-hierarchical approach to

knowledge; quality of ideas more

important than status of source

Intolerance for mistakes or need for help Accept and reward creative errors and

collaboration; no loss of status from not

knowing everything

The above findings show that:

Tacit knowledge is hard to capture.

25

There is a strong human element to the willingness of individuals to accept

other lessons as their own.

A shared sense of the world around us is important in being able to

understand and apply knowledge in an appropriate way.

These issues highlight the importance of a field of study that ignores the potential of

technology to manage knowledge and focuses on the potential of individuals to

creatively describe their experiences in a way that allows others to draw lessons

from them.

Something that is apparent from these suggested solutions is that they require a

culture change within an organisation. Something this dissertation aims to do is to

show which of the suggested improvements to knowledge management can be

achieved without extensive organisation support.

Rising & Derby (2003) use the metaphor of programming patterns to illustrate how

learning can be captured and generalized. They suggest that project retrospectives

are used to capture the lessons learned and that when a sufficiently generalizable

lesson is encountered, it is framed as a pattern and made available for future

reference. The key features of a pattern are: Name, context, problem, forces,

rationale and solution. They also suggest that a pattern should have been shown to

be relevant in three different settings before it is considered valid. They consider it

vital that a pattern has a short, descriptive, attention-grabbing title so that it can be

easily encapsulated and adopted by other teams. A review of existing patterns can

then become a part of project planning.

Madhavan & Grover (1998) advocate rich personal interaction as a key ingredient of

effective knowledge creation. Their context is new product development.

26

Brown et al. (2001) have suggested storytelling as a method of capturing the tacit

and explicit knowledge at project review points, and doing so in a form that can be

used by others. They believe that stories can provide a mechanism of sharing

knowledge between communities and can provide a sufficiently generalized context

to increase the perceived relevance of the knowledge being shared.

Clarke & Rollo (2001) give examples of where the codification of tacit knowledge has

been successfully achieved and look at the importance of the flow of that knowledge

being embedded into an organisation‘s processes.

Neve (2003) proposes a framework of semi-structured interviews to obtain a

narrative description that can tap in to the knowledge that participants did not know

they had.

Some of these techniques could be as valid for sharing explicit knowledge as tacit

knowledge. This study looks at the main issues identified by the participants and

considers how techniques such as these could enhance any of their identified

problems.

2.7 Finding the gaps in current processes

Another area of research in preparation for moving forward with this project has been

into the best way to compile that list of the issues that currently exist with knowledge

capture in software development. The nominal method has been used for this sort of

purpose. Input from a number of experts in the field is sought and can be combined

through a series of iterations into an agreed list. Van De Ven & Delbecq (1974) found

that this technique, or the Delphi technique where participants do not need to be in

the same location, produced a better result than a traditional group discussion. De

Ruyter (1996) found a similar result in a comparison of the Delphi method and focus

27

group interviews. The added benefit of the Delphi method is that it allows the

participants to remain anonymous. In this case, this will allow access to managers

from companies that work in a similar field and are in some cases competitors,

without them feeling inhibited from sharing their problems and successes.

In a particularly relevant paper, Dekleva (1992) used the Delphi technique to compile

a list of the problems in software maintenance. These included people being stuck in

a role because the organisation could not afford to lose their experience, inadequate

documentation, and high turnover of staff causing a loss of expertise.

2.8 Research question

This study aims to address the following question:

What techniques can be used to improve the practice of knowledge management in

software development projects, where the organisation as a whole does not provide

strong guidance?

It addresses the question in three parts:

What processes are currently used in management of software development

to capture knowledge gained

Where are existing processes lacking?

What frameworks or methods could provide help in the areas where current

processes are lacking?

28

2.9 Summary

The study of knowledge management can be seen to encompass a wide range of

disciplines, from the philosophy of knowledge and education through to advanced

computer system development. It involves the study of human interactions, the role

of the organisation and optimisation of business practices. At its root, however, is a

belief that sharing knowledge is a worthwhile pursuit, and that knowledge itself is a

resource that organisations should be taking seriously and investing in. Whilst the

organisation can play a very important role in introducing systems to facilitate the

sharing of knowledge it is perhaps more important for it to foster an environment

where the sharing of knowledge is valued and encouraged. The literature places

emphasis on the importance of organisation involvement in knowledge management,

so it was important for this study to take this into account.

29

Chapter 3 Research Methods

The research was split into two complementary tasks. In the first task, the input from

industry experts on their experience of knowledge management was gathered. This

was intended to discover what knowledge management techniques these experts

use as a matter of course, and where these techniques fall short.

In parallel with this, an analysis of the existing methods and frameworks suggested

by the literature was carried out. In particular, there was a focus on the means of

externalising knowledge, to assess how useful they might be in addressing some of

the problems identified by the experts on the panel.

3.1 Other techniques considered

The objective of this part of the study was to gather input from a number of subject

experts about their experiences. Although the Delphi technique was selected, other

options were considered:

3.1.1 Questionnaire

The participants were not knowledge management professionals, although they were

knowledge workers. It was thought that they might have differing views on what

knowledge management might mean, and that a simple questionnaire would not

allow them to feed off each other‘s experience. It was desirable to provide the

participants with an opportunity to review their own suggestions and those of others

when they were asked to provide a ranking.

30

3.1.2 Case study

It would also have been possible to look in detail at a smaller number of projects by

performing a case study. Every project has its own characteristics, however, and the

study was more concerned with the general approaches to knowledge management,

rather than what has been applied in one case. One of the very issues that the study

investigated was how to generalise learning from one project so that it applies across

many.

3.1.3 Interviews

Interviews could have been a useful part of the process – it may have been valuable

to have gathered the initial suggestions from participants in this way. Interviews in

themselves would not have been sufficient as there was great benefit to be gained

from the iterative nature of the Delphi technique.

3.2 The Delphi technique

The process of compiling the list was carried out using the Delphi technique. In this

technique, an expert panel is formed who are then asked for their input on a given

topic. The Delphi technique has been found to be more effective than a standard

focus group when trying to gather suggestions from a group of people. Van De Ven

& Delbecq (1974) found that the number and quality of ideas generated was higher

than in a focus group, and that the members of the group were more satisfied with

the outcome.

In the particular circumstances of this research, the Delphi technique is favoured

over the nominal technique because of the practicalities of getting the expert panel

from a variety of organisations together. The advantages of this approach is that

31

members can commit the time when they have it available, and there is less chance

of participants censoring themselves or being affected by the dynamics of the group.

The potential disadvantage is the extended length of time that the Delphi process

can take, compared with the nominal group method. This was in fact advantageous

in this case, as it allowed for research into the literature to be carried out in parallel to

the gathering of expert opinions.

3.2.1 Participants

The panel consisted of 11 people with current experience of managing the

development of software development projects. It included:

Managers of software development teams

Internal customers of software development teams

External customers of software development teams

Project managers who liase between customers and suppliers

10 of the 11 participants were based in the UK and 1 in India. They worked for 5

different organisations.

It has to be recognised that the group who are involved with the Delphi method for

this dissertation are an opportunity sample and from a narrow field of work – that of

the management of software development. Any findings will need to be considered

as representative only of that context.

The participants were assured of anonymity in their responses. This was particularly

important for this group, because within the group some of the participants were

32

involved in a commercial supplier-customer relationship and some in a competitive

relationship. It would have been inappropriate to have circulated information to all

participants that could be traced back to particular projects and could influence one

participant‘s view of another.

3.2.2 Stage 1: Gathering initial contributions

The first step of the Delphi process (having secured the involvement of the panel)

was to ask participants to complete a short questionnaire. The first half of the

questionnaire asked for input on a number of areas of knowledge management and

transfer.

In the second half of the questionnaire, participants were encouraged to list as many

examples as they thought relevant in the given categories. These responses to those

questions formed the foundation of the rest of the stages of the Delphi method. The

questionnaire is shown in Figure 2.

33

Figure 2. Questionnaire sent to participants in the Delphi method.

The responses supplied by the participants are given in Appendix A.

34

3.2.3 Stage 2: Collation

Once the suggestions and comments had been received, these were collated.

Where similar suggestions were given by multiple participants, these were

combined. The set of responses to three of the questions were taken forward to the

third stage of the process.

3.2.4 Stage 3: Refinement 1

The collated responses to the three key questions were sent out with a request that

participants rank and comment on them. The ranking was to be achieved by

allocating 50 points across all the items in a list, giving most marks to items that were

considered most relevant and fewer or no marks to those items that were considered

less relevant. Some editing had been necessary to combine similar responses, and

to remove company-specific information. This means that it was possible that

participants would feel that their original points were no longer reflected in the

collated lists. It was made clear to participants that they could elaborate on, disagree

with or otherwise refine the list.

The questionnaire sent out for this stage and the numerical responses received are

shown in Appendix B.

When the responses were received, the total marks allocated to each item on the list

were summed and the responses were ranked according to the total number of

marks received, from highest to lowest. Further details of the reasoning behind using

this approach to achieve the ranking are given in Section 4.3.

35

It was decided that the aim was to take between 6 and 8 items from each list forward

for further analysis and the purpose of the refinement stage in the Delphi method

was to compile the 6-8 items.

3.2.5 Stage 4: Refinement 2

The participants were informed that the top 6 items in each list would be taken

forward into the later stages of the study. They were given the list showing the

rankings from 1 to 6.

The participants were also provided with the lists of items that did not make the top 6

and were therefore going to be considered only peripherally in subsequent stages of

the study. They were asked to choose one item from this list as their ―wildcard‖, and

give a reason why it should be carried forward with the top 6. The intention here was

partly to provoke comments on what had been left out and secondly to check that the

top 6 was not missing any items that a number of people felt were important. There

was scope within the plans to take up to two of the anticipated eight wildcard

suggestions forward if it was felt they would contribute anything interesting to the

subsequent research.

The material sent out for this stage is show in Appendix C.

3.2.6 Stage 5: final data collection

The responses were received and the comments and suggested additional

inclusions were considered. The participants returned their questionnaires with their

suggestion of which of the wildcard entries should be considered further. The

36

intention was to see if there was any consensus about which entries to add back in

for consideration. This was a subjective process.

This was the final stage of the Delphi consultation with the panel and resulted in:

6 benefits that knowledge management brings to an organisation

6 of the most consistently used knowledge management techniques

6 areas where current knowledge management practice is failing to capture

the necessary information.

These items were then taken forward for consideration in the rest of the study.

3.3 Analysis of existing solutions

Running in parallel to the Delphi process, the existing literature was assessed to find

techniques that have been used to address the issues highlighted by feedback of the

participants. Section 5.5 provides details of the result of this assessment.

The aim of this study, as previously stated, is to find where existing research can be

used to help address the problems experienced by those trying to make use of

knowledge management techniques.

The solutions offered by the literature were mapped on to a grid that showed their

scope and their level of detail. This was a subjective process.

A more rigorous approach may have been to ask the participants to provide the

mapping to the grid. This additional validation was outside the scope of this study.

37

Chapter 4 Data Collection

4.1 Data Sources

The data required for this dissertation was obtained from a panel of project

managers working in the field of software development. They were all known to the

researcher. Eleven people agreed to participate. At each stage, the people who had

not responded to the previous stage played no further part in the research.

4.2 Process

The questionnaire was sent by email to the 11 participants. 9 were completed

electronically and returned. The responses can be seen in Appendix A: Delphi

method initial questionnaire results. The results from the first three questions are

taken directly into the results shown in Chapter 5.

Of the remaining questions, it was intended that Question 4, about the benefits of

knowledge management, would only provide background information and questions

5 – 7 would be used as the basis for the Delphi method. In fact the results from

Question 4 deserved further investigation and were taken forward. Question 7 was

about other knowledge management techniques the participants were aware of. It

did not provide a data set that would lend itself to further consideration and was not

taken forward. This left three lists that needed further analysis.

4.3 Preliminary Analysis

For each of questions 4, 5 and 6, the same process was applied in analysing the

data.

1. All the answers given by participants were combined into a single list

38

2. Any contribution that covered more than one point was split up so that each

item in the list covered only a single issue.

3. The main themes were identified and listed using language that was general

enough to cover similar issues received from multiple participants.

4. Each of the issues in the list was then mapped on to one of the main themes.

Every issue raised by a participant was reflected in this consolidated list.

5. The lists were presented in no particular order to be used in the later stage of

the Delphi method.

These three lists were sent back out to the 9 active participants and they were asked

to rank and comment on them. The ranking was to be achieved by awarding marks

to each item from a total pool of 50 marks per list. Items that were most relevant

were to be given highest marks and those that were least relevant were to be given

fewest or even zero marks. Before the ranked lists were returned, it was not known

what method would be used to measure the level of agreement between the

participants. An absolute ranking was not required – only an understanding of what

the most important items in each list were.

4.4 Analysing the ranking

Eight of the nine remaining participants returned their ranked lists. The total marks

given to each item in a list were summed and then the lists were sorted from highest

total marks to lowest total marks. This gave three lists that were sorted from most

relevant to least relevant.

This is a relatively simplistic approach to analysis of material gathered via the Delphi

method. Other research has used Kendall‘s coefficient of concordance (W). An

39

explanation of how to perform this calculation is found in Legendre (2005). A simpler

approach would be to calculate the standard deviation and see if it improves over

subsequent iterations. This method was used by Dekleva (1992). Iman & Conover

(1987) propose an alternative approach to calculating correlation that focuses on

agreement amongst the highest ranked items, minimising the significance on the

lower ranked items.

The output of this stage of the study was a list of issues to be taken forward for

further analysis. The study sought to find the most important benefits of knowledge

management, the most consistently used techniques and the hardest to capture

knowledge. It was not important which item was number 1 and which was number 2

because they were both going to be treated equally in the subsequent stages of the

study.

Although the process of simply ranking the totals seems primitive, some further

analysis was carried out to check that it served its purpose.

A second calculation was carried out which summed the totals but excluded the

highest and lowest mark awarded to each item. This should show whether the

ranking of a particular item was unduly influenced by any one contributor. For Lists 1

and 2, this second calculation provided exactly the same top 6 as the first calculation

did. For List 3, the results were slightly different.

The items were being ranked were on the following topic: ―What types of

knowledge have been hardest to capture and share effectively?‖ The item that

was number 2 on the list when all marks were included dropped to number 11 when

the highest and lowest marks were excluded. This was because of a single person

giving it 30 marks out of a total of 50. This item was ―Emails and documents can be

40

hard to keep track of if they are being contributed to by many people.‖ Nobody else

on the panel gave this item more than 5 marks and 3 of the 8 participants did not

give it any marks at all. This result does, therefore, seem to have been skewed by a

single participant. Replacing the item that was number 2 is the item ranked 9th on the

straightforward ranking of scores.

What this shows is that although there is broad agreement on 5 of the top 6 items,

there is some dispute as to the 6th. It was decided to use the straightforward ranking

based on the sum of all marks, but to allow each participant to make the case for a

―wildcard‖ – an additional item to be taken forward. If there was a strong case made

(as assessed subjectively) or a number of participants proposed the same wildcard

item, then it could be included for the latter stages of the study.

41

Chapter 5 Results

This chapter describes: the outcome of the contributions from the expert panel; the

analysis of some techniques that could improve knowledge management and

comparison of these techniques with the needs of the panel.

5.1 Background information

The information in Tables 5.1 – 5.3 was gathered from the participants in response

to the questionnaire.

These results suggest that none of the organisations considered have mature

knowledge management techniques, infrastructure or personnel structure. Any

improvements to knowledge management suggested by this study cannot rely on

buy-in at an organisational level.

Table 5.1. Level of maturity of organisational knowledge management approach.

Question 1 Answer Frequency

My organisation has a defined knowledge management process. 1 0

2 7

3 1

4 1

5 0

Mean 2.33

Table 5.2. Technical sophistication of organisational knowledge management approach.

Question 2 Answer Frequency

My organisation has an IT system designed 1 0

to help knowledge management. 2 2

3 5

4 2

5 0

Mean 3.00

42

Table 5.3. Personnel structure with respect to knowledge management.

Question 3a. Is anyone in your organisation responsible for

knowledge management?

Answer Frequency

Yes 2

No 7

Question 3b. If yes, what is their job title?

Knowledge Manager

Knowledge Consultant

5.2 Delphi method data

In the following sections, the data is presented in the form it evolved through the

Delphi method for each question.

5.2.1 What are the benefits of knowledge management?

When the original questionnaire was distributed, this question was intended as a

background question to gather information about how much value participants

placed on knowledge management. When the results came in, it was apparent that

this question was revealing some answers about what the participants are trying to

achieve through their re-use of knowledge. As this could have some impact on what

approaches to knowledge management best meet their needs, and what issues they

encounter, it was decided to treat this as an input into the Delphi method, and to

refine it further through that process.

43

The data was obtained from 9 participants who provided 23 suggestions. Once the

similar suggestions had been combined, this left 13 distinct suggestions to be taken

forward to the first refinement.

Each participant was asked to distribute 50 marks across the list. Eight participants

returned their marks. Appendix B shows the marks given by each participant. Table

5.4 shows the list sorted by total marks received and also gives the participants‘

comments for each item. The top 6 items in this list were sent out to the participants

with the request that they choose one of the items ranked 7-13 and make the case

for why they thought it should be considered further.

Table 5.4. Responses for list 1 ranked by total marks received, with comments added by participants. Benefits of knowledge management.

Rank Benefit Elaboration from comments received

Total

1 Improved consistency of information - everyone is referring to the same versions and documents

Saves time if someone can be pointed towards a document rather than being offered an explanation. Users need to be able to tell what version a doc is even once it has been printed out.

53

2 Reduced need to reinvent the wheel for every project

45

3 Improved speed of issues resolution by making existing solutions to past problems available

Needs appropriate filtering and keywords.

42

4 Training for new starters - provides a consistent set of information and experience for them to tap into

Very useful for contractors and handovers too.

41

5 Shared workload - no single person is the sole keeper of knowledge

Avoids disaster when someone leaves, but documentation usually fall behind what is in someone‘s head.

37

6 When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need

Usually easier to go to the source of the information directly.

34

7 New work can get off the ground more quickly

Why? 33

44

Rank Benefit Elaboration from comments received

Total

8 Lets customers know what we already know about a product and reduces support calls

29

9 Urgent information can be pushed to those who need to see it

Needs a certain amount of design and maintenance to ensure that the mailing lists remain up to date.

27

10 It encourages a culture of sharing and cooperation

No evidence that this is the case. 27

11 Increased quality of finished product Experience of other people useful in shaping future approaches

19

12 Increased ability of an organisation to recognise synergies and opportunities

Especially when departments are doing similar things that could better be achieved through co-ordination e.g. Business Continuity Plans.

11

13 The ability to share knowledge outside the business

Should be careful with customer communications.

2

5.2.2 What techniques are currently used?

The data was obtained from 9 participants who provided 49 suggestions. Once the

similar suggestions had been combined, this left 18 distinct suggestions to be taken

forward to the first refinement.

The marks awarded by the eight participants who responded are shown in Appendix

B.

Table 5.5 shows the total marks given to each item and shows the items ranked from

highest to lowest. Again, the top 6 items were sent back out to have their ranking

refined and the participants were asked to choose a wildcard from the remainder.

45

Table 5.5. Responses for list 2 ranked by total marks received, with comments added by participants. Commonly used knowledge management methods.

Rank Method Elaboration from comments received

Total

1 Common storage area / intranet for project documents

Good idea to storing key documents, but needs housekeeping and a good structure to ease finding documents. Invaluable on large projects.

66

2 Defect list for technical bugs Very useful measure of quality, remaining work, trends and pattern analysis.

38

3 Ongoing issues list throughout a project

Only useful if small enough to manage, and not too detailed.

37

4 Lessons learned written up and placed in shared location at end of project

Would be more useful if they force entry into a common database with relevant topic keywords.

35

5 Post project reviews Sometimes compiled and shared in meeting form. Useful to the individuals within the project team, but not used outside very much.

32

6 Progress reports shared with wide audience

Can be difficult to get these read by appropriate audience, perhaps because they are at the wrong level - too detailed or too high level.

29

7 List of unresolved issues at end of project

Duplicate of lessons learned 20

8 Best practice examples in a shared area

Very useful, especially if key lessons and tips are summarised.

20

9 End project reports Duplicate of lessons learned and usually a result of post-project review

18

10 Flowcharts Can be very useful with simple projects, parts of bigger projects or high level views. Can quickly deteriorate with more complex flows and detailed views.

17

11 Sharing checklists Need to balance between too detailed and too vague.

If too detailed:

- People tend to answer the checklist and don‘t consider issues not directly covered by the checklist.

- Reluctancy to address areas not

17

46

Rank Method Elaboration from comments received

Total

covered by the checklist.

- People can fall into the habit of carrying out irrelevant checks too much.

If too vague:

- Checks open to misinterpretation leading to false positives and negatives.

NOTE: Regular maintenance of the results and the contents are required and should form part of the project review process.

Useful for remote teams

12 Wiki Difficult to get agreement about who should have access to what.

17

13 Workshops with other project managers

Under-used, but can be very effective. We think of red team review when it‘s failing technically, but do not have a management equivalent.

Have these set up with another division and within the division on particular projects.

16

14 Project blogs Esp useful for noting technical problem solving.

Scored 1 because although I‘ve never used it, believe it could be useful.

11

15 Published articles that highlight known issues and share useful information

Same as open issues list? Or is this referring to a post-project ―issues that came up‖ list? In which case same as end project reports?

Good for technical information, but the system we have is not currently geared up for internal information.

8

16 Defect list for problems with processes

Not really used, but will use standard defect database when required.

8

17 Instant messaging Scored 1 because although I‘ve never used it, believe it could be useful – although may be distracting if not specific.

6

18 Daily log of conversations and phone calls

Overkill. 5

47

5.2.3 What other techniques are you aware of but have not personally

used?

The list that was obtained after collating all the contributions to this question is

shown in Table 5.6. The data was obtained from 5 participants who provided 8

distinct suggestions. This list did not lend itself to further refinement. The interesting

finding to be gained from these responses is that participants had little detailed

understanding of any knowledge management techniques that they did not currently

use.

Table 5.6. Collated responses to Question 6 of questionnaire.

What other techniques are you aware of but have not personally used?

Use of a project office to collate details that can be used for lessons learned Library of historical project metrics, e.g. estimates vs outcomes

Formal post-project reviews

Knowledge-based software Using help-desk tracker software to collate problems solved and questions answered.

Wikis

Blogs

Collaboration-enabling software

No further analysis of this list was undertaken as part of the Delphi process.

5.2.4 What types of knowledge have been hardest to capture and share

effectively?

The data was obtained from 9 participants who provided 23 suggestions. Once the

similar suggestions had been combined, this left 16 distinct suggestions to be taken

forward to the first refinement.

The marks awarded by each of the eight participants who responded are shown in

Appendix B.

48

Table 5.7 shows the total marks given and shows the items ranked from highest to

lowest. The top 6 items were sent back out to have their ranking refined and the

participants were asked to choose a wildcard from the remainder.

Table 5.7. Responses for list 3 ranked by total marks received, with comments added by participants. Hard to share knowledge.

Rank Knowledge type Elaboration from comments received

Total

1 Knowledge that will help provide more accurate estimates in future

This is an area that is usually poorly recorded:

- Resources too busy to focus on introspection.

- Time recorded poorly, sometimes entered weeks after the event.

- Unwillingness to be open and honest with themselves and others.

- Difficulties breaking down the tasks and including the intangible work e.g. thinking time.

- If work is unique, tendency to guess rather than seek equivalent parallels and use the data for predictions.

- Tendency to get stuck in to the detail rather than take a step back and discuss the work and estimates.

54

2 Emails and documents can be hard to keep track of if they are being contributed to by many people.

It‘s so easy to spend days trying to solve a technical issue and not write it down for next time. Bug trackers that include resolutions can be helpful for this.

43

3 The solution to specific technical issues that have had to be overcome

People can be protective of their approach, so difficult to break down into the constituent parts.

37

4 Subjective opinions - discussions on how something should have been done or how successful a particular approach was

Not something that is usually written down.

36

5 Relationship information - who is helpful and who is not constructive

Information needs better structure including considered keywords, facts detailed, inferences explained and key messages summarised.

33

49

Rank Knowledge type Elaboration from comments received

Total

6 Keeping information general enough to be applied in other scenarios without being too vague.

Knowledge needs to be reviewed regularly and trimmed on occasion.

29

7 Mistakes made early on in a project, long before a post-project review or lessons learned reports

End Stage Reports, Project Meetings. Don‘t need to wait for formal post project reviews.

25

8 Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work

This does cause problems, especially if there is a dispute with a developer.

Still go for documentation, as it often yields results. Can be kept as short and sweet as possible.

24

9 It can be hard to choose how much information to share to achieve a manageable quantity of information

Knowledge needs to be reviewed regularly and trimmed on occasion. Add ―next review dates‖, give someone the responsibility to manage data and prune accordingly.

24

10 Lessons learned in other teams Not widely communicated and either too detailed and therefore unread or too top level and out of context.

23

11 Root cause of delays and schedule slip

Duplicate 22

12 An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?

At the start of every new software project I emphasise that we should aim to produce a simple, elegant product, but every time we end up with something that is more complicated than it needed to be. Such products generate a lot of calls from customers.

Usually get the author and audience together to agree this.

14

13 Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.

Duplicate 14

14 Early warning signs Need people to manage their own risks, implement measures and alert when breached.

11

50

Rank Knowledge type Elaboration from comments received

Total

15 Cause and effect Especially when people focus on blaming each other rather.

8

16 It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.

Duplicate 3

5.2.5 Wildcards

The ranked lists were sent back out to the eight participants who had contributed.

These participants were asked to make the case for the inclusion of any of the items

that had not been ranked in the top six. Six participants responded. The full results

are given in Appendix C. No more than two people voted for any one of the items.

There was no particular weight behind any single item to suggest that it should be

taken forward for further analysis ahead of the others. For this reason, the lists to be

analysed further were left as the top six ranked items as explained above.

5.3 Results of Delphi process

In summary, the following results were obtained. According to the participants:

The strongest benefits that knowledge management can bring are:

1. Improved consistency of information - everyone is referring to the same

versions and documents

2. Reduced need to reinvent the wheel for every project

3. Improved speed of issues resolution by making existing solutions to past

problems available

51

4. Training for new starters - provides a consistent set of information and

experience for them to tap into

5. Shared workload - no single person is the sole keeper of knowledge

6. When you are aware that something similar has been done before, knowledge

management makes it easy to find the information you need

The most consistently used knowledge management techniques are:

1. Common storage area / intranet for project documents

2. Defect list for technical bugs

3. Ongoing issues list throughout a project

4. Lessons learned written up and placed in shared location at end of project

5. Post project reviews

6. Progress reports shared with wide audience

The problems the panel encounter in knowledge capture are:

1. Capturing knowledge that will help provide more accurate estimates in future

2. Emails and documents can be hard to keep track of if they are being

contributed to by many people.

3. Capturing the solution to specific technical issues that have had to be

overcome

4. Capturing subjective opinions - discussions on how something should have

been done or how successful a particular approach was

52

5. Capturing relationship information - who is helpful and who is not constructive

6. Keeping information general enough to be applied in other scenarios without

being too vague.

Given these findings, the existing literature on knowledge management was

consulted to examine how it might help improve current practice.

5.3.1 The connection between ambition and practice

From these results, it is possible to see how current practice meets the ambitions of

the participants. Figure 3 shows how each of the perceived benefits of knowledge

managements connects to one or more of the most consistently practiced

techniques.

There is at least one technique being used that should help to achieve each benefit,

and every technique is associated with a desired benefit. This shows a consistency

between ambition and practice that suggests the participants are focussing their

efforts on the areas that they perceive to matter most – and getting the greatest

return for their effort.

53

Figure 3. Connections between intended benefits and practice.

5.3.2 Gaps in current practice

Although Figure 3 suggests that the participants are focussing their efforts on exactly

the correct techniques to meet their aims, a different picture emerges when the gaps

in current knowledge management are considered.

Figure 4. Connections between problem areas and practice.

Figure 4 shows that there is no obvious connection to be found between two of the

identified gaps in knowledge capture and any of the techniques that are used most

54

consistently. It further shows that four of the gaps could be addressed by current

techniques. The fact that the participants identified these gaps suggests that the

application of these techniques could be improved, or that more suitable techniques

are required.

5.4 Comparison with the literature

Using the analysis of current practice above, this study examines the existing

literature on knowledge management with two aims:

1. To find ways to improve the implementation of existing techniques.

2. To find new techniques that will fill those gaps that are not currently

addressed at all.

To help identify what sort of new techniques the study should be considering, the

existing techniques and the gaps were plotted on a grid. The X axis of the grid is for

the level of detail and the Y axis is for the scope of the information.

In Figure 5 the currently used techniques are shown depending on the sort of

knowledge that they are good at managing.

Top left box in the grid is for detailed knowledge with project-specific

relevance.

The bottom left box is for more generalised knowledge that has cross-project

relevance.

Top right is for generalised knowledge but applying only to a specific project.

Bottom right is for general and cross-project.

55

There is a degree of subjectivity in this categorisation, but it provides a useful tool to

compare techniques and gaps.

Figure 5. Current knowledge management techniques categorised in terms of the scope of their relevance across projects and the level of detail they capture.

From Figure 5 is can be seen that the techniques used at the moment are heavily

biased towards the detailed side of the grid, and are also better for capturing

knowledge that is project-specific. Lessons learned reports and other published

project reports can have a wider audience but must be written in a way that makes

the information they contains seem digestible and relevant to other projects. In

general, the techniques used rely heavily on some form of documentation and seem

56

geared towards getting information down on paper (or stored on a computer) and

trying to make that information available to those who need it.

Figure 6 then shows the gaps that have been identified, plotted onto the same grid.

This shows that the participants have found it difficult to capture knowledge that can

be shared between projects and that is more general in nature. The exception to this

is the need to capture information on which to base future estimates. This

information is wanted so that it can be applied to future work, it is historical

information. It needs to be captured in detail in the context of what was done on a

specific historical project and metrics are needed to measure how long activities took

and in what context.

This comparison suggests that the study should focus on techniques that allow less

detailed project knowledge to be transferred between projects. It is also valuable to

consider some techniques that do not rely so heavily on written documentation. A

further requirement is that the techniques do not need a high level of organisation

involvement in order to be successful. The participants clearly indicated in their

responses to the first questionnaire that their organisations were not pro-active in

knowledge management.

57

Figure 6. The gaps in current practice categorised in terms of the scope of their relevance across projects and the level of detail they require.

In researching the existing literature, the methods in Table 5.8 were found to offer

some value over and above current knowledge management practice. They were

chosen because they provide a contrast to the very document-centric processes

identified above.

For each of the techniques identified in Table 5.8, this study will consider

Which gaps in knowledge management practice they could help to fill

Which currently used techniques they could complement or extend

58

What level of organisational support is necessary for them to be implemented.

This is particularly important given the lack of organisational involvement in

knowledge management identified by the participants.

Table 5.8. Literature sources of research into knowledge capture.

Method Origin Mode of transfer

Rich personal interaction (Madhavan & Grover, 1998) Socialization

Programming patterns (Rising & Derby, 2003) Externalization

Facilitates internalization

Storytelling (Brown et al., 2001)

Externalization

Facilitates socialization

Blogging (Ojala, 2005) Externalization

Combination

Discussion groups and

collaboration software

(Liebowitz & Megbolugbe

2003)

Socialization

Externalization

Semi-structured interviews (Neve, 2003) Externalizaton

5.5 Analysis of practice vs literature

This section will compare the problems identified by participants with the solutions

proposed in the literature. It will also look at how practical the solutions could be

based on the low level of organisational involvement in knowledge management

identified by the participants.

Tables 5.9 and 5.10 show the correlation between the solutions proposed by the

literature, the gaps we are concerned with filling through new solutions and the

currently used techniques. Each ―x‖ in these tables is dealt with in greater detail

below.

59

Table 5.9. Matrix showing which gaps in knowledge capture identified by the Delphi method are addressed by which techniques from the literature.

Gap __________

Technique Future estimates

Keeping track of email chains

Solutions to specific issues

Subjective opinions

Keeping information general

Relationship information

Rich personal interaction

X

X X X X

Programming patterns

X

X

Storytelling

X X

Blogging

X

X

Discussion groups

X X X

Semi-structured interviews

X

X

Table 5.10. Matrix showing which of the current techniques are extended or complemented by the additional techniques found in the literature.

Technique

Rich personal interaction

Programming patterns Storytelling Blogging

Discussion groups

Semi-structured interviews

Common storage for project docs

X

Issues list

X

X X

Defect list for technical bugs

Post-project reviews

X

X

Published project report

X

Lessons learned report

X X

X

5.5.1 Patterns

The research of Rising & Derby (2003) draws an analogy with programming

patterns. These are templates for common requirements in software development. A

programming pattern is an approach to solving a problem that can be used in many

different contexts. Whereas an issue list is project-specific, a pattern could be used

60

to take the resolution to that issue and try to generalise it so that it is more relevant

across a number of future projects. Rising & Derby‘s suggestion is that we take time

to reflect on our experiences by carrying out retrospectives.

―During a retrospective, the project team can identify likely patterns, and small

groups may write a pattern outline as part of processing what they‘ve learned.

[…] ask them to think carefully about the project and identify two or three of

the project‘s most critical moments. A critical moment can be a decision, a

turning point, or an action that overcame an obstacle or made a difference in

some other way. ‖

The issues list could be a good starting point for this work and the participants did

identify the post-project review as a useful technique. The purpose of the

retrospective and the use of patterns is to see how the resolution to an issue for one

project can be turned into a more general pattern for other projects. This also helps

to address one of the issues identified by Davenport & Prusak (1998) – there can be

a reluctance to use knowledge from outside a team, a resentment of being asked to

adopt something that was ―not invented here‖. A pattern provides a general

approach, often summarised by a memorable name that helps it to be shared and

captures the intent. The pattern itself is then a starting point for a team to be able to

understand the problems encountered on other projects and to add to their arsenal

of possible solutions to that sort of problem.

The idea of patterns is not new. They are analogous to a means of communication

that has been around for centuries – proverbs. ―A bird in the hand is worth two in the

bush‖ conveys a lesson learned in a memorable and accessible way. It is not

necessary to know what the experience was that led to the creation of the proverb –

we can all relate to the message and the wisdom.

61

A similar approach is taken in the military. Heath & Heath (2007) describe

Commander‘s Intent. This is a crisp, clear unambiguous message that is passed

from those commanding an operation to those implementing it.

―Suppose I‘m commanding an artillery battalion and I say, ‗We‘re going to pass this

infantry unit through our lines forward.‘ That means something different to different

groups. […] As a commander, I could spend a lot of time enumerating every specific

task, but as soon as people know what the intent is they begin generating their own

solutions.‖

The Delphi method found that solutions to specific issues were hard to capture and

that it was difficult to keep information general enough for others to use. Patterns can

address both these issues, and can be an extension of the existing techniques of

capturing issues and lessons learned, and conducting a post-project review.

Rising & Derby (2003) explain the organisational support that they advocate to get

the process of generating and using patterns adopted.

―Patterns and insights won‘t permeate the organization on their own. Someone

needs to sing the songs and tell the stories of successful solutions to devilishly

recurring problems — a patterns minstrel, if you will.‖

They suggest:

Post patterns on an intranet

Host a monthly tech forum to update the organization

Speak the language (Use pattern names in a way that helps the concepts

become embedded

Help newcomers to become familiar with the use of patterns

62

These are the actions that are necessary to help share the patterns across an

organisation and this is where the greatest benefits can be achieved. Without

organisational buy-in, however, there is still value in using the structure of patterns to

guide the production of lessons learned and the post-project review and to help

produce something that manages to generalise the specific experience from a

particular project.

5.5.2 Document management and collaboration software

Three of the gaps identified in the Delphi study were around the management of

explicit knowledge: Difficulty in keeping track of email chains, recording solutions to

particular problems and recording subjective opinions.

Technology can help here, and recent developments in collaboration and networking

software mean that it can now be implemented on a small scale. Standard office

software now allows for multiple people to comment on a document and for their

remarks to be stored within the document itself. This adds richness and functionality

to the common storage area for documents that the participants in the study

currently use. An extension of this would be an application like Google docs

(http://docs.google.com), that allows multiple users to be editing the same document

at the same time. There are numerous bulletin board or group applications that allow

users to thread discussions and keep track of the history of a discussion. In the past,

such a system would have required a substantial investment of time and money from

an organisation. Today, these systems can be managed at a corporate level or they

can be created for free or nearly-free. Google Groups (http://groups.google.com/)

provides discussion groups and document storage for free, as does Microsoft

(http://home.services.spaces.live.com/). These can be used to keep track of

63

discussions and documents on a project basis, and they can be used to publish

information more widely.

Using a shared, collaborative system for issue logging can help team members to

keep track of issues and can be used to record solutions and discussions around

particular issues. This extends a project issue list (already identified as valuable by

the participants in the study), by also capturing the discussion around an issue.

Participants identified the capture of subjective opinions as an issue. George Orwell

wrote ―History is written by the winners,‖ and it may prove difficult for someone who

has been a dissenting voice on a project to get their views captured in project

documentation if they have not been successful in having their ideas implemented.

Capturing the process of choosing a solution by recording the discussion online

rather than in email chains is one way to record all considered solutions, not just the

one that was implemented. This also ensures that all knowledge made explicit

through the discussions is captured.

Rising & Derby (2003) state that one of the best aids to problem-solving is a wide

range of experience and suggestions to draw from. A range of opinions on ways to

solve old problems could therefore be a useful starting point for future issue

resolution.

Bresnen et al. (2003) found that strong social networks help with knowledge sharing

and the communities that can be built up using social networking technology or

discussion groups can help here. When people know where information has come

from, and they feel they can trust the source, they are more likely to use that

information.

64

One of the fastest rising forms of publishing in recent years has been blogging. Ojala

(2005) found that blogs were particularly useful in sharing knowledge in cross-

cultural environments. This is another issues mentioned by Davenport & Prusak

(1998) and is also an avenue for people to publish their own views without a

consensus needing to be reached before it is seen by an audience. This form of

publishing is readily availably today and needs no organisational oversight. If an

organisation did want to embrace this technology, then it would be possible to create

a centralised implementation that could co-ordinate the metadata associated with

blog entries, index the entries and highlight any that were particularly worthy of wide

attention.

Although they would bring the greatest benefit if implemented at an organisational

level, these technologies are now available with very little set-up or cost. They are

therefore viable at a project level for use by a single team.

5.5.3 Storytelling

Two of the gaps identified by the participants in the Delphi study involve the sharing

of less concrete knowledge. They find it difficult to share knowledge in a way that

keeps lessons general, and they don‘t know how to share information about how

personalities and relationships impacted their projects. This is not quite tacit

knowledge. They know they have this information and that it would be useful to

share, but it does not lend itself to documentation. Storytelling could be one way of

meeting this need for an almost anecdotal approach to knowledge sharing. Denning

(2001) recounts the process of using storytelling at the World Bank. The focus of the

book is on how storytelling encouraged buy-in to the adoption of knowledge

management within the organisation, rather than storytelling as tool for knowledge

65

management itself. However, Brown et al. (2001) advocate storytelling as a means

of relaying information captured at project review points and see it as a useful

technique for knowledge management. They share the view of Denning that

storytelling builds a connection between listener and storyteller. The listener

becomes an active participant in the experiences being recounted:

―In narrative, there is thus an implicit invitation to the listeners to fill in the

missing links in the story. If the listeners accept the invitation, they are thus

inside the story, projecting themselves into the situation, living the

predicament of the protagonist, feeling what he or she was feeling,

experiencing the same hopes and fears. In such a lived-in experience, it is not

difficult for the participants to visualize the missing links. In fact, they will find it

difficult to resist adding necessary patterns and linkage to the narrative.‖

Denning (2001)

Similarly, Heath & Heath (2007) explain how the act of listening puts the listener into

a problem solving frame of mind. The listener is primed to look for connections and

solutions to the problem presented to them. More importantly, they extrapolate from

the specific scenario described in the story to try to gauge how it might be relevant to

them.

This characteristic of storytelling could be beneficial in overcoming the resistance to

taking on information from other projects. Storytelling allows the listener to fill in the

gaps and to think about what they would have done in the same situation. It is a

dynamic process, and allows a story about a specific project experience to have a

broader scope of interest and to be generalised. It adds to the experiences that the

listener can draw on when confronted with a similar situation.

Storytelling is something that can be carried out on a small scale, but would require

willingness from those on project teams to think about putting a story together and

willingness on the part of others to listen. It has the benefit that stories, or important

66

points from stories, can be passed on through the normal socialisation and

interactions of staff beyond the initial pool of recipients. Put simply, people tell each

other stories.

In the literature about implementing storytelling, one common thread is that it is

something that has been established in an organisation through the efforts of a few

―believers‖. It is an interesting concept because it is so far removed from the usual

document-based methods of capturing knowledge and it relates back to the work of

Nonaka & Takeuchi who look at sharing tacit knowledge and the use of metaphor to

go beyond what is normally recorded when you ask someone what they know. The

difficulty with this approach, however, is that storytelling is a skill and whilst it may be

possible to teach and learn this skill it is not one that a project manager can be

assumed to have ready access to. Even if project outcomes can be captured in this

way, setting up the process of sharing these stories is likely to require an evangelist

to get buy-in from other teams.

Storytelling could provide a new take on lessons learned reports and published

project reports but this is not a quick-win solution. It is one that either needs to be

driven from the grass-roots by an enthusiastic and talented storyteller or backed by

an organisation willing to lead by example.

5.5.4 Semi-structured interviews

One of the goals of knowledge management is to capture knowledge held by one

and capture it in a form that makes it available to many. This is particularly difficult to

achieve with the externalisation of tacit knowledge. This is a limitation of the post-

project reviews that the Delphi participants identified as one of their key knowledge

management techniques. Despite carrying out reviews and publishing the results,

67

the participants identified a gap in their ability to capture solutions to specific issues

and relationship information. Neve (2003) criticises the focus of some knowledge

management approaches on technical solutions, stating that:

―When knowledge systems are built, it often seems that we forget that we

cannot extract this from individuals without their participation, motivation, or

awareness of their knowledge. There is a neglect of a personal dimension.‖

Neve‘s solution is to motivate individuals to share information by conducting

interviews that seek to get interviewees to describe events, getting them to provide

examples and a narrative. This provided individuals with an opportunity to reflect on

and analyse their experience and to question their assumptions. When the process

of carrying out these interviews was trialled, Neve found that interviewees had

difficulty in understanding how much they really knew. The process of questioning

them with a view to extracting examples was found to help individuals to share their

tacit knowledge. Depending on the direction of the questioning, it could be possible

for this approach to also gather information about the roles played by individuals in a

project that would then help to provide a picture of which relationships were

successful and which were not. This was identified by the participants of this study

as a difficult area to gather and share information on. It is a sensitive area because

of the potential damage to reputations and the need to fairly represent an individual.

The focus on examples and narrative may allow a more concrete, constructive

description of relationship issues to be gathered and shared.

This process of externalisation could help add more depth to post-project reviews by

increasing the amount of tacit knowledge that is shared, and could also provide input

to the lessons learned report. Neve‘s view was that these interviews need to be

―carried out proactively and deliberately within an organisation,‖ but there does not

seem to be any reason why they could not be carried out for particular projects.

68

5.5.5 Rich personal interaction

As has been mentioned previously, all of the key knowledge management processes

used by the participants in the Delphi study are document-based. In contrast,

Madhavan & Grover (1998) advocate rich personal interaction. Their context is new

product development teams, but the principles are valid in a broader context. They

argue:

―Rich personal interaction, consisting of direct, frequent, and informal

interaction among team members, will influence the trust in the team

orientation of other members positively, which, in turn, is related positively to

the efficiency and effectiveness with which embedded knowledge is converted

to embodied knowledge.‖

This is also the point put forward by Ji-Hong Park (2006) who demonstrates the role

of trust in the reuse of knowledge. Madhavan & Grover discuss some of the factors

of rich person interaction that can contribute to its success:

Direct personal interaction is most effective

Frequent interaction is required, not just close physical proximity

Informal interaction is more effective than formal networks

Two of the many possible benefits of developing a team that practices rich personal

interaction are an increase in innovation and improved information redundancy. The

implications of this approach are most relevant to those constructing teams.

Madhavan & Grover recommend a team structure where there is a wide breadth of

interests and a diversity of personal networks. This approach would require buy-in

from those controlling personnel allocation and it is not always possible in practice to

pick and choose team members based on anything other than availability. Some of

69

Madhavan & Grover‘s recommendations, however, are more applicable at the

project level. They suggest constructing team goals for intermediate stages of a

project so as to build up trust within the team. They also recommend further

investigation of some of the virtual office practices, and are concerned that these

reduce the physical proximity of team members and therefore reduce the quality of

their interaction.

From Liebowitz & Megbolugbe (2003), it was stated in Section 2.4 that the

suggested knowledge management improvement with the lowest barrier to entry was

more face to face contact. It was proposed in this dissertation that any organisation

that values knowledge management would have this process in place. This study,

however, has found that there is a lack of interpersonal contact in the context of

managing knowledge.

Of all the techniques considered in this study, improvements in the quality of

personal interaction between team members have the best overlap with the

knowledge management gaps identified by the participants.

Providing context to enable future estimates to be based on other‘s

experience

Sharing solutions to specific issues, in an informal, ad hoc way

Sharing subjective opinions – increased trust will encourage forthright sharing

of views

Keeping information general – frequent interaction can help individuals to

share mental models which allows information to be transferred in a way that

all individuals can relate to.

70

Relationship information – the informal networks that are created when

interactions are direct and frequent should allow better relationships to be

established. Better trust between individuals can free them to share

information about their relationships with others.

It seems that a worthwhile balance to the rigour of documentation and explicit

knowledge capture would be an increased focus on improving the opportunities for

frequent, informal, direct interactions between team members. This is something that

can be achieved at a project level without the need for organisational involvement.

5.5.6 Estimates for future work

The approaches above have provided suggestions for how all except one of the

gaps identified by the participants in the Delphi study could be filled. Software

development estimation is a field of study in itself. Gray & MacDonnell (1997) is just

one of the many papers that compare different techniques. This study does not deal

with the various techniques of project estimation, but Gray & MacDonnell highlight

two issues that are relevant here:

Good estimates require historical data.

Future estimations need to take into account how closely the new tasks match

tasks that data is available for.

Time capture software is available to assist developers in keeping track of how much

time they spend on each task. It is outside the scope of this study to recommend

particular software applications to assist in this as it is not knowledge that is being

captured. It is management metrics and this can be done mechanically and

objectively.

71

The relevant point for knowledge management is that when this information is used,

it must be possible to relate it to both the work that was done and the work to be

estimated. A developer who is to provide an estimate must know how similar the

future work is to work that the developer has done before, so that there is a basis for

the estimate. The more experience they have, the more likely they are to be able to

relate the new work to existing work. The project manager then simply has to be able

to tell how good the developer is at estimating and this can be done by tracking how

well previous estimates have matched actual development time. Spolsky (2007)

provides one method for evidence based scheduling that follows exactly this

process. It meets the requirement of being something that can be applied at a team

level, as it relies on knowledge of how well individuals in a team have estimated in

the past. It does not rely on input from outside the team or from across an

organisation.

Once a process such as this is in place, more accurate estimates could be provided

by creating a larger pool of experience for each developer to call on when comparing

future work to past work. This then becomes a very similar issue to the other cross-

project issues identified above.

5.5.7 Summary of proposed techniques

As is so often the case in knowledge management, these approaches can be

considered to fall into two categories – the human and the technical. Techniques

such as rich personal interaction, storytelling and patterns all rely on the human side

of knowledge management. The document management and collaboration software

sit on the side of technical solutions. There is also now, however, a vibrant middle

ground where software can be used to build communities, extend relationships and

72

publish. In the past this would only have been possible with expensive bespoke

systems but it is now available to anyone with an internet connection. This opens up

the possibility of combining the technical with the interpersonal to extend the benefit

of both.

5.6 Validation

The research question evolved over the course of the study in response to the

feedback from the participants and the findings of the literature research. The study

gives good coverage to the current practice and gaps as identified by the Delphi

method participants. It demonstrates how the gaps are a possible consequence of

the techniques used, although it does not show that they are an inevitable

consequence, or that the findings are specific to software development.

The need to associate these findings with software development stems more from

the pool of participants than from the approach of this study. An expert panel drawn

from a wider range of organisations could allow for that limitation of the study to be

removed.

The proposed solutions were chosen because they provided a variety of different

techniques from those currently in use. In relating the proposed solutions, the issue

of the lack of organisational involvement in knowledge management is touched on,

but is not examined in detail in partnership with the participants. Similarly the

suitability of these techniques as solutions is not discussed with the participants and

this does mean that they can only be considered as suggestions rather than having

the weight of recommendations.

For this reason, the research question should perhaps be changed from ―What

techniques can be used…‖ to read:

73

―What techniques could be used to improve the practice of knowledge management

in software development projects, where the organisation as a whole does not

provide strong guidance?‖

5.6.1 Analysis

The use of the Delphi method evolved during the study. It was not clear enough at

the start of the study that an absolute ranking was not necessary. The initial stages

of the Delphi method were geared towards gathering a large number of ideas then

sorting them into an order. It was only when the first refinement was returned that it

became apparent that all that was needed to progress the study was agreement on

the most important areas.

When the initial responses to the questionnaire were received, it was necessary to

combine similar responses into a single point. As little editing as possible was carried

out, to avoid changing the sense of the items that were submitted by the participants.

This resulted in some quite similar items being sent out for refinement. This may

have led to some confusion in the marks and it is arguable that the marks for these

similar items should have been combined when the ranking was carried out. If this

process were repeated it may be desirable to spend more time amalgamating the

suggestions, but to have an independent third party audit the changes made.

The mapping of techniques and gaps in Figures 5 and 6 was a very useful visual

tool, but it is subjective. The mapping cannot be claimed to be definitive, empirical

analysis. To add this rigour it would have been necessary to ask a number of

independent parties to provide their mappings and to check correlation.

74

Although the figures point towards a possible reason for the gaps in knowledge

management, they cannot be proven to cause them. A more focussed study that

concentrated on techniques used vs knowledge management gaps within individual

organisations would be one way to demonstrate this causal link but that was not in

the scope of this study.

75

Chapter 6 Conclusions

The participants in this study identified many benefits that knowledge management

could bring to their organisations. They could also identify many areas where they

felt that they were not achieving all the benefits that were possible. These gaps in

knowledge management were considered and it was proposed that the areas that

proved problematic were those that required inter-project, general communication.

The study suggests that the strength of the techniques that were most commonly

used was in capturing project-specific information, and that none of the techniques

would excel in sharing knowledge between projects.

The study then puts forward some knowledge management techniques that contrast

with the very document-centric techniques currently being used by the participants.

Their potential for being implemented without extensive organisational support was

considered and it was found that whilst organisational co-ordination would be

valuable, it should be possible to implement these techniques at a project level. This

is possible because of the advances in technology that have provided cheap or free

off-the-shelf collaborative software.

Some of the techniques, for example creating patterns or storytelling, require skills

that may not already exist within project teams, and are only likely to be implemented

either through organisational change or through the actions of an evangelical core. It

could be considered that they are structured forms of perhaps the most valuable

technique of all – rich personal interaction. That is the technique that would seem to

provide improvements across the most problem areas and is the most distinct from

the techniques that are currently the focus of knowledge management activity.

76

6.1 Project review

The study has succeeded in gathering an overview of current knowledge

management practice by the participants. It has also analysed the impact that some

alternative, currently unused techniques, could have on the quality of knowledge

shared.

The literature suggests that tacit knowledge is the most difficult to capture and much

effort has been expended on investigating techniques to help in this area. It was

expected that the issues found by the participants in this study would reflect the

difficulty with tacit knowledge.

In fact, the gaps identified were more related to the difficulty in sharing knowledge

that had already been gained, particularly outside the project that the knowledge

originated from.

This dissertation was able to consider the gaps identified by the participants and look

for techniques that could help to improve their practice.

The study also identified that knowledge management practice was not mature in the

organisations that the participants are employed by, and was able to take that into

account when reviewing possible techniques.

6.2 Future research

An extension of this study would be to take the suggested additional techniques

forward with these participants or those in a similar position and investigate whether

they do indeed help to fill existing gaps in knowledge management practice.

As mentioned previously, the gaps identified by the participants seemed to be a

consequence of the techniques used, but it was not possible to prove cause. Another

77

study could investigate whether or not cause could be found, and if the gaps were an

inevitable consequence of the techniques used.

Organisations are not providing a corporate framework for knowledge sharing and

this gap is being filled by knowledge workers who find they need to make knowledge

management work for them in order for them to be successful in their jobs. Some

research into how to incorporate more techniques for knowledge management into

project management methodologies could be valuable.

Another area of interest could be an investigation into whether social networking and

widely available collaborative technologies are replacing, improving or circumventing

corporate IT solutions for knowledge management.

78

References

Ackoff, R.L. (1989): ‗From Data to Wisdom‘, Journal of Applied Systems Analysis,

vol. 16.

Ambrosini, V. & Bowman, C. (2001) ‗Tacit Knowledge: Some Suggestions for

Operationalization‘, Journal of Management Studies, vol. 38, no. 6.

Anquetila, N. et al. (2006) ‗Software maintenance seen as a knowledge management

issue‘, Information and Software Technology, vol. 59, no. 5.

Basili, V.R. & Romback, H.D. (1991) ‗Support for Comprehensive reuse‘, IEEE

Software Engineering Journal, vol. 6, no. 5.

Birk, A., Dingsøyr, T. & Stålhane, T. (2002) ‗Postmortem: Never Leave a Project

without It‘, IEEE Software, May/June.

Brand, A. (1998) ‗Knowledge management and innovation at 3M‘, Journal of

Knowledge Management, vol. 2, no. 1.

Bresnen, M. et al. (2003) ‗Social practices and the management of knowledge in

project environments‘, International Journal of Project Management, 21.

Brown, J.S. et al. (2001) Storytelling: Passport to the 21st Century,

http://www2.parc.com/ops/members/brown/storytelling/Intro0-table.html [Accessed

March 24 2007]

Clarke, T. & Rollo, C. (2001) ‗Corporate initiatives in knowledge management‘,

Education + Training, vol. 43, no. 4/5.

Collier, B., DeMarcot, T. & Fearey, P. (1996) ‗A Defined Process For Project

Postmortem Review‘, IEEE Software, July.

Crossan, M.M., Lane, H.W. & White, R.E. (1999) ‗An Organizational Learning

Framework: From Intuition to Institution‘, The Academy of Management Review, vol.

24, no.3.

79

Davenport, T.H. & Prusak, L. (1998) Working knowledge: How organizations

manage what they know, Harvard Business School Press, Boston.

De Ruyter, K. (1996) ‗Focus versus nominal group interviews: a comparative

analysis‘, Marketing intelligence and planning, vol. 14, no. 6.

Dekleva, S. (1992) ‗Delphi study of software maintenance problems‘, Proc. Conf.

Software Maintenance, 9-12 Nov 1992.

Denning, S. (2001) The springboard: How storytelling ignites action in knowledge-era

organizations, Elsevier, New York.

Desouza, K.C. & Awazu, Y. (2005) ‗Segment and destroy: the missing capabilities of

knowledge management', Journal of Business Strategy, vol. 26, no. 4.

Faniel, I.M. & Majchrzak, A. (2006) ‗Innovating by accessing knowledge across

departments‘, Decision Support Systems, in press.

Graham, A.K. (2000) ‗Beyond PM 101: Lessons For Managing Large Development

Programs‘, Project Management Journal, vol. 31, no. 4.

Grant, R.M. (1996) ‗Toward A Knowledge-Based Theory Of The Firm‘, Strategic

Management Journal, 17.

Gray, A.R. & MacDonell, S.G. (1997) ‗A Comparison of Techniques for Developing

Predictive Models of Software Metrics‘, Information and Software Technology, 39.

Heath, C. & Heath D. (2007) Made to Stick: Why Some Ideas Survive and Others

Die. Random House, New York.

Henninger, S. (2001) ‗Turning Development Standards into Repositories of

Experience‘, Software Process Improvement and Practice, 6.

Iman, R.L. & Conover, W.J. (1987) ‗A Measure of Top-Down Correlation‘,

Technometrics, vol. 29, no. 3.

Kant, I. (1787) Critic of pure reason. This edition published Cambridge University

Press, 1999.

80

Kasvi, J.J.J., Vartiainen, M. & Hailikari, M. (2003) ‗Managing knowledge and

knowledge competences in projects and project organisations‘, International Journal

of Project Management, 21.

Kodama, M. (2002) ‗The promotion of strategic community management utilizing

video-based information networks‘, Business Process Management Journal, vol. 8,

no. 5.

Legendre, P. (2005) ‗Species association: the Kendall coefficient of concordance

revisited‘, J. Agricultural, Biological, and Environmental Statistics, vol. 10, no. 2.

Leibowitz, J. & Megbolugbe, I. (2003) ‗A set of frameworks to aid the project

manager in conceptualizing and implementing knowledge management initiatives‘,

International Journal of Project Management, vol. 21, no. 3.

Madhavan, R. & Grover, R. (1998) ‗From embedded knowledge to embodied

knowledge: New product development as knowledge management‘, Journal of

Marketing, vol. 62, no. 4.

Marwick, A.D. (2001) ‗Knowledge management technology‘, Knowledge

Management, vol. 40, no. 4.

Neve, T.O. (2003) ‗Right Questions to Capture Knowledge‘, Electronic Journal of

Knowledge Management, vol. 1, no. 1.

Nonaka, I. & Takeuchi, H. (1995) The knowledge-creating company, New York,

Oxford University Press.

Nonaka, I. (1991) ‗The Knowledge-Creating Company‘, Harvard Business Review,

vol. 69, no. 6.

Ojala, M. (2005) ‗Blogging for knowledge sharing, management and dissemination‘,

Business Information Review, vol. 22, no. 4.

O‘Leary, D.E. (1998) ‗Using AI in knowledge management: knowledge bases and

ontologies‘, IEEE Intelligent Systems, May/June.

81

Park, J.-H. (2006) ‗The Role Of Trust On Knowledge Creation In A Virtual

Organization: A Social Capital Perspective‘, Journal of Knowledge Management

Practice, vol. 7, no. 4.

Rising, L. & Derby, E. (2003) ‗Singing the Songs of Project Experience: Patterns and

Retrospectives‘, Cutter IT journal, vol. 16, no. 9.

Robillard, P.N. (1999) ‗The role of knowledge in software development‘,

Communications of the ACM, vol. 42, no. 1.

Sipcic, S.R. & Makonnen, Z. (1998) ‗Web-based groupware support for knowledge

creation and competitive advantage‘, Proc. Seventh IEEE Int. Workshop on Enabling

Technologies, June 1998, pp. 155–160.

Spolsky, J. (2007) Evidence Based Scheduling,

http://www.joelonsoftware.com/items/2007/10/26.html [Accessed 12 January 2008]

Tautz, C. & Althoff, K.-D. (1997) ‗Using Case-Based Reasoning for Reusing

Software Knowledge‘, IESE-Report No. 004.97/e.

Tiwana, A. (2004) ‗An empirical study of the effect of knowledge integration on

software development performance‘, Information and Software Technology, vol. 43,

no. 13.

Tsoukas, H. & Vladimirou, E. (2001) ‗What is organisational knowledge?‘, Journal of

Management Studies, vol. 38, no. 7.

Van De Ven, A.H. & Delbecq, A.L. (1974) ‗The Effectiveness of Nominal, Delphi, and

Interacting Group Decision Making Processes‘, The Academy of Management

Journal, vol. 17, no. 4.

82

Bibliography

Davenport, Thomas H. (2005) Thinking for a living: how to get better performance

and results from knowledge workers, Boston, Harvard Business School Press.

O‘Dell. C.S. et al. (1998) If only we knew what we knew: the transfer of internal

knowledge and best practice, New York, The Free Press.

Von Krogh, G et al. (2000) Enabling knowledge creation, Oxford, Oxford University

Press.

83

Index

artificial intelligence, 10

blogging, 21, 46, 58, 64, 96

collaboration, 10, 20, 62, 71

Delphi method, 26, 30, 42

explicit knowledge, 9, 11, 15, 19, 23,

26, 62, 70

groupware, 10, 21

human factors, 10, 11, 12, 25, 71

innovation, 8, 9, 11, 20, 68

issues list, 45, 51, 60

Japanese approach, 9

metaphor, 9, 66

organisational involvement, 10, 17, 28,

41, 61, 66, 70, 75

patterns, 25, 58, 59

personal interaction. See rich personal

interaction

post-project review, vii, 60, 61, 66

rich personal interaction, 25, 68, 71, 75

semi-structured interviews, 26, 58, 66

social networking, 21, 63, 77

software development, 8

storytelling, 26, 64, 65, 71

tacit knowledge, 9, 11, 13, 15, 19, 21,

22, 23, 64, 66, 76

technological solutions, 9, 20, 62

video, 10

Western approach, 9

wiki, 21, 46

84

Appendix A: Delphi method initial questionnaire results

Questionnaire from Participant A

85

Questionnaire from Participant B

86

Questionnaire from Participant C

87

Questionnaire from Participant D

88

Questionnaire from Participant E

89

Questionnaire from Participant F

90

Questionnaire from Participant G

91

Questionnaire from Participant H

92

Questionnaire from Participant I

93

Appendix B: First refinement

B.1 Material supplied to participants

Questionnaire for MSc Dissertation research

Elaine Aitken

Stage 1

Thank you for sending back your response to the questionnaire. Stage 1: gathering

initial contributions is now complete.

Stage 2

Stage 2: Collation is also complete. I have been through all the responses and have

produced new lists that combine the comments provided by multiple people. There

was quite a lot of overlap between the responses and the comments have been

reworded to cover similar answers. This means that you may not find your exact

wording in the list but should find that your views have been reflected.

Stage 3 – Refinement.

There are three lists below that show the collated responses resulting from Stage 2.

The answers are not in any particular order. I would like you to do two things with

each list:

1. Allocate points to show how relevant you think each item is to the question

You will have a set number of points to give in each list, and should award most

points to the most relevant answer, and don‘t give any points to an answer you don‘t

think is relevant at all.

Note – you may award more than one item the same mark.

94

2. Add comments to any items that you think need further explanation or refinement, or

any that you feel particularly strongly about.

For example:

Question: What fruit is most important in protecting the health of people living in the

UK?

(Please award 30 points)

Fruit Points Comment

Oranges 8 I believe this is definitely the most important because it‘s a main source of vitamin C. Also lots of people drink orange juice.

Apples 5

Bananas 5

Grapes 3

Satsumas 6 Should be combined with oranges.

Lemon 0

Grapefruit 2

Berries 1 Needs to be more specific – split into raspberries, strawberries and blackberries

What happens next?

Once I have received all the responses, I‘ll produce a ranked version of each list that

takes into account the feedback received. I‘ll then send this back out to you for

further comment and refinement.

Thank you again for your help.

95

List 1: What benefits does / could knowledge management bring to your

organisation?

Please award a total of 50 points. It is ok to award 0 points to any answer.

Points Comments

Improved consistency of information - everyone is referring to the same versions and documents

When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need

Shared workload - no single person is the sole keeper of knowledge

It encourages a culture of sharing and cooperation

New work can get off the ground more quickly

Increased quality of finished product

Increased ability of an organisation to recognise synergies and opportunities

Urgent information can be pushed to those who need to see it

Training for new starters - provides a consistent set of information and experience for them to tap into

Reduced need to reinvent the wheel for every project

Improved speed of issues resolution by making existing solutions to past problems available

The ability to share knowledge outside the business

Lets customers know what we already know about a product and reduces support calls

96

List 2: Which techniques have you used for knowledge sharing?

Please award a total of 50 points

Points

Comments

Lessons learned written up and placed in shared location at end of project

Post project reviews

Common storage area / intranet for project documents

Progress reports shared with wide audience

Ongoing issues list throughout a project

End project reports

Workshops with other project managers

Wiki

Published articles that highlight known issues and share useful information

List of unresolved issues at end of project

Daily log of conversations and phone calls

Project blogs

Defect list for technical bugs

Defect list for problems with processes

Best practice examples in a shared area

Sharing checklists

Flowcharts

Instant messaging

97

List 3: What types of knowledge have been hardest to capture and share

effectively?

Please award a total of 50 points.

Points Comments

An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?

Mistakes made early on in a project, long before a post-project review or lessons learned reports

Subjective opinions - discussions on how something should have been done or how successful a particular approach was

Cause and effect

Relationship information - who is helpful and who is not constructive

Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work

Root cause of delays and schedule slip

The solution to specific technical issues that have had to be overcome

Knowledge that will help provide more accurate estimates in future

Early warning signs

Lessons learned in other teams

Keeping information general enough to be applied in other scenarios without being too vague.

98

It can be hard to choose how much information to share to achieve a manageable quantity of information

It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.

Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.

Emails and documents can be hard to keep track of if they are being contributed to by many people.

99

B.1 Results received from participants

Table B.1. Reponses to the first refinement of List 1.

Participant A B C D E F G H

Improved consistency of information - everyone is referring to the same versions and documents

10 8 10 6 2 5 - 12

When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need

3 - 5 8 5 8 5 -

Shared workload - no single person is the sole keeper of knowledge

5 8 - 6 8 - 10 -

It encourages a culture of sharing and cooperation

3 4 - - 4 6 10 -

New work can get off the ground more quickly 5 4 5 1 3 5 10 -

Increased quality of finished product 5 - 5 1 3 3 - 2

Increased ability of an organisation to recognise synergies and opportunities

3 - - - 2 4 - 2

Urgent information can be pushed to those who need to see it

2 - 10 8 - 4 - 3

Training for new starters - provides a consistent set of information and experience for them to tap into

2 6 5 - 8 5 5 10

Reduced need to reinvent the wheel for every project

5 8 - 2 7 5 10 8

Improved speed of issues resolution by making existing solutions to past problems available

5 6 - 10 6 5 - 10

The ability to share knowledge outside the business

1 - - 1 - - - -

Lets customers know what we already know about a product and reduces support calls

1 6 10 7 2 - - 3

100

Table B.2. Reponses to the first refinement of List 2.

Participant A B C D E F G H

Lessons learned written up and placed in shared location at end of project

5 - 10 5 - 8 5 2

Post project reviews 5 6 2 5 5 1 5 3

Common storage area / intranet for project documents

10 8 5 15 10 8 5 5

Progress reports shared with wide audience 5 6 2 10 3 - - 3

Ongoing issues list throughout a project 5 6 5 2 10 2 5 2

End project reports 5 - 5 2 - 6 - -

Workshops with other project managers - 6 1 - - 5 - 4

Wiki - - 5 - 7 - 5 -

Published articles that highlight known issues and share useful information

- - - - - 5 - 3

List of unresolved issues at end of project - 6 6 1 2 - 5 -

Daily log of conversations and phone calls 5 - - - - - - -

Project blogs - - - - - 5 5 1

Defect list for technical bugs 5 8 2 10 - - 5 8

Defect list for problems with processes - - - - 8 - - -

Best practice examples in a shared area - - 5 - - 8 - 7

Sharing checklists - 4 - - - 2 5 6

Flowcharts 5 - 2 - - - 5 5

Instant messaging - - - - 5 - - 1

101

Table B.3. Reponses to the first refinement of List 3.

Participant A B C D E F G H

An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?

- 10 - - 2 - - 2

Mistakes made early on in a project, long before a post-project review or lessons learned reports

10 - - 5 2 8 - -

Subjective opinions - discussions on how something should have been done or how successful a particular approach was

5 - 5 10 1 8 - 7

Cause and effect - - - - 2 - - 6

Relationship information - who is helpful and who is not constructive

5 8 2 - 5 5 5 3

Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work

- 4 3 10 7 - - -

Root cause of delays and schedule slip 5 8 - - 9 - - -

The solution to specific technical issues that have had to be overcome

- 4 10 5 5 3 10 -

Knowledge that will help provide more accurate estimates in future

5 - - 10 11 8 10 10

Early warning signs 5 - - - 3 - - 3

Lessons learned in other teams 5 8 - - - - 5 5 Keeping information general enough to be applied in other scenarios without being too vague.

- - - 10 - 5 10 4

It can be hard to choose how much information to share to achieve a manageable quantity of information

- 4 - - 2 5 5 8

It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.

- - - - - 3 - -

Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.

5 4 - - - 5 - -

Emails and documents can be hard to keep track of if they are being contributed to by many people.

5 - 30 - - - 5 2

102

Appendix C. Second Refinement

C.1 Material sent to participants

Chapter 1 Questionnaire for MSc Dissertation research

Elaine Aitken

Thank you for sending back your completed forms with all the lists ranked. You‘ll be

pleased to hear that the hard work is complete and there is just one final stage to be

completed.

I have added up the scores for each item and ordered each list. I‘ve then produced

shortened lists with just the top-scoring 6 items.

I propose to take the top 6 items in each list forward for further analysis within my

dissertation. This analysis will include looking at research in the field of knowledge

management to see if it offers any suggestions that might be useful in addressing the

issues raised.

I would now like you to have one more look at the lists below. Please choose a

wildcard item – from the items that did not make the top 6, choose one that you think

most deserves to be considered further, and give a reason.

103

List 1: What benefits does / could knowledge management bring to your

organisation?

Benefit Elaboration from comments received

1 Improved consistency of information - everyone is referring to the same versions and documents

Saves time if someone can be pointed towards a document rather than being offered an explanation. Users need to be able to tell what version a doc is even once it has been printed out.

2 Reduced need to reinvent the wheel for every project

3 Improved speed of issues resolution by making existing solutions to past problems available

Needs appropriate filtering and keywords.

4 Training for new starters - provides a consistent set of information and experience for them to tap into

Very useful for contractors and handovers too.

5 Shared workload - no single person is the sole keeper of knowledge

Avoids disaster when someone leaves, but documentation usually fall behind what is in someone‘s head.

6 When you are aware that something similar has been done before, knowledge management makes it easy to find the information you need

Usually easier to go to the source of the information directly.

104

List 1 wildcard

Choose your wildcard from the list below by adding a reason for your selection. If

you do not think any of the items below need to be considered further, leave the

Reason column blank.

Benefit Reason for considering further

New work can get off the ground more quickly

Lets customers know what we already know about a product and reduces support calls

It encourages a culture of sharing and cooperation

Urgent information can be pushed to those who need to see it

Increased quality of finished product

Increased ability of an organisation to recognise synergies and opportunities

The ability to share knowledge outside the business

105

List 2: Which techniques are most consistently practiced on your projects?

Rank Most consistently practiced techniques

Elaboration from comments received

1 Common storage area / intranet for project documents

Good idea to storing key documents, but needs housekeeping and a good structure to ease finding documents. Invaluable on large projects.

2 Defect list for technical bugs Very useful measure of quality, remaining work, trends and pattern analysis.

3 Ongoing issues list throughout a project

Only useful if small enough to manage, and not too detailed.

4 Lessons learned written up and placed in shared location at end of project

Would be more useful if they force entry into a common database with relevant topic keywords.

5 Post project reviews Sometimes compiled and shared in meeting form. Useful to the individuals within the project team, but not used outside very much.

6 Progress reports shared with wide audience

Can be difficult to get these read by appropriate audience, perhaps because they are at the wrong level - too detailed or too high level.

106

List 2 wildcard

Choose your wildcard from the list below by adding a reason for your selection. If

you do not think any of the items below need to be considered further, leave the

Reason column blank.

Benefit Reason for considering further

List of unresolved issues at end of project

Best practice examples in a shared area

End project reports

Flowcharts

Sharing checklists

Wiki

Workshops with other project managers

Project blogs

Published articles that highlight known issues and share useful information

Instant messaging

Defect list for problems with processes

Daily log of conversations and phone calls

107

List 3: What types of knowledge have been hardest to capture and share

effectively?

Rank Hardest to share knowledge types

Elaboration from comments received

1 Knowledge that will help provide more accurate estimates in future

This is an area that is usually poorly recorded:

- Resources too busy to focus on introspection.

- Time recorded poorly, sometimes entered weeks after the event.

- Unwillingness to be open and honest with themselves and others.

- Difficulties breaking down the tasks and including the intangible work e.g. thinking time.

- If work is unique, tendency to guess rather than seek equivalent parallels and use the data for predictions.

- Tendency to get stuck in to the detail rather than take a step back and discuss the work and estimates.

2 The solution to specific technical issues that have had to be overcome

It‘s so easy to spend days trying to solve a technical issue and not write it down for next time. Bug trackers that include resolutions can be helpful for this.

3 Subjective opinions - discussions on how something should have been done or how successful a particular approach was

People can be protective of their approach, so difficult to break down into the constituent parts.

4 Relationship information - who is helpful and who is not constructive

Not something that is usually written down.

5 Keeping information general enough to be applied in other scenarios without being too vague.

Information needs better structure including considered keywords, facts detailed, inferences explained and key messages summarised.

6 It can be hard to choose how much information to share to achieve a manageable quantity of information

Knowledge needs to be reviewed regularly and trimmed on occasion.

108

List 3 wildcard

Choose your wildcard from the list below by adding a reason for your selection. If

you do not think any of the items below need to be considered further, leave the

Reason column blank.

Benefit Reason for considering further

Mistakes made early on in a project, long before a post-project review or lessons learned reports

Lessons learned in other teams

Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work

Root cause of delays and schedule slip

Emails and documents can be hard to keep track of if they are being contributed to by many people.

Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.

Early warning signs

An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?

Cause and effect

It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.

109

C.2 Results

List 1 wildcard responses

Benefit Reason for considering further Votes

New work can get off the ground more quickly

A: Otherwise tend to repeat the same mistakes or forget lessons learned

1

Lets customers know what we already know about a product and reduces support calls

B: Helps reduce support calls (and hence cost) without requiring additional development effort – not always financially viable.

C: Reduces total cost to business and makes customers feel better (in a fluffy/loving sense) about the company they are dealing with as being knowledgeable and the authority on their own product.

2

It encourages a culture of sharing and cooperation

D: All of the above depends on people getting into the habit of thinking about sharing info

1

Urgent information can be pushed to those who need to see it

Increased quality of finished product

E: From the reasons above: consistency, less time re-inventing the wheel, and faster resolution can all have benefits for quality and can increase the amount of time spent improving the product rather than just getting the job done.

Increased ability of an organisation to recognise synergies and opportunities

F: Opportunities get lost because knowledge isn‘t shared effectively. People have to crack on with the next product on their list and currently do this without any reference to knowledge already gathered/lessons already learned.

1

The ability to share knowledge outside the business

110

List 2 wildcard responses

Benefit Reason for considering further Votes

List of unresolved issues at end of project

Best practice examples in a shared area

End project reports C: Summarises project and acts as good point of reference for anyone looking back. Also excellent for capturing lessons learnt, impact on the business and a summary of the project in one location.

1

Flowcharts

Sharing checklists F: We have started to do this very recently on our large secondary project and it is proving very useful.

A: Used frequently as a visual aid for discussion and helps when trying to confirm a process.

2

Wiki

Workshops with other project managers E: This isn't something I do at all (no other project managers to work with.), but it is something I think would be valuable to share experiences and problems.

1

Project blogs

Published articles that highlight known issues and share useful information

B: Essential at the end of the project to avoid support calls/escalations.

D: An easy place to find the solution to problems that have been solved before

2

Instant messaging

Defect list for problems with processes

Daily log of conversations and phone calls

111

List 3 wildcard responses

Benefit Reason for considering further Votes

Mistakes made early on in a project, long before a post-project review or lessons learned reports

B: This relies on people‘s memories. Need to track it ongoing, e.g. keep an open log all through the project and/or review at each stage boundary.

1

Lessons learned in other teams D: Systems tend to be locally based – people don‘t bring this info with them

1

Ad-hoc work, where there is an impression that documentation is not justified because of the small scale of the work

E: on ―dynamic‖ (i.e. constantly changing) software projects, there are often lots of minor ―tweaks‖, which end up being undocumented and/or arbitrary decisions made by developers.

F: We have had a number of projects carried out on an ad-hoc basis, with nothing of significance to other projects recorded. The editors who tend to work in this way tend to work in a fairly isolated way in their silos without learning from others or giving of their experience.

2

Root cause of delays and schedule slip A: This sometimes seems to be brushed under the carpet if the cause is resource from another team or people just being slow to respond and I think it is important to capture.

1

Emails and documents can be hard to keep track of if they are being contributed to by many people.

Making use of lessons learned from other projects is difficult because they are not communicated widely or not bought into at the right level.

Early warning signs

An understanding of the problems caused by too much complexity in a product specification - what constitutes simple?

Cause and effect

It's hard to share knowledge without defining the scope of the knowledge too narrowly to the extent that other teams cannot see its relevance to them.