African Journal of Business Management Vol. 5(26), pp. 10734-10746, 28 October, 2011 Available online at http://www.academicjournals.org/AJBM DOI: 10.5897/AJBM11.445 ISSN 1993-8233 ©2011 Academic Journals

Full Length Research Paper

Extracting key performance indicators (KPIs) new product development using mind map and Decision-Making Trial and Evaluation Laboratory (DEMATEL)

methods

Yu-Tien Cheng1, Han-Hsin Chou2 and Ching-Hsue Cheng1*

1Department of Information Management, National Yunlin University of Science and Technology, Yunlin, Taiwan. 2Department of Travel Management, Hsingwu Institute of Technology, New Taipei City, Taiwan.

Accepted 25 May, 2011

In an era focusing on performance management, finding the right key performance indicators (KPIs) can aid the implementation of company goals. By monitoring and managing these indicators, the effects of prevention, continual improvements, and innovations are achieved to shape the core competitiveness. This study used the Delphi method to interview 18 field experts. Through their rich experiences and ideas, the study investigated the KPIs of new product development (NPD). Through qualitative and quantitative analysis, this study provides managers with effective management suggestions, including communication and contacts, information sharing, analysis and judgment, decision-making, goal planning, improved control, and pay-for-performance. After empirical analysis, the NPD process was found to have a total of 9 constructs with 43 important in relation to KPIs. To analyze the cause and effect relationships and importance of these constructs, decision-making trial and evaluation laboratory (DEMATEL) was utilized. The order of importance was as follows: Project schedule management, operational risk management, total quality management and life cycle cost management, demonstrating that the schedules, risks, quality and costs are valued team aspects during product research and development. Therefore, the results can provide the manager to build a more complete KPIs system and reduce product development risks. Key words: Key performance indicators, new product development, Delphi method, knowledge maps, decision-making trial, evaluation laboratory

INTRODUCTION Since company product research and development is increasingly important, market competition has become very fierce. Regardless of industry, new product develop-ment consistently commands attention. The entire process depends on increasingly complex management structures. Therefore, to effectively manage various tasks of product development, effective channels must be sought to implement performance evaluation.

New product development (NPD) is a business process in which a company develops new products or concepts, encompassing the process from original conception to *Corresponding author. E-mail: ahosir8@gmail.com. Tel: 886-5-5342601 ext. 5312. Fax: 886-5-5321719.

manufacturing to finally market introduction markets (Hohenegge et al., 2007). For instance, the development of weapon systems involves not only product design, but also important indicators for enhancing national defense strength and evaluating military power. Unlike regular commodities, its research and development process is longer. Resource input and equipment output require teams to cooperate and apply project and performance management practices in order to deliver excellent quality. It used the concept of mind maps to illustrate key performance indicators (KPIs) in order to promote KPIs exploration and the effective discovery of NPD. In addition, KPIs distribution of product research and development are described in detail. The mind map expresses the KPIs of NPD domain, which assist users in discovering the direction of enterprise performance

indicators. Via online networking, KPIs mind maps provide the latest information to the NPD research and development teams, lowering transparency barriers and smoothing content distribution.

Among projects, individuals, and enterprises, the indicators of each NPD management focus are items for project responsibilities and performance assessments, guiding the project direction and goals to achieve the targets, performances, and benefits. The purposes of this study were the following: (1) Extracting KPIs of the NPD; (2) Employing mind maps to illustrate the KPIs diagram of the NPD; (3) Using the DEMATEL to analyze the KPIs cause and effect relationships of the NPD.

Based on the Delphi method, this study obtained nine constructs and 43 important and influential KPIs from the NPD important work operational situations. Furthermore, to understand the influence level and the cause-and-effect relationships such that managers and project co-workers may conduct activities, DEMATEL questionnaire design and return statistics were conducted, including communication and contacts, information sharing, analysis and judgment, decision-making, goal planning, improved control, and pay-for-performance. LITERATURE REVIEW To build the KPIs of the NPD, this section uses a literature review to define the concepts and derive relationships of the concepts. First, NPD and KPIs are introduced. Then, the literature data of the mind maps are explained. Finally, a comprehensive literature review is conducted. New product development Among competitive industries, companies usually focus on certain parts of the production process, such as design, parts and accessories production, assembly, testing, transportation, distribution and sales. Then, they engage in either vertical or horizontal cooperation, some supporting competing companies with the purpose of satisfying customer needs. Upon the product lifecycle management (PLM) stage, successful implementation of NPD must be achieved for most procedural components, including market demand, product concepts, detailed design processes, process planning, and production. It is important to maintain innovative, effective, and efficient operation (Chen et al., 2008). Souder (1988) defined a new product as a something that has never been previously owned. Regardless of existence in the markets, this definition is based on the perspective of enterprises. Kotler (1991) considered new products to encompass the following: (1) Original products; (2) Improved products; (3) Modified products; (4) New brands that are internally researched and developed.

Cheng et al. 10735

Sampson (1970) listed three characteristics of a new product: (a) Satisfying new needs, demands, or desires; (b) Satisfying customer needs better than other products; (c) Innovating products and communication. Cooper (1983) proposed seven stages of new product development: (1) Idea generation (2) Preliminary assessment (3) Review of the concept; (4) Product development; (5) Prototype testing; (6) Trial sales; (7) Product introduction. The NPD period includes 4 steps: (a) concept generation: The transformation of user feedback into conceptual descriptions; (b) product planning: the concept develop-ment, costs, appearances, and other purposes; (c) production engineering: the transformation of product targets into detailed blueprints; (d) manufacturing engine-ering: design workflow, tools/equipments and spare parts handling procedures (Liu et al., 2005). Key performance indicators To understand the enterprise performance in a highly competitive environment, enterprises pay increasing attention to appropriate performance evaluation indica-tors and implement evaluation and assessment systems for designing enterprise management strategies and ope-rational management. Performance evaluation requires indicators (Chang, 1998) and should be the indicators of advanced, forward-looking, and immediate responses. To build appropriate indicators, the work process must receive due attention. Ahmad and Dhafr (2002) believed that key performance indicators could be quantified and compared according to internal or external target stan-dards. Its values come from any process, activity data collection and calculation. Zairi (1994) mentioned that enterprise performance indicator measurement must continually be calculated and reviewed. These measure-ments include financial performance indicators, technical performance indicators, and efficiency indicators. Corbett (1998) believed that the production strategy KPIs were costs, quality, flexibility, delivery and inventory. Yeung et al. (2009) used the Delphi method to develop 7 KPIs to measure, monitor, improve and benchmark partner performance in construction projects and establish a diagram of performance measurement to identify problem areas and key success factors for good partnerships. KPIs are quantitative indicators that reflect an organiza-tion’s success factors. The choice for KPIs indicators

10736 Afr. J. Bus. Manage. varies with the organization type. Regardless of the selected KPIs, the indicators must be combined with organizational goals and quantitatively evaluated. KPIs make enterprises illuminate main department responsi-bilities for performance evaluation, providing an analytical quantitative foundation. Establishing clear and practical KPIs system is the key to good performance management.

Mind maps

Buzan (2004) pointed out that mind maps are manifestation of radioactive thinking, which comes naturally to human brains. Mind maps are like city maps. The central topics, like city centers, highlight the most important. The roads extending from city centers represent primary thinking. Meanwhile, secondary roads branching off from these roads represent secondary important thinking. Mind maps best match the human brain mechanism, effectively realizing the brain’s potential. Moreover, they can be applied to every life aspect, whether learning or thought clarification (Sun, 2007). Aydin (2009) believed that mind maps not only could be hand drawn, but also could be designed using computers. Certain software facilitates simple mind map design, featuring all types of useful illustrations and graphics. Concept maps help to organize a large amount of information, while the method itself has the functions and characteristics of conceptual mapping and thought integration. It is an assistant tool that aids learning and enhances teaching. Through visual graphics, they express thoughts, innovative ideas, and hidden know-ledge. KPIs must confirm the goals of NPD, and identify and analyze the necessary performance indicators. They provide a basis for evaluating directions, data, and facts. Meanwhile, the mind mapping method accommodates both the left and right brain hemispheres, employing a whole brain and diffusion-like way of thinking. It conceptually presents all ideas, knowledge, and learned material; in addition, it is a visual and graphical note taking skill, which can be applied to idea integration, brainstorming, free associations, creative inspirations, and effective memory. Enterprises develop visions and strategies and finally, through continuous feedback and review, achieve performance evaluation targets. This study intends to present KPIs as mind maps, that is, by integrating the thinking expression of a human mind model, words, categorical frameworks and graphics of the desired concepts are expressed. This is a new type of KPIs presentation style. Therein, KPI strategies and definitions, obtainment, distribution and influence are included so that every person may utilize KPIs of an organization to achieve better performance and have greater competitive advantages.

RESEARCH METHODS AND DESIGN

In resolving certain issues and achieving study targets, new

methods and design models were established. Concerning the study method selection, the Delphi method was used to investigate the KPIs of a weapon system NPD. Then, KPI mind maps that matched the organizational characteristics were developed, providing guidelines for performance management applications of follow-up projects. This chapter explains the research scope and framework and introduces the implementation steps of the Delphi method.

Research scope

This study focused on various KPIs investigations into the NPD of tank and armored vehicle weapons systems. Here, “weapons system” refers to weaponry with strong combat power that directs battlefield situations. The system consists of personnel, equipment (including supporting equipments), facilities, operations, file maintenance, and other software and hardware units; it is a whole system that continuously executes combat missions (Ministry of National Defense, 2009). In terms of tank and armored vehicle R&D design, two methods exist: The first is top-to-bottom that is first planning the overall maximum vehicle weight, size, and external factors, followed by the development of the engine, transmission,

and other subsystems. The other is inside-to-outside that is first selecting engines, transmissions, and other subsystems, and then determining the external size. For instance, tank mission priorities are to directly engage the frontline enemy weapons. The overall configuration is divided into weapons, power, protection, communication, electrical, suspension, transmission, and other special equipment systems.

The weapon system R&D primarily includes the transformation of

product demands, virtual ideas, and concept development, concept implementation of workable ideas and concrete presentations using concept maps, computer drawing diagrams or models. For each stage of development, design experiences and support information directly affect the final product. The entire R&D project includes the project schedule, costs, quality, risks, procurement, product integration, and team management. In addition, in the validation of test assessments, weapon R&D emphasizes system engineering, overall logistic support systems and knowledge obtainment. In total,

it involves product design, part standardization, production tech-niques, model making, assembly tools, schedule planning, testing standards, methods, process, mechanical instruments, operation and maintenance procedures, troubleshooting, weapon testing, information system design development, database setup, and other professional knowledge and experience. To ensure weapons system functioning, the NPD of this study achieved its target while taking into account demands, budgets, and obtainment factors.

Furthermore, planning, organization, leadership, and control were employed so that human resources, material resources, financial resources, technology and information satisfied the demands for product quality within the required schedule and budget. Study framework

This study used the Delphi method, extracting the KPIs of the weapon system development selected by the field experts and employed mind mapping to draw KPIs frameworks. Based on its results, the KPIs constructs were summarized and organized. Afterward, the DMATEL method conducted empirical analysis of the questionnaire design and return statistics, concluding the study. Figure 1 shows the study design flowchart.

Expert interview using the Delphi method

The Delphi method originated from a technique prediction study

Cheng et al. 10737

Questionnaire Design

Expert Selection

Open Questionnaires

Collecting and

Modifying

Questionnaires

Changing the Questionnaire Design

Obtaining KPIs

Conducting Question Surveys

Collecting Questionnaires and Performing Statistical Analysis

Reaching Consensus

No

Yes

Figure 1. Study framework and flowchart.

developed in 1944. At the time, General Arnold entrusted Theodor von Karman to forecast technology related to national defense (Michael and Erio, 1996). In 1964, Godden and Helmer of the R

and corporation published their long-term research reports, applying the Delphi method to other fields in addition to the national defense system. They primarily focused on predicting the social and economic impact of new technologies (Cornish, 1999). Until now, the Delphi method has been widely applied to government agencies, academic units, technologies, budgeting, and living quality. It has become one of the most important assistant decision tools for resource monitoring, evaluation of relevant resources and estimations of environmental impact (Kreisel, 1984; Richey et al.,

1985). The Delphi method is a team communication technique employed by experts, allowing them to systematically handle a complicated question or task. Its nature is quite easy to compre-hend, using emails or computer systems for communication. A series of questionnaires are sent to a prior selected expert group to elicit opinions (Kuo and Yu, 1999; Mendoza and Prabhu, 2000). It is considered one of the most effective methods for achieving consistent opinions and predicting events. It is also a suitable tool

for collecting opinions from a geographically dispersed group of experts incapable of face-to-face contact (Linstone and Turoff, 1975; Hung et al., 2008). The Delphi method integrates expert opinions through writing. First, an open questionnaire was used for the research topic design and sent to selected participants (usually scholars, experts and workers with an in-depth knowledge of the phenomenon; henceforth referred to as “experts.”) After the questionnaires were returned, the researcher organized the opinions into a narrative type of questionnaire and sent it to the

same set of experts for them to indicate whether or not their attitudes aligned with the proposal. Then, the expert opinions were organized into an analysis table and another round of

questionnaires were designed. Each expert was asked to fill out, supplement, or modify the questionnaires until a consistent result was reached. The actual implementation steps are as follows

(Figure 2 shows the flowchart):

Questionnaire design For the questionnaire content to thoroughly cover the investigated topic, this study collected relevant literature and organized it as the basis for the questionnaires, which the experts were then asked to proofread. Through pre-tests, suggestions for the questionnaire

design were proposed. Based on these suggestions, scholars and experts made suitable modifications. Expert selection For experts to offer predictions, they must have sufficient professional knowledge, experience and wisdom. Therefore, this

study referred to definitions of Public Construction Commission (2008) and hired experts with one of the following qualifications: (1) A current or previous public enterprise supervisor (including assistant supervisor) with 15 or more years of practical, expertise-related experience; (2) A doctorate and 7 or more years of practical, expertise-related experience; (3) A master’s degree and 10 or more years of practice, expertise-related experience; (4) A professional and technical staff with a professional diploma and 10 or more years of post-graduation practical experience.

10738 Afr. J. Bus. Manage.

Questionnaire Design

Expert Selection

Open Questionnaires

Collecting and

Modifying

Questionnaires

Changing the Questionnaire Design

Obtaining KPIs

Conducting Question Surveys

Collecting Questionnaires and Performing Statistical Analysis

Reaching Consensus

No

Yes

Figure 2. The delphi method flowchart

Based on the aforementioned qualifications, a total of 18 people were hired. Open questionnaire surveys

The first open questionnaire surveys was also organized and implemented for the relevant of the research study.

Expert opinions Opinion was also collected for the first round questionnaire data in order to carryout the research survey. Analysis table Expert opinions were organized into an analysis table and the next round of questionnaires was designed Each expert was asked to fill out the questionnaires, supplement, and then modify their answers.

Additional remarks The modified expert opinions and additional remarks were collected.

Integration of expert opinions

The expert opinions were integrated into general and consistent result. If such purpose could not be achieved, then the survey procedure, including its conduction, collection and compiling statistics was repeated to achieve consistent results. DATA ANALYSES AND RESULTS This section of research complied with previously mentioned research methodology and steps. First, open

questionnaires for experts were designed using Delphi method. Surveys were conducted and analyzed to obtain the KPIs and their dimensions. After the experts had reached an agreement, KPIs of NPD were constructed.

For the first round of questionnaires in the survey, we issued a total of 18 questionnaires, and retrieved 18 questionnaires, for a return rate of 100%. In discussing the dimensions of the KPIs, we designed an ascending assessment scale from 1 to 9: 1: Extremely disagree, 2: Strongly disagree, 3: Disagree, 4: Somewhat disagree, 5: Neutral, 6: Somewhat agree, 7: Agree, 8: Strongly agree, 9: Extremely agree. The numbers represented the degree of agreement towards the KPIs indicated by the experts. After retrieving the first round of questionnaires, the study used Microsoft Excel statistical software for data analysis. The functions included averaging, quartile deviation, median deviation, and standard deviation. The meanings of these functions are as follows: (1) “Average” refers to the average of a components score. A high average value would indicate that the experts considered this key indicator highly important. In this research we considered the “average” dimensions of KPIs to be ≧ 7 (7 is the initial number of agreement in the assessment scale). (2) “Quartile deviation” refers to the degree of dispersion among experts’ opinions. The number indicates different perspectives of experts, whose scores ranked from 25 to 75%. High “quartile deviation” would indicate that significant differences existed among experts in the group. Fahety (1979) suggested that in ascending assessment scales from 1 to 9, “quartile deviation” of ≦ 0.6 would indicate that experts had reached an accord in their opinions. (3) Median represents the mid-point in the ranking of expert’s scores. (4) “Standard deviation” represents the dispersion of the scores and differences among experts’ opinions. A high

Cheng et al. 10739 Table 1. Statistical analysis chart of NPD KPIs questionnaire.

KPI dimension KPI Average Quartile

deviation Median

Standard deviation

Project schedule management and control

Activity definition 8.33 0.50 8.00 0.65

Project schedule assessment 7.75 0.50 8.00 0.75

Schedule development 8.00 0.25 8.00 0.74

Schedule control 7.83 0.50 8.00 0.72

Life cycle cost management

Project resource plan 8.33 0.50 8.50 0.78

Cost estimating 8.00 0.25 8.00 0.74

Budgeting 7.67 0.50 8.00 0.65

Cost control 7.92 0.13 8.00 0.67

Total project quality management Quality control 8.33 0.50 8.50 0.78

Perform quality assurance 8.00 0.25 8.00 0.74

Quality auditing 8.33 0.50 8.50 0.78

Quality improvement 7.42 0.13 7.00 0.79

Quality control 8.00 0.25 8.00 0.74

Operational risk management

Risk planning 8.00 0.25 8.00 0.74

Risk identification 8.33 0.50 8.00 0.65

Risk analysis 8.17 0.13 8.00 0.58

Monitor and control risks 8.17 0.50 8.00 0.72

Procurement management

Procurement plan 8.17 0.50 8.00 0.72

Advertising and contracting 7.33 0.13 7.00 0.65

Contractor and contract management 8.33 0.50 8.00 0.65

Contractor-performance management 8.67 0.13 9.00 0.65

Administrative closure 8.50 0.50 9.00 0.67

Establishment of product integrated

team

Human resource plan 7.83 0.50 8.00 0.72

Establishment and development of integrated team

8.33 0.50 8.00 0.65

Management of integrated team 7.42 0.13 7.00 0.79

Target management 8.08 0.13 8.00 0.67

Equipment systems engineering

Requirement analysis 8.17 0.50 8.00 0.72

System analysis and management 8.25 0.13 8.00 0.45

Function analysis and collocation 7.75 0.50 8.00 0.62

System unit integration and plan 7.92 0.13 8.00 0.67

Tailor studies 8.00 0.25 8.00 0.74

Configuration management 8.58 0.13 9.00 0.79

Development of integrated logistics support

Technical menu compilation 8.00 0.25 8.00 0.74

Analysis of materials supply plan 8.25 0.50 8.00 0.62

Personnel training 8.33 0.50 8.00 0.65

Reliability engineering 8.50 0.50 9.00 0.67

Logistics and maintenance planning 8.17 0.13 8.00 0.58

Management of vanishing business resources

7.75 0.50 8.00 0.75

Verification of test and evaluation

Effectiveness assessment 8.67 0.13 9.00 0.65

10740 Afr. J. Bus. Manage. Table 1. Contd.

Suitability assessment 7.67 0.50 7.50 0.78

Operational measurement 8.33 0.50 8.00 0.65

Performance measurement 8.67 0.13 9.00 0.65

Availability analysis 7.58 0.50 7.00 0.79

value would indicate that a significant difference existed among some experts.

Results from analysis of the data collected in the first round of questionnaires were as follows: The KPIs related to 10 dimensions including “schedule auditing,” “risk assessment,” “risk management,” “factory manage-ment,” “file management,” “personnel selection,” “opera-tional objectives and strategic decision,” “equipment construction project support,” “teaching aids and tools,” and “the importation of consultation and support services.” The average <7 and quartile deviation ≧ 0.6 indicated that experts’ disagreement concerning the dimensions of KPIs. The opinions of the experts’ were not in agreement; we decided to delete the above dimen-sions in the second round of questionnaires. Another group of experts proposed that 15 KPIs including “activity definition”, “risk identification”, “procurement plans,” “advertising and contracting,” “contractor-performance management,” “establishment and development of integrated team,” “configuration management,” “reliability engineering,” “logistics and maintenance planning,” “management of vanishing business resources,” “measurement of effectiveness and efficiency,” “measure-ment of adaptability,” “operational measurement,” “performance measurement,” and “availability analysis,” were key components of NPD; all of which should be included in further discussions. Therefore, the above dimensions were incorporated in the second round of questionnaires in the survey. For the second round of questionnaires, we designed an attachment of data analysis charts from the first round of surveys. Each of the 18 questionnaires was supplemented, amended, issued, and filled out by an expert. The study retrieved 18 responses, for a return rate of 100%. After gathering the information, the study used Microsoft Excel statistics software for the statistical analysis of average, quartile deviation, median, and standard deviation. The experts managed to reach a consensus of opinions. Table 1 illustrates the statistics for the second round of questionnaires. KPIs mind map In this study we generated a knowledge map using the graphical techniques of mind mapping developed by Tony Buzan, president of Brain Foundation in England. Mind mapping is a whole-brain learning method that transforms thoughts into keywords and images. This

activates both the left and right hemispheres of the brain, which makes it possible for visual images, ideas and concepts to emerge (Wycoff, 1991). Hanf (1971) describes mind mapping as a way of rendering graphic images during ones’ mental thought processes. Learning to use mind maps is quite simple, and is easily grasped and accepted by the public. In addition to releasing potential brainpower, mind mapping facilitates memorization and enhances the clarity of thinking in ones’ study, work and life in general (Buzan, 2005). Mind Manager 7 is a typical software utilizing current mind mapping methods (Mindjet, 2009). It is a visual organization of the thinking process and a tool for special project planning. It allows users to compose, display and illustrate ideas and outlines. Mind mapping can be instrumental in enhancing the efficiency and productivity of an enterprise. This software is based on prominent mind mapping methods, enabling users to generate, organize, and disseminate information quickly and easily. KPIs mind map presents information and relation-ships through visualization. Visualization is a process of rendering an image, following the analysis of information. Such graphical presentations offer a number of benefits, such as improving learning speed and long-term memory retention. In the case of complex information or a nume-rous documents, an entire image can be divided into smaller parts through techniques such as classification and clustering, which facilitates easier control of system (Wang, 2002). In the field of artificial intelligence, some scholars use self-organized image maps, consisting of la-yered graphics to distinguish and integrate concepts. This study used the mind mapping method to transform the KPIs from Table 1 to images in Figure 3. This drawing process is as follows: (1) First, we designated the main theme as “The Mind Map of NPD KPIs”, and established it as the central node of all ideas. We began drawing from the center outward. (2) From the central node, we drew radiating lines, which represented the nine sub-nodal branches including “project schedule control,” “life-cycle cost management,” “total project quality management,” “operational risk management,” “procurement manage-ment,” “establishment of product integration team,” “equipment systems engineering,” “establishment of integrated logistic support system,” and “test and evalua-tion verification.” The main branches grew thicker from the center outward, connected to other joint branches. (3) We extended the lines from the nine sub-nodes outward to the next level of KPIs, creating a joint-node hierarchical and radiating structure. Take “project schedule control”

Cheng et al. 10741

Figure 3. The mind map of NPD KPIs

as an example. We extended it to the next level, consisting of “activity definition,” “activity schedule estimating,” “schedule development,” and “schedule control.” (4) We made use of colors, graphics, and layers to indicate the relationship between each branch. The study created the mind map of KPIs using mind manager software developed by Mindjet Corporation. Through built-in tools within the software package, we were able to publish knowledge based websites as illustrated in Figures 4 and 5. This feature transfers information from the mind maps to websites, allowing users to browse and share the latest information. Colleagues involved in special projects are able to browse the content of each KPI using internet or intranet; which improves communi-cation, and connects individual managerial systems within an organization, to create a more structured information system.

A mind map of NPD KPIs website is a unique expression of the World Wide Web. Harmeri and Puittinen (2003) proposed that the web added value to

the project delivery process, thanks to the versatility of the Internet as a global communication medium. The Internet can seamlessly integrate different stages of product life cycles, which facilitates execution of special projects through the application of information systems. The true potential of the web can be seen when we consider how it simplifies business processes, securely transferring vast amounts of business information to whomever needs it, whenever and wherever they need it. The Internet can immediately provide NPD Brand team with the newest information, while ensuring the accuracy of KPIs information sharing and expansion effectiveness DEMATEL method in empirical analysis This study integrated all of the opinions of the experts from the Delphi method questionnaires, to create the mind map of KPIs, as shown in Figure 4. Due to the diverse nature and inter-connectedness of KPIs, we were

10742 Afr. J. Bus. Manage.

Figure 4. The mind map of NPD KPIs website

not able to conclude with a definitive and detailed set of cause-and-effect relationships. We used DEMATEL method and the 9 dimensions of KPIs to design the questionnaires. The 18 experts chosen during the first round of surveys returned all 18 of the questionnaires, for a return rate of 100%. We follow with the origins, conceptual calculations, and results of the empirical analysis using DEMATEL. The origin and concept of DEMATEL method DEMATEL method was developed for science and human affairs programs in Battelle Memorial Institute of Geneva between 1972 and 1976. The intention was for it to solve complex issues, to enhance understanding of particular problems, to classify intertwined issues, and to identify feasible options by restructuring problems according to a hierarchical structure (Tzeng et al., 2007). Calculation steps for Demotes method The process of DEMATEL method is as follows (Hu et al.,

2009): Step1: Define quality and establish a scale of measurement Set a standard with which to evaluate the level of impact for various criteria. Define and divide the levels according to the degree of impact and designate these as 0, 1, 2, 3, 4. The numbers differentiate levels of impact, which includes “0: no impact,” “1: Low impact,” “2: Average impact,” “3: High impact,” and “4: Extremely high impact.” Step 2: Establish direct-relation matrix This matrix was based on questionnaire responses from “evaluators”, who had established two criteria with which to determine the level of impact. They matched to the scales we created in Step 1, and in so doing generated a direct-relation table. We incorporated the results from each evaluator to produce a direct-relation matrix (V), in

which n represents the index number, ｖij represents the

Cheng et al. 10743

Figure 5. The mind map of NPD KPIs website.

level of impact of Criteria Ci on Criteria Cj after the incorporating experts’ opinions. Some of the diagonal values were set as 0.

n

nn

n

n

n

CCC

VV

VV

VV

C

C

C

V

21

21

221

112

2

1

0

0

0

0

Step 3: Normalize direct-relation matrix After standardizing the direct-relation matrix V from Step 2, the study established a standardized-relation matrix. The standardization equation is as follows:

n

i

ijnj

n

j

ijni

vv1

11

1max,max

1

then the standardized-relation matrix X = λ × V is calculated. Step 4: Calculation of direct/indirect relation matrix (T) Once the standardized relation matrix X is obtained, it is converted into direct/indirect relationship matrix T by equation (1)

1

1)(k

k

ijXIXXtT

（1）

Where I is the unit matrix, and tij indicates criterion Ci’s level of impact on criterion Cj.

n

nnnn

n

n

n

CCC

ttt

ttt

ttt

C

C

C

T

21

21

22221

11211

2

1

10744 Afr. J. Bus. Manage.

Table 2. Direct/indirect relationship matrix for NPD KPIs.

Notation C1 C2 C3 C4 C5 C6 C7 C8 C9

C1 - 0.4661* 0.4962* 0.4612* 0.4568* 0.4283* 0.4501* 0.4466* 0.4560*

C2 0.3777* - 0.4002* 0.3769* 0.3808* 0.3772* 0.3652 0.3673 0.3886*

C3 0.3533 0.3470 - 0.3849* 0.3775 0.3664 0.3318 0.3403 0.3606

C4 0.4458* 0.4548* 0.4845* - 0.4233* 0.4089* 0.4304* 0.4352* 0.4416*

C5 0.3045 0.3309 0.3340 0.3221 - 0.3211 0.3259 0.3365 0.3424

C6 0.3090 0.3037 0.3230 0.2899 0.2906 - 0.3048 0.2980 0.3001

C7 0.2774 0.2747 0.3039 0.2835 0.2901 0.2595 - 0.2668 0.2710

C8 0.3219 0.3370 0.3549 0.3397 0.3321 0.3267 0.3257 - 0.3397

C9 0.2874 0.2962 0.3265 0.3020 0.3087 0.2956 0.2912 0.2850 -

* indicates that the value is greater than the threshold 0.3678 (the total average).

Step 5: Calculate the total degree of influence provided and received for each criteria We summed up each row and column of direct/indirect relation matrix T, to establish the value of D using Formula (2), as well as the sum of every row and the value of R using Formula (3). D value represents this degree of direct/indirect impact for each of the criteria on other criteria, where the R value represents the degree of impact other criteria’s on this criterion

n

j

iji tD1 ， i=1,2,…….,n

n

ijij tR

1 ，j=1,2,…….,n

Step 6: Calculate the degree of relatedness and impact Take the value of D and R from Step 5, to calculate D+R, D-R, in which D+R represents degree of relatedness between criteria, whereas D-R represent degree of impact as the criteria influence other criteria or are influenced by other criteria. Graph the value of D+R and D-R for different criteria, and analyze the causal impact and relationship among each. Create a cause-and-effect graph using degree of impact, degree of relation and threshold value for each of the criteria. Questionnaire data analysis In analyzing the data gathered from DEMATEL questionnaires, using expert opinions we built a direct-relation matrix on the original impact of each dimension. We used Microsoft EXCEL software to standardize the direct-relation matrix in order to generate a total impact relation matrix, calculate statistics in rows and columns

and arrive at the correlation between the causes D and the effects R. Coding of each KPIs dimension was as follows: “C1: Project schedule control,” “C2: Life-cycle cost management,” “C3: Total project quality manage-ment,” “C4Op: operational risk management,” “C5: Procurement management,” “C6: Establishment of product integration team,” “C7: equipment systems engineering,” “C8: Development of integrated logistics support,” and “C9: Testing evaluation verification.” A direct/ indirect relation matrix pertaining to NPD is shown in Table 2. The threshold value was the total average of relationship matrix, 0.3678. We employed DEMATEL method to study construction and calculation processes in Step 5. We summed up the value of total impact for each row and column (direct/indirect) for relationship matrix “T,” to determine the total influence of dimensions

iD and the total effect felt by dimensions jR. We

added up the sum of the row and the sum of the column (D+R), which indicated the strength of correlation between this dimension and other key dimensions. We subtracted the sum of column from the sum of row (D-R) to determine the degree of impact on other dimensions, or felt by other dimensions, as illustrated in Table 3. We used the sum of row and column (D+R) and difference between row and column (D-R) calculated in Table 3 as raw data. We crisscrossed the X-axis (D+R) and Y-axis (D-R), and plotted the key coordinates of the dimension in Figure 6, to illustrate cause-and-effect. Finally, we added up the sum of the rows and the sum of the columns to determine the (D+R). The higher the value of (D+R), the more important this dimension would be among all assessed dimensions. From among the dimensions, we chose the D+R value that exceeded the threshold value (the total average of 6.1822), to find four key NPD KPIs dimensions in order of importance as, “C1: special project schedule control,” “C4: Operational risk management,” “C3: Total project quality management,” and “C2: Life-cycle cost management.” For each dimension, we subtracted the sum of columns from the sum of rows to get the value of D-R. A high positive value

Cheng et al. 10745

Table 3. Sum and differences of rows and column.

KPIs dimension D R D+R D-R

Project schedule control (C1) 3.9982 3.0139 7.0121* 0.9843

Life-cycle cost control (C2) 3.3213 3.0978 6.4191* 0.2235

Total quality control (C3) 3.1545 3.3160 6.4706* -0.1615

Operational risk control (C4) 3.8603 3.0959 6.9562* 0.7644

Procurement management (C5) 2.8651 3.1077 5.9728 -0.2426

Establishment of product integration team (C6) 2.6409 3.0054 5.6463 -0.3645

Equipment systems engineering (C7) 2.4330 3.0312 5.4643 -0.5982

Development of integrated logistics support (C8) 2.9244 3.0224 5.9469 -0.0980

Testing and evaluation verification (C9) 2.6216 3.1290 5.7506 -0.5074

-1.00

-0.70

-0.40

-0.10

0.20

0.50

0.80

1.10

1.40

1.70

2.00

0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00D+R

D-R

Figure 6. Causal diagram for NPD KPIs.

indicated that this dimension had a huge direct influence on the other dimensions. A high negative value indicated that the dimension was heavily influenced by the other dimensions. Ranked in order of this value, “C1: Project schedule control” (positive maximum) was the most important dimension affecting other dimensions, while “C7: Equipment systems engineering” (the largest negative value) was the dimensions most heavily affected by the others. Conclusion This paper has extracted dimensions of KPIs using Delphi method, and generated a map of KPIs using mind mapping techniques. The map illustrates many concepts and relationships of KPIs contributed by experts from many fields. The study creates KPIs structures that are consistent with NPD requirements; thus, allowing users to understand KPIs of NPD from different dimensions

through an animated and interesting learning process. KPIs in weapon NPD will enable performance of

integrated weapon system life-cycles. The results can bring the following three benefits: (1) The graph of KPIs was constructed with a performance indicator, designed by experts, to facilitate quick and accurate understanding of the origin of the desired KPIs. It can take on the leading role in performance management by converting knowledge to wisdom and make a positive contribution to the decision making process of CEOs of management teams. (2) The mind map of KPIs makes it possible for staff in different fields to understand one another better. It lays the foundation for performance enhancement and facilitates innovation. The KPIs Mind mapping can also combine various stages of KPIs during the entire weapon development life cycle, thereby promoting cooperation and unity among staff. (3) The KPIs mind map websites allow the active sharing of KPIs and promotes trans-parency of KPIs. It provides a reference for other depart-ments through constant exchange and dissemination

10746 Afr. J. Bus. Manage. of information.

DEMATEL is employed to study the construction and calculation of steps. From empirical analysis, the KPIs dimension of NPD concluded with four dimensions ranked in order of importance; “special project schedule control,” “operational risk management,” “total project quality management,” and “life-cycle cost management.” It is critical therefore, that all levels of staff pay special attention to the dimensions of schedule, risk, quality and costs. We discovered that “project schedule control” was the key dimension influencing other dimensions, while “equipment systems engineering” was the key dimension influenced by others. The cause-and-effect graph and correlation graph indicated the degree of correlation among each of the dimensions of KPIs. With the knowledge of the key dimensions of KPIs, enterprises can consider how to collect, organize, extract, share, disseminate, recreate, and integrate the knowledge into core values controlling the operation of the organization. The radiating shape and hierarchical structure of KPIs' mind maps provide flexibility during the creation process and focuses clearly on important information through layered graphics. With an effective display of dimensions, a KPI mind map can be an auxiliary tool for incoming staff, researchers, and even the general public to learn about their research field and share information of the KPIs. The corporate world can also apply methods proposed in this study to construct graphs of KPIs for internal organization or external business domains. REFERENCES

Ahmad MM, Dhafr N (2002). Establishing and improving manufacturing performance measures. Robot. Comput. Integr. Manuf., 18: 171-176.

Aydin G., Balim AG (2009). Technologically-supported mind and

concept maps prepared by students on the subjects of the unit systems in our body. Procedia Soc. Behav. Sci., 1: 2838–2842.

Buzan T (2004). Mind maps at Work, Thorsons.

Buzan T (2005). The ultimate book of Mind Maps®. Thorsons, UK. Chang AS (1998). Development of Consultant Performance Measures

for Design Projects. Proj. Manage. J., 29: 39-54.

Chen HH, Kang HY, Xing X, Lee AHI, Tong Y (2008). Developing new products with knowledge management methods and process development management in a network. Comput. Ind., 59 (2-3): 242-

253. Corbett ML (1998). Benchmarking manufacturing performance in

Australia and New Zealand. Benchmarking, 5(4): 271-82.

Cornish E (1999). The study of the future: An Introduction to the Art and Science of Understanding and Shaping Tomorrow's World. Washington, DC, World Future Society.

Department of National Defense (2009). National Army Major Weapon

and Equipment System Engineering Rules and Regulation. Taipei: Armaments Bureau.

Hameri AP, Puittinen R (2003). WWW-enabled knowledge management for distributed engineering projects. Comput. Ind., 50 (2): 165-177.

Hanf MB (1971). Mapping: A Technique for Translating Reading into

Thinking. J. Reading, 225-230. Hohenegger J, Bufardi A, Xirouchakis P (2007). A new concept of

compatibility structure in new product development. Adv. Eng.

Inform., 21: 101-116. Hu HY, Lee YC, Yen TM, Tsai CH (2009). Using BPNN and DEMATEL

to modify importance–performance analysis model–A study of the

computer industry. Expert Syst. Appl., 36 (6): 9969-9979. Hung HL, James WA, Lee YF (2008). Methodological and conceptual

issues confronting a cross-country Delphi study of educational

program evaluation. Eval. Program Plann., 31: 191-198. Kreisel WE (1984). Representation of the Environmental Quality Profile

of a Metropolitan Area. Environ. Monitor. Assess., 4: 15–33.

Kuo NW, Yu YH (1999). Policy and Practice: An Evaluation System for National Park Selection in Taiwan. J. Environ. Plan. Manage., 42 (5): 735-745.

Linstone HA, Turoff M (1975). The Delphi method: Techniques and applications. MA: Addison Wesley Publishing.

Mendoza GA, Prabhu R (2000). Development of a Methodology for

Selecting Criteria and Indicators of Sustainable Forest Management: A Case Study of Participatory Assessment. Environ. Manage., 26: 659-673.

Michael A, Erio Z (1996). Gazing into the Oracle The Delphi Method and its Application to Social Policy and Public Health. Jessica Kingsley Publishers.

Mindjet (2009). Software for Visualizing and Using Information, Retrieved on 11th April from: http://www.mindjet.com.

Public Construction Commission (2008). Experts and scholars

suggested removing list of database and omitting key operational items. Taipei: Public Construction Commission.

Richey JS, Mar BW, Horner RR (1985). The Delphi Technique in

Environmental Assessment Implementation and Effectiveness. J. Environ. Manage., 21 (2): 135-146.

Sun YX (2007). Mind Mapping Bible, Taipei: Yale International Cultural

Publishing. Tzeng GH, Chiang CH, Li CW (2007). Evaluating Intertwined Effects in

e-Learning Programs: A Novel Hybrid MCDM Model Based on Factor

Analysis and DEMATEL. Expert Syst. Appl., 32 (4): 1028-1044. Wang SH (2002). Knowledge maps for managing Web-based business.

Ind. Manage. Data Syst., 102: 357-364.

Wycoff J (1991). Mind Mapping. New York: Berkley. Yeung JFY, Chan APC, Chan DWM (2009). A computerized model for

measuring and benchmarking the partnering performance of construction projects. Automat. Construct., 18: 1099-1113.

Zairi M (1994). Benchmarking the best tool for measuring competitiveness. Benchmarking, 1(1): 11-24.