a performance management system for the choose approach … · 2014-12-19 · a performance...
TRANSCRIPT
GHENT UNIVERSITY
FACULTY OF ECONOMICS AND BUSINESS
ADMINISTRATION
ACADEMIC YEAR 2013– 2014
A Performance Management System for the
CHOOSE Approach for Enterprise
Architecture for Small and Medium-Sized
Enterprises
Master thesis presented in order to acquire the degree of Master of Science in Applied Economics: Business Engineering
Lara Moons under the management of
Maxime Bernaert and Prof. Dr. Poels
GHENT UNIVERSITY
FACULTY OF ECONOMICS AND BUSINESS
ADMINISTRATION
ACADEMIC YEAR 2013– 2014
A Performance Management System for the
CHOOSE Approach for Enterprise
Architecture for Small and Medium-Sized
Enterprises
Master thesis presented in order to acquire the degree of
Master of Science in Applied Economics: Business Engineering
Lara Moons under the management of
Maxime Bernaert and Prof. Dr. Poels
PERMISSION
The undersigned states that the content of this master thesis may be consulted and/or
reproduced, if cited.
Lara Moons
I
PREFACE
This master thesis provides the last keystone to acquire the degree of Master of Science in
Applied Economics: Business Engineering at Ghent University, Belgium. The completion of this
work would not have been possible without the support and assistance of several persons. I
would like to take the opportunity to thank those people.
First of all, I would like to thank Mr. Maxime Bernaert and Prof. Dr. Geert Poels of Ghent
University. They gave me the opportunity to establish a master thesis tailored to my interests,
guided me through the process of writing this thesis and provided me with constructive criticisms
and feedback.
Furthermore, I would like to thank my parents for the great opportunity to complete my studies
and for the support during the completion of this master thesis. A special word of thanks goes to
my father, Jan Moons, who proofread my master thesis and made it possible to perform a case
study research at VMD NV. Furthermore, I would also like to thank my friend Noémie Renaerts
for proofreading this master thesis.
Finally, I wish to thank Ms. Kristel Walschots, Mr. Luc Jacobs, Mr. Peter Manet and my father for
the enjoyable collaboration during the implementation and evaluation of the CHOOSE approach.
In addition, I thank Mr. Peter Manet and Ms. Sofie Deboiserie for allowing me to participate in a
session about performance management of the MBA Highlights. Last but not least, I thank
performance management expert Mr. Mark Van Gastel for his evaluation of the CHOOSE model.
II
TABLE OF CONTENTS
Dutch Abstract ............................................................................................................................. 1
Introduction ................................................................................................................................. 2
PART I: Literature review ............................................................................................................ 4
Introduction Literature Review ................................................................................................. 4
Chapter I: Small and Medium-sized Enterprises ...................................................................... 5
Chapter II: Enterprise Architecture .......................................................................................... 7
2.1 Definition of Enterprise Architecture ........................................................................... 7
2.2 Positioning of EA: information management .............................................................. 7
2.3 Different architectural layers and views ...................................................................... 8
2.3.1 TOGAF ............................................................................................................. 10
2.4 Ingredients of EA ..................................................................................................... 11
2.5 Advantages of EA .................................................................................................... 12
2.6 EA for SMEs: the CHOOSE Approach ..................................................................... 13
2.6.1 CHOOSE Metamodel ....................................................................................... 14
2.6.2 CHOOSE Method ............................................................................................. 15
2.6.3 CHOOSE Tool Support ..................................................................................... 15
Chapter III: Performance Management ................................................................................. 16
3.1 Performance Measurement and Performance Management .................................... 16
3.1.1 Performance Measurement (System) ............................................................... 16
3.1.2 Performance Management (System) ................................................................ 17
3.1.3 Advantages of Performance Management (Systems) in SMEs ......................... 20
3.2 Goals ....................................................................................................................... 21
3.2.1 Vision, Mission and Values ............................................................................... 21
3.2.2 Critical Success Factors ................................................................................... 22
3.2.3 Strategies and Plans ......................................................................................... 22
III
3.2.4 Organization Structure ...................................................................................... 23
3.2.5 Goals and Organization Structure in SMEs ....................................................... 24
3.3 Performance Measurement...................................................................................... 24
3.3.1 Key Performance Indicators .............................................................................. 24
3.3.1.1 Concept and Advantages .............................................................................. 24
3.3.1.2 Characteristics .............................................................................................. 26
3.3.1.3 Attributes ....................................................................................................... 27
3.3.1.4 Classification ................................................................................................. 27
3.3.1.5 Development and Use ................................................................................... 29
3.3.1.6 Alignment of KPIs on different levels: the cascading concept ........................ 30
3.3.1.7 Number of KPIs on different levels ................................................................ 32
3.3.2 Performance Measurement System .................................................................. 32
3.3.2.1 The Performance Measurement Matrix ......................................................... 33
3.3.2.2 The Strategic Measurement And Reporting Technique ................................. 33
3.3.2.3 The Result and Determinants Framework ..................................................... 33
3.3.2.4 The Balanced Scorecard ............................................................................... 33
3.3.2.5 Strategy Maps ............................................................................................... 36
3.3.2.6 The Performance Prism ................................................................................ 37
3.3.2.7 Organizational Performance Measurement model ......................................... 37
3.3.2.8 Integrated Performance Measurement model for Small Firms ....................... 37
3.3.2.9 Performance Measurement System in SMEs ................................................ 37
3.4 Target, Evaluation and Reward practices................................................................. 38
3.4.1 Target Setting ................................................................................................... 38
3.4.2 Performance Evaluation.................................................................................... 39
3.4.3 Reward Systems .............................................................................................. 40
3.5 Implementation of the PMAS ................................................................................... 40
3.5.1 Information Flows, Systems, and Networks ...................................................... 40
IV
3.5.2 PMASs Use ...................................................................................................... 41
3.5.3 PMASs Change ................................................................................................ 41
3.5.4 Strength and Coherence ................................................................................... 42
Chapter IV: Links between EA and PM ................................................................................. 43
4.1 Relationship between KPI-related concepts and EA elements ................................. 43
4.2 EA Metamodel of Braun and Winter ......................................................................... 44
4.3 Unified Business Strategy Metamodel of Giannoulis et al. ....................................... 47
4.4 Gaps of Research .................................................................................................... 49
PART II: Problem Statement ..................................................................................................... 50
PART III: Solution ...................................................................................................................... 52
Introduction ........................................................................................................................... 52
Chapter I: Initial Solution after Literature Review ................................................................... 55
1.1 CHOOSE Metamodel............................................................................................... 55
1.2 CHOOSE Method .................................................................................................... 60
1.3 Extra CHOOSE Constraints ..................................................................................... 62
Chapter II: Case Study as Action Research .......................................................................... 63
2.1 Methodology ............................................................................................................ 63
2.1.1 Design Science ................................................................................................. 63
2.1.2 Action Research ............................................................................................... 64
2.1.3 Case Study Research ....................................................................................... 66
2.2 Case Study Research as Action Research .............................................................. 68
2.2.1 Description Case Study .................................................................................... 68
2.2.2 Case Study Protocol and Database .................................................................. 68
2.2.3 Development .................................................................................................... 69
2.2.3.1 Interview 1 ..................................................................................................... 70
2.2.3.2 Interview 2 ..................................................................................................... 71
2.2.3.3 Interview 3 ..................................................................................................... 72
V
2.2.3.4 Interview 4 ..................................................................................................... 73
2.2.3.5 Interview 5 ..................................................................................................... 74
2.2.3.6 Interview 6 ..................................................................................................... 76
2.2.3.7 Interview 7 ..................................................................................................... 77
2.2.3.8 Interview 8 ..................................................................................................... 80
2.2.3.9 Interview 9 ..................................................................................................... 83
2.2.3.10 Interview 10, 11 and 12 ................................................................................ 84
Chapter III: Final Solution ...................................................................................................... 86
3.1 Final CHOOSE metamodel ...................................................................................... 86
3.2 Final Extra CHOOSE constraints ............................................................................. 86
3.3 Final CHOOSE method ............................................................................................ 87
PART IV: Evaluation .................................................................................................................. 90
Introduction ........................................................................................................................... 90
Chapter I: Evaluation models ................................................................................................ 90
1.1 Method Evaluation Model ........................................................................................ 90
1.2 User Evaluations Based Quality Model for Conceptual Models ................................ 93
1.3 CHOOSE criteria ..................................................................................................... 95
Chapter II: Evaluation Results ............................................................................................... 97
2.1 Evaluation by the SME ............................................................................................. 97
2.1.1 General Evaluation ........................................................................................... 97
2.1.2 Method Evaluation ............................................................................................ 99
2.1.3 Model Evaluation .............................................................................................. 99
2.1.4 General Evaluation of the Existing CHOOSE Approach .................................. 100
2.2 Evaluation by an expert ......................................................................................... 100
PART V: Conclusion ................................................................................................................ 102
General Conclusion ............................................................................................................. 102
Limitations ........................................................................................................................... 103
VI
Future Research ................................................................................................................. 104
Bibliography ................................................................................................................................. I
Appendices ............................................................................................................................... XI
Appendix 1 ............................................................................................................................ XI
Appendix 2 .......................................................................................................................... XIII
Appendix 3 .......................................................................................................................... XVI
Appendix 4 ........................................................................................................................ XVIII
Appendix 5 ........................................................................................................................ XXIV
Appendix 6 ...................................................................................................................... XXVIII
Appendix 7 ...................................................................................................................... XXXIII
VII
ABBREVIATIONS
AIM Amsterdam Information Management model
ADM Architecture Development Method
ARIS Architecture of Integrated Information Systems
BSC Balanced Score Card
CAIRO Consulted, Accountable, Informed, Responsible, and Out of the loop
CEO Chief Executive Officer
CHOOSE “keep Control, by means of a Holistic Overview, based on Objectives and kept
Simple of your Enterprise”
CSF Critical Success Factor
EA Enterprise Architecture
HR Human Resources
ICT Information and Communication Technology
IS Information System
IT Information Technology
KPI Key Performance Indicator
KRI Key Result Indicator
MEM Method Evaluation Model
OPM Organizational Performance Measurement
PEOU Perceived Ease Of Understanding
PI Performance Indicator
PM Performance Management
PMAS Performance Management System
PMES Performance Measurement System
PSQ Perceived Semantic Quality
VIII
PU Perceived Usefulness
RACI Responsible, Accountable, Consulted, and Informed
RASCI Responsible, Accountable, Supportive, Consulted, and Informed
RI Result Indicator
SAM Strategic Alignment Model
SMART Strategic Measurement and Reporting Technique (Cross & Lynch, 1991)
SMART Specific, Measurable, Attainable, Realistic, and Time-Bound (Doran, 1981)
SMARTER Specific, Measurable, Attainable, Realistic, Time-Bound, Evaluated, and
Reviewed
SMBSC Strategy Maps and Balanced Score Card
SME Small and Medium-sized Enterprise
SMT Senior Management Team
SC Supply Chain
TAM Technology Acceptance Model
TOGAF The Open Group Architecture Framework
UBSMM Unified Business Strategy Meta-Model
UEBQM User Evaluations Based Quality Model
UML Unified Modelling Language
US User Satisfaction
VC Value Configuration
IX
LIST OF FIGURES
Figure 1: Positioning of EA in AIM (adapted from (R. Maes, 2007)) ............................................. 8
Figure 2: EA with more focus on the four core dimensions of the business architecture (from
(Bernaert et al., 2014)) ................................................................................................................ 9
Figure 3: Content metamodel and Motivation & Governance extension of TOGAF (from (The
Open Group, 2011)) .................................................................................................................. 10
Figure 4: Bow-tie model of EA (adapted from (Tavernier & Poels, 2012)) .................................. 11
Figure 5: Overview evaluation criteria for an EA technique for SMEs (adapted from (Ingelbeen,
Bernaert, & Poels, 2013)) .......................................................................................................... 13
Figure 6: Structure of the four integrated models (from (Bernaert & Poels, 2011a)) ................... 14
Figure 7: Current CHOOSE metamodel (from (Bernaert, Callaert, Poels, et al., 2013)) ............. 15
Figure 8: The PM Cycle (from (Eckerson, 2009)) ....................................................................... 17
Figure 9: Performance Management and Measurement are closely intertwined (adapted from
(Lebas, 1995)) ........................................................................................................................... 18
Figure 10: The Performance Management Systems framework (adapted from (Ferreira & Otley,
2009)) ........................................................................................................................................ 19
Figure 11: Journey from a Mission and Vision to performance measures that work (from
(Parmenter, 2011)) .................................................................................................................... 21
Figure 12: Example of a KPI (from (Niven, 2006)) ..................................................................... 25
Figure 13: Four types of performance measures (from (Parmenter, 2011)) ............................... 29
Figure 14: PM implementation model for SMEs (from (Parmenter, 2011)) ................................. 30
Figure 15: Using the Balanced Scorecard as a strategic management system (adapted from
(Kaplan & Norton, 1996b)) ......................................................................................................... 34
Figure 16: The Strategy Map template (from (Kaplan & Norton, 2004)) ..................................... 36
Figure 17: Aggregate framework metamodel (from (Braun & Winter, 2005)) ............................. 44
Figure 18: The Strategy Layer metamodel (from (Braun & Winter, 2005)) ................................. 45
Figure 19: The Organization Layer metamodel (from (Braun & Winter, 2005)) .......................... 46
Figure 20: Goal-Performance part of core metamodel (translated from (Winter, 2011)) ............. 47
Figure 21: The UBSMM (from (Giannoulis et al., 2012) ............................................................. 48
Figure 22: Why, How and What of a goal (from (Markovic & Kowalkiewicz, 2008)) ................... 50
Figure 23: Optional Success Factor class as specialization of Goal class ................................. 56
Figure 24: Indicator class together with Measurement relationship with Success Factor class .. 56
Figure 25: Many-to-many Source Relationship between Indicator and Object classes .............. 57
X
Figure 26: Indicator class with reflexive Includes relationship .................................................... 58
Figure 27: Milestone class is specialized in Target class and has a Comparison relationship with
the Indicator class ..................................................................................................................... 59
Figure 28: CHOOSE metamodel with Performance-oriented dimension after literature research
.................................................................................................................................................. 60
Figure 29: Goal dimension through action research using post-its ............................................. 70
Figure 30: From mission to the four highest-level BSC perspectives ......................................... 71
Figure 31: Elimination of Type attribute from Indicator class ...................................................... 72
Figure 32: Decomposition of ‘Stimulate closer collaboration between VMD and Biové’ into
SMART goal .............................................................................................................................. 73
Figure 33: Decomposition of 'Put the right people on the right spot' into SMART goal ............... 74
Figure 34: Replace Success Factor construct by Objective construct ........................................ 75
Figure 35: Performance-oriented viewpoint on ‘Reliable forecast from VMD to Biové’ ............... 78
Figure 36: Assigned (RACI) relationship with an association class with a Type attribute ........... 79
Figure 37: Example of the Assigned relationship between Actor and Indicator .......................... 80
Figure 38: Performance-oriented viewpoint on ‘Put the right people on the right spot’ ............... 81
Figure 39: Description attribute of the Milestone class ............................................................... 81
Figure 40: Example of the interrelation of the different viewpoints ............................................. 82
Figure 41: Performance-oriented viewpoint on ‘Realize a high level of profitability’ ................... 83
Figure 42: Example of an As-is/To-be and Degree of Satisfaction analysis ............................... 85
Figure 43: Final CHOOSE Metamodel ....................................................................................... 86
Figure 44: Method Evaluation Model (from (Moody, 2003)) ....................................................... 91
Figure 45: UEBQM (from (A. Maes & Poels, 2007)) .................................................................. 94
XI
LIST OF TABLES
Table 1: Overview of the used literature .................................................................................... 54
Table 2: Six-item scales of Perceived Usefulness and Perceived Ease of Use (from (Davis,
1989)) ........................................................................................................................................ 92
Table 3: Questionnaire for Perceived Usefulness and Perceived Ease of Use (from (Bernaert,
Callaert, Poels, et al., 2013)) ..................................................................................................... 93
Table 4: The UEBQM measurement instrument concerning the four quality dimensions (from (A.
Maes & Poels, 2007)) ................................................................................................................ 94
Table 5: Measurement instrument to evaluate the CHOOSE metamodel (from (Bernaert,
Callaert, Poels, et al., 2013)) ..................................................................................................... 95
Table 6: Ten design criteria of the CHOOSE approach (translated from (Bernaert, Callaert, &
Poels, 2013)) ............................................................................................................................. 96
Table 7: Evaluation results of CHOOSE metamodel and method by the CEO, SC manager, HR
manager and member of the board ........................................................................................... 98
Table 8: Evaluation results of the CHOOSE model by the external PM and ICT Architecture
expert ...................................................................................................................................... 100
Table 9: Evaluation results of the CHOOSE model by the internal PM expert ......................... 101
1
DUTCH ABSTRACT
Deze masterproef beschrijft de ontwikkeling van een performance management systeem voor
de CHOOSE techniek voor enterprisearchitectuur voor kleine en middelgrote ondernemingen.
De CHOOSE techniek is een techniek ontwikkeld aan de Universiteit Gent en modelleert de
bedrijfsarchitectuur aan de hand van vier dimensies met overeenkomende entiteiten en relaties:
de doelen van de onderneming (waarom), de actoren (wie), de operaties (hoe) en de objecten
(wat). De CHOOSE techniek bestaat uit een CHOOSE metamodel, een CHOOSE methode en
een CHOOSE tool support.
Het belangrijkste aspect van een enterprisearchitectuur is de dimensie van de doelen. Echter,
zonder een correcte meting van de vervulling van deze doelen blijft het controle gedeelte van
een enterprisearchitectuur zinloos. Performance management biedt hiervoor een oplossing door
de realisatie van doelen te koppelen aan metingen, streefcijfers en bijhorende acties.
Deze masterthesis stelt daarom een optionele performance georiënteerde uitbreiding op de
CHOOSE techniek voor. Deze uitbreiding is bedoeld voor kleine en middelgrote ondernemingen
die geconfronteerd worden met een toenemende complexiteit en de noodzaak voelen om de
realisatie van hun doelen te meten. Zowel het CHOOSE metamodel als de CHOOSE methode
worden uitgebreid met een performance georiënteerde dimensie. Deze uitbreiding ontwikkelde
zich in drie stappen. Als eerste werd er een initiële performance georiënteerde dimensie
toegevoegd aan de oorspronkelijke CHOOSE techniek op basis van een uitgebreid
literatuuronderzoek betreffende enterprisearchitectuur, performance management en de link
tussen beiden. Het metamodel en de methode die op deze manier werden ontwikkeld, zijn
daarna in de tweede stap verfijnd en aangepast aan de hand van een casestudie onderzoek bij
één bedrijf. Ten slotte werd de extensie positief geëvalueerd door het betrokken casestudie
bedrijf en door een externe performance management expert.
2
INTRODUCTION
Enterprise Architecture techniques represent all elements of an enterprise (business-,
information systems-, and technology-elements) in a structured way. The concept came into
being because information systems grew more complex and both the importance of ICT and the
alignment between ICT and business was emphasized (Sessions, 2007).
Although underrepresented in literature and practice, small and medium-sized enterprises could
especially benefit from enterprise architecture, as they are frequently confronted with problems
caused by lack of structure and control (Bernaert & Poels, 2011b). For that reason, Bernaert and
Poels (2011a) developed the CHOOSE approach, which is an acronym for “keeping Control, by
means of a Holistic Overview, based on Objectives and kept Simple of your Enterprise” and
consists of a metamodel, a method and tool support to compose an enterprise architecture
model of a small or medium-sized enterprise (Bernaert, Poels, Snoeck, & De Backer, 2013).
More recently, it was acknowledged that the strategy and goals (business elements) of an
enterprise are the most important and as a result, enterprise architecture should start from there
(Lankhorst, 2009; Ross, Weill, & Robertson, 2006). Nevertheless, current enterprise architecture
approaches often do not pay sufficient attention to this area (Cardoso & Almeida, 2010). Without
properly measuring the fulfilment of goals, the controlling dimension of enterprise architecture
remains pointless (Matthes & Monahov, 2012).
Performance management ensures the achievement of organizational goals by associating them
with measures, controlling those measures and executing appropriate actions (Popova &
Sharpanskykh, 2011). Managing performance has been greatly acknowledged and used by
large enterprises, but once more small and medium-sized enterprises are falling behind, even
though they could certainly benefit from it (Hudson, Smart, & Bourne, 2001).
Since performance management facilitates the realization of goals and goals are the most
important aspect of an enterprise architecture, this master thesis aims to provide a performance
management system for the CHOOSE approach for enterprise architecture for small and
medium-sized enterprises. Hence, both the CHOOSE metamodel and the CHOOSE method will
be extended with a Performance-oriented dimension. This extension is an optional add-on for
small and medium-sized enterprises that are facing increased complexities and perceive the
need to measure the achievement of their goals. The extension was accomplished in three
successive steps. First of all, an initial metamodel and method were built based on an extensive
3
literature review and on the current CHOOSE approach (Bernaert, Callaert, Poels, Snoeck, & De
Backer, 2013). Secondly, this metamodel and method were extended and refined through a
cyclical process of action research (by a case study research). Finally, the metamodel and
method are evaluated by the case study enterprise and by a performance management expert.
This master thesis is structured in five parts. Part one covers the literature review, part two
raises the problem statement, part three proposes a solution, part four evaluates this solution
and part five provides a general conclusion.
Part one (literature review) consists of four chapters. Chapter one provides an overview of the
typical characteristics of small and medium-sized enterprises. Chapter two explains the concept
of enterprise architecture, accompanied by the CHOOSE approach specifically for small and
medium-sized enterprises. The third chapter deals with performance management and with the
specific features of performance management for small and medium-sized enterprises. The last
chapter provides existing links between enterprise architecture and performance management.
The second part (problem statement) of this master thesis will explain the encountered problem
regarding a performance management system for the CHOOSE approach for small and
medium-sized enterprises.
Part three (solution) is structured in four chapters. In chapter one, an initial Performance-
oriented extension to the CHOOSE metamodel and method is proposed, based on the literature
review. Chapter two provides the case study as action research. First of all, the methodology is
clarified, which is mainly based on the principles of action research and case study research.
Afterwards, the case study is described, the case study protocol and database are provided, and
the refinement process through different action research cycles is explained. Finally, chapter
three will present the final solution.
The fourth part of this master thesis (evaluation) contains two chapters. Chapter one describes
the models that are used to evaluate the proposed solution. The participants of the case study
will evaluate the extension to the CHOOSE method and metamodel based on respectively the
Method Evaluation Model (Moody, 2003) and the User Evaluations Based Quality Model for
Conceptual models (A. Maes & Poels, 2007). In addition, an expert will evaluate the extension
based on the ten design criteria of the CHOOSE approach (Dumeez, Bernaert, & Poels, 2013).
Afterwards, the results of this evaluation will be discussed in Chapter two.
Finally, part five offers the overall conclusion of this master thesis, together with its limitations
and recommendations for future research.
4
PART I: LITERATURE REVIEW
INTRODUCTION LITERATURE REVIEW
In order to create a Performance-oriented dimension for the current CHOOSE approach, a
description of the most important concepts and developments regarding this subject is required.
Therefore this master thesis starts with a literature review, divided in four chapters.
In Chapter I, the most important characteristics of small and medium-sized enterprises are
described, together with the specific approach required to deal with these characteristics.
Chapter II explains the concept of enterprise architecture. First of all, a definition of the concept
is given, followed by the positioning of enterprise architecture in information management.
Subsequently, the different architectural layers and views, the ingredients of enterprise
architecture and its advantages are described. Finally, the enterprise architecture technique
specifically developed for small and medium-sized enterprises, namely the CHOOSE approach,
is explained.
Chapter III regards performance management. To start, the difference between performance
management (systems) and performance measurement (systems) is clarified. Afterwards, the
goal concept is described, together with a constraint on these goals (the organization structure).
Next, performance measurement is elucidated, divided in two parts: the first regards key
performance indicators and the second discusses several well-known performance
measurement systems. Furthermore, several target, evaluation and rewards practices are
mentioned. Finally, the implementation of the performance management systems is briefly
explained.
To conclude, Chapter IV provides the link between enterprise architecture (Chapter II) and
performance management (Chapter III). First, the relationship between key performance related
concepts and enterprise architecture elements is described, followed by two metamodels that
currently establish this link: the EA metamodel of Braun and Winter and the unified business
strategy metamodel of Giannoulis et al.
5
CHAPTER I: SMALL AND MEDIUM-SIZED ENTERPRISES
According to extract of Article 2 of the Annex of Recommendation 2003/361/EC, the definition of
a Small and Medium-sized Enterprise (SME) is as follows: “The category of micro, small and
medium-sized enterprises is made up of enterprises which employ fewer than 250 persons and
which have an annual turnover not exceeding 50 million euro, and/or an annual balance sheet
total not exceeding 43 million euro” (European Union, 2003).
SMEs have nine intrinsic characteristics that differ from large companies:
(1) Employees and management have only little time for both strategic matters and
performance management issues, because they are fully occupied with daily matters
(Bernaert & Poels, 2011b; Tenhunen, Rantanen, & Ukko, 2001; Vichitdhanabadee,
Wilmshurst, & Clift, 2009).
(2) SMEs have significantly smaller resources than larger companies in terms of
management, manpower and finance. This has several reasons: competitive
environment, financial constraints, lack of expertise and sensitivity to external influences
(Bernaert & Poels, 2011b; Hudson et al., 2001).
(3) SMEs have a limited Information Technology (IT) knowledge and technical skills
(Bernaert & Poels, 2011b).
(4) The CEO is the central figure who determines the direction of an SME. His skills and
preferences are of great importance (Bernaert & Poels, 2011b).
(5) The CEO takes all the important decisions. He decides if it is a yes or a no. An SME is
characterized by a very personalized management, with little devolution of authority
(Bernaert & Poels, 2011b; Hudson et al., 2001). Therefore, the expected returns must
exceed the risks and costs (Bernaert & Poels, 2011b; Parmenter, 2011).
(6) SMEs have a large demand for know-how transfer within the enterprise (Bernaert &
Poels, 2011b).
(7) The strategy of an SME is often informal, non-documented, dynamic and marked by high
innovatory potential. Furthermore, this strategy is mostly defined by the owner and not
well communicated to the employees (Chalmeta, Palomero, & Matilla, 2012; Hudson et
al., 2001).
(8) An SME has no culture of using indicators for performance management (Chalmeta et
al., 2012).
(9) SMEs are more short-term minded than larger companies are (McAdam, 2000).
6
Because of these characteristics, SMEs are in need of both an appropriate Enterprise
Architecture (EA) and Performance Management (PM):
An appropriate EA is necessary because SMEs struggle with problems related to a lack
of structure and overview of their business. However, the EA approaches created for
large enterprises are too complex for SMEs (Bernaert & Poels, 2012).
Furthermore, PM approaches are currently rarely completed in SMEs (Tenhunen et al.,
2001), as they do not take into account the fundamental differences between SMEs and
larger organizations (Garengo, Biazzo, & Bititci, 2005). Therefore, models that have been
developed for large enterprises do not seem to apply well for SMEs (Cassell, Nadin, &
Gray, 2001), which is confirmed by the gap between theory and practice (Cocca &
Alberti, 2010) and the low adoption rate of performance measurement practices in SMEs
(Garengo et al., 2005).
Each of the characteristics stated above, requires a specific approach for implementing EA and
PM in SMEs (Bernaert, Callaert, Poels, et al., 2013; Hudson & Smith, 2007). Day-to-day
business requires working in a time-efficient manner on strategic and performance issues (1).
Resource poverty requires an implementation without much assistance from external experts,
and a very resource efficient development to ensure viability (2). A limited IT knowledge and
technical skills require an easy, user-friendly approach, even for people with limited IT
knowledge (3). CEO as central figure requires a big involvement of the CEO (4). The fact that
the CEO takes the decisions requires a method where the expected returns are bigger than the
expected costs (5). A big demand for knowledge sharing requires a guide of descriptions on how
things are done in the company (6). Also, an informal and quickly changing strategy requires a
dynamic and flexible development to accommodate strategic changes and ensure continued
strategic relevance (7). The measures defined to achieve the strategy, should furthermore be
easy to understand (8). Finally, both short-term as well as long-term benefits have to be
established, to help maintain the enthusiasm of the team (9).
7
CHAPTER II: ENTERPRISE ARCHITECTURE
2.1 Definition of Enterprise Architecture
To understand the term ‘Enterprise Architecture’, it is required to first understand the term
‘Architecture’ on its own. Let’s first think about the architecture of a building. The architect has to
specify the spatial structure, dimensions, functions, materials, colors, etc. based on the
requirements of its future owners and users, and according to the applicable regulations
(Jonkers et al., 2006). When focusing on the architecture in the IT world, the definition of
Architecture according to the IEEE Standard 1471-2000 is as follows: “Architecture is the
fundamental organization of a system embodied in its components, their relationships to each
other, and to the environment, and the principle guiding its design and evolution (IEEE Computer
Society, 2000).” When architecture is considered at the level of the entire enterprise, the term
‘Enterprise Architecture’ (EA) is used. According to Sessions, EA means: “An architecture in
which the system in question is the whole enterprise, especially the business processes,
technologies and information systems of the enterprise (Sessions, 2007).” An important
characteristic of EA is the fact that it provides a holistic view of the enterprise (Jonkers et al.,
2006). This enables optimization of the company as a whole instead of doing local optimization
within individual domains (Bernaert, Poels, Snoeck, & De Backer, 2014). The definitions of
Architecture and EA stress that architecture is both a process and a product: it refers to both the
principles used in the design process of the system in line with the business objectives and to
the product of that design, in the form of a blueprint of the system and a guide for managers
(Jonkers et al., 2006).
2.2 Positioning of EA: information management
As the complexity of information systems (IS) has been growing rapidly, organizations are
finding it more and more difficult to keep those increasingly expensive IT systems aligned with
business need problems (Sessions, 2007). The need for better Business-IT alignment on both
the strategic as operational level has been described by Henderson and Venkatraman in their
Strategic Alignment Model (SAM) (Henderson & Venkatraman, 1993), which formed the
foundation for the development of the Amsterdam Information Management (AIM) model of
Maes (R. Maes, 2007). Maes emphasizes the role of information management within the
organization by adding an Information & Communication column and a Structure row to the SAM
(R. Maes, 2007). EA is located in the Structure level of the model (cf. Figure 1), and in this way it
8
takes care of business-IT alignment in a structural way: it develops a consistent architecture that
comprises the relationship between the business-, information/communication-, and technology-
architecture, but also the relationship with the strategy and operations blocks (R. Maes, 2007).
Figure 1: Positioning of EA in AIM (adapted from (R. Maes, 2007))
2.3 Different architectural layers and views
Information regarding an enterprise is demanded by a lot of different stakeholders, going from
top-level management to software engineers, and each stakeholder requires information in
another format (Jonkers et al., 2006). So when you ask Zachman (Zachman, 1987) the question
“What is information systems architecture?”, he answers “There is not an information systems
architecture, but a set of them that are additive and complementary.” By this he means that
architecture is relative: one is not right and another wrong. What you think architecture is,
depends on what you are doing (Zachman, 1987). In the aforementioned definition of EA
(Sessions, 2007), it is stated that architecture covers the whole enterprise. This includes many
different component types and thereby an enormous number of artefacts. As a consequence,
most EA approaches distinguish several architecture layers and architecture views in order to
reduce the number of artefacts per model (Schekkerman, 2004). A layered architectural
framework is useful in order to represent different aspects in different models as long as overall
consistency is preserved by appropriately modelling the framework’s metamodel (Braun &
Winter, 2005).
9
Wagter et al. (2005) state that the architectural layers can be grouped into three general kinds of
architecture: business architecture (product/service, process and organization architecture),
information architecture (data and application architecture) and technological architecture
(middleware, platform and network architecture) (Wagter, Van Den Berg, Luijpers, & van
Steenbergen, 2005). In the beginning of its existence, EA focused mainly on IT and its alignment
with the business (as it was developed by IT researchers) (Henderson & Venkatraman, 1993),
but nowadays the business architecture layer receives more attention as it is the basis where
everything starts (Lankhorst, 2009; Ross et al., 2006). Four dimensions that are shared by most
of the EA techniques can be distinguished in the business architecture layer: a strategic
dimension (Why?), an active actor (Who?), an operation (How?), and an object (What?)
(Bernaert et al., 2014). The layers are depicted in an inverted pyramid (cf. Figure 2) to
demonstrate the fact that business architecture is the foundation of a good EA, and an EA
technique has to start with the strategic question (Why?) (Bernaert et al., 2014).
Figure 2: EA with more focus on the four core dimensions of the business architecture (from (Bernaert et al., 2014))
Many EA techniques made abstraction of the strategy in the past, although this is the most
important part as different stakeholders all have different goals (M. Porter, 1985). The domain of
‘Motivation’, which can be seen as a particular viewpoint (cf. Section 2.4), has been recognized
as an important element by Zachman (1987). Goal modelling explicitly captures the goals of an
organization and thereby it documents the organization’s strategy (Andersson et al., 2008).
Despite the fundamental importance of goal modelling in EA efforts, limited attention was
dedicated to explicitly representing goals and using goal-related concepts to increase the value
of enterprise modelling techniques (Cardoso & Almeida, 2010). This master thesis aims to
provide an extension to the CHOOSE technique (cf. Section 2.6) in order to further elaborate the
domain of ‘Motivation’. Currently, some other EA techniques begin to incorporate this aspect as
well: both TOGAF and ArchiMate incorporate this dimension through a motivation extension in
their latest EA technique (The Open Group, 2011, 2012). The extensions in TOGAF will be
briefly discussed below.
10
2.3.1 TOGAF
The TOGAF Architecture Content framework (cf. left side Figure 3), which provides the
underlying structure for the better known Architecture Development Method (ADM), contains a
Content Metamodel. This metamodel is built of a core metamodel with a minimum feature set
and of additional metamodel concepts to support possible extensions. Two important extensions
regarding goal modelling are incorporated in the Motivation part of the Business Architecture
artefacts: the Motivation Extension (Drivers, Goals and Objectives) and the Governance
Extension (Measures) (The Open Group, 2011).
Figure 3: Content metamodel and Motivation & Governance extension of TOGAF (from (The Open Group, 2011))
11
A Driver is an external or internal condition that motivates the definition of organizational goals,
while a Goal is a high-level statement of intent or direction for an organization. An Objective is a
time-bounded milestone for an organization, used to demonstrate progress towards a goal. A
Measure is an indicator that can be tracked, usually on an ongoing basis, to determine success
or alignment with objectives and goals (The Open Group, 2011).
The Motivation Extension (cf. right side Figure 3) allows additional structured modelling of the
drivers, goals, and objectives that influence an organization to provide business services to its
customers, and thereby permits a better measurement of performance. The Governance
Extension supports in-depth operational governance, by allowing additional structured data to be
held against objectives. The extension permits to apply measures to objectives (The Open
Group, 2011).
2.4 Ingredients of EA
Obtaining a good EA technique requires several indispensable ingredients (Jonkers, Proper, &
Turner, 2009; The Open Group, 2003), which can be represented in an elementary ‘bow tie’
model of EA (cf. Figure 4).
Figure 4: Bow-tie model of EA (adapted from (Tavernier & Poels, 2012))
The ingredients are:
A Process (or method) that is the methodology for creating architectures consisting of
guidelines, techniques, and best practices;
A set or classification of Viewpoints that define the perspectives relevant for different
stakeholders;
A Language that describes the EA based on both a metamodel as the corresponding
notation; and
Viewpoints Process Language
Repository, (Reference) Models
12
A Repository that stores the architectural artefacts and possibly contains predefined
reference models.
The bow tie model permits EA proposals to be evaluated in terms of which ingredients are
covered and how well they are supported and furthermore permits different approaches to be
integrated (Tavernier & Poels, 2012).
An EA framework is a multidimensional classification schema that helps ensuring the
completeness of the EA (The Open Group, 2011). It is a communication model for developing an
EA (Schekkerman, 2004) and a tool which can be used for developing a broad range of different
architectures (The Open Group, 2011). It contains “a set of models, principles, services,
approaches, standards, design concepts, components, visualizations and configurations that
guide the developments of specific aspect of architectures (Schekkerman, 2004, p. 91)”.
2.5 Advantages of EA
An organization will only implement an EA approach if the expected benefits of the project
exceed the expected costs (Bernaert et al., 2014). Some possible benefits of an EA are:
EA approaches enable strategic business goals via better operational excellence (Ross et
al., 2006).
The enterprise architect takes the desires of every stakeholder into consideration when
making the mutual architecture, and thereby all the stakeholders agree with it (Bernaert &
Poels, 2011a).
On the one hand, the architect produces a general overview of the most important
domains of the enterprise, but on the other hand also makes it possible to only have a
look at a little part of this overview, without any irrelevant details by means of different
viewpoints. A person from the marketing department for example, may only need an
overview of the products and the specific customer group (Bernaert & Poels, 2011a).
The different viewpoints can be used as documentation (Bernaert & Poels, 2011a).
By capturing the goals of an enterprise the Motivation viewpoint manages business
improvements, as changes in a company’s strategy have significant consequences within
all organizational domains (Yu, 2011).
The reproduction and the interdependence between the elements of EA may serve for
analyses and optimization of the enterprise and for performing an impact-analysis of
changes as well (Bernaert & Poels, 2011a).
13
2.6 EA for SMEs: the CHOOSE Approach
Bernaert and Poels answered to the need of SMEs for EA by creating the CHOOSE approach
according to Einstein’s principle: “Everything should be made as simple as possible, but not
simpler” (Bernaert & Poels, 2012). CHOOSE is an acronym for “keeping Control, by means of a
Holistic Overview, based on Objectives and kept Simple of your Enterprise” (cf. Figure 5). This
acronym contains all the important criteria of a good EA technique (Dumeez et al., 2013):
(1) Control: Controlling the complexity of the enterprise;
(2) Holistic Overview: the EA has to capture the essentials of the enterprise (more stable);
(3) Objectives: Translation from corporate strategy to daily operations;
(4) kept Simple is the translation of ‘Suitability for its target audience’ (here: SMEs): it should
be understood by all those involved;
(5) Enterprise: Optimizations of the company as a whole, instead of doing local optimization
within individual domains.
The fourth criterion ‘Suitable for target audience (SME)’ is divided in six SME-related criteria (cf.
Chapter I).
Figure 5: Overview evaluation criteria for an EA technique for SMEs (adapted from (Ingelbeen, Bernaert, & Poels, 2013))
The approach is based on four integrated models (cf. Figure 6) to frame the business
architecture of an enterprise. The four models correspond with the four dimensions of business
architecture stated in Section 2.3: a strategic goal dimension (Know-Why), an active actor
14
dimension (Know-Who), an operation dimension (Know-How), and an object dimension (Know-
What). The relationships between those models are indispensable as well (Bernaert & Poels,
2011a).
Figure 6: Structure of the four integrated models (from (Bernaert & Poels, 2011a))
The CHOOSE approach is built of three artefacts: the CHOOSE metamodel, the CHOOSE
method and the CHOOSE tool support.
2.6.1 CHOOSE Metamodel
The metamodel was built as a first artefact of the CHOOSE approach in compliance with the
Language ingredient of a good EA (cf. Section 2.4) (Jonkers et al., 2009). It consists of meta-
concepts, -relationships, -attributes, and –constraints. The development of the metamodel is a
continuous trade-off between simplicity (adapted to the SME context) and completeness (EA
criteria) (Bernaert, Poels, et al., 2013).
In Figure 7 the current CHOOSE metamodel is displayed, which has been developed through
several case studies (Bernaert, Callaert, & Poels, 2013; Bernaert et al., 2014). This metamodel
will serve as the starting point for the development of the Performance-oriented extension of the
CHOOSE metamodel.
The CHOOSE metamodel blends the three EA layers (business, IS, IT) by providing Actors for
each layers (Human Actor / Role, Software Actor, Device) and enables the other three
viewpoints to be related to it. Goals, Operations and Objects could also originate from the three
different EA layers, however no explicit specialization is needed for this (Bernaert, Poels, et al.,
2013).
15
Figure 7: Current CHOOSE metamodel (from (Bernaert, Callaert, Poels, et al., 2013))
2.6.2 CHOOSE Method
As a second artefact, the CHOOSE method was developed in accordance with the process
ingredient of a good EA (cf. Section 2.4) (Jonkers et al., 2009). The CHOOSE method supports
the CHOOSE metamodel and contains step-by-step guidelines, best practices and heuristics
(Bernaert, Callaert, & Poels, 2013). It is built around two popular frameworks, the Balanced
Scorecard (BSC) (Kaplan & Norton, 1992) and Porter’s Value Chain (M. Porter, 1985), and
consists of three parts: a six step Roadmap-Method, a three step Interview-Method and four
Stop-Criteria (Bernaert, Callaert, & Poels, 2013). These can be found in Appendix 1. This
method will serve as the starting point for the development of the Performance-oriented
extension of the CHOOSE method.
2.6.3 CHOOSE Tool Support
Implementing the CHOOSE tool support lowers the threshold and increases the rate of adoption
of the CHOOSE approach for organizations (Bernaert, Callaert, & Poels, 2013). Several
CHOOSE tool supports have been developed already (Dumeez et al., 2013; Ingelbeen et al.,
2013; J. Maes, Poels, & Bernaert, 2013; Otte, Poels, & Bernaert, 2013; Puylaert, Bernaert, &
Poels, 2013; Verhulst, Bernaert, & Poels, 2013; Zutterman, Bernaert, & Poels, 2013). The
CHOOSE tool support goes beyond the research domain of this master thesis.
16
CHAPTER III: PERFORMANCE MANAGEMENT
Companies are currently competing in a globalized and competitive market, where customer
demand is changing rapidly (Nudurupati & Bititci, 2011). Furthermore, quality assurance has
been an important issue for some time, and the power of IT cannot be underestimated (Neely,
1999). In order to survive in a dynamic environment, companies need to be able to satisfy all
their stakeholders and simultaneously excel along all performance dimensions (Neely, Adams, &
Kennerley, 2002). Performance indicates the possibility of achieving a future successful
implementation of actions in order to reach the postulated objectives and targets (Lebas, 1995).
3.1 Performance Measurement and Performance Management
3.1.1 Performance Measurement (System)
A requirement to achieve high performance standards is being able to effectively measure and
monitor a company’s performance (Cocca & Alberti, 2010). Performance Measurement is
defined as “the process of quantifying the efficiency and effectiveness of actions” (Neely et al.,
2002). Measurement is useful for an organization in three ways (Amaratunga & Baldry, 2002):
(1) It provides the basis for an organization to assess how well it is progressing towards its
predetermined objectives,
(2) It helps to identify areas that need attention, and
(3) It determines future initiatives to enhance organizational performance.
To deal with the changing environment, it is furthermore required to have up-to-date and
accurate performance information, i.e. information that is integrated, dynamic, accessible and
visible to support fast decision-making (Nudurupati & Bititci, 2011). For that reason, a
Performance Measurement System (PMES) is created. A PMES is a balanced and dynamic
system that gathers, elaborates an analyses information to support the decision-making process
(Neely et al., 2002).
17
3.1.2 Performance Management (System)
Performance Management (PM) aligns performance with strategy (Eckerson, 2009). According
to Amaratunga and Baldry (2002) PM uses “performance measurement information to change
the organizational culture, systems and processes in a positive direction by helping to set
agreed-upon performance goals, assigning and prioritizing resources, informing managers to
either confirm or change current policy or program directions to meet these goals, and sharing
results of performance in pursuing those goals” (Amaratunga & Baldry, 2002). Eckerson (2009)
describes PM in a virtuous cycle. This cycle (cf. Figure 8) contains four steps: creating strategy,
creating plans, monitoring the execution of those plans, and adjusting activity and objectives to
achieve strategic goals. This four-step wheel turns around integrated data and metrics, which
provide a measurement framework that measures the efficiency and effectiveness of strategic
and management processes (Eckerson, 2009; Neely et al., 2002).
Figure 8: The PM Cycle (from (Eckerson, 2009))
The relation between Performance Measurement and Management can be described through an
iterative process (cf. Figure 9): management both precedes and follows measurement in an
upward spiral, hereby creating the context for measurement (Lebas, 1995).
18
Figure 9: Performance Management and Measurement are closely intertwined (adapted from (Lebas, 1995))
A Performance Management System (PMAS) is the philosophy supported by performance
measurement: the vision, teamwork, training and incentives that surround the performance
measurement activity throughout the entire organization (Lebas, 1995). Ferreira and Otley
declare a more elaborated definition by describing PMASs as “the evolving formal and informal
mechanisms, processes, systems, and networks used by organizations for conveying the key
objectives and goals elicited by management, for assisting the strategic process and ongoing
management through analysis, planning, measurement, control, rewarding, and broadly
managing performance, and for supporting and facilitating organizational learning and change”
(Ferreira & Otley, 2009, p. 264). PMASs cover the definition, control and management of both
the achievement of outcomes as well as the means used to achieve those results (Broadbent &
Laughlin, 2009).
Ferreira and Otley (2009) developed a PMAS framework (cf. Figure 10) that aims to provide a
broad view of the key aspects of a PMAS and that can be used as a descriptive tool to outline
the main features of the system in a comprehensive manner.
The framework consists of twelve questions that have brought a significant comprehension in
the various aspects of PMASs design and use, and that describe the following concepts: (1)
Vision and Mission, (2) Key Success Factors, (3) Organization Structure, (4) Strategies and
Plans, (5) Key Performance Measures, (6) Target Setting, (7) Performance Evaluation, (8)
Reward Systems, (9) Information Flows, Systems, and Networks, (10) PMASs Use, (11) PMASs
Change, and (12) Strength and Coherence (Ferreira & Otley, 2009).
19
Figure 10: The Performance Management Systems framework (adapted from (Ferreira & Otley, 2009))
The questions are presented as a heuristic tool to enable the rapid description of the significant
aspects of PMASs design and operation, and are not imposing a normative framework (Ferreira
& Otley, 2009). Furthermore, Ferreira and Otley state that contextual factors and organizational
culture should be taken into consideration as well, even if only implicitly. Broadbent and Laughlin
(2009) extend these contextual factors and organizational culture in their framework.
The remainder of this chapter will be structured according to this framework. First of all, the
questions regarding the PMAS design (Q1 to Q8 in Figure 10) will be split into three parts:
Section 3.2 first describes the goal questions (Q1, Q2 and Q4 in Figure 10) and the constraint
question (Q3 in Figure 10), Section 3.3 takes care of the performance measures (Q5 in Figure
10), and Section 3.4 finishes with the performance target, evaluation and reward practices (Q6,
Q7 and Q8 in Figure 10). Afterwards, Section 3.5 briefly deals with the questions regarding the
implementation of the PMAS (Q9 to Q12 in Figure 10).
Before continuing, the benefits of PM and PMASs in SMEs will be shortly described.
20
3.1.3 Advantages of Performance Management (Systems) in SMEs
PM has many advantages for SMEs, provided that it is properly adapted to their specific needs
(Garengo et al., 2005):
PM helps to formalize and communicate the organizational strategy and align the
operational activities with the strategy of the SME (Garengo et al., 2005). This is
particularly important, as SMEs frequently have informal and non-documented strategies
(cf. Chapter I, characteristic (7)).
PM makes it easier to decentralize decisions (Hudson et al., 2001). This is essential as
well, as the CEO of an SME currently takes all the decisions by himself (cf. Chapter I,
characteristic (5)).
PM improves the integration of the company’s IS (Bititci, Turner, & Begemann, 2000).
This integration reduces the costs of implementation, which corresponds with the limited
financial resources of SMEs (cf. Chapter I, characteristic (2)).
PM takes into account different perspectives of analysis in a balanced framework
(Tenhunen et al., 2001). This allows defining a proper strategy (cf. Chapter I,
characteristics (1) and (7)).
PM is dynamically adaptable to the changing internal and external environment of the
SME (Bititci et al., 2000), which also corresponds with the dynamic strategy (cf. Chapter
I, characteristic (7)).
A PMAS can only be effectively implemented and used if SMEs understand the benefits of it, yet
this is often not the case (Hudson et al., 2001). Therefore, a PMAS should respond to the
specific needs of SMEs, be efficient and easy to understand, and clearly state the benefits
(Garengo et al., 2005). The need to make a PMAS simple and easy to use should nevertheless
not compromise the completeness of a system, as would be the case if only single measures
were used or if PMAS models developed for big companies were purely simplified for SMEs by
reducing the number of measures without maintaining the holistic vision of the original
architecture (McAdam, 2000). This once more corresponds with Einstein’s principle (cf. Section
2.6): “Everything should be made as simple as possible, but not simpler”. When establishing a
PMAS for the CHOOSE approach in PART III, these constraints should be taken into account.
21
3.2 Goals
3.2.1 Vision, Mission and Values
Ferreira & Otley (2009) ask the following question in their twelve-question framework: “Question
one: What is the vision and mission of the organization and how is this brought to the attention of
managers and employees? What mechanisms, processes, and networks are used to convey the
organization’s overarching purposes and objectives to its members?”
A mission shapes “the overall purpose of the organization in line with the values or expectations
of stakeholders” (Johnson, Scholes, & Whittington, 2008, p. 9). It is a guiding light that may
never be reached for the organization and can remain the same for decades (Parmenter, 2011).
A vision sets out the aspiration of the organization and is also called the desired future state
(Johnson et al., 2008). The vision describes what the organization will achieve if it is successful,
and has a more specific future state and time-frame compared to the mission (Parmenter, 2011).
Values are what the organization stands for (Parmenter, 2011), they determine the boundaries in
which a strategy can be developed (Johnson et al., 2008).
Parmenter (2011) developed a framework (cf. Figure 11) that resembles the first five questions
of Ferreira and Otley’s twelve-questions framework (cf. Figure 10). The framework also starts by
a mission and vision (and values) statement, and incorporates critical success factors and
strategies as well.
Figure 11: Journey from a Mission and Vision to performance measures that work (from (Parmenter, 2011))
22
3.2.2 Critical Success Factors
Ferreira & Otley (2009) ask the following question in their twelve-question framework: “Question
two: What are the key factors that are believed to be central to the organization’s overall future
success and how are they brought to the attention of managers and employees?”
Critical Success Factors (CSFs) translate the mission and the vision of an organization into more
concrete and time specific terms (Ferreira & Otley, 2009). They determine the health, vitality and
well-being of an organization and stipulate on which areas the organization has to perform well
(Parmenter, 2011). PM would not make sense without defining clear CSFs, since performance
measurement, monitoring and reporting would be random processes (Parmenter, 2011).
Parmenter (2011) states that a perfect number of CSFs is five to eight regardless of the
organization’s size, although at first sight many more could have been found.
The selection of the CSFs is a very subjective exercise and the effectiveness and usefulness of
the chosen CSFs is highly dependent on the degree of analytical skill of those involved
(Parmenter, 2011). A technique to find the success factors that are most critical to the
organization is mapping cause-and-effect relationships between the different success factors
(Parmenter, 2011). This could be done by Strategy Mapping (cf. Section 3.3.2.5) (Kaplan &
Norton, 2004).
When linking a CSF to Key Performance Indicators (KPIs) (cf. Section 3.3.1), one CSF may
need several KPIs (Brooks & Chittenden, 2012; Parmenter, 2011). The contribution of each KPI
to the realization of the CSF can differ (Pourshahid, Amyot, Chen, Weiss, & Forster, 2007).
3.2.3 Strategies and Plans
Ferreira & Otley (2009) ask the following question in their twelve-question framework: “Question
four: What strategies and plans has the organization adopted and what are the processes and
activities that it has decided will be required for it to ensure its success? How are strategies and
plans adapted, generated and communicated to managers and employees?”
Strategy describes the way an organization wants to achieve its vision and objectives and wants
to obtain a competitive advantage (Parmenter, 2011). A strategy consists of both a strategy
formulation and a strategy implementation (Henderson & Venkatraman, 1993). The first regards
choices concerning which path to take to achieve the organization’s objectives, while the latter
ensures that the chosen road is followed and monitored to deliver the desired outcomes
23
(Mintzberg, Lampel, Quinn, & Ghoshal, 2003). Strategy implementation and communication are
frequently underdeveloped in an organization (Ferreira & Otley, 2009).
As the establishment of strategic goals and the development of their respective action plans are
the most critical steps in the management process, it is essential to write meaningful and
effective goals (Doran, 1981). It has been proven that specific and challenging goals lead to
better task performance than vague or easy goals (Locke & Latham, 1990). Doran (1981)
emphasized the fact that “there is a SMART way to write management’s goals and objectives”.
The most common meaning of a SMART goal is one that is Specific (target a specific area for
improvement rather than a general one), Measurable (obtain concrete criteria for measuring
progress towards the attainment of the goal), Attainable (create realistic and achievable goals),
Relevant (choose goals that matter), and Time-bound (specify when the results can be achieved
within a time-frame) (P. Meyer, 2003). The acronym does not imply that every objective will have
all five criteria, but the closer a company gets to the SMART criteria as a guideline, the smarter
the objectives will be (Doran, 1981). Sometimes two extra criteria, namely Evaluated and
Reviewed (or Recorded), are added to ensure that targets will not be forgotten, and thereby
SMART becomes SMARTER (Markovic & Kowalkiewicz, 2008; Yemm, 2012).
Many organizations act in the wrong direction by adapting their strategies to their existing and
established measurement instruments. However, the development of KPIs should start with the
strategy and the goals an organization wants to fulfil (Marr, 2012).
3.2.4 Organization Structure
Ferreira & Otley (2009) ask the following question in their twelve-question framework: “Question
three: What is the organization structure and what impact does it have on the design and use of
PMASs? How does it influence and how is it influenced by the strategic management process?”
The organization structure is linked with both the CSFs and the strategic decisions (Ferreira &
Otley, 2009). It can be seen as a constraint on the strategic management process and the
design and use of PMASs, as it is an essential control element that should be taken into account
in the design and implementation phase (Ferreira & Otley, 2009). Organization structures
determine who is responsible and accountable for which activities (Ferreira & Otley, 2009). They
can be classified on many ways, such as the degree of centralization or decentralization of
authority, the level of differentiation or standardization, and the degree of formalization of rules
and procedures (Johnson et al., 2008).
24
3.2.5 Goals and Organization Structure in SMEs
Employees and management of SMEs have only little time for strategic issues, because they are
fully occupied with daily matters (cf. Chapter I, characteristic (1)). Thereby, PM initiatives are
mostly introduced to solve a specific problem (Hudson et al., 2001). Nevertheless SMEs work in
a dynamic and rapidly changing environment, so their strategies have to be responsive and be
able to change quickly (Rompho, 2011). Defining the strategy should therefore be the first step
in designing a PMAS (Tenhunen et al., 2001).
The organizational structure of an SME is typically defined by its size, competitive position and
individual management practices, which all greatly differ from larger companies (Hudson et al.,
2001). Due to the limited resources in SMEs, significant difficulties can arise during the
implementation of a resource intensive system (McAdam, 2000). To overcome these structural
difficulties, the development process of the PMAS for the CHOOSE approach for SMEs in PART
III should offer the following features (Hudson & Smith, 2007):
Very resource efficient development to ensure viability (cf. Chapter I, characteristic (2));
Short-term as well as long-term benefits, to help maintain the enthusiasm of the
developments team over time (cf. Chapter I, characteristic (9));
The ability to surface informal strategies to overcome limited strategic capabilities
(Chapter I, characteristic (7)); and
Dynamic and flexible development to accommodate strategic changes and ensure
continued strategic relevance (cf. Chapter I, characteristic (7)).
3.3 Performance Measurement
To start, Section 3.3.1 will cover the theory of Key Performance Indicators. Afterwards in Section
3.3.2, six well-known PMESs will be discussed, followed by three systems especially developed
for SMEs.
3.3.1 Key Performance Indicators
3.3.1.1 Concept and Advantages
Ferreira & Otley (2009) ask the following question in their twelve-question framework: “Question
five: What are the organization’s key performance measures deriving from its objectives, key
success factors, and strategies and plans? How are these specified and communicated and
what role do they play in performance evaluation? Are there significant omissions?”
25
The difference between a regular measure and a Key Performance Indicator (KPI) (or Key
Performance Measure) is that the latter embodies a strategic objective and measures
performance against a multidimensional target (Eckerson, 2009). A performance measure can
only be a KPI if it is linked to a CSF and a strategic objective of the organization (Parmenter,
2011). According to Ferreira and Otley (2009), KPIs are “the financial or non-financial measures
used at different levels in organizations to evaluate success in achieving their objectives, CSFs,
strategies and plans, and thus satisfying the expectations of different stakeholders”. Appropriate
performance measures therefore have to be identified during the strategic implementation
process and have to be derived from the organization’s objectives, CSFs, and strategies and
plans (Johnson et al., 2008). Furthermore, they need to facilitate the alignment between strategy
and operations (Ferreira & Otley, 2009). In Figure 12, a typical example of a KPI is given.
Figure 12: Example of a KPI (from (Niven, 2006))
The power of a complete set of KPIs is that they can not only illuminate where the organization
is today, but also project where the organization will be and provide visibility into where key
individuals can participate to correct the followed road of the organization if the forward looking
KPIs are off track (Paulen & Finken, 2009).
Nevertheless, some caution has to be taken as well. Measurement can become a source of
division and conflict between managers and employees, because these initiatives are often seen
as managerial control devices that are solely beneficial for management. As follows, the
measurement could have the adverse effect (Parmenter, 2011).
26
3.3.1.2 Characteristics
In this section, an overview is given of the characteristics of KPIs necessary to facilitate positive
change and to create desired outcomes. These characteristics will be taken into account during
the establishment of the Performance-oriented extension of the CHOOSE approach in PART III.
A comprehensive review of literature was undertaken by Neely et al. (1997) which resulted in a
set of twenty-two characteristics of KPIs (Neely, Richards, Mills, Platts, & Bourne, 1997). Yet,
Hudson et al. (2001) revealed that many of the characteristics were duplicates and they reduced
the set to seven critical characteristics. These seven characteristics are found in most other
literature (for instance (Band, 1990; Eckerson, 2009; Neely et al., 2002; Niven, 2006; Parmenter,
2011)). Nevertheless, some important characteristics are not mentioned by Hudson et al (2001),
and are added here as well. KPIs should:
(1) Be derived from strategy (Hudson et al., 2001);
(2) Be clearly defined with an explicit purpose (Hudson et al., 2001);
(3) Be relevant and easy to maintain (Hudson et al., 2001);
(4) Be simple to understand and use (Hudson et al., 2001);
(5) Provide fast and accurate feedback (Hudson et al., 2001);
(6) Link operations to strategic goals (Hudson et al., 2001);
(7) Stimulate continuous improvement (Hudson et al., 2001);
(8) Have top management support (Band, 1990);
(9) Be reliable, valid and attributable to the source (Chang & Morgan, 2001; Eckerson,
2009);
(10) Be stated in a challenging target (Chang & Morgan, 2001);
(11) All together give a balanced or multidimensional picture of the company (Cocca & Alberti,
2010).
Characteristics in SMEs
The same characteristics were found for SMEs in literature (Chalmeta et al., 2012; Cocca &
Alberti, 2010; Garengo et al., 2005; Hudson et al., 2001; Nudurupati & Bititci, 2011; Rompho,
2011). One extra characteristic is added by Cocca and Alberti (2010): the required data for the
measure should be easy to collect.
27
3.3.1.3 Attributes
Not only goals should be written in a SMART way (cf. Section 3.2.3), but KPI attributes should
also be represented the same way (Shahin & Mahbod, 2007). After extensive research, Cardoso
(2013a) discovered that a KPI requires the following attributes:
(1) Name and Description,
(2) Scale (determines the measurement scale for the KPI, where different scales exist),
(3) Measure (determines how the KPI values are calculated),
(4) Current Value,
(5) Target Value,
(6) Threshold (represents the deviation in relation to the target value that is acceptable for
the current value to reach),
(7) Source (the internal or external sources used to extract the performance indicators),
(8) Frequency (frequency of KPI measurement during a specific time interval),
(9) Responsible,
(10) Informed, and
(11) Owned.
The last three attributes link the KPI to the organizational structure (cf. Section 3.2.4): who is in
charge of measuring the KPI (Responsible), who is somehow interested in the KPI (Informed),
and who is responsible for the achievement (Owned). A KPI meets the SMART criteria by
incorporating attributes three (Specific and Measurable) and five (Achievable), and by being
related to a goal (Relevant) and deadline (Time-targeted). To decide whether these attributes
will be implemented in the Performance-oriented extension of the CHOOSE approach (cf. PART
III), their usefulness will be evaluated taking into account the intrinsic characteristics of SMEs.
3.3.1.4 Classification
Outcome versus Driver KPI
KPIs are frequently divided into two fundamental types: outcomes and drivers (Eckerson, 2009).
Outcome KPIs, also called lagging indicators, measure the output of past activity (Eckerson,
2009). These are results of many actions and give a clear picture of whether an enterprise is
doing fine or not, but they do not indicate how to improve these results (Parmenter, 2011). They
are often financial in nature (Eckerson, 2009) and are reported typically on a monthly/quarterly
basis (Parmenter, 2011). Frequently used examples are net profit before tax, margins, return on
28
equity, return on capital employed, customer satisfaction, and employee retention (Eckerson,
2009; Parmenter, 2011).
Driver KPIs, also called leading indicators or value drivers, measure activities that have a
significant impact on outcome KPIs (Eckerson, 2009). They are reported on a shorter time
period (weekly, daily or even continuously) (Parmenter, 2011). Driver KPIs always measure
activity in their current or future state (Eckerson, 2009). Although driver KPIs are more influential
as outcome KPIs, the latter are currently used twice as much in practice (Eckerson, 2009).
However, the use of driver KPIs is currently becoming more and more important. According to
Gartner Inc. organizations that use predictive business performance metrics will increase their
profitability by twenty percent by 2017. “Using historical measures to gauge business and
process performance is a thing of the past. To prevail in challenging market conditions,
businesses need predictive metrics — also known as "leading indicators" — rather than just
historical metrics (aka lagging indicators)." (Gartner, 2014)
On a strategic level, executives fulfil strategy and manage performance (Eckerson, 2009). As
outcome KPIs show the result of many actions and give a clear picture of whether you are doing
fine are not, these are perfect here (Eckerson, 2009; Parmenter, 2011). Lower levels on the
other hand, want to know more about the causes of good or bad practices, and want to optimize
the performance of people and processes: they need both short- and long-term information
(Eckerson, 2009; Parmenter, 2011). Hence, managers on a tactical level need both outcome
and driver KPIs. On an operational level, the staff monitors and controls processes on an
intraday basis, and therefore they need driver KPIs and operational measures (that do not
contribute to strategy) (Eckerson, 2009).
There has been some discussion about the difference between outcome and driver KPIs.
Eckerson (2009) argues that there is not a lot of difference, because an outcome KPI of one
person could be the driver KPI of the next one (Eckerson, 2009). Parmenter (2011) makes use
of different names. He states that KPIs only cover leading indicators and gives a different name,
i.e. Key Result Indicators (KRI), to lagging indicators (Parmenter, 2011). He specifies four types
of performance measures and makes a comparison with an onion (cf. Figure 13) to describe
their relationship: “The outside skin describes the overall condition of the onion (KRI), the
amount of sun, water, and nutrients it has received; how it has been handled from harvest to
supermarket shelf. However, as we peel the layers off the onion, we find more information. The
layers represent the various Performance Indicators (PIs) and Result Indicators (RIs), and the
core represents the KPIs (Parmenter, 2011, p. 1).”
29
Figure 13: Four types of performance measures (from (Parmenter, 2011))
Qualitative versus Quantitative KPI
Another possible way to classify KPIs is by making the distinction between qualitative and
quantitative KPIs. Quantitative KPIs measure activity by counting, adding, or averaging
numbers, such as financial KPIs (Eckerson, 2009). Quantitative data is most common in
measurement and therefore forms the backbone of most KPIs (The KPI Institute, 2014).
Qualitative KPIs are based on a subjective understanding of an opinion of an organizational
participant on various issues, such as a KPI on customer satisfaction (Eckerson, 2009). While
the survey data itself is quantitative, the measures are based on a subjective interpretation of a
customer’s or employee’s opinion (The KPI Institute, 2014).
3.3.1.5 Development and Use
Parmenter (2011) developed a twelve-step model to develop and use KPIs. It is crucial to create
a sound environment in which the organization can develop and use its KPIs, but many
organizations still fail to do this properly, because they do not prepare themselves correctly
(Eckerson, 2009; Parmenter, 2011). Therefore, using a methodology in developing and
implementing KPIs is essential (Eckerson, 2009).
Step one involves having the Senior Management Team (SMT) fully committed, which will create
a dynamic environment in which projects can prosper. Secondly, a small, well-trained winning
KPI project team with a large consultative value has to be established. As a third step, the SMT
and winning KPI project team have to create a “just-do-it” culture and process, which will
stimulate people that they can do it without only relying on experts. Fourthly, the KPI project
needs to be placed in a holistic strategy to achieve best practice. Step five contains advertising
the KPI system to all employees to make sure they are ready for the changes KPIs will bring in
the organization. The CSFs are identified in step six. Step seven contains recording
performance measures in a database. In step eight, performance measures on a team-level are
30
selected. Afterwards, the organizational KPIs will be chosen in step nine. A proper alignment
between lower and higher level KPIs is essential and will therefore be separately discussed in
Section 3.3.1.6. In step ten, a reporting framework has to be developed, which accommodates
the requirements and reporting frequencies at different levels. Step eleven makes sure that the
KPIs become widespread in an organization and not buried when staff members move on.
Finally, step twelve covers the notion of managing and refreshing the measurement system so
that it remains relevant and useful.
Development in SMEs
Parmenter (2011) argues that for SMEs the twelve steps can be truncated and merged into eight
steps, with a shorter timeframe (from twelve to six weeks) (cf. Figure 14). To start, step one and
four are merged. Secondly, Parmenter declares that a two-day CSF workshop would help selling
the concept of PM and delivering CSFs to the management and board (step one and six). Even
if the PM project would be cancelled, the workshop would attribute a lot. Thirdly, step two and
three are merged. Subsequently, step five, seven and eight are merged. Finally, step nine, ten,
eleven and twelve are established separately.
Figure 14: PM implementation model for SMEs (from (Parmenter, 2011))
3.3.1.6 Alignment of KPIs on different levels: the cascading concept
Performance measures are used on every level of the organization and every level needs
different measures (as explained in Section 3.3.1.4). The effectiveness of a measurement
system depends on the clear alignment between the measures on these different levels (Chang
& Morgan, 2001; Eckerson, 2009; Parmenter, 2011). This alignment process is mostly known
through the concept of cascading (Chang & Morgan, 2001; Eckerson, 2009): KPIs cascade from
the highest level to the lower levels of the organization, ensuring an aligned organization
31
(Eckerson, 2009). Once these cascading measures have been defined, they have to be mapped
in order to collectively view the relationships among all the KPIs and their impact on each other
(Eckerson, 2009). This results in a well-balanced and coherent view of the measures and a
correct measure for every responsibility (Chang & Morgan, 2001). Eckerson’s study (2009)
revealed that most organizations use two (30%), three (29%) or four (16%) levels of cascading.
Several types of cascading KPIs exist; Eckerson (2009) distinguishes four types. Generally, the
high-level KPIs are duplicated at lower levels. Going to a higher-level, KPIs are either summed
or averaged, and sometimes weighted averages or Boolean rules are used. Financial measures
are for instance mostly duplicated. A derived KPI is another method to cascade: a higher and
lower level KPI measure the same activity, but in different way. When different regions in a
global company for instance calculate their net sales differently, their calculations are
synchronized into one coherent KPI at a higher level. Thirdly, conglomerate KPIs comprise two
or more lower-level KPIs that are summed or averaged. The “research and development
expenses” KPI can for example be decomposed of the “total research expenses” KPI and the
“total development expenses” KPI. Finally, unique KPIs are measures that only exist at one
level.
Considered from a different point of view, Popova and Sharpanskykh (2011) identify three other
types of relationships between KPIs: causality, correlation and aggregation. A causality relation
expresses that the first KPI causes changes in the same or opposite direction to the second KPI.
A refinement relation between goals often corresponds with a causality relation between the
corresponding KPIs. A correlation relation expresses that a change in one KPI implies a change
in the same direction of the other KPI. An aggregation relationship corresponds with the
conglomerate relation stated above.
Another frequently used link is the customized relationship, which enables the qualification of
further customized relations (Cardoso, 2013b; Frank, Heise, Kattenstroth, & Schauer, 2008).
Cascading KPIs has three essential benefits. First of all, it permits executives to obtain an
overall view of performance across all groups and levels of performance. As a result, they can
monitor the execution of the strategy and intervene as needed (Eckerson, 2009). Furthermore,
employees at all levels will be able to understand how their efforts contribute to the whole
(Eckerson, 2009), which has an empowering and motivating effect, and ensures team ownership
of their performance measures (Parmenter, 2011). Thirdly, it stimulates friendly competition
among peer groups within an organization, which can improve productivity (Eckerson, 2009).
32
Alignment in SMEs
SMEs usually do not have the capacity to identify, design and quantify indicators in an integrated
way. Thereby, they are unable to establish relations among the indicators from the strategic to
the operational level and aligning them with the strategy of the enterprise (Chalmeta et al., 2012;
Hudson et al., 2001)
3.3.1.7 Number of KPIs on different levels
Defining the number of KPIs is an important issue, because the time of managers is limited and
more measures reduce the impact of each single measure (Ferreira & Otley, 2009).
Kaplan & Norton (1996a) recommend a maximum of twenty-five performance measures in their
Balanced Scorecard (explained in Section 3.3.2.4). Parmenter (2011) advises a guiding rule of
ten lagging KPIs, eighty PIs and RIs and ten leading KPIs. According to a study that Eckerson
performed (2009), most people can only focus on a maximum of five to seven items at once and
only a handful of metrics really impacts the desired outcomes. Therefore, he suggests that the
number of KPIs at the individual level should be limited to that number. Eckerson’s study (2009)
furthermore revealed that in practice on average sixteen KPIs exist at the executive level,
twenty-two at the business unit level, twenty-four at the department and workgroup levels, and
ten at the individual level.
3.3.2 Performance Measurement System
A PMES establishes a sound environment in which KPIs can operate and develop (Parmenter,
2011). As mentioned in Section 3.1.1, the PMES has to be balanced and dynamic and should
support the decision-making process by gathering, elaborating and analyzing information (Neely
et al., 2002).
Since the late 1980s and 90s, many academics have criticized the problems with the former
financial measurement systems, as they give an internal and historically based view of the
company (Kaplan & Norton, 1992; Keegan, Eiler, & Jones, 1989; Neely et al., 1997; Nudurupati
& Bititci, 2011). Traditional financial performance measures, which worked well for the previous
economic era dominated by tangible assets, were out of step at the end of the 20th century,
when intangible assets became the major source of competitive advantage (Kaplan & Norton,
1992). Exclusive reliance on financial indicators could promote behavior that sacrifices long-term
value creation for short-term performance (M. E. Porter & Summers, 1992). Furthermore, the
level of financial performance achieved today is a function of decisions made in the past (Neely
33
et al., 2002). As a result, many companies started to recognize non-financial measures such as
quality, customer satisfaction, and innovation (Nudurupati & Bititci, 2011). From that moment on,
the number of PMESs boomed. According to Neely (1999), from 1994 to 1996 there were more
than 3600 articles published on Performance Measurement.
The PMESs that contributed the most in literature and practice are discussed in the following
paragraphs, divided in six general PMESs and three specifically for SMEs. In PART III, one of
these PMESs will be chosen as the foundation for the Performance-oriented extension of the
CHOOSE approach, based on their specific advantages and disadvantages.
3.3.2.1 The Performance Measurement Matrix
The Performance Measurement Matrix is one of the first balanced models helping a company
define its strategic objectives and translating these objectives into performance measures
through a hierarchical and integrated approach (Keegan et al., 1989). Cost/non-cost
perspectives and external/internal perspectives are combined in a two-by-two matrix. The matrix
is cited for its simplicity and flexibility, but frequently criticized in literature as well as it does not
consider several important perspectives and relationships that are made explicit in other models,
such as the Balanced Scorecard (cf. Section 3.3.2.4) (Neely et al., 2002).
3.3.2.2 The Strategic Measurement And Reporting Technique
The Strategic Measurement And Reporting Technique (SMART) pyramid also incorporates the
need for both internally and externally focused measures of performance. It adds the notion of
cascading measures down the organization so that measures at department level reflect the
corporate vision and internal and external business unit objectives (Cross & Lynch, 1991).
3.3.2.3 The Result and Determinants Framework
The Result and Determinants Framework focuses on six dimensions, divided into results and
determinants of those results. The framework emphasizes the careful definition of performance
indicators needed to achieve the performance objectives and is developed especially for service
companies (Fitzgerald, Johnson, Brignall, Silvestro, & Voss, 1991).
3.3.2.4 The Balanced Scorecard
In 1992, Kaplan and Norton discovered that managers needed a balanced presentation of both
financial measures (results of actions already taken) and operational measures (drivers of future
financial performance) and introduced a Balanced Score Card (BSC): a set of measures that
gives top managers a fast but comprehensive view of the business. The BSC delivers the
34
framework for the translation of the strategy to operational terms (Kaplan & Norton, 1996a) (cf.
Figure 15). For example, it demands that managers translate their general mission statement on
customer service into specific measures that reflect the factors that really matter to customers
(Kaplan & Norton, 1992). The BSC puts strategy, mission and vision at the center instead of
control, and the objectives and measures are always deducted from these concepts (Kaplan &
Norton, 1996a). The BSC establishes goals, but assumes that people will adopt whatever
behaviors and take whatever actions needed to arrive to those goals (Kaplan & Norton, 1992).
The BSC provides an answer to four basic questions, classified in four perspectives (Kaplan &
Norton, 1992):
(1) Financial Perspective: How do we look to shareholders?
(2) Customer Perspective: How do customers see us?
(3) Internal Processes Perspective: What business processes must we excel at?
(4) Learning and Innovation Perspective: Can we continue to improve and create value?
Figure 15: Using the Balanced Scorecard as a strategic management system (adapted from (Kaplan & Norton, 1996b))
35
The BSC includes both the lagging indicators of (1) financial performance and (2) customer
value proposition, and the leading indicators of (3) internal processes and (3) learning and
growth (Kaplan & Norton, 2004)(cf. Section 3.3.1.4).
The BSC is balanced on several aspects. Firstly, it enables companies to track financial results
while simultaneously monitoring progress in building the capabilities and acquiring the intangible
assets they need for future growth (Kaplan & Norton, 1996b). Secondly, it gives comprehensive
information from four different perspectives, while it also minimizes information overload by
limiting the number of measures used (Kaplan & Norton, 1992). As mentioned before in Section
3.3.1.7, Kaplan and Norton suggest a maximum of twenty-five performance measures in one
scorecard (Kaplan & Norton, 1996a).
Critics on BSC
Several authors (such as (Chang & Morgan, 2001; Marr, 2012; Parmenter, 2011)) argue that the
four perspectives of the BSC are rather limited and should therefore be enlarged. Parmenter
(2011) adds two new perspectives: employee satisfaction and environment/community. Chang
and Morgan (2001) do not use a fixed number of perspectives in their performance scorecards,
neither a fixed name per perspective: it is the company itself who should choose this, and
thereby align the perspectives with its current and future strategies. In their more recent work,
Kaplan and Norton (2003) acknowledge themselves that the four perspectives should be seen
as a template that admits extensions.
Furthermore, Neely et al (2002) criticize the fact that the BSC does not incorporate a broad
enough view of the stakeholders who interact with the organization. For instance, no mention of
end-users, employees, suppliers, regulators, pressure groups, or local communities is made.
Kaplan and Norton (2003) argue that employees are already included in the learning and growth
perspective, and suppliers as well, if strong connections exist. Furthermore, they claim that only
those stakeholders who have causal relationships with the strategy of the organization are
admitted in the BSC.
BSC in SMEs
The BSC is very popular in large organizations (Kaplan & Norton, 1996b, 2001, 2004), however
literature on the use of the BSC in SMEs is rare (Rompho, 2011). Most SMEs are not aware of
the BSC technique and the usage rate is very low compared to large organizations, although the
BSC is believed to be as beneficial for SMEs as it is for larger organizations (McAdam, 2000).
Due to limited resources (cf. Chapter I, characteristic (2)), SMEs frequently tend to focus on the
36
financial aspects before anything else when it comes to defining goals, decisions and outcomes
(Chalmeta et al., 2012). While a large organization benefits from the BSC implementation
through effective communication of their strategy, SMEs gain more from the description of
strategic objectives with priorities (Andersen, Cobbold, & Lawrie, 2001).
3.3.2.5 Strategy Maps
A Strategy Map is a general representation of the four perspectives of the BSC in a cause-and-
effect manner and facilitates the communication of direction and priorities across the enterprise
(Kaplan & Norton, 2004). It links strategic objectives across multiple perspectives to display
which objectives drive others. The Learning & Growth Perspective supports the Internal
Perspective that in turn realizes the Customer and Financial Perspectives (Kaplan & Norton,
2004) (cf. Figure 16). Each objective is represented by one or more KPIs (Eckerson, 2009).
Through the BSC, these goals are then extended to a set of targets using measures to evaluate
their achievement, and initiatives are identified to achieve these targets (Giannoulis, Zdravkovic,
& Petit, 2012).
Figure 16: The Strategy Map template (from (Kaplan & Norton, 2004))
37
3.3.2.6 The Performance Prism
Neely et al. (2002) point out that most of the currently used performance measurement
frameworks are only partial solutions. Therefore, they introduced the Performance Prism, which
is holistic in orientation and starts with the question ‘Who are our stakeholders and what do they
want and need?’. Afterwards, it prompts questions about what strategies are required to deliver
value to these stakeholders, what processes need to be put in place to execute these strategies,
and what capabilities are required to underpin these processes. The Performance Prism makes
the distinction between what the stakeholders want of the organization and what the
organization wants of its stakeholders.
3.3.2.7 Organizational Performance Measurement model
The Organizational Performance Measurement (OPM) model is developed especially for SMEs
and is based on three principles: (1) Alignment, i.e. the selected performance measures support
the alignment between people’s actions and the company strategy, (2) Process thinking, i.e. the
measurement system makes reference to the process monitoring, control and improvement
system, and (3) Practicability, i.e. at any level in the company there is a consistent process for
identifying measures that should be considered and for ensuring the quality and suitability of
data (Chennell & Dransfield, 2000).
3.3.2.8 Integrated Performance Measurement model for Small Firms
The Integrated Performance Measurement model, which is ‘a hybrid accounting system,
connecting the traditional view and the activity-based costing together in a causal chain’, was
specifically designed for SMEs as well (Laitinen, 2002). It is based on five internal dimensions
(costs, production factors, activities, products and revenues) used to monitor the whole
production process and two external dimensions (financial performance and competitiveness)
used to monitor the company’s position in its competitive context (Garengo et al., 2005; Laitinen,
2002).
3.3.2.9 Performance Measurement System in SMEs
Cocca and Alberti (2010) composed a list of the necessary features of a PMES especially for
SMEs. A PMES should: (1) be flexible, rapidly changeable and maintainable, (2) be balanced
(internal/external, financial/non-financial), (3) be synthetic, (4) be easy to implement, use and
run, (5) show causal relationships, (6) be strategically aligned, (7) be graphically and visually
effective, (8)be incrementally improvable, (9) be linked to a rewarding system, and (10) be
integrated with IS.
38
3.4 Target, Evaluation and Reward practices
3.4.1 Target Setting
Ferreira and Otley (2009) ask the following question in their twelve-question framework:
“Question six: What level of performance does the organization need to achieve for each of its
key performance measures (identified in the above question), how does it go about setting
appropriate performance targets for them, and how challenging are those performance targets?”
As already stated above (cf. Section 3.3.1.1), the difference between a regular measure and a
KPI is the fact that the latter embodies a strategic objective and measures performance against
a multidimensional goal. This goal associated with KPIs is known as a target because it requires
a measurable outcome rather than a conceptual destination (Eckerson, 2009). Research has
confirmed that performance is affected by the target levels and that especially moderately
difficult targets enhance performance (Fisher, Peffer, & Sprinkle, 2003). The difficulty of target
setting lies in making the distinction between what is desired and what is thought to be feasible
in determining targets for all aspects of organizational performance (Otley, 1999).
Several types of targets exist. Eckerson (2009) mentions five:
(1) Achievement, used when performance has to reach or exceed the target and anything
above the target is valuable but not required (e.g. revenues and satisfaction);
(2) Reduction, used when performance has to reach or be inferior to the target and anything
less than the target is valuable but not required (e.g. overtime and attrition);
(3) Absolute, used when performance should exactly equal the target and both a higher or
lower value are unfavorable (e.g. in-stock percentage and on-time delivery);
(4) Minimum/Maximum, used when performance should be within a range of values and
anything above or below the range is unfavorable (e.g. mean time between repairs); and
(5) Zero, used when performance should equal zero, which is the minimum value possible
(e.g. employee injuries and product defects).
The target to which the KPI is compared is multidimensional because a target has ranges,
encodings, thresholds, time frames and benchmarks (Eckerson, 2009). Ranges, which are
usually percentage increments above or below target, are used to help workers assess their
performance. Most organizations have three ranges: above target, on target, or below target.
These ranges are encoded in software to enable the visual display of performance. A color
coding could for instance be used, where red stands for below target, orange for on target and
green for above target (in an Achievement target type). The boundary lines between ranges are
39
called thresholds. Furthermore, time frames are assigned to targets: they stipulate when the
target must be accomplished and influence how KPIs are calculated. Generally annual time
frames are used and these long-term targets are often divided into smaller intervals to provide
milestones of performance on a more frequent basis. Targets are measured against
benchmarks, which are the starting point for improving performance. A typical benchmark is the
result of the previous year, which is an internal benchmark. However, external benchmarks,
such as the industry leader or closest competitor, are used as well (Hope & Fraser, 2003).
3.4.2 Performance Evaluation
Ferreira and Otley (2009) ask the following question in their twelve-question framework:
“Question seven: What processes, if any, does the organization follow for evaluating individual,
group, and organizational performance? Are performance evaluations primarily objective,
subjective or mixed and how important are formal and informal information and controls in these
processes?”
Managers focus the most on areas that senior management finds important, and achieving
success in these areas could positively affect the organization. The unit of evaluation can be an
individual, various groups of individuals (such as teams or departments) or the organization as a
whole (Ferreira & Otley, 2009). Furthermore, Grafton, Lillis, and Widener (2010) found that in
order to stimulate managers to use multiple financial and non-financial performance indicators,
performance evaluation schemes have to be designed to reflect these measures as well.
Performance evaluations can be objective, subjective or in-between those two (Ferreira & Otley,
2009). Objective evaluations only asses actual results, and the weightings placed on the various
dimensions of performance are known and measurable. In subjective evaluations on the
contrary, the evaluator determines the weightings. Recently, relative performance evaluations
become more popular, where the performance of an individual or entity is measured in relation
to that of another in order to eliminate distortions caused by uncontrollable factors.
Enterprises often take the easy way, by choosing to measure what is easy to measure, rather
than what is right to measure (Neely et al., 2002). However, some patterns of performance
measures can incite employees to act unsuitable, called the gaming behavior: employees make
a game out of the performance measures and use gaming tactics to fulfil the performance
measures (Ferreira & Otley, 2009; Neely et al., 2002). Especially when monetary incentives are
involved, gaming behavior increases (Eckerson, 2009). Neely et al. (2002) phrased this problem
remarkably well: “You certainly get what you measure, but you may not get what you want.”
40
3.4.3 Reward Systems
Ferreira and Otley (2009) ask the following question in their twelve-question framework:
“Question eight: What rewards — financial and/or non-financial — will managers and other
employees gain by achieving performance targets or other assessed aspects of performance
(or, conversely, what penalties will they suffer by failing to achieve them)?
Rewards range from financial rewards, to recognition by senior management, to long-term
progression or promotion, and can both be positive (i.e. reward) or negative (i.e. penalty). The
relationship between rewards, motivation and performance is complex and includes questions
such as whether to implement extrinsic or intrinsic rewards, or whether to use group or
individuals rewards (Ferreira & Otley, 2009).
3.5 Implementation of the PMAS
3.5.1 Information Flows, Systems, and Networks
Ferreira and Otley (2009) ask the following question in their twelve-question framework:
“Question nine: What specific information flows -feedback and feedforward-, systems and
networks has the organization in place to support the operation of its PMASs?”
Without information flows, systems, and networks, a PMAS cannot properly function (Otley,
1999). According to Ferreira and Otley (2009) “they act like the nervous system in the human
body, transmitting information from the extremities to the center and from the center to the
extremities” (Ferreira & Otley, 2009, p. 273). Systems are a part of the IS and IT infrastructure
and are used to organize accounting and other information (Ferreira & Otley, 2009). Systems for
financial information are well-developed, but this is not always the case for those for non-
financial measures (Ferreira & Otley, 2009). Dechow, Granlund, and Mouritsen (2006)
emphasize the mutual and interdependent relationship between accounting and IT, where
accounting needs IT to report and manage performance, and IT needs accounting to justify its
existence. Systems are often organized in networks. Both formal and informal networks play an
important role in the distribution of information, and the degree of formality depends on the
organizational culture (Ferreira & Otley, 2009).
Information flows in SMEs
Innovation in IT increases the opportunity to create a relationship between PM and a company’s
IS, and thereby decreases the costs of implementing a PMAS (Neely, 1999). This is particularly
41
important for SMEs, as they have limited financial resources (cf. Chapter I, characteristic (2)).
Furthermore, since knowledge is mainly tacit and context-specific, the information required to
implement and use a PMAS is difficult to gather (Marchini, 1995).
3.5.2 PMASs Use
Ferreira and Otley (2009) ask the following question in their twelve-question framework:
“Question ten: What type of use is made of information and of the various control mechanisms in
place? Can these uses be characterized in terms of various typologies in the literature? How do
controls and their uses differ at different hierarchical levels?”
The use made of information and control is a cornerstone of PMASs (Ferreira & Otley, 2009).
Case studies even suggest that the use of control information can be more significant than the
formal design of the control system (Ferreira, 2002).
PMASs Use in SMEs
Keeping PMASs continuously updated is challenging for every organization, but particularly for
SMEs: on the one hand they need to be extremely flexible and reactive to market changes ,while
on the other hand they are characterized by lack of resources and managerial expertise
(Garengo et al., 2005). Research through case studies has shown that, although there is a
widespread understanding of the importance of strategic PM among SME managers, very few of
them act on it, which indicates substantial barriers for implementation (Hudson et al., 2001).
Most of the time, the process is described both too resource intensive, which contrasts with the
low resource availability of SME (Chapter I, characteristic (2)), and too strategically oriented,
which contrasts with the more dynamic, emergent strategy styles of SMEs (Chapter I,
characteristic (7)). Furthermore, SMEs have a lack of human resources, which means that all the
staff involved in the activities of managing daily work have no extra time for additional activities,
such as implementing a PMAS (Chapter I, characteristic (1)).
3.5.3 PMASs Change
Ferreira and Otley (2009) ask the following question in their twelve-question framework:
“Question eleven: How have the PMASs altered in the light of the change dynamics of the
organization and its environment? Have the changes in PMASs design or use been made in a
proactive or reactive manner?”
Environments and organizations change continuously, so PMASs have to change as well to stay
relevant and useful. The idea of change both concerns the design infrastructure that builds the
42
PMAS (such as the KPIs), and also the way PM information is used (such as which aspects are
emphasized and which are not) (Ferreira & Otley, 2009). Critical to consider as well, is the scope
of strategic change in the current increasingly competitive environment, as strategy is the core
component of a PMAS (Ferreira & Otley, 2009).
PMASs change in SME
PMASs in SMEs are frequently introduced in a reactive, spontaneous manner to solve specific
problems (Hudson, Bennett, Smart, & Bourne, 1999). Consequently, performance measurement
is often characterized by a poor alignment between the measures and strategy (Garengo et al.,
2005). Furthermore, McAdam (2000) remarks that managers of SMEs are sceptic about the
long-term benefits of change, as they are faced with an ever changing environment. Therefore,
he suggests that improvement projects should always have adequate short-term benefits next to
the long-term potential.
3.5.4 Strength and Coherence
Ferreira and Otley (2009) ask the following question in their twelve-question framework:
“Question twelve: How strong and coherent are the links between the components of PMASs
and the ways in which they are used (as denoted by the eleven questions above)?”
It is important that all the individual components of the PMAS fit well together, otherwise control
failures can occur (Ferreira & Otley, 2009). Chenhall (2003) proposes that the strength and
coherence of PMASs should be evaluated on the extent to which the control system “considers
multiple stakeholders; measures efficiency, effectiveness and equity; capture(s) financial and
non-financial outcomes; provides vertical links between strategy and operations and horizontal
links across the value chain; provides information on how the organization relates to its external
environment and its ability to adapt” (Chenhall, 2003, p. 136).
43
CHAPTER IV: LINKS BETWEEN EA AND PM
In most existing approaches the relationship between performance indicators and goals remains
implicit (Popova & Sharpanskykh, 2008), although they facilitate the alignment between strategy
and operations (Ferreira & Otley, 2009). Thereby, implementing a PMAS could further
encourage the translation from strategy to operations, which is one of the main features of EA
(R. Maes, 2007). Cardoso (2013a) made a review of the enterprise modelling approaches and
EA frameworks that support modelling KPI-related concepts. There are already some EA
approaches who acknowledge the existence of KPIs as means of quantifying the organizational
performance (Frank et al., 2008; Iacob, Quartel, & Jonkers, 2012; Kronz, 2006; Lankhorst,
Quartel, & Steen, 2010; Popova & Sharpanskykh, 2008). Yet, a considerable gap exists on
modelling performance indicators and integrate them into modern enterprise modelling
frameworks, and thereby there is not much automated support for KPI measurement and
measurement of goal achievement (Cardoso, 2013a). The remainder of this chapter will first
describe the relationship between KPI-related concepts and EA elements, and afterwards
present two EA metamodels that incorporate performance measures.
4.1 Relationship between KPI-related concepts and EA elements
Cardoso (2013a) proposes a top-down approach: start by modelling the relevant goals,
afterwards derive appropriate KPIs from these goals, and finally link these with other EA
elements. Several proposals are described to measure the KPI values to evaluate the
achievement of goals. These measures represent a particular way of collecting information
about the properties of the elements of the EA to obtain the value of a given KPI. There are
several different proposals to formulate the measures, which can for example be distinguished
according to the following dimensions: number of instances, type of measure, type of scope,
type of monitoring object and monitoring point (Cardoso, 2013a). Monitoring objects represent
the association between the KPI language and the other viewpoints of the enterprise, and are
used as EA elements (Cardoso, 2013a). They represent the architectural elements whose
properties are measured and are the elements where the monitoring operates on to have the
values of the KPI determined. Monitoring points are required to perform the actual measurement
of the monitoring objects. They define the points in the process from which the data is gathered
to calculate the KPIs (Kronz, 2006), and can be attached to any monitoring object on which it is
intended to gather information.
44
4.2 EA Metamodel of Braun and Winter
In 2005, Braun and Winter developed a metamodel framework for EA which is constructed of
four architectural layers, with different design goals for each layer: the Strategy layer, the
Organization layer, the Application layer and the Software Component layer. Particularly
important is the mapping of the interdependencies between the models on different framework
layers in a suitable way (Braun & Winter, 2005). Figure 17 gives a simplified overview of the
framework’s metamodel of the first three layers1.
Figure 17: Aggregate framework metamodel (from (Braun & Winter, 2005))
In the Strategy layer metamodel, a distinction is made between several different views, marked
with different shades of grey (Figure 18). One of these views is the Balanced Scorecard (BSC)
view (in white), which is used to specify performance indicators that serve as a foundation for the
design and PM of business processes. A Strategy defines a Goal. Goals have CSFs, and are
realized through Actions. The implementation of an Action is the responsibility of a particular
Organizational Unit. The Goal, Action and Organizational Unit classes show a great
resemblance with the current CHOOSE approach (cf. 2.6). Therefore, the metamodel of Braun
and Winter will be very useful during the solution phase in PART III.
1 Regular arcs represent references between constructs, dotted arrows represent primary dependencies
between framework layers, and broken arrows represent aggregation relationships between constructs.
45
Each CSF is operationalized by one or more Indicators. On the one hand, the importance of an
optimal aggregation of Indicators is stressed, where the number of indicators should be reduced
to a minimum and the indicators themselves should be kept as simple as possible. On the other
hand, the Indicators should always reflect the real state of affairs as truthfully as possible. As
individual Indicators can be interpreted wrong, it is advised to use several indicators for one
CSF, integrated in one system like the BSC (Braun & Winter, 2007).
Figure 18: The Strategy Layer metamodel (from (Braun & Winter, 2005))
The Organization layer metamodel is built of several parts as well (cf. Figure 19). The relevant
part for PM, is the Process Control-part (in white). A CSF on the organizational level describes a
performance-critical feature of a process. It is derived from the business strategy and
operationalized by one or more Indicators. The Process Goal, Indicator and Success Factor are
summarized in the Report, which represents the causes of the deviation between and the extent
of the targeted and actual results and goes to one or more Organizational Units. The
organizational performance indicators are derived from the KPIs/CSFs defined in the BSC view
on the Strategy layer (Braun & Winter, 2005). In fact, the same KPI might be used on both
Strategy and Organization layer, but in a different way, e.g. for strategic planning/controlling in
the first, while for process PM in the latter (Winter, 2011).
46
Figure 19: The Organization Layer metamodel (from (Braun & Winter, 2005))
There is an important interdependency between the metamodel construct on the Strategy layer
and on the Organization layer of the framework. The BSC defines targets in terms of CSFs and
performance indicators (Strategy layer) which form the foundation for the design of process
management on the Organization layers. They are supposed to guarantee the adherence to
process goals (Braun & Winter, 2005). Furthermore, the performance indicator at the
Organization layer is linked with a monitoring point at the Application layer.
More recently, a general EA metamodel was developed (cf. Figure 20) where PM is investigated
from a Business Engineering perspective (Winter, 2011). The same classes arise and in addition
there is also made notion of a Target Value class. However, Winter argues that target setting is
only implicitly incorporated in the metamodel, and requires a more comprehensive description in
order to use it for strategic leadership and management of business processes (Winter, 2011).
The goals of an organization that are defined on a very aggregate level (for instance in a BSC),
are subsequently decomposed into more and more specific Success Factors and KPIs, resulting
47
in a multi-layer goal/indicator aggregation hierarchy. When target values and actual values differ,
appropriate Actions can take place (Winter, 2011).
Figure 20: Goal-Performance part of core metamodel (translated from (Winter, 2011))
4.3 Unified Business Strategy Metamodel of Giannoulis et al.
Giannoulis et al. (2012) cite that business strategy formulations are frequently overlooked in EA
proposals, although the business strategy should be clear in order to support an enterprise in
achieving its vision and in defining an architecture that supports that vision. Due to this
ignorance, IT solutions cannot be traced back to business strategy in a clear and unambiguous
way (Giannoulis et al., 2012). To establish a link and thereby solve the problem, they propose
the Unified Business Strategy Meta-Model (UBSMM). The UBSMM is a formalization of the
rigorous integration of two metamodels established in previous works: Strategy Maps and BSC
metamodel (SMBSC) (Giannoulis, Petit, & Zdravkovic, 2011a, 2011b) and the Value
Configuration meta-model (VC) (Svee, Giannoulis, & Zdravkovic, 2011). The metamodel,
formalized in UML, is depicted in Figure 21. As will be shown in the next paragraph, PM is
included in EA by linking Business Strategy to EA.
48
Figure 21: The UBSMM (from (Giannoulis et al., 2012)
From a PM perspective, especially the StrategicGoal, Objective, Measure, Milestone, Target,
Value Activity and Actor classes are interesting. StrategicGoal captures the defined goals. The
causality relations between different goals are captured through the self-association (influences,
isInfluencedBy) between Goals. Objectives refer to measurable goals that are used for building
BSCs, which suggests that not necessarily all goals are used to build a BSC. Through the
generalization, the transition of goals from strategy maps into BSC is captured. Measure
supports the evaluation of the achievement of an Objective. A final, long-term Target must be
defined for each Measure determining whether the Objective has been achieved. The Target
specifies a deadline and a threshold (desired value for the milestone to be completed) attribute
for the Measure. A Milestone is any short-term or intermediate target needed prior to the final
target. A Value Activity encompasses all actions or activities required to achieve an Objective
that is derived from a Strategic Goal (Giannoulis et al., 2012). It carries a resource attribute,
which captures both time and money available for an activity. The Value Activity Strategic
Compliance captures whether a value activity actually delivers value and fits with the strategy.
Actor captures the organization, unit or individual that defines a strategy or that performs a value
activity. Those could be the same, but do not have to be necessarily (Giannoulis, 2011).
49
A number of constraints are worth mentioning as well. For Goals in the financial perspective of
the BSC, no initiatives are launched because targets capture the results of initiatives from the
other perspectives. Consequentially, a Measure belonging to the financial perspective, has no
initiatives. Furthermore, a Measure can have several Milestones, but has only one Target
(Giannoulis, 2011).
4.4 Gaps of Research
To the knowledge of the author and according to Cardoso (2013a), a gap exists in the current
literature regarding which level of goal abstraction should be associated with KPIs.
Nevertheless, it is known that in order to link Goals to KPIs, these Goals should be written as
Objectives, which are SMART statements of intent whose achievement supports a goal (Object
Management Group, 2013).
50
PART II: PROBLEM STATEMENT
The concept of Goals is the most important part of an EA approach (cf. PART I, Section 2.3). A
goal can be considered as an end (why) for a refined goal, as well as means (how) for a goal at
a higher level (Markovic & Kowalkiewicz, 2008). The higher-level goals are refined into more
specific goals through AND or OR-decompositions, where the goals are decomposed into
alternative sub goals (OR) or combinations of sub goals (AND) that must be satisfied. However
at the same time, a goal at each refinement level also describes what needs to be done (cf.
Figure 22). Thereby, the goal is described in more detail: it has for example an assigned
measure for controlling the progress, a deadline for completion and a priority.
Figure 22: Why, How and What of a goal (from (Markovic & Kowalkiewicz, 2008))
The current CHOOSE metamodel incorporates the why-part and how-part of the goal
decomposition through the Refinement and Goal constructs of the Goal dimension, but does not
include the what-part of the goals. It is important to understand the difference between the what-
part of the goal dimension described above and the what-dimension in the current CHOOSE
approach. The current CHOOSE approach distinguishes between four separate dimensions
(Bernaert, Poels, et al., 2013):
A Goal dimension (Know-Why), for the motivational part, where the goals are refined
through why and how questions (so no what questions are currently asked here).
An Object dimension (Know-What), for the description of the concepts and relationships.
An Operation dimension (Know-How), for the behavioral part.
An Actor dimension (Know-Who), for the active performers.
51
Without measuring the fulfilment of the goals, the controlling dimension of EA remains pointless
(Matthes & Monahov, 2012). KPIs evaluate the performance of organizational operations and
use this information to measure the achievement of operational and strategic goals (Popova &
Sharpanskykh, 2011). Furthermore, KPIs are an important part of EA, representing the
relationships between organizational structure, business processes, applications, and
infrastructure elements (Wittenburg, 2007).
Abundant EA approaches are already described in literature, but very few of them incorporate
PM even though its importance has been argued (Braun & Winter, 2005; Cardoso, 2013a;
Wittenburg, 2007). Several EA proposals (such as ARIS (Scheer, 2000), ArchiMate 2.0 (The
Open Group, 2012), and TOGAF (The Open Group, 2011)) include performance measures in
their models without any information on performance management. While the PMESs (such as
the BSC, the Performance Prism, the SMART pyramid, etc. discussed in PART I, Section 3.3.2)
provide a strong methodological support for suggesting, deriving and adapting indicators, they
only specify the indicators and the goals derived from these indicators in a natural language,
without specifying important information concerning a goal-based measurement of indicators
(Cardoso, 2013a). If the indicators and goals are not represented through a (formal) modelling
language, this could cause problems regarding ambiguity, incompleteness, and traceability.
Braun and Winter (cf. PART I, Section 4.2) incorporate Measures, Targets, Required Actions,
Reports, Monitoring Points and links with Processes in their EA metamodel, but argue that the
target setting should be elaborated (Braun & Winter, 2005; Winter, 2011). Giannoulis et al. (cf.
PART I, Section 4.3) integrate Objectives, Measures, Milestones, Targets, and ValueActivities in
their metamodel in order to link Business Strategy with EA (Giannoulis et al., 2012). In addition,
Cardoso (cf. PART I, Section 4.1) describes the relationship between KPI-related concepts and
EA elements (Cardoso, 2013b).
To the knowledge of the author, there has never been developed an EA that incorporates a
PMAS especially for SMEs (Cardoso, 2013a). On the one hand, there have been a few
proposals for PM in SMEs (such as (Cocca & Alberti, 2010; Garengo et al., 2005; Hudson et al.,
2001; Hudson & Smith, 2007; Nudurupati & Bititci, 2011; Rompho, 2011)), but none of them are
linked to EA. On the other hand, Bernaert et al. developed the CHOOSE approach (cf. Section
2.6) as an EA technique for SMEs (Bernaert, Poels, et al., 2013). Therefore, this master thesis
intends to provide the link between those two, by developing a PMAS for the CHOOSE
approach for EA for SMEs.
52
PART III: SOLUTION
INTRODUCTION
The CHOOSE approach consists of four dimensions, with corresponding constructs and
relationships: a Goal dimension (why), an Actor dimension (who), an Operation dimension (how)
and an Object dimension (what) (Bernaert, Poels, et al., 2013). Furthermore, the CHOOSE
approach comprises three artefacts: the CHOOSE metamodel, the CHOOSE method and the
CHOOSE tool support. The CHOOSE method consists of three parts: the Roadmap method, the
Interview method, and the Stop criteria (Bernaert, Callaert, & Poels, 2013). The CHOOSE
approach is based on the five criteria of a good EA technique (cf. PART I, Chapter 2.6), where
the criteria of “Suitability for its target audience” is translated in Kept Simple which contains six
SME-related criteria (cf. Figure 5). In previous research (Bernaert, Callaert, & Poels, 2013;
Bernaert et al., 2014) the metamodel and method are refined by action research through several
case studies.
This master thesis will provide a PMAS for the CHOOSE approach, starting from the existing
CHOOSE approach (Bernaert, Callaert, Poels, et al., 2013). The Performance-oriented
extension is an optional add-on for the existing CHOOSE approach for SMEs that are facing
increased complexities and perceive the need to measure the achievement of their goals. It will
mainly focus on the Control criteria of a good EA and further enhance the Objective criteria (i.e.
translating the corporate strategy into operations). Furthermore, the Suitability for SMEs has to
be kept in mind as well.
The main objective of the CHOOSE approach is to provide an EA approach for SMEs. During
this realization, some efforts have already implicitly been made to incorporate PM. First of all,
the Performance relationship between the actor and the operation is characterized by an
association class including the attribute Type, to enable the use of a RACI chart (Bernaert,
Poels, et al., 2013). RACI stands for (1) Responsible: who is responsible for the activity, (2)
Accountable: who is accountable for the activity, meaning who provides direction and authorizes
the activity, (3) Consulted: who should be consulted for the activity, and (4) Informed: who must
be informed about the activity. However, Bernaert et al. (2013) still give the SME the opportunity
to choose another assignment matrix (such as RASCI, CAIRO, etc.). Secondly, a PM
framework, namely the BSC of Kaplan and Norton, is used in the CHOOSE method in order to
elicit the highest-level Goals as holistically as possible, and afterwards find the lower-level
53
Goals. However, the BSC is currently only used to display the goals and is not used as a
framework for the measures. Thirdly, the CHOOSE approach gives the possibility to give an
integrated as-is/to-be view of the enterprise through the two different types of links between the
Goal class and the Refinement class. As-is and to-be Goals can be modelled using respectively
an AND- or OR-Refinement.
Bernaert, Poels, et al. (2013) started the CHOOSE approach with the metamodel, and secondly
developed the method. Additionally, Cardoso articulates that in order to obtain an aligned PMAS,
researchers must rely on a metamodel and systematic methodology (Cardoso, 2013c).
Therefore, both the metamodel and method will be discussed in this master thesis. The third
artefact of the CHOOSE approach, namely the CHOOSE tool support, goes beyond the
research domain of this master thesis.
To start, in Chapter I the current CHOOSE metamodel and method will be elaborated with a
Performance-oriented viewpoint based on an extensive study of existing literature on EA, PM
and on the link between EA and PM (cf. PART I). An overview of the extensions, the reasons for
extensions and relevant literature are summarized in Table 1. When several different
possibilities arose in literature, the chosen solution is highlighted in bold. The choices were
made according to the five design principles of Paige, Ostroff and Brooke (2000) to verify the
utility and effectiveness of a Unified Modelling Language (UML): seamlessness, uniqueness,
consistency, simplicity and reliability, and according to the exhaustiveness criteria proposed by
Bernaert, Poels, et al. (2013).
Afterwards in Chapter II, the established metamodel and method will be evaluated and refined
through a Case Study research as Action Research, where the choices between the different
possible refinements are also made according to the five design principles of Paige et al. (2000)
and the exhaustiveness criteria of Bernaert, Poels, et al. (2013). During the Action Research, the
researcher is an active participant rather than a passive observer: it is a cyclical process of
actively participating in an enterprise change situation while doing research (Susman & Evered,
1978).
Lastly in Chapter III, the final solution will be given, consisting of the final CHOOSE metamodel,
the final extra CHOOSE constraints and the final CHOOSE method.
54
METAMODEL Extension Reason Literature
1 Performance-oriented viewpoint as fifth dimension
- Control complexity (Bernaert, Poels, et al., 2013; Braun & Winter, 2005; The Open Group, 2003)
2 Accommodation of all strategic terms in Goal class
- Simplicity principle - Consistency principle
(Bernaert & Poels, 2011b; Chalmeta et al., 2012; Hudson et al., 2001; Tenhunen et al., 2001; Vichitdhanabadee et al., 2009) (Braun & Winter, 2005; Giannoulis et al., 2012)
3
Success Factor class as optional specialization of Goal class for those goals that can be measured
- Cost vs. Benefits - Focus on Control - Some goals do not need
measurement - Commonly used term
(Bernaert & Poels, 2011b; Braun & Winter, 2005; Brooks & Chittenden, 2012; Ferreira & Otley, 2009; Giannoulis, 2011; Hudson et al., 2001; Kaplan & Norton, 2004; Parmenter, 2011)
4
Indicator class rather than metric, measure, … class with Measurement relation with Success Factor class
- Uniqueness principle - Simplicity principle - Integration with existing metamodel
(Braun & Winter, 2005) (Eckerson, 2009)
(Giannoulis & Zikra, 2013) (Ferreira & Otley, 2009) (Parmenter, 2011)
5 Association class for Measurement relationship with Contribution attribute
- Indicates level of contribution of Indicator to Success Factor
(Pourshahid et al., 2007)
6 Attributes of Indicator class: Name, Description, Value, Frequency and Type
- Consistency principle - Simplicity principle - Widely known
(Cardoso, 2013a)
7 Source relation between Indicator and Object classes
- Integration with existing viewpoints - Consistency principle - Simplicity principle
(Cardoso, 2013a)
8
Reflexive Includes relation with association class with Type attribute for Indicator class
- Simplicity principle - Exhaustiveness - Flexibility for SME
(Eckerson, 2009) (Popova & Sharpanskykh, 2011) (Cardoso, 2013a; Frank et al., 2008)
9 Milestone class and optional Target specialization class
- SMEs require both long and short term results
(Eckerson, 2009; Giannoulis et al., 2012; McAdam, 2000) (Braun & Winter, 2005; Cardoso, 2013a)
10 Attributes of Milestone class: Value, Deadline and Reward
- Uniqueness principle (Cardoso, 2013a; Eckerson, 2009; Ferreira & Otley, 2009; Giannoulis et al., 2012)
11 Comparison relation with association class with Type and Threshold attribute
- Exhaustiveness principle - Flexibility for SME
(Cardoso, 2013a; Eckerson, 2009)
12 Achievement relationship between Milestone and Operation class
- Simplicity principle - Uniqueness principle - Integration between viewpoints
(Braun & Winter, 2005; Eckerson, 2009; Ferreira & Otley, 2009; Giannoulis et al., 2012; Kaplan & Norton, 1996a)
METHOD Extension Reason Literature
1 Start with Goal dimension - Most important part of EA - SMEs lack formalized strategy
(Bernaert & Poels, 2011a; Eckerson, 2009; Hudson et al., 2001; Kaplan & Norton, 1996a; Parmenter, 2011; Tenhunen et al., 2001)
2 BSC as goal-mapping framework
- Broaden strategic view - Widely known and used - Approved for SME - Goal-oriented - Coherence with existing method
(Bernaert et al., 2014; Kaplan & Norton, 1992, 1996a; Rompho, 2011; Schelp & Stutz, 2007; Solingen & Berghout, 1999)
3 Define Success Factors and write them in a SMART way
- -facilitates translation to SMART Indicators
(Doran, 1981; Ferreira & Otley, 2009; Kaplan & Norton, 2004; P. J. Meyer, 2003; Parmenter, 2011)
4 Find well-balanced equilibrium in time spent
- SMEs have not much prior knowledge
- Importance of motivated team - SMEs do not have many time for
strategic issues
(Chalmeta et al., 2012; Chang & Morgan, 2000; Eckerson, 2009; Parmenter, 2011) (Bernaert & Poels, 2011a; Tenhunen et al., 2001; Vichitdhanabadee et al., 2009)
5 Define Indicators, Milestones and Targets in accordance with soft constraints
- Avoid information overload (Cardoso, 2013a; Eckerson, 2009; Ferreira & Otley, 2009; Parmenter, 2011)
Table 1: Overview of the used literature
55
CHAPTER I: INITIAL SOLUTION AFTER LITERATURE REVIEW
1.1 CHOOSE Metamodel
Bernaert et al. discovered during their action research that even though the CHOOSE
metamodel regards SMEs, the CHOOSE models tend to become very large (Bernaert, Poels, et
al., 2013). To control this complexity, different viewpoints can be used. Furthermore, many other
EA models (such as (Braun & Winter, 2005; The Open Group, 2003)) do the same. For these
reasons, the PMAS will be introduced as the fifth viewpoint (next to the Goal, Actor, Operation
and Object viewpoint) in the CHOOSE metamodel: the Performance-oriented viewpoint.
Every PM approach of the studied literature mentions the importance of goal-related concepts
(such as mission, vision, values, goals, objectives, CSFs and strategies) and most approaches
even describe these concepts very extensively. The metamodel of Braun and Winter (2005)
distinguishes for example a Strategy, Goal, Success Factor and Perspective class (cf. Figure
18), and Giannoulis et al. (2012) define a Strategic Goal, Objective, Group, Strategic Theme,
Strategy Plan, … class (cf. Figure 21). However, literature concerning SMEs revealed that these
organizations often have limited time for strategic matters and have an informal and non-
documented strategy (cf. characteristic (1) and (7) of PART I, Chapter I). For this reason, and
according to the design principle of simplicity and consistency (Paige et al., 2000), the choice
was made not to implement all these different constructs, but simply accommodate all these
concepts in the current Goal construct of the CHOOSE metamodel. In this manner, the SME is
free to choose which concepts to use.
Not all goals can and have to be measured. First of all, the costs of doing so would be greater
than the benefits (cf. characteristic (5) of PART I, Chapter I). Furthermore, the Performance-
oriented extension focuses on the Control part instead of the Holistic Overview part of the EA
criteria (cf. PART I, Chapter 2.6). And thirdly, some (more general) goals do not require a
measurement (cf. PART I, Chapter 4.3). Therefore, it is desirable to implement a specialization
of the Goal class for those goals that can be measured. The term Success Factor will be used to
create a class that specifies the Goal class (as an optional specialization) (cf. Figure 23), as this
term is most commonly used (Braun & Winter, 2005; Brooks & Chittenden, 2012; Ferreira &
Otley, 2009; Kaplan & Norton, 2004; Parmenter, 2011).
56
Figure 23: Optional Success Factor class as specialization of Goal class
Several names are given to the actual measurement of performance of an organization, such as
metrics (Eckerson, 2009), measures (Giannoulis & Zikra, 2013), indicators (Braun & Winter,
2005), key performance measures (Ferreira & Otley, 2009), key performance indicators, key
result indicators (Parmenter, 2011), … A more profound analysis however revealed that they are
all about the same core concept: a type of measurement used to quantify performance over
time, that is linked to the organization’s goals. I chose to add an Indicator construct to the
metamodel, which will have a Measurement relation with the Success Factor specialization of
the existing Goal construct (cf. Figure 24), because of three reasons:
(1) The principle of uniqueness, as there is already a Performance relation in the CHOOSE
metamodel between the Actors and Operation constructs (Paige et al., 2000),
(2) The principle of simplicity, as the sentence “An Indicator measures a Goal” is easy to
understand (Paige et al., 2000), and
(3) The fact that it promotes the integration with the existing CHOOSE metamodel.
One Success Factor is linked to zero to many Indicators and an Indicator is linked to exactly one
Success Factor (Braun & Winter, 2005; Brooks & Chittenden, 2012; Giannoulis et al., 2012;
Parmenter, 2011). The Measurement relationship is characterized by an association class,
including a Contribution attribute to indicate the level of contribution a specific Indicator has in
the realization of the Success Factor (cf. PART I, Chapter 0) (Pourshahid et al., 2007).
Figure 24: Indicator class together with Measurement relationship with Success Factor class
57
A comprehensive list of eleven attributes an indicator should contain, is given in PART I, Chapter
3.3.1.3. Their usefulness or redundancy for the Performance-oriented extension of the CHOOSE
metamodel is evaluated below:
Name and Description are added as attributes of the Indicator construct, based on the
design principle of consistency (Paige et al., 2000).
Current Value and Frequency are also added as attributes of the Indicator construct (as
respectively Value and Frequency), since these are mentioned in almost every studied
source.
As the Scale of the Indicator is only incorporated in a few models, for simplicity I chose
not to include this in the CHOOSE metamodel (Paige et al., 2000).
For the calculation of the indicator, a separate attribute could be defined, or this
calculation could be incorporated in the Description attribute. I chose the second option
based on the principle of simplicity (Paige et al., 2000).
The Target Value and Threshold are not integrated as attributes of the Indicator
construct, as target setting is multidimensional and has several attributes for itself (cf.
PART I, Chapter 3.4.1). Hence, a separate class in needed (cf. infra).
The Source attribute is neither included as an attribute of the Indicator class, but as a
many to many Source relationship between the Indicator class and the existing Object
class (cf. Figure 25). This relationship specifies where the information that is needed to
calculate the Indicator can be found. For instance, the information for a financial ‘Return
on Investment’ Indicator can be found in the ‘Balance sheet analysis’ Object. The Source
relationship promotes on the one hand a better integration of the different viewpoints,
and on the other hand consistency and simplicity of the metamodel (Paige et al., 2000).
As the CHOOSE metamodel blends the three architectural layers (cf. PART I, Chapter
2.6.1), the Monitoring Points and Monitoring viewpoints (cf. PART I, Chapter 4.1) are
implicitly incorporated in the Source relationship.
Figure 25: Many-to-many Source Relationship between Indicator and Object classes
58
The Responsible, Informed, and Owned attributes are already implicitly located in the
metamodel through the wish/assignment relationship between the Goal and Actor
constructs, and not included again because of simplicity and uniqueness (Paige et al.,
2000). The Actor who is assigned to a specific Goal, is automatically Responsible for the
Indicator and is the Owner of the Indicator. The Actor who wishes a specific Goal, is
automatically Informed about the Indicator.
Furthermore, an indicator can be classified in many different ways (cf. PART I, Chapter
3.3.1.4). Therefore, a Type attribute is included as well, that can display which type of
Indicator is used.
Figure 26: Indicator class with reflexive Includes relationship
In PART I, Chapter 3.3.1.6, the importance of aligned indicators is highlighted. Therefore, it is
desirable to add a reflexive Includes relationship to the Indicator construct. While Eckerson
(2009) defines duplicated, derived and conglomerate relationships, Popova and Sharpanskykh
(2011) talk about causality, correlation, and aggregation relations, and Frank et al. (2008) and
Cardoso (2013a) about customized relationships. Due to the design principle of simplicity (Paige
et al., 2000) and exhaustiveness (Bernaert, Poels, et al., 2013), and in accordance with the
customized relationship (Cardoso, 2013a; Frank et al., 2008), I chose to implement an
association class to the Includes relationship, including a Type attribute to specify the kind of
relationship the Indicators have (cf. Figure 26). Furthermore, this rather generic label gives the
SME the flexibility to choose other relationships themselves.
As above-mentioned, target setting requires a different class because it is multidimensional (cf.
PART I, Chapter 3.4.1 and Chapter 4). The time frames that stipulate when the targets must be
accomplished, are frequently divided into smaller intervals, called milestones (Eckerson, 2009;
Giannoulis et al., 2012). Particularly for SMEs this is important, as they require both short-term
and long-term results (cf. PART I, Chapter I, characteristic (9)). For that reason, I suggest
modelling a Target class as an optional specialization of the more general Milestone class and
59
make a Comparison relationship between the Milestone and Indicator class (cf. Figure 27),
following the example of Giannoulis et al. (2012). An Indicator is compared to one or more
Milestones, while a Milestone only belongs to one Indicator.
Figure 27: Milestone class is specialized in Target class and has a Comparison relationship with the Indicator class
As displayed in Figure 27, the Milestone class will need three attributes. Firstly, a Value attribute
that displays the short-term (milestone) or long-term target value (attribute number (5) in PART I,
Chapter 3.3.1.3). Secondly, a Deadline attribute that displays when the Value should be
obtained (cf. PART I, Chapter 3.4.1 and Chapter 4.3). The Ranges and Benchmarks that
Eckerson (2009) mentions (cf. PART I, Chapter 3.4.1) do not need a separate attribute
according to the uniqueness design principle (Paige et al., 2000): the first can be calculated by
comparing the Value attribute of the Indicator class and the Value attribute of the Milestone
class, while the latter is already reflected in the chosen target value in the Value attribute of the
Milestone class. Thirdly, the importance of reward practices has been emphasized in PART I,
Chapter 3.4.3. Therefore, a Reward attribute is added to the Milestone class as well. This
attribute belongs to every milestone and not only to the end target, because SMEs need both
short- and long-term benefits to stay motivated (cf. PART I, Chapter 3.4.3).
The above-mentioned Threshold attribute displays the acceptable deviation of the actual value
(Value attribute of Indicator class) from the targeted value (Value attribute of Milestone class).
Furthermore, Eckerson (2009) mentions five possible types of relationships between the actual
and targeted value (cf. PART I, Chapter 3.4.1) (Achievement, Reduction, Absolute,
Minimum/Maximum, and Zero). Therefore, the Comparison relationship between the Indicator
and Milestone class in characterized by an association class, including an attribute Threshold
and an attribute Type (cf. Figure 27).
Many authors state the importance of introducing initiatives/actions/activities when actual and
target values differ, which are executed by an individual or an organization unit (Braun & Winter,
60
2005; Eckerson, 2009; Ferreira & Otley, 2009; Giannoulis et al., 2012; Kaplan & Norton, 1996b).
These initiatives/actions/activities can be seen as a project (or even a continuous process), and
can thereby be incorporated in the Operation construct of the existing CHOOSE metamodel,
which enhances simplicity and uniqueness (Paige et al., 2000) and better integrates the different
viewpoints. As both the metamodels discussed in PART I, Chapter 4.2 and 4.3 add a
relationship between the target class and the action class, this seems the best solution: a many-
to-many Achievement relationship between the Milestone and Operation constructs.
Automatically, the required action has a Performance relationship with the Actor.
The proposed CHOOSE metamodel after extensive literature research is displayed in Figure 28.
Figure 28: CHOOSE metamodel with Performance-oriented dimension after literature research
1.2 CHOOSE Method
Both PM techniques (cf. PART I, Chapter 3.2) and EA techniques (cf. PART I, Chapter 2.3)
highlight the importance of starting the methodology by the strategy of the enterprise. Regarding
PM, almost every studied approach starts by outlining the goals (Eckerson, 2009; Kaplan &
Norton, 1996a; Parmenter, 2011). Especially for SMEs, who often lack a formalized strategy, it is
crucial to start by defining the strategy (cf. PART I, Chapter 3.2.5) (Hudson et al., 2001;
Tenhunen et al., 2001). Regarding EA, the CHOOSE Roadmap-method (cf. Appendix 1) starts
61
by asking the CEO to elicit the highest-level Goal in each of the four BSC dimensions. In the
second step, these highest-level Goals are broken down into lower-level Goals. A complete goal
tree is found through the ‘Why/How?’-questions (Bernaert & Poels, 2011a).
The BSC (Kaplan & Norton, 1992) will continue to be the goal-mapping framework for several
reasons. Firstly, it has been specifically developed to broaden the strategic view of a company
on more than only financial aspects to give managers a fast but comprehensive view of the
business (Kaplan & Norton, 1996a). Secondly, it is widely known and used (Schelp & Stutz,
2007). Thirdly, measurements should always be goal-oriented, and this is incorporated in the
BSC (Solingen & Berghout, 1999). Furthermore, studies revealed that the BSC is suitable for
SMEs (Rompho, 2011). Finally, the BSC is already used in the existing CHOOSE approach and
will thereby guarantee the most coherence with it (Bernaert et al., 2014).
Because of this alignment, the first step of the Performance-oriented extension corresponds
partially with step one and two of the existing Roadmap-method, and with the first part of the first
interview of the Interview-method (cf. Appendix 1). Nevertheless, some extra requirements have
to be included to these steps:
The Success Factors of the organization should be defined (cf. PART I, Chapter 0).
These Success Factors should be written in a SMART way to afterwards facilitate the
translation into SMART Indicators (cf. PART I, Chapter 3.2.3 and Chapter 3.3.1.3).
As characteristic (8) of PART I, Chapter I mentions, SMEs frequently do not have many prior
knowledge about indicators, in contrast to the other constructs of the existing CHOOSE model
(Goals, Operations, Actors and Objects). Furthermore, several authors state the importance of
establishing a motivated team to introduce the indicators (Chang & Morgan, 2000; Eckerson,
2009; Parmenter, 2011). Therefore, sufficient time should be spent to explaining the concepts
and motivating the management. Conversely, according to characteristic (1) of PART I, Chapter
I, the management does not have much time to spend on strategic and performance matters.
Therefore, a well-balanced equilibrium should be found.
After modelling the relevant Goals and Success Factors, the corresponding Indicators should be
derived together with their Milestones and Targets (Cardoso, 2013a; Eckerson, 2009; Ferreira &
Otley, 2009; Parmenter, 2011). If the total number of Indicators exceeds twenty-five (the
proposed number by Norton and Kaplan, cf. PART I, Chapter 3.3.1.7), it is advisable to revise
the indicators based on importance. Furthermore, if the number of Indicators belonging to one
Human Actor (through the Assignment relationship of the corresponding Success Factor)
62
exceeds seven (the proposed number by Eckerson, cf. PART I, Chapter 3.3.1.7), the same
approach should be taken. These two constraints are included as soft constraints in the
metamodel (cf. 1.3).
If the deviation between the value attribute of the Indicator class and the value attribute of the
Milestone class is bigger than the defined threshold attribute in the association class of the
Comparison relationship, a warning should be given. This constraint is also included as a soft
constraint in the metamodel (cf. 1.3).
As a last step, the other EA elements, which are the Objects related to the Indicator, the
Operations necessary to achieve the Targets, and the Actors responsible for the Indicators and
Operations are included. This sequence is in line with the proposed methodology of (Cardoso,
2013a).
1.3 Extra CHOOSE Constraints
Several soft constraints should be included as well:
(1) The total number of Indicators should not exceed twenty-five.
(2) The number of Indicators belonging to one Human Actor (through the Assignment
relationship of the corresponding Success Factor) should not exceed seven.
(3) If the deviation between the actual and the target value ( which is the difference between
Value attribute of Indicator class and the Value attribute of Milestone class) is larger than
the defined Threshold, a warning should be given.
(4) The Actor who is assigned to a specific Goal, is automatically Responsible for the
Indicator and the Owner of the Indicator. The Actor who wishes a specific Goal, is
automatically Informed about the Indicator.
(5) If an Operation is defined to achieve a Milestone, the relationship between the Goal,
Indicator, Milestone and Operations classes should behave as a ‘closed circle’: an
Indicator that measures a certain Objective, is compared against Milestones. The
Operation that is defined to achieve such a Milestone, should also Operationalize a Goal.
This Goal, in turn, should be a lower-level goal of the original Objective. (An example of
this constraint is given in Section 2.2.3.8.)
(6) Every Indicator should be compared to exactly one Target and can be compared to zero
to many other Milestones.
63
CHAPTER II: CASE STUDY AS ACTION RESEARCH
The purpose of the case study as action research is the further development, refinement and
evaluation of the proposed Performance-oriented viewpoint extension to the CHOOSE approach
(cf. Chapter I: Initial Solution). To ensure consistency in the overall development of the
CHOOSE approach, this research is in line with the methodology of (Bernaert, Callaert, & Poels,
2013).
Therefore Chapter two will start with the explanation of the methodology of this research in
Section 2.1. First of all, the area of Design Science will be briefly explained. According to
Järvinen (2007), Action Research is an instance of the design science methodology (Hevner,
March, Park, & Ram, 2004) that is suitable when few theoretical background or experience is
available (which is the case for the application of a PMAS for EA in SMEs), and thereby this
research will be mainly structured around the cyclical process of Action Research (Susman &
Evered, 1978). As the Action Research program was conducted via a case study, the literature
around Case Study Research will moreover be reviewed as well.
Secondly, the actual case study will be discussed in Section 2.2. First a short introduction of the
particular case study will be given, followed by the case study protocol and database, and to
finish the incremental development- and refinement process of the CHOOSE metamodel and
method are discussed.
2.1 Methodology
As the Action Research program in this master thesis is conducted through a case study, and as
Action Research is an instance of the more general Design Science (Järvinen, 2007), the
research methodology is guided by both Design Science, Action Research, and Case Study
Research. These principles will be shortly described before using them.
2.1.1 Design Science
Design Science is essentially a problem solving process: it creates and evaluates IT artefacts
with the intention to solve identified organizational problems (Hevner et al., 2004). Design is both
a verb and a noun: a process, i.e. a set of activities, and a product, i.e. an artefact. This
enhances a continuous shifting between design processes (the world as acted upon) and
designed artifacts (the world as sensed) for the same complex problem (Hevner et al., 2004).
64
Design has four artefacts: constructs, models, methods, and instantiations. It this research, the
(meta-)model and method artefact will be developed. Furthermore, Hevner et al. (2004) present
seven guidelines to conduct a good design science, which are applied during the following case
study research:
(1) Design as an Artefact. A metamodel and a method are developed in the previous chapter.
(2) Problem Relevance. The artefacts foster PM in EA for SMEs.
(3) Design Evaluation. The utility, quality, and efficacy of the metamodel and method are
evaluated in a case study, which is called an Observational Design Evaluation Method.
(4) Research Contributions. The CHOOSE approach is one of the only EA techniques
especially developed for SMEs and the inclusion of a PMAS in EA is sporadic as well.
(5) Research Rigor. The metamodel and method development is based on an extensive
literature research on PM and on criteria of existing EA and PM in SMEs.
(6) Design as a Search Process. Since Design Science is an iterative process, multiple
cycles of Action Research are used to refine the metamodel and method.
(7) Communication of Research. The approach is implemented and tested in an SME and is
communicated in this master thesis.
2.1.2 Action Research
Action Research has been proven to become more and more popular and accepted as a
research method for information systems (IS) (Baskerville & Myers, 2004). Following the
example of Bernaert, Callaert and Poels (2013), the six characteristics and the five stage cyclical
process of Action Research of Susman and Evered (1978) are applied to this research.
The six characteristics of Action Research, that constitute a correction towards the deficiencies
of positivist science (Susman & Evered, 1978), are the following:
(1) Future-oriented. This research is oriented at extending the existing CHOOSE metamodel
and method with a PMAS.
(2) Collaborative/participatory. The researcher works together with the CEO and several
other participants of the SME.
(3) Implies system development. The research is structured around the five stages of
Susman and Evered (1978), which will be explained further on in this section.
(4) Generates theory grounded in action. The research contributes to the development of a
theory (CHOOSE metamodel and method) by taking actions directed by the theory and
65
the subsequent evaluation of these actions. Based on this evaluation, the developed
theory will be supported or revised.
(5) Agnostic. The theories and prescriptions for action of the CHOOSE metamodel and
method are the product of previously taken action and therefore are subject to re-
examination and reformulation upon entering every new research situation. The action
researcher also recognizes that the objectives, the problem, and the method of the
research must be generated from the process itself, and that the consequences of
selected actions cannot be fully known ahead of time.
(6) Situational. It is recognized that an extensive part of the modelled constructs and
relationships are a function of the situation like the relevant actors (such as the CEO, the
HR manager, …) describe it.
The five stages of the cyclical process are used in this master thesis to refine the metamodel
and method. They are all crucial for a comprehensive definition of Action Research (Baskerville,
1997; Susman & Evered, 1978) and are the following:
(1) Diagnosing. The primary problems that are the underlying causes of the organization’s
desire for change, are identified. The notes and voice recordings are analyzed after each
round, based on which the problems are identified that necessitate changes to the
metamodel or method.
(2) Action Planning. The organizational actions that should relieve or improve these primary
problems, are specified. During the implementation process of the CHOOSE approach at
the case study enterprise, several actions are proposed to improve the CHOOSE
metamodel and/or method to deal with these challenges.
(3) Action Taking. After careful consideration, the planned actions can be implemented. The
proposed new metamodel construct can for example be incorporated in the CHOOSE
metamodel.
(4) Evaluating. During the evaluation step, it is verified whether the theoretical effects of the
proposed (in step two) and executed (in step three) actions were realized, and whether
these effects relieved the problems (identified in step one). After implementing the
actions, the CHOOSE model (as an instantiation of the metamodel) and method are
evaluated by the participants of the case study and an external expert.
(5) Specifying learning. The cyclical process of Action Research can be continued to
develop further knowledge about the case study enterprise and the validity of the
theoretical frameworks. After evaluating the incorporated adjustments (e.g. a new model
66
construct), the evaluation results (cf. PART IV) will influence the adoption of the actions
in the CHOOSE metamodel/method.
The first three phases (diagnosing, action planning, and action taking) will be applied to structure
the development and refinement process of the Performance-oriented extension of the
CHOOSE metamodel and method (cf. Section 2.2). Afterwards, the last two phases (evaluating
and specifying learning) will be used to evaluate both CHOOSE artefacts (cf. PART IV)
(Baskerville, 1997; Susman & Evered, 1978).
2.1.3 Case Study Research
Benbasat et al. (1987) state that Case Study research is a good IS research strategy for three
reasons: (1) it permits to study IS in their natural setting, (2) it enables the researcher to ask
‘why’- and ‘how’-questions to understand the nature and complexity of a particular process, and
(3) it is a good way to study a domain that has never been investigated before (Benbasat,
Goldstein, & Mead, 1987).
As scientific research can be approached from different epistemological positions, Yin claims
that three philosophical perspectives can be applied: a critical, positivist and interpretive
perspective (Devos, 2011; Yin, 2009). In a critical research perspective, the world is constrained
by social, cultural, and political predominance; this will not be discussed further. A positivist
researcher considers reality as an objective given. However in this master thesis, an interpretive
perspective will be taken. In contrast to the positivist perspective, this perspective starts with the
assumption that access to reality is only possible through social constructions such as language,
consciousness, and shared meaning. Thereby, the interpretive research is not value-free and
the researcher subjectively observes the reality (Yin, 2009).
Although the interpretive perspective is chosen, the four design tests of the positivist perspective
should still be applied to increase the quality of the research design (Yin, 2009):
(1) Construct validity. Since the case study is used as an Action Research program to
design new artifacts, there are no testable constructs derived from theory.
(2) Internal validity. As this is mainly a concern for explanatory case studies (Yin, 2009) and
as this case study is used for designing artifacts, this is not a concern.
(3) External validity. The criteria makes sure that the findings and conclusions of the study
are generalizable to other case studies (Yin, 2009). A limitation of the case study
evaluation is the fact that only one case study is investigated and that the results are not
statistically generalizable (Yin, 2009).
67
(4) Reliability. By this criteria, the number of errors and biases are minimized and the
falsifiability of the study is maximized (Yin, 2009). To ensure this, a case study protocol
was followed and a case study database was developed. The protocol describes that
interviews should be used where the researcher asks questions and that all interviews
should be recorded. The database is constructed according to Yin’s guidelines (Yin,
2009) and an outline can be found in Appendix 2.
Yin (2009) distinguishes statistical and analytical generalization. The first is not relevant for case
studies, as the researcher will make a statement (inference) about a population based on
empirical data collected from a sample. To perform an analytical generalization, a prior
developed theory (cf. Chapter I) should be used as a template against which the experimental
results can be tested (cf. Chapter II, Section 2.2). As a unit-of-analysis, the complete enterprise
is taken into account (single unit of analysis) (Yin, 2009). A limitation of this master thesis is the
fact that only one case study has been performed (single-case design) (Yin, 2009).
68
2.2 Case Study Research as Action Research
2.2.1 Description Case Study
VMD NV is a Belgian pharmaceutical laboratory established in 1973, located in Arendonk. The
company is a privately owned manufacturer of veterinary pharmaceuticals. Since 1973, it has
matured into one of the most respected companies in its field, despite the fierce competition.
With forty years of experience in veterinary practice, VMD has acquired extensive knowledge of
veterinary work and the requirements of veterinary profession. VMD employs thirty-nine people
and had an annual turnover of approximately € 20 million in 2013. Thereby, it complies with the
definition of a medium-sized enterprise (Commission, 2005). Since a merger in 2007 with Biové
(a French company), VMD is characterized by an effort to establish a closer collaboration
between both companies. Furthermore, the complete manufacturing department of VMD (expect
for the local labelling department) has moved to France, whereby the core activities of VMD
currently are sales, administration, registrations and logistics. The VMD Group (VMD and Biové)
employs hundred-thirty people and had a consolidated annual turnover of approximately € 40
million in 2013.
This SME complies with the common characteristics of SMEs (Bernaert, Callaert, Poels, et al.,
2013): management has little time to look at strategic matters, no EA experts are employed,
there is discussion about the responsibility of employees for certain tasks, the CEO is the central
figure, and the CEO takes the decisions whether or not to adopt a new approach. However, an
external consultant is part of the Board of Directors. Furthermore, the merger with Biové has
caused increasing complexities for the firm, whereby the firm is a good candidate for the
Performance-oriented extension of the CHOOSE approach.
The CHOOSE model of VMD was composed by Mr. Jan Moons (CEO and owner of the
company), Mr. Peter Manet (external member of the Board), Mr. Luc Jacobs (member of the
Board and head of HR, Administration and Finance), Ms. Kristel Walschots (Supply Chain
Manager) and myself (the CHOOSE architect).
2.2.2 Case Study Protocol and Database
Both the case study protocol and case study database are described to increase the reliability
(positivist design test) of this research.
69
The case study protocol entails the instrument, the general rules and the procedures to execute
the protocol (Yin, 2009). An interview guide per interview round is prepared and can be found in
the database.
The case study database should include five different source types to increase the reliability of
the investigated data based on triangulation: documentation, archival records, interviews, direct
observation, and physical artefacts (Yin, 2009). Therefore, the database comprises voice-
recordings of interviews, documents (from the enterprise), notes (prepared by the researcher),
and the developed CHOOSE model of the enterprise. Documentation and archival records
correspond with the documents, notes, and models. Interviews, direct observation, and physical
artefacts correspond with the voice recordings of the interview.
As the greatest part of the data involves strategic issues, a limitation of this research is that the
case study database contains confidential data and therefore cannot be made public. Hence,
only the references of these documents are displayed in Appendix 2, while the original
documents are submitted on a CD-ROM with this master thesis.
2.2.3 Development
The incremental implementation and refinement process of the Performance-oriented extension
of the CHOOSE metamodel and method will be described chronologically. Every modification or
addition to the metamodel and method will be subjected to the three-step process of Mohr
(1982):
(1) Problem/shortcoming Identification, where the problem or shortcoming to the CHOOSE
metamodel or method is identified,
(2) Opening the Solution Space, where the different (potential) solutions are described and
analyzed, and
(3) Closing the Solution Space, where the best alternative is chosen based on the defined
criteria. Furthermore, the solution space should be collectively exhaustive.
The incremental implementation and refinement process entails twelve successive interviews.
Afterwards in PART IV, the modifications and extensions to the metamodel and method will be
evaluated and will be used to develop further knowledge. Thus, the five phases of the cyclical
process of action research are followed (cf. Section 2.1.2) (Baskerville, 1999; Susman & Evered,
1978).
70
2.2.3.1 Interview 1
Description
During the first interview (with the CEO and the external member of the Board) that took an
entire day, the complete Goal dimension of the CHOOSE model was established, by using post-
its and a large whiteboard (cf. Figure 29). Afterwards, the CHOOSE Editor tool support in
Eclipse was used to get a clear overview (Zutterman et al., 2013). The anguage notation legend
of this tool can be found in Appendix 3. The complete Goal viewpoint (established after several
improvement cycles) can be found in Appendix 4.
Figure 29: Goal dimension through action research using post-its
Refinements to CHOOSE approach
The current Roadmap-method of the CHOOSE method states that the Goal dimension should
start by asking the CEO to define the highest-level goal in each of the four BSC dimensions (cf.
Appendix 1). However, based on this interview (Problem/shortcoming Identification, (Mohr,
1982)) and based on a workshop I participated in concerning strategy implementation through
KPIs (Solvay Brussels School of Economics and Management, VOKA, & TiasNimbas Business
71
School, 2014), I discovered that it is easier to start by defining the overall strategy/vision/mission
of the firm before dividing it into the four BSC perspectives. Furthermore, this is also mentioned
in literature (Kaplan & Norton, 1996a).
I see two possible solutions to this problem (opening the solution space, (Mohr, 1982)):
(1) introducing a Highest Strategy construct as a specialization of the Goal construct in the
CHOOSE metamodel, or (2) including this requirement in the first step of the CHOOSE
Roadmap-method. In order not to overload the metamodel and based on the simplicity design
criteria (Paige et al., 2000), the second option was chosen (closing the solution space). The
case study example is given in Figure 30.
Figure 30: From mission to the four highest-level BSC perspectives
2.2.3.2 Interview 2
Description
The second interview was held with the external consultant of the Board. The goals were further
refined and ambiguities were solved.
Value
The interviewee acknowledged the usefulness of writing goals (or success factors) in a SMART
way (part of the CHOOSE method).
72
2.2.3.3 Interview 3
Description
The third interview took place with only the CEO. The refined goal tree was proposed, a few
aspects were adapted and finally the goal tree was approved (cf. Appendix 4). Furthermore, the
Success Factors were defined. The CEO states that the following goals are currently of crucial
importance to the company: (1) Establish a closer collaboration between V.M.D. and Biové (the
French company it merged with), (2) Put the right people on the right spot (HR policy), and (3)
Realize a high level of profitability. In the following interviews, these three goals will be
measured through performance indicators.
Refinements to CHOOSE approach
During the explanation of the type attribute of the Indicator class, which distinguishes between
different classification types of indicators (e.g., KPI, KRI, leading, lagging, PI, … discussed in
PART I, Chapter III, Section 3.3.1.4), the case study participants had problems understanding
this. I see two possible solutions for this problem. Firstly, a detailed explanation of the different
classifications could be given in the CHOOSE method. Secondly, the type attribute could be
deleted from the metamodel. The second option was chosen for several reasons (cf. Figure 31):
Figure 31: Elimination of Type attribute from Indicator class
This is the optimal solution according to the simplicity design criteria (Paige et al., 2000);
The ‘right’ classification type and its importance has been argued a lot in literature (cf.
PART I, Chapter III, Section 3.3.1.4);
SMEs often do not have prior knowledge of indicators (cf. PART I, Chapter I,
characteristic (8)) and could thereby get easily confused; and
The increase in efficiency (cost decrease) of deleting the attribute seems larger than the
decrease in effectiveness (benefits) (Moody, 2003).
73
Value
During the interview, the CEO frequently asked the same question: “Defining the indicators is
one part, but the key question I’m concerned about is who will take care of the monitoring?”
Therefore, the Performance (RACI) relationship between the Operation (that will be performed to
achieve the target) and the Actor doing it, is of great value.
2.2.3.4 Interview 4
Description
This interview concerned the goal “Stimulate a closer collaboration between VMD and Biové”,
and had the Supply Chain Manager as an interviewee. Yet, as will be explained in Interview 5
(cf. infra), the lower-level goal “Perform a reliable forecast from VMD to Biové”2, which is wished
by the CEO and assigned to the Supply Chain Manager is used as a starting point here. During
the interview, the success factor was written in a SMART way (and refined after listening to the
voice recordings) (cf. Figure 32), several Indicators were proposed, as well as their
corresponding required Operations (actions/initiatives).
Figure 32: Decomposition of ‘Stimulate closer collaboration between VMD and Biové’ into SMART goal
Refinements to CHOOSE approach and Value
Thesse will be discussed in Interview 5, as both interviews happened right after each other.
2 Biové is the French company that merged with VMD. All VMD’s products are now produced by Biové,
and therefore, VMD has to perform a good forecast of its needed products to Biové in order to get the
products on time and establish a good relationship (and closer collaboration) with Biové.
74
2.2.3.5 Interview 5
Description
This interview concerned the goal “Put the right people on the right spot” (which is wished by the
CEO and assigned to the HR manager), and had the HR manager as an interviewee. The HR
process of the company has been managed rather informally in the past. However since the
merger with the company in France, the company feels the need to formalize its ways of
evaluating and hiring people so that the two companies become more aligned. The goal was
refined into two more specific goals, which were written in a SMART way (cf. Figure 33) and
afterwards linked to the Indicators, which on their turn are linked to the Operations. The “Perform
a good selection process” goal clearly shows that not only the lowest-level goals (also called the
‘leaves’ of the tree) can be written SMART in order to be measured.
Figure 33: Decomposition of 'Put the right people on the right spot' into SMART goal
75
Refinements to CHOOSE approach
During interview four and five, some difficulties were discovered when writing the defined
Success Factor in a SMART way. Success Factors are often rather high-level goals (cf. PART I,
Chapter II, Section 3.2.2) and thereby they are often too general to be written SMART. The goal
“Stimulate a closer collaboration between VMD and Biové” exemplifies this.
This problem could be solved in two ways. The first solution is to define a certain level of
abstraction in the goal tree from which goals could be measured (e.g., only the third level of
refinement). The second solution is two-fold. To start, the Success Factor construct is replaced
by an Objective construct, which is defined as “a statement of an attainable, time-targeted, and
measurable target that the enterprise seeks to meet in order to achieve its goals” (Object
Management Group, 2013). Furthermore, the CHOOSE Roadmap-method will include a
statement about the organization’s success factors: ‘Determine which areas of the goal tree
currently are the most critical to the success of the company’. The second solution (cf. Figure
34) is chosen for several reasons:
It is according the simplicity design criteria (Paige et al., 2000).
Several well-known authors acknowledge the value of using an Objective construct to
link Goals and Indicators constructs (Giannoulis et al., 2012; Kaplan & Norton, 1996a;
Object Management Group, 2013). The other solution on the contrary, is riddled with
many ambiguities and literature review did not reveal a particular method to do this (cf.
PART I, Chapter IV, Section 4.4).
The term Objective implicitly incorporates the SMART criteria throughout its definition,
and thereby a greater consistency between the CHOOSE metamodel and method is
achieved (Paige et al., 2000).
Figure 34: Replace Success Factor construct by Objective construct
The extra statement about the company’s success factors in the CHOOSE Roadmap-method
still allows the critical success areas of the company to be defined, but these can now be further
refined into more specific and clear objectives.
76
2.2.3.6 Interview 6
Description
In the sixth interview (with the CEO), the Performance-oriented view of the “Stimulate a closer
collaboration between VMD and Biové” was revised and some adjustments were made.
Furthermore, the third critical area of the goal tree was discussed, i.e. “Realize a high level of
profitability”. The first idea was to measure the underlying “Stimulate cost awareness” goal,
however the CEO was not convinced about the usefulness anymore after comparing the benefits
with the costs.
Refinements to the CHOOSE approach
After asking to identify Indicators that measure the fulfilment of the goals in interview four, five
and six with respectively the Supply Chain manager, the HR manager and the CEO, I
discovered that we frequently were talking too long about several indicators that afterwards
seemed inefficient to measure:
In the interview with the Supply Chain manager, an indicator was designed to measure
the number of late deliveries to the customers. However, these data were not recorded,
and several other employees would have to participate to retrieve them.
In the interview with the HR manager, we were brainstorming about a skill rating system
that could be used to determine whether employees are over-, or underqualified for their
job. First of all, the question was raised concerning which aspects to measure (e.g.,
being able to work efficiently with the in-house software system). Secondly, the
importance of these aspects differs for different functions in the company (e.g., while the
sales persons require a basic knowledge of the software system (such as data entry),
the bookkeeper needs an in depth knowledge). All these aspects would have to be
combined into one measuring scale.
In the interview with the CEO, indicators were searched to measure the stimulation of
cost awareness in the company. This is a qualitative goal (“being aware of something”)
and defining quantitative measures for it (e.g., through a survey or a questionnaire)
turned out to be very challenging.
The indicators that resulted (or would have resulted) out of these three examples, would violate
three of the characteristics that a good indicator should possess (defined in PART I, Chapter III,
Section 3.3.1.2): an Indicator should be easy to maintain, simple to use and (specifically for
SMEs) easy to collect. Furthermore, the costs of implementing these measures would be greater
77
than the benefits (cf. PART I, Chapter I, characteristic (5)). Regarding the first example, other
indicators could measure the same goal with less effort. The ‘Perform a reliable forecast’ goal
could for example be measured through the ‘out-of-stocks of products ordered from Biové’
Indicator, which can easily be retrieved from the software system. Regarding the second
example, the HR manager and CEO know every employee personally and see them every day,
as SMEs are rather small. Thereby, informal conversations or performance appraisals could be
more beneficial. Regarding the third example, the same explanation as the second example
applies (e.g., observing the cost awareness of employees rather than measuring it).
I see two possible solutions for this problem. Firstly, the Objects necessary to perform the actual
measurement of the Indicator, are currently incorporated as a last step in the method (cf. PART
III, Chapter II, Section 1.2). This step could be moved forward and asked directly after the
identification of an Indicator, before Milestones, Targets and Actors are defined. Secondly, an
extra step could be added to the method after the identification of the Indicators to ensure that
cost and benefits are always compared before continuing the other steps. As the first solution
would require a change to the existing Roadmap-method and Interview-method (Bernaert,
Callaert, Poels, et al., 2013) and the second solution only requires adding one extra sentence,
the latter solution is chosen based on the design principle of simplicity (Paige et al., 2000) and
on the coherence with the existing CHOOSE approach.
Value
When searching for SMART descriptions and indicators, more goals were often elicited and
other goals were refined and adjusted, which facilitates the translation from goals into
operations. Furthermore, the CEO was really convinced about defining short-time milestones
and not only a long-term target, which confirms characteristic (9) of PART I, Chapter I.
2.2.3.7 Interview 7
Description
The seventh interview was conducted with the Supply Chain manager, in order to further
complement the Indicators belonging to the Objective “Reshape the forecasting system from
VMD to Biové so that Biové’s delivery time to VMD becomes 8 to 10 weeks”. The model consists
of seven Indicators belonging to the Supply Chain Manager and one Indicator belonging to the
CEO (which is possible through a metamodel adjustment explained in the next paragraph) (cf.
Figure 35). Furthermore, the Includes relationships between the different Indicators are
78
displayed, together with their corresponding Type attribute. Finally, the relationships with the
Object and Operation viewpoints are established as well (not displayed here, but in Appendix 5).
Figure 35: Performance-oriented viewpoint on ‘Reliable forecast from VMD to Biové’
Refinements to CHOOSE approach
When drafting the indicators, it became clear that the actor assigned to a certain objective is not
always responsible for the indicator. Instead, several other actors are involved as well. Three
indicators exemplify this:
The indicator “Loss on out-of-stocks”, which is derived from the indicator “Out-of-stocks
of Biové products for Belgium”, is of great importance to the CEO of the company.
The indicator “Expired Products” cannot be calculated if the IT specialist of the company
does not create it.
OUT-OF-STOCKS of Biové products for
Belgium (ind. 1)
DELIVERY TIME from Biové to
VMD (ind. 7)
On time forecasting fromspecial export customers
(ind.6)
EXPIRED PRODUCTS (ind.2)
VARIANCE in Belgian Forecasting
(ind.3)
On time forecasting from important & special export
customers (ind.4)
On time forecasting from important export
customers (ind.5)
Causality
Inverse correlation
Sum
LOSS ON OUT-OF-STOCKS
Derived
CEO
79
The indicator “Variance in Belgian Forecasting” has to be established by both the
responsible for Sales Belgium and the Supply Chain Manager, and both persons also
have an interest in this indicator.
There are two possible ways to solve this problem. In the first solution, a many-to-many
Assigned relationship between the Indicator and Actor constructs is set up. As several Indicators
can be linked to several Actors, this relationship needs an association class. This association
class comprises a Type attribute to enable the use of a RACI labelling of the Assigned
relationship (Responsible, Accountable, Consulted and Informed). The second solution on the
other hand, does not create an extra solution, but uses the existing relationships to solve the
problem: the Actor who needs to set up the Indicator is specified through the Source relationship
with the Object class, and the person who will be informed on the Indicator, is specified by the
Wish relationship between the Actor and Goal construct.
The first solution was chosen (cf. Figure 36) because of the exhaustiveness principle (Bernaert,
Poels, et al., 2013) and the consistency design criteria (Paige et al., 2000) (the Actor
specialization of the Object class is used as a passive element and could thereby not produce
an Indicator). In addition, the generic Type attribute enables flexibility for the SME to choose for
another responsibility assignment matrix. Furthermore, it corresponds with the Performance
(RACI) relationship of the current CHOOSE metamodel (Bernaert, Callaert, Poels, et al., 2013)
and with what is broadly agreed upon in literature (Cardoso, 2013a): an indicator should have a
responsible, give information to specific persons, and be owned. An example is given in Figure
37 (framed in red).
Figure 36: Assigned (RACI) relationship with an association class with a Type attribute
This refinement additionally gives the ability to improve an above-mentioned soft-constraint (cf.
PART III, Chapter II, Section 1.1) on the number of indicators per person: the number of
Indicators for which an Actor is Responsible (through the Assigned (RACI) relationship) should
not exceed seven.
80
Figure 37: Example of the Assigned relationship between Actor and Indicator
2.2.3.8 Interview 8
Description
Interview number eight (with the HR manager) further complements the indicators belonging to
the objectives “Retain neither over, nor under qualified persons for a function” and “Hire neither
over, nor under qualified persons for a function” (cf. Figure 38). The HR manager is Accountable
for all ten indicators, but only Responsible for seven of them. The other three indicators are the
responsibility of respectively the managers of the Quality Department (Leo in Figure 38), the
Belgium sales department (PJ in Figure 38) and the Export sales department (Jan in Figure 38).
Furthermore, the Includes relationships between the different Indicators are displayed, together
with their corresponding Type attribute. Also shown is the Measurement relationship between
the objective and its corresponding indicators, with the specific contribution of each indicator.
Finally, the relationships with the Object and Operation viewpoints are established as well (not
displayed here, but in Appendix 6).
Name Get more input from registrations, Sales
Namevariance in Belgian forecasting Description
vb. Antibiotica in België aan banden gelegd, had
eerder moeten gecommuniceerd worden
Description
|estimated value of the last twelve months-
real value of the last twelve months |
/ real value of the last twelve months
Who (RACI) registration department, sales
Actual Value 33% Where /
Frequency Monthly Deadline jun/15
Where Excel list: ForecastBelgium
Type of target Reduction
Name Include promo actions in forecasting
Description When a product is sold as a discount, more is sold,
so the forecast should be adjusted
deadline 1 dec/14 Who (RACI) R: PJ, A: Kristel
value 1 25% Where /
deadline 3 jun/15 Deadline jun/15
target value 15%
RACI Responsible: Supply Chain Manager,
Consulted: Sales Belgium
Indicator 3
Action 1
Action 2
Actor
Milestones
3
81
Figure 38: Performance-oriented viewpoint on ‘Put the right people on the right spot’
Refinements to the CHOOSE approach
During all the interviews, the Reward attribute of the Milestone class has not been used at all.
This could be caused by the fact that SMEs have a lack of financial resources (PART I, Chapter
1, Characteristic (2)). Because of the design principle of simplicity (Paige et al., 2000), the
attribute is deleted from the metamodel. The same applies for the Threshold attribute of the
association class of the Comparison relationship between the Indicator and Milestone classes:
the CEO and other managers did not see the benefits of implementing it. As the increase in
efficiency (cost decrease) of deleting the attribute is larger than the decrease in effectiveness
(benefits) (Moody, 2003), this attribute is deleted as well.
Figure 39: Description attribute of the Milestone class
Instead, a Description attribute is added to the Milestone class (cf. Figure 39). This attribute
ensures consistency with the other classes of the CHOOSE metamodel and can entail extra
information, such as rewards practices and other issues.
ADDED VALUE HUMAN CAPITAL
(ind.1)
NUMBER OF RECRUITMENTS WITH ASSESSMENTS OF HIGHLY SKILLED
PERSONS
NUMBER OF PERFORMANCE APPRAISALS FROM SALES BELGIUM DEPARTMENT
(Ind.7)
AVERAGE SCORE OF WORKERS ON EVALUATION
(Ind. 2)
TOTAL NUMBER OF PERFORMANCE
APPRAISALS(Ind.5)
NUMBER OF PERFORMANCE
APPRAISALS FROM HR MANAGER
(Ind.6)
NUMBER OF PERFORMANCE
APPRAISALS FROM QUALITY DEPARTMENT
(Ind.8)
AVERAGE SCORE OF WORKERS ON GOOD
BEHAVIOR
AVERAGE SCORE OF WORKERS ON ATTENDANCE
NUMBER OF PERFORMANCE
APPRAISALS FROM SALES EXPORT DEPARTMENT
(Ind.9)
Contribution
SumSum
PJ JanLeo
50%30%20%
82
Value
To start, the Assigned relationship between the Indicator and Actor constructs has been valued
by the HR manager. To exemplify, the “Total number of performance appraisals” Indicator will be
explained. The company decided that performance appraisals to evaluate the employees would
be very useful, since the company was getting bigger. However, several different persons would
need to perform the appraisals, as the HR manager does not have enough technical knowledge
to interview for instance the employees working in the Registration department. Therefore, the
indicator is divided into four different indicators: one for the manager of Sales Belgium, one for
the manager of Sales Export, one for the manager of the Registration department, and one for
the HR manager (cf. Figure 38 and Appendix 6).
Secondly, as mentioned before (cf. Section 2.2.3.5), the “Perform a good selection process” goal
shows that not only the lowest-level goals (also called the ‘leaves’ of the tree) can be
transformed in an Objective in order to be measured. Furthermore, as depicted in Figure 40, the
operations required to achieve a milestone, are on their turn linked with a goal, which is a lower-
level refinement goal of the original objective. This clearly shows the interrelation of the different
viewpoints and corresponds with soft constraint (5) described in Section 1.3.
Figure 40: Example of the interrelation of the different viewpoints
Indicator: Number of
recruitments with
assessment of highly
skilled people
Target: 100% by the
end of 2014
Comparison
(Absolute)
Measurement
Wish
Assignment
Responsible
Achievement
Accountable
Responsible
Consulted
Performance
Responsible &
Accountable
Consulted
Operationalization
83
2.2.3.9 Interview 9
Description
During this interview with the CEO, the third goal described in Interview 3, “Realize a high level
of profitability”, is measured by indicators. Two factors are of crucial importance to the CEO: the
EBITDA and the Gross Margin (cf. Figure 41). Furthermore, the Includes relationships between
the different indicators are displayed, together with their corresponding Type attribute. Also
shown is the Measurement relationship between the objective regarding the Gross Margin and
its indicators, with their specific contribution.
Figure 41: Performance-oriented viewpoint on ‘Realize a high level of profitability’
Refinements to the CHOOSE approach
The CEO did not find any actions (Operations) necessary to achieve the target, as the financial
goals were too high-level. Literature review confirms the same (cf. PART I, Chapter 3.3.1.4):
financial indicators give a clear picture of whether you are doing fine or not, but do not tell you
how to improve these results (Parmenter, 2011). Therefore, a soft-constraint is added to
metamodel following the example of Giannoulis et al. (2012): The Milestones of an Indicator
belonging to an Objective coming from the financial perspective of the BSC, do not require
corresponding Operations.
84
2.2.3.10 Interview 10, 11 and 12
Description
During interview ten, eleven and twelve, the last adjustments were made to the Performance-
oriented dimension of the three objectives that are critical to the organization:
Interview ten regards the “Stimulate a closer collaboration between Biové and VMD” goal
and a complete overview of the Performance-oriented viewpoint can be found in
Appendix 5.
Interview eleven regards the “Put the right people on the right spot” goal and a complete
overview of the Performance-oriented viewpoint can be found in Appendix 6.
Interview twelve regards the “Realize a higher level of profitability” goal and a complete
overview of the Performance-oriented viewpoint can be found in Appendix 7.
Refinements to the CHOOSE approach
During the last three interviews, the CHOOSE metamodel and method stayed robust.
Value
Because of the inclusion of the value attribute of the Indicator class and the value attribute of the
Milestone class, and because of the Contribution attribute of the association class of the
Measurement relationship between the Indicator and Success Factor classes, it will be possible
to perform a more elaborated as-is/to-be analysis than was the case previously. Additionally, the
degree of satisfaction of a certain success factor can be calculated through the degree of
satisfaction of its underlying Indicators (calculated by the Contribution attribute of the
Measurement relationship). An example is given in Figure 42.
85
Figure 42: Example of an As-is/To-be and Degree of Satisfaction analysis
Goal (Objective ):
AS ISTO BE
(end 2014)
TO BE
(end 2015)
TO BE
(end 2016)
€ 5 745 mio € 5 450 mio € 5 848 mio € 6 165 mio
Strive for a consolidated EBITDA of € 6.165
million by the end of 2016
AS IS / TO BE analysis
DEGREE OF SATISFACTION analysis
86
CHAPTER III: FINAL SOLUTION
3.1 Final CHOOSE metamodel
Figure 43 displays the final CHOOSE metamodel extended with a Performance-oriented
viewpoint (in orange) after the Literature Research and Action Research.
Figure 43: Final CHOOSE Metamodel
3.2 Final Extra CHOOSE constraints
The Performance-oriented extension of the CHOOSE metamodel includes the following soft-
constraints:
(1) The total number of Indicators should not exceed twenty-five.
(2) The number of Indicators for which an Actor is Responsible (by the Assigned
relationship) should not exceed seven.
(3) Every Indicator should be compared to exactly one Target and can be compared to zero
to many other Milestones.
(4) The Milestones of an Indicator belonging to an Objective derived from the financial
perspective of the BSC, do not require corresponding Operations.
87
(5) If an Operation is defined to achieve a Milestone, the relationship between the Goal,
Indicator, Milestone and Operations classes should behave as a ‘closed circle’: an
Indicator that measures a certain Objective, is compared against Milestones. The
Operation that is defined to achieve such a Milestone, should also Operationalize a Goal.
This Goal, in turn, should be a lower-level goal of the original Objective. (An example of
this constraint is given in Section 2.2.3.8.)
3.3 Final CHOOSE method
The CHOOSE method consists of the Roadmap-method, the Interview-method and the Stop-
criteria (Bernaert, Callaert, Poels, et al., 2013). The extensions that are made related to the
Performance-oriented viewpoint during this master thesis, are highlighted in bold.
The Roadmap-method
(1) (Goal dimension): The CEO of the enterprise is asked to first define the strategy, vision
and mission of the enterprise and afterwards define the highest-level Goal in each of
the four Balanced Scorecard dimensions.
(2) (Goal dimension): From the four highest-level Goals, the Goals are broken down into
lower-level Goals. More Goals are found through the 'Why/How?'-questions (Bernaert
and Poels 2011). Afterwards determine which areas of the goal tree currently are
the most critical to the success of the company. These Goals will be linked to the
Performance-oriented dimension and have to be written in SMART Objectives.
(3) (Actor dimension): The Human Actors, Roles, Software Actors, and Devices are added
through interviewing and consulting other secondary sources (e.g., organization charts).
(4) (Performance-oriented dimension): the Indicators (corresponding with the
Objectives through the Measurement relationship and with the Actors through the
Assigned relationship) are defined together with their Milestones and Target
through the Comparison relationship. Indicators are related to each other through
the Includes relationship. The benefits of adding an Indicator should always be
compared against the costs.
(5) (Operation dimension): The different activity groups of Porter’s Value Chain (Porter and
Millar 1985) are used to elicit the Operations (Processes and Projects). The Operations
are linked with the other three dimensions (Goals, Actors and Milestones).
88
(6) (Object dimension): The Objects and corresponding relationships with Goals (Concern),
Actors (Monitoring/Control), Indicators (Source) and/or Operations (Input/Output) are
added to the model based on the interviews and visual inspection.
(7) (Validation): The last step of the Roadmap is a validation round of the model by the CEO.
The Interview-method
(1) During the first interview, Step 1 and 2 of the Roadmap are executed in order to create a
holistic Goal tree of the enterprise. The second part of the interview focuses on the Actor
tree by executing Step 3.
(2) The second interview is divided in four parts. First of all, a questionnaire, created after
the analysis and implementation of the Goal and Actor tree of the first interview, is used
to fill the gaps (e.g., missing elements or relationships) of the Goal and Actor tree.
Secondly, the Indicators and Milestones belonging to the SMART Objectives are
defined (Step 4). Thirdly, the activity groups of Porter’s Value Chain are used to define
the Operations of the enterprise in each activity group (Step 5). Thirdly, the Objects of
the enterprise are added based on their relationships with the other four dimensions
(Step 6).
(3) The third interview consists of two parts. Firstly, after analysis and implementation of the
Indicators, Milestones, Operations tree and Objects, the interviewer tries to find missing
constructs and relations or resolve ambiguities. Secondly, the interviewee (CEO of the
SME) visually validates the final CHOOSE-model (Step 7).
The Stop-criteria
(1) Goal dimension: The Goal-check is twofold.
(a) Does the downward Goal Refinement path of each Goal AND-Refinement and of
one alternative of each Goal OR-Refinement only stop if at least one of its fine-
grained Goals is Operationalized?
(b) Is every lowest-level (AND-Refinement) Goal Operationalized by an Operation?
If this is not possible, there are two possibilities: it is a to-be Goal (another
alternative of a Goal OR-Refinement) or there is a misfit between strategy and
Operations (Henderson and Venkatraman 1993).
(2) Actor dimension: The Actor-check is also twofold:
89
(a) Are all employees (Human Actors and Roles) of the organization chart, all
other relevant stakeholders, and all relevant Software Actors and Devices
included in the Actor dimension?
(b) Is every Actor linked to at least one Operation?
(3) Operation dimension: First of all, take into consideration that relationships can
sometimes be transferred to a related element (Bernaert et al. 2013a): a superOperation
groups the relations (with e.g., Goals) of its subOperations and a lower-level Goal
inherits the relations of its related higher-level Goals (with e.g., Operations). If for
example a superOperation does not Operationalize any Goal, however one of its
subOperations does, then this superOperation actually also Operationalizes this Goal
through one of its subOperations. The Operation-check is then fourfold:
(a) Are all Performed Operations for every Actor included?
(b) Does each Operation contribute to a Goal?
(c) Is every Operation linked to an Actor?
(d) Are all Operations contributing to the achievement of a Milestone included?
(4) Performance-oriented dimension: The same relationship transfer is applicable
here. For example, an Objective is linked to an Indicator, which on its turn is built
of two other Indicators. Then these two Indicators also measure the Objective. The
Performance-oriented-check is fourfold:
(a) Does each Indicator measure an Objective?
(b) Does each Indicator have a responsible Actor?
(c) Does each Indicator have one Target?
(d) The Indicators derived from the financial perspective of the BSC excluded,
does each Milestone belonging to an Indicator have a corresponding
Operation?
(5) Object dimension: In analogy with the four other dimensions, the Object-check is also
fourfold:
(a) Are all products and resources (either physical or not) of the SME represented in
the Object model?
(b) Are all the necessary Objects (Input/Output) of the Operations (Operation
dimension) included in the Object dimension?
(c) Are all the necessary Objects of the Goals included in the Object dimension?
(d) Are all the necessary Objects of the Indicators included in the Object
dimension?
90
PART IV: EVALUATION
INTRODUCTION
To ensure consistency, the pursued method of evaluation for the Performance-oriented
viewpoint extension of the CHOOSE metamodel and method is in line with the previously used
evaluation methods in the CHOOSE approach (Bernaert, Callaert, Poels, et al., 2013). In
Chapter I, the literature concerning the used evaluation techniques will be described. The
CHOOSE method will be evaluated according to the Method Evaluation Model (Moody, 2003),
the developed instance of the CHOOSE metamodel (CHOOSE model) will be evaluated
according to the User Evaluations Based Quality Model for Conceptual Modelling Scripts (A.
Maes & Poels, 2007) and the CHOOSE model will also be evaluated with respect to the
CHOOSE criteria (Bernaert et al., 2014). Afterwards, in Chapter II, the results of this evaluation
will be described.
CHAPTER I: EVALUATION MODELS
1.1 Method Evaluation Model
The Method Evaluation Model (MEM) (Moody, 2003) is a theoretical model to evaluate IS design
methods, and is based on both the Technology Acceptance Model (TAM) (Davis, 1989) and the
Methodological Pragmatism (Rescher, 1977). Both theories will be briefly presented before
explaining the MEM.
The TAM is a well-known model concerning the adoption of IS technology, and consists of four
important constructs (Davis, 1989):
(1) Perceived Usefulness, which is “The degree to which a person believes that using a
particular system would enhance his or her job performance”.
(2) Perceived Ease of Use, which is “The degree to which a person believes that using a
particular system would be free of effort”.
(3) Intention to Use, which is “The extent to which a person intends to use a particular
system”, and
(4) Actual System Use, which is “The extent to which a system is used in practice”.
91
The Intention to Use is influenced by both the Perceived Usefulness and Perceived Ease of Use,
which is a person’s subjective appraisal of respectively performance and effort (Davis, 1989).
The Methodological Pragmatism emphasizes that methods only have pragmatic value rather
than truth-value: “The validity of a method can only be established by applicative success in
practice” (Rescher, 1977). Pragmatic success is defined as “the efficiency and effectiveness with
which a method achieves its objectives” (Rescher, 1977). The objective of validation is not
demonstrating that the method is ‘correct’, but that it is rational practice to adopt the method
based on its pragmatic success (Rescher, 1977). For this reason, the Performance-oriented
extension of the CHOOSE approach was implemented and tested in a case study.
Based on TAM (Davis, 1989) and the Methodological Pragmatism (Rescher, 1977), Moody
(2003) constructed the MEM, which is displayed in Figure 44.
Figure 44: Method Evaluation Model (from (Moody, 2003))
The model has the following six constructs:
(1) Actual Efficiency, which is “The effort required to apply a method” (Methodological
Pragmatism) (Rescher, 1977)
(2) Actual Effectiveness, which is “The degree to which a method achieves its objectives”
(Methodological Pragmatism) (Rescher, 1977)
(3) Perceived Ease of Use (TAM) (Davis, 1989)
(4) Perceived Usefulness (TAM) (Davis, 1989)
(5) Intention to Use (TAM) (Davis, 1989)
(6) Actual Usage (TAM) (Davis, 1989)
92
The causal relationships between the constructs of the model shown in Figure 44 by the arrows,
are the following:
Actual Efficiency measures the effort required to apply the method, by which the
Perceived Ease of Use (i.e. the perception of effort required) is influenced.
Actual Effectiveness measures how well the method achieves its objectives, by which the
Perceived Usefulness (i.e. the perception of its effectiveness) is influenced.
Perceived Ease of Use has an impact on the Perceived Usefulness (TAM) (Davis, 1989).
Both Perceived Ease of Use and Perceived Usefulness have an impact on Intention to
Use (TAM) (Davis, 1989).
The Intention to Use a method determines its Actual Usage. This is according to TAM
(Davis, 1989) and the Theory of Planned Behavior (Ajzen, 1991), which states that
perceptions influence intentions which in turn influence the actual behavior of the
individual.
It is important to notice that Intention to Use is only indirectly influenced by Actual Efficiency and
Effectiveness through the Perceptions of Ease of Use and Usefulness (in contrast to TAM). This
emphasizes the fact that subjective reality is more important than objective reality, as
perceptions are also influenced by other factors not included in the evaluation model, such as
prior knowledge, experience with particular methods and normative influences (Moody, 2003).
According to Moody (2003), the relative importance of Perceived Ease of Use in making
decisions about method adoption is much higher for practitioners than it is for undergraduate
students. Therefore, it is essential to make sure that EA techniques can be easily applied by
practitioners in SMEs, and that these practitioners moreover are willing to apply the method
(Bernaert et al., 2014).
Moody (2003) reformulated the TAM six-item scales for Perceived Usefulness and Perceived
Ease of Use (cf. Table 2) on a 7-point Likert scale from ‘strongly disagree’ to ‘strongly agree’.
Perceived Usefulness Perceived Ease of Use
1. Work More Quickly 2. Increase Job Performance 3. Increase Productivity 4. Increase Effectiveness 5. Make Job Easier 6. Useful
1. Easy to Learn 2. Controllable 3. Clear and Understandable 4. Flexible 5. Easy to Become Skilful 6. Easy to Use
Table 2: Six-item scales of Perceived Usefulness and Perceived Ease of Use (from (Davis, 1989))
93
During the evaluation phase of the action research in this master thesis, four different persons
evaluated the CHOOSE method by a questionnaire compiled by Bernaert, Callaert, Poels, et al.
(2013) based on the six-item scales of Moody (2003) and Davis (1989) (cf. Table 3). The
evaluators are four participants of the case study enterprise: the CEO (Mr. Jan Moons), the HR
manager (Mr. Luc Jacobs), the Supply Chain manager (Ms. Kristel Walschots) and the external
consultant of the Board (Mr. Peter Manet), who also is a PM expert.
Table 3: Questionnaire for Perceived Usefulness and Perceived Ease of Use (from (Bernaert, Callaert, Poels, et al., 2013))
1.2 User Evaluations Based Quality Model for Conceptual Models
The User Evaluations Based Quality Model (UEBQM) proposed by Maes and Poels (2007)
consists of four model constructs (quality dimensions: PEOU, PU, PSQ, US) and five
relationships between those constructs (H1, H2, H3, H4, and H5) (cf. Figure 45). The
relationships and the explanatory power of the model have been proven empirically (A. Maes &
Poels, 2007). The model constructs are:
The Perceived Ease of Understanding (PEOU), which defines to which extent a person
believes that using a model for understanding the problem domain and IS requirements is
free of mental effort.
The Perceived Semantic Quality (PSQ), which measures the accuracy and completeness by
which the reality is represented by the model, as perceived by a model user.
Perceived Usefulness
PU1 The CHOOSE method enabled me to more quickly create the CHOOSE model.
PU2 With the aid of the CHOOSE method, I was able to create a better and more consistent model.
PU3 The structured CHOOSE method contributes to an increased
productivity.
PU4 The applied CHOOSE method increases the effectiveness of the enterprise architecture process and contributes to the
final result.
PU5 The offering of the method facilitates the enterprise architecture process.
PU6 I consider the applied method as a useful aid in creating the
enterprise architecture model.
Perceived Ease of Use
PEU1 It is easy to learn to work with the CHOOSE method.
PEU2 The CHOOSE method is easily controllable.
PEU3 The structure of the CHOOSE method is straightforward, clear, and understandable.
PEU4 The CHOOSE method can be applied flexibly.
PEU5 I feel that I will quickly get to master the CHOOSE method.
PEU6 The CHOOSE method is easy to use.
94
The Perceived Usefulness (PU), which defines the effectiveness of a model related to the
expression and communication of the user’s view of the domain and the IS requirements.
The User Satisfaction (US), which measures the satisfaction of the users about the model
with respect to its purpose and thereby gives a general evaluation of the model quality.
Figure 45: UEBQM (from (A. Maes & Poels, 2007))
Table 4 displays the proposed multi-item measurement instrument for the four constructs (A.
Maes & Poels, 2007). For each item, a 7-point Likert scale ranging from ‘strongly disagree’ to
‘strongly agree’ is recommended.
Table 4: The UEBQM measurement instrument concerning the four quality dimensions (from (A. Maes & Poels, 2007))
PEOU1 It was easy for me to understand what the conceptual model was trying to model PEOU2 Using the conceptual model was often frustrating
PEOU3 Overall, the conceptual model was easy to use PEOU4 Learning how to read the conceptual model was easy
US1 The conceptual model adequately met the information needs that I was asked to support US2 The conceptual model was not efficient in providing the information I needed
US3 The conceptual model was effective in providing the information I needed US4 Overall, I am satisfied with the conceptual model for providing the information I needed
PU1 Overall, I think the conceptual model would be an improvement to a textual description of the business process
PU2 Overall, I found the conceptual model useful for understanding the process modelled PU3 Overall, I think the conceptual model improves my performance when understanding the
process modelled PSQ1 The conceptual model represents the business process correctly
PSQ2 The conceptual model is a realistic representation of the business process PSQ3 The conceptual model contains contradicting elements
PSQ4 All the elements in the conceptual model are relevant for the representation of the business process
PSQ5 The conceptual model gives a complete representation of the business process
95
This measurement instrument is used to evaluate the developed CHOOSE model by the same
four participants of the case study enterprise. The CHOOSE metamodel will thereby be indirectly
evaluated by evaluating its model instance developed in the case study. The proposed adapted
measurement instrument for the CHOOSE model of Bernaert, Callaert, Poels et al. (2013) (cf.
Table 5) is used, with a specific focus on the Performance-oriented viewpoint.
Table 5: Measurement instrument to evaluate the CHOOSE metamodel (from (Bernaert, Callaert, Poels, et al., 2013))
1.3 CHOOSE criteria
The CHOOSE approach entails ten design criteria (cf. Figure 5): five criteria for a good EA
technique, where the fourth criteria ‘Suitable for target audience (SME)’ is replaced by the six
SME-related criteria (Bernaert et al., 2014). These criteria are used as well to evaluate the
Performance-oriented viewpoint extension of the CHOOSE metamodel and method. The criteria
are summarized in Table 6. The evaluation will be done by an external PM and ICT Architecture
expert, Mr. Mark Van Gastel and by the internal PM expert of the case study company, Mr. Peter
Manet.
PEOU1 It was easy for me to understand what the CHOOSE model was trying to model
PEOU2 Using the CHOOSE model was often frustrating
PEOU3 Overall, the CHOOSE model was easy to use
PEOU4 Learning how to read the CHOOSE model was easy
US1 The CHOOSE model adequately met the information needs that I was asked to support
US2 The CHOOSE model was not efficient in providing the information I needed
US3 The CHOOSE model was effective in providing the information I needed
US4 Overall, I am satisfied with the CHOOSE model for providing the information I needed
PU1 Overall, I think the CHOOSE model would be an improvement to a textual description of the company
PU2 Overall, I found the CHOOSE model useful for understanding the company modeled
PU3 Overall, I think the CHOOSE model improves my performance when understanding the
company modeled
PSQ1 The CHOOSE model represents the company correctly
PSQ2 The CHOOSE model is a realistic representation of the company
PSQ3 The CHOOSE model contains contradicting elements
PSQ4 All the elements in the CHOOSE model are relevant for the representation of the company
PSQ5 The CHOOSE model gives a complete representation of the company
96
EA 1 The CHOOSE model has succeeded in managing and controlling the complexity of
the enterprise (and its corresponding systems, processes, …). (Control)
EA 2 The CHOOSE model describes the enterprise completely/holistically. (Holistic
Overview)
EA 3 The CHOOSE model facilitates the translation from strategic objectives to
operational processes. (Objectives)
SME 1 The CHOOSE model allows the SME to work time efficiently on strategic matters.
SME 2 A person with few IT skills must be able to use and/or adapt the CHOOSE model.
SME 3 The CHOOSE model is useful without support from external specialists.
SME 4 The CHOOSE model enables clear process description and overview.
SME 5 The CEO must be involved in the development of the CHOOSE model.
SME 6 The expected returns of the CHOOSE model must be bigger than the expected
costs and risks.
EA 5 The model optimizes the company as a whole instead of doing local optimization
within individual domains. (Enterprise)
Table 6: Ten design criteria of the CHOOSE approach (translated from (Bernaert, Callaert, & Poels, 2013))
97
CHAPTER II: EVALUATION RESULTS
2.1 Evaluation by the SME
After the development and refinement of the Performance-oriented extension of the CHOOSE
approach, both the CHOOSE model (as an instance of the metamodel) and method were
evaluated by the CEO, HR manager, Supply Chain (SC) manager and the external member of
the board (who is a PM expert) by means of a questionnaire based on the questions in Table 3
and Table 5. It is important to notice that the metamodel and method were refined several times
during the cyclical process of Action Research (Susman & Evered, 1978), and thereby the final
CHOOSE metamodel and method were only established after some time. As a result, the
evaluators cannot be considered as independent evaluators. Furthermore, as the case study
research consisted of only one case study, the results cannot be generalized statistically (Yin,
2009).
The results of the method and model evaluation are shown in Table 7. The results of the
negatively formulated statements of PEOU2, US2, and PSQ3 are interpreted as the inverse
results (so 1 becomes 7, etc.) to calculate the average and interpret these results correctly.
2.1.1 General Evaluation
The participants of the enterprise in general evaluated the CHOOSE approach on average as
‘very good’ on a 5-point Likert scale (from ‘very bad’ to ‘very good’) (cf. General Evaluation in
Table 7, average: 4.75).
The CEO is the only person who evaluated the CHOOSE approach as ‘good’ rather than ‘very
good’. He argues that the CHOOSE approach could be a very useful tool to manage the
complexities of the firm. Identifying measures through the Performance-oriented viewpoint
furthermore stimulates a good way of thinking and a positive attitude in the organization.
However, he remarks that the failure or success of the model will depend on the persons
keeping the model up to date. Furthermore, he wonders who will construct the initial model and
whether or not this should be an external person. These remarks corresponds with the critical
remark of the CEOs in case studies three and six of Bernaert, Callaert and Poels (2013): “Who
within the SME will make an initial EA model and will keep it up to date?”
The HR manager states that the model “gives an organized and clear view of what really matters
and clearly shows where the company can still improve, but the difficulty will be to keep a
98
general overview without having an overload of information”. In addition, the SC manager wishes
to use the model on a permanent basis in the future, as it clearly shows the pressing issues the
company is currently dealing with. The external member of the board, who is an expert in PM,
furthermore argues the following: “The performance-oriented viewpoint is a substantial
improvement to the CHOOSE approach. As I am big believer of ‘What you measure is what you
get’, I certainly suggest including this extension in the final CHOOSE approach”.
Table 7: Evaluation results of CHOOSE metamodel and method by the CEO, SC manager, HR manager and member of the board
99
2.1.2 Method Evaluation
The CHOOSE method was evaluated based on MEM (Moody, 2003). To remind the participants
of the case study company of the applied method, a transcript of the applied CHOOSE method
(cf. PART III, Section (5)) was first given before filling in the questionnaire.
The adapted six-item scales of Davis (1989) (cf. Table 3) of both variables (Perceived
Usefulness and Perceived Ease of Use) are positively evaluated. The participants on average
agree that the CHOOSE method has a high degree of Perceived Usefulness (cf. PU in Table 7,
average: 6.08). The external member of the board states the following: “The strength of the
CHOOSE approach lies in the structured process representation (i.e. the method). Without this
method, it would be difficult to interpret the metamodel.” Nevertheless, the HR manager only
gives an average score in between ‘neutral’ and ‘slightly agree’ on the Perceived Usefulness.
This could be caused by the fact that the HR manager was the only person who did not have
any prior knowledge of the used framework (BSC) and of performance indicators in general. The
persons involved also agree that the CHOOSE method has a high degree of Perceived Ease of
Use (cf. PEU in Table 7, average: 5.96).
The high score of the Perceived Usefulness and Ease of Use will lead to an Intention to Use and
to the Actual Usage of the CHOOSE method and thereby to the Actual Usage of the CHOOSE
approach. This inference is based on the MEM (cf. Figure 44) (Moody, 2003).
2.1.3 Model Evaluation
Based on the UEBQM (A. Maes & Poels, 2007), the participants of the case study agreed that
the CHOOSE model (as an instance of the CHOOSE metamodel) has a good Perceived Ease of
Understanding (cf. PEOU in Table 7, average: 6,13) and Perceived Semantic Quality (cf. PSQ in
Table 7, average: 5,95).
The SC manager did however have a remark concerning the third question of the Perceived
Ease of Understanding (cf. PEOU3 in Table 5). She noted that it was difficult and ambiguous to
answer this question correctly, since the model was constructed by the CHOOSE Architect
rather than the participants of the case study. The same remark has also been made during the
second and third case study of Bernaert, Callaert and Poels (2013). Therefore, it could be useful
to more actively involve the participants of the company during the construction of their
CHOOSE model.
100
In accordance with the causal relationships in UEBQM (cf. Figure 45), this resulted in an
approval with a good Perceived Usefulness (cf. PU in Table 7, average: 6.25) and with a good
User Satisfaction (cf. US in Table 7, average: 5.88).
2.1.4 General Evaluation of the Existing CHOOSE Approach
As a side note, some general evaluation remarks concerning the existing CHOOSE approach
established by Bernaert, Callaert and Poels (2013) are given as well. During the first interview
where the Goal dimension was elaborated, both the CEO and the external member of the board
found the following three aspects very useful:
(1) The refinements of Goals through the ‘How’ and ‘Why’ questions are a convenient and
efficient way to establish the goal tree.
(2) By the clear distinction between the different dimensions (Goal, Actor, Operation and
Object) and by starting with the elaboration of the goal tree, the CHOOSE approach
ensures that the CEO first thinks on a very general level and does not immediately
proceed to the corresponding actions (Operations).
(3) The elaboration of the goal tree is of great value during a strategy meeting with the
board.
2.2 Evaluation by an expert
The CHOOSE model was evaluated based on the ten initial criteria for EA in an SME context (cf.
Table 6) by Mr. Mark Van Gastel, an expert in PM and ICT Architecture. The aim of this
evaluation step is to indirectly evaluate the CHOOSE metamodel towards these criteria, as the
CHOOSE model is an instance of the CHOOSE metamodel. The results are shown in Table 8.
CHOOSE
Criteria EA 1 EA 2 EA 3 SME 1 SME 2 SME 3 SME 4 SME 5 SME 6 EA 5
Yes / No YES NO YES YES YES NO YES YES YES YES
Table 8: Evaluation results of the CHOOSE model by the external PM and ICT Architecture expert
As aforementioned in the introduction of PART III, the Performance-oriented dimension focuses
mainly on the Control criteria (EA 1) and the Objective criteria (EA 3), while keeping in mind the
Suitability for SMEs (SME 1 to SME 6). The expert positively evaluated these criteria, expect for
the third SME criteria (SME 3 in Table 8), which states that the CHOOSE model could be used
101
without support from external specialists. Furthermore, the expert believes that the CHOOSE
model does not describe the enterprise completely or holistically (EA 2 in Table 8).
Nevertheless, this could be caused by the intention of this particular case study research: the
CHOOSE model (as an instance of the CHOOSE metamodel) was constructed to develop and
refine the Performance-oriented dimension of the CHOOSE approach. Thereby, this master
thesis did not aim to construct a complete CHOOSE model and provide a holistic overview.
Moreover, the expert confirms the value in practice of the added sentence in step one of the
Roadmap-method (Goal dimension: “The CEO of the enterprise is asked to first define the
strategy, vision and mission of the enterprise and afterwards define the highest-level Goal in
each of the four Balanced Scorecard dimensions”, cf. PART III, Section 3.3(5)). He states that
“in practice, it is more convenient to start with the value proposition before starting with the four
BSC perspectives”.
In addition, another PM expert (Mr. Peter Manet) evaluated the CHOOSE model based on the
CHOOSE criteria. As this expert is a member of the board of the case study company, the
evaluation results are not independent and value-free, but they can still be important. The results
(cf. Table 9) correspond with the results of the external expert in Table 8, except for the second
SME criteria (SME 2 in Table 9).
CHOOSE
Criteria EA 1 EA 2 EA 3 SME 1 SME 2 SME 3 SME 4 SME 5 SME 6 EA 5
Yes / No YES NO YES YES NO NO YES YES YES YES
Table 9: Evaluation results of the CHOOSE model by the internal PM expert
102
PART V: CONCLUSION
First of all, this part provides a general conclusion of the master thesis, then it presents its
limitations and finishes with several guidelines for future research.
GENERAL CONCLUSION
This master thesis presented a Performance Management System for the CHOOSE approach
for Enterprise Architecture for SMEs. Two artifacts of the existing CHOOSE approach (Bernaert,
Callaert, Poels, et al., 2013) are extended with a Performance-oriented dimension: the
metamodel and the method. This sets the total number of viewpoints to five: a Goal, Actor,
Operation, Object and Performance-oriented viewpoint. The extension can be considered as an
optional add-on to the current CHOOSE approach, specifically developed for SMEs that are
facing increasing complexities and that perceive the need to measure the achievement of their
goals.
The additional viewpoint permits SMEs to measure their goals through indicators, provided that
these goals are written in a SMART way (Specific, Measurable, Attainable, Relevant and Time-
bound). An indicator has an assigned actor, can include another indicator and is measured
against a long-term target and possibly against several other short-term milestones as well. To
achieve these milestones, appropriate actions (operations) can be defined. Furthermore, the
Performance-oriented viewpoint gives the SME the opportunity to calculate the degree of goal
satisfaction and to perform a more profound and detailed as-is/to-be goal analysis than was the
case previously.
The Performance-oriented viewpoint was accomplished in three consecutive phases: a literature
research phase, a case study research phase and an evaluation phase. Since the CHOOSE
approach is an EA approach for SMEs (Bernaert et al., 2014), the literature research first of all
focused, where possible, on studies concerning SMEs and secondly, the selected case study
enterprise complies with the definition of SMEs. Despite the fact that the case study research
was executed from an interpretive perspective, the two positivist design tests (external validity
and reliability) were taken into consideration (Yin, 2009).
Based on the literature research, four classes (two general classes and two specialization
classes) and five relationships were added to the existing CHOOSE metamodel (Bernaert,
103
Callaert, Poels, et al., 2013). Furthermore, several guidelines were given for the CHOOSE
method.
Secondly, the metamodel and method were extended and refined through the cyclical process of
Action Research by a Case Study Research (Susman & Evered, 1978) that consisted of twelve
interviews. By developing and refining the CHOOSE model (as an instance of the CHOOSE
metamodel) during the first nine interviews, it was possible to make ten adjustments to the
metamodel. The metamodel stayed robust during the last three interviews. In addition, the
CHOOSE method was adapted: one step was added to the CHOOSE Roadmap-method, an
additional step was added to the second interview of the Interview-method and another step was
added to the Stop-criteria.
As a final stage, the CHOOSE approach with the Performance-oriented dimension was
positively evaluated by the participants of the case study enterprise. Furthermore, a PM and ICT
Architecture expert evaluated the developed CHOOSE model towards the initial criteria for EA in
an SME context (Bernaert et al., 2014). This evaluation step confirmed the fit between the
CHOOSE model and these initial criteria.
The impact of this master thesis can be substantial as, to the knowledge of the author, it is one
of the first EA approaches for SMEs that incorporates a performance management system
(Cardoso, 2013a). Previous research clearly indicates the necessity for a tailored approach
adjusted to the specific needs of SMEs, both for EA (Bernaert et al., 2014) as for PM (Chalmeta
et al., 2012; Cocca & Alberti, 2010; Garengo et al., 2005; Hudson et al., 2001; Nudurupati &
Bititci, 2011; Rompho, 2011). The key contributions of this master thesis are thus the
development and refinement of a Performance-oriented viewpoint for the CHOOSE metamodel
and method, adapted to the characteristics of an SME context.
LIMITATIONS
This master thesis has four main limitations. The first limitation concerns the limited number of
case studies. Because the Action Research program consists of only one case study (single-
case design) (Yin, 2009), the results cannot be demonstrated statistically.
Secondly, design criteria from literature (Bernaert, Poels, et al., 2013; Paige et al., 2000) were
used during the evaluation of the potential solutions in the solution space (for the development
and refinement of the CHOOSE metamodel and method). Although the evaluation of these
104
criteria for the possible solutions was carefully conducted, it is still by its nature subjective, which
is the second limitation of this master thesis.
Thirdly, the evaluation of the CHOOSE model was conducted by five persons. Four out of five
evaluators were participants of the case study itself, so they cannot be seen as independent
evaluators. As a result, the CHOOSE model is only evaluated by one external expert.
Furthermore, no use was made of an objective measuring instrument.
Finally, most of the data of the case study involve strategic issues. A last limitation of this
research is the fact that the case study database contains confidential data and thus cannot be
made public.
FUTURE RESEARCH
First of all, future research could increase the number of case studies to further test and
evaluate the additional Performance-oriented dimension of the CHOOSE metamodel and
method. On the one hand, implementing the CHOOSE approach in more SMEs would result in a
more robust CHOOSE metamodel and method. On the other hand, implementing the CHOOSE
approach in more companies in general (SMEs and larger companies) would result in more
reliable evaluation results.
Secondly, these case study companies should be monitored on a longer term to assess the
amount of effort they have spent to keep their CHOOSE model up to date. This master thesis
focused on the use of CHOOSE in developing EA models integrated with performance
indicators, but more attention should be given to the management of such models.
Thirdly, it would be advantageous if the Performance-oriented dimension of the CHOOSE
metamodel and method were visualized through the CHOOSE tool support. A color coding could
for instance be used to measure the degree of goal realization and alarms could ring when
actual and target values differ too much. This tool support would certainly enhance the usability
of the Performance-oriented viewpoint of the CHOOSE approach.
I
BIBLIOGRAPHY
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision
Processes, 50(2), 179–211.
Amaratunga, D., & Baldry, D. (2002). Moving from performance measurement to performance
management. Facilities, 20(5/6), 217–223.
Andersen, H., Cobbold, I., & Lawrie, G. (2001). Balanced scorecard implementation in SMEs:
reflection on literature and practice. In 4th SME International Conference, Alborg University,
Denmark.
Andersson, B., Bergholtz, M., Edirisuriya, A., Ilayperuma, T., Jayaweera, P., Johannesson, P., &
Zdravkovic, J. (2008). Enterprise sustainability through the alignment of goal models and
business models. In Proceedings of 3rd International Workshop on Business/IT-Alignment
and Interoperability (BUSITAL’08) CEUR Workshop Proceedings.
Band, W. (1990). Performance Metrics Keep Customer Satisfaction Programs on Track.
Marketing News (MNW), 24.
Baskerville, R. (1997). Distinguishing action research from participative case studies. Journal of
Systems and Information Technology, 1(1), 24–43.
Baskerville, R. (1999). Investigating information systems with action research. Communications
of the AIS, 2, 4.
Baskerville, R., & Myers, M. (2004). Special Issue on Action Research in Information Systems:
Making IS Research Relevant to Practice: Foreword. Mis Quarterly, 28(3), 329–335.
Benbasat, I., Goldstein, D., & Mead, M. (1987). The case research strategy in studies of
information systems. MIS Quarterly, 369–386.
Bernaert, M., Callaert, M., & Poels, G. (2013). Business Architectuur Modellering in KMO’s:
Case Study Onderzoek ter Verfijning en Validatie van de CHOOSE Methode en
Metamodel. Ghent University.
Bernaert, M., Callaert, M., Poels, G., Snoeck, M., & De Backer, M. (2013). Enterprise
Architecture for Small and Medium-Sized Enterprises: Action Research to Develop, Refine,
and Evaluate the CHOOSE Method and Metamodel.
Bernaert, M., & Poels, G. (2011a). De zoektocht naar know-how, know-why, know-what en
know-who: architectuur voor kleinere bedrijven in vier dimensies. Informatie (Amsterdam),
53(9), 34–41.
II
Bernaert, M., & Poels, G. (2011b). The Quest for Know-How, Know-Why, Know-What and
Know-Who: Using KAOS for Enterprise Modelling. In Advanced Information Systems
Engineering Workshops (pp. 29–40). Springer.
Bernaert, M., & Poels, G. (2012). Enterprise architecture management for small and medium
sized enterprises: a case study and tool support. Ghent University, Faculty of Economics
and Business Administration.
Bernaert, M., Poels, G., Snoeck, M., & De Backer, M. (2013). CHOOSE: Towards a Metamodel
for Enterprise Architecture in Small and Medium-sized Enterprises. Ghent Universtiy.
Bernaert, M., Poels, G., Snoeck, M., & De Backer, M. (2014). Enterprise Architecture for Small
and Medium-Sized Enterprises: A Starting Point for Bringing EA to SMEs, Based on
Adoption Models. Information Systems for Small and Medium-Sized Enterprises, 67–96.
Bititci, U. S., Turner, T., & Begemann, C. (2000). Dynamics of performance measurement
systems. International Journal of Operations and Production Management, 20(6), 692–704.
Braun, C., & Winter, R. (2005). A comprehensive enterprise architecture metamodel and its
implementation using a metamodeling platform. Desel, Jörg; Frank, Ulrich, 24–25.
Braun, C., & Winter, R. (2007). Modellierung der Unternehmensarchitektur: Weiterentwicklung
einer bestehenden Methode und deren Abbildung in einem Meta-Modellierungswerkzeug.
Broadbent, J., & Laughlin, R. (2009). Performance management systems: A conceptual model.
Management Accounting Research, 20(4), 283–295.
Brooks, P., & Chittenden, J. (2012). Metrics for Service Management: Designing for ITIL. Van
Haren Publishing.
Cardoso, E. (2013a). Challenges in Performance Analysis in Enterprise Architectures. In 2013
17th IEEE International (Ed.), Enterprise Distributed Object Computing Conference
Workshops (EDOCW), 2013 17th IEEE International (pp. 327–336). 2013 17th IEEE
International (pp. 327-336). IEEE.
Cardoso, E. (2013b). Challenges in Supporting a Goal-Oriented Enterprise Architecture
Analysis. In Proceedings of the 5th Central-European Workshop on Services and their
Composition. Zeus.
Cardoso, E. (2013c). Methodologies for Goal-Oriented Process Performance Management.
Proceedings of the 6th Ph. D. Retreat of the HPI Research School on Service-Oriented
Systems Engineering, 76, 11.
Cardoso, E., & Almeida, J. (2010). On the support for the goal domain in enterprise modelling
approaches. In Enterprise Distributed Object Computing Conference Workshops (EDOCW),
2010 14th IEEE International. IEEE.
III
Cassell, C., Nadin, S., & Gray, M. (2001). The use and effectiveness of benchmarking in SMEs.
Benchmarking: An International Journal, 8(3), 212–222.
Chalmeta, R., Palomero, S., & Matilla, M. (2012). Methodology to develop a performance
measurement system in small and medium-sized enterprises. International Journal of
Computer Integrated Manufacturing, 25(8), 716–740.
Chang, R., & Morgan, M. (2000). Performance scorecards: Measuring the right things in the real
world.
Chang, R., & Morgan, M. (2001). Prestatiemeting met scorecards. Academic Service.
Chenhall, R. H. (2003). Management control systems design within its organizational context:
Findings from contingency-based research and directions for the future. Accounting,
Organizations and Society.
Chennell, A., & Dransfield, S. (2000). OPM: a system for organisational performance
measurement. In Proceedings of the performance measurement–past, present and future
conference (pp. 19–21). Cambridge.
Cocca, P., & Alberti, M. (2010). A framework to assess performance measurement systems in
SMEs. International Journal of Productivity and Performance Management, 59(2), 186–200.
Commission, E. (2005). The new SME definition: user guide and model declaration. Office for
Official Publications of the European Communities.
Cross, K., & Lynch, R. (1991). The “SMART” way to define and sustain success. National
Productivity Review, 8(1), 23–33.
Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of
information technology. MIS Quarterly, 319–340.
Dechow, N., Granlund, M., & Mouritsen, J. (2006). Management control of the complex
organization: relationships between management accounting and information technology.
Handbooks of Management Accounting Research, 2, 625–640.
Devos, J. (2011). Methodologie voor wetenschappelijk IT onderzoek. University of Ghent. Ghent
University.
Doran, G. T. (1981). There’s a S.M.A.R.T. way to write management's goals and objectives.
Management Review, 70, 2.
Dumeez, J., Bernaert, M., & Poels, G. (2013). Development of Software Tool Support for
Enterprise Architecture in Small and Medium-Sized Enterprises. In CAiSE International
Workshops. Springer.
Eckerson, W. (2009). Performance management strategies. Business Intelligence Journal.
IV
European Union. (2003). Commission recommendation of 6 may 2003 concerning the definition
of micro, small and medium-sized enterprises. Official Journal of the European Union, L,
2003/361/E.
Ferreira, A. (2002). Management accounting and control systems design and use: an
exploratory study in Portugal. Lancaster University.
Ferreira, A., & Otley, D. (2009). The design and use of performance management systems: An
extended framework for analysis. Management Accounting Research, 20(4), 263–282.
Fisher, J. G., Peffer, S. A., & Sprinkle, G. B. (2003). Budget Based Contracts, Budget Levels,
and Group Performance. Journal of Management Accounting Research.
Fitzgerald, L., Johnson, R., Brignall, S., Silvestro, R., & Voss, C. (1991). Performance
Measurement in Service Businesses. London: CIMA.
Frank, U., Heise, D., Kattenstroth, H., & Schauer, H. (2008). Designing and Utilising Business
Indicator Systems within Enterprise Models-Outline of a Method. MobIS, 89–105.
Garengo, P., Biazzo, S., & Bititci, U. (2005). Performance measurement systems in SMEs: a
review for a research agenda. International Journal of Management Reviews, 7(1), 25–47.
Gartner. (2014). Gartner Says Organizations Using Predictive Business Performance Metrics
Will Increase Their Profitability 20 Percent by 2017. Press Release. Retrieved May 08,
2014, from http://www.gartner.com/newsroom/id/2650815
Giannoulis, C. (2011). Schema Integration Process for UBSMM.
Giannoulis, C., Petit, M., & Zdravkovic, J. (2011a). Modeling business strategy: A meta-model of
strategy maps and balanced scorecards. In Research Challenges in Information Science
(RCIS) (Vol. Fifth Inte). IEEE.
Giannoulis, C., Petit, M., & Zdravkovic, J. (2011b). Modeling competition-driven business
strategy for business IT alignment. In Advanced Information Systems Engineering
Workshops (pp. 16–28). Springer Berlin Heidelberg.
Giannoulis, C., Zdravkovic, J., & Petit, M. (2012). Model-driven strategic awareness: From a
unified business strategy meta-model (UBSMM) to enterprise architecture. Enterprise,
Business-Process and Information Systems Modeling, 255–269.
Giannoulis, C., & Zikra, I. (2013). A Comparative Analysis of Enterprise Modeling Approaches
for Modeling Business Strategy. In The Practice of Enterprise Modeling: Proceedings of the
6th IFIP WG 8.1 Conference on the Practice of Enterprise Modeling (pp. 193–204).
Grafton, J., Lillis, A., & Widener, S. (2010). The role of performance measurement and
evaluation in building organizational capabilities and performance. Accounting,
Organizations and Society.
V
Henderson, J. C., & Venkatraman, H. (1993). Strategic alignment: Leveraging information
technology for transforming organizations. IBM Systems Journal.
Hevner, A. R., March, S. T. S., Park, J., & Ram, S. (2004). Design Science in Information
Systems Research. MIS Q, 28(1), 75–105.
Hope, J., & Fraser, R. (2003). New Ways of Setting Rewards: THE BEYOND BUDGETING
MODEL. California Management Review, 45, 104–119.
Hudson, M., Bennett, J., Smart, A., & Bourne, M. (1999). Performance measurement for
planning and control in SMEs. In Global Production Management (pp. 219–225). Springer.
Hudson, M., Smart, A., & Bourne, M. (2001). Theory and practice in SME performance
measurement systems. International Journal of Operations & Production Management, 21,
1096–1115.
Hudson, M., & Smith, D. (2007). Implementing strategically aligned performance measurement
in small firms. International Journal of Production Economics.
Iacob, M., Quartel, D., & Jonkers, H. (2012). Capturing business strategy and value in enterprise
architecture to support portfolio Valuation. In Enterprise Distributed Object Computing
Conference (EDOC), IEEE 16th International (pp. 11–20).
IEEE Computer Society. (2000). IEEE Recommended Practice for Architectural Description of
Software-Intensive Systems. IEEE Std 1471-2000.
Ingelbeen, D., Bernaert, M., & Poels, G. (2013). Enterprise Architecture Software Tool Support
for Small and Medium-Sized Enterprises: EASE (pp. 1–11). 19th Americas Conference of
Information Systems (AMCIS).
Järvinen, P. (2007). Action Research is Similar to Design Science. Quality & Quantity, 41(1), 37–
54.
Johnson, G., Scholes, K., & Whittington, R. (2008). Exploring corporate strategy: text & cases.
Jonkers, H., Lankhorst, M., ter Doest, H., Arbab, F., Bosma, H., & Wieringa, R. J. (2006).
Enterprise architecture: Management tool and blueprint for the organisation. Inf Syst Front,
8(2), 63–66.
Jonkers, H., Proper, E., & Turner, M. (2009). TOGAF 9 and ArchiMate 1.0. White Paper.
Kaplan, R., & Norton, D. (1992). The Balanced Scorecard – Measures that Drive Performance.
Harvard Business Review, Januray -, 71–79.
Kaplan, R., & Norton, D. (1996a). The balanced scorecard: translating strategy into action.
Harvard Business Press.
Kaplan, R., & Norton, D. (1996b). Using the balanced scorecard as a strategic management
system. Harvard Business Review, (October 1993), 75–86.
VI
Kaplan, R., & Norton, D. (2001). Transforming the balanced scorecard from performance
measurement to strategic management: Part I. Accounting Horizons.
Kaplan, R., & Norton, D. (2004). Strategy maps: Converting intangible assets into tangible
outcomes.
Keegan, D., Eiler, R., & Jones, C. (1989). Are your performance measures obsolete?
Management Accounting, 70, 45–50.
Kronz, A. (2006). Managing of process key performance indicators as part of the ARIS
methodology. In Corporate performance management: ARIS in practice. Springer.
Laitinen, E. (2002). A dynamic performance measurement system: evidence from small Finnish
technology companies. Scandinavian Journal of Management, 18, 65–99.
Lankhorst, M. (2009). Enterprise Architecture at Work: Modelling, Communication and Analysis.
The Enterprise Engineering Series, (Vol. 3, p. xviii, 334 s.). Berlin, Heidelberg: Springer.
Lankhorst, M., Quartel, D., & Steen, M. (2010). Architecture-based IT portfolio valuation. In
Poceeding of practie-driven research on Enterprise Transformations, The Netherlands (pp.
78–106). Springer.
Lebas, M. (1995). Performance measurement and performance management. International
Journal of Production Economics.
Locke, E., & Latham, G. (1990). A theory of goal setting & task performance.
Maes, A., & Poels, G. (2007). Evaluating quality of conceptual modelling scripts based on user
perceptions. Data & Knowledge Engineering, 63(3), 701–724.
Maes, J., Poels, G., & Bernaert, M. (2013). Business architecture modelling in CHOOSE: a
visual application for Android tablets. Ghent University.
Maes, R. (2007). An integrative perspective on information management. Information
Management: Setting the Scene, 11–26.
Marchini, I. (1995). Il governo della piccola impresa, vol. 3 - La gestione delle funzioni. Genova:
ASPI/ISN-EDIT.
Markovic, I., & Kowalkiewicz, M. (2008). Linking business goals to process models in semantic
business process modeling. In IEEE (Ed.), Enterprise Distributed Object Computing
Conference, 2008. EDOC’08. 12th International IEEE (pp. 332–338).
Marr, B. (2012). Key Performance Indicators (KPI): The 75 measures every manager needs to
know.
Matthes, F., & Monahov, I. (2012). A Template-Based Design Method to Define Organization-
Specific KPIs for the Domain of Enterprise Architecture Management. DASMA Software
Metrik Kongress.
VII
McAdam, R. (2000). Quality models in an SME context: A critical perspective using a grounded
approach. International Journal of Quality & Reliability Management.
Meyer, P. (2003). What would you do if you knew you couldn’t fail? Creating SMART Goals. In
Attitude Is Everything: If You Want to Succeed Above and Beyond.
Meyer, P. J. (2003). Attitude Is Everything: If You Want to Succeed Above and Beyond.
Mintzberg, H., Lampel, J., Quinn, J., & Ghoshal, S. (2003). The Strategy Process: Concepts,
Contexts, Cases. Pearson Eduction.
Mohr, L. (1982). Explaining Organizational Behavior (Vol. 1). Jossey-Bass San Fransisco.
Moody, D. (2003). The method evaluation model: a theoretical model for validating information
systems design methods. ECIS, 1327–1336.
Neely, A. (1999). The performance measurement revolution: why now and what next?
International Journal of Operations & Production Management, 19(2), 205–228.
Neely, A., Adams, C., & Kennerley, M. (2002). The performance prism: The scorecard for
measuring and managing business success (p. 393). London: Prentice Hall Financial
Times.
Neely, A., Richards, H., Mills, J., Platts, K., & Bourne, M. (1997). Designing performance
measures: a structured approach. International Journal of Operations & Production
Management, 17, 1131–1152.
Niven, P. R. (2006). Balanced Scorecard Step-by-Step: Maximizing Performance and
Maintaining Results (2nd ed., p. 318). Wiley.
Nudurupati, S., & Bititci, U. (2011). State of the art literature review on performance
measurement. Computers & Industrial Engineering, 60(2), 279–290.
Object Management Group. (2013). Business Motivation Model (BMM) specification. Retrieved
April 13, 2014, from http://www.omg.org/spec/BMM/1.2/
Otley, D. (1999). Performance management: a framework for management control systems
research. Management Accounting Research, 10, 363–382.
Otte, M., Poels, G., & Bernaert, M. (2013). Next Generation Media: A used-friendly iPad
application for business architecture modelling. Ghent University.
Paige, R., Ostroff, J., & Brooke, P. (2000). Principles for modeling language design. Information
and Software Technology, 42(10), 665–675.
Parmenter, D. (2011). Key Performance Indicators: Developing, Implementing,and Using
Winning KPIs (2nd ed.). John Wiley & Sons.
Paulen, B., & Finken, J. (2009). Key performance Indicators. Pro SQL Server 2008 Analytics:
Delivering Sales and Marketing Dashboards, 32–52.
VIII
Popova, V., & Sharpanskykh, A. (2008). Formal Goal-based Modeling of Organizations.
MSVVEIS.
Popova, V., & Sharpanskykh, A. (2011). Formal modelling of organisational goals based on
performance indicators. Data & Knowledge Engineering Journal, 70(4), 335–364.
Porter, M. (1985). Competitive advantage: creating and sustaining superior performance. Free
Press, New York.
Porter, M. E., & Summers, L. (1992). Capital Disadvantage : America ’ s Failing Capita
investment System. Harvard Business Review.
Pourshahid, A., Amyot, D., Chen, P., Weiss, M., & Forster, A. (2007). Business Process
Monitoring and Alignment: An Approach Based on the User Requirements Notation and
Business Intelligence Tools. WER, 80–91.
Puylaert, O., Bernaert, M., & Poels, G. (2013). Business architectuur modellering in CHOOSE:
een gebruiksvriendelijke applicatie voor de iPhone. Ghent University.
Rescher, N. (1977). Methodological Pragmatism: A Systems-Theoretic Approach to the Theory
of Knowledge. Blackwell.
Rompho, N. (2011). Why the Balanced Scorecard Fails in SMEs: A Case Study. International
Journal of Business & Management, 6(11).
Ross, J. W., Weill, P., & Robertson, D. (2006). Enterprise Architecture as Strategy: Creating a
Foundation for Business Execution. Harvard Business Review Press.
Scheer, A.-W. (2000). ARIS - Business Process Modeling (Vol. 3). Springer.
Schekkerman, J. (2004). How to Survive in the Jungle of Enterprise Architecture Frameworks:
Creating or Choosing an Enterprise Architecture Framework. Trafford Publishing.
Schelp, J., & Stutz, M. (2007). A balanced scorecard approach to measure the value of
enterprise architecture. Journal of Enterprise Architecture, (1), 5–12.
Sessions, R. (2007). Comparison of the Top Four Enterprise Architecture Methodologies.
Shahin, A., & Mahbod, M. (2007). Prioritization of key performance indicators: An integration of
analytical hierarchy process and goal setting. International Journal of Productivity and
Performance Management, 56, 226–240.
Solingen, R. Van, & Berghout, E. (1999). The Goal/Question/Metric Method: A Practical Guide
for Quality Improvement of Software Development.
Solvay Brussels School of Economics and Management, VOKA, & TiasNimbas Business
School. (2014). MBA Highlights. module 2, friday 22/02/2013. Retrieved from
http://www.voka.be/limburg/diensten/mba-highlights/
IX
Susman, G. I., & Evered, R. D. (1978). An Assessment of the Scientific Merits of Action
Research. Administrative Science Quarterly, 582–603.
Svee, E., Giannoulis, C., & Zdravkovic, J. (2011). Modeling Business Strategy: A Consumer
Value Perspective. The Practice of Enterprise Modeling.
Tavernier, V., & Poels, G. (2012). Ontwikkeling van een gevalstudie voor enterprise architecture.
Master of science in de Toegepaste Economische Wetenschappen: Handelsingenieur.
Universiteit Gent.
Tenhunen, J., Rantanen, H., & Ukko, J. (2001). SME-oriented implementation of a performance
measurement system. Department of Industrial Engineering and Management -
Lappeenranta University of Technology, Lahti.
The KPI Institute. (2014). On Key Performance Indicators. Retrieved May 08, 2014, from
http://www.smartkpis.com/key-performance-indicator-KPI
The Open Group. (2003). TOGAF “Enterprise Edition” Version 8.1. Retrieved April 28, 2014,
from http://pubs.opengroup.org/architecture/togaf8-doc/arch/
The Open Group. (2011). TOGAF Version 9.1. Van Haren Publishing. Retrieved April 28, 2014,
from http://www.opengroup.org/togaf
The Open Group. (2012). ArchiMate 2.0. Retrieved April 28, 2014, from
http://www3.opengroup.org/subjectareas/enterprise/archimate
Verhulst, P., Bernaert, M., & Poels, G. (2013). Business Architectuur modellering in CHOOSE:
een gebruiksvriendelijke applicatie aangepast aan de user interface van de iPad. Ghent
University.
Vichitdhanabadee, J., Wilmshurst, D., & Clift, R. (2009). Peformance management practice in
the Thai SMES. In Performance Measurement Association Conference.
Wagter, R., Van Den Berg, M., Luijpers, J., & van Steenbergen, M. (2005). Dynamic Enterprise
Architecture: How to Make It Work. John Wiley & Sons.
Winter, R. (2011). Business engineering navigator - Gestaltung und Analyse von
Geschäftslösungen“ Business-to-IT.” Springer.
Wittenburg, A. (2007). Softwarekartographie: Modelle und Methoden zur systematischen
Visualisierung von Anwendungslandschaften.
Yemm, G. (2012). FT Essential Guide to Leading Your Team: How to Set Goals, Measure
Performance and Reward Talent.
Yin, R. (2009). Case study research: Design and methods (Vol. 5). Sage Publications.
Yu, E. (2011). Modelling strategic relationships for process reengineering. Social Modeling for
Requirements Engineering, 11.
X
Zachman, J. A. J. (1987). A framework for information systems architecture. IBM Systems
Journal, 26(3), 276–292.
Zutterman, S., Bernaert, M., & Poels, G. (2013). Development of a Tool for Business
Architecture Modeling in Eclipse. Ghent University.
XI
APPENDICES
APPENDIX 1
The Roadmap-method
(1) (Goal dimension): The CEO of the enterprise is asked to define the highest-level Goal in
each of the four Balanced Scorecard dimensions.
(2) (Goal dimension): From the four highest-level Goals, the Goals are broken down into
lower-level Goals. More Goals are found through the 'Why/How?'-questions (Bernaert
and Poels 2011).
(3) (Actor dimension): The Human Actors, Roles, Software Actors, and Devices are added
through interviewing and consulting other secondary sources (e.g., organization charts).
(4) (Operation dimension): The different activity groups of Porter’s Value Chain (Porter and
Millar 1985) are used to elicit the Operations (Processes and Projects). The Operations
are linked with the other two dimensions (Goals and Actors).
(5) (Object dimension): The Objects and corresponding relationships with Goals (Concern),
Actors (Monitoring/Control), and/or Operations (Input/Output) are added to the model
based on the interviews and visual inspection.
(6) (Validation): The last step of the Roadmap is a validation round of the model by the CEO.
The Interview-method
(1) During the first interview, Step 1 and 2 of the Roadmap are executed in order to create a
holistic Goal tree of the enterprise. The second part of the interview focuses on the Actor
tree by executing Step 3.
(2) The second interview is divided in three parts. First of all, a questionnaire, created after
the analysis and implementation of the Goal and Actor tree of the first interview, is used
to fill the gaps (e.g., missing elements or relationships) of the Goal and Actor tree.
Secondly, the activity groups of Porter’s Value Chain are used to define the Operations
of the enterprise in each activity group (Step 4). Thirdly, the Objects of the enterprise are
added based on their relationships with the other three dimensions (Step 5).
(3) The third interview consists of two parts. Firstly, after analysis and implementation of the
Operations tree and Objects, the interviewer tries to find missing constructs and relations
XII
or resolve ambiguities. Secondly, the interviewee (CEO of the SME) visually validates the
final CHOOSE-model (Step 6).
The Stop-criteria
(1) Goal dimension: The Goal-check is twofold.
(c) Does the downward Goal Refinement path of each Goal AND-Refinement and of
one alternative of each Goal OR-Refinement only stop if at least one of its fine-
grained Goals is Operationalized?
(d) Is every lowest-level (AND-Refinement) Goal Operationalized by an Operation?
If this is not possible, there are two possibilities: it is a to-be Goal (another
alternative of a Goal OR-Refinement) or there is a misfit between strategy and
Operations (Henderson and Venkatraman 1993).
(2) Actor dimension: The Actor-check is also twofold:
(a) Are all employees (Human Actors and Roles) of the organization chart, all
other relevant stakeholders, and all relevant Software Actors and Devices
included in the Actor dimension?
(b) Is every Actor linked to at least one Operation?
(3) Operation dimension: First of all, take into consideration that relationships can
sometimes be transferred to a related element (Bernaert et al. 2013a): a superOperation
groups the relations (with e.g., Goals) of its subOperations and a lower-level Goal
inherits the relations of its related higher-level Goals (with e.g., Operations). If for
example a superOperation does not Operationalize any Goal, however one of its
subOperations does, then this superOperation actually also Operationalizes this Goal
through one of its subOperations. The Operation-check is then threefold:
(e) Are all Performed Operations for every Actor included?
(f) Does each Operation contribute to a Goal?
(g) Is every Operation linked to an Actor?
(4) Object dimension: In analogy with the three other dimensions, the Object-check is also
threefold:
(e) Are all products and resources (either physical or not) of the SME represented in
the Object model?
(f) Are all the necessary Objects (Input/Output) of the Operations (Operation
dimension) included in the Object dimension?
(g) Are all the necessary Objects of the Goals included in the Object dimension?
XIII
APPENDIX 2
Case Study Database
INTERVIEW SOURCE FILE
Interview 1
Participants: * CEO (Jan Moons) * External member of the board (Peter Manet)
Duration: 7h30min Date: February, 12
Case study notes
Image_Goal_Tree.JPG (1 to 5) InterviewGuide.docx Goaltree.pptx
Case study documents
Organigram_VMD.xls Organigram_Biové.xls RACI_chart.docx Workshop_VOKA_Notes.pdf Workshop_VOKA_Slides.pdf
Interviews 1.m4a to 10.m4a (10 parts)
Interview 2
Participants: Peter Manet
Duration: 2h Date: March, 7
Case study notes
InterviewGuide.pptx NotesDuringInterview.pdf NotesAfterInterview.docx
Case study documents
/
Interviews 1.m4a tot 4.m4a (4 parts)
Interview 3
Participants: Jan Moons
Duration: 2h10min
Date: March, 16
Case study notes
GoalDimension_beforeInterview.pptx InterviewGuide.docx NotesAfterInterview.pdf NotedDuringInterview.pdf
Case study documents
/
Interviews 1.m4a to 6.m4a (6 parts)
Interview 4
Participants: Supply Chain Manager (Kristel Walschots)
Duration: 50min
Date: March, 21
Case study notes
InterviewGuide.docx ModelConstructionAfterInterview.xlsx NotesAfterInterview.pdf
Case study documents
EmailAfterInterview.docx Example_ForecastFromClient.pdf Example_M-Soft.pdf ForecastProductsBelgium.xlsx OutOfStock_2010_A.xls to OutOfStock_2010_E.xls OutOfStock_2013.xlsx ProductionFromBiové.pdf
Interviews 1.m4a 2.m4a
XIV
Interview 5
Participants: HR manager (Luc Jacobs)
Duration: 1h
Date: March, 21
Case study notes
InterviewGuide.docx NotesAfterInterview.pdf RightPeopleRightSpotModelAfterInterview.xlsx
Case study documents
/
Interviews 1.m4a 2.m4a 3.m4a
Interview 6
Participants: Jan Moons
Duration: 50min
Date: April, 4
Case study notes
InterviewGuide.docx NotesAfterInterview.pdf ReliableForecast_ModelAfterInterview.xlsx ReliableForecast_ModelBeforeInterview.xlsx
Case study documents
/
Interviews 1.m4a 2.m4a 3.m4a
Interview 7
Participants: Kristel Walschots
Duration: 50min
Date: April, 4
Case study notes
InterviewGuid.docx NotesAfterInteriew.pdf ReliableForecast_ModelAfterInterview.xlsx ReliableForecast_ModelBeforeInterview.xlsx
Case study documents
ActivityForForecasting_VEToFlash.docx ExpiryDateProducts.xlsx NumberInStockProducts.xlsx
Interviews 1.m4a 2.m4a
Interview 8
Participants: Luc Jacobs
Duration: 1h
Date: April, 14
Case study notes
InterviewGide.docx NotesDuringInterview.pdf RightPeopleRigheSpot_ModelAfterInterivew.docx RightPeopleRightSpot_ModelBeforeInterview.docx
Case study documents
BalanceSheetAnalysis.xlsx EmployeesEvaluation.xlsx PersonnelRegister.xls
Interviews 1.m4a 2.m4a
Interview 9
Participants: Jan Moons
Duration: 1h
Date: April, 21
Case study notes
FinancialGoalsAfterInterview.xlsx InterviewGuide.docx NotesDuringInterview.pdf
Case study documents
BorrowingBaseCalculation.xsl ConsolidatedP&LAccount2012.pdf ConsolidatedP&LAccountEstimates.xlsx
Interviews 1.m4a 2.m4a
XV
Interview 10
Participants: Kristel Walschots
Duration: 40min
Date: May, 5
Case study notes
InterviewGuide.docx ReliableForecast_ModelBeforeInterview.xls
Case study documents
/
Interviews 1.m4a 2.m4a
Interview 11
Participants: Luc Jacobs
Duration: 20min
Date: May, 5
Case study notes
InterviewGuide.docx RightPeopleRightSpot_ModelBeforeInterview.xlsx
Case study documents
/
Interviews 1.m4a
Interview 12
Participants: Jan Moons
Duration: 10min
Date: May, 5
Case study notes
InterviewGuide.docx
Case study documents
/
Interviews /
Total Case Study Model
GoalDimension.choose_diagram.xlsx FinalCHOOSEModel.xlsx
Interview 13
E-mail Correspondence
Evaluation Questionnaires
Evaluatie_JanMoons.docx Evaluatie_KirstelWalschots.docx Evalutie_LucJacobs.docx Evaluatie_PeterManet.docx Evaluatie_CHOOSEcriteria_PeterManet.docx Evaluatie_CHOOSEcriteria_MarkVanGastel.docx
XVI
APPENDIX 3
Summary of the CHOOSE concepts and Language notation, from (Zutterman et al., 2013).
XVII
XVIII
APPENDIX 4
XIX
Mission, Values, Strategy and highest-level BSC perspectives
XX
Financial Perspective
XXI
Customer Perspective
XXII
Internal Processes Perspective
XXIII
Learning and Innovation Perspective
XXIV
APPENDIX 5
OUT-OF-STOCKS of Biové products for
Belgium (ind. 1)
DELIVERY TIME from Biové to
VMD (ind. 7)
On time forecasting fromspecial export customers
(ind.6)
EXPIRED PRODUCTS (ind.2)
VARIANCE in Belgian Forecasting
(ind.3)
On time forecasting from important & special export
customers (ind.4)
On time forecasting from important export
customers (ind.5)
Causality
Inverse correlation
Sum
LOSS ON OUT-OF-STOCKS
Derived
CEO
NAME Use Reliable forecasting system from VMD to Biové RACI
Description
Reshape the forecasting system from VMD to Biové to
a reliable system so that Biové's delivery time to VMD becomes 8 to 10 weeks.
Responsible
Accountable
Consulted
Informed
(Verantwoordelijk)
(Eindverantwoordelijk)
(Geraadpleegd)
(Geinformeerd)
Who wishes this? Jan Moons
who is assigned? Kristel Walschots
OBJECTIVE
XXV
Name Investigate unexpected peaks/declines
NameOut-of-stocks of Biové-products for Belgium Description
Are there unexpected peaks or declines for specific products?
When? Reason? Estimation of augmentation/decline?
Description
= number of Biové-products for Belgium out of stock
where number out of stock = (stock gecontroleerd) +
(stock niet gecontroleerd) - reservaties
Who (RACI) Responsible: belgische area manager: PJ; Accountable: Kristel
Actual Value 82 Where Experience + Past sales
Frequency Two-weakly Deadline End 2014
Type of target Reduction
Object: m-soft: stocklijst, niet gecontroleerde stocklijst, reservaties Name Are is always particular products that are out of stock?
Actor (RACI):
R: Kristel Description
Compare the out-of-stock lists from the
previous months, years, … and search for
specific products that are more out-of-stock than others
Who (RACI) Resp: Mine/Kristel; Acc.: Kristel
deadline 1 end 2014 Where out-of-stock lists previous years in excell
value 1 50 Deadline End 2014
deadline 2 /
value 2 /
deadline 3 jun/15 Name Perform a stricter and fairer policy for out-of-stocks
target value 10 Description When there is an out-of-stock, the person
who made a forecast, gets priority to the person who did not.
Who (RACI) R: Kristel; I: area manager
Where msoft
Deadline end 2014
Name Investigate why Biové delivers too late for a specific product
Description Are we out-of-stock because Biové delivers too late?
Who (RACI) R: Kristel; C: Laurence (assistente planning)
Where /
Deadline End 2014
Relationship with other classes
INDICATORS Actions
1
Indicator 1
1
2Milestones
3
3
Name Provide on-time information on expiry dates to sales persons
NameExpired products Description
Send a notification well in advance to all sales person
so that they can still sell the product WITHOUT big discounts
Description value of Biové-products in stock expired in the previous year Who (RACI) R: Kristel; A: Kristel; I: sales persons
Actual Value 12500 Where Msoft
Frequency Daily Deadline end 2014
Type of target Reduction
Object: M-soft: 2 different places by Mine and Excell Name Give a weakly list if stock goes under or over min or max stock
Actor (RACI): R: Kristel; Consulted: Mine (she makes the list) Description Automatic alarm when min or max stock is attained
Who (RACI) R: Mine; A: Kristel
deadline 1 end 2014 Where Msoft
value 1 10.000 Deadline end 2014
deadline 2 /
value 2 /
deadline 3 jun/15 Name Get information from registration expiries
target value 5.000 Description Better communication about which registrations are going to expire
Who (RACI) R: Registration department; A: Kristel
Where Msoft
Deadline end 2014
Name Stock & Expiry date together in one document
Description Provide one document with both the stock and expiry date so
that Kristel can check this DAILY
Who (RACI) R: Mine; A: Kristel
Where Msoft
Deadline end 2014
2
Indicator 2
1
Relationship with other classes
2Milestones
3
4
XXVI
Name Get more input from registrations, Sales
Namevariance in Belgian forecasting Description
vb. Antibiotica in België aan banden gelegd, had
eerder moeten gecommuniceerd worden
Description
|estimated value of the last twelve months-
real value of the last twelve months |
/ real value of the last twelve months
Who (RACI) registration department, sales
Actual Value 33% Where /
Frequency Monthly Deadline end 2014
Type of target Reduction
Object: Excel list: ForecastBelgium Name Include promo actions in forecasting
Actor (RACI):
Responsible: Supply Chain Manager,
Consulted: Sales BelgiumDescription
When a product is sold as a discount, more is sold,
so the forecast should be adjusted
Who (RACI) R: PJ, A: Kristel
deadline 1 dec/14 Where /
value 1 25% Deadline end 2014
deadline 2
value 2
deadline 3 jun/15
target value 15%
Name Awareness actions to customer
Name
On time forecasting from important & special
export customersDescription
Show to benefits of the forecasting
(shorter delivery times, …)
Description
number of forecast from important &
special export customers received quarterly /
total number of important & special export customers
Who (RACI) R: Area managers, A: Kristel
Actual Value 40-50% Where /
Frequency Quarterly Deadline june 2015
Type of target Absolute
Object: list of agents Name Awareness actions to area managers
Actor (RACI): R: Kristel; C: agents/sales persons Description Stimulate the area managers to stimulate their clients
Who (RACI) Kristel Walschots
deadline 1 june 2015 Where /
value 1 60% Deadline june 2015
deadline 2 june 2017
value 2 80%
deadline 3 end 2019 Name Reminder actions to customer
target value 100% Description Send emails, …
especially to nigeria
Who (RACI) R: Area managers / partners; A: Kristel
Where /
Deadline june 2015
4
Indicator 4
1
Relationship with other classes
2Milestones
3
3
Indicator 3
1
Relationship with other classes
2Milestones
XXVII
Name On time forecasting from export important customers
Description
number of forecast for the next 3 months from
important customers received quarterly /
total number of important customers
Actual Value 40-50%
Frequency Quarterly
Type of target Absolute
Object: list of agents
Actor (RACI): R: Kristel; C: agents/sales persons
deadline 1 june 2015
value 1 60%
deadline 2 june 2017
value 2 80%
deadline 3 end 2019
target value 100%
5
Indicator 5
Relationship with other classes
Milestones
Name on time forecasting from export special products
Description
number of forecasts from special products
received before december /
total number of special export products
Actual Value 40-50%
Frequency Quarterly
Type of target Absolute
Object: list of agents
Actor (RACI): R: Kristel; C: agents/sales persons
deadline 1 june 2015
value 1 60%
deadline 2 june 2017
value 2 80%
deadline 3 end 2019
target value 100%
Name Delivery time from Biové to VMD
Description Week of delivery product - Week of order product
Actual Value >12 weeks
Frequency monthly
Type of target Min/Max
Object: mSoft
Actor (RACI): R: Kristel; I: Jan
deadline 1 end 2014
value 1 between 10 and 12 weeks
deadline 2 /
value 2 /
deadline 3 jun/15
target value between 8 and 10 weeks
7
Indicator 7
Milestones
6
Indicator 6
Milestones
Relationship with other classes
Relationship with other classes
XXVIII
APPENDIX 6
ADDED VALUE HUMAN CAPITAL
(ind.1)
NUMBER OF RECRUITMENTS WITH ASSESSMENTS OF HIGHLY SKILLED
PERSONS
NUMBER OF PERFORMANCE APPRAISALS FROM SALES BELGIUM DEPARTMENT
(Ind.7)
AVERAGE SCORE OF WORKERS ON EVALUATION
TOTAL NUMBER OF PERFORMANCE
APPRAISALS
NUMBER OF PERFORMANCE
APPRAISALS FROM HR MANAGER
(Ind.6)
NUMBER OF PERFORMANCE
APPRAISALS FROM QUALITY DEPARTMENT
(Ind.8)
AVERAGE SCORE OF WORKERS ON GOOD
BEHAVIOR
AVERAGE SCORE OF WORKERS ON ATTENDANCE
NUMBER OF PERFORMANCE
APPRAISALS FROM SALES EXPORT DEPARTMENT
(Ind.9)
Contribution
SumSum
PJ JanLeo
NAME Perform a good internal HR Management RACI
Description
Retain neither over, nor underqualified persons for a function
Responsible
Accountable
Consulted
Informed
(Verantwoordelijk)
(Eindverantwoordelijk)
(Geraadpleegd)
(Geinformeerd)
Who wishes this? Jan Moons
who is assigned? Luc Jacobs
Name Added value of human capital
Description
'= added value / average number of FTE
where added value = (turnover) + (other operating revenues) - (trade
goods, raw and auxiliary materials)-(servies and other goods)-(other
operating expenses
Actual Value 98.000 € per FTE
Frequency monthly
Type of target Minimum/Maximum
Object: Where? Balance sheet analysis (excell)
Actor (RACI): Who? Luc Jacobs
deadline 1 dec/14
value 1 between 100 and 110
deadline 2 /
value 2 /
deadline 3 dec/15
target value between 110 and 120
OBJECTIVE
Measures Actions
1
Indicator 1
Relationship with other classes
Milestones
XXIX
Name Average score of workers on evaluation
Description = 50% * Value Indicator 3 + 50 % * Value of Indicator 4
Actual Value 299
Frequency Yearly
Type of target Achievement
Object: Where? Indicator 3 and Indicator 4
Actor (RACI): Who? R: Luc Jacobs; C: Kristel Walschots; I: Jan Moons
deadline 1
value 1
deadline 2
value 2
deadline 3 end 2015
target value 310
Name Implement a reward system
Name
Average reward for attendance Description
The workers can chose between a group insurance or an extra bonus
on their salary
Description
= average sum of (bonus * multiplicator*gross salary) of all workers
Where multiplicator is dependent on number of sick and leave days
(0 days: 2; 1 day: 1,75; 2: 1,50; 3: 1,25; 4-5:0,90; 6-10:0,60;11-15:0,30; 16-
20: 0,10; >20:0)
Who (RACI) R: Luc Jacobs, C: Kristel Walschots, I: Jan Moons
Actual Value 168 Where /
Frequency yearly Deadline end 2015
Type of target Achievement
Object: Where? Excell file 'WorkersEvalution' Name
Actor (RACI): Who? Luc Jacobs Description
Who (RACI)
deadline 1 Where
value 1 Deadline
deadline 2
value 2
deadline 3 end 2015
target value 175
Name Provide training
Name Average score on good behavior Description Provide training on GMP, cleanliness and tidiness, …
Description
=average sum of bonus*gross salary*( sum of 4 scores regarding good
behavior on ten)/10 of all workers
1: application of labor regulations
2: compliance with GMP
3: work volume
4: attitude/loyalty
Who (RACI) R: Luc Jacbobs
Actual Value 130 Where /
Frequency yearly Deadline End 2015
Type of target Achievement
Object: Where? Excell file 'WorkersEvalution'
Actor (RACI): Who? Luc Jacobs
deadline 1
value 1
deadline 2
value 2
deadline 3 end 2015
target value 135
4
Indicator 4
Action 1
Relationship with other classes
Milestones
2
Indicator 2
Relationship with other classes
Milestones
3
Indicator 3
Action 1
Relationship with other classes
Action 2Milestones
XXX
Name Gather infromation on basic elements of performance appraisal
NameTotal number of performance appraisals Description
What should always be included in a performance appraisal? For
example absence rate, Functioning in group
Description Number of performance appraisals per person per year Who (RACI) A: Luc
Actual Value 0 Where
Frequency half-yearly Deadline end 2014
Type of target Absolute
Object: Where? List in excell Name Establish a postive athmosphere around the interviews
Actor (RACI): Who?
R: Jan, Luc, Leo, PJ; A: Luc Description
Make sure that the employees do not feel threatened by the
interviews, but that they understand its benefits.
Open culture, everything can be said, ,..
Who (RACI) R: Luc Jacobs
deadline 1 1 Where
value 1 end 2014 Deadline end 2014
deadline 2 /
value 2 /
deadline 3 2
target value end 2015
Name Gather infromation on specific elements of performance appraisal
Name
Total number of performance appraisals in
Administration, Production, … Description
Which elements are specific for a certain department
(administration)?
Production: GMP
Description
Number of performance appraisals per person per year in the
following departmentsWho (RACI) R: Luc, A: Luc, Consulted: Jan
Actual Value 0 Where
Frequency half-yearly Deadline end 2014
Type of target Absolute
Object: Where? List in excell Name Gather information on personal elements
Actor (RACI): Who? Luc Jacobs Description
What is a characteristic of a certain person, how does he/she work,
does he/she has remarks/suggestions for the company, …
Who (RACI) R: Luc, A: Luc, Consulted: Jan
deadline 1 1 Where
value 1 end 2014 Deadline end 2014
deadline 2
value 2
deadline 3 2 Name Make a planning and list for the next 6 months
target value end 2015 Description Luc has to do 8 performance appraisals (production not included)
Who (RACI) R: Luc ; A: Luc; C: Ingrid, mine, els, cindy, linda, kristel, griet, chantal,
Where
Deadline end 2014
Name Gather infromation on specific elements of performance appraisal
Name
Total number of performance appraisals in
Sales department BelgiumDescription Which elements are specific for a certain department (sales)?
Description
Number of performance appraisals per person per year in the sales
departmentWho (RACI) R: PJ, A: Luc, Consulted: Jan
Actual Value 0 Where
Frequency half-yearly Deadline end 2014
Type of target Absolute
Object: Where? List in excell Name Gather information on personal elements
Actor (RACI): Who? R: PJ; A: Luc; C: Jan Description
What is a characteristic of a certain person, how does he/she work,
does he/she has remarks/suggestions for the company, …
Who (RACI) R: PJ, A: Luc, Consulted: Jan
deadline 1 1 Where
value 1 end 2014 Deadline end 2014
deadline 2
value 2
deadline 3 2 Name Make a planning and list for the next 6 months
target value end 2015 Description PJ has to do 5 performance appraisals
Who (RACI) R: PJ ; A: Luc; C: Linus, Christel, Tanja, Anne-Sophie, Luc
Where
Deadline end 2014
7
Indicator 7
Action 1
Relationship with other classes
Action 2Milestones
Action 3
6
Indicator 6
Action 1
Relationship with other classes
Action 2Milestones
Action 3
5
Indicator 5
Action 1
Relationship with other classes
Action 2Milestones
XXXI
Name Gather infromation on specific elements of performance appraisal
Name
Total number of performance appraisals in
Registration departmentDescription Which elements are specific for a certain department (sales)?
Description
Number of performance appraisals per person per year in the
registration departmentWho (RACI) R: Leo, A: Luc, Consulted: Jan
Actual Value 0 Where
Frequency half-yearly Deadline end 2014
Type of target Absolute
Object: Where? List in excell Name Gather information on personal elements
Actor (RACI): Who? R: Leo; A: Luc; C: Jan Description
What is a characteristic of a certain person, how does he/she work,
does he/she has remarks/suggestions for the company, …
Who (RACI) R: Leo, A: Luc, Consulted: Jan
deadline 1 1 Where
value 1 end 2014 Deadline end 2014
deadline 2
value 2
deadline 3 2 Name Make a planning and list for the next 6 months
target value end 2015 Description Leo has to do 4 performance appraisals
Who (RACI) R: Leo ; A: Luc; C: Tinne, Mieke, Leen, Chris, Sophie
Where
Deadline end 2014
Name Gather infromation on specific elements of performance appraisal
Name
Total number of performance appraisals in
Sales department exportDescription Which elements are specific for a certain department (sales)?
Description
Number of performance appraisals per person per year in the sales
departmentWho (RACI) R: Jan, A: Luc, Consulted: Jan
Actual Value 0 Where
Frequency half-yearly Deadline end 2014
Type of target Absolute
Object: Where? List in excell Name Gather information on personal elements
Actor (RACI): Who? R: Jan; A: Luc Description
What is a characteristic of a certain person, how does he/she work,
does he/she has remarks/suggestions for the company, …
Who (RACI) R: Jan, A: Luc, Consulted: Jan
deadline 1 1 Where
value 1 end 2014 Deadline end 2014
deadline 2
value 2
deadline 3 2 Name Make a planning and list for the next 6 months
target value end 2015 Description Jan has to do 5 performance appraisals
Who (RACI) R: ; A: Luc; C: Christoph, Véronique, Teresa, Mark, Yawar
Where
Deadline end 2014
9
Indicator 9
Action 1
Relationship with other classes
Action 2Milestones
Action 3
8
Indicator 8
Action 1
Relationship with other classes
Action 2Milestones
Action 3
XXXII
NAME Perform a good selection process RACI
Description
Hire neither over, nor underqualified persons for a function
Responsible
Accountable
Consulted
Informed
(Verantwoordelijk)
(Eindverantwoordelijk)
(Geraadpleegd)
(Geinformeerd)
Who wishes this? Jan Moons
who is assigned? Luc Jacobs
Name Perform an assessment on new person registration dep
NameNumber of recruitments with assessment of highly skilled people Description
Perform an assessment on the person who will replace
Ann Bracke in registration departement
Description
= number of recruitments that is done with assesments/number of recruitments that
should be done with assesmentsWho (RACI) R: Registration manager (Leo), A: HR Manager (Luc), C: CEO (Jan)
Actual Value 0,3 Where Mr. Goelen (assesment centre)
Frequency half-yearly Deadline end 2014
Type of target Absolute
Object: Where? Personnel register in Excell + personnel map in archive Name Perform an assessment on new assistant controller
Actor (RACI): Who? R: HR manager (Luc Jacobs) Description Perform an assessment on new function of assistant controller
Who (RACI) R: HR (Luc), A: HR (Luc), C: CEO (Jan)
deadline 1 Where Mr. Goelen (assesment centre)
value 1 Deadline end 2014
deadline 2
value 2
deadline 3 end 2014
target value 1
Action 2Milestones
OBJECTIVE
Measures Actions
1
Indicator 1
Action 1
Relationship with other classes
XXXIII
APPENDIX 7
Consolidated
EBITDA (Ind.1)
EBITDA VMD
(Ind.2)
EBITDA Biové
(Ind.3)
Consolididated
Absolute GROSS
MARGIN (Ind.1)
Consolidated
Relative GROSS
MARGIN (Ind.2)
Consolidated sum
Derived
NAME Realize a high level of profitability (financial perspective)
Description
Strive for a consolidated EBITDA of € 6.165 million by the end of
2016
Who wishes this? Jan Moons
who is assigned? /
Name Consolidated EBITDA
Description
= consolidated turnover - costs (except interests, taxes,
amortizations and depreciations)
Actual Value € 5.745 million
Frequency half-yearly
Type of target Achievement
Object: Where? Consolidated Profit & Loss account
Actor (RACI): Who? R: Revisor; A: Jan Moons; C: Luc Jacobs
deadline 1 End 2014
value 1 5.450 mio
deadline 2 End 2015
value 2 5.848 mio
deadline 3 End 2016
target value 6.165 mio
OBJECTIVE
Measures
1
Indicator 1
Milestones
Relationship with other classes
XXXIV
Name EBITDA VMD
Description
= turnover - costs (except interests, taxes, amortizations and
depreciations) of VMD
Actual Value 1.823 mio
Frequency Monthly
Type of target Achievement
Object: Where? Profit and Loss account VMD
Actor (RACI): Who? R: Luc; A: Jan
deadline 1 End 2014
value 1 1,729 mio
deadline 2 End 2015
value 2 1,856 mio
deadline 3 End 2016
target value 1,956 mio
Name EBITDA Biové
Description
= turnover - costs (except interests, taxes, amortizations and
depreciations) of Biové
Actual Value 2.925 mio
Frequency three-monthly
Type of target Achievement
Object: Where? Profit & Loss acount Biové
Actor (RACI): Who? R: KPMG; A: Jan
deadline 1 End 2014
value 1 2,775 mio
deadline 2 End 2015
value 2 2,977 mio
deadline 3 End 2016
target value 3,139 mio
3
Indicator 3
Relationship with other classes
Milestones
2
Indicator 2
Relationship with other classes
Milestones
XXXV
NAME Optimize the gross margin
Description
Monitor both the absolute and relative consolidated gross margin,
so that the relative gross margin is around 50%
Who wishes this? Jan Moons
who is assigned?
Name Consolidated Absolute Gross Margin
Description = turnover - costs
Actual Value 20.040 mio
Frequency Half yearly
Type of target Achievement
Object: Where? Consolidated P&L account
Actor (RACI): Who? R: revisor, A: Jan Mppns, C: Luc Jacobs
deadline 1 end 2014
value 1 19.950 mio
deadline 2 end 2015
value 2 20.548 mio
deadline 3 end 2016
target value 21.165 mio
Name Consolidated Relative Gross Margin
Description = gross margin / turnover
Actual Value 50%
Frequency half yearly
Type of target Achievement
Object: Where? Consolidated P&L account
Actor (RACI): Who? R: Biové ; A: Jan
deadline 1
value 1
deadline 2
value 2
deadline 3 2016
target value 50%
2
Indicator 2
Relationship with other classes
Milestones
OBJECTIVE
Measures
1
Indicator 1
Relationship with other classes
Milestones
XXXVI