it governance according to cobit
DESCRIPTION
IT GovernanceTRANSCRIPT
-
Master Thesis
IT GOVERNANCEACCORDING TO COBIT
How does the IT performance within one ofthe largest investment banks in the world
compare to COBIT?
JOEL ETZLER
Stockholm, Sweden
XR-EE-ICS 2007:14
-
1
ABSTRACT
To improve the governance of IT and comply with regulatory demands,
organizations are using best practice frameworks to facilitate the work. One of these IT governance frameworks is COBIT (The Control Objectives for Information and related Technology). COBIT provides guidance on what could be done within an IT organization in terms of controls, activities, measuring and
documentation. This framework is however large and require specific knowledge in order to enable full use of its potential. This project was initiated to use a straightforward method of working with COBIT while assessing the maturity of an
organization. The method was developed by myself and my advisor at The Royal Institute of Technology in Stockholm and describes one way of using COBIT. The
organization under evaluation is one of the largest, most well known investment banks in the world, in this project referred to as The Firm.
A specific part of the IT organization within The Firm was evaluated with COBIT as a starting point and the gap between the framework and the organization was underlined. COBIT provides an incremental measurement scale, where the internal
processes are measured in terms of how defined and structured they are. The scale expresses levels of maturity and The Firm reached a level 3.3 out of 5.
The strongest and weakest areas have been emphasized and improvements on the weaker areas have been suggested. These improvement actions could enable organizations to better govern IT and facilitate compliance to regulatory
requirements.
Keywords: IT Governance, IT Management, COBIT, ITIL, Align IT to business, Sarbanes and Oxley.
-
2
PREFACE
This is my Master Thesis and it constitutes the final part in my Master of Science
education in Electrical Engineering at the Royal Institute of Technology in Stockholm. Conducting this project has been a great experience for me. I have met many, very kind and helpful people and would like to express my gratitude to all involved. Above all I would like thank, my advisor at ICS, Mrten Simonsson and key stakeholders at The Firm; Moss, Nikki, Andrew and Trevor. Thank you!
Joel Etzler
Stockholm, 16th of May, 2007
-
3
TABLE OF CONTENTS
1 INTRODUCTION ................................................................................................................... 5
1.1 BACKGROUND ....................................................................................................................... 5 1.2 PROBLEM ............................................................................................................................... 7
1.3 PURPOSE ................................................................................................................................ 7
1.4 DELIMITATIONS ..................................................................................................................... 7
1.5 THESIS DISPOSITION AND READING ADVICES ......................................................................... 7
2 METHODOLOGY .................................................................................................................. 9
2.1 INITIATION ............................................................................................................................ 9 2.2 CASE STUDY .......................................................................................................................... 9 2.3 THEORETICAL STUDY .......................................................................................................... 10 2.4 EVALUATION METHOD......................................................................................................... 11
3 THEORETICAL FRAMEWORK ....................................................................................... 12
3.1 CORPORATE GOVERNANCE .................................................................................................. 12
3.2 IT GOVERNANCE .................................................................................................................. 18
3.3 IT GOVERNANCE FRAMEWORKS .......................................................................................... 20
3.4 COBIT ................................................................................................................................ 22
3.5 COBIT FACILITATES COMPLIANCE WITH SARBANES-OXLEY .............................................. 31
4 ANALYTICAL FRAMEWORK.......................................................................................... 33
4.1 DATA COLLECTION .............................................................................................................. 33
4.2 MODELING .......................................................................................................................... 37
4.3 ANALYSIS ............................................................................................................................ 38
5 EMPIRICAL STUDY ........................................................................................................... 39
5.1 PROCEDURE ......................................................................................................................... 39 5.2 THE FIRM ............................................................................................................................ 39 5.3 PROJECT DEFINITION ........................................................................................................... 40 5.4 CASE STUDY AT THE FIRM................................................................................................... 41
6 RESULTS ............................................................................................................................... 43
6.1 GENERAL RESULTS WITHIN THE MARKETS DIVISION ............................................................. 43 6.2 WEAKNESSES AT THE FIRM ................................................................................................. 47
7 DISCUSSION......................................................................................................................... 49
7.1 DISCUSSING THE RESULTS ................................................................................................... 49 7.2 HOW TO IMPROVE THE WEAKNESSES ................................................................................... 51 7.3 VALIDITY ............................................................................................................................ 53 7.4 RELIABILITY ........................................................................................................................ 53
-
4
8 CONCLUSION ...................................................................................................................... 54
LIST OF FIGURES
FIGURE 1 FRAMEWORK LINKING CORPORATE GOVERNANCE TO IT GOVERNANCE8 ............................ 13
FIGURE 2 POSITIONING OF IT GOVERNANCE AND IT MANAGEMENT. SOURCE: PETERSON, SEE
GREMBERGEN, 2004. ............................................................................................................................................... 19
FIGURE 3 COBIT, OVERLYING FRAMEWORK PRINCIPLES. SOURCE: IT GOVERNANCE INSTITUTE,
COBIT 4.0 ................................................................................................................................................................. 23
FIGURE 4 COBIT, STRUCTURE AND INTERRELATIONSHIP OF PROCESSES. SOURCE: IT GOVERNANCE
INSTITUTE, COBIT 4.0 ........................................................................................................................................... 24
FIGURE 5 COBIT, OVERALL FRAMEWORK. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ........ 25
FIGURE 6 METRICS. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ................................................. 28
FIGURE 7 RACI-CHART. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ......................................... 30
FIGURE 8 DOCUMENTS. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ......................................... 30
FIGURE 9 MAPPING TO PCAOB TO COBIT. SOURCE: ITGI (2006), IT CONTROL OBJECTIVES FOR
SARBANES-OXLEY, THE ROLE OF IT IN THE DESIGN AND IMPLEMENTATION OF INTERNAL CONTROL
OVER FINANCIAL REPORTING. ............................................................................................................................... 31
FIGURE 10 WEIGHTED RESULTS ON ALL COBIT PROCESSES...................................................................... 44
FIGURE 11 TOP AND BOTTOM PROCESSES EMPHASIZED .............................................................................. 45
FIGURE 12 THE STRONGEST AREAS .................................................................................................................. 45
FIGURE 13 THE WEAKEST AREAS ...................................................................................................................... 47
FIGURE 14 SUGGESTED IMPROVEMENTS, CONTROLS AND METRICS ......................................................... 51
-
5
1 INTRODUCTION
This chapter gives the reader an introduction to the subject of matter. I present background to the research, a problem description, the purpose of my thesis where I display my research question, then delimitations of this thesis and finally, my thesis disposition.
1.1 BACKGROUND
Companies growing and merging with other businesses demand great changes to their infrastructure. The equities market space is constantly evolving and the implications to the IT systems and processes within the organizations are
substantial. Companies today depend to a great extent on the information stored and managed through IT and many would not be able to operate without a functional IT structure. The increasing regulatory demands also put a pressure on the accounting, documenting and reporting through IT. The systems are required not only to support the operations of the companies, but to report and store financial and organizational data to meet external demands. It is no longer enough
to look at talented individuals to manage IT projects, the projects regularly need to be structured as sustainable processes, where documentation and measuring is
standardized. Many companies acknowledge this need and put more effort into standardizing the IT structure, policies and procedures and focus on aligning them to the business objectives. This practice is called IT governance and will be further explained and discussed throughout this report.
To facilitate the governing of IT there are several frameworks available on the market. One of the most frequently used and chosen in this work is called COBIT1,
the Control Objectives for Information and Related Technology, further described
1 IT governance institute (2005), Control objectives for Sarbanes-Oxley
-
6
in section 3.4. COBIT gives guidance from best practices derived from major global IT-related standards, practices and frameworks on processes and its constituents to aid in the work of governing IT. The framework defines a set of processes, to which there is a number of activities, suggested documentation and
measuring. It provides a high level view of an IT organization and what could be done within it. COBIT also associates a maturity model that can be used to
benchmark the performance and level of definition to each process in a standardized manner. The scale, which is obtained from the Capability Maturity Model (CMM), described in section 3.3.3 spans from 0 to 5, with 5 being the highest.
To many organizations, the help of external best practices is a cost efficient and
effective alternative to creating own frameworks and standards. This thesis will highlight the work with one of these frameworks, namely COBIT and look at the possibilities to improve the governance on a specific IT organization through the help of that framework. The project has been performed at one of the largest investment banks in the world at a global division on the IT side. The project has followed the organizations desire to externally assess their IT performance with
COBIT as a frame for benchmarking.
The organization is in this thesis referred to as The Firm and the specific part of The Firm that the project is focused on is called The Markets division. This is further described in section 5.2. My advisor at the department of Industrial Information and Control Systems (ICS) at the Royal Institute of Technology is PhD student Mrten Simonsson. My advisor at The Firm is the European Head of Technology Business Development. Key stakeholders at The Firm are the European Head of Technology Business Development, the Head of Development at
The Markets Division and the people responsible for the scope and implementation phase of the COBIT initiative at The Firm. The Head of Development did
participate in interviews, but when referred to as key personnel, they do not represent a respondents view.
-
7
1.2 PROBLEM
How should IT be governed and how could COBIT be used as guidance? In this project, there are two key issues I have addressed.
The framework itself does not say how it should be used; it merely states guidance on its defined processes.
The Markets division wanted to know how it compared to industry standards and see how the effectiveness and efficiency of the IT organization could be improved.
1.3 PURPOSE
The purpose of the project was to do an assessment of The Markets division at The Firm with COBIT serving as a starting point. The assessment could be resembled by a gap analysis where the difference between the framework and the actual organization is emphasized. Derived from that assessment is the information about strengths and weaknesses within the IT organization, in comparison to COBIT. The four strongest and weakest areas should be emphasized and suggestions on how to improve the weaker areas should be presented. The question I tried to answer was:
How does the IT procedures and processes at The Markets division compare to COBIT- how big are the gaps, what could be improved and how?
1.4 DELIMITATIONS
The project was decided to be a high level assessment and was limited to gathering information on the COBIT processes from one person per process. The definition
of a process is described in section 3.4 COBIT.
This project covers what is being done in respect to COBIT, not processes outside those borders. The project was also limited to The Markets division which is further described in section 5.2.
1.5 THESIS DISPOSITION
1. Introduction
-
8
This chapter gives the reader an introduction to the subject of matter. I present background to the research, a problem description, the purpose of my thesis where I display my research question, then delimitations of this thesis and finally, my thesis disposition
2. Methodology This chapter provides the projects course of action and motivates why I have chosen this approach to address the given problem. I describe the initiation, the method of collecting data, required theoretical knowledge and finally how I evaluated the data
3. Theoretical framework This chapter provides the theoretical foundation of the thesis. Initially I will discuss theory around corporate and IT governance, leading up to the ways
IT could be governed. Brief reviews of possible IT governance frameworks are presented to facilitate the governing of IT and the framework used in this study, COBIT, will be described closer.
4. Analytical framework - In this chapter I explain the method of collecting data in detail, the analysis of the collected data and the method I have chosen to derive my results.
5. Empirical study - This chapter portrays the data collection specific for the assessment at The Firm and a description of the organization.
6. Results - In this chapter I reveal my results of the assessment beginning with general results. I then explain the results for the stronger and weaker areas closer.
7. Discussion - This chapter will discuss the results of the assessment and highlight relevant and interesting findings throughout the project.
8. Conclusion - This chapter describes the conclusions that can be drawn from this
assessment and answers the question posed in the purpose section.
-
9
2 METHODOLOGY
This chapter provides the projects course of action and motivates why I have chosen this approach to address the given problem. I describe the initiation, the method of collecting data, required theoretical knowledge and finally how I evaluated the data.
2.1 INITIATION
The reason why the project was initiated relates to the research of PhD student Mrten Simonsson and the department of Industrial Information and Control Systems at the Royal Institute of Technology, previously described in section 1.1.
The purpose, also prior described, is evaluating a part of an IT organization with COBIT as a starting point. The first problem of the thesis project was to find a sponsoring company that would be willing to participate in this project. During a previous employment, I came in contact with The Firm and proposed my project. The Firm felt as a suitable sponsor where my project could be of value. This is further described in section 5.2. The project was also further limited to The Markets division, also described in section 5.2 as that area seemed to be just the right size for my study.
2.2 CASE STUDY
The case study is but one of several ways of doing social
science research. Other ways include experiments, surveys,
-
10
histories, and the analysis of archival information (as in
economic studies).2
The way to fulfill the purpose of this project has mainly been through a case study. A more quantitative method, like questionnaires would possibly have been
applicable to this project as well. According to Holme & Solvang3 the qualitative and quantitative methods both have their advantages and disadvantages. As COBIT was new to many of the participants in the study, explanations were in several cases necessary.
In general, case studies are the preferred strategy when
how or why questions are being posed2
The study required the presence of someone with knowledge in COBIT to facilitate the questions- and answering process. This is the reason why I chose to do interviews. That way I could participate as an interviewer with specific knowledge in the COBIT framework and easier get accurate answers from the respondents. I
used COBIT as a starting point and asked the respondent to evaluate the maturity on each activity within one process. I also asked them to answer how many of the
suggested documents and metrics The Markets division was actually using. Finally I asked how the role assignment suggested in the RACI-chart corresponded to the structure at The Markets division. COBIT specifics can be found in section 3.4.
2.3 THEORETICAL STUDY
After determining the method of gathering information there were a few areas I
needed more theoretical knowledge in. This also constitute a part of the curriculum of a master thesis and motivates chapter 3, Theoretical framework where the research is presented as needed to understand the empirical study. The research is partly about corporate governance and its constituents. This along with the relationship to IT governance depicts the foundation for the thesis subject. The way to govern IT is suggested with help and guidance from an assessment framework and the currently available frameworks are presented briefly as a benchmark for
2Yin, Robert K. (1994), Case study research, Design and methods, second edition. 3 Holme & Solvang (1997).
-
11
comparative analysis in respect to COBIT, the framework of choice in this project. COBIT was chosen because it is considered
arguably the most appropriate control framework to help
an organization ensure alignment between use of
Information Technology (IT) and its business goals4
The analysis shows the competitive advantages of COBIT compared to its alternatives. COBIT is then described in detail in section 3.4, COBIT, as it
constitutes a large portion of the required theoretical knowledge in this thesis. The way COBIT can be useful to organizations will be presented and examined in terms
of what drives the implementation of the framework in general. It will be shown that COBIT is an effective framework as to assure compliance to regulatory requirements and provide a way to enhance efficiency within the IT organization and for the company as a whole. Various regulatory requirements will be described
along with their relationship to COBIT.
2.4 EVALUATION METHOD
After collecting the data from the interviews I needed a way to aggregate them into
results. Discussions with my advisor from ICS lead to the evaluation method. We decided to take all results from all parts of the data collection and add them together. The mean value generated the maturity on each process, and the mean value on all 34 COBIT processes gave the overall maturity level.
4Ridley G. et al (2004), COBIT and its Utilization: A framework from the literature. Proceedings of the 37th Hawaii International Conference on System Sciences, IEEE
-
12
3 THEORETICAL FRAMEWORK
This chapter provides the theoretical foundation of the thesis. Initially I will discuss theory around corporate and IT governance and the regulatory demands in that space. This leading up to the ways IT could be governed. Brief reviews of possible IT governance frameworks are presented to facilitate the governing of IT and the framework used in this study, COBIT, will be described closer.
3.1 CORPORATE GOVERNANCE
In order to understand the concept of IT governance one needs insight into the principles of corporate governance and its constituents.
"Corporate Governance is concerned with holding the
balance between economic and social goals and between
individual and communal goals. The corporate governance
framework is there to encourage the efficient use of
resources and equally to require accountability for the
stewardship of those resources. The aim is to align as
nearly as possible the interests of individuals, corporations
and society" 5
The Organization for Economic Cooperation and Developments 1999 published the OECD Principles for Corporate Governance which defines corporate governance as providing the structure through which the objectives for the company is set and the ways to align and achieve those objectives and monitor the performance is determined. It also set the relationships between an organizations
5 Sir Adrian Cadbury (2000), in 'Global Corporate Governance Forum', World Bank.
-
13
board, management, shareholders and additional key stakeholders.6 IT governance closely relates to corporate governance, the structure of the IT organization and its objectives and alignment to the business objectives.
Corporate Governance issues cannot be addressed without
considering IT Governance issues7
Weill and Ross8 have created a framework for linking the corporate governance
and IT governance principles together, which can be seen in figure 1. The areas that relates to IT governance are marked in grey.
Figure 1 Framework linking corporate governance to IT governance8
There are several ways of looking at the connection between corporate governance and IT governance. Another is described by Van Grembergen, De Raes and
6 OECD (1999), Principles of Corporate Governance.
7 Van Grembergen, De Raes o Guldentops (2004), Structures, Processes and Relational
Mechanisms for IT Governance, Idea Group inc. 8 Weill & Ross (2004), IT Governance
8 Van Grembergen, De Raes o Guldentops (2004), Structures, Processes and Relational
Mechanisms for IT Governance, Idea Group inc.
-
14
Guldentops8. They use Shleifer, A. & Vishnys9 work and mention three key questions that they say the management team should address to display the connectivity between corporate governance and IT governance.
Corporate Governance Questions: - IT Governance Questions:
How do suppliers of finance get
managers to return some of the
profits to them?
- How does management get their CIO and
IT organization to return some business
value to them?
How do suppliers of finance make sure
that managers do not steal the capital
they supply or invest it in bad
projects?
- How does top management make sure that
their CIO and IT organization does not steal
the capital they supply or invest in bad
projects?
How do suppliers of finance control
management?
- How does top management control their
CIO and IT organizations?
Table 1 Corporate and IT governance questions10
3.1.1 REGULATORY REQUIREMENTS ON CORPORATE GOVERNANCE
With the amount of effort still needed to address Sarbanes-
Oxley, Basel II, and the European 8th Directive---to name
but a few---compliance with regulations is expected to
maintain its position as the top driver for information
security going forward10
These regulatory requirements constitute a large portion of the need for structure
within organizations and the implications on IT are substantial. In coordination with various financial and regulatory requirements, a new era of high level
corporate and IT thinking has emerged. A key driver for IT governance have the last couple of years, been these external demands and the most significant one so
far has been the Sarbanes-Oxley act, described below. There are a few other important regulations, like Basel II, the European 8th Directive and Mifid but they will not be discussed in this study and their implications to IT will not be taken into account.
9 Schleifer A. & Vishny (1997), A survey on corporate Governance. The Journal of Finance, 52(2)
10 Ernst &Young (2005), Global Information Security Survey
-
15
THE SARBANES-OXLEY ACT OF 2002
The Sarbanes-Oxley act of 2002, SOX, has changed the world of reporting accountabilities as we know it. A number of corporate and accounting scandals,
most notably Enron, Tyco International and WorldCom reinvigorated the debate on regulating corporate governance. The loss of trust in large corporations accounting
and reporting practices became apparent. To restore the lack of trust investors and shareholders experienced, the Sarbanes-Oxley act was created. The act was passed on as United States federal law on July 30, 2002 initiated by the naming sponsors,
Senator Paul Sarbanes and Representative Michael G. Oxley.
All companies, including subsidiaries, American or not, listed on American stock exchanges like NYSE, the New York Stock Exchange, or NASDAQ are required to comply with the Sarbanes-Oxley act. The act establishes standards for all such companys boards, managements and public accounting firms. Containing eleven
titles, details in appendix 1, the act ranges from describing the increased corporate board responsibilities to criminal penalties for corporate wrongdoing. It also obligates the SEC, Securities and Exchange Commission, to implement rulings and accounting standards for compliance. The titles or sections of the act can be seen
below and are of varying importance in regards to this thesis.
Title I Public Company Accounting Oversight Board Title II Auditor Independence Title III Corporate Responsibility Title IV Enhanced Financial Disclosures Title V Analyst Conflicts of Interest Title VI Commission Resources and Authority Title VII Studies and Reports Title VIII Corporate and Criminal Fraud Accountability Title IX White Collar Crime Penalty Enhancements Title X Corporate Tax Returns Title XI Corporate Fraud Accountability
Title III and IV are the titles that are closest related to this work.
-
16
The two sections that should concern IT executives the
most are 302 and 404(a) because they deal with the
internal controls that a company has in place to ensure the
accuracy of their data. This relates directly to the software
systems that a company uses to control, transmit and
calculate the data that is used in their financial reports.11
Section 302 is characterized mainly by the CEOs and CFOs responsibility of
internal control regarding the annual financial reporting.
Section 404 demands each annual report to contain an internal control report which shall
(1) state the responsibility of management for establishing and maintaining an adequate internal control structure and procedures for
financial reporting; and (2) contain an assessment, as of the end of the most recent fiscal year of the issuer, of the effectiveness of the internal control structure and procedures of the issuer for financial reporting.12
Even though the act is focused on accounting and financial reporting, the
importance of appropriate IT systems as an integral part in the reporting procedure is evident. The systems ensure the validity of information and provide fundamental
structure to the reporting standards and assessments of financial data. Section 409 of the act expresses the real time accounting demands and is central to the IT
systems involved.
11 Dietrich, Robert (2004). Sarbanes-Oxley and the Need to Audit Your IT Processes, MKS
12 Sarbanes and Oxley act of 2002 Section 404. PUBLIC LAW 107204
-
17
REAL TIME ISSUER DISCLOSURES.Each issuer reporting
under section 13(a) or 15(d) shall disclose to the public on a
rapid and current basis such additional information
concerning material changes in the financial condition or
operations of the issuer, in plain English, which may
include trend and qualitative information and graphic
presentations, as the Commission determines, by rule, is
necessary or useful for the protection of investors and in the
public interest.]13
The relationship between IT systems and section 409 is described by Rob Smith, Co-Chair of Industry Solutions SOX Committee and Michael Kuhbock, Co-Chairman and Founder of the Integration Consortium.
The only way for issuers to be aware of real time
information and trends on operations or the physical
activities of their organization is for the issuers systems to
report on anomalies and trends in real time and on an
exception basis. As well, the integration of any new system
into an organization will have to pass SOX compliancy
before it is either selected or plugged in. Failure of control
process, due to a systems failure will strictly fall under the
409 clause regarding material change.14
This could very well be the most grueling challenges in the compliance work and one of the reasons corporations struggle to find easily adopted, implemented and administered frameworks to facilitate the process of compliance. A framework is
required by the act; however the choice of version is free. One such framework is provided by COBIT and another by COSO, described in section 3.4 and 3.3.2
respectively.
13 Sarbanes and Oxley act of 2002 Section 409. PUBLIC LAW 107204
14Smith R. Kuhbock M.. Sarbanes Oxley 404/409-Integration Organizations and SOX.
www.integrationconsortium.org
-
18
COSOs framework is the most frequently used when implementing compliance procedures today.15 It is also recommended by the SEC to aid in such tasks. COSO, does not provide a great deal of guidance to assist companies in the design and implementation of IT controls.16 COBIT on the other hand has its main focus on
controls within the IT organization.
The auditing standards are set by the PCAOB, The Public Company Accounting Oversight Board. The PCAOB is created by Sarbanes-Oxley and described in title I of the act. The purpose is to supervise and regulate the work done by auditing companies. It also sets the working principles for the auditing companies.
3.2 IT GOVERNANCE
IT Governance is the organisational capacity exercised by
the Board, executive management and IT management to
control the formulation and implementation of IT strategy
and in this way ensure the fusion of business and IT.17
These are the words of well renowned, IT governance theorist, Grembergen in
2002. There have been several different ways of defining IT governance, below are a few additional of the more famous displayed.
IT governance is the responsibility of the board of directors
and executive management. It is an integral part of
enterprise governance and consists of the leadership and
organisational structures and processes that ensure that
the organisations IT sustains and extends the
organisations strategies and objectives.18
The organisational capacity to control the formulation and
implementation of IT strategy and guide to proper direction
15IT Governance Institute (2005), IT Control objectives for Sarbanes-Oxley 16
IT governance institute (2006), IT control objectives for Sarbane-Oxley 17
Grembergen, (2002) 18
IT governance institute (2003)
-
19
for the purpose of achieving competitive advantages for the
corporation19
The theory of IT governance as mentioned before is partly driven by the external regulatory demands. Besides that, an increasing number of companies acknowledge
that a well defined structure and high level of guidance truly can contribute to the overall cost efficiency and performance of IT. One of the key focuses of IT governance according to Grembergen, (2004) is to align IT to business objectives. As an explanation it could be said that IT governance is the mix between corporate
governance and IT management. According to Peterson, figure 2 can be used to describe the relationship between IT management and IT governance.
FIGURE 2 POSITIONING OF IT GOVERNANCE AND IT MANAGEMENT. SOURCE: PETERSON, SEE
GREMBERGEN, 2004.
The difference between them could help provide a better view of what IT governance is, as confusion easily occurs. Weill and Ross (2004), says that governance determines who should make decisions and management is the process
of making and implementing the decisions.
19 The Ministry of International Trade and Industry (1999)
-
20
3.3 IT GOVERNANCE FRAMEWORKS
3.3.1 ITIL
The IT Infrastructure Library, ITIL, was created by the British Office of Government Commerce, OGC, to more effectively manage IT within British authorities as well as public companies. The principles of the ITIL framework were
derived out of best practice with regards to observed companies within the IT sector. It is now a fully documented set of best practice documents for IT service
management and the most widely accepted approach to IT service management in the world.20 It consists of several books, hence the term library. At the moment there are eight books:
1. Service Delivery 2. Service Support 3. ICT Infrastructure Management 4. Security Management 5. The business perspective
6. Application management 7. Software Asset Management 8. Planning to Implement Service Management
ITILs main objectives are to provide best practice definitions and criteria for operations management within two key areas, namely Service Support and Service
Delivery2122. In these areas ITIL focuses on the operational, organizational and functional attributes required for optimized operations management. These areas also have a number of supporting subcategories. ITIL, however does not cover the strategic impact of IT and the relation between IT and the business.2021
20 Office of Government Commerce, OGC. http://www.itil.co.uk/
21 Office of Government Commerce: IT Infrastructure Library Service Support. The
Stationery Office (2002) 22
Office of Government Commerce: IT Infrastructure Library Service Delivery. The Stationery Office (2002)
-
21
3.3.2 COSO
COSO or the Committee of Sponsoring Organizations of the Treadway commission was established in 1985. In 1992 COSO released the Internal Control Integrated framework. It was originally developed to cope with the fraudulent financial reporting present in the world of corporate accounting.23 The framework COSO consists of five interrelated Internal control components and three Enterprise risk management components. The ERM components and the Enterprise Risk Management Integrated Framework, were created in collaboration with PriceWaterhouseCoopers in 2004. All components are shown below with risk management components in bolded fonts.
Internal Environment
Objective Setting Event Identification
Risk Assessment
Risk Response
Control Activities
Information and Communication
Monitoring
COSO is a voluntary private sector organization dedicated
to improving the quality of financial reporting through
business ethics, effective internal controls, and corporate
governance.23
The five components of internal control that COSO identifies can be resembled by the guidance COBIT provides for IT.24
23 COSO-The Committee of Sponsoring Organizations of the Treadway commission, www.coso.org
24 Damianides, Marios (2005), SarbanesOxley and IT governance: New guidance on it control and
compliance http://www.infosectoday.com/SOX/Damianides.pdf
-
22
3.3.3 CMMI
Capability Maturity Model Integration (CMMI) is a process
improvement approach that provides organizations with the
essential elements of effective processes. It can be used to
guide process improvement across a project, a division, or
an entire organization.25
CMMI, (Capabilities Maturity Model Integration), previously CMM developed by the Software Engineering Institute (SEI), provides a model to improve the efficiency in processes across an organization. As the name implies, a key element
in the model is the evaluation of maturity through a maturity model. This maturity model is further described in section 3.4.1.
3.4 COBIT
COBIT is short for the Control Objectives for Information and Related Technology and was developed by the Information Systems Audit and Control Foundation, ISACF in 1996. ISACF, founded 1969 later became ISACA, Information Systems Audit and Control Association. ISACA, is now a global organization with over 50 000 members in more than 140 countries. The founders, a group of IT auditors, recognized the increasing need for control within IT organizations and decided to
create a network for information and guidance in the field. In 1998 ISACA established the IT Governance Institute, ITGI, who is now responsible for COBIT. During the fall of 2005, ITGI released a version 4.0 of COBIT which constitutes the framework of reference in this thesis.
COBIT was originally developed as a tool to control IT and reduce risk within IT
organizations, primarily in the banking and e-business industries. It has evolved to become more business oriented and now gives a high level image on what to accomplish within an organization rather than how. It is designed to provide fundamental guidance to management and process owners to in best way possible allocate the assets of the organization. Figure 3 shows the overlying framework principles.
25Software Engineering Institute (SEI) http://www.sei.cmu.edu/cmmi/general/general.html
-
23
The COBIT framework has the aspiration to be both responsive and practical in the sense of the business needs, while at the same time being independent to the technical and structural differences within various organizations.
COBIT uses ideas from all frameworks above and even more standards when
creating its definitions and controls.
For this COBIT update (COBIT 4.0), six of the major global IT-
related standards, frameworks and practices were focused
on as the major supporting references to ensure appropriate
coverage, consistency and alignment26
The standards, frameworks and practices mentioned in the quote above are:26
Committee of Sponsoring Organisations of the Treadway Commission (COSO): Internal ControlIntegrated Framework, 1994
Enterprise Risk MangementIntegrated Framework, 2004
Office of Government Commerce (OGC): IT Infrastructure Library (ITIL), 1999-2004
International Organisation for Standardisation: ISO/IEC 17799:2005, Code of Practice for Information Security Management
Software Engineering Institute (SEI): SEI Capability Maturity Model (CMM), 1993 SEI Capability Maturity Model Integration (CMMI), 2000
Project Management Institute (PMI):
26IT Governance Institute (2005), COBIT 4.0
FIGURE 3 COBIT, OVERLYING FRAMEWORK PRINCIPLES.
SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0
-
24
Project Management Body of Knowledge (PMBOK), 2000 Information Security Forum (ISF):
The Standard of Good Practice for Information Security, 2003
Originally the framework was based on three separate documents: Control Objectives is the first of the documents that describes the 34 processes and the control objectives to each process employed by COBIT. The maturity levels are not regarded in this section. Management Guidelines presents the maturity levels and the two measurable indicators connected to each process type. Audit Guidelines is based on Management Guidelines and provide advice on who to interview and what kind of information is demanded to each process type.
THE COBIT FRAMEWORK
COBIT provides a detailed and easily used model to govern IT. The structure and
interrelationship of the processes that COBIT treats is shown in Figure 4. The COBIT control objectives document is divided into four domains that describe the risks and activities within IT that needs to be managed. The domains in turn are divided, in all into 34 different high level control objectives or processes. The processes each encompass detailed control objectives, activities, roles, different metrics and an incremental measurement scale. The roles in turn have
responsibilities associated to the activities.
FIGURE 4 COBIT, STRUCTURE AND INTERRELATIONSHIP OF PROCESSES. SOURCE: IT
GOVERNANCE INSTITUTE, COBIT 4.0
-
25
The processes apply at different levels of the IT organization and each domain could help to provide an understanding of the purpose of the processes. The names of all the COBIT processes are displayed in Figure 5.
The four COBIT domains; Plan and Organise, Acquire and Implement, Deliver and Support and Monitor and Evaluate as shown in figure 5, are clarified below.
Plan and Organise (PO) describes how the business objectives are best reached through the use of IT. This domain administrates the use of tactics and strategy to
FIGURE 5 COBIT, OVERALL FRAMEWORK. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0
-
26
plan, communicate and manage the different perspectives throughout the organization.
Acquire and Implement (AI) depicts the identifying and acquiring of IT solutions. Furthermore this domain explains the solutions integration to the business processes
and how to manage and upkeep the existing systems.
Deliver and Support (DS) handles the actual delivery of the information at hand and see to the management of service levels, performance and capacity, configurations, operations and the physical environment, to name a few. This domain is also responsible for the identification and allocation of costs and the training of users.
Monitor and Evaluate (ME) describes the monitoring and evaluation of all the processes employed by the IT
organization. This domain also delivers the final statement to provide IT governance
3.4.1 ASSESSMENT WITH THE COBIT FRAMEWORK
MATURITY MODEL
It is not easy to know how to benchmark an organization and to what grade of accuracy the evaluation should be scaled. COBIT suggests an incremental measurement scale of six maturity levels. Going from 0, Non-existent to 5, Optimized, COBIT covers the entire spectrum of maturity in a process. The structure and design of the scale is the same as the one used by Capability Maturity
Model, (CMM), described in section 3.3.3. These maturity levels are individually explained for each of the 34 processes but the general structure could be seen in table 2.
-
27
ACTIVITIES
The activities are a significant part of the suggested guidance COBIT describes for each process. They say what should be done and they are also associated to the
roles, further described under Roles and Responsibilities. An example of activities is shown in figure 7, RACI-chart. As previously mentioned; COBIT also
describe detailed control objectives. The detailed control objectives often correspond to the activities and their purpose is the same. COBIT is not entirely
consistent about this but in many cases, the activities are just simplified detailed control objectives.
METRICS
To improve the efficiency and effectiveness of the processes, COBIT suggest a set of metrics to use as measurement to each process. The metrics are different for each process but some of the outlines are similar. The metrics are in the version
used in this study, COBIT 4.0, Key Performance Indicators, Process Key Goal
0 Non-ExistentComplete lack of any recognisable processes. The organisation has not even
recognised that there is an issue to be addressed.
1 Initial
There is evidence that the organisation has recognised that the issues exist
and need to be addressed. There are however no standardised processes but
instead there are ad hoc approaches that tend to be applied on an individual
or
2 Repeatable
Processes have developed to the stage where similar procedures are followed
by different people undertaking the same task. There is no formal training or
communication of standard procedures and responsibility is left to the
individual. There is a high degree of reliance on the knowledge of individuals
and therefore errors are likely.
3 Defined
Procedures have been standardised and documented, and communicated
through training. It is however left to the individual to follow these processes,
and it is unlikely that deviations will be detected. The procedures themselves
are not sophisticated but are the formalisation of existing practices.
4 Managed
It is possible to monitor and measure compliance with procedures and to take
action where processes appear not to be working effectively. Processes are
under constant improvement and provide good practice. Automation and tools
are used in a limited or fragmented way.
5 Optimised
Processes have been refined to a level of best practice, based on the results
of continuous improvement and maturity modelling with other organisations.
IT is used in an integrated way to automate the workflow, providing tools to
improve quality and effectiveness, making the enterprise quick to adapt.
TABLE 2 MATURITY MODEL. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0
-
28
Indicators and IT Key Goal Indicators. For the process, Manage the IT investment the metrics are shown in figure 6.
Just to clarify what is shown in the image, one metric COBIT suggests could be to
measure the percentage of projects with benefit defined upfront. That metric can be seen in the upper left corner of the Key Performance Indicators box in figure 6.
According to Guldentops27 the primary purpose of the guidelines is to enable
corporate management to:
Measure Performance What are the indicators of good performance?
Profile their IT control Whats important? What are the critical success factors for control?
Enhance their awareness What are the risks of not achieving our objectives?
Benchmark the organization What do others do? How do we measure and compare?
The indicators are the key inputs in the benchmarking process. The Management guidelines indicators are Key Goal Indicators (KGIs), Key Performance Indicators (KPIs) and maturity models.
The Key Goal Indicators represents what has to be accomplished in order to achieve the process goals. They define measures that tell if business objectives
27Guldentops, E in Van Grembergen, W (2004). Strategies for Information Technology Governance. Idea Group Inc. Chapter 11 Governing Information Technology through COBIT.
FIGURE 6 METRICS. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0
-
29
have been met for a specific process and are often defined as the target to achieve. Business requirements are generally expressed in terms of information criteria:
Availability of information needed to support the business needs
Absence of integrity and confidentiality risks
Cost-efficiency of processes and operations Confirmation of reliability, effectiveness and compliance
The Key Performance Indicators define measures to explain to what extent the process is fulfilling its objectives, how well its performing. They are the most important indicators in revealing whether or not a goal will be reached and are often used to in an early stage tell if the KGIs will be difficult to achieve.
ROLES AND RESPONSIBILITIES
COBIT describes a number of different roles that an IT organization should use. The roles suggested by COBIT can be seen below.
Chief executive officer (CEO) Chief information officer (CIO) Business executives Chief financial officer (CFO) Head operations Chief Architect Head development Head IT administration The project manager office (PMO) Compliance, audit risk and security
To every process there are a number of activities with the responsible employee or employees conveyed in a chart, called a RACI-chart, see figure 7. To be more
precise COBIT defines four different ways in which a person or role should be connected to an activity. The different ways are Responsible, Accountable, Consulted and Informed, hence the name RACI. The Responsible person is the one responsible for the execution of an activity while Accountable is the one who authorizes it. Consulted is someone who should be asked or consulted when an
-
30
activity is performed while the function of Informed is merely one who should know about the activity. Figure 7 shows the roles as functions and their relationship to the activities of the process Manage the IT investment. The activities extend the understanding of the process and its purpose. To each activity there is either a
Responsible or an Accountable role to see to that the activity is executed in a proper manner.
DOCUMENTS
Relevant documentation renders repetition and effective feedback of the processes possible. COBIT defines which documents should exist at the initiation stage and
which should be produced during the process. They are referred to as Inputs and Outputs, shown in figure 8.
FIGURE 7 RACI-CHART. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0
FIGURE 8 DOCUMENTS. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0
-
31
3.5 COBIT FACILITATES COMPLIANCE WITH SARBANES-OXLEY
As mentioned above, COBIT is one applicable assessment framework that could help in the compliance of SOX. COBIT aligns 12 of the IT control objectives with the PCAOB Auditing standards No 2, displayed in figure 9. COBIT focuses on IT as opposed to COSO which is focused on controls for financial processes. This
means that COBITs guidance is centered on the IT processes which in reality are the way through which financial auditing is conducted.
COBIT enables clear policy development and good practice
for IT control throughout organizations. ITGIs latest version
COBIT 4.0 emphasizes regulatory compliance, helps
organizations to increase the value attained from IT,
enables alignment and simplifies implementation of the
COBIT framework.28
Appendix 2 shows the IT Governance Institutes compliance to SOX, roadmap.
28 www.Isaca.org
FIGURE 9 MAPPING TO PCAOB TO COBIT. SOURCE: ITGI (2006), IT
CONTROL OBJECTIVES FOR SARBANES-OXLEY, THE ROLE OF IT IN THE
DESIGN AND IMPLEMENTATION OF INTERNAL CONTROL OVER FINANCIAL
REPORTING.
-
32
While implementing procedures to comply with SOX regulations, many companies choose to overlook the IT structure to see what else could be improved during the reconstructuring. Ernst & Young have interviewed 1300 companies regarding
information security practice. They experienced a surprisingly low 41 percent of the interviewees using the opportunity of restructuring IT while complying with
external regulatory requirements. According to Ernst & Young its the ideal time to improve and streamline the business structure while a structural change still is inevitable due to the external regulatory demands.29
29 Ernst&Young (2005), Global Information Security Survey
-
33
4 ANALYTICAL FRAMEWORK
In this chapter I explain the method of collecting data in detail, the analysis of the collected data and the method I have chosen to derive my results.
4.1 DATA COLLECTION
There are no rules that govern the way to use COBIT and to what extent it is to be implemented. Each organization may adopt the framework to meet their business
objectives in which way they see fit.
COBIT works as a helping hand, providing guidance to the management on how,
according to best practice to use the assets and people within the organization. However, the complexity of COBIT could make the usage difficult and time consuming. Furthermore it leaves room for interpretation, which means that two
interviewers could obtain incomparable results on the same assessment. It is not a given that for instace the COBIT-defined activities, are interpreted the same way by two separate people. While the purpose of COBIT is to provide guidance on IT
governance, it does require a substantial amount of expertise with regards to the framework. This has led to the creation of a tool through which COBIT can be used
in a more formalized and straightforward way. This improves the validity and makes the framework more usable. It was created by PhD student Mrten Simonsson at the department of Industrial Information and Control Systems (ICS) at the Royal Institute of Technology. I will here describe how the data can be
collected, the modeling tool used and how to analyze the results.
-
34
As presented in section 2.4 the interviews will provide the input information to the project. The vast majority of the respondents should be executives with management functions as their knowledge is most likely to correspond to the kind of strategic information COBIT deals with. The descriptions below explains the
steps to take when working with COBIT and conducting the interviews.
1. Who to speak to about what With key personnel, map each of the suggested roles in COBIT to corresponding person at the organization under evaluation. From that mapping, talk to the person with the highest responsibility on each COBIT process. Through this method some individuals could easily
become potential respondents to many processes. To even out the time spent with each individual, discuss together with key
stakeholders at the organization under evaluation and try to find other people that could answer questions on some of those processes.
2. Short introduction to the project. Send by email a short PowerPoint briefing about the project and also information regarding the subject of the interview. This generally makes the face-to-face introduction shorter. Many times the respondent will not have time to review the material beforehand, which leads to the need of a background description of the project and COBIT anyway.
3. Explanation of respondents role Ask the respondent to explain his/hers role at the organization under evaluation. This could make it easier to appreciate from where the answers
come.
4. Evaluation of a process The respondents should be asked about the activities within each process he/she is either Accountable or Responsible to, according to the RACI-chart. The question is on what level of maturity in terms of the maturity model the respondent places that
activity, section 3.4.1.
-
35
The respondent should also be asked about the documents associated to the process and the measured KPIs and KGIs. This will be yes or no questions, adding up to a total which later in the analysis is compared to the maximum number of metrics defined by COBIT. In more detail the interviews can be done as follows.
1. The respondents should be asked to assess the maturity on each activity
suggested by COBIT. Table 3 could be used to assign maturity for each activity: (For help and guidance, the maturity model provided on each process in the COBIT document can be used)
MATURITY
LEVEL
ACTIVITY EXECUTION
LEVEL 0 NO AWARENESS OF THE IMPORTANCE OF ISSUES RELATED TO THE ACTIVITY. NO MONITORING IS PERFORMED. NO
DOCUMENTATION EXISTS.
NO ACTIVITY IMPROVEMENT ACTIONS TAKE PLACE.
LEVEL 1 SOME AWARENESS OF THE IMPORTANCE OF ISSUES RELATED TO THE ACTIVITY. NO MONITORING IS PERFORMED. NO
DOCUMENTATION EXISTS. NO ACTIVITY IMPROVEMENT ACTIONS TAKE PLACE.
LEVEL 2 INDIVIDUALS HAVE KNOWLEDGE ABOUT ISSUES RELATED TO THE ACTIVITY AND TAKE ACTIONS ACCORDINGLY. NO
MONITORING IS PERFORMED. NO DOCUMENTATION EXISTS. NO ACTIVITY IMPROVEMENT ACTIONS TAKE PLACE.
LEVEL 3 AFFECTED PERSONNEL ARE TRAINED IN THE MEANS AND GOALS OF THE ACTIVITY. NO MONITORING IS PERFORMED.
DOCUMENTATION IS PRESENT. NO ACTIVITY IMPROVEMENT ACTIONS TAKE PLACE.
LEVEL 4 AFFECTED PERSONNEL ARE TRAINED IN THE MEANS AND GOALS OF THE ACTIVITY. MONITORING IS PERFORMED.
DOCUMENTATION IS PRESENT. THE ACTIVITY IS UNDER CONSTANT IMPROVEMENT. AUTOMATED TOOLS ARE
EMPLOYED IN A LIMITED AND FRAGMENTED WAY
LEVEL 5 AFFECTED PERSONNEL ARE TRAINED IN THE MEANS AND GOALS OF THE ACTIVITY. MONITORING IS PERFORMED.
DOCUMENTATION IS PRESENT. AUTOMATED TOOLS ARE EMPLOYED IN AN INTEGRATED WAY, TO IMPROVE QUALITY
AND EFFECTIVENESS OF THE ACTIVITY
TABLE 3 ACTIVITY ASSESSMENT
A mean value for all activities within a process, the average activity
maturity (AM), should then be calculated. The values are threshold values, i.e. all criteria for level 3 have to be fulfilled in order to achieve level 3 maturity.
2. The RACI-chart should be discussed on each point to see how well it corresponds to the role assignment of the organization under evaluation. It
is broadly visualized in table 4. For more details, see appendix 5, Role assignment
-
36
3. The documents should be asked on one by one and the number of documents that actually exists within the organization is to be compared to those suggested by COBIT. The percentage of documents gives the
maturity value, according to table 4.
4. The same procedure for the metrics (Key Performance Indicators, Process Key Goal Indicators, IT Key Goal Indicators) as with the documents. How many of the suggested metrics they actually used as measurement should be counted. This also shown in table 4.
TABLE 4 METRIC, DOCUMENT AND METRIC ASSESSMENT.
The process maturity (PM) for the entire process is then calculated as the mean of the average activity maturity (AM) , The assigned responsibilities maturity (RM), the documents in place maturity (DM) , and the Metrics monitored maturity (MM).
PM= (AM+RM+DM+MM)/4 The values are also threshold values, i.e. all criteria for level 3 have to be fulfilled in order to achieve level 3 maturity. This means that it requires 100% usage of the metrics suggested in COBIT in order to achieve level 5.
MATURITY
LEVEL
ASSIGNED RESPONSIBILITIES
DOCUMENTS IN PLACE
METRICS
MONITORED
LEVEL 0 NO RELATIONS EXIST
0 % 0 %
LEVEL 1 AT LEAST 20 % OR RELATIONS IN LINE WITH COBIT. 20 % 20 %
LEVEL 2 AT LEAST 40 % OR RELATIONS IN LINE WITH COBIT. 40 % 40 %
LEVEL 3 AT LEAST 60 % OR RELATIONS IN LINE WITH COBIT. 60 % 60 %
LEVEL 4 AT LEAST 80 % OF RELATIONS IN LINE WITH COBIT. 80 % 80 %
LEVEL 5 100 % OF RELATIONS IN LINE WITH COBIT. 100 % 100 %
-
37
Regarding weights for separate metrics, the basic assumption is that all metrics have the same weight. It is up to each organization to do their own weighting but a guideline could be that activities should have the highest weight followed by the metrics.
As an optional final step, the respondent should be asked to evaluate where he/she thinks the entire organization or the suggested silo would land on the maturity scale. This should not be used in the assessment but is interesting to collected for
future benchmarking and evaluation of the maturity assessment method.
4.2 MODELING
The modeling phase represents the aggregation of all the collected data and the creation of a map showing all the COBIT processes and its relations to the
activities, metrics, roles and documents used by the organization. The reason for creating an architectural map is to easier get an overview of the processes and their relationships and to set definitions so that information about the model more easily can be derived. The map in this case study was created with a modeling program called Metis, a Troux technologies30 product. Metis is the software chosen by ICS, which is why I used it for this study. User specific functionality in Metis is done
through an application Programming Interface (API) that supports Visual Basic and Java script. At ICS an own Meta model that incorporates the definitions, rules and
restrictions of the model I used in this project has previously been created. That Meta model describes what could be modeled, which processes, metrics, documents and relations could be used in the model. It holds a reference model of the complete COBIT framework to which the model of the organization under
evaluation could be compared. The gap between the reference model and the model under evaluation generates the basis for the results and give the maturity to the
processes. The complete map can be seen in appendix 4, Model of The Firm. The modeling in Metis is a method that is still under evaluation by ICS. It will be used to a greater extent in future research as the benefit of using it increases the more defined this method gets. One of the key beneficial aspects of the model, is that it
can be used to easier change relations to the processes.
30 Troux Technology, Metis http://www.troux.com
-
38
4.3 ANALYSIS
The analysis is where the results are reviewed from the modeling and which conclusions could be drawn from the work. As one of the goals in the thesis was to
find areas or processes with lower and higher maturity level and suggest improvements, the conclusion of the modeling was crucial in this study. The
processes with more and less mature nature have been examined in detail. This is further described in chapter 6, Results. From the interviews I have tried to figure out which are the key gaps or specific strengths within those areas. To find out more about the current state and the reason for the strong or weak procedures and
policies within those areas, key personnel from The Firm was involved and questioned.
-
39
5 EMPIRICAL STUDY
This chapter portrays the data collection specific for the assessment at The Firm and a description of the organization.
5.1 PROCEDURE
This project will initially be described with a short introduction of the company where the study was done. After that follows in chronological order the phases of
the project with the Initiation followed by Project definition and Case study at The Firm.
5.2 THE FIRM
For security reasons the name of the company where the study took place will not
be revealed, it will instead be given a fictitious name, The Firm. The company I have chosen to call The Firm is one of the largest and most well known investment banks in the world. It operates on a global basis and house more than 50 000 employees. The Firm has taken a silo like approach to enterprise structure, which
means that each division functions almost as a separate organization. Each silo has got roles equivalent to what a normal company would have, like CIO (Chief Information Officer) and CFO (Chief Financial Officer). As this thesis mainly is about IT governance and the structure around IT processes, the following description is focused on the IT organization at The Firm.
Many roles are clearly defined within each silo. Their responsibilities are most often tied to the area they are stationed in but their superior officers
responsibilities could vary from central isolated groups to officers controlling
-
40
several silos. As many separate groups perform functions that are of use to all areas
at The Firm, those groups are in a way a part of all the silos. As will be described in section 1.5 the purpose of this project is to do an assessment of a specific division or silo at The Firm called The Markets division. The silo I, together with key stakeholders from The Firm, chose for this project is not really a silo but a mixture of three silos. The reason for choosing The Markets division was a result of several discussions with people who later became key stakeholders in the project.
Because many external auditors and regulators use COBIT, The Firms internal audit section has chosen to use it. Thereby they talk the same language. COBIT
is also the basis for the structure of their new global IT policy program31, which is why I found this company to be a suitable sponsor of this project.
5.3 PROJECT DEFINITION
As the need for structure and definition of the project was evident, many introductory interviews contributed to the project layout. These interviews along with discussions with my advisor at The Firm lead to the definition of the project. The assessment really had two different possible ways of being performed. One being a very high-level with the role mapping on European executives level. The COBIT roles, CEO, CIO, and CFO would correspond to the level of The Firms European CEO, CIO, and CFO and so on. As The Firms IT organization keeps a
silo like structure, each silo functions as a small organization with between 200-1000 employees within IT. A proper high-level assessment would require
interviews with respondents within each silo and from those with responsibilities spanning the entire organization. My advisor at The Firm and I agreed that this project was too large within the given timeframe so we turned to the second alternative, to focus on one division within The Firm. Discussions throughout the
organization resulted in a desire to assess The Markets division. It seemed to present a reasonably sized IT organization, 33 employees globally, where this
relatively small, and short project could find interesting results and still deal with complex systems and structures, much like the other silos.
31 Information from a global IT policy conference at The Firm the 24th of April, 2007
-
41
5.4 CASE STUDY AT THE FIRM
This project was performed at the companys European headquarter in London between the 15th of January 2007 and the 27th of April 2007. The method I used in this study is described in chapter 4, Analytical framework. As previously mentioned, the case study was based on interviews with selected personnel at The
Firm. Every interview was conducted in the same way and the questions were posed in a standardized manner, but to different subject areas. The areas were represented by the COBIT processes. In most cases the interviewee was the most responsible within that area. For instance I interviewed the European Head of
Operational Risk when talking about the Assess and manage IT risk process, the CFO of The Markets division regarding the Manage the IT investment process and the CIO of The Markets division regarding the Manage Operations process. In this example the Assess and manage IT risk process was managed by a central group and the maturity on that process would be the same for a different silo since that work is done across the board. In some cases one individual answered questions on several processes, which meant that we had to be clear that the role had been change since the last interview and that this new process required a different focus. On average, one process took around 30 minutes to go through, which was good since I could often get a one hour meeting and do two interviews, when it was necessary.
As COBIT has a way of describing processes that was not familiar to all
respondents, explanations were often required. The problem occurred most frequently when discussing the maturity on the activities. COBIT describes detailed control objectives to each process that often corresponds to the activities. The framework does not provide a consistent approach to this. Some of the activities
cannot be explained by a corresponding detailed control objective. Below is an example of when an activity can be further explained by a detailed control
objective associated to the same process. It is taken from process PO5 - Manage the IT investment.
Activity: Establish and maintain IT budgeting process Detailed control objective: IT budgeting process
-
42
Described by the detailed control objective as: Establish a process to prepare and manage a budget reflecting the priorities established
by the enterprises portfolio of IT-enabled investment programmes, and including the
ongoing costs of operating and maintaining the current infrastructure. The process should
support development of an overall IT budget as well as development of budgets for
individual programmes, with specific emphasis on the IT components of those
programmes. The process should allow for ongoing review, refinement and approval of
the overall budget and the budgets for individual programmes.
Some interviewees suggested ways to improve the COBIT framework with ideas
that made sense to the work they were doing at The Firm. One suggestion was to include a Quality Assurance role to the RACI-chart. This was motivated by the fact that in all the work done at The Firm there is interaction from a Quality Assurance function that makes sure that the quality policies are followed. There were also numerous suggestions on metrics and documents that could be added to improve the framework. One example could be to add a document called space planning to the process Procure IT resources. That document would describe the available space within each area of company so that there was adequate space for
the manpower and hardware.
The results of this assessment will be described in the next chapter in the way they
have been weighted in this study. Together with the group responsible for the initiation phase of the COBIT initiative at The Firm, I decided to give more weight to the activities and metrics. The activities received weight 4 and the metrics weight 2, the documents and role assignment stayed at weight 1. This meaning that the activities were four times as important as the documents to the results.
-
43
6 RESULTS
In this chapter I reveal my results of the assessment beginning with general results. I then explain the results for the stronger and weaker areas closer.
6.1 GENERAL RESULTS WITHIN THE MARKETS DIVISION
As described in chapter 1.5 and 5.2, the assessment was done at a specific division within The Firm, called The Markets division. There were however difficulties keeping the assessment to only The Markets division since many of the areas or functions are centrally governed and managed. In those cases where one of the COBIT processes was managed at a central level, the interview was conducted with
personnel working in that group, i.e. outside The Markets Division. Table 5 shows where each process belongs.
Central at The Firm Both Local within The Markets division
PO2 PO1 PO5
PO4 PO3 PO8
PO6 AI2 PO10
PO7 AI6 AI1
PO9 ME1 AI4
AI3 ME2 AI7
AI5 DS3
DS1 DS4
DS2 DS9
DS5 DS10
DS6 DS11
DS7 DS13
DS8
DS12
ME3
ME4
TABLE 5 PROCESS LOCATION AT THE FIRM
-
44
As shown in the table, almost half of the processes are managed on a central level and operate across the board. Another relevant issue to consider, when revealing the results, is the fact that The Markets division is a mix of three silos within The Firm. That contributes to the rather high amount of centrally managed processes
which in some cases only stretches to the boundaries of these three silos and not the entire company.
The complete results of this assessment can be seen in detail in appendix 4, where the maturity level, (the result) is displayed and specified by activities, metrics, documents and role assignment for each process. Since The Firm had desires to
weight the final results, the activities have weight 4, the metrics weight 2, the
documents and role assignment weight 1. The aggregated process maturity results
after weighting can be seen in figure 10. The average maturity across all processes was 3.3 after weighting. The activity maturity was 3.1, metrics 2.9, documents 4.0 and role assignment 3.9. Since the activities and metrics were heavier weighted, the result sank to 3.3, from an un-weighted result of 3.5.
Figure 11 shows the maturity on all the processes, with the top and bottom four highlighted. Their definition according to COBIT can be seen in appendix 6.
FIGURE 10 WEIGHTED RESULTS ON ALL COBIT PROCESSES.
Average maturity, 3.3.
-
45
These processes will be described further in the following sections to clarify how big the gaps to COBIT are in these areas, which was a part of the purpose of this
project. The results and information are based on the interviews.
As seen in figure 12, the most mature processes based on the results of this case study are Manage quality, Procure IT resources, Identify and allocate costs and Manage the physical environment.
FIGURE 11 TOP AND BOTTOM PROCESSES EMPHASIZED
FIGURE 12 THE STRONGEST AREAS
-
46
All of them have policies and procedures which are set from central groups, which means, they cannot just be tracked back to the work within The Markets division. Though some of the work is being done within The Markets division, the standards and guidelines are set outside those borders.
The manage quality process has got strong procedures and a lot of work is being done within that area. The Firm currently has various quality approaches and
systems for different groups and tasks. Methods like Six Sigma and Lean Production is applied to improve processes by eliminating defects and waste within them. According to the Head of Development at The Markets IT division, all processes involved in their software development lifecycle interact with their
quality assurance function and align to the business objectives. All of those processes are managed through a bug tracking tool called Jira32. Jira is an Atlassian
product that also supports measuring of the processes to improve the performance. Jira can also be used for issue tracking and escalation procedures.
The identification and allocation of costs also follows a structured approach. Costs of services provided are identified, verified, allocated and reported to management, business process owners and users in a standardized manner. According to the
Business Manager at The Markets IT division there is a fair bit of documentation and measuring being done as well. This work is primarily done by a group called IT Finance, to which each group within IT reports. IT Finance holds the systems that support the measuring and are responsible for optimizing the process performance.
The procurement of IT resources has a well defined overlying IT procurement plan and specific procurement policies for almost every vendor along with strong, reviewed contractual policies33. The vendors are carefully selected for their
excellence and their offers are reviewed to the extent that the responsible personnel at the IT procurement team requires. According to key personnel at the IT
procurement team, the contracts could be reviewed more frequently but it would be
32 Jira - http://www.atlassian.com/software/jira/
33 Information from interview with key personnel in the IT procurement team
-
47
important to find a balance between constantly reviewing contracts and rely on vendor track record.
According to responsible personnel within the security team the Management of the physical environment (offices, datacenters and sites), is clearly defined and set on a global basis. The procedures and policies are strong and all sites are managed
centrally. This meaning that the responsible group has taken the entire companys sites into consideration when determining the strategy. They have developed a framework for the standard of the security on the sites and a level where they would like to be. In comparison to COBIT they do all the measuring and
documentation suggested, and more. There is a lot of focus on improving the security on the sites, partly driven by terrorist attacks like 9/11 in New York City and the bombings in the London underground.
6.2 WEAKNESSES AT THE FIRM
The processes that showed to have the least defined procedures and the biggest gap to COBIT, were Define and manage service levels, Define a strategic IT plan, Manage the IT investment and Manage problems. The four processes with the lowest maturity can be seen in figure 13.
FIGURE 13 THE WEAKEST AREAS
-
48
The define and manage service levels process has got a structured approach when dealing with service levels between vendors and IT, but the organization lacks an IT service catalogue to agree service levels with the business. According to the global head of ITIL34 this fact is recognized by involved personnel. One of the
goals for 2007 is to build an IT service catalogue and go towards a more defined framework with Service Level Agreements (SLAs) towards the business. This is partly done through the current ITIL initiative, which involves a big change process to address this issue35.
The process called definition of a strategic IT plan seems to be more focused on the tactical IT planning which allows the organization to adapt to the fast changing industry and the policies and procedures in long term planning can more easily be
changed36. The interaction with the business and alignment to the business objectives are not as developed as COBIT suggests. They would like the IT sourcing and acquisition strategy to be more evolved. At the moment it is more tactical than strategic. 34
Manage the IT investment is a process with relatively low maturity as well. The
allocation of responsibility for IT investment and financial planning is done on an ad hoc basis and the project portfolio is inconsistently used in that area37.
Identifying, classifying, fixing and recording problems resides in a process called Manage problems. It follows a repeatable approach but it does no reach the level of a defined process. There is tracking and recording of problems but the root cause analysis does not follow a standardized method.
34 Information from interview with the Global Head of ITIL at The Firm the 23rd of April, 2007.
35 Information from interview with Account Managers at The Firms IT department, the 14th of
March, 2007 36
Information from interview with key personnel at The Markets divisions IT department, the 13th of March, 2007 37
Information from interview with the CFO at The Markets divisions IT department, the 19th of March, 2007
-
49
7 DISCUSSION
This chapter will discuss the results of the assessment and highlight relevant and interesting findings throughout the project.
7.1 DISCUSSING THE RESULTS
In order to understand the maturity results and whether or not they are any good,
one needs to compare it to something. That benchmarking is crucial when drawing the actual conclusions on comparative analysis. The average results of a 3.3, average maturity can seem quite high, but how high are they really? Where would
other companies place on the scale? As this is one of the first studies made by ICS, I really do not have any basis for benchmarking The Firm to other companies. My results will however together with other assessments form the basis for comparative benchmarking in future studies made by ICS.
The results of the assessment were initially un-weighted and the average maturity
was 3.5. The group responsible for the initiation phase of the COBIT initiative at The Firm suggested putting a higher weight on activities and metrics. They also
considered the results to be very high.38 We agreed that a weight of 4 on activities and 2 on metrics was adequate to form results that reasonably would reflect the performance of the IT processes at The Markets division. The activities section is the only input to the results where the respondent is able to grade the performance
on a measurable scale. That, in my opinion, makes the chosen weighting logical. On metrics and documents it is either on or off. During the interviews the
discussions were slightly focused on the activities, which is another reason for
38 Information from discussion with key personnel for the initiation phase of the COBIT initiative at
The Firm, April 20th, 2007.
-
50
them to have a more significant weight. For future reference, the weighting method could be improved by further analysis to reach a suitable state.
It is interesting to see that the documentation reaches a relatively high maturity
level; 4.0. I believe one reason for that could be that the pressures from external regulatory demands like SOX and Basel II, to document financial data. It could
drive the overall documentation to a more standardized level. Documentation procedures and systems that support documentation are likely to be in place. This affects The Firm and other banks in particular because Basel II for instance is focused on that industry.
The final results were discussed together with my advisor from ICS and key
stakeholders in the project at The Firm. We agreed that further analysis on the processes with the highest and lowest maturity could be of interest. This due to the fact that the least mature processes could possibly be improved and the most mature processes could be reviewed to see if they are more defined than necessary.
By cutting down on the effort in those areas, the company could possibly achieve cost savings. The results on these areas are described in section 6.2. These four stronger and weaker areas actually gave one of the most notable acknowledgements that I have received on my results. The processes I have highlighted as the least and most mature seemed to correspond to the views of key personnel at The Firm. One could imply that this increases the reliability of the results since the key personnel did not have a subjective role in the assessment. Furthermore the results still seemed accurate after aggregating the activities, metrics, documents and role assignment, which is another sign that the results provide a true image.
An interesting observation when comparing the different processes and their
maturity results is that the centrally managed processes in general reached a higher maturity. There are several functions or groups within The Firm that are
responsible for only one of the COBIT processes. This could be quality, risk or IT procurement for instance. Those groups have clearly defined policies and procedures. One reason for this, I believe could be that since their work needs to correspond to all areas within the IT organization, with different objectives and
-
51
characteristic, those groups profit from standardization. Ad hoc solutions to support operations would be time and money consuming.
As the goal of this project was to see how mature The Markets division at The Firm was in respect to COBIT and suggest improvement actions to the least mature areas, I will here give my suggestions and discuss the possible benefits of using
COBIT for improvement. The least mature processes were described in more detail in the previous chapter.
7.2 HOW TO IMPROVE THE WEAKNESSES
What is important to notice is that a low maturity does not necessarily mean that the company is performing badly. It could be a conscious choice to leave some
areas less defined, with less documentation and measuring in order to stay nimble, agile and responsive to change. These suggestions below are more or less the gaps on the four least mature processes to COBIT. If The Firm would like to use COBIT as guidance, these suggestions could be useful. As previously mentioned, a few of these suggestions have already been acknowledged and is something The Firm is working on improving. What should be done within the process is suggested in the top boxes in figure 14. The lower boxes show the suggested metrics.
FIGURE 14 SUGGESTED IMPROVEMENTS, CONTROLS AND METRICS
-
52
In order to work with these suggestions the company will need an action plan. It is important to know where to start and evaluate what to focus on. Since there currently is a large global IT policy program running at The Firm, it is important that those procedures and standards are followed. In my opinion the first steps
would be to:
1. Make sure the above results are accurate by engaging more people in i