audit engagement planning audit execution audit reporting audit … · 2018. 7. 21. · audit...

TWD-OGM-ICS Internal Controls Manual (Part 2) Page | 37

III. THE AUDIT PROCESS

Unless otherwise specified, all information are copied from the Philippine

Government Internal Audit Manual (PGIAM) of the Department of Budget

Management (DBM).

A. Four Phases

The Audit Process is divided into four phases, namely: audit engagement

planning, audit execution, audit reporting, and audit follow-up.

This audit process is applicable for both management and operations audit. For

each phase, there are specific criteria to ensure a successful audit engagement.

1. Audit Engagement Planning

Description

Purposes:

a. Understanding the control environment and the organization;

b. Outlining the scope and objectives of the audit;

c. Establishing the basis for budgeting (time, cost, personnel);

d. Identifying the evidence required to develop the audit findings;

e. Assisting in choosing/determining the audit procedures (nature, extent

and timing); and

f. Establishing the basis for coordinating the staff.

Audit Engagement Planning

Audit Execution

Audit Reporting

Audit Follow-up

- Most important part of the audit

- Entails familiarization with the objectives, processes, risks and

controls of the auditee and activity to be audited, and

developing a strategy and approach in conducting the audit

- Involves the listing down of audit activities per audit

engagement based on the AWP


Steps:

a. Document understanding of the program and project

- Involves the following:

i. selection of specific internal controls and focusing on the degree

of compliance with laws, regulation and policies of specific

program, project, system, process for evaluation

ii. evaluation of the control effectiveness

iii. determination of whether or not operations are conducted

economically, efficiently, ethically and effectively

- For Management Audit:

i. involves understanding of management controls

ii. should be based on a sound understanding of the internal

control system, operating & support systems, & processes

- For Operations Audit:

i. involves the selection of a specific activity and focusing only on

a specific program, project, process for evaluation, being

concerned with the economy, efficiency, ethicality and

effectiveness of operations

ii. Audit plan should be based on a sound understanding of the

objectives, accountability, internal control system, and operating

& support processes

iii. Common drawbacks and recommended adjustments:

No. Drawback Adjustment

1 Program objectives are not clear enough

Policy review

2 Measurement systems are inadequate

Restudy the system

3 Subject matter is difficult to measure

Focus the audit on measurable subject matters

4 Purely systematic review may not be adequate

Identify appropriate audit procedures

Document understanding

Audit objective, scope, criteria

& evidence

Audit plan & program

Determine KPIsSecure

approval


No. Drawback Adjustment

5 Time constraints Prioritize audit activities

b. Determine the audit objective, scope and criteria and audit evidence

This step is broken down as follows:

i. Determine audit objective

o What are audit objectives?

1) What the audit aims to accomplish

2) Normally expressed in terms of what questions the audit

is expected to answer about the performance of an

activity

3) Ideally would be consistent with the achievement of the

objectives of the organization / program, project

o Involves the following activities:

o Relate to why the audit is being conducted. If controls are

weak, the ICS traces the root cause and recommends to top

management courses of action to address the deficiency

o For management audits:

Determining the types of audit to be performed

Identifying the focus of the audit & aspect of performance to be examined

Preliminary gathering of docs / info


o For operations audits:

The ICS may choose from any of the following objectives, or

may formulate more which are appropriate to the results of

the audit planning:

ii. Determine audit scope

o What is audit scope?

1) The framework or limits of the audit

2) Normally defined by stating what the audit intends to

cover and the relevant time frames

o Steps in determining audit scope

One of the objectives is to ascertain if the operations has its measurement and evaluation system which will be used to review and improve performance and

assess compliance with laws, rules, methods and procedures

If self-assessment is in place, the ICS evaluates the components of

the performance evaluation system for adequacy,

appropriateness of the measures and reliability of the reporting, a

well as the evaluation result

If self-assessment is not in place, the ICS assesses the internal control system built in the

operating & support system under audit to determine if there are

compensating controls

To determine if the program or project is achieving its target

To validate the reported accomplishments of the program or project as of a certain period

from the data source to the consolidation and preparation of

the final report

To assess and gauge the level of achievement of the program or

project objective


1) Define the parameters and nature of the audit work to

achieve the audit objectives

2) Determine the audit tools, techniques and methodology

to be utilized, and

3) Select the sampling method to be utilized

o For operations audits, audit scope includes the

determination of:

1) Which phase of the program or project will be examined?

2) What will be the duration of the program or project?

3) What portion of the program or project will be covered in

audit?

4) What will be the sources of information for examination?

o For management audits, audit scope includes review and

appraisal of the:

1) Systems (operating & support) & procedures / processes

2) Organizational structure

3) Assets management practices

4) Financial and management records

5) Reports and performance standards

iii. Determine audit criteria & evidence

o What are audit criteria?

1) Reasonable standards against which existing conditions

are assessed

2) Reflect a normative condition for the subject of the audit

3) Expectations of the program/project as to what should be

4) Includes statutory and / or managerial requirements,

process requirements, and citizens’ requirements, needs

& expectations

o To come up with sound criteria, auditors must:

1) Gather / Identify the standards for audit evaluation

2) Set reasonable and attainable standards of performance,

statutory or managerial policies for evaluation

3) Identify pieces of audit evidence required by law and

standards and the approaches to be utilized in obtaining

them


c. Determine the resource required for the audit and the target milestone

/ dates

- Involves assessing the following:

i. Current staff capability / capacity

ii. Technological resources (e.g. computers, software)

iii. Financial resources (budget requirements)

iv. Other considerations

- Target milestones / dates for the completion or accomplishment of

critical elements during the audit process should be established to

keep track of the progress of the engagement and check on the

quality of the outputs

d. Develop the audit plan and audit program

- What is an audit plan?

i. A document that provides the main guidance of the whole audit

process in order to achieve the audit objective in an efficient and

effective way

ii. Provides an integrated description of the auditee and the audit

by serving as guide for the whole audit

- Contents of an audit plan: For Management Audit

Element Information

Introduction A brief description of the management controls or the plan of organization and all the methods and measures adopted within an agency to ensure: o That resources are used consistent with

laws, regulations and managerial policies; o That resources are safeguarded against

loss, wastage and misuse; o That financial and non-financial information

are reliable, accurate and timely; and o That operations are economical, efficient,

ethical and effective

Audit objective & scope

Overall objective and scope of the work to be accomplished

Assessment of controls

Critical processes identified by the ICS during the planning phase which led to the selection of the audit area approved by the GM and the


formulation of the audit objective

Audit approach Compliance audit and management control process audit

Resources / inputs

Statutory policies, mandates, managerial policies, government regulations, established objectives, systems and procedures/processes, etc.

Audit criteria Set of reasonable and attainable standards of performance, statutory or managerial policies, laws and regulations, etc.

- Contents of an audit plan: For Operations Audit

Element Information

Introduction A brief description or background information of the program or project, including: o the main activities and significant events; o information on the structure of the program or

project, systems and processes: 1) which lead to the attainment of the output

or the aggregate of the outputs to achieve the outcome,

2) which process is underperforming causing delays in completion

Audit objective & scope

Overall objective and scope of the work to be accomplished

Assessment of controls

Critical points identified by the ICS during the understanding phase which led to the selection of the audit area approved by the GM and the formulation of the audit objective

Audit approach Audit of program or project results

Resources / inputs

Statutory policies, mandates, managerial policies, citizens’ needs and expectations, manpower, materials, equipment and timelines

Audit criteria Set of reasonable and attainable standards of performance, statutory or managerial policies, laws and regulations, etc.

- What is an audit work program?

i. A document which contains:

o the audit objective

o the step-by-step audit procedures to accomplish the audit

objective,

o the auditor responsible to perform the procedures, and

o the specified time frame


ii. Guidelines for action during the execution phase of the audit

iii. Set out the detailed audit procedures for cost effective collection

of evidence

iv. Describes the details of the planned audit and enumerates the

processes or methods and tools for identifying, analyzing and

recording information gathered during the engagement

e. Determine the Key Performance Indicators (KPIs) of the audit

engagement

- What are KPIs?

i. Performance measures that are utilized to assess the outputs /

outcomes contributing to the overall organizational efficiency

and effectiveness

ii. In evaluating performance, KPIs are employed to gauge the

ICS’ accomplishments and to determine whether or not:

o Audit objectives are met as reflected in the audit findings and

recommendations;

o Findings and recommendations are based on facts,

substantial evidence and in compliance with relevant laws,

rules and regulations;

o There is compliance with Internal Auditing Standards

(NGICS, PGIAM and other relevant standards) under

COA/DBM rules and regulations;

o Findings and recommendations promote the adequacy of

internal control under COA rules and regulations; and

o High standards of ethics and efficiency of public officials and

employees are being observed under OMB and CSC rules

and regulations.

iii. Should be aligned with the internal audit strategic plan and the

annual work plan

iv. Help drive the performance that the organization expects from

the ICS

v. Incorporated in the audit plan to guide the auditors during the

execution of the audit engagement


f. Secure approval of the audit plan and audit work program and KPIs

- Recommended steps for large ICS teams:

- For small ICS teams, only Step 3 may be applicable

Step 1:

The audit plan, audit work program and KPIs, are submitted by the ICS team leader to the Head of ICS for review and approval prior to the commencement of the audit execution.

Step 2:

The Head of ICS will evaluate the documents to assess the relevance, significance, auditability and other factors affecting the conduct of the audit.

Step 3:

After the documents have been approved, management should be informed about the approved audit plan, audit work program and the KPIs. The audit plan and the KPIs should be discussed with management but the audit work program should not be shared.


2. Audit Execution

Steps:

a. Entry conference

- Sets the tone for the audit

- Done to discuss the focus, requirements and time lines of the audit,

as well as to obtain the audited entity’s views and expectations for

the overall framework for the conduct of the audit

- Matters arising from the entry conference must be recorded (as

entry conference notes) and should be considered during the

conduct of the engagement planning

b. Conduct compliance audit

- What is it?

i. The evaluation of the extent or degree of compliance with laws,

regulations, managerial policies and operating processes in the

agency, including compliance with accountability measures,

ethical standards, and contractual obligations

ii. A necessary first step to, and part of, management and

operations audits:

o In management audit, only when there is compliance that

control effectiveness is determined. If there is no

compliance, the probable cause for such non-compliance is

determined.

o In operations audit, compliance audit is done to determine

whether government operations are in accordance with the

organization’s mandate and explicit objectives

Entry conference

Conduct compliance

audit

Conduct system /

process audit

Exit conference


- Steps

c. Conduct system / process audit

- Involves the following:

i. documentation of the process or system under audit

ii. identification of the control procedures

iii. verification and validation on whether or not such control

procedures are complied with and are working effectively

e. Integrate audit findings and prepare the highlights of the audit findings

Do this in terms of the 4Cs:Criteria, Condition, Conclusion & Cause

d. Prepare the working papers

The ICS should record relevant information to support the audit results

c. Determine the probable causes

Acts or ommissions which could have caused the non-compliance

Establish also the why, what and how of the non-compliance

b. Compare conditions with criteria to draw conclusion

Conclusion of facts which is defined as an inference drawn from the subordinate or evidentiary fact.

a. Gather and analyze evidence to establish the condition that the auditee is in

Findings of facts which is defined as a fact, supported by substantial evidence (includes consequence, effects or impact).


- Objectives of process audits:

- Steps:

Operations process audit

• Designed to evaluate the effectiveness, efficiency, ethicality and economy of operating systems selected for audit

Management process audit

• Aims to evaluate control effectiveness

e. Integrate & prepare the highlights of the audit findings

Do this in terms of the 4Cs:Criteria, Condition, Conclusion & Cause

d. Prepare the working papers

Record of relevant information to support audit results

c. Determine the root cause/s

A structured investigation that aims to identify the true cause of a problem & actions necessary to eliminate it

b. Compare conditions with criteria to draw conclusion

Conclusion of facts which is defined as inference

(Drawn from the subordinate or evidentiary fact)

a. Gather and analyze evidence to establish the condition

Findings of facts defined as a fact and supported by substantial evidence

(includes consequence,effects or impact)


d. Exit conference

- The purpose is to discuss the highlights of the audit findings with

the auditee and/or the responsible official who has sufficient

knowledge about the audit area

- Provides an opportunity to get the auditee’s comments or

management comments and insights about the significant audit

issues as a way of validating the findings:

i. Management’s comments should be taken into consideration so

as to arrive at workable recommendations and obtain the

auditee’s commitment towards performing remedial actions.

ii. The auditee’s comments / responses are recorded in the audit

findings sheet and integrated into the draft report.

3. Audit Reporting

Represents the culmination of the audit execution and the associated

analysis and considerations made during the audit

The audit report sets out the findings in appropriate format: provides the

pieces of evidence gathered to arrive at the audit findings and the

recommendations

Steps:

a. Develop audit findings

- What are audit findings?

i. Can be developed by analyzing the pieces of evidence gathered

for each of the audit elements

ii. Should align with the audit objectives

iii. Should be rational and based on specific standards and criteria.

iv. Compare the conditions with the audit criteria, and determine

the causes

- Audit findings on probable cause of illegality of a transaction

constitute a violation of law while irregularity constitutes a violation

of regulations

Audit findings

Audit recommend-

ations

Draft audit report

Update the GM

Final audit report


- Types of evidence:

- What are “conditions” compared with the audit criteria?

- Once an audit finding has been identified, two (2) complementary

forms of assessment take place:

i. Assessment of the significance of the findings

ii. Determination of the probable cause/s and the root cause/s

- All audit findings should be formulated based on the four Cs:

Physical Documentary Testimonial Analytical Electronic

•Standards against which a condition is compared with

•e.g. laws, regulations, policies

Criteria

•A fact, backed up by substantial evidence

•What is currently being done or the current situation

•What the auditor actually finds as a result of the review

Condition

•Evaluation of the criteria & conditions that could either result in compliance or non-compliance with laws, regulations and policies, as supported by substantial evidence

•Determination of adequacy or inadequacy of controls

•Determination of the efficiency, effectiveness, ethicality, and economy of agency operations

Conclusion

•Immediate and proximate reasons/s for the condition for which substantial evidence will be used as basis of the audit recommendation

•Probable cause that could have caused non-compliance and root cause

Cause

Factual and evidentiary conditions such as the current state /

practices or what is obtaining, and their effects


b. Develop audit recommendations

- What is it?

i. Management / Legal remedies to avoid occurrence

ii. Provide courses of action as the basis for improving internal

controls

iii. Should:

o Be clear,

o Be based on science of facts, conditions and evidence

o Consist of practicable, incontestable and workable

solutions that can stand alone and address the issue(s)

at hand

- Issues to consider in developing recommendations are as follows:

Officer primarily responsible

• General Manager

Recommended courses of action

• Should indicate what needs to be done, but not how to do it.

• The “how” of it is the responsibility of the unit and/or management concerned.

Other items to be included

• Circumstances that aid or hinder the organization in achieving the criteria

• The feasibility and cost-benefit analysis of adopting a recommendation

• Alternative courses for remedial actions

• Effects of the recommendation (positive and negative)


c. Prepare draft audit report

- Prepared by laying out and analyzing the pieces of evidence

gathered to arrive at preliminary audit findings and

recommendations

- When preparing a draft audit report, the auditor should

i. Delineate the objectives and scope and report within that scope,

unless other issues of substance are identified;

ii. Identify all criteria;

iii. Report significant matters – positive or negative;

iv. Describe the context and background of the reported matter

only as far as is necessary to provide an understanding of the

issue;

v. State initial findings, management’s comments and team’s

rejoinder, if any;

vi. Present the audit findings in a manner that is concise, fair and

objective; and

vii. State the recommendations so that they indicate what needs to

be done but not how to do it.

d. Update the GM

- The GM should be updated on the results of the audit engagement

e. Prepare the final audit report

- The draft report may then be finalized integrating the following as

parts of the final report:

i. Table of Contents;

ii. Executive Summary;

iii. Detailed Audit Findings;

iv. Management Comments and Team’s Rejoinder;

v. Monitoring and Feedback on Prior Year’s Recommendations;

vi. Recommendations; and

vii. Appendices.

- The final audit report should be presented to the GM who decides

on the distribution of the audit report based on the recommendation

of the ICS


4. Audit Follow-up

A monitoring and feedback activity undertaken to ensure the extent and

adequacy of preventive / corrective actions taken by the Management to

address the inadequacies identified during the audit

Aims to increase the probability that recommendations will be

implemented

Purposes:

Steps

a. Monitor implementation of approved audit findings and

recommendations

- It is a sound practice to monitor the implementation of approved

recommendations (management/legal remedies) to avoid the

occurrence (preventive measures) and recurrence (corrective

measures) of control weaknesses/incidences after a reasonable

period from the report submission date.

• To increase the probability that recommendations will be implemented

Increase the effectiveness of

audits

• To propose necessary actions to the GM and other officials

Assist the government

• Provides basis for evaluationEvaluate the ICS

Performance

• May contribute to better knowledge and improved practice

Create incentives for learning &

development

Monitor implementation

Resolve non- and inadequate

implementation

Prepare Audit Follow-up Report


- The benefits of internal audit report recommendations are reduced,

and deficiencies remain, if recommendations are not implemented

within the specified timeframe.

- It is management’s responsibility to implement approved findings

and recommendations, but the internal audit is in a good position to

monitor the progress of implementation of the recommendations

b. Resolve non-implementation / inadequate implementation of audit

recommendations

- In the event of non-implementation of recommendation /

inadequate action, the ICS recommends appropriate legal and/or

management remedies for non-implementation of recommendation

and inadequate preventive / corrective actions.

c. Prepare audit follow-up report

- Results of the audit follow-up should be recorded and reported in

order to apprise the GM of the status of actions on the approved

recommendations.

- The reasons for the lack of action or non-completion of action on

any recommendation should be documented and further action

considered on significant recommendations that have not been

acted upon.

- Where possible, the report should:

i. Describe the results of the auditor’s analysis of actual against

projected benefits for the period under review;

ii. Summarize the extent of implementation of the approved

recommendations;

iii. Highlight cases where auditee’s performance in implementing

recommendations have been particularly inadequate; and

iv. Describe the actions, if any, that the auditor intends to take in

relation to inadequate auditee’s actions.


B. Gathering and Analysis of Evidence

1. Steps

2. Sufficiency and appropriateness of audit evidence

What is sufficient and appropriate is the result of the auditor’s sound

evaluation and is dependent on:

Sufficiency and appropriateness of audit evidence are interrelated:

Sufficient and appropriate means that the audit evidence must be

substantial enough to influence or convince the GM to implement the

recommended courses of action. Substantial evidence is more than a

mere scintilla of evidence. It means such relevant evidence as a

Identify the control tested

Consider the evidence available

to support or contradict

Select the method of obtaining the

necessary evidence

Collect and evaluate that

evidence to form audit findings

Nature of the control deficiency

Materiality

Source of information

and evidence

Prior audit experience

Results of other audit procedures

Sufficiency

•the measure of the quantity of audit evidence

•affected by the auditor‟s assessment of the impact of control deficiencies (the higher the impact, the more audit evidence is likely to be required) and also by the quality of such audit evidence (the higher the quality, the less may be required).

•If no evidence is obtainable for certain deficiencies, the particular area/topic is not auditable

Appropriateness

•measure of the quality of audit evidence

•its relevance and reliability in providing support for the audit findings.

•It should assist in meeting the audit objectives and is credible.


reasonable mind might accept as adequate to support a conclusion, even

if other minds equally reasonable might conceivably opine otherwise

3. Characteristics of evidence

4. Types of Audit Evidence

• One having value in reason as tending to prove any matter provable in an action

Relevant

• That which proves the fact in dispute without the aid of any inference or presumeption

Direct

• Proof of a fact or facts from which, taken either singly or collectively, the existence of the partiicular fact in dispute may be inferred as a necessary or probable consequence

Circumstantial

• Additional evidence of a different character to the same point

Corroborative

• Any testimonial, documentary or tangible evidence that may be introduced in orderto establish or bolster a point;

• Must be relevant, no prejudicial, reliable

Admissible

Physical Testimonial Documentary Analytical Electronic


Physical Evidence

Testimonial Evidence

Documentary Evidence

Hierarchy of reliability:

Description

•obtained by direct observation

•may require proof of anoher evidence (such as documentary or photographic evidence)

Examples

•cash count

•project site visits

•inventory count

Sources

•observation of processes and procedures

•site visits to gain personal knowledge of the practicality and physical state of work as they are at a point in time

•physical verification of assets

Description

•obtained from others through oral or written statements in response to inquiries or through interview

Examples

•Interview notes

•Recorded conversations

•Corroborated evidence or testimonies from other people that have knowledge of the issue at hand

Sources

•comes from interviews with interested parties

Description

•most commonly used source of evidence

•more reliable than oral representations

Examples

•Manuals

•Files

•Reports

•Instructions

•Contracts

•Invoices

•Vouchers

Sources

•solicitation (ask for or request)

•elicitation (draw, extract, obtain)

Independent external evidence

Internally provided evidence


Note: Internal evidence is more reliable when related internal controls are

satisfactory

Analytical Evidence

Electronic Evidence

5. Use of evidence

Overreliance on any one form of evidence may impact on the validity of the

findings. One should gather a wide variety of evidence for purposes of

triangulation of multiple forms of diverse and corroborating types of evidence.

This is to check the validity and reliability of the findings. Thus, more cross-

checks on the accuracy of the decision should be undertaken.

Pieces of evidence in support of the findings should be corroborative as a

result of triangulation of evidence gathered in at least three approaches.

Triangulation involves employing multiple forms of corroborating diverse types

and sources of evidence and perspectives. By using multiple forms of

evidence and perspectives, a veritable portrait of the facts and conditions can

be developed.

6. Audit approaches and techniques in gathering audit evidences

In selecting the audit techniques to be used, the IA should first determine

what needs to be done and what pieces of evidence to obtain.

There are a number of audit approaches and techniques that can be

adopted in gathering audit evidence:

Description

•built up by analyzing the information obtained from other sources

Examples

•cost-benefit analysis

Sources

•may not be easily available in a ready-made format

•usually developed by the auditor

Description

•derived from different types of electronic devices

•collecting requires careful planning and execution, preferably by experts

•may be challenged on the basis of unreliability, but can be countered if it can be shown that controls are in place

Examples

•Hardware & network diagrams

•Operating systems software

•Network & communications software

•Journal & activity logs

•Application programs

•Flow diagrams


a. Inquiries and interviews

b. Sampling

Inquiries and

interviewsSampling CAATs

•A question and answer session to elicit specific information

•A way of gathering facts and information, and gaining support for a variety of arguments

•Basis of most audit work, but should not be relied on as a sole source

•Carried out at different stages of the audit

Description

•Fact-finding conversations & discussions

•Unstructured interviews (with open-ended questions)

•Structured interviews (with closed questions)

Methods

•Preparatory interviews

•Interviews to collect or validate material information

•Interviews to generate and assess facts and pieces of evidence

Types

•Must be compiled and documented in a way that facilitates analysis and reliability of information

•Can be sources of conditions, causes and potential recommendations for the development of audit findings and recommendations

Results

• A scientific method of selecting the transactions to be subjected to audit

• Provides efficiency and economy in the audit process

• Allows auditor to test less than 100% of the population to form audit findings, on the assumption that the sample selected is representative of the population

Description


c. CAATTs (Computer-Assisted Audit Techniques and Tools)

• Systematic

• Statistical

• Non-statistical

• Random

• Simple random

• Stratified

Types

• See Appendix 9 for details

Procedures

• computer tools and techniques in performing auditing procedures and improving the effectiveness and efficiency of obtaining and evaluating audit evidence

• provides effective tests of controls and substantive procedures where a wide range of techniques and tools are used to automate the test procedures for evaluating controls, obtaining evidence and data analysis

Description

• Type 1: CAATTs used to validate programs / systems

• Type 2: CAATTs used to analyze data files

• Results can indirectly help auditor to reach conclusions regarding the quality of programs but they do not test the validity of the programs

Types

• Type 1:

• Detailed examination of program coding

• Involves a fair degree of programming skill & a thorough knowledge of program specification

Procedures


Generally, an audit will involve a combination of such approaches.

The audit approach selected should be the most time and cost-effective

given the objectives and scope of the audit.

It should aim to collect sufficient and appropriate evidence that enables

the auditor to come to well-founded audit findings about the program or

activity under review and to make appropriate recommendations.

Decisions will have to be made at each stage of the audit about the need

for specific testing, data collection and analysis by the internal audit and

the extent that reliance can be placed on the work of other internal or

external reviewers.

7. Techniques in the analysis of evidence

All audit findings must therefore be based on appropriate analyses and

evaluation of the information and/or evidence

Include:

a. Structured or semi-structured interviews

b. Delphi Technique

c. Root cause analysis

d. Fault tree analysis

e. Cause-consequence analysis

f. Cause and effect analysis

g. Bow tie analysis

h. Cost/benefit analysis


C. Root Cause Analysis

1. What is it?

A method used to address a deficiency to determine the root cause of the

problem

Used to correct or eliminate the cause and prevent the problem from

recurring

Attempts to identify the root or original causes, instead of dealing with the

immediately obvious symptoms

A structured review and evaluation that aims to identify the true cause of a

deficiency and the courses of action necessary to address it

Means continuing to ask “why” the control deficiency occurred until the

fundamental process element that failed is identified

2. Basic Steps

3. Techniques

Selected techniques that can be used are as follows:

a. Establishing the scope and objectives of the RCA;

b. Gathering data and evidence relating to the non-compliance;

c. Performing a structured analysis to determine the root cause; and

d. Developing solutions and making recommendations.

5 Whys FMEA FTA Fishbone Pareto


a. 5 Whys

- A simple technique done by repeatedly asking “why” to peel away

layers of cause and sub-causes

- The following discussion is derived from various sources, including

the author’s work experience. Example:

Why? 5

No strategic plan to increase collections

Why? 4

Poor cash management / low collections

Why? 4TWD cannot afford the high collection cost charged by 3rd party

collecting agents

Why? 3

Plans to add payment centers have not yet materialized

Why? 2

There are only 2 payment centers

Why? 1

Long customer queues during payment due dates

Problem

Low customer satisfaction rating


- Guidelines:

i. Reasons presented should only include those that are within the

control of the organization

ii. Doesn’t have to be wordy

iii. Doesn’t have to be always composed of 5 reasons. It can be

more or less than 5, as long as the root cause is identified

iv. How to know if it is the root cause? When there is no other

answer for the “Why”.

v. For each arrow going from left to write, read it using the word

“because”

Example:

For “Why? 3”, it is not enough to say “high collection

costs” because that is beyond the control of the

organization. However, if it is said that “the organization

cannot afford the high collection costs”, then it can be an

acceptable cause.

Example:

The problem is we have a low customer satisfaction rating…

Because: of long customer queues during payment due dates…

Because: there are only 2 payment centers

Because: plans to add payment centers have not materialized

Because: we can’t afford the high cost charged by 3rd parties

Because: we have poor cash management / low collections

Because: we have no strategic plan to increase collections


vi. To check if the analysis makes sense, read the reasons

backwards, starting with the last “Why” and connecting it with

the previous “Why” by using the word “therefore”

vii. “The 5 Whys technique is a simple technique that can help you

quickly get to the root of a problem. But that is all it is, and the

more complex things get, the more likely it is to lead you down a

false trail. If it doesn't quickly give you an answer that's

obviously right, then you may need to use a more sophisticated

problem solving technique such as Root Cause Analysis or

Cause and Effect Analysis.” (Mind Tools Ltd., 2013)

b. FMEA (Failure Mode & Effects Analysis)

- Used to identify the ways in which the components, systems or

processes can fail to fulfill their design intent

- Identifies:

i. All potential failure modes of the various parts of a system (a

failure mode is what is observed to fail or to perform incorrectly,

i.e., the deficiency in control design and control operation);

ii. The effects these failures may have on the system;

iii. The mechanisms of failure; and

iv. How to avoid the failures and/or mitigate the effects of the

failures on the system.

Example:

We have no strategic plan to increase collections…

Therefore: we have poor cash management / low collections

Therefore: we can’t afford the high cost charged by 3rd partie

Therefore: plans to add payment centers have not materialized

Therefore: there are only 2 payment centers

Therefore: long customer queues during payment due dates

Therefore: we have a low customer satisfaction rating


- Background and history, according to the FMEA website

i. Formally developed and applied by NASA in the 1960’s to

improve and verify reliability of space program hardware.

ii. Used as a reliability evaluation technique to determine the effect

of system and equipment failures. Failures were classified

according to their impact on mission success and

personnel/equipment safety.

iii. The procedures called out in MIL-STD-1629A are the most

widely accepted methods throughout the military and

commercial industry

(FMEA-FMECA.com, 2006)

- Procedures

i. Get an overview of the system:

o Determine the function of all components.

o Create functional and reliability block diagrams.

o Document all environments and missions of sys.

ii. ID all potential failure modes of each component.

iii. Establish failure effect on the next level of the sys.

o Determine failure detection methods.

o Determine if common mode failures exits.

iv. Determine criticality of the failure, ranking & CIL.

o Develop CIL

o Corrective actions/retention rationale.

v. Provide suitable follow-up or corrective actions.

(NASA Lewis Research Center, 2006)

- Procedure Flowchart



- Worksheet Template


- Example

(Avaluation.com, 2009)


c. FTA (Fault Tree Analysis)

- Used for identifying and analyzing the factors that can contribute to

a specified undesired event (top event)

- Causal factors are deductively identified, organized in a logical

manner and represented pictorially in a tree diagram which depicts

the causal factors and their logical relationship to the top event

- Process overview:

i. If the technique is being applied in a formal, scheduled session,

take the necessary steps to prepare for conducting the FTA.

o If technological methods will be used, acquire concept

mapping software, a computer, a projection device (for

example, a video projector), and a projection surface or

screen.

o If non-technological methods will be used, ensure that you

have access to a large surface area (that is, a whiteboard or

chalkboard) on which you can create the concept map, as

well as thick markers in various colors, tape, and so on.

o If you are doing the concept mapping session with a large

number of participants, consider identifying a colleague or

assistant who is able to create the actual concept map while

the facilitator mediates the session.

o Identify and invite participants who are experts on the

system that will be the focus of the FTA.

o Schedule the FTA activity session.

ii. Using your list of information required for the needs

assessment, define the system that will be the focus of the FTA.

iii. Identify the “what should be” for the system either by identifying

the system’s mission, purpose, or goals, or by defining the

criteria for what the “ideal situation” would look like.

iv. Working with an expert on the system of focus, begin the

process of building the fault tree (see figure 3B.3). Determine, in

specific terms, “the top undesired event” for which you want to

identify the underlying causes. Write the top undesired event at

the top of the tree. This undesired event will be the foundation


on which the FTA will be constructed, so it is important that it be

identified in clear terms.

v. Identify the factors (conditions) that are in the immediate vicinity

of the top undesired event and that could be causing it. Write

those key factors immediately below the top of the tree.

vi. Look at each of the key factors you have identified in the

previous step. What sub-factors could be causing the key

factors? Identify the sub-factors, and place them underneath the

appropriate factor on the tree. Do not move on to the next level

of analysis until there is consensus that all factors at the current

level have been identified.

vii. Continue this procedure—building the tree-like graphic—until

there is a general consensus that the tree is finished.

viii. After the fault tree has been completed, work with experts to

carefully and systematically analyze it for accuracy. Compare

the fault tree’s factors and structure against the actual system

being analyzed.

ix. Analyze the fault tree. This analysis can be done either

statistically or through informal nonstatistical methods (such as

brainstorming). To analyze quantitatively, use statistical analysis

to determine the probability of all the contributing factors you

have listed in the tree. This analysis can be complex, and we

recommend doing additional readings before completing the

analysis.

x. By drawing on your analysis, you should be able to identify the

potential factors, as well as the sequences of factors, that may

account for the performance problem that you identified as the

top undesired event.

xi. Focus particularly on the factors that appear lowest in the tree,

because remedying or preventing these root causes is the most

effective and efficient way to obstruct or eliminate the critical

paths leading to the top undesired event.

(Ryan Watkins, 2008)


- Tips for Success

i. The FTA technique works best for problems that have a medium

level of complexity. For very complex problems, this technique

can be difficult to manage or overwhelming for people to

interpret.

ii. Remember that the expert insight that is used to construct the

fault tree is generally of a very subjective nature. Take steps to

consult as many experts as possible and to externally validate

the fault tree and its outcomes. Both of these steps will reduce

the subjectivity to some extent.


- Example



d. Fishbone or Ishikawa Diagrams

- What is it?

i. A cause and effect analysis method to identify many possible

causes of an undesirable event or problem

ii. Can be used to structure a brainstorming session

iii. Sorts ideas into useful categories

- Procedures

i. The Problem Statement. Write the problem statement at the

center right of the document / flipchart / whiteboard / screen.

Draw a box around it then draw a horizontal line / arrow from the

box to the left side of the sheet. The box would be the head and

the line the vertebra / backbone of the fish.

ii. The Categories. Draw five (5) diagonal lines stemming from the

main horizontal line: three (3) on top and two (2) below (or

reverse). The lines should be thinner than the horizontal line.

Label each diagonal line as follows:

o Surroundings

o Suppliers

o Systems

o Skills

o Safety


iii. Causes. Write all the possible causes of the problem and connect these to the “cause” diagonal

lines. Again, the lines should be thinner than the diagonal line. Ask: “Why does this happen?” As

each idea is given, write it as a branch from the appropriate category. Causes can be written in

several places if they relate to several categories.


iv. Sub-causes. Again, ask “Why does this happen?” about each cause. Write sub-causes branching

off the causes. Lines should be thinner than the lines for the causes. Continue to ask “Why?” and

generate deeper levels of causes. Layers of branches indicate causal relationships.

v. Root causes. Encircle the sub-causes which do not have further sub-causes. These are the root

causes.

(American Society for Quality, 2013) & (The Business Tools Store, 2012)


e. Pareto Analysis

- A method using statistics to discover the most important causes of

an effect based on the “Pareto Principle” which states that only

“vital few” factors (20%) are responsible for producing most of the

problems (80%). If these few key causes are corrected, then there

will be a greater probability of success

- Procedures

i. Identify and list the problems.

ii. Identify the root cause of each problem using other techniques

(5 Whys, Fishbone, Fault Tree, etc.).

iii. Form a table listing the causes and their frequency as a %.

iv. Arrange the causes in decreasing order of importance.

v. Add a cumulative percentage column to the table.

No. Causes

Count %

1 No policy 5 25%

2 Insufficient number of staff 6 30%

3 Unequal distribution of work load 4 20%

4 Poor cashflow management 2 10%

5 Poor collection 3 15%

20 100%

Frequency

No. Causes

Count %

1 Insufficient number of staff 6 30%

2 No policy 5 25%

3 Unequal distribution of work load 4 20%

4 Poor collection 3 15%

5 Poor cashflow management 2 10%

20 100%

Frequency

No. Causes

Count % Count %

1 Insufficient number of staff 6 30% 6 30%

2 No policy 5 25% 11 55%

3 Unequal distribution of work load 4 20% 15 75%

4 Poor collection 3 15% 18 90%

5 Poor cashflow management 2 10% 20 100%

20 100%

Frequency Cumulative Freq


vi. Plot values in a Pareto Diagram. To do this:

o Manually:

a) Set-up: Use x-axis to plot the causes. There will be two y-

axes: Percentage on the left (primary axis) and

Cumulative percentages on the right (secondary axis).

b) Plot the frequency of each cause using a bar graph.

c) Plot the cumulative frequency of each cause using a line

graph, placed on top of the bar graph.

d) Draw a horizontal line corresponding to the 80% mark at

the secondary y-axis (cumulative percentage). Find out

where in the line graph this horizontal line intersects. At

this point, draw a broken vertical line. This broken line

separates the important causes on the left and the less

important on the right.

o Through Microsoft Excel

Note: Adapted from the following resources:

(Mind Tools Ltd, 2013) & (Haughey, 2013)


D. Other Considerations

1. Substantive Tests

A comprehensive analysis by using ratios, analytical procedures, inquiries,

confirmation and other tools and techniques

Executed audit procedures enumerated in the audit work program on

samples selected

Procedures seek to provide evidence as to the various control

attributes/features established during the planning stage of the audit:

a. Existence

b. Occurrence

c. Completeness

d. Validity

e. Adequacy

f. Efficiency

g. Effectiveness

h. Economy, etc.

2. Work of Other Experts

When there is a need to make use of other experts’ work to corroborate or

substantiate the facts/evidence gathered by the internal auditors, they

remain responsible for its use.

Experts are those who have acquired special knowledge, skill, experience

or training in a particular field other than auditing. The auditor may use the

work of an expert as evidence but the auditor retains full responsibility for

the contents of the audit report.

Expert task in auditing is expertise gained in the course of audit activities.

Expert tasks are performed in a way that does not endanger the

impartiality of audit activities. Expert tasks include participating in working

groups or projects, presenting initiatives to correct observed deficiencies

in administration, issuing statements and arranging trainings.

The steps the auditor should take are:

a. Obtain information on the qualifications, competence or specialization

of the experts and the context of their assignment. For instance,

opinions on information technology (IT) process should not just be from

a computer science graduate but from a recognized and reputable IT

practitioner demonstrating a profound level of expertise;


b. Consider the nature, complexity and materiality of the matter,

assumptions used, and corroborative evidence available;

c. Consider the objectivity of the expert; and

d. Advise the expert on what the work is being used for and the purpose

3. Integration and Preparation of Highlights of Audit Findings

In the preparation of audit findings, the conditions, conclusions and the

causes must be supported by sufficient audit evidence. The quantum of

evidence required to support an audit finding is substantial evidence. Such

substantial evidence would lead to the determination/finding of a probable

cause or a prima facie case and would draw a reasonable conclusion that

more likely than not, a non-compliance or failure of control/supervision

was established, and that an offense may have been committed.

a. “Substantial evidence is more than a mere scintilla of evidence. It

means such relevant evidence as a reasonable mind might accept as

adequate to support a conclusion, even if other minds equally

reasonable might conceivably opine otherwise.”

b. A finding of probable cause for non-compliance needs only to rest on

evidence showing that more likely than not the act/s or omission/s of

the person responsible had caused the non-compliance with laws,

regulations and managerial policies and operating procedures in the

agency, including compliance with accountability measures, ethical

standards and contractual obligations, which may warrant the conduct

of administrative proceeding by the disciplining authority. It must be

noted that to come up with the determination of probable cause/s, the

ICS must be able to establish, not only the facts and circumstances,

but also the why’s, the what’s and the how’s of the non-compliance.

c. “Prima facie requires a degree or quantum of proof greater than

probable cause… [i]t denotes evidence, which, if unexplained or

uncontradicted, is sufficient to sustain a prosecution or establish the

facts as to counterbalance the presumption of innocence and warrant

conviction x x x.”

This could also give rise to a disputable presumption of non-

compliance with a regulation or rule. “A disputable presumption has

been defined as a species of evidence that may be accepted and


acted on where there is no other evidence to uphold the contention for

which may be overcome by other evidence.”

The Supreme Court in Balbastro vs. COA, G.R. No. 171481, 30 June

2008, found the petitioner guilty on the basis of the audit report which

constitutes substantial evidence. The pertinent ruling reads:

“In fine, petitioner‟s arguments only render more pronounced

the correctness of the Ombudsman‟s decision finding her guilty

on the basis of the audit report which constitutes substantial

evidence. As Balbastro v. Junio held, an administrative case

also involving herein petitioner:

As to the findings of the Ombudsman, it is settled that

in administrative proceedings, the quantum of proof

required for a finding of guilt is only substantial

evidence – that amount of relevant evidence which a

reasonable mind might accept as adequate to justify a

conclusion. x x x.”

The audit findings supported by substantial evidence are deemed

admitted by the auditee if not controverted by any evidence to

overcome the same. In this case, the burden of proof now lies with the

auditee. “Burden of proof is the duty of a party to present such amount

of evidence on the facts in issue as the law deems necessary for the

establishment of his claim.”


E. References

American Society for Quality. (2013). Fishbone (Ishikawa) Diagram. Retrieved June 13, 2013, from ASQ:

http://asq.org/learn-about-quality/cause-analysis-tools/overview/fishbone.html

Avaluation.com. (2009). Failure Modes & Effects Analysis Worksheet

(http://perspectives.avalution.com/2009/risk-assessment-purpose-and-pitfalls-2/). Retrieved June 11,

2013, from www.bing.com:

http://www.bing.com/images/search?q=fmea+sample&qpvt=fmea+sample&FORM=IGRE#view=detail&i

d=B4B1FE44BDC3761198453C5193E138999CFE61A3&selectedIndex=12

FMEA-FMECA.com. (2006). What is a FMEA? Retrieved June 11, 2013, from FMEA-FMECA.com:

http://fmea-fmeca.com/what-is-fmea-fmeca.html

Haughey, D. (2013). Pareto Analysis Step by Step. Retrieved June 13, 2013, from ProjectSmart.co.uk:

http://www.projectsmart.co.uk/pareto-analysis-step-by-step.html

Mind Tools Ltd. (2013). Pareto Analysis: Using the 80:20 Rule to Prioritize. Retrieved June 13, 2013, from

Mind Tools: http://www.mindtools.com/pages/article/newTED_01.htm

Mind Tools Ltd. (2013). 5 Whys: Quickly Getting to the Root of a Problem. Retrieved June 11, 2013, from

MindTools: http://www.mindtools.com/pages/article/newTMC_5W.htm

NASA Lewis Research Center. (2006). Tools of Reliability Analysis -- Introduction and FMEAs. Retrieved

June 11, 2013, from FMEA-FMECA.com: http://fmea-fmeca.com/fmea-examples.html

Ryan Watkins, M. W. (2008). Fault Tree Analysis. Retrieved June 11, 2013, from RyanRWatkins.com:

http://ryanrwatkins.com/na/guidebook/Fault%20tree%20analysis.pdf

The Business Tools Store. (2012). Cause and Effect Ishikawa Fishbone Diagram - Excel Template User

Guide. Retrieved June 13, 2013, from The Business Tools Store:

http://www.businesstoolsstore.com/content/User%20Guides/Cause%20and%20Effect%20Ishikawa%20

Fishbone%20Diagrams%20Excel%20Template%20User%20Guide.pdf

audit engagement planning audit execution audit reporting audit … · 2018. 7. 21. · audit...

Documents