every five years! - acuaacua.org/acua_resources/documents/c-3everyfiveyears... · every five years!...
TRANSCRIPT
9/12/2017
1
Every Five Years!
Preparing for an External QAR
Quality Assessment ReviewQARQA
Peer ReviewExternal Quality Assessment
EQA
9/12/2017
2
When we finish this session you should be able to
describe elements of an external QAR,
identify evidence that supports conformance with the Standards,
recognize supporting materials typically used by an external assessment team,
evaluate your department’s readiness for an external QAR, and
develop a work plan to prepare your department for an external QAR.
Does this look familiar?
International Professional Practices Framework (IPPF)®
9/12/2017
3
Quality Assessment Manual For the
Internal Audit Activity
Copyright © 2013 by The IIA Research Foundation
Advice & counsel from practitioners with decades of experience and broad exposure to best IA functions
Sounding board
Leverage for funding, authority, independence, & training
Pipeline to audit committee & senior management
Compliance with the Standards
What are the benefits of a QAR?
9/12/2017
4
Know IIA Standards
Mandatory vsrecommended guidance
Maintain a QAIP
Quality operations
Assess Readiness
Gap analysis
Conduct QAR
Low stress external assessment
Blueprint for a Successful QAR
Know QAR Process
Assessor expectations
Blueprint for a Successful QAR
Know QAR Process
Core Principles for the Professional Practice of Internal Auditing, Code of Ethics, Definition of Internal Auditing
International Standards for the Professional Practice of Internal Auditing
Audit governance, staff, management, and process
Opportunities for improvement & best practices
What is the focus of a QAR?
9/12/2017
5
Blueprint for a Successful QAR
Know QAR Process What happens during the QAR?
Assess compliance with the Core Principles, Code of Ethics, and StandardsReview charters, policies, procedures, and practicesSurvey clients and interview stakeholdersReview staff experience & qualificationsInspect work papersAssess conformance, evaluate and report
Agree on the scope and objectives
Select & prepare external assessment team
Request & review planning guides
Clarify document request
Agree on surveys and interviews
Discuss physical arrangements
Typical Planning Topics
Know QAR Process
9/12/2017
6
Some documents will be requested in advance. Others will only need to be available onsite.
The QAR is easier when the planning documentation is completed by internal audit prior to the onsite visit.
Document Requests
Know QAR Process
• Executive Leadership and Operating ManagementSelect individuals from areas where engagements have been performed in the last one or two years.
• Internal Audit StaffWith larger departments completion by all staff should be mandatory. With smaller departments consider using interviews to obtain input.
Surveys
Know QAR Process
9/12/2017
7
• Surveys may be sent out by the assessor or by Internal Audit
• Completed surveys should always be returned directly to the external assessor so responses can remain confidential.
Surveys
Know QAR Process
• Assessor selects individuals from organization chart.• Audit Committee Chair
• Chief Executive Officer• Chief Financial Officer• Chief Information Officer
• Chief Operating Officer• Chief Risk Officer• Chief Compliance Officer
• External Auditors
• Internal Audit typically schedules the interviews.
Interviews
Know QAR Process
9/12/2017
8
• Ensure the availability of all IA staff (including CAE)• Provide a work location that affords some privacy• Provide internet connection and printing• Coordinate interviews with the board, operating management
• Expect prompt feedback if significant issues are identified
Host Onsite Visit
Know QAR Process
• Summary of observations and recommendations including successful practices and deficiencies
• Consider a closing conference• Review a draft report; consider comments as appropriate
• Final report is issued to CAE and individuals or groups initiating the external assessment
Receive Report
Know QAR Process
9/12/2017
9
What is important?
and
What is NOT as important?
Blueprint for a Successful QAR
Know IIA Standards
Mandatory vsrecommended guidance
Mandatory Guidance
• Core Principles•Definition of Internal Auditing• Code of Ethics• International Standards for the Professional Practice of Internal Auditing
Know IIA Standards
Mandatory guidance
9/12/2017
10
Mandatory Guidance
11 Major Standards
40 Supporting Standards
51 Implementation Standards
Know IIA Standards
Mandatory guidance
The Standards
•The Standard
•Evidence of Conformance
•Typical Document Request
Know IIA Standards
9/12/2017
11
Governance
Management
Staff
Process
1000, 11001300, +
20002100, 2600
2200, 23002400, 2500
1200
Determine the level of conformance with the spirit and intent of
• Standard 1000: Purpose, Authority, & Responsibility• Standard 1100: Independence & Objectivity• Standard 1300: Quality Assurance & Improvement Program
• Core principles, Code of Ethics, Definition of Internal Auditing
9/12/2017
12
• Defined and approved by the Board (1000).• Defines nature of assurance and consulting services (1000.A1 & .C1).
• References mandatory nature of Core principles for the Professional Practice of Internal Auditing, Code of Ethics, Standards, and the Definition of Internal Auditing (1010).
Purpose, Authority, and Responsibility
• The approved internal audit charter• Minutes of board meetings showing periodic review of charter
Evidence of Conformance
9/12/2017
13
• The approved internal audit charter• Institution’s organization chart• Minutes of board meetings
Typical Document Requests
Independence and Objectivity
• Appropriate reporting relationships (1110, 1111, 1112).• No restrictions on the scope, resources, and access of internal audit (1110 & 1130).
• Objectivity (Code of Ethics, 1120, 1130).• Roles beyond internal auditing (1112)
9/12/2017
14
• Approved internal audit charter• Institution’s organization chart showing reporting lines• Policies on independence, objectivity, addressing conflicts, performance evaluation
• Conflict‐of‐interest disclosure forms
• Board meeting agendas and minutes
Evidence of Conformance
• Audit committee charter
• Institution’s organization chart
• Internal audit aharter• Policy on objectivity• Policy on Code of Ethics
• Audit committee agendas and minutes
• Job descriptions• Specific laws or regulations• Institution’s latest annual report
Typical Document Requests
9/12/2017
15
Quality Assurance & Improvement Program• Formally documented, consistent with Standards (1310).
• Periodic self‐assessment (1311)
• External assessments (1312)
• Results communicated to board, senior management (1320)
• Use of “conforms” statement (1321).
• Disclosure of nonconformance (1322).
• Documentation of QAIP
• Results of internal and external assessments
• Actions to improve internal audit
• Reports to board, minutes of meetings
Evidence of Conformance
9/12/2017
16
• Documentation of QIAP policy/procedure
• Reports to the board, minutes of meetings
• Internal audit values and customer service standards
• Performance measures
• Documentation of periodic internal assessments
• Report of external assessors
Typical Document Requests
How frequently is IA charter reviewed? Most recent review?
Does CAE confirm independence at least annually?
Are there impairments to independence or objectivity?
Is there an expectation of conformance?
Example Planning Questions
9/12/2017
17
Are there performance metrics?
How often are internal assessments performed?
Was the external assessment discussed with the board?
Do you use the “conforms” statement?
Example Planning Questions
Determine the level of conformance with the spirit and intent of
• Standard 1200: Proficiency and Due Professional Care.
9/12/2017
18
• Staff have knowledge, skills, & other competencies to complete annual plan (1210 & 2030).
• Staff have knowledge, skills, & other competencies to complete individual engagements (1210, Code of Ethics, & 2230).
Proficiency and Due Professional Care
• Department has effective recruiting and development policies/practices (1210 & 1230).
• Staff aware of due professional care standard (1220).• Appropriate training including IT and fraud (1230).• Continuing professional development (1230).
Proficiency and Due Professional Care
9/12/2017
19
• Competency assessments of internal audit
• Annual audit plan and engagement plans showing appropriate allocation of staff
• Recruitment and training strategy, job descriptions, certifications, resumes
• Internal audit policies, procedures• Evidence policies, procedures were communicated
Evidence of Conformance
• Internal audit organization chart• Staff profile (i.e., experience, certifications)• Policy on staff development
• Staff training records• Job descriptions• Staff appraisal documents
Typical Document Requests
9/12/2017
20
Are there job descriptions, competency frameworks, models to evaluate knowledge and skills of staff?
Do audits require specialization? How do you staff audits?
Are auditors aware of requirement for due professional care?
How is staff performance reviewed?
What is your policy on continuing professional development?
Example Planning Questions
Determine the level of conformance with the spirit and intent of
• Standard 2000: Managing the Internal Audit Activity
• Standard 2100: Nature of Work
• Standard 2600: Communicating Acceptance of Risk
9/12/2017
21
• Risk‐based audit plan (2010, 2010.A1 & 2010.C1, 2120.A1, 2120.A2, 2130.A1)
• Input and expectations for plan (2010.A1 and .C1)• Expectations for overall opinions and other conclusions (2010.A2 & 2450).
• Performance Measures
Managing Internal Audit
• Resources and financial budgets are appropriate (2030 and 2020)
• Coordinated with external providers (2050)• Third‐party/external provider understanding (2070)
Managing Internal Audit
9/12/2017
22
• CAE communicates annual plan and interim changes (2020)
• Reporting includes significant risk exposures and control and governance issues (2060).
• Policies and Procedures documented and communicated (2040).
• Policies and Procedures are understood (2040).
Managing Internal Audit
• Client surveys and other feedback• Internal and external assessments
• Supervisory evaluations• Reviews of individual internal auditors• Documented internal audit plan
Evidence of Conformance
9/12/2017
23
• Board meeting agendas, minutes
• Internal audit staff meeting agendas, minutes
• Communications to audit staff
• Communications with co‐source providers
Evidence of Conformance
• Audit plan/risk assessment policy/procedure
• Current and prior year audit plans and supporting documentation
• Comparison of plan to actual engagements
• Reports to board and senior management
• Policy/procedure on communicating results
Typical Document Requests
9/12/2017
24
• Sufficient coverage (2010 & 2110)• Governance process (2110)• Ethics‐related objectives, programs & activities (2110)• Information Technology (2010, 2110, 2120, & 2130)• Risk management processes (2010 & 2110)• Control processes (2130)
• Scope of engagements (2120.A1, 2130.A1, & 2201)
Nature of Work
• Documentation of internal audit roles and responsibilities related to governance, risk management, and control.
• Internal audit risk assessment and plan
• Engagement risk assessments and plans
• Minutes of meetings
Evidence of Conformance
9/12/2017
25
• Institution’s governance structure and policies• Institution’s risk management framework, policies
• Institution’s control policy• Reports to audit committee and senior management
• Risk assessment
• Work papers
Typical Document Requests
• CAE must discuss unacceptable risk with management
• If not resolved, CAE must communicate to board
Communicating Acceptance of Risks
9/12/2017
26
• Minutes of meetings
• Memos to file
• Status reports showing disposition of recommendations
Evidence of Conformance
• Recent status reports showing disposition of recommendations
Typical Document Requests
9/12/2017
27
What is your process for developing a risk‐based audit plan?
How do you assure completeness of audit universe?
Describe periodic reporting to senior management and board.
How do you develop, maintain, and implement IA policy /procedure manual?
Example Planning Questions
If you co‐source, describe the process.
What is IA’s role in risk management?
How do you contribute to ethics and the improvement of governance, risk management, and control processes?
Example Planning Questions
9/12/2017
28
Determine the level of conformance with the spirit and intent of
• Standard 2200: Engagement Planning
• Standard 2300: Performing the Engagement
• Standard 2400: Communicating Results
• Standard 2500: Monitoring Progress
• Engagement Objectives (2210)
• Engagement Scope (2220)
• Engagement Resource Allocations (2230)
• Engagement Work Program (2240)
Engagement Planning
9/12/2017
29
• Documented engagement plan covering • planning considerations, • engagement scope and objectives,
• resource allocation, • approved work program
• Communications to engagement client
Evidence of Conformance
• Policy on engagement planning/risk assessment
• Engagement work papers
Typical Document Requests
9/12/2017
30
• Sufficient, reliable, relevant, and useful information (2310).
• Appropriate analysis and evaluation (2320).• Support for results and conclusions (2330).• Workpapers have controlled access (2330.A1, .A2, & .C1)
• Proper supervision (2340).
Performing the Engagement
• Engagement work papers that describe• actions, analyses, and evaluations• logic supporting conclusions, opinions, and/or advice• description of CAATs or software used
• Engagement work papers approved by supervisor
• Completed work paper review checklist
Evidence of Conformance
9/12/2017
31
• Internal audit policy on performing engagements
• Engagement work papers
• List of software or CAATs
Typical Document Requests
• Engagement’s objectives, scope, and results (2410).
• Applicable conclusions, recommendations (2410)
• Accurate, objective, clear, concise, constructive, complete, and timely (2420).
• Communication of Errors (2421).
Communicating Results
9/12/2017
32
• Final approved communication documents
• Policies regarding communication of noncompliance with laws, regulations, etc.
• Policies for communicating sensitive information
• Policies for communicating outside the institution
Evidence of Conformance
• Engagement work papers
• Internal audit policy on communicating results
Typical Document Requests
9/12/2017
33
• System for communicating disposition of results to senior management (2500)
• System to monitor results (2500.A1)• Ensure implementation• Management has accepted risk
• Monitor consulting as agreed (2500.C1)
Monitoring Progress
• Routinely updated exception tracking system• Corrective action status reports
Evidence of Conformance
9/12/2017
34
• Policy/procedure on monitoring progress
• Recent status reports used to monitor disposition of recommendations
Typical Document Requests
What is your mix of assurance and consulting projects?
Do you develop and document plans for each engagement?
Are engagement scopes sufficient?
Do you assign appropriate resources to each engagement?
Do work papers include all relevant information?
Example Planning Questions
9/12/2017
35
Are conclusions and results based on appropriate analysis, supported by documentation?
How are engagements supervised?
How are results communicated?
Example Planning Questions
Quality Assurance & Improvement Program
The chief audit executive must develop and maintain a quality assurance and improvement program that covers all aspects of the internal audit activity.
Maintain a QAIP
Quality operations
9/12/2017
36
Quality Assurance & Improvement Program
A QAIP is designed to enable an evaluation of internal audit’s compliance with the Standards and an evaluation of whether internal auditors apply the Code of Ethics. It assesses efficiency and effectiveness of internal audit and identifies opportunities for improvement.
Maintain a QAIP
Quality Assurance & Improvement Program
How do you ensure the quality of your audit engagements?
Do you conduct periodic internal assessments?
Do you take corrective actions when deficiencies are identified?
Maintain a QAIP
9/12/2017
37
Gap Analysis
• Compare the typical document requests to what you have; consider whether alternate documents provide the same evidence
• Rate yourselfAddress any standard rated “does not comply”; any standard rated “partially complies” represents an opportunity for improvement
Assess Readiness
Gap analysis
Low Stress External Assessment
• Internal Audit, Management, and Board are prepared
• Select an appropriate assessor
• Schedule the QAR when management and internal audit staff will be present
Conduct QAR
Low stress external assessment
9/12/2017
38
Engage an independent party to evaluate your department
Full External Assessment
Overall conformance evaluation
Individual standards rated less than generally conforms
List of all Standards and conformance levels
Identification of noteworthy strengths
Process improvement opportunities
Full External Assessment Report
9/12/2017
39
Perform your own internal assessment ... and then engage an independent party
to review your work.
Self‐Assessment with Independent Validation (SAIV)
• Content is the same as a full external assessment
• Created by the internal audit department
• External Assessor only provides a validation statement • Confirm conclusions• Disagree as appropriate
SAIV Report
9/12/2017
40
Generally ComplyPartially ComplyDoes Not Comply
Capability Maturity Model
Initial
Infrastructure
Integrated
Managed
Optimized
9/12/2017
41
• Qualified, independent, from outside your organization
• Competent in professional practice of IA
• Competent in external assessment process
• Similar size, complexity, sector, and technical issues
Choosing an Assessment Team
9/12/2017
42
The IIA
Commercial audit firms
Triangular arrangement
ACUA resources
Bilateral arrangement
How do I find a team?
How big is a team?
# internal auditors1‐23‐67‐1011+
Typical team size
1‐22‐33‐44
9/12/2017
43
How much time will the team spend onsite?
# internal auditors1‐23‐67‐1011+
Typical days on site1‐32‐43‐44+
• Fee for team
• Staff time to prepare
• Travel expenses for team members
How much does it cost?
9/12/2017
44
Your Work Plan
Start where you are
Align your practices with the Standards
Build in policies and procedures
Document your work
Standards are mandatory guidance.
They are not rigid. There is often more than one way to comply.
9/12/2017
45
No one fully complies
Almost everyone generally complies
A few partially comply
Almost no one does not comply
Relax a little
The Chief Audit Executive is responsible for communicating results of the assessment to
senior management and the board.
9/12/2017
46
You should now be able to
describe elements of an external QAR,
identify evidence that supports conformance with the Standards,
recognize supporting materials typically used by an external assessment team,
evaluate your department’s readiness for an external QAR, and
develop a work plan to prepare your department for an external QAR.
Every Five Years
Jim Sleezer, MBA
918‐853‐5114