state of the region: icbf self- assessment report 2016 · 2 afrosai-e icbf self-assessment report...

ICBF SELF-ASSESSMENT

REPORT

STATE OF THE REGION:

2016

PRESENTED AT THE AFROSAI-E GOVERNING BOARD MEETING IN NAIROBI, KENYA, MAY 2017

AFROSAI-E ICBF SELF-ASSESSMENT REPORT 20162

A WORD FROM OUR CEOWESSEL PRETORIUS

It is my great pleasure to present the 2016 State of the Region: ICBF Self-Assessment Report, formerly the Transversal Activity Report.

1.

3.

4.

The report is based on the annual questionnaire sent out to the AFROSAI-E member SAIs.

2. The report provides member-SAIs with information from an institutional perspective on the status of their development and that of public-sector auditing in the region through self-assessments.

The questionnaire and ICBF Self-Assessment report are built on the Institutional Capacity Building Framework (ICBF) developed by AFROSAI-E and adopted by its Governing Board (GB) in 2006.

The conclusions of the report are also guided by the general findings of the Quality Assurance Support visits reviews carried out at eight SAIs in 2016.

WHAT IS THE ICBF? The AFROSAI-E Institutional Capacity Development Framework (ICBF) questionnaire was developed to enable SAIs to do a self-assessment of their performance and measure their compliance with the ISSAIs. The ICBF consists of five development levels, level 1 to 5, and five institutional development areas or domains:

1. INDEPENDENCE AND LEGAL FRAMEWORK

3. HUMAN RESOURCES

4. AUDIT STANDARDS AND METHODOLOGY

5. COMMUNICATION AND STAKEHOLDER MANAGEMENT

2. ORGANISATION AND MANAGEMENT

AFROSAI-E ICBF SELF-ASSESSMENT REPORT 2016 3

ELEMENTS OF ALL FIVE DOMAINS IN THE ICBF

INDEPENDENCE AND LEGAL FRAMEWORK

ORGANISATION AND MANAGEMENT

HUMAN RESOURCES

AUDIT STANDARDS AND METHODOLOGY

COMMUNICATION AND STAKEHOLOER MANAGEMENT

INDEPENDENCE OF THE SAI:

• Appropriate and effective constitutional/statutory/ legal framework

• Financial autonomy • Managerial and

administrative autonomy

• Appropriate human, material and monetary resources

LEADERSHIP AND DIRECTION

STRATEGIC PLANNING

ANNUAL OPERATIONAL PLANNING

ORGANISATION OF THE SAI

• Organisational development

• Existence of a performance audit function

• Existence of an IS audit function

• Existence of an IT support function

HUMAN RESOURCE AND PROFESSIONAL DEVELOPMENT POLICY

Including: • Recruitment• Remuneration• Performance

management system • Career development • Training • Staff welfare • Professional

development • Job rotation • Retaining • Exit

ANNUAL AUDIT PLAN

Covering: • Assessments of

constraints • Current issues

and stakeholder expectations

• Risk assessments in place for prioritizing audit risk

• Clear statement of audit coverage

• Activity plans for regularity and performance audits

• Addressing of backlogs

COMMUNICATION POLICY AND STRATEGY COVERING INTERNAL AND EXTERNAL COMMUNICATIONS

Based on: • Legal framework• Vision, mission and

values • Stakeholder

analysis (including prioritization)

• SWOT or similar analysis

• Gap analysis considerations

CHANNELS OF COMMUNICATION

Between: • SAI and Parliament• PAC and Judiciary

INDEPENDENCE OF THE HEAD OF SAI AND MEMBERS OF COLLEGIAL INSTUTIONS:

• Security of tenure • Legal immunity in the

normal discharge of their duties

INTERNAL CONTROL SYSTEM

in line with International standards

DEVELOPMENT PLANS

Aligned with: • Strategic plan • Annual operational

plan

AUDIT MANUALS

• Aligned to international standards

• Connected to a training program

• Reviewed and updated regularly

AD HOC MEETINGS

With: • Ministry of Finance

and oversight bodies

SUFFICIENTLY BROAD MANDATE

• A broad mandate and full discretion in the discharge of SAI functions

• Direct submission of reports to Parliament

• Access to information • Discretion in selection

of audit issues • Freedom to decide

on content, timing of audit reports and to publish and disseminate them

USE OF RESOURCES

• A management information system (MIS) tracking key management information

• A time recording system to enable reporting of staff costs

MANAGEMENT OF PERSONNEL

• Recruitment • Development • Staff welfare• Performance

appraisals • Retaining • Filling of vacant posts • Exit

QUALITY CONTROL MEASURES AND QUALITY ASSURANCE

• SAI policy and procedures

• Roles and responsibilities

• Type of review specified and planned, including nature, scope and frequency

• Implementation of a quality assurance handbook or guidance for full compliance to international standards

INTERNAL COMMUNICATION INCLUDING:

• Alignment of staff to SAi’s vision, mission,goals and objectives

• Implementation of effective information sharing practices

Each domain contains a number of elements:


ELEMENTS OF ALL FIVE DOMAINS IN THE ICBF

INDEPENDENCE AND LEGAL FRAMEWORK

ORGANISATION AND MANAGEMENT

HUMAN RESOURCES


COMMUNICATION AND STAKEHOLOER MANAGEMENT

EFFECTIVE FOLLOWUP MECHANISM

at the SAI on its audit recommendations

SAI’S REPORTING ON ITS OWN ACTIVITIES AND USE OF RESOURCES

OVERSIGHT AND ACCOUNTABILITY

The Parliament or an oversight body appoints the SAI’s externaI auditors

CODE OF ETHICS

and its monitoring

TRAINING ASPECTS WITH MONITORING AND EVALUATION MECHANISMS

For:• New entrants• Management

and Leadership development

• On the job training• Secondments to other

SAIs• Audit/ accounting

quaIifications• Coaching and

mentoring process

CAPACITY TO TRAIN ITS STAFF

To:• use information• develop knowIedge• develop skilI

QUALITY ASSURANCE PROCESSES PERFORMED BY OTHERS

AUDIT IT-TECHNIQUES

Such as:• Electronic working

papers• Computer assisted

audit techniques (CAATS)

IMPLEMENTATION OF THE ENGAGEMENT COMMUNICATION STRATEGY

for the audit process with the auditees

COMMUNICATION

With:• Relevant experts• Professional bodies• Relevant journals• Internal audit• Other public sector

audit institutions

SAI REPORTING

Should include:• Follow up on previous

recommendations• Standard structure of

reports, user-friendly with materiality considerations

PROMOTION OF THE SAI

Via:• Engagement with:• Media• The public• Academic institutions• International

community and organisations

• Use of effective information sharing practices

AUDIT PERFORMANCE AND RESULTS

Such as:• Audit coverage of

expenditure• Number of signed and

issued or published performance audit reports

• Number of / percentage of performance auditors in relation to total audit staff

• Integration of IS audit in regularity and performance audit

• Coverage of IS audit• Time for submission

of the annual audit report to Parliament from the beginning of the year

• Time for implementation of the recommendations

• Key stakeholders view on the benefit of the audit

The ICBF uses the International Standards of Supreme Audit Institutions (ISSAIs) and other international and regional best practices as benchmarks against which development and performance is measured.

OBJECTIVE OF THE TOOL:

T O H E L P T H E S A I F O C U S O N H O W I T C A N I M P R O V E I T S R E S U L T S .


YEARNUMBER OF

SAIs WHO RESPONDED

TOTAL NUMBER OF

MEMBER SAIs2006 12 232007 13 232008 17 232009 22 232010 23 232011 20 232012 21 242013 24 242014 23 252015 25 262016 24 26

The development levels of the ICBF:

INSTITUTIONAL CAPACITY BUILDING FRAMEWORK (ICBF)

INSTITUTIONAL PERSPECTIVE

LevelINDEPENDENCE

AND LEGAL FRAMEWORK

ORGANISATION AND

MANAGEMENT

HUMAN RESOURCES


COMMUNICATION AND STAKEHOLDER

MANAGEMENT

5 O p t i m i s e d L e v e l

4 M a n a g e d L e v e l

3 E s t a b l i s h e d L e v e l

2 D e v e l o p i n g L e v e l

1 F o u n d i n g L e v e l

DEV

ELO

PMEN

T LE

VEL

ABOUT THE QUESTIONNAIRE

IMPORTANT NOTE: The questions used in the 2016 questionnaire have been materially adjusted in view of the following:

• The new 2015-2019 strategic imperatives

• The principles of ISSAI 12, 100 – 400 and ISSAI 30

• The need to align the questions with the criteria used in the SAI-PMF.

TIMEFRAME: This report concerns itself with the development of public sector auditing in the region for the twelve months period ending 31 December 2016.

48 Questions, outputs, outcomes and environment

IT IS NOT used to measure the SAI development levels

IT IS used to provide context and add to the narrative content of the report.

N U M B E R O F QUESTIONS: 185 R E S P O N S E

RATE 2016: 92%A D D I T I O N A LSECTION: 48

questions


Credibility of Self-Assessments In general, credibility and validity concerns are raised when addressing the issue of self-assessments. It is not within the scope of this report to discuss the merits and demerits of self-assessments. However, based on the regional history of self-assessments over a period of ten years, the region has observed significant improvements in terms of SAIs submitting objective responses that the SAIs in-turn use for adjusting and reviewing their strategic and operational plans. Nonetheless, when using this report the following factors are still important to consider:

First, the responses are based on self-assessments by member SAIs and are therefore a product of perceptions on performance. In order to increase the relevance and reliability of responses, and to ensure consistency, an abridged version of the ICBF guideline was sent together with the questionnaire to assist and guide members to carry out the self-assessment. The abridged guideline is meant to assist SAIs internalise the application of the ICBF tool and improve objectivity when assessing the level of development in a given area.

Second, in situations where there have been sudden or extreme variations compared with the previous years, this may be due to different people answering the questions over the years since inception to date. To mitigate against extreme variations, SAIs were encouraged to establish cross-functional teams to draft responses before final approval is given by the head of SAI.

Third, a simple averaging methodology is used to calculate the scoring under each domain and imperative for the period under review and this may also blur the true development level of individual SAIs and the region. To mitigate these weaknesses, results of the quality assurance support visits by the Secretariat were used to interpret and develop conclusions under each domain.

THE ICBF AND SAI-PERFORMANCE MEASUREMENT FRAMEWORK (SAI-PMF) AFROSAI-E was a member of the Performance Measurement Framework (SAI-PMF) task team, providing inputs on the final version that was endorsed during 2016 INCOSAI in Abu Dhabi.

DOMAINS OF THE SAI-PMF AND ICBF

SAI-PMF DOMAINS ICBF DOMAINS

Domain A: SAI Reporting Domain 1: Independence & Legal Framework

Domain B: Independence and Legal Framework Domain 2: Organisation & Management

Domain C: Strategy for Organisational Development Domain 3: Human Resources

Domain D: Audit Standards and Methodology Domain 4: Audit Standards & Methodology

Domain E: Management and Support StructuresDomain 5: Communication & Stakeholder Management

Domain F: Human Resources and Leadership

Domain G: Communication and Stakeholder Management

Although the ICBF and the PMF tools assign different domain names, on a general level the two frameworks use similar criteria to measure performance.

• Both tools are based on the ISSAIs and best practice.

• The development of the SAI-PMF was partly based on the ICBF work and frame of reference.

• The SAI-PMF can be done at least once every two years, whereas the ICBF questionnaire will be done annually through self-assessment.

GOING FORWARD:

The ICBF questionnaire

will complement the SAI-PMF.


AFROSAI-E DEVELOPMENT TOOLS

More than 16 SAIs have benefited from the in-country roll-outs of Management Development (MDP) courses and follow-up workshops

to customise and implement the range of development tools that are available.

BASED ON THE ICBF AFROSAI-E HAS EMBARKED ON A

STRUCTURED PROCESS TO CAPACITATE SAI MEMBERS to comply with ISSAI’s focussing on the following key activities:

Developing audit tools and training interventions

Providing support and assistance on request of members

Establishing country and regional quality assurance practices

Monitoring compliance with THE ISSAI’s

IN SUPPORT OF

THE DEVELOPMENTAL TARGETS SET BY SAIs and the Governing Board for the region as a whole, the Secretariat produces a range of capacity building tools. These include:

Guidance materials (handbooks, guidelines and manuals)

Courseware (including e-learning programmes)

Training workshops (regional and in-country)

On-site support and quality assurance reviews

THE FOLLOWING DEVELOPMENTAL TOOLS

were developed and launched during 2016:

Financial Audit manual (exposure draft)

Compliance Audit manual (exposure draft)

Performance Audit manual

SINCE 2012:

AFROSAI-E in partnership with the Swedish National Audit Office developed and delivered the Executive Leadership Development

Programme (ELDP) to six SAIs at a regional level to support the establishment of top executive teams at SAIs.


EXECUTIVE SUMMARY2016 PROGRESS REPORT

Absence of effective communication policies and tools at both the SAI business and strategic levels may also be a factor.

is significantly being reduced

IN THIS YEAR’S REPORT:

THE RESULTS generally show that the responses are stabilising.

Shows that SELF-AWARENESS OF THE SITUATION also is interpreted in a more consistent way between the years, which is good.

HOWEVER, THERE ARE STILL AREAS WHERE SAIs overrated themselves across the five domains.

When self-assessment results are compared with the results of the QA support visits reviews carried out by the SAI, there is a correlation of about 70%.

70%correlation

TENDENCY TO OVERRATE SCORES BY SAIS

as more SAIs become aware of the purpose of the annual assessments.

However, some of these gaps are of concern as they can threaten the effectiveness of SAIs.

SAIs should be encouraged to explore ways in which they can earn and maintain their independence.

Challenges were faced in the existence of SAI policies and systems versus the actual implementation thereof.

SAI PERFORMANCE GAPS AND CHALLENGES ARE BEING CLOSED AT AN INCREASING RATE WHICH IS GOOD FOR THE REGION.

Funding and skills gaps to drive the policies, strategies and deployment of the tools.

OTHER CHALLENGES:

IMPLICATIONS FOR SAIs AND STAKEHOLDERSThe report observations raise several different capacity building questions for the attention of SAI leadership teams. The implication for stakeholders and SAIs may be in the area of capacity development opportunities that could be exploited both at the SAI and regional levels. In exploiting these opportunities the principles enshrined in ‘The Values and Benefits of SAIs’ (ISSAI 12) are instructive. These opportunities exist across all the five domains of the ICBF as demonstrated in some of the following main issues:

1. INDEPENDENCE AND LEGAL FRAMEWORK: The identified gaps and challenges in operationalising SAI independence presents opportunities for SAIs to develop tools and capacities necessary for lobbying the legislature and the executive to create an enabling environment necessary for the SAI to effectively execute its mandate. Within this domain the development and implementation of effective follow-up mechanisms by SAIs at the institutional level is also relevant as a means to demonstrate value and benefits to citizens.


2. ORGANISATION & MANAGEMENT: Building capacity of SAIs during this period of uncertainties, global financial crises, disruptive political change and demands by stakeholders, and operating environments that are being shaped by continuous technological changes and advancements; calls for adaptable and visionary leadership to show the way. Further, in view of the organisational principles and requirements of ISSAI 12 and the newly endorsed ISSAI 30 SAI leadership are presented with opportunities to establish balanced integrity control systems within their organisations as a means of promoting preventive approaches to the management of integrity and ethical challenges at the work place.

3. HUMAN RESOURCES: The challenges identified in this domain also present opportunities for SAIs to leverage into the professionalisation and competency framework development initiatives being championed by the INTOSAI Capacity Building Committee (CBC) as well as the Secretariat. Within these initiatives the development of critical leadership and management that would inspire and drive the SAI change agenda is fundamental. The advantage is that the goal for becoming model organisations can then be achieved in an efficient and effective manner. Further, the development and implementation of HR policies and strategies aligned to the SAI strategic goals and objectives is also relevant.

4. AUDIT STANDARDS & METHODOLOGY: The gaps and challenges identified in this domain can provide various training and development opportunities for member-SAIs to acquire relevant auditing knowledge and skills in financial, performance, compliance, IT auditing and other thematic auditing disciplines. Critical though, is the need for SAIs to prioritise and address the IT/IS auditing capacity gaps created by automation of the public financial management systems and processes in the region. Automation of the SAI audit processes is also relevant as a means of improving audit efficiency.

5. COMMUNICATION AND STAKEHOLDER MANAGEMENT: Challenges indicate the need for member SAIs to continue prioritising the value of both internal and external stakeholders. As such the development and implementation of stakeholder engagement tools, policies and strategies that can be customised to specific SAI environment(s) is critical. In this era of financial funding constraints SAIs can also partner with development partners at both the bilateral and regional levels as a way to leverage into external expertise and funding. Another communication tool that can enhance the value and benefits of SAIs is in the development and publication of SAI own performance reports.

SUMMARY OF FINDINGS2016 was the second year of AFROSAI-E’s five-year strategic plan period spanning from 2015 to 2019.

PAST 11 years:

Better awareness by the SAIs on the use and purpose of the ICBF self-assessment questionnaire – i.e. process of self-adjustment.

GENERAL IMPROVEMENT across five ICBF domains

HOWEVER IN 2016 the region experienced marginal decreases 2016

REASON?

Notwithstanding, the impact of the increased capacity building interventions in the region can better be observed at the element levels of the ICBF domains.

Capacity gaps identified development and implementation of ISSAI compliant audit tools and practical usage by auditors and SAIs. There is still absence of effective monitoring and evaluation systems and processes within the majority of the member-SAIs.


CHALLENGE encountered by SAIs: Implementation of developed policies, the various audit tools, systems and processes necessary for the improvement of the quality of audit services.

IN THIS REPORT the implementation challenges have been attributed to:

Leadership challenges

Funding challenges

Technical skill gaps

Lack of strategies and strategic implementation

Absence of monitoring systems

Absence of effective strategic communication

SECONDLY absence of effective communication at both the business and strategic levels may also be a factor in the challenges being faced by SAIs. E.g. need for stakeholder engagement by SAIs as a means for improving the quality of their audit services.

1. Independence and legal framework domain – Identified gaps and challenges in operationalising SAI independence presents opportunities for SAIs to develop tools and capacities necessary for lobbying the legislature and the executive to create an enabling environment necessary for the SAI to effectively execute its mandate.

2. Organisational principles and requirements of ISSAI 12 – Presented with opportunities to establish balanced integrity control systems within their organisations as a means of promoting preventive approaches to the management of integrity and ethical challenges at the work place.

3. Professionalisation and competency framework – Opportunities to leverage into the development initiatives being championed by the INTOSAI Capacity Building Committee (CBC) as well as the Secretariat.

4. Implementation of ISSAIs – Possibilities for partnerships and collaboration initiatives can be explored by SAIs to enable their staff to acquire requisite auditing knowledge and skills in the disciplines of financial, performance, compliance, IT auditing and other relevant thematic topics.

5. Communication gaps and challenges – SAIs need to continue prioritising the value of both internal and external stakeholders. As such the development and implementation of stakeholder engagement tools, policies and strategies that can be customised to specific SAI environment(s) is critical.

ICBF DOMAIN AVERAGE SCORESThe self-assessment on the domains ranges from level 0-5. The regional target is to achieve level 3 and above of the ICBF.

0 1 2 3 4 5

2014 2015 2016

3,14

2,77

2,52

2,73

2,45

ICBF DOMAINS

Independence & Legal Framework

Organisation & Management

Human Resources

Audit Standards & Methodology

Communication & Stakeholder Management


Although in specific areas the region experienced marginal improvements on average there were marginal decreases across the domains.

1. Organisation & Management marginally remained constant at 2.77 when compared with the 2015 scores

2. Independence and Legal Framework marginally dropped from 3.17 (2015) to 3.14 in 2016

3. Audit Standards and Methodology decreased from 2.82 (2015) to 2.73 in 2016

4. Human Resources shaded off ten fractions from 2.62 (2015) to 2.52 in 2016

5. Communication and Stakeholder Management moved from 2.55 (2015) to 2.45 in 2016

A significant number of SAIs commented that their scores were reduced because of the involvement of the executive in setting their budget ceilings as well as budgetary release delays that were experienced in 2016.

NOTE:

OVERALL: The independence and legal framework domain achieved the targeted level 3 of the ICBF since 2013

and has been improving.

Remaining domains are yet to achieve the targeted level of development.

Interestingly, the development levels are contextualised at the SAI levels. This is more revealing when the development levels of each SAI are aggregated against the five domains of the ICBF. Appendix 1 shows the development levels of each SAI in accordance with the ICBF. Within this framework, the responsibility of prioritising and driving the SAI to higher development levels lies with the leadership of each SAI within the constraints of their specific political, economic, social and legal environment(s)

At regional level capacity building support is provided to SAIs in form of: bilateral support visits

training workshops

development of audit guidance materials

contributions towards efficient and effective ISSAI implementation

CHALLENGE: For SAIs to use such information and materials developed at the regional level to identify gaps within the framework of their individual resource constrains as inputs that would ensure the deliverance of value and benefits to their societies.


PERFORMANCE ON DOMAINS FROM 2014 TO 2016Targeted level of development: 40% of the SAIs to retain level 3 and above

Comparison between the number of SAIs on level 3 and 4 in the years 2013, 2014, 2015 and 2016 for the 26 countries included (2015 scores were used for Angola, and Somalia who did not submit responses)

NUMBER OF SAIs IN THE REGION ON LEVEL 3 AND 4 IN THE YEARS 2013, 2014, 2015 AND 2016

Year & ICBF Level of Development

Year 2013 Year 2014 Year 2015 Year 2016

LEVEL 3

LEVEL 4

LEVEL 3

LEVEL 4

LEVEL 3

LEVEL 4

LEVEL 3

LEVEL 4

INDEPENDENCE 12 0 15 0 15 0 16 0

ORGANISATION & MANAGEMENT 9 0 6 0 6 0 7 0

HUMAN RESOURCES 7 0 6 0 3 0 6 1

AUDIT STANDARDS & METHODOLOGY 6 0 4 0 10 0 7 0

COMMUNICATION & STAKEHOLDER MANAGEMENT

4 0 4 0 5 0 4 0

1. A majority of SAIs have reached level 3 in the independence and Legal Framework domain. 2. Observable improvements in the number of SAIs who reported that they were now at level 3 on

Organisation and Management domain.

3. The Human Resources domain experienced an increase from 3 SAIs in 2015 to 7 (26.9%) SAIs 2016 at level 3.

4. Audit Standards and Methodology experienced a marginal decrease in the number of SAIs that scored aggregates of level 3.

5. Within the Communications & Stakeholders Management domain, the number of SAIs scoring level 3 remained adjusted downwards from 5 SAIs in 2015 to 4 SAIs in 2016.

AUDIT RESULTS

ISSAI 12 principle 4 requires SAIs to publicly report audit results in order to enable the public to hold government and public-sector entities accountable. Effective communication of audit results delivers value to SAI stakeholders.

All SAIs (100% scoring level 3 and above) have freedom to decide on the content and timing of audit reports as well as publish and distribute them.

HOWEVER only 75% of the SAIs indicated that they have direct reporting to Parliament.


REPORT SUBMISSIONS

In 2016 the 24 SAIs who responded indicated that 86 performance audit reports were completed and signed off compared with 113 in 2015.

To date a total of 1189 performance audit reports have been produced in the region since performance audit was introduced.

The production time period for performance audits ranged from 3 to more than 24 months.

The average production time for performance audits was 11.95 months compared with 10.9 months in 2015 as calculated from the time of starting a pre-study to publication.

0%

Level 2 and below Level 3 and below

HOW SAIs ARE COMMUNICATING AUDIT RESULTS

54%

46%

33%67%

29%

71%

38%63%

21%79%

46%54%

17%83%

42%58%

25%75%

100%

50% 100%

The SAI submits the annual audit report to the Parliament/Executive within six months following the end of the financial/fiscal years.

The SAI has implemented ad hoc meetings with oversight bodies (Such as Parliament, PAC, other committees, anticorruption institutions).

The SAI has established communication channels to strengthen relations with PAC.

The SAI has established communication channels to strengthen relations with Parliament.

The SAI has implemented a standard structure for reports that is user friendly with materiality considerations.

The SAI reports to Parliament on the implementation status of the PAC recommendations.

The SAI has implemented a follow-up of previous recommendations to be included in the present audit reporting.

The SAI submits an annual report on its own performance to Parliament and other key stakeholders.

The Head of SAI and his/her staff (the SAI) has direct submission of reports to Parliament.

The SAI has freedom to decide on content and timing of audit reports and to publish and disseminate them.


SUBMISSION OF REPORTS

THE ANNUAL AUDIT REPORT IS TABLED IN PARLIAMENT

Later than 24 months

Between 12 and 24 months

Within 8 months of year end



4%

0% 20% 60%40%

28%

48%

16%

4%

MEASURING SAI PERFORMANCE

SAIs as model organisations need to continuously optimise their organisations to enhance their performance. According to principle 6 of ISSAI 20-Principles of Transparency and Accountability, SAIs are expected to:

Assess and report on their performance in all areas of their operations on an annual basis.

Follow up their public visibility, outcomes and impact through external feedback.

Assess and publicly report on their operations and performance in all areas being covered by their mandate.

Level 3 and below Level 2 and below

13%

88%

The SAI annually reports on the percentage of key stakeholders who consider that performance audit adds value to the stakeholders and the country.

The SAI annually reports on the percentage of key stakeholders who consider that regularity audit adds value to the stakeholders and the country.

The SAI is monitoring the implementation of the communication policy/strategy and reporting on the outcomes in the annual performance report.

The SAI has implemented an overall audit planning process covering all types of audits that is aligned to its operational and strategic planning objectives.

The SAI has implemented a system to monitor and evaluate all the different aspects of the HR development strategy and policies the results of which are included in its annual performance report.

The SAI has established a system for annually carrying out reviews of its internal control environment and publishes the results of such reviews in the annual performance plan.

The SAI has implemented a system for monitoring and evaluation of the annual operational plan.

The SAI has developed and implemented a monitoring and evaluation mechanism to provide input on the achievement of strategic goals and objectives as well as input into the next strategic planning phase.

The SAI submits an annual report on its own performance to Parliament and other key stakeholders.

13%

88%

33%

67%

75%

25%

75%

25%

54%

46%

75%

25%

75%

25%

58%

42%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

MONITORING AND REPORTING ON SAI PERFORMANCE



NEXT: PERFORMANCE OF THE ICBF DOMAINS


DOMAIN 1: INDEPENDENCE AND LEGAL FRAMEWORK DOMAIN ABOUT THE DOMAIN: This domain covers the demands on the independence of the SAI as formulated in the Lima Declaration (ISSAI 1) and the Mexico Declaration (ISSAI 10). It is concerned with the existence and mandate of the SAI as to whether it is appropriately and broadly prescribed in the constitution or a comparable legal framework that may not be amended without a two thirds legislature majority vote and/or in line with the national laws. It seeks to assess whether the SAI has operational, financial and administrative independence stipulated in the law(s) as well as whether the independence of the head of the SAI and staff is guaranteed in the same laws.

CRITERIA: Constitutional and legal framework

Financial/managerial/administrative autonomy and appropriate human resources

Financial and material resources

Independence of Head of SAI and staff of SAI

PERFORMANCE FOR THE YEAR UNDER REVIEW

2016 SAI Comparison on Independence & Legal Framework5.00

4.50

4.00

3.50

3.00

2.50

2.00

1.50

1.00

0.50

0.00

Leve

l

Ango

la

Som

alia

Nig

eria

Seyc

helle

s

Nam

ibia

Ethi

opia

Leso

tho

The

Gam

bia

Sout

h Su

dan

Keny

a

Tanz

ania

Zam

bia

Moz

ambi

que

Sout

h Af

rica

Mau

ritius

Eritr

ea

Bots

wan

a

Uga

nda

Mal

awi

Gha

na

Libe

ria

Zim

babw

e

Swaz

iland

Suda

n

Rwan

da

Sier

ra L

eone

SAI

NUMBER OF SAIs: 24

Most countries have achieved or are about to achieve level 3. For instance, 62% (16) SAIs indicated achievement of level 3 and above which was a similar number of SAIs in 2015.

More than 88% (23) of SAIs are at level 2.5 and above of the ICBF. While, on aggregate SAIs are progressing satisfactorily towards the targeted level 3. There is still work that SAIs need to do to ensure achievement of higher level of development.

While the majority of SAIs now have legal frameworks that provides them with a fair amount of independence, operationalising independence in practice is a challenge many of them are encountering.


46% of SAI budgets still require executive approval and release.

Only 54% of the SAIs indicated that the executive does not take part in the preparations and approval of their budgets.

In the event of the SAIs requiring additional funding, only 33% can approach Parliament directly to submit their requests.

67% of the SAIs compared still have no direct leeway to approach Parliament for approval of additional funding request.

50% of the SAIs had their own external auditors appointed by Parliament, the executive or other appropriate bodies.

According to ISSAI 10 and 11, SAIs are expected to have autonomy in appointing, remunerating, promoting and disciplining their own staff. A majority of SAIs (75%) scored level 3 and above when it comes to the determination of their own organisation.

DOMAIN 2: ORGANISATION & MANAGEMENTABOUT THE DOMAIN: This domain is based on the principles of ISSAI 12, 20, 30, 40, INTOSAI Govs. 9100 and 9130 as well as best practices. The domain covers organisational and management tools such as plans and how they should be used efficiently and effectively. The domain also covers how managers should behave as leaders, and important issues concerning how the SAI is run and organised. The key aspects of the domain are based on organisational planning processes and their implementation, thus integrating both strategic and operational levels of the organisation.

KEY ASSESMENT ISSUES:

Head of SAI and senior management: Demonstrate visionary, exemplary and strategic leadership based on ethical and professional behaviour(s).

Organisational level: Whether the SAI has efficient and effective strategic and operational planning systems anchored in robust monitoring and reporting mechanisms.

To promote credibility and integrity the SAI expected to develop and implement a Code of Ethics and establish effective integrity management control systems.


2016 SAI Comparison on Organisation & Management5.00

4.50

4.00

3.50

3.00

2.50

2.00

1.50

1.00

0.50

0.00

Leve

l

Ango

la

Som

alia

Nig

eria

Seyc

helle

s

Nam

ibia

Ethi

opia

Leso

tho

The

Gam

bia

Sout

h Su

dan

Keny

a

Tanz

ania

Zam

bia

Moz

ambi

que

Sout

h Af

rica

Mau

ritius

Eritr

ea

Bots

wan

a

Uga

nda

Mal

awi

Gha

na

Libe

ria

Zim

babw

e

Swaz

iland

Suda

n

Rwan

da

Sier

ra L

eone

SAI


6 SAIs are still at level 2.5 and below. The majority of these SAIs fall under the fragile state category of states. Need to develop and deliver capacity building interventions to support this group to effectively execute their mandates.

Significant number of SAIs achieved the established level and above scoring 27% (7 SAIs) compared with 25% (6 SAIs) in 2015. This is two percentage point improvement when compared with 2015.

81% (21 out of 26) of the SAIs compared with 75% (18) of the SAIs scored averages of level 2.5 and above.

92% of SAIs scored level 3 and above on the development and implementation of strategic plans.

While 88% (scoring level 3 and above) of SAIs responded that they had a code of ethics that is aligned to the standards, the application thereof is still facing challenges.

Based on ISSAI 12 and 20 SAIs are required to employ sound management practices, including appropriate internal controls over their financial management and operations.

79% of SAIs have approved organisations structures, scoring level 3

75% of SAIs scored level 3 and below when it comes to operationalising the same documents.

A majority of SAI indicated that they have no documented management information systems (60% of SAIs scored level 2 and below).

18 out of 24 SAIs who responded scored level 3 and above – i.e. their performance audit units are made up of at least ten performance auditors per SAI.

54% of the SAIs have established an IT support function whose responsibilities include hardware, software and network support – at least one IT person per 30 staff members.

54% (scoring level 3 and above) indicated that they had developed and implemented an ICT strategy that is aligned to the SAI strategy.

While SAIs have documented policies, structures and systems in place there is still need for more work on improving the content and process quality of these documents, implementation of the same as well as the establishment of monitoring mechanisms to enhance the effectiveness of the policy tools. Encouraging though is that SAIs appear to be eager to develop the policies, systems and processes necessary for the efficient and effective organisation of their work.

KEY ISSUES THAT ATTRACT ATTENTION UNDER THIS DOMAIN:

Strategic and operational planning and implementation

Integrity management control systems

Establishment of performance audit and IT functions

DOMAIN 3: HUMAN RESOURCESABOUT THE DOMAIN: The domain covers elements of importance for the management and development of the SAI managers and staff. Specifically, the domain comprises aspects that include a Human Resource and professional development policy, development plans like competency frameworks, organisational development policies, management of personnel, training aspects with monitoring and evaluation mechanism, and capacity to train staff. Critical is the need to monitor, evaluate and report on the effectiveness of HR policies and procedures within the SAI.

KEY ISSUES TO BE ADDRESSED:

Recruitment

Professional development

Promotion

Retention

GOING FORWARD:


Personnel with suitable qualifications

Developing and training employees


2016 SAI Comparison on Human Resources5.00

4.50

4.00

3.50

3.00

2.50

2.00

1.50

1.00

0.50

0.00

Leve

l

Ango

la

Som

alia

Nig

eria

Seyc

helle

s

Nam

ibia

Ethi

opia

Leso

tho

The

Gam

bia

Sout

h Su

dan

Keny

a

Tanz

ania

Zam

bia

Moz

ambi

que

Sout

h Af

rica

Mau

ritius

Eritr

ea

Bots

wan

a

Uga

nda

Mal

awi

Gha

na

Libe

ria

Zim

babw

e

Swaz

iland

Suda

n

Rwan

da

Sier

ra L

eone

SAI

The Human Resources domain did not experience much progress during the year – only 7 out of 26 SAIs are at level 3 and above, i.e. only 27% of the SAIs in the region.

Several SAIs recorded an increase in the number of qualified human resources practitioners to man human resources functions over the last few years.

15 SAIs averaging scores of level 2.5 and above which is 58% of the SAIs in the region.

A great number of SAIs do not have fully operational human resources management functions established – e.g. 37.5% indicated that they do not have fully established human resources management functions.

Only 33.3% of the SAIs have developed and implemented overarching integrated HR strategies to support the implementation of their strategic goals and objectives.

41% of the SAIs scored level 2 and below, indicating that the HR plans are not aligned to the strategic and operational plans objectives.

75% SAIs scored level of 2 and below indicating that the SAI had not adopted/developed and implemented an Integrated Competency Framework for Audit and Non-Audit professionals.

In the establishment of the establishment of Training and Development policies and Performance management policies, 70.8% and 62.5% are on level three and above respectively.

Note: the majority of SAIs are still under the public service and such policies are not the responsibilities of the SAI.

A significant number of SAIs (37.5% scoring level 2 and below) indicated absence of approved performance management systems.

79% of the SAIs scored level 3 and above indicating that they had adequate capacities to train and develop their own staff as well as assist them to obtain professional qualifications in relevant disciplines.

A sizeable number of SAIs acknowledge the need to improve the documentation and delivery of on-the-job training programmes for all staff.

A majority number of SAIs (59% scoring level 2 and below) indicated that they do not have their own established management and or leadership development programme(s).

A majority of SAIs (79% with a score of level 3 and above) indicated that they established professionalisation training programmes, however mostly made possible through the support of donors and development partners.


Small number of SAIs on level three and above can be partly explained by: The absence of administrative independence in most SAIs.

Majority of SAIs are still under the public service and as such the policy development and implementation processes can be slow and bureaucratic.

The absence of financial independence in a majority of SAIs tend to constrain the recruitment, training and retention of qualified and experienced staff.

DOMAIN 4: AUDIT STANDARDS & METHODOLOGYABOUT THE DOMAIN: The domain covers the audit process(es) from the planning stage to reporting. Certain elements in this domain are closely linked to other domains such as Human Resource and Communication and Stakeholder management. Key ISSAIs under this domain include ISSAI 100 to 400 as well as audit process guidance standards at level 4 of the ISSAI framework. In this domain SAIs are expected to develop, implement and maintain appropriate up-to-date ISSAI compliant audit tools like manuals, guidelines, and templates in accordance with their mandate and national legislative requirements. The implementation of these same tools should be facilitated by robust policies, systems and processes that recognise the importance of quality delivery of audit services in accordance with the requirements of ISSAI 10, 12, 20, 30 and 40.

KEY ELEMENTS:

SAIs development and implementation of overall annual audit plans, development, implementation and maintenance of audit manuals.

Audit tools expected to be linked training programs that are reviewed and updated regularly.

Quality control measures and quality assurance, audit techniques such as electronic working papers and Computer Assisted Audit Techniques (CAATS), during the audit process.


2016 SAI Comparison on Audit Standards & Methodology5.00

4.50

4.00

3.50

3.00

2.50

2.00

1.50

1.00

0.50

0.00

Leve

l

Ango

la

Som

alia

Nig

eria

Seyc

helle

s

Nam

ibia

Ethi

opia

Leso

tho

The

Gam

bia

Sout

h Su

dan

Keny

a

Tanz

ania

Zam

bia

Moz

ambi

que

Sout

h Af

rica

Mau

ritius

Eritr

ea

Bots

wan

a

Uga

nda

Mal

awi

Gha

na

Libe

ria

Zim

babw

e

Swaz

iland

Suda

n

Rwan

da

Sier

ra L

eone

SAI

GOING FORWARD:

With several reviews of SAI legal

frameworks now recognising the

independence of SAIs, it is expected that in the coming years this domain

may experience improved scoring

levels.


NUMBER OF SAIs: 24

11 SAIs (42%) achieved averages of below level 2.5. The majority of these eleven SAIs fitted the mode of medium to small sizes.

Small to medium SAIs including as well those from fragile states within the region are confronted with capacity and funding challenges. Consequently, more technical and funding assistance to these SAIs are required.

While in 2015 10 SAIs (40%) scored averages of level 3 and above under this domain. In 2016 this number decreased to 7 SAIs (27% scoring level 3 and above).

If level 2.5 scores are included there is a decrease by two SAIs to 16 SAIs or 62% of SAIs at level 2,5 and above compared with 18 (72%) SAIs in 2015.

Good improvement in the development and customisation of the Regularity Audit Manual (RAM) and the Performance Audit Manual (PAM) by SAIs.

82% indicated that they had annual overall audit plans for all types of audits that cover adequate assessments of constraints and risk profiling of audit clients.

33% indicated that they are yet to fully align the allocation of the available resources based on the results of the risk assessments.

In 2016 from the 24 SAIs who responded to the questionnaire a total of about 10354 regularity auditors was recorded compared to 9483 in 2015.

During the year 2016 a total of 9866 regularity audits were completed out of a planned or mandated total of 14291 audits.

Of these planned or mandated audits about 4100 audits could not be completed in the prescribed timeframes due to late submission of accounts and resource constraints.

92% of the SAIs indicated that their RAMs are ISSAI compliant.

75% indicated scores of level three and above when it comes to the need for clear documented definitions of roles and responsibilities of all team members, team leaders, etc.

71% indicated scores of level 3 and above when it comes to the need for clear documented specifications of requirements for types and levels of reviews including their nature, scope and frequency.

A significant number of SAIs were yet to establish quality assurance functions to monitor compliance with their policies, processes, structures and systems.

The number of auditors who are employed on a full-time basis as performance auditors in 2016 increased to 556 compared with 526 in 2015 and 494 in the year 2014.

A majority of SAIs have explicit legal mandates to carry out performance audits, with 87.5% of SAIs scoring level 3 and above.

The majority of the SAIs (54% scoring level 3 and above) indicated that they established adequate quality control management systems for performance auditing.

At the end of 2016 there were 261 IT auditors in the region as compared to 243 in 2015. However, 149 of the IT auditors in the region are in South Africa.

52% of governments in the region use Oracle as their platform of choice followed by Epicor and SAP.

In 2016 the SAIs produced 972 IT audits (compared with 901 in 2015) of which 706 of them were from South Africa followed by Mauritius with 100 audits.

More needs to be done in terms of establishing robust quality control management systems in SAIs, training of the technical staff, monitoring the effectiveness of the same systems, computerising the audit processes as well as building the performance audit and IT audit capacities, both in terms of numbers and competencies.

GOING FORWARD:



DOMAIN 5: COMMUNICATION & STAKEHOLDER MANAGEMENTABOUT THIS DOMAIN: This domain covers internal and external communication at the SAI. The main focus of this domain is on communication elements such as the establishment of communication policy and strategy covering internal and external communication, establishing communication channels between the SAI and Parliament, PAC and the judiciary and stakeholders like Ministry of Finance and other oversight bodies, promotion of the SAI via engagement with the media, public, academic institutions and international community and organisations as well as communication of audit performance and results.


2016 SAI Comparison on Communication & Stakeholder Management5.00

4.50

4.00

3.50

3.00

2.50

2.00

1.50

1.00

0.50

0.00

Leve

l

Ango

la

Som

alia

Nig

eria

Seyc

helle

s

Nam

ibia

Ethi

opia

Leso

tho

The

Gam

bia

Sout

h Su

dan

Keny

a

Tanz

ania

Zam

bia

Moz

ambi

que

Sout

h Af

rica

Mau

ritius

Eritr

ea

Bots

wan

a

Uga

nda

Mal

awi

Gha

na

Libe

ria

Zim

babw

e

Swaz

iland

Suda

n

Rwan

da

Sier

ra L

eone

SAI

4 SAIs scored averages of level 3 and above namely; Ghana, Sierra Leone, South Africa and Tanzania.

If average scores of levels 2.5 and above are used the number increases to 15 (62.5%) as compared to 16 (64%) SAIs in the year 2015. Decrease may be attributed to challenges encountered by several SAIs to recruit, develop and retain qualified and competent communication units.

Interventions like the SAI-PAC programme, annual communication workshops and the MDP programme are important factors attributing to SAIs moving upwards to level 2.5 and above.

Challenge: keeping a sustained momentum of the development to support the majority of SAIs to move to level 3 and above.

Based on the results of quality assurance support visits there may have been instances of SAIs over scoring in some of the questions, especially in the area of internal communication and stakeholder engagement.

62% of the SAIs have developed communication policies and strategies based on the SAI’s legal framework, vision, mission and values.

50% of the SAIs scored levels 2 and below, indicating that their communication policies and strategies are yet to be fully developed and implemented.

54% of the SAIs scored level 3 and above for implementing internal communication strategies that are aligned to their vision, mission and objectives.

62.5% indicated that internal communication strategies being implemented included effective information sharing practices, websites, meetings and newsletters.

The majority of SAIs do not have systems in place to effectively monitor, gather, and report on feedback information they could receive from their stakeholders.


CONTROL & QUALITY ASSURANCEDuring the 2006 Governing Board meeting, the Heads of SAIs resolved to carry out external quality assurance support visits based on a cycle of at least two to three years per member SAI. This resulted in the development of a regional quality assurance model as an ICBF tool to assist SAIs to build the requisite capacities that would enable them to achieve higher desired levels of development. Consequently, quality control and assurance remain an area of operational focus in the region.

QUALITY CONTROL MEASURES FOR SAIs

Below level 2.5 Between level 2.5 and 2.99 Level 3 and above

60%

50%

40%

30%

20%

10%

0%

2016201520142013

Figure 2: Quality Control Levels calculated on a subset of the quality control questions in the self-assessment questionnaire. The percentage is based on 24 SAIs in 2016; 25 SAIs in 2015; 25 SAIs in 2013 and 2014, that have reached level 3 and higher, those between level 2.5 and 2.99 and below level 2.5

SELF-ASSESSED LEVELS OF SAIs COMPARED WITH THE RESULTS OF QA REPORTS 2016

COMPARISON BETWEEN THE SAIs’ SELF-ASSESSED LEVEL ON A NUMBER OF RELEVANT QUESTIONS IN THE QUESTIONNAIRE AND THE SAIs’ LEVEL ACCORDING TO THE QA REPORTS FOR YEAR 2016

QUESTION 1 2 3 4 5 6 7 8 9 10 Total %

Self-assessed level is higher than in QA report

3 2 1 2 1 2 4 3 2 3 23 29%

Self-assessed level is equal to QA report

4 6 5 4 5 6 4 5 4 5 48 60%

Self-assessed level is lower than in QA report

1 0 2 2 2 0 0 0 2 0 9 11%

8 SAIs in the region were subject to quality assurance support visits by AFROSAI-E during 2016. The table shows the difference between the maturity levels in accordance to the quality assurance reports on the one hand and the SAIs’ own self-assessed rating in the self-assessment questionnaire for 2016.


NEXT: COUNTRY LEVEL PRESENTATIONS


APPENDIX 1: COUNTRY LEVEL PRESENTATIONS 2016Appendix 1 shows the development levels in accordance with the institutional capacity building framework (ICBF) within the AFROSAI-E region. The values are based on annual self-assessment questionnaires for the three years from 2014 to 2016.

2014 2015 2016

ANGOLA



Human Resources Audit Methodology &

Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

BOTSWANA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

ERITREA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

25


2014 2015 2016

ETHIOPIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

GAMBIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

GHANA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

KENYA




Standards


5.00

4.00

3.00

2.00

1.00

0.00


2014 2015 2016

LESOTHO




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

LIBERIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

MALAWI




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

MAURITIUS




Standards


5.00

4.00

3.00

2.00

1.00

0.00


2014 2015 2016

MOZAMBIQUE




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

NAMIBIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

NIGERIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

RWANDA




Standards


5.00

4.00

3.00

2.00

1.00

0.00


2014 2015 2016

SEYCHELLES




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

SIERRA LEONE




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2015 2016

SOMALIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

SOUTH AFRICA




Standards


5.00

4.00

3.00

2.00

1.00

0.00


2014 2015 2016

SOUTH SUDAN




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

SUDAN




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

SWAZILAND




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

TANZANIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00


2014 2015 2016

ZAMBIA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014 2015 2016

UGANDA




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2015 2016

ZIMBABWE




Standards


5.00

4.00

3.00

2.00

1.00

0.00

2014

Issued August 2017

CONTACT NUMBERS & EMAILPhone: +27 (0)10 286 0104Email: [email protected]

PHYSICAL ADDRESS3rd Floor, Pegasus 2 BuildingMenlyn Maine, PretoriaSouth Africa0181

POSTAL ADDRESSSuite 20 M, Private Bag X2Woodhill, Pretoria, South Africa0076

Visit our website at www.afrosai-e.org.za

state of the region: icbf self- assessment report 2016 · 2 afrosai-e icbf self-assessment report...

Documents