Special Report The certification bodies’ new role on CAP expenditure: a positive step towards a single audit model but with significant weaknesses to be addressed

(pursuant to Article 287(4), second subparagraph, TFEU)

EN 2017 NO 07

1977 - 2017

The ECA’s special reports set out the results of its performance and compliance audits of specific budgetary areas or management topics. The ECA selects and designs these audit tasks to be of maximum impact by considering the risks to performance or compliance, the level of income or spending involved, forthcoming developments and political and public interest.

Audit team

This report was adopted by Audit Chamber I of the ECA — headed by ECA Member Phil Wynn Owen — which is responsible for the audit for sustainable use of Natural Resources. The audit was led by ECA Member João Figueiredo, supported by Paula Betencourt, private office attaché; Sylvain Lehnhard, acting principal manager; Luis Rosa, head of task and Marius Cerchez, deputy head of task. The audit team consisted of Ioan Alexandru Ilie, Jindrich Dolezal, Michal Machowski, Michael Spang, Antonella Stasia, Heike Walz. Michael Pyper assisted with drafting the report.

From left to right: Michael Pyper, Paula Betencourt, Sylvain Lehnhard, João Figueiredo, Luis Rosa, Heike Walz, Jindrich Dolezal.

2

CONTENTS

Paragraph

Glossary

Abbreviations

Executive Summary I - VII

Introduction 1 - 12

Expenditure under the Common Agricultural Policy 1 - 3

The legal and institutional framework for shared management under the CAP 4 - 7

Role and responsibilities of certification bodies 8 - 12

Audit scope and approach 13 - 21

Audit scope 13 - 17

Audit approach 18 - 21

Observations 22 - 85

The CBs’ new role in relation to the legality and regularity of CAP expenditure is a positive step towards a single audit model 22 - 28

DG AGRI’s assurance model remains based on the Member States’ control results, 29- 37

Risk assessment and sampling method 38 - 61

Basing the risk assessment on the accreditation matrix may inflate the level of assurance the CBs can derive from PAs’ internal control systems 38 - 47

The CBs’ sampling method for IACS transactions, based on the PAs’ lists of randomly selected on-the-spot checks, entailed a series of risks that were not overcome 48 - 58

A portion of the non-IACS transactions upon which the CBs perform substantive testing is not representative of expenditure in the financial year audited 59 - 61

Substantive testing 62 - 71

3

For most transactions, the guidelines do not require the CBs to perform on-the-spot testing at final beneficiary level 62 - 67

The guidelines require the CBs only to re-perform PAs’ initial checks, rather than carry out all the audit procedures the CBs consider necessary to obtain reasonable assurance 68 - 71

Conclusion and audit opinion 72 - 85

The guidelines require the CBs to calculate two different error rates and the use made of these rates by the CBs and DG AGRI is not appropriate 72 - 78

The CBs’ opinion on the legality and regularity of expenditure is based on an understated total error 79 - 85

Conclusions and recommendations 86 - 104

Annex I - The Commission’s shared management model, as presented in its 2015 AAR

Annex II - The Commission’s assurance model for legality and regularity of expenditure, as presented in its 2015 AAR

Annex III - An example of an accreditation matrix, adapted from DG AGRI’s Guideline No 3 for the certification audit of the accounts

Annex IV - The timetable of the PAs’ and CBs’ substantive testing and reporting to the Commission

The Commission’s replies

4

GLOSSARY

Accreditation: A process to certify that paying agencies have an administrative organisation

and internal control system which provide sufficient guarantees that payments are legal and

regular, and properly accounted for. Such certification is to be made by MSs on the basis of

the paying agencies’ compliance with a set of criteria (“accreditation criteria”) regarding

internal environment, control activities, information and communication, monitoring.

Adjusted error rate: This is the Commission’s estimate of the level of residual error affecting

the CAP payments by PAs to beneficiaries after all checks have been carried out. The

Commission calculates them and publishes them in its AAR.

Administrative checks: Formalised documentary checks carried out by PAs on all

applications in order to verify that they comply with the terms under which aid is granted.

For IACS expenditure, the information contained in IT databases is used for automatic cross-

checks.

Annual Activity Report (AAR): An annual report published by every Directorate-General

detailing its achievements, initiatives it has taken during the year and the resources it has

used. DG AGRI’s annual activity report also includes an assessment of the functioning and

the results of the CAP management and control systems at PA level.

Conformity clearance procedure: Commission’s procedure based on an assessment of the

PAs’ internal control systems, to ensure that Member States apply EU and national law and

that any expenditure that infringes these rules in one or more financial years is excluded

from EU financing via a financial correction.

Control statistics: Annual reports submitted by Member States to the Commission that

contain the results of the paying agencies’ administrative and on-the-spot checks.

European Agricultural Fund for Rural Development (EAFRD): Finances the Union’s financial

contribution to rural development programmes.

5

European Agricultural Guarantee Fund (EAGF): Provides funding for direct payments to

farmers, for the management of agricultural markets and for information and promotion

measures.

Financial clearance procedure: Results in Commission’s annual financial decision concerning

the completeness, accuracy and veracity of the annual accounts of each accredited PA.

Financial corrections: Commission’s exclusions from EU financing of expenditure that

Member States have not effected in conformity with the applicable EU and national law. For

CAP expenditure, the exclusions always take the form of financial corrections treated as

assigned revenue.

Integrated Administration and Control System (IACS): An integrated system consisting of

databases of holdings, applications, agricultural areas, animals and, where applicable,

payment entitlements. These databases are used for administrative cross-checks on aid

applications for area and animal related payments.

Internationally accepted audit standards (IAAS) comprise auditing standards specified by

different public and professional standard-setting bodies, such as the International

Standards on Auditing (ISAs) issued by the International Auditing and Assurance Standards

Board (IAASB) or the International Standards of Supreme Audit Institutions (ISSAI) issued by

the International Organisation of Supreme Audit Institutions (INTOSAI).

Management declaration (MD): A yearly declaration of the Directors of each PA as to the

completeness, accuracy and veracity of the accounts and the proper functioning of the

internal control systems, as well as to the legality and regularity of the underlying

transactions. Several annexes accompany the MD, among which one regarding the statistics

of the results of all the administrative and on-the-spot checks performed by the PA (“the

control statistics”).

On-the-spot checks: Checks carried out by PAs’ inspectors in order to verify that the final

beneficiaries comply with the applicable rules. These can take the form of classical on-the-

spot visits (to agricultural holdings or investments subject to aid) or of remote sensing

6

(review of recent satellite images of parcels) to be complemented with rapid field visits in

cases of doubts.

Paying agency (PA): The body(ies) responsible within a Member State for the management

and control of CAP expenditure, notably controls, calculation and payment of CAP aid to the

beneficiaries and their reporting to the Commission. Part of a PA’s work may be done by

delegated bodies, but not payments to beneficiaries and their reporting to the Commission.

Programme period: Multiannual framework for planning and implementing EU policies,. The

current programme period runs from 2014 to 2020 Rural development programmes funded

by EAFRD are managed under this multiannual framework; EAGF is managed on a yearly

basis.

7

ABBREVIATIONS

AAR Annual activity report

CA Competent Authority of the Member State (usually the Ministry of

Agriculture)

CAP Common Agricultural Policy

CB Certification body

CF Cohesion Fund

DG AGRI Directorate General for Agriculture and Rural Development

EAFRD European Agricultural Fund for Rural Development

EAGF European Agricultural Guarantee Fund

ERDF European Regional Development Fund

ERR Error rate

ESF European Social Fund

IACS Integrated Administrative and Control System

IRR ‘Incompliance’ rate

ISAs International Standards on Auditing

ISSAIs International Standards of Supreme Audit Institutions

MD Management Declaration

PA Paying Agency

8

EXECUTIVE SUMMARY

I. With a budget of 363 billion euro (in 2011 prices) for the 2014-2020 period

(around 38 % of the total amount of the multiannual financial framework 2014-2020), the

Common Agricultural Policy (CAP) is under shared management between the Commission

and the Member States (MSs). While the Commission delegates the implementation of the

budget to paying agencies (PAs) designated by MSs, it retains ultimate responsibility and is

required to ensure that the EU’s financial interests are protected as though the Commission

were performing the delegated budget implementation tasks itself.

II. For this purpose, certification bodies (CBs) appointed by MSs have been entrusted with

the role of being the PAs’ independent auditors since 1996. They have been required, since

the 2015 financial year, to also provide an opinion, prepared in accordance with

internationally accepted audit standards, stating whether the expenditure for which

reimbursement has been requested from the Commission is legal and regular. The 2015

financial year was thus the first for which the Commission could use the CBs’ enhanced work

on legality and regularity.

III. In this context, we assessed whether the framework set up by the Commission enabled

the CBs to draw an opinion on the legality and regularity of CAP expenditure in accordance

with the applicable EU regulations and internationally accepted audit standards, thus

providing reliable results for Commission’s assurance model. We conclude that the

framework designed by the Commission for the first year of implementation of the new

work of the CBs has significant weaknesses. As a result, the CBs’ opinions do not fully

comply with the applicable standards and rules in important areas.

IV. The CBs' new role is a positive step towards a single audit model because the CBs'

outputs have the potential to help MSs to strengthen their control systems, reduce audit and

control costs and enable the Commission to obtain independent additional assurance as to

the legality and regularity of expenditure.

V. Against this backdrop, we analysed the Commission’s existing assurance model and the

changes introduced to take account of the CBs’ enhanced role. We noted that the

Commission’s assurance model remains based on the Member States’ control results. For

9

the 2015 financial year, the CBs’ opinion on legality and regularity were merely one factor

taken into account when the Commission calculated its adjustments of the errors reported

in the Member States control statistics. As those opinions are the only source which provides

independent assurance on legality and regularity on an annual basis, the CBs’ work, once

done in a reliable manner, should become the key element for the Commission’s assurance.

VI. Our examination of the Commission’s guidelines compliance with the applicable

regulations and internationally accepted audit standards identified the following

weaknesses:

(a) For the risk assessment procedure, we observed that the Commission required CBs to

use the accreditation matrix which creates the risk of inflating the level of assurance

that the CBs can derive from PAs’ internal control systems;

(b) The CBs’ sampling method for transactions based on the PAs’ lists of randomly selected

on-the-spot checks entailed a series of additional risks that were not overcome: in

particular, the CBs’ work can only be representative if the samples initially selected by

the PAs are themselves representative. A portion of the sampling for non-IACS

transactions is not representative of expenditure and is thus is not representative of the

financial year audited;

(c) As regards substantive testing, the Commission required CBs, for part of the sampling,

to carry out only a re-performance of the PAs' administrative checks;

(d) The Commission required CBs for their substantive testing, only to re-perform the PAs’

initial checks. While re-performance is a valid audit collection method, internationally

accepted audit standards also state that auditors should choose and perform all audit

steps and procedures that they themselves consider appropriate in order to obtain

sufficient audit evidence to form a reasonable assurance opinion;

(e) As regards the drawing of the auditor’s conclusion, we found that the guidelines

required the CBs to calculate two different error rates in relation to legality and

regularity, and that the use made of these rates not only by the CBs, but also by DG

AGRI, was not appropriate;

10

(f) Finally, we concluded that the CBs’ opinion on the legality and regularity of expenditure

was based on an understated total error. Indeed, the errors detected and reported by

the PAs in their control statistics were not taken into account by the CBs in calculating

their estimated level of error.

VII. We make a number of recommendations to address these observations:

(a) The Commission should use the CBs’ results, when the work is defined and performed in

accordance with the applicable regulations and internationally accepted audit

standards, as the key elementof its assurance model regarding the legality and

regularity of expenditure;

(a) The Commission should revise its guidelines as follows:

(i) focus the CBs’ risk assessment as regards legality and regularity on the key and

ancillary controls already used by the Commission;

(ii) require CBs, for the selection by the CBs of IACS transactions from the list of claims

randomly selected by the PAs for on-the-spot checks, to put in place appropriate

safeguards concerning the representativeness of the CBs’ samples; the timely CBs

on-the-spot visits; the non-disclosure of the CBs’ sample to the PAs;

(iii) for the sampling of non-IACS expenditure, require the CBs to draw their samples

directly from the lists of payments;

(iv) allow the CBs to carry out: on-the-spot testing for any transaction audited, and to

carry out all audit steps and procedures that they themselves consider appropriate,

without being limited to re-performing the PAs initial checks;

(v) require the CBs to calculate only a single error rate regarding legality and

regularity;

(vi) for IACS transactions which are sampled from the list of PAs’ random on-the-spot

checks, the overall error calculated by the CBs, to be able to issue an opinion on

legality and regularity of expenditure, should also include the level of error

reported by the PAs in the control statistics, extrapolated to the remaining

11

transactions not subject to PA on-the-spot checks . The CBs have to ensure that the

control statistics compiled by the PAs are complete and accurate.

12

INTRODUCTION

Expenditure under the Common Agricultural Policy

For agriculture, EU support is granted through the European Agricultural Guarantee 1.

Fund (EAGF) and the European Agricultural Fund for Rural Development (EAFRD). The total

allocation for these two funds amounts to 363 billion euro (in 2011 prices) for the 2014-2020

programme period, which represents around 38 % of the total MFF for the period 2014-

20201

Common Agricultural Policy (CAP) expenditure under both funds can be grouped in two 2.

categories:

.

- Integrated Administrative and Control System (IACS) expenditure, which is entitlement-

based2

- non-IACS expenditure, which are reimbursement-based payments, consisting mainly of

investments in farms and rural infrastructure, as well as interventions in agricultural

markets.

and consists mainly of annual per hectare payments.;

The split between IACS and non-IACS expenditure for the two funds (around 86 % and 3.

14 % of CAP expenditure respectively) for the 2015 financial year is show in Figure 1

1 Annex I to Council Regulation (EU, Euratom) No 1311/2013 of 2 December 2013 laying down the multiannual financial framework for the years 2014-2020 (OJ L 347, 20.12.2013, p. 884).

below:

2 Where a payment is based on meeting certain conditions.

13

Figure 1 – The breakdown of CAP expenditure for the 2015 financial year (in billion euro)

The legal and institutional framework for shared management under the CAP

Responsibility for managing the CAP is shared between the Commission and the 4.

Member States. About 99 % of the CAP budget is spent under shared management, as

defined in the Financial Regulation3. The CAP Horizontal Regulation4 lays down ground rules

specifically for the financing, management and monitoring of the CAP. The Commission has

been empowered to further specify these rules through implementing regulations5

3 Article 59 of Regulation (EU, Euratom) No 966/2012 of the European Parliament and of the Council of 25 October 2012 on the financial rules applicable to the general budget of the Union and repealing Council Regulation (EC, Euratom) No 1605/2002 (OJ L 298, 26.10.2012, p. 1) (the “Financial Regulation”).

and

guidelines.

4 Regulation (EU) No 1306/2013 of the European Parliament and of the Council of 17 December 2013 on the financing, management and monitoring of the common agricultural policy and repealing Council Regulations (EEC) No 352/78, (EC) No 165/94, (EC) No 2799/98, (EC) No 814/2000, (EC) No 1290/2005 and (EC) No 485/2008 (OJ L 347, 20.12.2013, p. 549) (the “CAP Horizontal Regulation”).

5 Commission Implementing Regulation (EU) No 809/2014 of 17 July 2014 laying down rules for the application of Regulation (EU) No 1306/2013 of the European Parliament and of the Council with regard to the integrated administration and control system, rural development measures

42.2

6.3

2.7

5.4

0 10 20 30 40 50

EAGF

EAFRD

IACS

non-IACS

14

Under shared management, the Commission remains ultimately responsible for the 5.

budget, but delegates its implementation to bodies specially designated by the Member

States: the paying agencies (PAs). The Commission’s Directorate-General for Agriculture and

Rural Development (DG AGRI) oversees the PAs’ implementation of the budget, verifying and

reimbursing the expenditure declared monthly (for the EAGF) and quarterly (for the EAFRD)

by the PAs and, ultimately, assessing whether expenditure is legal and regular by means of

the conformity clearance procedure6. If this procedure reveals weaknesses in a PA’s control

system which have a financial impact on the EU budget, the Commission may apply financial

corrections to that PA. DG AGRI’s shared management model, setting out the roles of the

different parties involved, is presented in Annex I7

At Member State level, the CAP budget is managed by PAs accredited by the relevant 6.

Member State’s competent authority (CA)

.

8. The PAs perform administrative checks on all

project applications and payment claims received from beneficiaries, as well as on-the-spot

checks on a minimum sample of 5 % for the vast majority of supporting measures9

and cross compliance (OJ L 227, 31.7.2014, p. 69) and Commission Implementing Regulation (EU) No 908/2014 of 6 August 2014 laying down rules for the application of Regulation (EU) No 1306/2013 of the European Parliament and of the Council with regard to paying agencies and other bodies, financial management, clearance of accounts, rules on checks, securities and transparency (OJ L 255, 28.8.2014, p. 59).

.

Following these checks, the PAs pay the beneficiaries the amounts due and declare those

amounts monthly (EAGF)/quarterly (EAFRD) to the Commission for reimbursement. All the

amounts paid are then recorded in the PAs’ annual accounts. The director of each PA

provides the Commission with these annual accounts, along with their declaration (the

6 DG AGRI procedure based on an assessment of the PAs’ internal control systems, to ensure that Member States apply EU and national law and that any expenditure that infringes these rules in one or more financial years is excluded from EU financing (Article 52 of the CAP Horizontal Regulation).

7 CAP shared management model presented in DG AGRI 2015 AAR, p. 35.

8 As required by Article 7 of the CAP Horizontal Regulation.

9 Articles 30 to 33 of Regulation (EU) No 809/2014. For the 2014-2020 period, the Commission has introduced further possibilities for Member States to reduce the minimum control rate down to 1 %. Member States may use such possibilities as provided by Article 41 of the Regulation (EU) No 908/2014 and by Article 36 of the Regulation (EU) No 809/2014.

15

management declaration (MD)) regarding the effectiveness of their control systems, which

also summarises the levels of error stemming from their control statistics10

The Financial Regulation requires the Commission, in entrusting the PAs with budget 7.

implementation tasks for the CAP under shared management, to ensure that the EU’s

financial interests are protected to the same standard as though the Commission were

performing these tasks itself

. The

Commission’s authorising officer (DG AGRI’s Director General for the CAP) takes these

annual accounts and MDs into account in DG AGRI’s annual activity report (AAR).

11

Role and responsibilities of certification bodies

. This includes establishing appropriate control and audit

responsibilities, such as the examination and acceptance of accounts. For the CAP, the

certification body (CB) assumes these audit responsibilities at Member State level.

The CBs have been performing their role as independent auditors of the PAs since the 8.

1996 financial year12

The Financial Regulation

. The role required the CBs to issue a certificate, in accordance with

internationally accepted audit standards, stating whether the accounts to be transmitted to

the Commission were true, complete and accurate, and whether the internal control

procedures had operated satisfactorily, with particular reference to the accreditation

criteria.

13 and the CAP Horizontal Regulation14

10 Annual reports submitted by Member States to the Commission that contain the results of the paying agencies’ administrative and on-the-spot checks.

have increased the role 9.

and responsibilities of CBs. In addition to their responsibility for the accounts and internal

control, CBs have been required, since the 2015 financial year, to provide an opinion stating

11 Article 59 and recital 22 of the Financial Regulation.

12 Article 3 of Commission Regulation (EC) No 1663/95 of 7 July 1995 laying down detailed rules for the application of Council Regulation (EEC) No 729/70 regarding the procedure for the clearance of accounts of the EAGGF Guarantee Section (OJ L 158, 8.7.1995, p. 6).

13 Article 59(5) of the Financial Regulation.

14 Article 9 of the CAP Horizontal Regulation.

16

whether the expenditure for which reimbursement has been requested from the

Commission is legal and regular. This opinion, drawn up in accordance with internationally

accepted audit standards, must also state ‘whether the examination puts in doubt the

assertions made in the MD’.

The CAP horizontal regulation also stipulates that a CB must be a public or private audit 10.

body designated by the CA and that it must have the necessary technical expertise and be

operationally independent from the PA, as well as from the authority which has accredited

that PA. For the 2015 financial year, CAP expenditure in the 28 Member States was

administered by a total of 80 PAs, which were in turn audited by 64 CBs15. Forty six of these

CBs were public bodies16

The aim of the CBs’ work on legality and regularity is to increase DG AGRI’s assurance on 11.

the legality and regularity of expenditure. The CBs’ role in DG AGRI’s assurance model is

described in

and 18 were private audit companies.

Annex II

The Commission established the framework for the CBs’ audit work through an 12.

implementing regulation

. Possible uses of the CBs’ work in the context of a “single audit model”

are detailed in paragraphs 24 to 28.

17 and additional guidelines18

15 In several Member States, more than one paying agency is audited by the same certification body.

. According to this regulation, and in

order to obtain reliable reports and opinions from the CBs, the Commission needs to ensure

that:

16 Such as departments within Ministries of Finance, departments of regional governments, or Supreme Audit Institutions.

17 Articles 5 to 7 of the Regulation (EU) No 908/2014.

18 Guideline no. 1 on accreditation requirements; Guideline no. 2 on the annual certification audit; Guideline no. 3 on reporting requirements; Guideline no. 4 on the Management Declaration; and Guideline no. 5 on irregularities.

17

(a) its guidelines provide appropriate instructions in accordance with internationally

accepted audit standards19

(b) the work carried out by the CBs based on such guidance is sufficient and of appropriate

quality.

;

AUDIT SCOPE AND APPROACH

Audit scope

The new requirement for CBs to provide an opinion on the legality and regularity of the 13.

expenditure plays a key role in the Commission’s assurance model for the 2014-2020 period.

In this audit, we focused our analysis on the framework established by the Commission 14.

for the CBs’ work on legality and regularity for the first year of implementation (2015

financial year). This audit did not assess the substantive testing carried out by the CBs.

The audit aimed at assessing whether the CBs’ new role was a step towards a single 15.

audit approach and if the Commission took due account of it in its assurance model. The

audit also aimed at assessing whether the framework set up by the Commission enables the

CBs to draw an opinion on the legality and regularity of CAP expenditure in accordance with

the applicable EU regulations and internationally accepted audit standards.

In particular, we examined whether the Commission’s guidelines to CBs ensured : 16.

- an appropriate risk assessment by the CBs and a representative sampling;

- an appropriate level of substantive testing; and

- a correct estimate of the level of error and audit opinion.

After the finalisation of our audit field work (September 2016), the Commission finalised 17.

in January 2017 new guidelines to be applied by CBs from 2018 financial year. These revised

guidelines have not been subject to our review.

19 Article 9 of the CAP Horizontal Regulation.

18

Audit approach

The audit evidence was obtained at both Commission (DG AGRI) and Member State 18.

level. The starting point for the work was an analysis of the relevant legal framework, in

order to identify the requirements applicable to all parties involved: the Commission, the

CAs and the CBs.

At Commission level, the audit involved: 19.

- a review of the Commission’s guidelines for CBs applicable to the 2015 financial year, as

well as documents from supporting activities such as expert group meetings20

- an assessment of the Commission’s desk reviews of CBs’ reports and opinions for 25 PAs

in 17 Member States, which in 17 cases also included a review of DG AGRI’s reports on

audit visits to CBs. We also reviewed DG AGRI’s 2015 AAR and the use made therein of

CBs’ work on legality and regularity;

(we

participated as observers in one such meeting in Brussels on 14 and 15 June 2016);

- interviews with relevant DG AGRI staff .

At Member State level, the audit involved: 20.

- a survey of 20 CAs and CBs in 13 Member States21

20 Expert group meetings were organised by DG AGRI, with the involvement of representatives from CAs and CBs from all Member States, as a forum for sharing knowledge and experience and clarifying issues related to the CBs’ work.

, selected based on the value of the

expenditure they certified (in total, the selection accounted for 63 % of EU CAP

expenditure in 2014) and on the CB’s legal status (i.e. whether it was a public or private

entity). The topics covered by the surveys were amongst others the Commission’s

guidance and support to the CBs;

21 Belgium, Czech Republic, Germany, Ireland, Spain, France, Italy, Austria, Poland, Portugal, Romania, Slovakia and United Kingdom.

19

- six visits22

Our findings were examined against the relevant provisions in the regulations for the 21.

2014-2020 programme period and the requirements laid down in the relevant International

Standards of Supreme Audit Institutions (ISSAIs) or International Standards on Auditing

(ISAs).

to Member States, carrying out interviews with representatives from the CAs

and CBs for the purpose of better understanding the information provided in the replies

to the survey, as well as examining the CBs’ contractual arrangements, checking the

timeliness of their audit procedures and reviewing the main results of their work as

presented in their audit reports and opinions covering the 2015 financial year.

OBSERVATIONS

The CBs’ new role in relation to the legality and regularity of CAP expenditure is a positive

step towards a single audit model

While there is no single recognised definition of the term “single audit”, the concept is 22.

premised upon the need to avoid uncoordinated, overlapping controls and audits. In the

context of the EU budget, and for shared management in particular, “single audit” would

provide assurance as to the legality and regularity of expenditure on two fronts: for

management (internal control) purposes on the one hand and for audit purposes on the

other. A single audit approach should be an effective control framework in which each layer

builds on the assurance provided by others.

In the area of Cohesion23, Member State audit authorities have been, since the 2007-23.

2013 period, one of the main sources of information in the Commission’s assurance

framework for the legality and regularity of expenditure. Also in our Opinion No 2/200424

22 Germany (Bavaria), Spain (Castilla y Léon), Italy (AGEA), Poland, Romania and United Kingdom (England).

,

we analysed the “single audit” model for the purposes of internal control, i.e. as a source of

23 Cohesion policy is funded through the European Regional Development Fund (ERDF), the European Social Fund (ESF) and the Cohesion Fund (CF).

24 OJ C 107, 30.4.2004, p. 1.

20

assurance for the Commission’s management. Furthermore, in our Special Report

No 16/201325

For CAP expenditure, the Commission draws assurance that expenditure is legal and 24.

regular from three layers of information: the checks performed by the PAs, the audit work

carried out by the CBs and the results of the Commission’s checks thereon. When such

assurance is considered insufficient, the Commission exercises its ultimate responsibility for

legality and regularity through its own conformity clearance procedures which may result in

financial corrections.

, we made specific recommendations on improving the Commission’s

procedures for using the work of national audit authorities in the area of Cohesion.

The CBs’ new role of issuing an opinion on the legality and regularity of expenditure has 25.

the potential, during the 2014-2020 period, to considerably strengthen a “single audit”

model for the Commission’s management of agricultural expenditure. According to DG AGRI,

such a model will yield greater assurance that control systems are working well26 and that it

will make it possible to derive additional management assurance from the audit work carried

out by the CBs rather than from DG AGRI’s own checks27

The CBs’ audit work can enable a “single audit” model to operate efficiently for CAP 26.

expenditure:

alone.

- by helping Member States to strengthen their control systems;

- by helping to reduce audit and control costs;

- by enabling the Commission to obtain independent additional assurance as to the

legality and regularity of expenditure.

25 http://eca.europa.eu

26 In DG AGRI’s audit manual version of 4.11.2015 - section 4.4 – conformity audit on legality and regularity of expenditure for EAGF and EAFRD.

27 In DG AGRI Directorate J’s Multi-annual Working Programme 2015-2018.

21

We consider a “single audit” approach is the right way to obtain assurance as to the 27.

legality and regularity of expenditure at Member State level. Under EU law, just as the

European Court of Auditors acts as the external auditor for the EU budget as a whole, the

CBs act as operationally independent auditors of agricultural spending at Member

State/regional level. In chronological terms, they are the first auditors to provide assurance

as to the legality and regularity of agricultural spending, and they are the only source of such

assurance at national/regional28 level. CBs draw their assurance from a combination of their

own substantive testing and the PAs’ control systems. In the context of a “single audit”

model for the 2014-2020 CAP, such assurance could potentially be used by the Commission,

as indicated in Figure 2

Figure 2 – A single audit model for the CAP 2014-2020

below, provided an appropriate audit trail is ensured.

Source: European Court of Auditors.

28 Belgium, Germany, Spain, Italy and United Kingdom have decentralised the management of CAP funds through regionalised PAs.

European Commission

(EC)

Paying Agency (PA)

Certification Body (CB)

Final Beneficiaries

Syst

ems

chec

ks

Man

agem

ent

Decl

arat

ion

Administrative and on-the-spot checks

CB report and opinion

Systems audit

Substantive testing

EC review CB work

Management checks

Management/audit report Audit

Management declaration

22

We believe that the Commission, being ultimately responsible for the management of 28.

CAP funds and for the regulatory requirements in force, could make the following use of the

CBs’ work on legality and regularity within the framework of a “single audit” approach, when

this work is reliable:

- improving how it estimates residual error and establishes reservations for its annual

activity report29

- determining whether or not the estimated level of error exceeds 50 000 euro or 2 % of

the relevant expenditure

;

30

- determining more accurately and completely the amounts to be excluded from the

EU budget, making more extensive use of extrapolated financial corrections

and, hence, whether or not to start a conformity clearance

procedure;

31

- assessing whether to ask MSs to review the accreditation status of PAs in cases where

there are insufficient guarantees that payments are legal and regular

;

32

.

DG AGRI’s assurance model remains based on the Member States’ control results

The Financial Regulation33

The assurance model used for this purpose by DG AGRI up until the 2015 financial year, 30.

which determined the amounts at risk from a legality and regularity perspective, took as its

requires DG AGRI’s Director-General to prepare an AAR, 29.

declaring that he has reasonable assurance that the control procedures in place give the

necessary guarantees concerning the legality and regularity of the underlying transactions.

29 Article 66(9) of the Financial Regulation.

30 Articles 34 and 35 of Commission Implementing Regulation (EU) No 908/2014.

31 Article 52 of the CAP Horizontal Regulation.

32 Article 7(2) and (5) of the CAP Horizontal Regulation.

33 Article 66 of Regulation (EU, Euratom) No 966/2012 (the Financial Regulation).

23

starting point the control statistics reported by Member States. Then top-ups were added on

the basis of known weaknesses highlighted by DG AGRI’s conformity audits and also by our

audit work. DG AGRI needed to increase the level of error where it found that part of the

errors were not detected by the PAs and therefore not reflected in MSs’ control results. This

calculation produced an adjusted error rate for each PA and each fund.

The aggregated adjusted error rates for the 2014 and 2015 financial years show that DG 31.

AGRI continued to make adjustments on top of the levels of error reported in the PAs’

controls statistics (see Table 1

Table 1 – Results of adjustments made by DG AGRI on top of the control statistics reported

by the PAs for the 2014 and 2015 financial years

).The information from the CB’s new opinions on legality and

regularity had only a marginal effect on these adjustments for the 2015 financial year:

Fund Financial year Average level of error reported in the PAs’ control

statistics

Aggregated adjusted error rate calculated by DG

AGRI EAGF 2014 0.55 % 2.61 %

2015 0.68 % 1.47 % EAFRD 2014 1.52 % 5.09 %

2015 1.78 % 4.99 %

In its 2015 AAR34

In fact, DG AGRI’s assurance model remains based on the control results. For the 2015 33.

financial year, the CBs' opinions on legality and regularity were merely one factor taken into

DG AGRI explained that it had made “very limited” use of the CBs’ 32.

opinions on legality and regularity. This was mainly due to this being the first year in which

the CBs had produced such opinions, the timing of their work, their lack of technical skills

and legal expertise, their inadequate audit strategies and the insufficient sizes of the

samples audited. In addition to the reasons indicated by DG AGRI, we consider that, based

on our findings, the way the Commission defined the CBs’ work in its guidelines has also an

important impact on the reliability of their work.

34 For the calculation of the 2015 adjusted error rates, DG AGRI used the CBs’ results to top up the PAs error rates in only three cases for the EAGF and nine cases for the EAFRD.

24

account when the Commission calculated its adjustments of the errors reported in the MSs'

control statistics. The CBs’ opinions on legality and regularity are the only source which

provides an independent assurance on legality and regularity on an annual basis. Thus, once

the CBs’ work is done in a reliable manner this independent assurance should in our view

become the key element for the Commission’s Director-General for DG AGRI when assessing

whether expenditure is legal and regular.

The continuing focus on the control statistics stemming from the PAs management and 34.

control systems is confirmed by DG AGRI in its 2015 AAR which stated that “the opinion on

legality and regularity should confirm the level of errors in the management and control

system of the CAP that is operated in the Member States”35. DG AGRI stated further that

“through testing of transactions (based on a statistical sample), the Certification Body

auditors are requested to confirm the level of errors found in the initial eligibility checks

performed by the Paying Agency and, if not confirmed, to give a qualified opinion”36

Some important aspects of the Commission’s guidelines, such as those listed below that 35.

we assess in this report, have thus been specifically put in place to accommodate DG AGRI’s

existing assurance model based on the validation of the PAs’ control results:

.

- use of two samples, with Sample 1 drawn from the list of on-the-spot checks performed

by the PAs (see paragraphs 48 to 58);

- limitation of the CBs’ on-the-spot testing to Sample 1 transactions (see paragraphs 62 to

67);

- limitation of the CBs’ scope to re-perform the PAs’ on-the-spot and administrative

checks (see paragraphs 68 to 71);

- calculating two error rates and DG AGRI’s use of the ‘incompliance’ rate (IRR) instead of

the error rate (ERR) (see paragraphs 72 to 78);

35 Page 66 of the annexes to the DG AGRI 2015 AAR.

36 Page 67 of the annexes to the DG AGRI 2015 AAR.

25

- understatement of the error rates based on which CBs express their opinions (see

paragraphs 79 to 85).

Furthermore, in order to apply the Commission’s model, CBs need to perform a higher 36.

volume of work than what is strictly required by the CAP Horizontal Regulation. This is

because the control statistics summarised in the PAs’ management declarations are

calculated separately for the IACS and non-IACS strata (see paragraph 2). DG AGRI also

calculates the adjusted error rates for each of the two strata. This means that CBs are also

required to validate control statistics at strata level (IACS / non-IACS) and not only at fund

level (EAGF/EAFRD). In order to have sufficient evidence to validate these statistics at strata

level, CBs need to significantly increase the size of their samples compared to a scenario

where only a validation at fund level is required.

The next sections also examine the extent to which the Commission’s guidelines comply 37.

with internationally accepted audit standards at the different stages of the audit process

(risk assessment and sampling method, substantive testing, conclusion and audit opinion).

Risk assessment and sampling method

Basing the risk assessment on the accreditation matrix may inflate the level of assurance

the CBs can derive from PAs’ internal control systems

As per DG AGRI’s guidelines37, the CBs obtain their overall level of assurance on legality 38.

and regularity from two sources: their assessment of the PAs’ control environment and their

substantive testing of transactions. Such a model is in line with internationally accepted

audit standards, which prescribe that the auditor should ‘understand the control

environment and the relevant internal controls and consider whether they are likely to

ensure compliance’38

37 Section 5 of Guideline No 2 on sampling methodology.

.

38 Paragraph 53 of the International Standards of Supreme Audit Institutions (ISSAI) 400 – Fundamental principles of compliance auditing.

26

The higher the CBs rate the PAs’ internal control systems, and hence the greater the 39.

assurance they derive from these systems, the fewer transactions will undergo substantive

testing. Table 2 illustrates this inverse relationship using information from DG AGRI’s

guidelines39

Table 2 – Internal control system rating vs size of samples for substantive testing

:

Internal control system rating

Works well Works Works partially Does not work

Number of items included in substantive testing

93 111 137 181

DG AGRI required the CBs to base their rating of the PAs’ internal control systems on a 40.

matrix for assessing the PAs’ compliance with the accreditation criteria (the “accreditation

matrix”). Only accredited PAs (i.e. those fulfilling the accreditation criteria) may manage CAP

expenditure40

As

.

Annex III41

This overall score is used both by the CAs (to assess whether PAs should keep their 42.

accreditation) and by the CBs (to help them form their opinion as to whether the PAs’

internal control systems are functioning properly).

shows, the accreditation matrix consists of six functions and eight 41.

assessment criteria, making a total of 48 assessment parameters. Each individual parameter

is given a score from one (“Not working”) to four (“Works well”) and a weighting. The overall

score yielded by the accreditation matrix represents the weighted average of these

48 assessment parameters.

However, it is inappropriate to use the overall score yielded by the accreditation matrix 43.

to assess the PAs’ internal control systems in relation to legality and regularity of

39 As per the table on p. 9 of Annex 3 to Guideline No 2 on sampling methodology, for a high inherent risk and an estimated error of 10 %.

40 Article 7 of the CAP Horizontal Regulation.

41 Example adapted from DG AGRI’s Guideline No 3.

27

expenditure, as these depend mainly on only two of the 48 assessment parameters:

administrative checks and on-the-spot checks on the processing of claims, including their

validation and authorisation (under “Control activities”).

Under the framework proposed by DG AGRI, low scores on these two parameters may 44.

be compensated for by higher scores on others (e.g. information and communication,

monitoring). These other factors are not directly related to legality and regularity and should

not, therefore, be used to compensate in this way. For legality and regularity purposes, given

the overwhelming importance of these two parameters, such compensation may lead CBs to

overrate internal control systems and derive more assurance from them than is warranted.

Furthermore, this framework does not sufficiently take into account systemic weaknesses

previously reported both by DG AGRI and by us following audits of PAs and final

beneficiaries.

In three42 of the six Member States visited, we found that the CBs had not taken 45.

sufficient account of known weaknesses in their systems assessments. Box 1

Box 1 – Situations where the accreditation matrix did not produce reliable results for legality and

regularity purposes

below presents

the situation we found in one of these Member States (Germany (Bavaria)), along with the

opposite situation found in Romania, where the CB actually made full use of such known

weaknesses in its assessment. These situations support our view that the accreditation

matrix is an inappropriate tool for assessing the functioning of internal control systems for

legality and regularity.

In Germany (Bavaria), the accreditation matrix yielded a rating of “Works” for the state’s EAFRD

(non-IACS) internal control system. This rating was not consistent with the following evidence, which

suggested that the control system for such expenditure was weaker:

- for the 2013 and 2014 financial years, DG AGRI had calculated that, in relation to the legality

and regularity of expenditure, there was a material level of error, and

42 Germany (Bavaria), Italy (AGEA) and the United Kingdom (England).

28

- the CB’s own verifications showed that a significant number of transactions contained financial

errors (25 from the 2015 financial year, i.e. one third of the audited population).

In Romania, the accreditation matrix initially yielded a rating of “Works” for the internal control

systems for both funds (EAFRD and EAGF) and both strata (IACS and non-IACS). However, applying its

professional judgement using the information available, including known weaknesses previously

reported by DG AGRI and by us, the CB disregarded this outcome and reduced the corresponding

ratings to “Does not work” for EAGF (IACS) and “Works partially” for EAFRD (IACS) and EAGF (non-

IACS).

During its conformity clearance inquiries, DG AGRI uses key and ancillary controls 46.

(see Box 2

Box 2 – What are key and ancillary controls?

for definitions) to assess whether the internal control systems in place at Member

State level are capable of ensuring the legality and regularity of expenditure.

43

“Key controls shall be the administrative and on-the-spot checks necessary to determine the

eligibility of the aid and the relevant application of reductions and penalties”.

“Ancillary controls shall be all other administrative operations required to correctly process claims”.

Even though DG AGRI itself uses key and ancillary controls to assess whether PAs’ 47.

internal control systems ensure the legality and regularity of expenditure, it requires CBs to

use a different tool for the same purpose: the accreditation matrix, which is not fit for that

purpose.

The CBs’ sampling method for IACS transactions, based on the PAs’ lists of randomly

selected on-the-spot checks, entaileds a series of risks that were not overcome

As mentioned above the CAP Horizontal Regulation requires CBs to provide an opinion 48.

on the legality and regularity of the expenditure for which reimbursement has been

requested from the Commission during the financial year being audited. Therefore, when

43 Article 12 (6)(a) and (b) of Regulation (EU) No 907/2014.

29

designing an audit sample, the CBs ’shall consider the purpose of the audit procedure and

the characteristics of the population from which the sample will be drawn’44

DG AGRI guidelines

.

45

- Sample 1: drawn by CBs from the list of beneficiaries randomly

split the work for substantive testing into two samples: 49.

46

- Sample 2: drawn by CBs from all payments for the year in question. For Sample 2, CBs

need to re-perform only administrative checks, not on-the-spot checks (see

paragraphs 62 to 67).

selected by the PAs for

on-the-spot checks (see paragraph 6). For Sample 1 transactions, CBs are required to re-

perform both the on-the-spot checks carried out by the PAs and the full range of

administrative checks (receipt of the claim and eligibility checks, validation of

expenditure, including the authorisation, execution and recording in the accounts of the

corresponding payment).

The timetable of the CBs substantive testing and reporting to the Commission are 50.

described in Annex IV

Sample 1 transactions are chosen from a list of randomly selected on-the-spot checks 51.

performed by the PA on claims, which, if they are IACS, are very likely to lead to payments

during the corresponding financial year. Hence, the results of CBs’ testing not only serve the

purpose of expressing an opinion on the legality and regularity of expenditure, but also help

the CBs to form an opinion on the functioning of the PAs’ internal control systems, which is

one of the other audit objectives laid down in the CAP Horizontal Regulation.

.

The number of transactions upon which CBs re-perform all of the PA checks (see Table 3

44 Paragraph 6 of International Standard on Auditing (ISA) 530 – Audit sampling.

52.

below paragraph 64) is often considerably greater than the number of transactions usually

45 Section 6.2 of Guideline No 2 on sampling methodology.

46 The random sample represents between 20 to 25 % of PAs’ on-the-spot checks. The remaining PAs’ on-the-spot checks are sampled based on risk.

30

required for compliance testing on internal control systems47

For audit results to be reliable, it is important that all samples used for substantive 53.

testing be representative. In order to be representative, samples must be drawn from the

total population using statistically valid methods and remain unchanged. Because the CBs

extract their samples in part from the PAs’ samples, the results of their work can only be

representative if the PAs’ samples are themselves representative. Furthermore, any

replacement of transactions contained in the initial samples must be appropriately justified

and documented.

. This gives them greater

assurance that the PAs’ systems for administrative and on-the-spot checks comply with the

applicable regulations.

DG AGRI requires CBs to assess the representativeness of the PAs’ samples. However, in 54.

order to comply with such a requirement, both the PAs and the CBs need to maintain a

sufficient and reliable audit trail to confirm that the samples are representative, drawn from

the entire population and have remained unchanged.

DG AGRI requires Sample 1 IACS transactions for substantive testing to be selected 55.

based on the amounts claimed by farmers (claim-based selection) rather than the amounts

they are actually paid48

However, for this method to work as intended, close cooperation and communication 56.

between the PAs and the CBs is important. The CBs need to be updated regularly on checks

performed by the PA so that the CB can re-perform these checks shortly after the PA’s visit.

, so that the CBs can perform their re-verifications as soon as possible

after the PAs’ on-the-spot checks. The advantage of the claim-based selection is that the

conditions the CBs find when they inspect the selected farms on the spot will be very similar

to those found by the PAs.

47 Annex 3 of Guideline 2 recommends a minimum of 30 transactions for each Fund.

48 Section 6.7 of Guideline No 2 on sampling parameters.

31

We found that in three of the five Member States visited where the CB re-performed the

IACS on-the-spot checks49, this condition was not met (see Box 3

Box 3 – Delays in the CBs’ re-performance of IACS Sample 1 on-the-spot checks for the 2014 claim

year (2015 financial year)

).

In Poland, the CB completed the re-performance of 53 of the 60 EAGF IACS Sample 1 transactions

after the end of the 2014 calendar year, with some being re-performed as late as March 2015. This

was because the PA had been late in sending the CB the reports on its on-the-spot checks.

In Romania, the CB re-performed all six classical on-the-spot checks50

In Germany (Bavaria), up to three months had passed between the PA’s initial farm visits and the

CB’s on-the-spot re-verifications for EAGF IACS transactions, and up to ten months for EAFRD IACS

transactions.

for EAFRD IACS Sample 1

transactions on average six months after the PA’s checks.

Also, for this approach to be valid, the CBs need to have immediate access to the list of 57.

on-the-spot checks initially selected by the PAs, in order to ensure that this list is not

subsequently changed. Moreover, a CB must not be allowed to exclude transactions from its

sample as a result of delays in the PA’s decision granting the relevant aid. Allowing the CBs

to exclude such transactions from the sample increases the risk of PAs intentionally delaying

their decisions and payments for certain transactions, which might lead to errors in the CBs’

calculations. Such cases not only undermine the representativeness of the results and the

validity of their extrapolation to overall expenditure, but may also cause an incorrect

reduction in the error rate found by the CBs (see Box 4

49 Italy is not counted for this purpose because for the 2015 financial year the CB did not test IACS transactions on-the-spot because of its late appointment.

).

50 On-the-spot checks can take the form of classical on-the-spot visits (to agricultural holdings or investments subject to aid) or of remote sensing (review of recent satellite images of parcels).

32

Box 4 – Exclusion of transactions from IACS Sample 1 because the PA did not execute payment

until after the CB had completed its audit work

In Romania, the CB excluded five transactions from its EAFRD IACS Sample 1 after completing its

audit work because the PA had not yet executed the corresponding payments. Consequently, the CB

did not take into consideration the results of the tests on these transactions in calculating the overall

error rate upon which its opinion on the legality and regularity of transactions was based.

The application of the claim-based selection method also relies on the CBs not letting 58.

the PAs know which transactions they have selected until after the latter have already

performed their checks, so that the PAs do not know which transactions the CBs will later

scrutinise (the Commission has also recognised this risk in cases where CBs accompany the

PAs on their initial checks)51. However, we found that in Italy such safeguards were not in

place (see Box 5

Box 5 – The re-performance exercise was compromised in Italy because the CB had given the PA

advance notice of which beneficiaries would be subject to re-performance

).

In Italy, for the 2015 claim year (2016 financial year), the CB gave the PA the list of transactions it had

selected for Sample 1 IACS for both funds (EAGF and EAFRD) before the PA had carried out the

majority of its initial on-the-spot checks. As a result, the PA already knew before starting its on-the-

spot checks which transactions would subsequently be subject to re-performance by the CB.

If, when carrying out its on-the-spot checks, the PA knows which transactions are scheduled for re-

performance by the CB at a later date, it is likely to scrutinise these transactions more closely. As a

result, these checks will be more accurate than those not scheduled for re-performance, so the CB

will find fewer errors.

Moreover, since the results for these transactions will be used to extrapolate the level of error for

the entire population, these biased transactions will render the overall error rate unrepresentative

and more than likely result in it being understated.

51 Section 6.2 of Guideline No 2 on sampling methodology.

33

A portion of the non-IACS transactions upon which the CBs perform substantive testing is

not representative of expenditure in the financial year audited

For non-IACS transactions (both EAGF and EAFRD), there is a significant disparity 59.

between:

- the period for which the on-the-spot checks are reported, which is the calendar year

(from 1 January to 31 December 2014 for the 2015 financial year, in this case), and

- the period for which expenditure is paid, which for the 2015 financial year is from

16 October 2014 to 15 October 2015.

As a result, some of the beneficiaries subject to on-the-spot checks performed during 60.

the 2014 calendar year were not reimbursed in the 2015 financial year52

As a result, the number of transactions the CBs use in calculating the overall error rate is 61.

lower, thus reducing its accuracy and, in turn, affecting the reliability of their opinions on the

legality and regularity of expenditure for the financial year.

. The CBs cannot use

the results of such transactions in their calculation of the error rate for the financial year

concerned. Such results may only be used to express an opinion on the functioning of the

internal control system and provide a statement on the assertions in the MD and its

underlying control statistics.

Substantive testing

For most transactions, the guidelines do not require the CBs to perform on-the-spot

testing at final beneficiary level

According to internationally accepted audit standards, ‘in performing a reasonable 62.

assurance audit, public sector auditors gather sufficient appropriate audit evidence to

52 The beneficiaries reimbursed before 16 October 2014 were recorded in the previous financial year and those (exceptional cases) reimbursed after 15 October 2015 were recorded in the following financial year.

34

provide a basis for the auditors’ conclusions’53. They should use a variety of techniques for

this purpose, such as observation, inspection, inquiry, re-performance, confirmation and

analytical procedures. Furthermore, under EU law, the CBs’ substantive testing of

expenditure must cover verifying the legality and regularity of the underlying transactions at

final beneficiary level54

Inspection involves examining books, records and other elements (such as the 63.

beneficiaries’ right of use for the land in question, fertilisation registers, building permits,

etc.) or physical assets (such as land, animals, equipment, etc.). Inquiry involves obtaining

information and explanations from final beneficiaries, in the form of either written

statements or more informal oral discussions.

.

As mentioned above, for sample 2, the Commission’s guidelines only require CBs to re-64.

perform administrative checks (see paragraph 49). In Table 3

53 Section 7.1 of ISSAI 4200 - Compliance Audit Guidelines. Compliance Audit Related to the Audit of Financial Statements.

, we calculated the proportion

of Sample 2 transactions in the total sample for the six Member States visited.

54 Article 7(3) of Regulation (EU) No 908/2014.

35

Table 3 – Proportion of Sample 2 transactions used for substantive testing by the six Member States visited

Member State EAGF EAFRD Sample 1 Sample 2 Total

sample % of Sample 2

in total Sample 1 Sample 2 Total

sample % of Sample 2

in total (1) (2) (3) = (1)+(2) (4) = (2)/(3)*100 (5) (6) (7) = (5)+(6) (8) = (6)/(7)*100

Germany (Bavaria) 30 41 71 58 % 60 146 206 71 % Spain (Castilla y Léon) 52 123 175 70 % 50 122 172 71 % Italy (AGEA1) 30 192 222 86 % 30 91 121 75 % Poland 90 58 148 39 % 97 33 130 25 % Romania 161 292 453 64 % 140 245 385 64 % United Kingdom (England) 30 40 70 57 % 59 26 85 31 % Average proportion of Sample 2 in the Member States/Regions visited (with no on-the-spot testing)

62 % 56 %

1 The ‘Agenzia per le erogazioni in agricoltura’ (AGEA) is one of the 11 Italian PAs.

36

As the table above shows, in four of the Member States/Regions visited, Sample 2, for 65.

which no on-the-spot testing is done, accounted for the majority of the overall sample used

by CBs for substantive testing.

For sample 2, gathering evidence based only on a re-performance of administrative 66.

checks at PA level, will often not provide the CBs with sufficient and appropriate audit

evidence as required by ISSAI 4200. By doing so, the CBs are not using two audit collection

methods that are very important in the context of the CAP expenditure: inspection and

inquiry. Without these methods, important audit evidence cannot be collected, such as

evidence that farmers are using their land as stated in their declarations and proof of the

existence of physical assets purchased through investment projects. This approach therefore

falls short of internationally accepted audit standards.

Without sufficient on-the-spot testing, the CBs may not be able to obtain reasonable 67.

assurance that the relevant expenditure is legal and regular: as we have demonstrated in our

annual reports, most of the errors in CAP expenditure (particularly for IACS) are found on the

spot55

The guidelines require the CBs only to re-perform PAs’ initial checks, rather than carry out

all the audit procedures the CBs consider necessary to obtain reasonable assurance

.

‘The nature, timing and extent of procedures performed […] are determined by public 68.

sector auditors applying professional judgement’56

DG AGRI required CBs, for their substantive testing, only to re-perform (re-verify) the 69.

PAs’ initial checks, for both Sample 1 and Sample 2 transactions

. With regards to the standards applicable

when performing the reasonable assurance audit see also paragraph 62.

57

55 See, for example, paragraphs 7.19 and 7.26 of the Court’s annual report for the 2014 financial year (OJ C 373, 10.11.2015) and paragraphs 7.14, 7.18 and 7.22 of the Court’s annual report for the 2015 financial year (OJ C 375, 13.10.2016).

. For example, if the initial

56 Paragraph 21 of ISSAI 4200.

57 Section 6.2 of Guideline No 2 on sampling methodology.

37

on-the-spot checks had been done by the PAs using remote sensing58, then the CBs were

always required to re-perform them using the same method59

Re-performance is defined by internationally accepted audit standards

.

60

While re-performance is a valid audit evidence collection method, the Commission 71.

should not, in our opinion, require the CBs to limit themselves to such method for all

transactions subject to substantive testing but instead leave it up to the CBs to determine

the extent of its use. CBs might choose and perform audit steps and procedures that they

themselves consider appropriate, which should not be limited to re-performance.

Substantive testing based only on re-performance might not allow the CBs to obtain

sufficient audit evidence to form a reasonable assurance opinion.

as 70.

‘independently carrying out the same procedures already performed by the audited entity’,

in this case the PA. For example, CBs may re-perform administrative checks to test whether

the PAs made the correct decisions when granting aid.

Conclusion and audit opinion

The guidelines require the CBs to calculate two different error rates and the use made of

these rates by the CBs and DG AGRI is not appropriate

In order to comply with DG AGRI guidelines61, CBs must calculate two error rates for 72.

legality and regularity purposes (see Table 4

58 Computer-based checks and detailed photo interpretation of ortho-images performed on recent high-resolution satellite or aerial images of parcels.

):

59 Section 9.2 of Guideline No 2 on the performance of on-the-spot checks.

60 Section 7.1.5 of ISSAI 4200.

61 Glossary of Guideline No 2.

38

Table 4 - The error rates to be calculated in accordance with DG AGRI guidelines

Characteristics Error rate (ERR) ‘Incompliance’ rate (IRR) Use made by the CB To express an opinion on the

completeness, accuracy and veracity of the PA’s annual accounts, on the proper functioning of its internal control system and on the legality and regularity of the expenditure for which reimbursement has been requested from the Commission.

To assess whether their examination puts in doubt the assertions made in the MDs, including the errors reported by PAs in the control statistics.

Population to which the rate refers to

Expenditure for the financial year (e.g. 16 October 2014 to 15 October 2015).

Controls performed by the PA during the calendar year (e.g. 1 January 2014 to 31 December 2014).

PA procedures taken into account

The entire control process performed by the PA, from receipt of claims until payment and accounting.

Only PA’s primary eligibility controls (administrative and on-the-spot checks), before sanctions are applied.

Errors taken into account Only overpayments by the PA are taken into account.

It takes into account both the over- and under-validations by the PA.

Article 9 of the CAP Horizontal Regulation requires the CBs to express a “limited” 73.

(negative) assurance conclusion62

Thus, in order to be compliant with internationally accepted audit standards, the 74.

conclusion on the MD can be drawn from the results of the audit work done in relation to

the reliability of accounts, the proper functioning of the internal control system and the

legality and regularity of transactions (including the ERR), i.e. as a “by-product” of this work.

in relation to the MD. Such a conclusion does not require

fully-fledged audit work, but merely a review, usually limited to analytical procedures and

enquiries. Tests of details, such as substantive testing of transactions, are not required. A

limited assurance review therefore provides a lower level of assurance than a reasonable

assurance audit. The limited assurance conclusion is usually worded as follows: “nothing has

come to our attention that would indicate that the subject matter is not in compliance, in all

material respects, with the stated criteria”.

62 For a more detailed definition of limited assurance and how it differs from reasonable assurance, see paragraphs 20 and 21 of ISSAI 4200.

39

However, DG AGRI guidelines additionally require the CBs to calculate the IRR in order to

draw a conclusion on the MDs.

Furthermore, in its 2015 annual activity report63

The IRR does not represent the level of error in the expenditure, but rather the financial 76.

impact of weaknesses in the PA’s administrative and on-the-spot checks.

, DG AGRI explained that, in principle, it 75.

intended to use the IRR to estimate an adjusted error rate for legality and regularity

purposes. So, whereas the CBs’ opinions on the legality and regularity of expenditure are

based on the ERR, DG AGRI‘s assurance model uses a different indicator of error, the IRR, in

order to estimate the amounts at risk from a legality and regularity perspective.

DG AGRI’s guidelines require the CBs to use the ERR calculated for legality and regularity 77.

purposes to also form an opinion on the completeness, accuracy and veracity of the PAs’

annual accounts (see Table 4

However, this is contrary to the CAP Horizontal Regulation

). In its annual review of the PAs’ accounts (the “financial

clearance procedure”), DG AGRI itself also uses the error rate for legality and regularity in

order to assess the reliability of the PAs’ accounts.

64

63 Explanatory Box: Annex 10-2.1.

which, for the financial 78.

clearance procedure, clearly separates the issue of the reliability of the PAs’ accounts from

that of the legality and regularity of expenditure. This distinction is appropriate because, for

example, a payment may be properly accounted for despite not being legal and regular (e.g.

because a farmer claimed aid for land worked by another farmer); and, vice-versa, a

payment may be legal and regular despite being incorrectly accounted for (e.g. recorded as

reimbursement of an investment project instead of as annual area-based payment). Given

this separation, DG AGRI’s use of the ERR to assess the reliability of the PAs’ accounts is

inappropriate.

64 Article 51 and recital 35 of the CAP Horizontal Regulation, which states that “The decision of the clearance of accounts should cover the completeness, accuracy and veracity of the accounts but not the conformity of the expenditure with Union law”.

40

The CBs’ opinion on the legality and regularity of expenditure is based on an understated

total error

The two indicators of error (ERR and IRR) are calculated as the difference between what 79.

the CB considers eligible and what the PA has previously validated. The differences between

what beneficiaries claimed and what the PA validated after its on-the-spot checks, which

constitute the errors reported by the PAs in the control statistics, are not taken into

consideration in the CBs’ ERR.

As paragraph 6 states, as a general rule, only 5 % of claims are subject to on-the-spot 80.

checks by the PAs. And yet, for the six Member States visited, the average share of

transactions subject to PA on-the-spot checks in the overall sample audited by CBs

(see Table 3) is 38 % for EAGF and 44 % for EAFRD. Hence, such transactions are

overrepresented in the overall CB samples. Figure 3

Figure 3 – The proportion of transactions subject to PA on-the-spot checks in the EAGF

population and in the average CB sample

below illustrates this situation with the

average figures for the EAGF fund.

Source: European Court of Auditors.

Where transactions have previously been subject to on-the-spot checks, the PA has 81.

already detected and reported errors for these in the control statistics. These errors will no

longer be identified by the CBs when comparing their results to the PAs’ results. However,

EAGF population

Transactions subject to PA on-the-spot checks

Transactions subject only to PA administrative checks

Sample 1

Sample 2

41

potential errors will remain uncorrected for the 95 % of the population not subject to on-

the-spot checks.

Thus, the CBs must accurately reflect the proportion of Sample 1 transactions that have 82.

previously been subject to on-the-spot checks ensuring a true representation of the

characteristics of the total population. Therefore, they need to add to their own error rate

the PA’s error rate, from the control statistics, for its random on-the-spot checks on the

share of Sample 1 transactions in excess of 5 %65

The PA’s error rate from the control statistics likewise needs to be added to the CB’s 83.

error rate for those Sample 2 transactions which have not been subject to any on-the-spot

checks, either by the PA

that have previously undergone such

checks.

66

This adjustment is needed because CBs are required to express an opinion on the 84.

legality and regularity of the entire population of payments, and not only on the

effectiveness of the PAs’ procedures for on-the-spot checks. In the absence of such

adjustments, the ERR is likely to be significantly understated for both Sample 1 and

Sample 2.

or by the CB.

In addition to the lack of such adjustments, the weaknesses reported in paragraphs 56, 85.

57, 58, 67 and 71 of this report are also likely to result in the error rate used by CBs for their

opinion on legality and regularity being understated.

CONCLUSIONS AND RECOMMENDATIONS

With a budget of 363 billion euro (in 2011 prices) for the 2014-2020 programme period, 86.

CAP expenditure is under shared management between the Commission and the Member

States. While the Commission delegates the implementation of the budget to PAs

65 5 % is the general rule, but where PAs have verified on-the-spot a different percentage of beneficiaries, that rate should be taken into consideration instead.

66 Given, however, that Sample 2 is randomly drawn from the list of payments, 5 % of this sample can be expected to have undergone an on-the-spot check by the PA.

42

designated by Member States, it retains ultimate responsibility and is required to ensure

that the EU’s financial interests are protected to the same standard as though the

Commission were performing the delegated budget implementation tasks itself.

The Commission draws assurance that expenditure is legal and regular from three layers 87.

of information: the checks performed by the PAs, the audit work carried out by the CBs and

the results of the Commission’s checks thereon. When such assurance is considered

insufficient, the Commission exercises its ultimate responsibility for legality and regularity

through its own conformity clearance procedures which may result in financial corrections.

The CBs have been entrusted with their role as the PAs’ independent auditors since 88.

1996. While they were initially responsible for issuing a certificate on the reliability of

accounts and on internal control procedures, their responsibilities were increased for the

2014-2020 period. CBs have been required, since the 2015 financial year, to also provide an

opinion, drawn in accordance with internationally accepted audit standards, stating whether

the expenditure for which reimbursement has been requested from the Commission is legal

and regular. The 2015 financial year, then, was the first for which the Commission could use

the CBs’ enhanced work on legality and regularity in preparing its AAR.

In this context, the present report assessed whether the CBs’ new role was a step 89.

towards a single audit model and if the Commission took due account of it in its assurance

model. We also assessed whether the framework set up by the Commission enabled the CBs

to draw an opinion on the legality and regularity of CAP expenditure in accordance with the

applicable EU regulations and internationally accepted audit standards.

We conclude that although the CBs’ new role is a positive step towards a single audit 90.

model, the Commission could take very limited assurance from the CBs’ work on legality and

regularity . In addition, we conclude that the framework designed by the Commission for the

first year of implementation of the new work of the CBs has significant weaknesses. As a

result, the CBs’ opinions do not fully comply with the applicable standards and rules in

important areas.

We consider the CBs' new role in relation to the legality and regularity of expenditure to 91.

be a positive step towards a single audit model, in which the different layers of control and

43

audit can be complementary, thus avoiding uncoordinated, overlapping controls and audits.

This is because the CBs' outputs have the potential to help Member States to strengthen

their control systems, reduce audit and control costs and enable the Commission to obtain

independent additional assurance as to the legality and regularity of expenditure.

Furthermore, the CB's work on legality and regularity can also help the Commission in

improving how it estimates the overall residual error, assessing the need for conformity

audits, determining more accurately and completely the amounts to be excluded from the

EU budget, making more extensive use of extrapolated financial corrections and reviewing

the accreditation status of PAs (paragraphs 22 to 28).

Against this backdrop, we analysed the Commission’s existing assurance model and the 92.

changes introduced to take account of the CBs’ enhanced role. We noted that the

Commission’s assurance model remains based on the Member States’ controls results. For

the 2015 financial year, the CBs' opinions on legality and regularity were merely one factor

to be taken into account when the Commission calculated its adjustments of the errors

reported in the Member States' control statistics. For the financial years 2014 and 2015, the

Commission made top-ups which increased the overall residual error by between two to

four times what was reported by Member States. The CBs’ opinions on legality and regularity

are the only source which provides independent assurance on legality and regularity on an

annual basis. Thus, once the CBs’ work is done in a reliable manner this independent

assurance should in our view become the key element for the Commission’s Director-

General for DG AGRI when assessing whether expenditure is legal and regular

(paragraphs 29 to 36).

Recommendation 1 - Assurance from the CBs’ work on legality and regularity

The Commission should use the CBs’ results, when the work is defined and performed in accordance

with the applicable regulations and internationally accepted audit standards, as the key element of

its assurance model regarding the legality and regularity of expenditure.

We then examined the extent to which the Commission's guidelines complied with the 93.

applicable regulations and internationally accepted audit standards at the different stages of

the audit process. For the risk assessment procedure, we observed that the Commission

required CBs to use the accreditation matrix in order to determine the extent to which they

44

should rely on the PAs' internal control systems. The accreditation matrix is used by CBs to

assess the PAs' compliance with accreditation criteria and contains 48 assessment

parameters, out of which only two have a significant impact on the legality and regularity of

expenditure: the administrative and the on-the-spot checks on the processing of claims.

Thus, it is inappropriate to use this matrix for legality and regularity purposes, because 94.

this may increase the level of assurance the CBs derive from PAs’ internal control systems

and decrease the size of their samples for substantive testing. During its conformity

clearance inquiries, the Commission uses a different tool to assess the effectiveness of the

PAs internal control systems: a list of key and ancillary controls, which comprise the

administrative checks and physical on-the-spot checks, as well as other administrative

operations, required to ensure the correct calculation of the amounts to be paid

(paragraphs 38 to 47).

Recommendation 2 – Risk assessment focused on key and ancillary controls

The Commission should revise its guidelines so that the CBs’ risk assessment as regards legality and

regularity is focused on the key and ancillary controls already used by the Commission,

complemented by any other evidence that the CBs judge appropriate in accordance with

internationally accepted audit standards.

For the sampling of transactions, the Commission required CBs to use two samples 95.

whose results are then merged: Sample 1, drawn from the list of beneficiaries randomly

selected by the PAs for on-the-spot checks, and Sample 2, drawn from all payments for the

financial year in question. This approach allows the CBs to test a significant number of

transactions (Sample 1) which have also been verified on-the-spot by the PAs, so that the

CBs gather evidence regarding the functioning of the PAs internal control system and the

reliability of the PAs control statistics where the results of the PAs on-the-spot checks are

reported. However, because the CBs extract Sample 1 from the PAs’ random on-the-spot

checks, their work can only be representative if the samples initially selected by the PAs are

themselves representative.

Furthermore, for IACS transactions, which are mainly annual area-based payments, 96.

Sample 1 was selected based on the amounts claimed by farmers, before the PAs had

45

determined the amounts to be paid. The Commission chose this solution so that the CBs

could perform their re-verifications as soon as possible after the PAs’ checks, when the

conditions the CBs found when inspecting the selected farms on-the-spot would be very

similar to those found by the PAs. However, this solution entails several risks, such as:

insufficient cooperation between the PAs and CBs which significantly delays the timing of

the CBs testing, the risk of transactions potentially affected by error being replaced in the CB

sample or the risk of the CBs announcing their visit before the PAs carry out their initial

checks (paragraphs 48 to 58).

Recommendation 3 – Safeguards for IACS sampling based on PAs’ on-the-spot checks

For the selection by the CBs of IACS transactions from the list of claims randomly selected by the PAs

for on-the-spot checks, the Commission should reinforce its guidelines, requiring CBs to put in place

appropriate safeguards that:

- ensure that the CBs’ samples are representative, are transmitted upon request to the

Commission and that an appropriate audit trail is maintained. This means that the CBs need to

check whether the Pas’ samples are representative;

- enable the CBs to plan and carry out their visits shortly after the PAs have carried out their on-

the-spot checks;

- ensure that CBs do not disclose their sample to the PAs before the latter carried out their on-

the-spot checks.

For non-IACS expenditure, which is mainly reimbursement-based expenditure for 97.

investments in farms and rural infrastructure, there is a disparity between the period for

which PAs’ on-the-spot checks are reported (e.g. from 1 January to 31 December 2014 for

the 2015 financial year) and the period for which expenditure is made (e.g. from

16 October 2014 to 15 October 2015 for the 2015 financial year). This means that those

payments executed before 16 October 2014 or after 15 October 2015 will not represent

expenditure for the 2015 financial year (paragraphs 59 to 61).

46

Recommendation 4 – Non-IACS sampling based on payments

The Commission should revise its guidelines with regards to the sampling method for non-IACS

expenditure, so that the CBs sample non-IACS expenditure directly from the list of payments

executed during the financial year being audited.

For Sample 2, the Commission required CBs to carry out only a re-performance of the 98.

PAs' administrative checks. In most Member States visited, Sample 2 accounted for the

majority of the total CB sample used for substantive testing. However, gathering evidence

based only on a documentary review at PA level, without any on-the-spot testing at final

beneficiary level, will often not provide the CBs with sufficient and appropriate audit

evidence as required by internationally accepted audit standards, as it deprives the auditor

of two audit evidence collection methods that are very important in the context of the CAP

expenditure: inspection and inquiry (paragraphs 62 to 67).

The Commission required CBs, for their substantive testing, only to re-perform (re-99.

verify) the PAs’ initial checks, for both Sample 1 and Sample 2 transactions. Re-performance

is defined by internationally accepted audit standards as ‘independently carrying out the

same procedures already performed by the audited entity’, in this case the PA. However,

these standards state that auditors should choose and perform all audit steps and

procedures that they themselves consider appropriate. As such, because the Commission

limits the CBs to re-performance, their substantive testing is incomplete and does not

provide them with sufficient audit evidence to form a reasonable assurance opinion

(paragraphs 68 to 71).

Recommendation 5 – Substantive testing carried out on-the-spot

The Commission should revise its guidelines to allow the CBs to carry out:

- on-the-spot testing for any transaction audited;

- all audit steps and procedures that they themselves consider appropriate, without being limited

to re-performing the PAs’ initial checks.

47

The Commission requires the CBs to calculate two indicators of error in relation to 100.

legality and regularity:

- the ERR, which is based on the CB’s audit of the PA’s entire control process, from

receipt of claims until payment and accounting, and is used to express an opinion on the

legality and regularity of expenditure; and

- the IRR, which measures only the financial impact of errors in the PA’s primary eligibility

controls (administrative and on-the-spot checks) before sanctions are applied and is

used to assess whether the CBs’ examination casts doubt on the assertions made in the

MDs, including the levels of error reported by PAs in the control statistics.

However, the CAP Horizontal Regulation requires only limited assurance regarding the MDs,

which is usually based on work that is limited to analytical procedures and enquiries, with no

tests of details.

The IRR is not necessary for ensuring compliance with internationally accepted audit 101.

standards, as the conclusion on the MDs can be drawn as from the results of the audit work

done in relation to the reliability of accounts, the proper functioning of the internal control

system and the legality and regularity of transactions. Furthermore, the Commission

envisaged using the IRR instead of the ERR in the calculation of the adjusted error rate,

despite the fact that the IRR does not represent the level of error in the expenditure but

rather the financial impact of weaknesses in the PA’s administrative and on-the-spot checks.

On the other hand, however, the ERR has been wrongly used by both the CBs and the

Commission to form a view on the completeness, accuracy and veracity of the PAs' accounts

(paragraphs 72 to 78).

48

Recommendation 6 – A single error rate in relation to legality and regularity

The Commission should revise its guidelines so that they do not require the calculation of two

different error rates regarding legality and regularity. A single error rate, based on which the CBs

expressed their reasonable assurance opinion on legality and regularity of expenditure and limited

assurance conclusion on the assertions made in the management declaration, would satisfy the

requirements of the CAP Horizontal Regulation.

The Commission and the CBs should not use such an error rate for legality and regularity to judge

the completeness, accuracy and veracity of the PA’s annual accounts.

The ERR is calculated as the difference between what the CB considers eligible and what 102.

the PA has previously validated. The differences between what beneficiaries claimed and

what the PA validated after its on-the-spot checks, which constitute the errors reported by

the PAs in the control statistics, are not taken into consideration in the ERR which is the basis

of the CBs’ opinion on legality and regularity of expenditure. Where transactions have

previously been subject to on-the-spot checks, the PAs have already detected and reported

the corresponding errors in their control statistics. These errors will no longer be identified

by the CBs when comparing their results to the PAs’ results, and the CBs are therefore likely

to find fewer errors for such transactions. Thus, the more transactions have previously been

subject to PA checks in the CB sample, the lower the overall level of error found by the CBs

will be.

In the six Member States visited, the average share of transactions subject to PA on-the-103.

spot checks in the overall sample audited by CBs was 38 % for EAGF and 44 % for EAFRD.

However, as a general rule, the percentage of transactions in the total population subject to

PA on-the-spot checks is only 5 %. Thus, in the CB samples, the transactions which had

previously been subject to PA on-the-spot checks – and which were therefore less affected

by error in CB testing –were overrepresented in the overall CB samples. This led to

underestimated levels of error being reported by the CBs, because the CBs had not made

any adjustments to ensure a faithful representation of the much smaller proportion of

transactions previously subject to PA on-the-spot checks in the actual population

(paragraphs 79 to 85).

49

Recommendation 7 – An error rate representative of the population covered by the audit opinion

The Commission should revise its guidelines so that:

- for IACS transactions which are sampled from the list of PAs’ random on-the-spot

checks, the overall error calculated by the CBs also includes the level of error reported

by the PAs in the control statistics, extrapolated to the remaining transactions not

subject to PA on-the-spot checks .The CBs have to ensure that the control statistics

compiled by the PAs are complete and accurate;

- for the transactions sampled by the CBs directly from the full population of payments,

no such adjustment is necessary because the sample would be representative of the

underlying populations audited.

The above recommendations 2 to 7 have as their implementation target the next 104.

revision of the Commission's guidelines applicable from the financial year 2018 onwards. The

Commission finalised in January 2017 new guidelines to be applied by CBs from 2018

financial year. These revised guidelines have not been subject to our review (see

paragraph 17).

This Report was adopted by Chamber I, headed by Mr Phil WYNN OWEN, Member of the

Court of Auditors, in Luxembourg at its meeting of 22 March 2017.

For the Court of Auditors

Klaus-Heiner LEHNE President

1

ANNEX I

The Commission’s shared management model, as presented in its 2015 AAR

Accredited Paying AgencyAdministrative checks on all aid applications by 15/05 n-1)

Selection of random and risk based samples for on-the-spot checks (OTSC)Performance of OTSC

Authorisation and payment of the aid (12/n-1 to 6/n)

Beneficiaries

Lodging of aid claims to

paying agency

Payment of aid to the

beneficiary (12/n-1 to

6/n)

On-the-spot-checks by PA on sample of

aid claims

Certification BodyOpinion

- on the annual accounts of the PA- on the proper function of the internal control

system in the PA- on the legality and regularity of the expenditure- whether the examination carried out by CB puts

in doubt the assertions made by the PA's director in his/her Management Declaration

CB opinion on PA report

CB reperforms and reverifies administrative and OTSC on a

sample of individual

transactions

European Commission : DG AGRIVerification of and reimbursement of monthly and quarterly payment declarations

Verification and clearance of annual accountsDG AGRI systems audits and conformity clearance procedure

Net financial corrections

Monthly (EAGF) and Quarterly (EAFRD)

declarations; Annual

Accounts,Control

Statistics,Management Declaration

DG AGRI conformity clearance systems audits to

Paying Agency

CB report & opinion

Reimburse-ment of

expenditure declarations;

Annual financial

clearance;Conformity clearance decisions;

Net financial corrections

1

ANNEX II

The Commission’s assurance model for legality and regularity of expenditure, as presented in its 2015 AAR

Accredited Paying Agency (PA)

Ex-ante administrative checks on 100 % of aid claimsEx-ante risk and random on-the-spot-checks (OTSC)

Reduction of aid claimsPayment of reduced amounts

aid applications

payment of aid

MS reporting

output

CB reperforms and reverifiesadministrative and OTSC on a

sample of individual transactions

DG AGRI conformity audits and net financial corrections

Reports to DG AGRI

Opinion sent to DG AGRI

Chec

kson

PA

DG AGRI certification audits

DG AGRI's accreditation audits

DG AGRI financial clearance

administrative and OTSC

Shared Management – CAP assurance model for legality and regularity

Where does DG AGRI get its assurance?

Architecture of MS management structure- accredited PAs

- independent audit bodies (CBs)

DG AGRI's conformity audits

DG AGRI's accreditation audits

DG AGRI audit and management opinions

Management Declaration of PAs

Opinions of the CBs

European Court of Auditors' findings

Net financial corrections

Ove

r 8 m

illio

n be

nefic

iarie

s

Annual AccountsAnnual Management Declaration

Results of controls carried out

Certification Body (CB)

Opinion- on the annual accounts of the PA

- on the proper functioning of the internal control system in the PA- on the legality and regularity of the expenditure

- whether the examination carried out by CB puts in doubt the assertions made by the PA's director in his/her Management Declaration

CB opinion onPA reports

1

ANNEX III

An example of an accreditation matrix, adapted from DG AGRI’s Guideline No 3 for the certification audit of the accounts

S T at 10 % or 15 %

S T at 5 % S T at 5 % S T at 50 % S T at 5 % S T at 10 % S T at 10 % S T at 5% W T

Administrative controls

4 0.40 4 0.20 2 0.10 3 1.50 3 0.15 3 0.30 3 0.30 4 0.20 20% 0.00 0.63

On-the-spot controls

4 0.40 4 0.20 2 0.10 2 1.00 3 0.15 3 0.30 2 0.20 4 0.20 20% 0.00 0.51

4 0.60 4 0.20 3 1.50 3 0.15 3 0.30 3 0.30 4 0.20 15% 0.00 0.49

4 0.60 4 0.20 3 1.50 3 0.15 3 0.30 3 0.30 4 0.20 15% 0.00 0.49

4 0.60 4 0.20 3 1.50 3 0.15 3 0.30 3 0.30 4 0.20 10% 0.00 0.33

4 0.60 4 0.20 2 1.00 3 0.15 3 0.30 3 0.30 4 0.20 20% 0.00 0.55

General conclusion 2.99

Assessment of the internal control system Works

Debts management

Advances and securities

Accounting

Execution of payments

Weighting/ Scoring

Claims processing, including Validation and autorisation

Internal audit

MATRIX - Assessment of the ICS for the IACS population

Assessment

Component procedure

Internal environment Control activities

Information and communication

Monitoring Evaluation at assessment

criteria

General conclusion

Organisation Human resources

Delegation Communication Information systems and security

Ongoing monitoring

1

ANNEX IV

The timetable of the PAs’ and CBs’ substantive testing and reporting to the Commission

5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6

PA inspections of IACS (EAGF and EAFRD) applicants, administrative

cross-checks run from claim submission

Control statistics (claim year 2014)

sent to the Commission Accounts and

management declaration

FY 2015 submitted to the

Commission (15 Feb)

PA work

Majority of payments to IACS scheme applicants

(advances paid from 16 Oct 2014; final payments from 1 Dec 2014)

PA samples of applicants to be

inspected are being drawn

Non-IACS population (EAGF and EAFRD)Receipt of applications, administrative and

on-the-spot checks, payments

Deadline for area-related aid applications (EAGF

and EAFRD)(15 May)

2014 2015 2016

CB re-performances of PA IACS inspections on 2014 applications

(EAGF and EAFRD)

CB should draw a random

sample of IACS inspections to

be re-performed

CB re-performances of PA non-IACS inspections(EAGF and EAFRD)

CB checks on payments made (re-performance of administrative checks) for IACS and non-IACS population

(EAGF and EAFRD)

CB reports for FY 2015 to be

submitted to the Commission

(15 Feb) CB work on PA

COM audit visits to MSs (CBs)

DG AGRI AAR for 2015 issued

(30 Apr)COM

work on CBs

Claim year 2014

Financial year 2015

2

REPLIES OF THE COMMISSION TO THE SPECIAL REPORT OF THE EUROPEAN

COURT OF AUDITORS

"THE CERTIFICATION BODIES' NEW ROLE ON CAP EXPENDITURE: A POSITIVE

STEP TOWARDS A SINGLE AUDIT MODEL BUT WITH SIGNIFICANT WEAKNESSES

TO BE ADDRESSED"

EXECUTIVE SUMMARY

Common Commission's reply to paragraphs II-IV:

The management and control system of the CAP expenditure is following a single control and audit

approach that is integrated in a pyramid of controls where each upper layer builds its work upon the

results of the previous layer, and where each layer may use the results of the upper layer to improve

its own controls. It is a dynamic model aiming not only at estimating an error rate but also aiming at

detecting the sources of errors, implementing remedial actions and reducing the error rates year

after year. For instance, the Integrated Administration and Control System (IACS) which governs

around 86 % of CAP expenditure is a good system for preventing and reducing the level of errors.

In the CAP assurance model applied by the Commission the independent CBs are a layer of the

pyramid of control. The CBs deliver every year an opinion on the PA's accounts and systems and,

from financial year 2015, on legality and regularity of expenditure. The Commission provides them

with guidelines.

DG AGRI auditors check the reliability of the opinions of the CBs on legality and regularity, in

order to assess whether they can be used as a source of assurance for the EU expenditure covered.

DG AGRI auditors also perform system audits which result where necessary, i.e. assurance not

obtained, in remedial action plans and net financial corrections, as well as interruptions, reductions

or suspensions of payments when the MS concerned does not take remedial actions.

The CAP single audit model

The CAP assurance model results in lower error rates and is cost effective. The overall error rate for

Natural resources (where the CAP represents 98 % of the expenditure) established by the Court for

financial year 2015 was 2.9 %.

V. The CAP assurance model already follows a "single audit" approach as it is an effective control

framework in which each layer builds on the assurance provided by others. This single audit model

gives reliable assurance on the legality and regularity of expenditure when taking into account all

corrective actions.

The new work of CBs reinforces assurance on the management and control system and on the

legality and regularity of the expenditure as:

3

- it creates a shorter circuit between the PA and its new first auditor, favouring quicker remedial

actions where necessary;

- it allows the Commission to better identify the expenditure at risk and focus its own audit and

corrective actions on it.

Where the CBs’ work cannot be relied on, DG AGRI continues to obtain independent assurance on

legality and regularity of expenditure from its own audits.

VI. To ensure continuity, the guidelines for 2015 – first year for the enhanced work of the CBs –

were elaborated on the basis of guidelines used for previous years' financial clearance. This was

also reflected in the integrated approach referred to in the new legislation. To take account of the

experience from the first 2015 exercise, the guidelines to CBs have already been revised. New

guidelines were finalised and presented to the MSs on 19 January 2017, for application from FY

2018, i.e. the earliest possible year.

(a) A new matrix has been introduced in the new guidelines for FY2018 for better reflection of the

level of assurance from the PA's internal control systems.

(b) The random samples controlled by PAs shall be selected in accordance with detailed EU rules

and guidance that include representativeness. The role of the CBs is to check that these rules have

been respected by the PAs and to extract second level samples that must be representative for the

random sample controlled by PAs. The issue for a portion of the sampling for the non-IACS

expenditure is addressed in the new guidelines for FY2018.

(c) In its single control and audit approach and considering the effectiveness of IACS, the

Commission considers that the audit work of the CBs should be focused on checking, by re-

performing them, the reliability of the controls done by the previous level in the pyramid of

controls.

(d) Re-performance is an audit methodology which, in accordance with IAAS, can be used to form

an audit opinion on legality and regularity. The Commission considers, as it is set out in the

guidelines, that, in the CAP framework, re-performance is sufficient to form an opinion on legality

and regularity of the expenditure. Furthermore, by definition, guidelines should never be understood

as mandatory instructions preventing those to whom they are addressed from taking their own

responsibilities. The guidelines should be seen as the minimum that is necessary to satisfy the

Commission, and not as a limit for CBs to do more.

Common Commission's reply to (e) and (f):

The Commission considers that the issues of the calculation of different error rates for legality and

regularity are addressed in the new guidelines for FY2018, taking into account Article 9(2)(b) of

Regulation (EC) No 1306/2013 which provides that, where appropriate, CBs may use an integrated

sample for both legality and regularity of expenditure and reliability of the accounts.

An incompliance rate (IRR) (cf. paragraph 74 for further details), to be added to the error rate found

by PAs and certified by CBs, is sufficient to form an audit opinion based on controls done by

previous layer of controls.

VII.

(a) The Commission considers that, whereas the guidelines for FY2015 can be improved, where

they are correctly applied they already in their current form enable the CBs to give an opinion on

legality and regularity in compliance with applicable rules and standards.

Where the audit work of the CBs is done in accordance with the applicable regulations and

guidelines their opinion are an important and valuable additional building block and should be the

key element for the Commission's assurance. This is in line with the audit strategy of DG AGRI and

the CAP single audit/control pyramid.

4

(b)

(i) The key and ancillary control lists were only agreed with MSs in the spring of 2015, hence not

available for CBs when guidelines were drafted. The new guidelines for FY2018 provide for the use

of the key and ancillary controls lists that have been updated for the period 2014-2020.

(ii) Taking into account the first year experience, the new guidelines for FY2018 address more

precisely these difficulties.

(iii) The new guidelines for FY2018 address this issue.

(iv) Requiring or allowing the CBs to go on-the-spot for transactions which have not been

controlled on-the-spot by the PAs would not be in line with the CAP single audit model and would

not bring comparable results vis-à-vis the PA's checks. Moreover, and especially for non-IACS

transactions, checking on-the-spot transactions previously not controlled on-the-spot by PAs would

generally not result in valuable additional evidence. The new guidelines for FY2018 provide

guidance to CBs on how to strike the right balance between costs and benefits in their work.

(v) The new guidelines for FY2018 address this issue.

(vi) The new guidelines for FY2018 address this issue. As provided for in them, the overall error to

be calculated by the CB for legality and regularity of the IACS transactions will be based on re-

verifications of on-the-spot checks, to be extrapolated to the remaining population.

The IRR represents the level of errors that were not detected by the PAs. Where the Commission is

satisfied that it can rely on the opinion of CBs, the overall rate of errors in the expenditure is

calculated as the sum of the error rate reported by the PA and the IRR estimated by the CB. This

logical approach of topping up the errors reported by PAs with estimates of non-detected errors has

been used in previous years by the Commission in DG AGRI Annual Activity Report.

INTRODUCTION

6. The Commission would like to precise that the minimum rate of on-the-spot checks can be

reduced down to 3% or even down to 1% under specific conditions, notably that the CB has

certified an error rate below 2%. In particular, a reduction to 1% may apply to basic payment

scheme, single area payment scheme, redistributive payment and small farmers' scheme.

12. The legal framework requires the CBs to issue an opinion drawn up in accordance with

internationally accepted audit standards. Moreover, the legislator empowers the Commission to

adopt implementing acts laying down rules on the tasks of the CBs, including the audit principles

and the audit methods (for instance the use of a single integrated sample). To that end, Article 6(4)

of Regulation (EU) No 908/2014 provides that the Commission shall establish guidelines to provide

further clarification and guidance in respect of the certification audit to be performed, and the

determination of the reasonable level of audit assurance to be achieved from audit testing.

OBSERVATIONS

22. The Commission considers that the CAP assurance model complies with the description of the

single audit as laid down by the Court and is a very good example of an effective control framework

in which each layer builds on the assurance provided by others. The model is based on solid

foundations that include accredited PAs managing the financial relationship with both beneficiaries

and the Commission, as well as comprehensive, detailed and precise rules on the management and

controls systems to be set-up (e.g. Land Parcel Identification System – LPIS). The effectiveness of

the model is evidenced by the relatively low error rates for EAGF and the continuous decrease in

the last 4 years in EAFRD error rates.

5

23. Common Commission's reply to paragraphs 23 and 24: For the CAP expenditure, the

Commission has built an assurance model that integrates the different layers of controls. The

control pyramid below shows how assurance is built upon the work carried out by the previous

layer, starting with the PAs' procedures and embedded controls in the internal control system, as

well as assurance from the internal audit service of the PA. In the Commission's model, the CBs'

work is based on the PA internal control system testing and reliance, as well as on their own

substantive testing.

The CAP single audit model

The Commission uses the results of its own audits on PAs to adjust (top-up) the error rate reported

by PA with an amount representing the level of errors not detected by PAs during their primary

controls before payments. The adjusted error rates are reported, at the level of each paying agency,

in the Annual Activity Report of DG AGRI and, together with tailored reservations and action

plans, are the basis for the declaration of assurance of the Director General of DG AGRI.

25. DG AGRI Audit Strategy 2014-2020 identified the new role of the CBs in giving an opinion on

legality and regularity as an opportunity to achieve more synergies with that of DG AGRI’s own

assurance building work reported in the Annual Activity Reports. This implies a gradual refocusing

of DG AGRI audits towards the CBs instead of the PAs in the mid and long term. However, the

audit work of the CBs will have to be checked before it will be possible to rely on the assurance

reported by them as regards legality and regularity. The objective is to place more and more

reliance on the assurance that can be obtained from the opinion of the CBs on legality and

regularity, as soon as it is verified that their work is to standard and can be relied upon.

27. The model described by the Court in Figure 2 is only one of the many possible models. The

Commission has chosen a more integrated approach taking the form of a pyramid of controls where

all layers of control give valuable evidence on both amounts and source of errors, thus maximising

their cost effectiveness. See Commission's reply to paragraphs 23 and 24.

28. The Commission welcomes the Court's suggestions on how the work of the CBs could improve

its assurance on the legality and regularity of EU expenditure and stresses that those have been

already implemented to the extent possible, i.e. when the work of CBs is considered reliable. For

instance:

First indent:

the Commission already uses the CBs' work for its adjustments (top-ups) to the error rates reported

in DG AGRI Annual Activity Report and which are close to those reported in the annual report of

the Court.

Second indent:

conformity clearance procedures are systematically launched as a result of the CBs' work in case the

errors exceed 50 000 EUR or 2% of the expenditure.

6

Third indent:

extrapolated error as calculated by the CB on the basis of their audit tests and statistical samples are,

where possible, used by the Commission for determining the amounts of financial corrections.

Fourth indent:

observations from the CBs can trigger requests to the competent authorities to review the

accreditation status and supervise the implementation of remedial actions.

30. When DG AGRI auditors identify deficiencies in the control system of PAs, they estimate the

amount at risk that was not detected by the PA and add it as a top-up to the error rate reported by

the PA. The result of the work of CBs on annual accounts was also used for top-ups in previous

Annual Activity Reports. The adjusted error rates are the basis for the declaration of assurance by

the Director General of DG AGRI.

31. DG AGRI could only make a limited use of the CBs' opinions for 2015 because in many cases

the work done was not yet reliable.

32. The Commission confirms the reasons why it could only make a very limited use of the

opinions in the first year of implementation of the new opinions on legality and regularity and refers

to DG AGRI Annual Activity Report 2015 for more information on the underlying reasons such as:

late appointment, late timing of the on-the-spot re-verifications, lack of experience, etc.

The Commission considers that, whereas the guidelines for FY2015 can be improved, where they

are correctly applied they already in their current form enable the CBs to give an opinion on legality

and regularity in compliance with applicable rules and standards.

33. The CAP assurance model (cf figure of pyramid of controls in the reply to paragraph 23 and 24)

is based on the control results of the PAs. The audit strategy of DG AGRI clearly sets out that the

ultimate objective for DG AGRI would be to get the necessary assurance from the audit work of the

CBs. The Commission considers that this assurance can be obtained where the CBs work is

considered reliable and their opinions are built upon the work carried out by the previous layer (i.e.

the PA's procedures and controls). Where the CBs’ work cannot be relied on, DG AGRI continues

to obtain independent assurance on legality and regularity of expenditure from its own audits.

34. In a single audit model the CB auditors should report on the errors (not) detected on the sampled

PA checks. PAs receive on an annual basis millions of aid applications, 100% of which are checked

administratively. In addition, PAs conduct on-the-spot controls before authorising the payment. For

most schemes, the on-the-spot check control rate is 5%, however, for some is 30% or even 100%. If

after its checks of the primary controls, a CB considers that the PA's procedures do not give

assurance on legality and regularity then it will give a qualified opinion, also affecting the error rate

provided by the PA.

35. The Commission's guidelines have been developed with the view to integrate the work of the

CBs in the CAP pyramid of controls, following a single audit approach, in accordance with EU

rules and ensuring that Member States could build on and further develop the methods and systems

already in place.

First indent:

See Commission's replies to paragraphs 48 to 58 and Recommendation 6.

Second indent:

See Commission's replies to paragraphs 62 to 67 and Recommendation 5.

Third indent:

See Commission's replies to paragraphs 68 to 71 and Recommendation 5.

7

Fourth indent:

See Commission's replies to paragraphs 72 to 78 and Recommendation 6.

Fifth indent:

See Commission's replies to paragraphs 79 to 85 and to Recommendation 2.

36. The Commission considers that, in accordance with internationally accepted audit standards, the

structure of the opinions shall reflect the different management and control systems.

There are two distinct agricultural funds: EAGF and EAFRD on which the CBs need to issue audit

opinions. In turn, the two Funds are divided into IACS1 and Non-IACS expenditure, depending on

the management and administration of the schemes/measures under the Fund. The IACS

schemes/measures are processed through a system with automated cross-checks embedded in it and

the system follows the whole cycle of the claim until the payment. In contrast, the Non-IACS

measures are not managed in one system. Furthermore, the timing of the controls in non IACS and

IACS is different. These inherent differences in the design of the schemes/measures prevents the

CBs from testing all measures in one Fund altogether. In order to comply with internationally

accepted audit standards (e.g. ISA315- Identifying and assessing the risks of material misstatement

through understanding the entity and its environment), the CB needs to first understand the

environment, identify risks and tailor appropriate audit procedures to cover those risks. In order to

be able to deliver a solid opinion on legality and regularity at Fund level, the CBs should assess the

risks separately for IACS and Non-IACS and design different audit procedures to cover those risks.

Thus, the guidelines on the Audit Strategy that the CBs are encouraged to use are in accordance

with internationally accepted audit standards. In addition, this approach serves the very purpose of

the opinion on legality and regularity, i.e. to ultimately and where necessary take appropriate

management decisions.

37. Please see Commission's replies paragraphs 37 to 84.

Common Commission's reply to paragraphs 41 to 44:

To ensure some continuity with the previous guidelines and enable CBs to build on the experience

from work done over many years for the annual financial clearance of accounts guidelines for

FY2015 did not substantially modify the way to use the accreditation matrix to form an opinion on

whether the PAs’ internal control systems are functioning properly. The accreditation matrix is

based on the accreditation criteria as laid down in Annex I of Regulation (EU) No 907/2014. Those

accreditation criteria are inspired from the COSO model which is recognized as a leading

framework for designing, implementing, and conducting internal control and assessing the

effectiveness of internal control.

The 2015 Guidelines were based on an integrated approach and sampling, in line with Article

9(2)(b) of Regulation (EU) No 1306/2013 and under which the testing, error evaluation and

reporting of errors under legality and regularity is combined with those for the annual accounts. In

this context, the usage of the whole accreditation matrix was deemed appropriate.

Notwithstanding the above, the Commission agrees with the Court that whilst the accreditation

criteria provide a solid basis for assessing the overall control framework of the PAs (what the audit

standards refer to as entity-level controls), the internal control system assessment for legality and

regularity on one hand, and, for the annual accounts2 on the other, needs to be separated.

1 Integrated Administration and Control System.

2 Execution of payments, accounting, advances and securities, debt management.

8

The approach to clearly separate the processes/procedures linked to legality and regularity from the

ones linked to the annual accounts is followed in the new guidelines for FY2018, which were

finalised in January 2017. This involves also amendment in the way the internal control system is

assessed for the purposes of the annual accounts and for the purposes of legality and regularity.

45. The Commission agrees with the Court. Similar observations were made by the Commission in

the letters sent to several MSs as a result of the 2015 financial clearance exercise.

Moreover, the Commission recommends to the CBs to look at action plans, which aim at addressing

known weaknesses. This link was discussed at the Expert Group Meeting of June 2016.

Nonetheless, this point is further emphasised in the new guidelines for FY2018.

In any event this must also be seen in the light of this being the first year that the CBs carried out

the new work on legality and regularity.

Box 1 – Situations where the accreditation matrix did not produce reliable results for legality

and regularity purposes

Please see Commission's reply to paragraph 45.

The weakness such as the one identified in Bavaria is provided for in the 2015 guidelines – section

6.5. In case the CB finds that the results of the substantive testing do not match with the previous

assessment of the Internal Control System (ICS), then the sample should be enlarged to

accommodate the real situation of the internal control system.

46. Comprehensive and detailed lists of key and ancillary controls have been elaborated by the

Commission in close cooperation with the MSs, for each of the type of measures of the CAP 2014-

2020. They are to be used at each control level.

47. The accreditation criteria should cover the overall set-up and functioning of the PA (i.e. the

entity-level controls), among other things, the appropriateness of processes and procedures put in

place.

When assessing the administrative and on-the-spot (OTS) process (the authorisation of claims) for a

given population, the CB's assessment should be based on the key and ancillary controls. The CBs

are also using the key and ancillary controls when they assess the legality and regularity of the

expenditure (under their compliance and substantive testing).

Nonetheless, the fact the CBs need to assess the process and procedures of authorisation of claims

(including administrative and on-the-spot controls) on the basis of the Key and ancillary controls is

clearly provided for in the new guidelines.

49. The split of the work for substantive testing into two samples resulted from the need to ensure

some continuity for Member States between the guidelines to CBs that were applicable before 2015

and those applicable from 2015. It also took into account the possibility established by Article

9(2)(b) of Regulation (EU) No 1306/2013 to use where appropriate a single integrated sample.

However, the experience of the first year of implementation showed that it was necessary to review

the approach and clearly distinguish the audit work and opinion on legality and regularity from the

opinion on the accounts, whilst keeping the possibility of dual purposes samples where appropriate

and with a view to avoid unnecessary administrative burden.

51. For sample 1 that is extracted from transactions controlled on-the-spot by the PAs, guidelines

require that CBs re-perform all controls done by the PAs, i.e. that CBs systematically re-perform the

on-the-spot controls. This approach allows CBs firstly to give an opinion on the control system and

secondly to calculate an incompliance rate (IRR) reflecting the difference they found between the

results of their re-performance and those of the primary controls by the PAs.

9

52. On the basis of the internal control system assessment, CBs should determine the level of

substantive testing (both sample 1 and sample 2). Thus, it is a CB's prerogative to increase the level

of substantive testing in order to cover the risks. In the table 3 presented by the Court, it is shown

that some CBs decided to increase the samples of substantive testing above the minimum 30

transactions in order to address the high risks in the given population.

55. The CBs are required to make their sample selection on claims for the IACS populations due to

the changing nature of the agricultural reality. Thus, it is crucial that the CB goes on the spot shortly

after the PA in order to be able to re-verify all eligibility elements. If re-verification is performed

late, there is a huge risk that not all eligibility elements can be verified or the reality has changed

completely, making re-verification ineffective and the opinion unreliable.

56. Due to the agricultural reality, there is indeed a risk related to the timing of the primary on-the-

spot checks and their re-verifications.

The guidelines address that risk and the CBs were repeatedly invited by the Commission to

adequately plan their re-verifications taking into account the time constraints imposed by the nature

of the re-verifications.

As reported in DG AGRI Annual Activity Report 2015, inadequate timing of the re-verifications

was unfortunately one of the main reasons why the Commission was not able to rely on many of the

CBs' opinions on 2015 expenditure.

Box 3 – Delays in the CBs’ re-performance of IACS Sample 1 on-the-spot checks for the

2014 claim year (2015 financial year)

Delays in appointing the CBs and in re-verification of the controls in claim year 2014 (financial

year 2015) were the main reasons why the Commission could not rely on CBs' opinions on legality

and regularity for its Annual Activity Report 2015.

Already during its 17 audits done in 2015, the Commission has observed in numerous cases

significant delays in the re-performance of controls. The Commission has issued a Note to the MSs

on 09/11/2015 in which it explains that the on-the-spot re-verifications should start in the summer

of n-1 for financial year n.

57. Under the FY2015 guidelines, the CB's re-verification sample is to be drawn from the list of

transactions selected for random on-the-spot check by the PA. Thus, the CB's selection and actual

re-verification should normally be done before the PA starts processing payments. As a result, it is

very unlikely that the CB will not select an item because the PA is delaying the payment.

Box 4 – Exclusion of transactions from IACS Sample 1 because the PA did not execute

payment until after the CB had completed its audit work

The work of the CBs on legality and regularity is threefold:

- opinion on the effectiveness of the internal control system;

- opinion on the legality and regularity of expenditure;

- opinion on the assertions made in the Management declaration (i.e. the control results reported by

the PA).

According to the guidelines, the CB should have still used these five transactions when assessing

the assertions reported in the Management Declaration, as well as on the effectiveness of the

internal control system. However, Article 9 of Regulation 1306/2013 requires an opinion on the

legality and regularity of the expenditure. Thus, the CB cannot issue an opinion on non-executed

payments. In this case, an additional item should have been tested to comply with the originally

determined sample size for the financial year in question. In order to anticipate such events, the CBs

10

are advised to select some additional items in advance, above the minimum 30 foreseen in the

guidelines.

The CB should consider the result of these tests for the opinion on the legality and regularity of

expenditure regarding the financial year in which this transaction will be paid. Those tests should be

taken into account irrespective of any correction the PA could do before payment.

58. Article 7 of Regulation (EU) No 908/2014 allows the CBs to accompany the second-level on-

the-spot checks. Accompaniment of the first-level checks is limited to the check that could not be

re-performed at a later stage (such as EAGF Non-IACS measures which involve operations linked

to distillation, storage or withdrawal operations). The idea behind the re-performance is that the PA

should do the check, so that the CB re-verifies it afterwards.

Notwithstanding the above, the guidelines request CBs to perform their tasks following professional

audit standards and not compromising the results of the substantive testing. Thus, the CB should

never share its selection before the PA conducts its checks. The shortcoming described by the Court

in Box 5 confirms that CBs should follow the guidelines to avoid inadequate audit practices.

Box 5 – The re-performance exercise was compromised in Italy because the CB had given the

PA advance notice of which beneficiaries would be subject to re-performance

CBs shall put in place safeguards for not disclosing their samples to the PAs before the latter have

concluded their own checks and advised the CBs accordingly.

61. There is a mismatch between the controls performed by the PA in a given year and the

payments for the financial year for the Non-IACS populations. Nonetheless, since the majority of

the Non-IACS measures are not time-constrained, the CB could test additional transactions in case

some of the already tested ones were not paid during the year. Unpaid transactions could still be

used to confirm or not the Management declaration and the control results.

62. In the CAP framework, where a legally binding and effective management and control system is

to be put in place by the PAs, auditors can in many cases, once they have sufficient assurance that

the PA's management and control system is designed and implemented in accordance with EU law,

gather sufficient and appropriate evidence by re-performing the PA controls.

Common Commission's reply to paragraphs 64 and 65:

For sample 1 that is used for establishing the IRR, guidelines require CBs to re-verify both

administrative and on-the-spot primary controls. For sample 2 that is used for establishing the EER,

the Commission's guidelines do not require on-the-spot checks. The proportion between the two

samples is to be defined by the CBs following their professional judgement.

66. The Commission considers that re-performance of administrative checks is sufficient and

appropriate audit evidence. It does include inquiry to the PA's personnel as to the checks performed,

checks of the aid application and supporting documents, as well as checks in registers and LPIS for

IACS schemes. For non-IACS expenditure, administrative checks and their re-verification also

include in-situ visits (inspections) to, for instance, check the existence of physical assets built or

purchased through investment projects.

It is up to the CB's professional judgement to design the audit procedures in order to gather

sufficient and appropriate audit evidence. Section 6.1 of Guidelines 2 for FY2015 says: "The test of

detail on operational transactions – includes, among others…These cover inspection and enquiry,

not only documentary review".

67. For testing the legality and regularity of the IACS expenditure, CBs have to select their sample

from the population that was controlled on-the-spot by the PAs and to re-perform all the primary

controls, both administrative and on-the-spot.

11

For non-IACS expenditure, it should be noted that most of the errors (ineligibility of

beneficiaries/activities/projects/expenditure and non-compliance with public procurement rules) are

found at the level of administrative controls and that around two thirds of the errors came from

insufficient use of available information or errors by the national authorities. It should also be noted

that all administrative controls (to be re-performed by the CBs) on investment operations must

include an in-situ visit to verify the realisation of the investment, except if the paying agency

justifies that the operation is small, low risk or is included in the on-the-spot sample.

Furthermore, taking into account the need to provide guidance to CBs on how to strike the right

balance between costs and benefits, the Commission's guidelines for FY2018 request CBs to have

statistically representative samples for the on-the-spot re-verifications for IACS. For non-IACS

expenditure a minimum of 30 on-the spot re-verifications are requested. The results of the re-

verifications will be extrapolated to the entire population.

69. The Commission considers that under the CAP integrated management and control system,

assessing the checks of the PA as regards legality and regularity, and afterwards, confirming from a

sample of transactions, that the PA has effectively implemented the procedures for primary

controls, is sufficient to provide assurance on the legality and regularity of the expenditure.

71. The Commission considers that in the CAP framework, checking the control systems and re-

performing the primary controls of the PAs generally allows obtaining sufficient and appropriate

evidence and forming a reasonable assurance opinion on the CAP expenditure.

In addition, the Commission guidelines do not prevent CBs from doing any audit work that they

would themselves consider appropriate. It is CBs' role to determine if re-performance is enough.

Inquiry and inspection may also be used in the re-verification of administrative checks. Cf

Commission's reply to paragraph 66.

74. Due to the integrated approach used in the current guidelines, CBs were requested to calculate

two rates of error that deal with legality and regularity. Nonetheless, this approach is revised from

FY2018.

In the new guidelines that are applicable from FY2018 and were presented in January 2017, the

conformity and financial clearance procedures are clearly separated. To that end, CBs are to

calculate two separate rates of error: one error rate (ERR)3 – including only errors from the annual

accounts testing, and one incompliance rate (IRR)4 including all legality and regularity errors. The

Commission already foresees in its new guidelines for FY2018 that the IRR will be used both for

the audit opinion on legality and regularity and to confirm (or not) the assertions in the Management

declaration.

Common Commission's reply to paragraphs 75 and 76:

In cases where there is sufficient evidence that CBs have worked in accordance with the guidelines,

the Commission uses the IRR estimated by the CBs in its assessment of the top-up to the error rate

calculated on the basis of the control statistics reported by MSs at the level of each PA. In addition,

the ERR was taken into account for assessing the level of the top ups.

77. With the new guidelines from FY2018, the ERR includes only errors from the annual accounts

testing and will be used only to assess the reliability of the PAs' accounts.

3 ERR – error rate used to estimate the misstatements in the audited sample and population with a financial impact on

the completeness, accuracy and veracity of the accounts. 4 IRR – incompliance rate estimating the potential financial impact due to deficiencies in the eligibility controls.

12

78. The Commission draws the attention to the integrated sampling foreseen by the legislation and

confirms that it separates the financial and conformity clearance procedures. The new guidelines for

FY2018 clarify it further, with an error rate for legality and regularity and another one for reliability

of the annual accounts.

Common Commission's reply to paragraphs 79 to 84:

As reported in DG AGRI's Annual Activity Report, Annex IV, materiality criteria, the Commission

uses control statistics reported by MSs to calculate a "residual reported error rate" by extrapolating

to the entire population the errors found by PAs in the population randomly selected for on-the-spot

controls. This residual reported error rate is incomplete because it does not take into account the

risk that deficiencies in the controls done by PAs might result in errors not being detecting and

reported.

Therefore, in a second step, DG AGRI collects all available information that would reveal

deficiencies in the first level controls, including the results of its own audits, of ECA's audits or any

other reliable source of evidence. DG AGRI auditors analyse this information, in the light of the

Commission's guidelines on the calculation of the financial corrections (Document C(2015) 3675

final) and using their professional judgement, and adjust the residual reported error rate with a top-

up that estimates the rate of errors not detected by PAs. The Commission uses the adjusted error

rate for estimating the amounts at risk and analysing whether reservations are necessary.

From 2015, the CBs' opinions on legality and regularity of the expenditure constitute potentially

another source of audit evidence for the Commission. In particular, the IRR may be used by the

Commission as a reliable estimate of the top-up for adjusting the error rate for the expenditure

under each PA. In addition the ERR was taken into account for assessing the top ups. However, in

2015 the opinions were often not yet sufficiently reliable for the Commission to use them to a large

extent.

The Commission considers that this use of the IRR to reinforce its assurance on the legality and

regularity of the CAP expenditure is appropriate and that, when established in accordance with its

guidelines, the use of the IRR to adjust the error rate does not understate the total error.

85. The Commission is aware of these potential risks that the opinion of the CBs on legality and

regularity would not be reliable and considers that they are covered in its guidelines. The

Commission will continue to supervise closely that CBs follow the guidelines as well as the audit

standards, including the application of their professional judgement, and conclude on legality and

regularity with the necessary qualification.

CONCLUSIONS AND RECOMMENDATIONS

Common Commission's reply to paragraphs 89 and 90:

The pyramid of controls of the CAP expenditures, which already corresponds to the single audit

approach described by the Court in paragraph 22, has been complemented from FY2015 with the

integration of the CBs between the PAs and the Commission layers, reinforcing the CAP assurance

model. In accordance with Article 6(4) of Regulation (EU) No 908/2014 the Commission has

established guidelines to provide further clarification and guidance in respect of the certification

audit to be performed, and the determination of the reasonable level of audit assurance to be

achieved from audit testing. The Commission considers that, whereas the guidelines for FY2015

can be improved, where they are correctly applied they already in their current form enable the CBs

to give an opinion on legality and regularity in compliance with applicable rules and standards.

Building on the experience from the first year of implementation, the Commission has elaborated

new improved guidelines to be applied for FY2018 onwards.

13

92. The Commission is satisfied with the CAP assurance model that keeps the error rate under

control at relatively low levels. The overall error rate for Natural resources (where the CAP

represents 98 % of the expenditure) established by the Court for financial year 2015 was 2.9 %. For

EAGF, where payments are granted every year to close to 8 million beneficiaries, DG AGRI

estimates that the error rate has been below materiality in the last two years, while the Court's most

likely error (MLE) was stable at 2,2 %. For EAFRD, the level of errors established by the

Commission has persistently decreased in the last 4 years.

Such results would not be achievable without the solid management and control system of the CAP,

notably IACS and the pyramid of controls where assurance is built upon the work carried out by the

previous layers and where the work of the CBs starts from the PAs' control results. In this respect,

CBs are requested to confirm the results of the controls performed by PAs and, if not confirmed, to

establish an incompliance rate and to give a qualified opinion. However, DG AGRI could only

make a limited use of the CBs' opinions for 2015 because in many cases the work done was not yet

reliable.

Where the CBs’ work cannot be relied on, DG AGRI continues to obtain independent assurance on

legality and regularity of expenditure from its own audits and where necessary used them as a basis

for top ups to the error rates reported by the PAs. The resulting adjusted error rates remained

relatively low (cf table 1 and above).

Where the Commission can rely on the CBs' work, the incompliance established by CBs can be

added to the error rate reported by PAs to calculate a reliable error rate that reflects the level of

assurance on the legality and regularity of CAP expenditure.

See also Commission's reply to Recommendation 1.

Recommendation 1 - Assurance from the CBs’ work on legality and regularity

The Commission partially accepts the recommendation, as follows:

The Commission does not accept the recommendation that using the opinion of CBs for

assurance would be possible only once the work is carried out as defined in recommendations 2 to

7, and not where the work is done as defined in the guidelines applicable from the financial year

2015. The Commission considers that, whereas the guidelines for FY2015 can be improved, where

they are correctly applied they already in their current form enable the CBs to give an opinion on

legality and regularity in compliance with applicable rules and standards.

The Commission accepts the recommendation that where the audit work of the CBs is done in

accordance with the applicable regulations and guidelines their opinion are an important and

valuable additional building block and should be the key element for the Commission's assurance.

This is in line with the audit strategy of DG AGRI and the CAP single audit/control pyramid below:

The CAP single audit model

14

93. The accreditation criteria should cover the overall set-up and functioning of the PA (i.e. the

entity-level controls), among other things, the appropriateness of processes and procedures put in

place. However, when assessing the administrative and on-the-spot (OTS) process (the

authorisation of claims) for a given population, the CB's assessment should be based on the key and

ancillary controls. Thus, the key and ancillary controls that define the controls for the particular

population (or scheme/measure) should be used on top of the overall accreditation criteria. The CBs

are also using the key and ancillary controls when they assess the legality and regularity of the

expenditure (under their compliance and substantive testing). (Cf. Commission's reply to paragraphs

41-44).

Nonetheless, the fact the CBs need to assess the process of authorisation of claims (including

administrative and on-the-spot controls) on the basis of the Key and ancillary controls is made clear

in the new guidelines for FY2018.

94. The Commission acknowledges that whilst the accreditation criteria provide a solid a basis for

assessing the overall control framework of the PA (what the audit standards refer to as entity-level

controls), the internal control system assessment for legality and regularity and for the annual

accounts 5 need to be separated. (Cf. Commission's reply to paragraphs 41-44).

The approach to clearly separate the processes/procedures linked to legality and regularity from the

ones linked to the annual accounts is adopted in the new guidelines for FY2018, which were

finalised in January 2017.

Recommendation 2 – Risk assessment focused on key and ancillary controls

The Commission accepts the recommendation and considers it already implemented in the new

guidelines for FY2018. The CBs are advised to use the key and ancillary controls when testing the

procedures for authorisation of claims.

95. The Commission stresses that sample 1 is used not only to test the functioning of the internal

control system and the reliability of the control statistics, but also to conclude on the legality and

regularity of expenditure at beneficiary level. Nonetheless, the new guidelines for FY2018 requires

a separation of audit objectives where the sample for legality and regularity will be used for the

following purposes:

- to conclude on the on the legality and regularity of expenditure;

- to conclude on the control results reported in the Management Declaration and the control

statistics;

- for the internal control system testing if dual-purpose testing is done by the CB;

5 Execution of payments, accounting, advances and securities; debt management.

15

- to certify an error rate for the purposes of the on-the-spot control rate reduction in accordance with

Article 41 of Regulation (EU) No 908/2014.

For IACS expenditure, the applicable legislation and the Commission guidelines to PAs include

methods aimed at assuring the representativeness of the sample with regards to the total population.

In the first step of their sampling operations, the CBs have to check that the PAs' samples are legal

and regular, i.e. that they have been selected in accordance with EU rules and guidelines to PAs,

including representativeness. In turn, the CBs' samples need to be representative for the population

selected by the PAs.

96. The issues related to the timing of the on-the-spot checks and the re-verifications for IACS

transactions are inherent to the changing agricultural reality in the fields.

Thus, it is crucial that the PAs do the primary control at the right time, i.e. before payments, and

that CBs go in the field shortly after the PAs in order to be able to re-verify all eligibility elements.

If the re-verification is performed late, there is a risk that not all eligibility elements will be verified

in the field or the reality has changed completely, so a re-verification will not be possible.

Recommendation 3 – Safeguards for IACS sampling based on PAs’ on-the-spot checks

The Commission accepts the recommendation and considers that it is already implemented in the

new guidelines for FY2018. The new guidelines are reinforced as regards the following aspects:

First indent:

the CB should check that the PA sample is selected in accordance with relevant legislation, rules

and guidelines including representativeness and then, the CB should check the representativeness of

its own sample;

Second indent:

crucial importance of the timing for the IACS re-verifications: a recommended timing of the audit

activities is included in the guidelines;

Third indent:

as part of the audit procedures, the CB should check the completeness and assurance of the control

results reported in the control statistics;

97. The Commission agrees that there is a mismatch between the controls performed by the PA in a

given year and the payments for the financial year for the Non-IACS populations. Nonetheless,

since the majority of the Non-IACS measures are not time-constrained, the CB could test additional

transactions in case some of the already tested ones were not paid during the year. The unpaid

transactions could still be used to confirm or not the Management declaration and the control

results, as well as the effectiveness of the internal control system.

Recommendation 4 – Non-IACS sampling based on payments

The Commission accepts the recommendation and considers that it is already implemented in the

new guidelines for FY2018. In case of time constraints, e.g. for measures entailing many payments

close to the end of the year, the CB is recommended to use estimated payments.

98. The Commission considers that in the CAP framework and notably when taking into account

the IACS contribution to preventing and reducing the level of errors, re-performance of

administrative checks is appropriate audit evidence. It does include inquiry to the PA's personnel as

to the checks performed, checks of the aid application and supporting documents, as well as checks

in registers and LPIS for IACS schemes. For non-IACS expenditure, administrative checks and

16

their re-verification by CBs also include in-situ visits (inspections) to, for instance, check the

existence of physical assets built or purchased through investment projects.

The Commission also considers that re-performing the checks done by PAs is generally sufficient

for CBs to assess the legality and regularity of the expenditure. To check on-the-spot payments not

controlled on-the-spot by PAs would generally not result in valuable additional evidence and will

generate incomparable results vis-à-vis the PA' controls.

99. The guidelines should be seen as the minimum that is necessary to satisfy the Commission and

cannot limit CBs from doing more if appropriate.

The Commission considers that in the context of the CAP expenditure and its management and

control system, re-performance is the most efficient way to reinforce assurance and to form a

reasonable assurance audit opinion.

As explained in the Commission's replies to paragraphs 60-70, the Commission's guidelines

recommend to the CB to perform the same checks (on-the-spot check and administrative checks or

only administrative checks) as the PA so that comparable results are obtained. Moreover, the

guidelines do not limit the CBs to do re-performance but re-performance in combination with other

techniques so that sufficient and appropriate audit evidence is collected. Since the CBs re-verify

also the administrative checks, inspection of documents and registers, as well as inquiry, are also

common techniques used by them.

In the new guidelines for FY2018, the Commission recommends the CBs to have two samples

dedicated to the separate audit objectives (audit of the accounts and audit of legality and regularity).

The Commission has already identified and took on board an approach that allows for an efficient

and effective audit such as: dual-purpose testing between audit objectives and between substantive

and compliance testing6, etc. The audit technique is not limited to re-performance but to the scope

of the PA's checks. Thus, if a PA only checked a transaction administratively, the CB should do the

same – re-verify the administrative checks through re-performance, inspection, inquiry, analytical

checks, etc.

Taking into account the need to provide guidance to CBs on how to strike the right balance between

costs and benefits, the Commission's guidelines for FY2018 request CBs to have statistically

representative samples for the on-the-spot re-verifications for IACS. For non-IACS expenditure a

minimum of 30 on-the spot re-verifications is requested. The results of the re-verification will be

extrapolated to the entire population.

Recommendation 5 – Substantive testing carried out on-the-spot

The Commission partially accepts the recommendation as follows:

1) The Commission does not accept the recommendation where it would entail that the CBs go

on-the-spot for transactions which have not been controlled on –the-spot by the PA. This would not

be in line with the Commission’s CAP assurance model and would not bring comparable results

vis-à-vis the PA's checks. Moreover, and especially for non-IACS transactions, checking on-the-

spot transactions previously not controlled on-the-spot by PAs would generally not result in

valuable additional evidence.

Taking into account the need to provide guidance to CBs on how to strike the right balance between

costs and benefits, the Commission's guidelines for FY2018 request CBs to have statistically

representative samples for the on-the-spot re-verifications for IACS. For non-IACS expenditure a

minimum of 30 on-the spot re-verifications are requested. The results of the re-verification will be

6 The possibility to use one transaction for various audit tests.

17

extrapolated to the entire population. The Commission considers that in the CAP framework this

balanced approach would be sufficient for the CBs to deliver a reliable opinion on the legality and

regularity of expenditure.

2) The Commission accepts the recommendation as regards allowing the CBs to carry out all the

audit steps and procedures that they themselves consider appropriate and considers that it is already

implemented.

100. Due to the integrated approach used in the current guidelines, CBs were requested to calculate

two rates of error that cover legality and regularity. Nonetheless, this approach is revised in the

guidelines for FY2018 (cf. Commission's reply to Recommendation 6 below).

101. To ensure continuity, the guidelines for 2015 – first year for the enhanced work of the CBs -

were elaborated on the basis of guidelines used for previous years' financial clearance. Furthermore,

the 2015 Guidelines were based on an integrated approach and sampling, in line with Article 9(2)(b)

of Regulation (EU) No 1306/2013 and under which the testing, error evaluation and reporting of

errors under legality and regularity is combined with those for the annual accounts. It follows that

under the integrated approach, the ERR is used for assessing legality and regularity of the

expenditure, as well as the annual accounts.

Recommendation 6 – A single error rate in relation to legality and regularity

The Commission accepts the recommendations and considers that it is already implemented.

The new guidelines for FY2018 foresee to clearly separate the conformity and financial clearance

procedures. To that end, CBs are to calculate two separate rates of error: an error rate (ERR)

including only errors from the annual accounts testing, and an incompliance rate (IRR) including all

legality and regularity errors not detected by the PA. The IRR will be used both for the audit

opinion on legality and regularity and to confirm (or not) the assertions in the Management

declaration. The IRR will be used to top up the control results reported by the PAs.

102. The incompliance rate (IRR) is designed to estimate by how much the PA's controls differ

from the CB's re-verification or in other words, how much the PA failed to detect. Thus, it is

reasonable to assume that if the PA finds an error, this error will be corrected before payment. The

CB's error should reflect only how much was not detected and as a result, not corrected before

payment. That is why in order to avoid double-counting of errors, the CB should not include the

beneficiary's claim, as the difference will include a part that was already corrected by the PA and

reported in the control statistics.

103. The CBs work is based on samples in accordance with internationally accepted audit standards.

In any event, all findings found during the on-the-spot re-verifications should be extrapolated to the

rest of the population for the calculation of the incompliance rate (whether or not they were

resolved before payment).

A similar approach is used by DG AGRI in the calculation of the residual error rate in the Annual

Activity Report.

Recommendation 7 – An error rate representative of the population covered by the audit

opinion

The Commission accepts the recommendation and considers that it is already implemented.

The new guidelines for FY2018 foresee that the incompliance rate for the IACS populations be

based on the PA's random on-the-spot check sample, however, the extrapolated result for the part of

the population is then extrapolated to the whole population in order to conclude on the legality and

regularity of all IACS expenditure.

For the Non-IACS populations, a slightly different approach is foreseen to take into account the

different design of the measures and the fact that the PA's random samples may not be

18

representative in all cases. Thus, the incompliance rate to be calculated by the CBs should be based

on a sample drawn from all expenditure which is then extrapolated.

104. The Commission considers that with the new guidelines for financial year 2018

recommendations 2, 3, 4, 6 and 7 are already implemented and that the part of recommendation 5

that is accepted is also already implemented. Since IACS related payments in financial year 2018

correspond to beneficiaries' claims introduced in Spring 2017 and to be checked during the same

crop period, the implementation should be realistic: implementing the new guidelines will require

preparatory work for the MSs, including public procurement procedures in cases where CBs are not

public bodies. Most MSs have already informed the Commission that they could apply the new

guidelines only from financial year 2019.

Event Date

Adoption of Audit Planning Memorandum (APM) / Start of audit 13.1.2016

Official sending of draft report to Commission (or other auditee) 6.2.2017

Adoption of the final report after the adversarial procedure 22.3.2017

Commission’s (or other auditee’s) official replies received in all languages

26.4.2017

This audit examined the role of the Certification Bodies which provide opinions on the legality and regularity of spending under the Common Agricultural Policy at Member State level. The Common Agricultural Policy accounts for almost 40 per cent of the EU Budget. We assessed whether a new framework set up in 2015 by the European Commission enables the Certification Bodies to form their opinions in line with EU regulations and international audit standards. Although the framework is a positive step towards a single audit model, we found that it is affected by significant weaknesses. We make a number of recommendations for improvement, to be included in new Commission guidelines due into force from 2018.

EUROPEAN COURT OF AUDITORS 12, rue Alcide De Gasperi 1615 Luxembourg LUXEMBOURG

Tel. +352 4398-1

Enquiries: eca.europa.eu/en/Pages/ContactForm.aspx Website: eca.europa.euTwitter: @EUAuditors

© European Union, 2017

Reproduction is authorised provided the source is acknowledged.

1977 - 2017