1eea presentation/complete set english DGE

European Environment Agency (EEA)Anita Künitzer

http://www.eea.eu.int

‘Lessons learned in designing effective assessments’

2

The EEA's Mission

... is to deliver

timely, targeted, relevant and reliable information

to policy-makers and the public for the development and implementation of sound environmental policies in the European Union and other EEA member countries.

Domingo Jiménez-BeltránExecutive Director, EEA

3

EU 15 member states+ Iceland LiechtensteinNorwayEEA candidatecountriesStability Pactcountries TACIS

EEA member countries

5

Production process of assessment

•Identification of users (politicians, scientists, school children,...)

•Policy issues to be addressed: what should the assessment achieve?

•Process of the assessment

•Launch: at which policy event

6

Designing effective assessments:

The role of participation, science

and governance, and focusWorkshop co-organised by

the European Environment Agency and

the Global Environmental Assessment Project

2001

7

Integrated Environmental Assessment

An interdisciplinary process of structuring knowledge elements from various scientific disciplines in such a manner that all relevant aspects of a complex societal problem are considered in their mutual coherence for the benefit of (sustainable) decision-making.

8

policy preparation

policy formulation

policy execution

policy evaluation

The Policy Cycle

9

Framework for scientific assessment process to inform policy makers

1. Example: IPCC (Intergovernmental Panel on Climate Change)

• Involved only expert scientists in defined disciplines, no political stakeholders

• Production of lengthy reports

2. Example: CLRTAP (Long-Range Transboundary Air Pollution)

• Less clear science-policy distinction• Few formal reports

10

Effective assessments

•What is effective?

–Cost-effectivness–Improvements in the natural environment–Fulfilling political objectives

•Attributes for effective assessments:

–Credibility–Salience–Legitimacy

11

Credibility

• Lack of credibility:–Assessment based on shoddy methods–Assessment ignores important empirical evidence–Assessment draws inappropriate conclusions from data

• Gain credibility:–Through the process by which the information is created (example: data obtained by good laboratory practise)

–By the credentials or other characteristics of producers of the assessment (example: assessment done by well-known, highly regarded scientist)

12

Salience or relevance

• Lack of salience:–Produced report is never referred to and never heard from again

–Assessment addresses questions to which the user is not interested in the answers

• Gain salience:–Assessment is able to address the particular concerns of a user

–User is aware of the assessment–User considers the assessment relevant to current policy

13

Legitimacy

measure of the political acceptability or perceived fairness of an assessment to a user

• Lack of legitimacy:–In ‘global’ assessments inputs from less powerful countries are not included or their interests are ignored

• Gain legitimacy:–Users and participants interests, concerns, views, perspectives have been taken into account

–The assessment process has been a fair one

14

Assessment design (1)

1. Historical context of the assessment• Characteristics of the issue area• Position of the issue on the political agenda

2. Characteristics of the intended user• Interest in the issue and/or assessment• User capacity to understand the results• User openness to different sources of advice

15

Assessment design (2)

3. Assessment characteristics

• Participation: who is involved in the assessment process?

• Science and governance: how are assessments conducted with respect to the interactions between scientific experts and policy makers?

• Focus: how broadly (multidisciplinary) or narrowly (technically) focussed should the assessment be? How consensus-based should it be?

16

Conceptual framework for considering effective assessments

Ultimate determinants Proximate pathways Assessment effectiveness

Historical context

•Issue characteristics

•Linkage

•Attention cycle

User characteristics

•Concern

•Capacity

•openness

Assessment characteristics

•Science/governance

•Participation

•focus

Salience

Credibility

Legitimacy

Effectiveness

17

Participation: critical issues• The capacity of partners, clients and/or users to participate

in the assessment (travel costs, administrative capacity, time for the assessment itself).

• Are scientists participating in their individual capacity (good for scientific credebility) or are they accountable to governments?

• Encourage participation of stakeholders to whom the assessment is designed to: NGOs, policy making community, country representatives to make them interested in the final report

• Process of participation might be more important than content: inclusion as author or attending a meeting increases legitimacy of assessment.

• Broad review of an assessment done by few scientist by several international organisations can increase the legitimacy

18

Science and governance: critical issues

• Assessments on issue areas, which are scientific controversial should be undertaken by institutions accountable to the scientific community to minimise credibility concerns.

• While scientist prefer credibility of assessments, politicians prefer salience. Therefore such assessments on more mature scientific areas might better be undertaken by other organisations more focussed on policy needs.

• Including policy recommendations in a scientific assessment can be dangerous. Here assessment of ’boundary organisations’ that are accountable to science and policy may be the solution.

19

Focus: critical issues

• Succesful assessment avoid adressing controversial issues.

• Broadly focussed assessments include more relevant factors and increase the audience and might be more relevant to decision makers.

• Most assessment to date have been too simple by excluding too many factors and causal chains.

• Assessment should be kept comprehensive despite all interactions. Periodic separate thematic assessments could be produced instead of one big comprehensive assessment.

20

M: Monitoring

D: Data

I: Information

A: Assessment

R: Reporting

R: Reporting

A: Assessment

I: Information

D: Data

M: Monitoring

Use MDIAR to analyse the information provision process

MDIAR stands for:

22

The DPSIR framework

Responses

State

Drivers

Pressures Impact

e.g. Clean Production, Public Transport, Regulations, taxes Information, etc.

e.g. Ill health, Biodiversity loss, Economic damage

e.g. Transportand Industry

e.g. PollutingEmissions

e.g. Air, Water, Soil quality

23

A performance indicator

Emissions of ozone precursors, EU15

target

24

The link between indicators and the policy process – distinguishing the differences

and improving relevance

1 2 3

Indicators linked to

quantitativetargets

Indicatorslinked tostated

objectives

Indicators linked to

policy intentionsor public

expectations

25

What are scenarios?

Scenarios are archetypal descriptions of alternative images of the future, created from mental maps or models that reflect different perspectives on past, present and future developments.

26

Measuring is Not Knowing:The Marine Environment and the

Precautionary Principle

‘The enormous number of papers in the marine environment means that huge amounts of data are available, but …we have reached a sort of plateau in …the understanding of what the information is telling us …. We… seem not to be able to do very much about it or with it. This is what led to the precautionary principle, after all – we do not know whether, in our studied ecosystem, a loss of diversity would matter, and it might’.

Marine Pollution Bulletin, Vol 34, No. 9, pp. 680-681, 1997

27

Precautionary principle in assessments

•Levels of proof: Assessments for public policy making need lower levels of proof than normal good science

•Multidisciplinary approaches: improve the quality of an assessment by considering aspects of the problem from different perspectives

•Early warnings: successful prevention of environmental impacts and associated cost needs early warnings

Levels of proof - some illustrations

Beyond all reasonable doubt

reasonable certainty

balance of probabilities/evidence

strong possibility

scientific suspicion of risk

negligible/insignificant

29

Organisation through Interest Groups

On each of

the 33 servers

across Europe

30

CIRCLE Library Service

31

Data Flows in EIONET

National Layer

EEA Warehouse

European Layer

Inform

ation R

etrieval System

Data access and visualisation

E2RCReportsand Reference Centre Concept

32