tool #65. how to use visual aids and present quantitative data · tool #65. how to use visual aids...
TRANSCRIPT
TOOL #65. HOW TO USE VISUAL AIDS AND PRESENT QUANTITATIVE DATA
1. INTRODUCTION
Many interventions deal with a complex range of issues and highly technical subject
matter. Illustrations and visual aids can provide both expert and non-expert readers with a
clear overview of the problems and their drivers, policy objectives and solutions, as well
as the different steps in the analytical process.
This tool presents three visual aids which are particularly relevant for policy
interventions: problem trees, objective trees and intervention logic diagrams. These tools
can help illustrate complex ideas and facilitate common understanding and better
communication both inside the Commission and with external stakeholders. The text
below also presents tips on how to present quantitative data clearly.
2. PROBLEM TREES
What are problem trees?
A problem tree is a highly effective communication tool that helps to demonstrate the
need for intervention. It is used to visualise the identified problems, the interactions
between these problems, their underlying drivers and likely consequences. The output is
a graphical presentation of problems arranged according to ‘causes’ and ‘effects,’ joined
by a core, or focal, problem. The problem tree should provide a simplified but robust
representation of the reality. It also encourages a logical, comprehensive and coherent
narrative and structure to the analytical process and report.
How to create problem trees
The graphs can be created in MS Word, Excel, equivalent open-source tools or
specialised software such as DoView, MS Visio, Smart Draw or Lucidchart.
Box 1 shows an example on disclosure of non-financial and diversity information by
certain large companies and groups (amending Council Directives 78/660/EEC and
83/349/EEC)766. The links between the problems, their drivers and the consequences can
be clearly seen. For example, poor quality financial information is a problem that is
caused, inter-alia, by a lack of incentive for companies to disclose such information.
Problem trees can also usefully illustrate what is outside the scope of the EU
intervention; for instance because not all of the problem(s) can or should be addressed by
EU level action. The example in Box 2 on the initiative on e-invoicing in public
procurement767 illustrates this:
766 http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013SC0127&from=EN
767 http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013SC0222&from=EN
337
Box 1. Problem tree on the disclosure of nonfinancial and diversity information by
certain large companies and groups
Insu
ffic
ient
ince
nti
ves
for
com
pan
ies
to h
ave
div
ersi
fied
Boar
ds
Insu
ffic
ien
t ch
all
en
ge o
f se
nio
r m
an
agem
en
t deci
sion
s by t
he
Board
Insu
ffic
ient
div
ersi
ty
of
vie
ws
wit
hin
the
Boar
d (
group
thin
k)
Tra
nsp
are
ncy
: in
suff
icie
nt
qu
ali
ty o
f n
on
-fin
an
cial
info
rmati
on
Dis
close
d info
rmat
ion i
s not
suff
icie
ntl
y m
ater
ial, a
ccu
rate
,
tim
ely
, cl
ear,
com
par
able
, an
d
reliab
le
Tra
nsp
are
ncy
: in
suff
icie
nt qu
an
tity
of
non
-fin
an
cial
info
rmati
on
.
~ 9
4%
of
EU
lar
ge c
om
pan
ies
do n
ot
dis
close
any
non
-fin
anci
al i
nfo
rmat
ion
(incl
udin
g div
ersi
ty)
Reg
ula
tory
Fai
lure
:
AD
req
uir
emen
t
inef
fect
ive.
Leg
al
fram
ework
frag
men
ted,
wit
h
sign
ific
ant
dif
fere
nce
s
amongs
t M
ember
Sta
tes
Mar
ket
Fai
lure
: In
suff
icie
nt/
unev
en i
nce
nti
ves
for
com
pan
ies
to d
iscl
ose
non
-fin
anci
al i
nfo
rmat
ion
des
pit
e an
incr
easi
ng
dem
and f
rom
sta
keh
old
ers
Com
pan
ies
perc
eiv
ed a
s n
ot
suff
icie
ntl
y a
ccou
nta
ble
an
d
un
met
info
rmati
on
dem
an
ds
from
civ
il s
oci
ety
Less
eff
icie
nt
mark
ets
: in
ves
tors
m
ay f
ail
to b
uild r
elev
ant
non
-
finan
cial
info
rmat
ion i
nto
thei
r
dec
isio
n-m
akin
g p
roce
ss/t
ake
info
rmed
dec
isio
ns
Low
er
com
pan
ies'
perf
orm
an
ce
Pote
nti
al inef
fect
ive
over
sigh
t by
the
Boar
d.
Man
agem
ent
and B
oar
ds
may
not
fully
inte
grat
e non
-fin
anci
al i
ssues
into
busi
nes
s an
d s
trat
egy
Insu
ffic
ient
iden
tifi
cati
on o
f
risk
s an
d o
pp
ort
unit
ies
in
gener
al
Imp
acts
on t
rust
in
busi
nes
s an
d t
he
mar
ket
Subop
tim
al a
lloca
tion o
f
cap
ital
Sin
gle
Mar
ket
pote
nti
al f
or
sust
ainab
le g
row
th a
nd e
mp
loy
men
t not
fully
exp
loit
ed
Dri
ver
s
Pro
ble
ms
Conse
quen
ces
338
Box 2. Problem tree on e-invoicing
3. OBJECTIVE TREES
The "objectives tree" is a graphical illustration of the different layers of objectives. Box
3 illustrates how the different levels of objectives can be represented graphically. It is an
example of the Directive 2009/142/EC relating to appliances burning gaseous fuels
(GAD)768.
Box 3. Objective tree concerning appliances burning gaseous fuels.
General objective Specific objective
Better protect health and safety of
users of gas appliances and fittings as
well as to ensure their appropriate
performance
Ensure that economic operators have
adequate safety and performance relevant
data available on the framework conditions
Ensure clarity of the requirements
Improve the fair playing field for
economic operators in the gas
appliance sector
Ensure legal clarity regarding the application
of more specific EU product harmonisation
legislation
Simplify the European regulation
environment in the field of gas
appliances and fittings
Ensure that legislation is up to date
Ensure clarity of the scope
768 http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52014SC0151&from=EN
339
4. INTERVENTION LOGIC DIAGRAMS
Intervention logic diagrams/result-chains are synthetic and diagrammatic representations
of the different steps of the analytical process. They can help to improve the coherence of
the report, by making the links between problems, objectives and options more explicit.
Box 4 shows how a diagram can be used to map the problems and objectives and relates
to the initiative laying down maximum authorized weights and dimensions for certain
road vehicles circulating within the Community769.
Box 4. Intervention logic diagram: authorized weights of road vehicles
Problem/Driver Objective
Problem (Part 1)
Certain limits on weights and dimensions
set by the Directive constitute obstacles to
energy efficiency improvements of road
vehicles and to intermodal transport
operations
General objective (Part 1)
To facilitate energy efficiency of road
transport and intermodal transport by
revising certain limits on weights and
dimensions of vehicles while maintaining
the balance with the requirements of
infrastructure maintenance, road safety
and the protection of the environment.
Root cause 1
Certain maximum weights and dimensions
prevent the market uptake of more
aerodynamic electric hybrid trucks and
reduce the attractiveness of certain coach
services.
Specific objective 1
To enable the market uptake of more
aerodynamic electric hybrid trucks and to
increase the attractiveness of certain coach
services.
Root cause 2
Certain maximum weights and dimensions
have not kept pace with the technical
development of intermodal transport and
containerisation.
Specific objective 2
To enhance the development of
intermodal/combined transport
Problem (Part 2)
The Directive is not applied in an effective
manner.
General objective (Part 2)
To improve the internal market for road
transport by providing a fairer playing
field for hauliers.
Root cause 3
Lack of common and dissuasive
enforcement methods.
Specific objective 3
To ensure better enforcement of the
maximum weights and dimensions across
the EU.
It is also possible to map objectives with policy options (in the IA context) or
implemented measures (in the context of retrospective evaluation) – Box 5 contains an
example from the IA accompanying the initiative laying down maximum authorized
weights and dimensions for certain road vehicles circulating within the Community. To
address the problem and its drivers, and given the substantial list of possible measures
769 http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013SC0108&from=EN
340
needed, it was proposed to form policy packages (PP) of measures for further assessment.
This example shows an overview of measures proposed in three policy packages.
Box 5. Intervention logic mapping objectives to policy options (packages).
In evaluation, the intervention logic provides a description or diagram summarising how
the intervention was expected to work. Usually this shows how different
inputs/activities/outputs triggered by the EU intervention were expected to interact to
deliver the promised changes over time and ultimately achieve the objectives. The
intervention logic should also consider external factors which may influence both the
performance of the EU intervention, or generate the same type of effects.
The evaluation intervention logic is a dynamic took and it is quite normal for it to
develop further during the evaluation project as assumptions are tested. The final
intervention logic may look quite different to the initial starting point, providing key
input to the evaluation on how actual behaviours and performance differed from original
expectations.
Box 6. Describing causal pathways
In evaluation, the concept of intervention logic combines elements of similar
methodologies. Among others, it involves ideas from the Logframe approach and the
Theory of Change. Both approaches are used to describe causal pathways in interventions
and the mechanisms that enable them.
Theory of Change is usually oriented towards the objectives that are intended to be
achieved under specific conditions. This concept is employed to show how activities
towards the objectives on the one hand and outcomes on the other can be connected in
various ways and through different mechanisms. Theories of Change are often used in
early stages of an intervention and serve as a tool to engage stakeholders and the persons
who will implement the intervention. They usually try to capture as many factors in the
341
environment of the (potential) programme as possible that might facilitate or hinder the
intended change.
The Logframe approach is an analytical tool to reconstruct the causal pathways between
needs, overall and specific objectives, resources, activities, output, outcomes, and the
impact of a specific intervention. External factors beyond the control of the intervention
are included in the model in case they influence its effects.
The differences between the Logframe approach and the Theory of Change are not clear-
cut and they are used in various definitions. In application, their meaning often overlaps.
Some examples of intervention logics used in evaluation are presented below. Others can
be found on the evaluation collaborative space770.
Intervention logic for the Birds and Habitats directives:
770 https://myintracomm-collab.ec.europa.eu/networks/IAWG/eval_network/SitePages/Home.aspx
Intervention logic for health and nutrition claims
DRIVERS PROBLEMS General
OBJECTIVES Operational OBJECTIVES
INPUTS: Actions by
public authorities, food business operators, EFSA
OUTPUTS
RESULTS
Lack of truthful, clear, reliable and useful information for consumer on the nutritional status of foods
Foods contain false health claims and thus mislead the consumer
Problems with the free circulation of foods bearing nutrition and health claims in the Internal Market
No level playing field for food business operators for making health and nutrition claims on foods
High level of consumer protection from untruthful and misleading claims and facilitating consumers' healthier food choices
Improve the free movement of foods bearing nutrition and health claims within the Internal Market
Guarantee legal certainty for food business operators on the use of nutrition and health claims
Ensure the same level of scientific evidence for the substantiation of nutrition and health claims
Ensure that only authorised nutrition and health claims may be used on the EU market
Ensure that nutrition and health claims are coherent with nutritional advice
Regulatory framework requiring scientific substantiation of nutrition and health claims
Absence of scientific criteria for making nutrition and health claims on foods
Different rules governing the use of nutrition and health claims made on foods in different Member States
A harmonised list of permitted nutrition claims
Conditions for the use of nutrition and health claims
Restriction of claims on alcoholic beverages
Nutrient profiles the establishment of nutrient profiles
A harmonised list of authorised health claims
Ensure fair competition in the area of foods bearing nutrition and health claims
Procedure for the establishment of the list of authorised health claims by the collection of national lists by Member States, scientific evaluation by EFSA, adoption of the list of authorised health claims by Commission
Procedure for authorisation of new health claims via application by Food Business Operators, scientific evaluation by EFSA, authorisation by Commission
- Harmonised use of nutrition and health claims
- Better alignment of nutrition and health claims with nutritional advice by public health authorities
- Increased trust of consumer in nutrition and health claims
- Reduction of misleading nutrition and health claims on foods marketed in the EU
- Smoother functioning of the internal market
- Legal certainty for food business operators when using health and nutrition claims
- Fair competition in the area of foods bearing nutrition and health claims by creating a level playing field for food business operators
5. PRESENTING QUANTITATIVE DATA
How to present quantitative data?
Data can be presented in the text, in a table, or pictorially as a chart, diagram or graph.
Any of these may be appropriate for demonstration. Detailed tables should be put in an
annex, with a summary in the main text for demonstration purposes. In general the
following is a guide to presenting numerical data:
Text alone should not be used to convey more than three or four numbers.
Sets of numerical results should usually be presented as tables or graphs: (a) well-
presented tables and graphs can concisely summarise information which would be
difficult to describe in words alone; (b) on the other hand, poorly presented tables and
graphs can be confusing or irrelevant.
The text should always include mention of the key points in a table or graph. If it does
not warrant discussion it should not be there. You should ensure the message in the text
is consistent with that in the table.
There are two main types of graph:
• Line graphs can show more detail than bar charts. They should be used when the
horizontal axis represents a continuous quantity (such as time).
• When the horizontal axis is a qualitative factor - such as countries, products, etc. -
bar charts are natural.
Tables used for demonstration purpose are intended to be assimilated quickly by the
reader. They should be clear and well-presented and reduce numbers to relatively few
significant digits.
It is preferable not to use overly large tables: (a) if the information is all necessary, split it
into manageable components; (b) omit any column which can be readily calculated from
data in other columns. Less relevant categories can be combined.
344
Box 7. Tables versus graphs
In general, tables are better than graphs for giving structured numeric information. For
instance:
In general, graphs are better than tables for indicating trends, making comparisons, or
showing relationships.
Good practice
Tables and graphs should be self-explanatory: the reader should be able to understand
them without detailed reference to the text; users may well pick things up from tables or
graphs without reading the whole text.
The titles of the tables/graphs should be informative
Rows and columns of tables or axes of graphs should be clearly labelled: what is
measured, where, units, timeframe, source (as relevant)
345
Box 8. Clear tables
A first version sent to the IAB:
Revised version:
Comparing options in the IA report
In order to be an effective aid to decision-making, the IA reports need to present a
credible set of alternative policy options, and their comparison should clearly outline
the advantages and disadvantages of each option. Visual aids can be helpful in this
regard.
A variety of presentational options are frequently included in multi-criteria analysis
which can reduce efforts for presenting the comparison of options (in particular these
tools frequently offer useful complementary views such as ranking by type of
stakeholder).
Radar charts
Radar charts can be used to compare options. They are particularly well-suited to show
outliers and commonalities or when one option outranks another; they are less well-
suited to show trade-offs. To make any sense, you need at least 5 quantifiable criteria.
The order of criteria is important to convey meaning.
346
Box 9. Example of visual aids used to compare options
Example 1: IA on communication analysing the specific challenges for the space components
of GMES (Global monitoring for environment and security). “In quantitative terms, the
options can be compared most easily on the basis of their generated NPV in the period under
consideration.”
Is this the most appropriate kind of graph? The IA report presents the graph below
instead: different styles of graphs can
enhance clarity and generate interest
Example 2:
347
Box 10. Example of radar charts