ex post evaluation of cohesion policy 2007-2013 –...

Transport Evaluation Economics

DG REGIO 20/3/2015

Ex post evaluation of Cohesion Policy 2007-2013 – Work Package 5 Transport

Inception Report 20th March, 2015

Prepared by: ............................................................. Checked by: /s/ .................................................................. John Finnegan Jacopo Signorile Regional Director Senior Advisor Approved by: ............................................................. Richard Redfern Regional Director

Rev No Comments Checked by Approved by

Date

1 Initial Draft for Steering Group of 18th February JS RR 10.2.2015 2 Revised Draft following Steering Group meeting JS RR 10.3.2015 3 Edits following comments from DG REGIO JS RR 20.3.2015

Ground Floor, Grand Canal House, Upper Grand Canal Street, Dublin 4, Republic of Ireland Telephone: +353 (0) 1 238 3100 Website: http://www.aecom.com This document has been prepared by AECOM Limited for the sole use of our client (the “Client”) and in accordance with generally accepted consultancy principles, the budget for fees and the terms of reference agreed between AECOM Limited and the Client. Any information provided by third parties and referred to herein has not been checked or verified by AECOM Limited, unless otherwise expressly stated in the document. No third party may rely upon this document without the prior and express written agreement of AECOM Limited.

Section A: Introduction ..................................................................................................................................................................2

Section B: Task 0 Inception.........................................................................................................................................................10

Section C: Task 1 Analysis of Achievements.............................................................................................................................13

Section D: Task 2 Analysis of Major Projects ............................................................................................................................24

Section E: Task 3 Project Case Studies .....................................................................................................................................42

Section F: Task 4 Catalogue of Challenges ...............................................................................................................................52

Section G: Task 5 Member State Case Studies..........................................................................................................................57

Section H: Task 6 Seminar...........................................................................................................................................................66

Section I: Task 7 Reporting .........................................................................................................................................................72

Section J: Project Management and Quality Control ................................................................................................................76

Appendix: Long List of Major Projects .......................................................................................................................................83

Table of Contents

Section A: Introduction

AECOM/KPMG Ex post evaluation of Cohesion Policy 2007-2013 – Work Package 5 Transport 2 Capabilities on project: Transport Evaluation Economics

A1 Introduction The Directorate General for Regional and Urban Policy of the European Commission (DG REGIO) is currently undertaking an ex-post evaluation of Cohesion Policy programmes between 2007 and 2013. As part of this work DG REGIO has commissioned a study referred to as “Work Package 5: Transport” from AECOM and KPMG. A kick off meeting for this study took place in Brussels on 10th December, 2014. A meeting to discuss a draft of this Inception Report between the consultants and the Steering Group for the study took place on 18th February, 2015. This updated Inception Report was prepared following these meetings and other preliminary work by the consultants. After some brief background remarks on the role of transport investments in Cohesion Policy this introduction outlines the overall approach that AECOM and KPMG will take to this study and describes the structure of this Inception Report, which sets out in detail the methodology that will be adopted for this study.

A2 Context Between 2007 and 2013, the Cohesion Policy of the EU invested €347 billion of European Regional Development Fund (ERDF), Cohesion Fund (CF) and European Social Fund (ESF) resources to strengthen economic, social and territorial cohesion across European regions and Member States. In the context of Cohesion Policy, infrastructure and accessibility play an important role in promoting regional competitiveness. This is reflected in an €82 billion strong allocation of ERDF and CF resources to the transport sector which amounts to about 25% of total available resources.1 The European Union’s Transport Policy supports sustainable mobility for people and goods, and aims at ensuring efficient, safe, and environmentally friendly means of transport2 throughout the European Union. Amongst other things, it promotes the development of pan-European transport networks through the Trans-European Network – Transport (TEN-T) programme.

Of 309 Operational Programmes negotiated with the European Commission (ERDF and CF), 238 included support to the transport sector. Of these, the majority were implemented at a national level (representing 67 percent of the total allocation of funds). Almost all of the funds were allocated to regions under the Convergence objective (95 percent of the total allocation of funds), reflecting EU, national, and regional strategies, and attempting to balance support across road, rail, and sustainable means of transport. Green mobility was of particular importance in urban areas.

The largest share of the allocations went to road and railway projects (about 50 and 30 percent of the sum respectively). Almost €41 billion were made available for road infrastructure (12 percent of total Cohesion policy resources), while about €24 billion were allocated to Railway (6.8 percent of the total). Other areas of focus were urban and multimodal transport projects, ports, airports, and inland waterways. Of particular importance were investments in support of TEN-T, which represented more than 10 percent of total Cohesion Policy investments, or €38 billion.

The figure below provides an overview of allocations to the transport sector across Member States, as planned in the beginning of the 2007-2013 programming period:

1http://ec.europa.eu/regional_policy/activity/statistics/2007_transport.pdf 2 White Paper on Transport

Section A: Introduction

http://ec.europa.eu/regional_policy/activity/statistics/2007_transport.pdf


Transport Allocatons as a % of Total Cohesion Policy

Source: European Commission

As stated in the Specifications of this Tender, almost 77 percent of resources allocated to transport were invested in major projects (i.e. projects costing more than €50 million). These major projects require formal approval by the European Commission on the basis of feasibility studies and cost-benefit analyses. The European Commission approved 349 of such projects in the period 2007-2013. However, only 125 major projects had been completed by April 2014.

It must be acknowledged that projects in the transport sector are amongst the slowest in delivering outputs and results due to their scale and complexity. Previous evaluations carried out by the European Commission have highlighted a number of pitfalls not only in the delivery of transport projects (e.g. delays and cost-overruns), but also in their long-term financial sustainability.3 While the recent economic and financial crisis is likely to have negatively affected project implementation, pre-crisis studies point at similar occurrences. The questions of what the main underlying issues are, why they occur, and what can be done to mitigate their effects will repay further study.

As regards performance, transport projects are mainly reviewed in terms of core output indicators defined by DG REGIO, namely number of transport projects, kilometres of new or reconstructed roads and railways, and additional population served with improved urban transport4. Results can be captured by looking at indicators such as reduced journey time, increased numbers of journeys or passengers and reductions in accidents. However, these are often difficult to estimate.

3http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/expost2006/wp5a_final_report.pdf 4 For a more detailed list, please consult the core indicators identified in the EC Evaluation Guideline “Concepts and Recommendations”

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/expost2006/wp5a_final_report.pdf


In February 2015 “Work Package Zero” of the ex post evaluation of Cohesion Policy was completed5. Amongst other things, this work collated and corrected the latest data available from Member States on output indicators. For example, the Tables below show achievements in the construction of new roads according to the corrected 2013 Annual Implementation Reports (AIRs).

Table A1: Km of new roads Member State Km

Poland 1,595

Hungary 443 Czech 293

Portugal 290 Spain 279

Germany 274 Romania 162 Bulgaria 141 Slovakia 72 Estonia 70

Italy 61 France 28

UK 12 Sweden 9 Slovenia 3

3,733 Table A2: Km of new TEN-T roads

Member State Km

Poland 834 Bulgaria 141 Romania 140 Portugal 137 Hungary 114 Czech 103 Spain 88

Greece 88 Germany 71 Slovenia 52 Slovakia 39

UK 6 Cyprus 3

1,817 Against this background, careful evaluation of Cohesion Policy investments in the transport sector is a compelling task for at least two reasons. First, it will provide important information as to whether European support contributed to the creation of a comprehensive transport network. This is relevant for accountability to European stakeholders 5 European Commission “Final Report: Ex-post evaluation of Cohesion Policy programmes 2007-2013. Work Package Zero: Data collection and quality assessment” February 2015


and taxpayers. Secondly, this evaluation will for the first time drill down into the long term financial sustainability of investments, by looking at practical instances at the project level. It will review assumptions behind financial analysis and demand forecast (e.g. demographic trends, consumer behaviour, tariffing), informing on systematic biases and compiling a catalogue of common pitfalls/challenges and mitigating actions. This is all the more true if the size of allocations to transport projects, the scope of targeted population, and their importance in underpinning economic development are taken into consideration.

A3 Overall Approach to this Study The main objectives and requirements of this evaluation are to:

• Identify the main achievements of cohesion policy in all areas of transport related infrastructure;

• Analyse to what extent the EU support contributed to the creation of comprehensive transport networks; and

• Identify weaknesses of Cohesion Policy intervention in transport and explore directions this policy should take in the future.

The evaluation will be carried out following a methodological approach specifically tailored to answer the evaluation questions relevant to each of the Commission’s objectives. The hypotheses to be tested will emerge from the very early stages of the evaluation exercise (Task 1) and will be tested through a deductive approach of investigation through project and Member State case studies.

Each of the tasks has a strong interrelation with the others. In addition, the structure of the proposed team will assure the opportune sharing of information and results within the team, and externally, with the Commission and the main stakeholders.

A summary of our approach is shown and described below.


Figure A1: Overview of our approach

Task 1: Analysis of

achievements

Task 2:Review financial

analyses

Task 3:Project case

studies

Task 2a

Task 2b

Task 2c

Task 4:"Catalogue of Challenges"

Task 5:Case studies of CP

Task 6:Seminar

Task 7:Final Report

15 Member States

20 major projects

10 project case studies

6 Member States

Task 1 will provide an overview of Cohesion Policy. An analysis of the achievements of Cohesion Policy in the field of transport over the 2007-2013 programming period will be carried out. This task encompasses a quantitative analysis of data on inputs and outputs of Cohesion Policy in the field of transport, and a qualitative aspect: tailored stakeholder interviews designed to elicit the views of informed commentators. In order to carry out the qualitative analysis, a list of 15 Member States that has been proposed.

Task 2 will be based on a selected list of 20 major projects. We will examine the documentation on these and provide a judgment on demand and financial assumptions of their ex-ante financial analysis. We will identify the key assumptions and theoretical foundations for demand and financial analysis within these major projects. The findings provided by this task will be used in tasks 3 and 5

Task 3 will provide case studies of 10 completed major projects. The 10 projects will be selected from the 20 addressed in Task 2 and reviewed in more depth. The hypotheses established as to the main causes of observed bias and forecast inaccuracies will be explored further in these 10 case studies. These case studies will be of projects that have been fully implemented and that are operational, to ensure that evaluation can consider the full implementation and initial maintenance life cycle stages. The analysis of the case studies will be presented in the Seminar organised in Task 6.

The evidence from task 2 and 3 will be included in Task 4. We will prepare a “Catalogue of Challenges” that sets out descriptions of the most common problems encountered in the financial analysis of major projects in the field of transport. The draft catalogue will be used as a basis to discuss challenges in the Seminar (Task 6). In addition, conclusions from the Seminar will be incorporated into the Catalogue.


Task 5 will propose 6 Member States to the Commission for use as case studies from the list of 15 identified in Task 1. Task 5 will adopt a similar approach and detailed methodological approach as Task 3. The analysis of the case studies will be presented at the Seminar organised in Task 6.

Task 6 is a Seminar to discuss and deepen analysis the emerging findings of the case studies prepared in Tasks 3 and 5 and their tentative conclusions. Conclusions from the Seminar will be incorporated in the “Catalogue of Challenges” prepared in Task 4.

Task 7 will summarise the evidence of the overall analysis in a Final Report.

The table below presents the objectives, input and output of each task.

Table A1: Overview of Tasks

We will implement a mixed method approach, using a combination of several elements such as desk research, broad literature review, interviews with different kind of stakeholders, and benchmark analysis.

The success of the work therefore depends on the following:

• Setting the analysis at an appropriate level of detail in order to identify weakness of Cohesion Policy intervention in transport;

Section headingPage heading

Task Objects Input Output

1 Prepare and present an analysis of the achievements and the contribution of Cohesion Policy in the field of transport over the 2007-2013 programming period.

Propose 15 Member States where conduct a qualitative analysis through in-depth interviews.

From Commission;

EC documents;

Literature review;

Desk search;

Interviews.

Task 1 Tasks 2 and 5

2 Propose a list of 20 selected projects and provide a judgement on their demand and financial analysis.

Describe and analyse patterns across the selected projects, indicate possible individual and systematic methodological errors in financial analysis.

Project application documents from Commission;

Output from Task 1;

EC documents;

Desk researches;

Internal and external database;

Benchmark analysis.

Tasks 2a and 2b Task 2c

Task 2c Tasks 3, 4 and 5

3 Analyse further and in more depth 10 project case studies from among 20 projects examined in Task 2.

Output from Task 2;

Stakeholders interviews;

Policy documentations;

Programme documents;

Desk research on available information sources (e.g. articles, news);

Internal and external database.

Task 3 Tasks 4 and 6

4 Organize a "Catalogue of Challenges" that describes the most common problems occurred in the financial analysis of major projects in the field of transport infrastructure and solutions to avoid them.

Output from Tasks 2 and 3;

Conclusions from Task 6.

Task 4 Tasks 6

5 Identify 6 Case Studies to explore in detail the contribution of Cohesion Policy to national and EU transport policy.

Output from Task 1;

Desk search;

Interviews.

Task 5 Tasks 6

6 Organize the evidences collected under Tasks 3 and 5 and prepare the basis for a discussion analysis on the emerging findings with a wider group of stakeholders.

Output from Task 3 and 5;

Initial evidences from Task 4.

Task 6 Tasks 4

7 Prepare a Final Report with the evidence of the overall analysis.

Output from Task 1, 2, 3, 4, 5 and 6.


• Maximising the benefit of information already available and the experience of the team members;

• Maximising the benefit of knowledge already held by previous analysis; and

• Assuring the quality of the analysis and its interpretation through strong quality control measures.

A4 Structure of this Document Sections B to I of this Inception Report describe in detail the approach and methodology that will be adopted for each of Tasks 0 to 7. Section J then describes the project management arrangements that are in place and the timetable for completion of this work.

Section B: Task 0 Inception


B1 Introduction The study has started with an Inception Phase. This comprises a kick-off meeting, which took place on 10th December 2014, preparation of a draft Inception Report, discussion of the draft Inception Report with the Steering Group for the Study on 18th February, 2015 and some preliminary research to refine the methodology for the study. B2 Task Methodology

Task 0.1: Kick-Off Meeting The initial activity following the confirmation of this assignment was a Kick-off Meeting with DG REGIO on 10th December 2014. This covered: • Introducing key members of our team and confirming channels of communication and points of contact; • Confirming the evaluation objectives, methodology and discussing DG REGIOs views on the scope of our

proposal and any necessary revisions; • Agreeing the format of the Inception Report to be prepared; • Presenting initial ideas on project case study selection and agreeing and confirming the approach to be

followed; • Discussing the fieldwork arrangements to ensure effective and efficient interaction with officials and experts in

the Member States and the European Commission; • Confirming the programme of delivery and dates for any subsequent meetings; and • Facilitating the exchange of relevant data/information, particularly relating to the 50 candidate projects. Task 0.2: Inception Period Following the Kick-off Meeting we revised the originally proposed methodology and approach in the light of the discussion that had taken place, and followed up certain issues with DG REGIO. The main issues considered during this Inception Period were: Major project Case Studies • Review of overview data and detailed project documentation for 58 major transport projects selected as a “long

list” by DG REGIO; • Confirmation of the completion status of these projects through our respective office networks; and, • Selection of 20 major projects for further examination as set out in the description of Task 2 below. Member State Case Studies • Finalisation of a list of 15 Member States for consideration in Task 1b. Stakeholder Consultation • List of project and Member State contacts/stakeholders; • Template designs: Including content, structure and order; and, • Seminar agenda update and approach. Commission Management • The programme of work: Confirming agreed deadlines for key deliverables (Inception Report, Interim Reports, 2

Project Pilot Case studies, Draft Final Report and Final Report);

Section B: Task 0 Inception


• Dates for Steering Group meetings were also confirmed to ensure that critical delivery pathways are defined. This will promote our proactive approach to managing this commission, minimising risk. The results of this work are set out in the Section of Project Management below; and

• Updated resourcing plan (working days per person/task): To reflect the agreed scope of works, techniques to be adopted and programme of work.

Task 0.3: Inception Report A Draft Inception Report was prepared for DG REGIO and circulated by DG REGIO to the Steering Group for this study. This draft was discussed with the Steering Group at a meeting on 18th February. Following this meeting some further investigation of the data sources for this study took place and this Inception Report was finalised.

Section C: Task 1 Analysis of Achievements


C1 Context and Approach The first task in this evaluation is to prepare and present an analysis of the achievements of Cohesion Policy in the field of transport over the 2007-2013 programming period. This analysis is split into two sub-tasks. These are Task 1a, an analysis of inputs and outputs and Task 1b, an in-depth analysis of contribution.

Task 1a requires the consultants to gather the information available on the inputs and outputs of Cohesion Policy in the field of transport. In particular, we will assemble and present:

• The inputs, in terms of financial support given, of Cohesion Policy in the field of transport analysed by sector and Member State;

• The outputs of Cohesion Policy in the field of transport, analysed by sector and Member State; and,

• An analysis of the role of major projects in these inputs and outputs and a collation of the geo-reference data for these major projects allowing mapping of their geographical distribution.

The “In-depth analysis of contribution” requires an analysis of the contribution of the outputs of Cohesion Policy in the field of transport, i.e. the contribution of the transport investments part-funded by Cohesion Policy to the development of comprehensive transport networks in the EU. This analysis will include their contribution to the development of the TEN-T, and the transport networks of individual Member States. This task also requires the evaluator to develop an overview of the relevant trends and developments in the areas of transport in Member States, which formed the context for the operation of Cohesion Policy. Finally, the evaluator is to form a judgement, based on the results of this work, of the main strengths and weaknesses of Cohesion Policy in the field of transport infrastructure between 2007 and 2013.

The proposed approach to these two sub-tasks is outlined below:

C2 Task 1a Input/Output Analysis Methodology

Completion of this task requires the assembly and analysis of the best available data on:

• The total financial input of Cohesion Policy to transport projects;

• The financial input of Cohesion Policy to “major” transport projects, i.e. those transport projects costing more than €50m where Cohesion Policy funding is subject to specific approval by the Commission;

• The output of all transport project part funded by Cohesion Policy; and

• The output of “major” projects.

The 2007-2013 programming period for Cohesion Policy has been marked by significant attention to gathering relevant data for subsequent evaluation during the course of the Programme. In particular, the Managing Authorities for each Operational Programme funded by the Cohesion Fund and the European Regional Development Fund have been obliged to submit detailed Annual Implementation Reports (AIRs) to the Commission on these Operational Programmes. These AIRs contain detailed information on the absorption of financial support by the Operational Programmes and the progress of projects.

Section C: Task 1 Analysis of Achievements


In 2013, DG REGIO of the Commission procured a study to “gather and quality assess physical data reported by Managing Authorities in their Annual Implementation Reports and to assess the feasibility of gathering financial data broken down by priority theme to the NUTS2 and 3 levels of region”. This study formed “Work Package 0” of the ex post evaluation of Cohesion Policy 2007-2013. The results of Work Package 0 were made available to the consultants carrying out the other Work Packages in February 2015. This consists of a database covering each Operational Programme funded by Cohesion Policy. For each Operational Programme the database sets out corrected data on the indicators reported for that Programme. The range of data available for Operational Programme is set out in Figure C1 below: Figure C1: Data Available from Work Package Zero Master table column

Master table headings

A Consecutive numberProgramme reference

B MSC Programme_TitleD CCI_EuRefE Progr. ObjectiveF Priority Axis CodeG Priority Axis name

Indicator referenceH Indicator level

I FundJ Indicator name EnglishK Core and common indicators official name

L Indicator Specification

MSelected core and common indicators relevant to WP Zero

N Core Indicator CodeO Future Indicator CodeP Significant indicator tag

Q ExPost Evaluation ThemeR Unit of measurement

WP Zero Corrections

SConsistency between indicator programme definition and EC guidelines

TIndicator definition provided by the programme [if not in line with EU recommendations]

U WP Zero selected core reported in SFC 2013

VExplanation for inconsistency between programme and the EC definition

W Explanation for target correction (2013)X Explanation for achievement correction (2013)

2012 valuesY 2012 Corrected TargetZ 2012 Corrected Achievement

2013 valuesAA 2013 Corrected TargetAB 2013 Corrected Achievement

Work Package Zero has also collated certain data on the major projects included in the Operational Programmes. A database is available that sets out for each Major Project some or all of the following details (Details of outputs were not provided for many of the Major Projects in the Annual Implementation Reports):

• Operational programme reference; • Sector;


• Major project reference; • Reporting status; • Physical work started (Yes/No); • Total investment; • Community Amount; • Total investment cost; • EU assistance; • Certified expenditure; • Km of new roads; • Km of new TEN-T roads; • Km of new railways; • Km of reconstructed railways; and, • Additional population served by waste water projects.

During the Inception Phase of this study, DG REGIO has provided AECOM and KPMG with relevant data on transport investments funded by Cohesion Policy from the Commission’s own information systems.

Task 1a.1 Capture Data in a usable form As discussed above, this sub-task requires data on:

• The total financial input of Cohesion Policy to transport projects; • The amount of this input that relates to “major” transport projects; • The output of all transport projects part funded by Cohesion Policy; and • The amount of this output that arises from “major” projects.

Data on each of these will be collated as follows: The financial data available for this study covers the allocation of Cohesion Policy Funding to transport projects by Member States. Specifically, the data extracted from the “SFC” information system and provided by DG REGIO sets out - for every operational programme and for each year from 2008 to 2014 – the allocation by the Member State in question of the total Cohesion Policy funding for the Operational Programme to the different “Priority Themes” defined by the Commission. This data will be collated by Member State for the relevant Priority Themes. The resulting data will produce a picture of the allocation of Cohesion Policy funding to Transport projects by each Member State and the evolution of this allocation over time. In a similar way the data available from the SFC information system covers the allocation of funding by Member States to Major projects. This data gives details of the Priority themes for these Major projects, so allows identification of the allocation of funds to Major projects in the area of transport. The relevant data from this dataset will be collated to give values for the allocation of funding by each Member State to Major projects in the area of Transport, analysed by the different sub-sectors of transport covered by the data. As described above, Work Package 0 has developed a comprehensive data set of the output of all of the Operational Programmes funded by Cohesion Policy. This can be interrogated to produce a picture of the output of the total set of transport projects funded by Cohesion Policy. The type of analysis to be performed is described in the next sub-section.


During the completion of Work Package 0 it was not possible to extract data on the outputs of many of the Major Projects from the Annual Implementation Reports submitted by Member States. However, the output does include a definitive list of all Major projects. DG REGIO has provided a list of all of the Major projects in the field of transport. As described under Task 2 below, AECOM and KPMG have determined which of these projects are complete as of early 2015. In order to assess the contribution of Major projects to the total output of Cohesion Policy funded projects in the field of transport it will be necessary to measure the output of these complete projects. During the Inception Phase the potential sources of this information were investigated and discussed with DG REGIO. It was decided that the planned outputs of these projects, as documented in the applications for funding held by DG REGIO would constitute a suitable measure of the output of these projects and allow a comparison of the scale of output of Major transport projects with the output of all Cohesion Policy funded transport projects. Geocoding data for each of these major projects will be identified and recorded at this stage. Where this data is not readily available from the application file, AECOM’s own mapping resources will be used to determine the relevant Geocoding information. Although a certain amount of work will be needed to assemble this data, the resulting data sets will be relatively simple. (The expected form of these dataset, and the analysis and presentation to be carried out is described further in Task 1a.2 below.) As a result, the most effective format for the capture and analysis of this data will be as a set of Excel files. Once this has been completed, we will carry out standard reviews and cleaning of the data, eliminating obvious errors and corrupted data, repairing these faults where possible.

Task 1a.2 Presenting the scale of transport investments funded by Cohesion Policy The next task will be to prepare an overview of the relevant aggregate values for the inputs and outputs of Cohesion Policy over the period. A number of basic tables and graphics will be a useful context for later work and will provide an introduction to any future presentation of the work of Cohesion Policy in the area of transport.

With respect to inputs we will prepare tables and graphics of allocations of Cohesion Policy financial support to transport projects, analysed by Member State and by priority theme. The changes in this allocation over the programming period will also be identified and presented. In addition any interesting groupings of Member States exhibiting similar behaviour or policies that can be identified will be presented.

We will also present the output of Cohesion Policy funded investments in the field of transport analysed by Member State and Core Indicator. At this stage the emphasis will be on simple, uncluttered presentation of key data.

Task 1a.3 Analysis of the role of major projects in achieving these outputs

The data collated on the financial support allocated to completed Major projects, and the outputs of these Major projects will be presented and analysed in a similar way to the data on Cohesion Policy transport projects as a whole. These results will be compared with the values in Task 1a.2 to demonstrate the role of Major projects in achieving the outputs of Cohesion Policy in the field of transport.

C3 Task 1b In-depth analysis of contribution As described above, Task 1a will produce a quantitative description of the aggregate inputs and outputs of Cohesion Policy in the field of transport. It will also draw conclusions on the operation of Cohesion Policy and areas for study or improvement to the extent that this is possible based purely on the quantitative data.

Task 1b goes on to analyse the contribution of Cohesion Policy to transport as a whole in a selection of 15 Member States. This task can be split into two elements:


• High level quantitative work, where the inputs and outputs of Cohesion Policy in the field of transport are compared with the total resources, from all sources, devoted to transport investment in Member States and the outputs achieved; and,

• The qualitative analysis necessary to identify the relevant trends and developments in the area and so understand the contribution of Cohesion Policy to transport in Member States and the EU as a whole.

The best central sources of information on transport infrastructure in Member States are the transport statistics published by Eurostat. Eurostat provides information on Motorway and Railway infrastructure across the EU, and changes in the amount of infrastructure from year to year. However this information is limited. In particular the Eurostat data only goes up to 2012. In addition Eurostat does not give any information on financial inputs into transport infrastructure. Limited information is available from other central sources. The International Transport Forum publishes statistics on total expenditure on transport infrastructure by each Member State, but this information only goes up to 2011. Other bodies such as the World Bank appear to rely on Eurostat information. This level of data, supplemented by enquiries in specific Member States made through AECOM and KPMG’s networks of offices will provide sufficient data on the scale of transport infrastructure stock and investment in Member States to allow an assessment of the role of Cohesion Policy.

For the qualitative part of this task, desk research and interviews with key stakeholders will be carried out to determine:

• The contribution of Cohesion Policy on the provision of comprehensive transport networks in individual Member States and in the EU as a whole. These are two related but separate questions. Each Member State needs a specific transport network that meets the needs of its residents, which will be determined by patterns of settlement and economic activity and by travel and trade with other Member States. The EU as a whole also needs a transport network that meets the needs of EU citizens and businesses for cross frontier travel. Ultimately this network must serve and drive the future full integration of EU Member State economies as well as meeting current needs. The priorities of individual Member States and the EU as a whole can diverge. For example a key road or rail corridor across the EU could have relatively little relevance for one of the Member States through which it passes. Cohesion Policy can fund transport investments in a Member State that meet the transport needs of the Member State, that meet the needs of the EU as a whole or that contributes to both networks. In all three cases the funds provided by Cohesion Policy will be supplemented by public funds and possibly private funding in the Member State in question.

• The share of Cohesion Policy Funding in total spending on transport projects in Member States.

• The result of funding for transport will depend on a range of other trends and developments in the sector. These need to be explored to understand the contribution of Cohesion Policy in the area of transport and to identify any improvements to the operation of Cohesion Policy that are possible in this area; and,

• Greater understanding of the contribution of Cohesion Policy funded investments in transport on Member State and EU wide transport networks, and insights into the relevant trends and development in the field of transport will highlight strengths and weaknesses of Cohesion Policy. In addition stakeholders familiar with these areas will have insights into the strengths and weaknesses of Cohesion Policy from their own experiences and observations. It is required that these insights are collated and reported.

Our proposed approach to the requirements for these tasks is based on:


• Desk research across all Member States;

• Identifying 15 Member States for stakeholder consultation;

• Identifying key relevant stakeholders in each of these representative Member States;

• Carrying out investigative interviews with these stakeholders based on tailored questionnaires; and,

• Analysis of Cohesion Policy effects on transport in each Member State

These five steps are discussed in more detail below:

Task 1b.1 Desk Research The team will carry out a literature review to identify existing studies on the contribution of Cohesion Funding to transport in the EU. Likely sources include:

• The work of DG REGIOs evaluation network;

• Evaluations by Member States of individual Operational Programmes that include transport investments, and of the role of Cohesion Policy as a whole;

• Commentary by economic research institutes in Member States; and,

• Academic research in the fields of transport policy and regional development.

This material will be reviewed for key insights relevant to this task. This review will also highlight other issues that will have affected the development of transport networks over the period. Examples of such issues are:

• Emergence and roll out of new technologies such as ITS for roads;

• Adoption of relatively new modes such as High Speed Rail;

• Broader economic environmental factors such as investor confidence and the state of financial markets;

• Changing public tastes and attitudes to environmental issues and responsibilities;

• Public attitudes to tolls and access charges, and to private ownership or operation of transport infrastructure;

• Changing regulatory regimes and market structures in areas such as rail and air transport; and

• Long term trends in fuel costs. Task 1b.2 Selection of 15 Member States A preliminary suggested group of 15 Member State where consultations would take place was included in AECOM and KPMG’s technical proposal to carry out this study. This selection was discussed with DG REGIO at Kick-off. It was agreed that the group of 15 should be chosen to maximise the level of insight gained into the spending of Cohesion Policy funds on transport projects.

The selection of Member States should therefore favour those with the highest levels of spending on transport projects. This Task also presents the opportunity to gain sight into the Member States where we will be carrying out case studies of major projects, so these Member States should be included to the extent possible.

The total allocation of funds to transport projects in each of the 28 Member States is set out in Table C1 below:


Table C1: Cohesion Policy Funding Allocated to Transport Projects by Member State A selection of 15 Member States has been made by:

• Selecting the 13 Member States where the major projects chosen for Task 2 are located. These Member States are overwhelmingly those where the greatest amount of funding has been allocated; and

• Choosing two of the remaining Member States with high level of allocated funding (Portugal and Lithuania).

The set of Member States resulting from this process are presented in Table C2 below:

Country €m Austria 6Belgium 60Bulgaria 2,022Croatia 229Cyprus 81Czech Republic 7,822Denmark -Estonia 692Finland 40France 1,081Germany 3,157Greece 6,255Hungary 6,679Ireland 84Italy 4,184Latvia 1,141Lithuania 1,570Luxembourg -Malta 148Netherlands 50Poland 25,655Portugal 1,713Romania 5,471Slovakia 3,383Slovenia 946Spain 8,226Sweden 77United Kingdom 498Cross-Border Programmes 1,058 82,328


Table C2: Member States Selected for Task 1

Country Bulgaria Czech Republic Estonia France Germany Greece Hungary Italy Lithuania Poland Portugal Romania Slovakia Slovenia Spain

Task 1b.3 Identification of Key Stakeholders The core study team is already familiar with the Member States suggested above. In addition the core team will be able to draw on the knowledge of their colleagues in the office networks of AECOM and KPMG. The project management Section of this Inception Report identifies a set of local correspondents drawn from the office network of AECOM and KPMG who will be attached to the study team to provide expertise on local conditions and institutions. This expertise will be used to help plan and execute these interviews and the various case studies and other interviews involved in subsequent tasks in this study.

A number of principles can be set out at this stage that will guide the selection of interviewees for this task:

• Member State Managing Authorities and Ministries of Transport will be highly knowledgeable and will certainly be included in the final lists of interviewees. However they may not be entirely objective in their assessment of the issues facing transport in their Member State, e.g. if institutional weakness of one of these bodies is a key issue in a particular Member State;

• In some cases civil society bodies representing transport users, industry, environmental concerns or cities will exist and will be able to give a valuable perspective on transport issues in a Member State;

• Academics who study the operation of Cohesion Policy or who specialise in transport matters will be able to provide a valuable overview of relevant issues in some Member States; and


• Many Member States will have non-governmental research institutes that track and comment on issues of economic and physical development. These will be a valuable source of insight where they are available.

To give an example for a specific Member State, consider Romania, which has benefited from extensive funding from Cohesion Policy and from international financial institutions for investments in transport infrastructure. Relevant stakeholders to consult for this task would include:

• The Ministry of Transport;

• The national Railway Infrastructure Manager;

• The national road authority;

• The JASPERS office in Romania which has advised the Romanian authorities on the development of major projects in the field of transport and on various horizontal transport issues;

• The World Bank office in Romania which has been providing finance and advice to transport stakeholders in Romania; and

• The consultant team currently finalising a national transport strategy for Romania.

Task 1b.4 Carrying Out Stakeholder Interviews Having identified the relevant stakeholders in each Member State the core team will carry out structured interviews. These will be based on a discussion guide prepared in advance to ensure consistency across Member States and comprehensive coverage. These interviews will elicit the views of informed commentators and stakeholders on the impact of Cohesion Policy transport investments in the Member States in question. Non-government and independent stakeholders such as academics and staff of independent research institutes will be interviewed to the greatest extent possible to gain an objective assessment. These interviews will cover:

• The contribution of Cohesion Policy transport investments to Member State transport policy. i.e. have they fulfilled priority transport needs for the Member State in question;

• The related but separate question of the contribution of these investment projects to European Transport Policy. In theory significant investments could take place in the TEN-T network in a particular Member State that might be of limited relevance to the specific needs of the Member State;

• The contribution of the projects to economic cohesion and regional development. e.g. Have the investments improved the transport links between a “Cohesion” Member State to markets elsewhere in the EU? Have the investments connected underdeveloped regions of the Member State to markets and opportunities elsewhere in the Member State or in the EU as a whole; and

• What other trends and contextual factors have been relevant in the development of transport networks over the period. These could range from the emergence of new technologies, to changes in sentiment in financial markets over the period.

Task 1b.5 Analysis of Cohesion Policy Effects on Transport in each Member State Finally the results of this work (the analysis of inputs and outputs using Commission monitoring data, the desk research and the interviews carried out in 15 Member States) will be drawn together into conclusions on:

• The contribution of Cohesion policy to the creation of the transport networks of individual Member States;


• The contribution of Cohesion policy to the creation of pan-EU transport networks, particularly the TEN-T network;

• The relative importance of Cohesion policy funding in these investments;

• Other developments and trends that have affected the development of transport networks in the programming period; and

• Based on the above, the strengths and weaknesses of Cohesion Policy in the field of transport policy and transport infrastructure.

Section D: Task 2 Analysis of Major Projects


D1 Introduction Task 2 of this study is an analysis of 20 Major Projects funded by Cohesion Policy. During the Inception phase DG REGIO has provided AECOM and KPMG with:

• A “long list” of 58 Major Projects in the field of Transport;

• The full documentation held by DG REGIO on these projects including the application for funding and the supporting documents such as Feasibility Studies; and,

• Summary data on major projects extracted from DG REGIO’s information system.

AECOM and KPMG used the network of correspondents in their offices around the EU to obtain up to date information on the state of completion of these projects. This information is summarised in a Table in the Appendix to this report.

20 major projects were selected from this long list based on the following criteria:

• At least one Major project should be selected from each Member State where a Major project or projects has been approved. Although the sample of 20 cannot be a statistically representative one, this will ensure that the 20 projects selected will illustrate the range of issues that arise with these projects across the EU;

• At least 10 of the major projects selected should be complete, to allow the preparation of 10 case studies of completed Major Projects in Task 3;

• At least one project should be chosen from each of the sector descriptions occurring in the long list: i.e. Motorways, TEN-T Motorways, National Roads, Railways, TEN-T Railways, Urban Transport, Regional Roads and Mobile rail assets. The aim of selecting these 20 Major Projects for further study is to identify and illustrate the issues that arise in the funding of transport investments, so it is appropriate to cover as wide a range of investment types as possible; and

• Once these criteria are satisfied, the selection should be geared to larger projects, to aim to cover the more significant Major Projects.

This approach to the selection of a sample of 20 Major Projects led to the set of projects set out in Table D1 below; this includes 2 motorway, 4 TEN-T motorway, 2 National Road, 1 Railway, 6 TEN-T Railway, 3 Urban Transport, 1 Regional/Local Road and 1 Mobile Rail projects.

Section D: Task 2 Analysis of Major Projects


Table D1: Major Projects to be Analysed in Task 2

Code Member State Description Type €m

2010BG161PR002 Bulgaria Completion of Trakia Motorway, Lots 2, 3 and 4 Motorways (TEN-T) 363

2011BG161PR003 Bulgaria Modernisation of the Railway Section Septemvri-Plovdiv – Part of Trans-European Railway Network Railway (TEN-T) 322

2009CZ161PR013 Czech Republic Track modernisation Votice – Benešov, near Prague Railway (TEN-T) 3512009DE161PR009 Germany Leipzig City Rail Tunnel – Modules 5 and 6 Railway (TEN-T) 1782009EE161PR009 Estonia Reconstruction of Ülemiste Juncton in Tallinn Regional/local roads 812010GR161PR007 Greece Completion of the Elefsina – Corinth – Patras road Motorways (TEN-T) 2,025

2009ES161PR012 Spain Madrid – Castilla – Valencia– Murcia High Speed Rail Line. Railway (TEN-T) 1,455

2012FR161PR002 France “Route des Tamarins” Motorway - Tranche 2 Motorways 1,091

2012FR162PR005 France Completion of the third phase of the Valenciennes tramway Urban transport 183

2009IT161PR020 Italy Completion of Line 1 of the Naples Metro Railways 1,577

2009IT161PR012 Italy Double tracking sections of the Palermo – Messina railway Railway (TEN-T) 3,269

2008HU161PR003 Hungary Budapest metro line 4, section i Urban transport 1,418

2013PL161PR028 Poland Passenger rolling stock to meet long term railway needs

Mobile rail assets (TEN-T) 492

2010PL161PR030 Poland Construction of A1 Toruń – Stryków section Motorways (TEN-T) 3,335

2011PL161PR001 Poland Modernisation of railway line E30/C-E 30, Kraków – Rzeszów section, Phase III Railway (TEN-T) 1,129

2011PL161PR032 Poland Integrated urban public transport system for Lublin Urban transport 131

2008RO161PR002 Romania Motorway Construction on TEN-T 7 , Cernavodă - Constanța Section Motorways (TEN-T) 848

2009RO161PR048 Romania Rehabilitation of national road DN Alexandria - Craiova National roads 456

2013SI161PR002 Slovenia Poljane by-pass of Škofja Loka (R1-210 stretch 1087, R1-210 stretch 1110) National roads 52

2009SK161PR001 Slovakia R1 Road: Zarnovica - Sášovské Podhradie Motorways 216 18,972

There will be a strong interaction between Task 2 and 3 according the following approach:

• Task 2 will provide an assessment of the assumptions used in the financial analysis. In cases where assumptions were not coherent or showed a significant variance from the mean figures, these projects should be considered for more in depth analysis to the carried out in Task 3; and

• Task 3 will review 10 completed projects in more detail. If our work in Task 2 indicates that incorrect assumptions have been made in the modelling and development of these projects we will estimate corrected forecasts and results based on corrected assumptions.

Projects to be analysed in Task 3 will be selected from the projects addressed in Task 2 for a more in depth analysis according to the following criteria (apart from those mentioned tender):


• Complex financial structure (i.e. blending of different EU funds; PPP structures; innovative sources of revenues; cross financing); and

• Very limited difference (best practice) and large differences (worst practices) between initial hypotheses and actual figures (i.e. in terms of Capital Costs, Operating Costs, traffic, interest rate, tariffs, etc.): as the projects are operating it will be possible to compare at least a few years’ worth of data.

The 20 selected projects will be the object of the analysis in Task 2a and 2b as described below. During the Inception phase a check was carried out of the availability of suitable data, this is described in Section D2 below. Sections D3 to D5 describe the detailed methodology for Tasks 2a to 2c.

D2 Availability of Data

AECOM and KPMG have been provided with access to the DG REGIO application file on each of the 58 Major transport projects from which this set of 20 have been chosen. During the Inception Phase AECOM and KPMG have ensured that the information available on these projects is sufficient to allow the methodology described below to be applied in practice. A data collection template was prepared to allow a systematic review of the application files for these Major projects to ensure that the methodology described in detail in Sections D3 to D5 below can be applied in practice. This template has been used to review the application files for all of the above Major projects on a pilot basis. This pilot review confirmed that the files contained sufficient information to apply the methodology proposed for this Task.

D3 Task 2a Judgement on Demand Analysis - Methodology A key element in any feasibility study for a transport investment is a forecast of user demand for the asset in question. Forecasts of future use drive the assessment of the likely benefits of an investment for travellers, the wider economic benefits of an investment, the environmental benefits of the investment and the likely level of fares, tolls and/or operating costs.

High quality forecasting of demand is therefore the cornerstone of high quality planning and appraisal of transport investments. This aspect of the evaluation is an excellent opportunity to draw lessons in this area that are revealed by this study. This will improve not only the effectiveness of Cohesion Policy, but the effectiveness of all transport investment by Member States. Demand forecasting depends in making high quality assumptions on:

• Demographics;

• Competing modes and alternative routes;

• Consumer behaviour, including the effects of particular measures (i.e. aiming to reduce congestion or travelling time, to increase traffic safety, shift from private car use to public transport, etc.);

• Affordability: how the analysis took into account the relationship between user charges and income level of the users – methods used, main parameters; and

• Tariffing: how user charges were planned to be set, including user/polluter pays principle and institutional mechanisms for price control in the relevant sector.

As specified in the invitation to tender we will gather and analyse independent information on each of these issues. This will allow an assessment of the quality of demand analysis undertaken by Member States when they developed


transport projects to the point where they could seek funding under Cohesion Policy. More importantly this task will identify areas for improvement and best practices in this respect, so that this key activity can be done better by all Member States in future.

Task 2a.1: Data Acquisition The relevant data on demand analysis is embedded in a number of documents such as the feasibility study itself, transport demand reports, and the cost benefit analysis. In each case the relevant material will be reviewed to obtain the relevant data. Care will be taken to ensure that where demand analyses were revised or updated, the last relevant exercise forms the basis of the assessment. Information gathering will focus on ensuring a comprehensive picture is obtained of all the data sources that were considered, those that were taken forward in the analysis, and the checks which were undertaken to ensure the relevancy and robustness of those datasets.

As described in Section D2 above we have carried out preliminary work to determine the availability of relevant data from the application files in the hands of DG REGIO. This has confirmed that the approach described here will be feasible. Detailed work in these projects may reveal gaps in the data available from DG REGIO’s files. This may have to be supplemented by one to one discussions with the technical staff who worked on the project to ensure all relevant information is captured. We would adopt a standard pro forma to record these discussions, allowing for sufficient free text entry to accommodate variations from expected responses. This will be carried out in conjunction with the Task 3 case study interviews to the maximum extent possible.

Task 2a.2 Data Analysis Assessment of the quality of the demand analysis must be based on a review of:

• The demand modelling approach adopted;

• The forecast economic and demographic variables that underpin any projection using the model;

• The pricing/charging regime proposed; and

• The consumer behaviour envisaged.

Task 2a.3 Assessment of the Demand Modelling Approach • Appropriate Modelling Techniques

The demand modelling approach required to achieve quality demand forecasts varies with project type. Simple econometric demand analyses may suffice, especially where network and mode change effects are not envisaged to be substantial. Where the reactions of the consumer are considered to be more complex, more extensive network based transportation planning models are required. For example, urban public transport projects require more complex modelling approaches, as a variety of consumer responses are possible, due to the availability of alternative modes and trip destinations. Based on our substantial experience in transportation modelling, we will assess the extent to which appropriate modelling techniques were used.

• Competing Modes and Alternative Routes

The assessment of any new transport infrastructure needs to start with an understanding of the likely market of that facility. Any transport facility will provide a route from a point ‘A’ to a point ‘B’ albeit with usually many connecting points en route. For private transport i.e. car or goods vehicle, there will typically be a number of alternative routes which could provide the same function. Any assessment should seek to understand the relative appeal of these alternatives, which will be a combination of: Travel time; Reliability of travel time; Travel comfort;


Route suitability (particularly in the case of goods vehicles); and Cost, which, for private vehicles, would include fuel, other operating costs, tolls and parking charges and any other charges such as vignettes for goods vehicles.

Public transport may also offer an alternative via either metro, bus or rail or some combination of those. These are relevant when considering an improved public transport facility. In either case, the relevant considerations are: Fare; Travel time; Frequency of service and other timetable issues; and Interchange time and facilities offered at interchange.

These issues would normally be dealt with by a formal modelling process. However, not all models are capable of dealing with the issues of mode choice adequately. In many cases variable demand modelling is required. A focus of the study will be to check that the modelling approach adopted for a study was appropriate and had the capacity to capture mode and route choices.

Task 2a.4 Economic and Demographic Variable Inputs

• Macro-Economy The 2007 -2013 Programming period was dominated by a deep economic recession that affected the economies of all Member States to varying degrees. A key requirement in determining the future financial stability of projects is to establish the extent to which any demand shortfall occurring to date is the result of the poor macroeconomic performance. This will require an analysis of the macroeconomic performance in the relevant Member State and the extent to which this could have diminished demand below that forecast by the models. There is substantial literature on the income elasticity of demand for travel that would inform such an assessment. 6 This will provide a valuable insight into whether unreasonable economic assumptions were made in the modelling process or whether renewed economic growth is likely to make good any shortfall.

• Demographics Clearly the demographics, or the quantifiable statistics of a given population, are fundamental in any work on transport forecasting. We would interpret, in this context, relevant demographics parameters as including: Location; Gender; Age; Car ownership; and Socio-economic group / income band.

The evolution of the age profile is key, as are the changing levels of car ownership, particularly for those countries starting from a relatively low base. These can see a dramatic turnaround as economies strengthen and a consequent lowering of the appeal of even quite high standard public transport options. Our approach would be to establish whether the demographic assumptions used were linked to official national or EU demographic projections that would in part underpin their validity. Additionally we will contrast the population growth rates used with prior trends to establish whether they represented a departure from previous experience.

6 See Glaister etc.


• Pricing/Charging Regimes The financial stability of projects is determined in part by the pricing of charging regimes. This is most obviously the case for road projects that involve tolling and public transport projects that involve service expansions, and thus increased patronage and revenues. However, a number of issues may arise: • The pricing and charging regimes originally proposed at the time the application for funding was made may

not have been implemented in full; and

• The consumer response (price elasticity of demand) envisaged by the modelling of demand may have been inaccurate, leading to, for example, a prediction of greater demand than was actually realised.

• Setting Prices and Charges We will review the proposed prices/ charges envisaged, the proposals for adjusting those charges over time and the institutional framework for setting charges.

• Consumer Behaviour and Affordability Any assessment will have included a consideration of traveller response to the new facility provided. There can be a tendency to overestimate the appeal of a new facility and it will be important to understand what assumptions have been made within the modelling or evaluation process about the elasticity of response for each potential element of the market, to the key levers identified above such as travel time and fare or cost. We will draw on our extensive knowledge of the time and price elasticities of demand to establish whether the assumptions made were realistic.

An assessment of affordability is a critical element in informing the pricing charging decision in terms of the consumer response and the capacity of policy makers to implement and sustain the envisaged level of charges. We will assess the extent to which the assessment of affordability complied with the relevant guidance and with best practice. Any assessment of the impacts of the proposed tariff regime will need to consider the real impacts of the proposed tariff on the various segments within the overall forecast population.

D4 Task 2b Judgement on Financial Analysis - Methodology The main aim of this task is to perform an in-depth and critical review of the financial analysis of each selected projects. The basis of the review will be the main items and trends that have characterised the major transportation infrastructure projects, the network developed and the providers of funds, i.e. investors and financial institutions, and the social, economic and regulatory framework of the different countries where the projects are located.

This task will refer to EU guidelines7 that illustrate the standard methodology for carrying out the financial analysis conducted by the project sponsors, leveraging on the Consultants’ extensive experience and database of transport infrastructure projects in Europe. According to EU guidelines, financial analysis should be accomplished through subsequent, interlinked, accounts with reference to:

• Investment costs; • Operating costs and revenues; • Financial return to investment; • Sources of financing; • Financial sustainability; and • Financial return to capital.

7 “Guide to Cost Benefit Analysis of Investment Projects”, July 2008.


There are some critical issues usually encountered in the financial analysis of transport infrastructure projects. Some examples of such issues, broken into Cost examples and Revenue examples, are set out below:

• Cost: the time lapse between project costing and construction period, construction delay, optimism bias, lack of experience with certain technologies (e.g. tunnelling), poor ground survey, etc; and,

• Revenue: using unrealistic rates and charges, unaffordable charges, not recognizing price elasticity, etc.

The following methodological approach is proposed in order to perform a review and a critical judgment on the financial analysis.

• Task 2b.1 will compare the assumptions used in the development of these projects with subsequent market developments and industry benchmarks from the professional experience of AECOM and KPMG. This task will also highlight the main risks affecting the financial performance of the transport infrastructure projects;

• Task 2b.2 will review the financial analysis, identifying and verifying the main assumptions directly influencing financial performance of each selected project. The tasks will be performed in accordance with market trends, best practices (Task 2b.1) and EU guidelines; and

• Task 2b.3 will review the sensitivity analysis undertaken for each project. This exercise will first identify the

‘critical variables’ (i.e. those that could have the greatest impact on a project’s financial performance) and then verify whether these were tested sufficiently at the outset. The projects’ financial risks analysis will also be reviewed with the aim of assessing the probability distributions for the critical variables. Finally, we will provide an overall judgment on financial analysis according with market trends, best practices (Task 2b.1) and EU guidelines.

Task 2b.1: Market and benchmark analysis on key financial assumptions

The first task encompasses the collection of historical data and forecasts on similar projects using information gathered by some international databases and other available public sources, literature on specific aspects of project development (for example cost overruns) and in depth knowledge from our own experience), and experience in infrastructure projects. The aim of the market and benchmark analysis includes:

• Identifying the key assumptions and the theoretical foundations used in the main transportation projects that could influence financial performance as well as errors and risks; and,

• Establishing benchmark values / expectations for the key assumptions.

The results of the analysis will be used to test the assumptions in financial analysis carried out in the selected projects (Task 2b.2). Indicative examples of the main assumptions which will be the object of the analysis are shown below.

• Recent trends and expected evolution in the financing of major transport projects (i.e. role of PPP/PFI, using of national funds, etc.);

• Size of the total investment costs (define the tenor of the investment and the additional costs occurred during the project life cycle);

• Source of financing (percentage of equity);


• General financial rates and financial conditions applied; • Funding gap method; • Revenues characteristics (toll revenues, guarantee, etc.); and • Major players and best practice.

The Team will set up a dataset in order to present the trend scenario for the chosen variables. The dataset will be used to identify relevant levels of variance against the hypotheses in the project documentation provided by Member States.

The Team will provide a synthesis of the results of the market and benchmark analysis. It will include the preparation of graphical summaries of key trend data. The results of the analysis will be useful in addressing the review of the financial analysis (Task 2b.2) and in comparing the assumptions and the output to the general practices. The main evidence of the tasks will be used also as a base of discussion in the seminar (Task 6).

Task 2b.2: Review of the financial analysis

This task will examine the documentation provided by the Commission for the different projects (i.e. the results of the feasibility studies; cost benefit analysis; an analysis of the environmental impact; the financing plane) and the preliminary classification carried out in the selection of the 20 projects.

The aim is to perform a critical assessment of the financial analyses encompassed in the project applications and verify the main assumptions directly influencing financial performance. The Team will verify the correctness of the data and calculations in the financial analysis, identifying differences from standard practice as found in the research conducted in Task 2b.1 (in particular market and benchmark analysis and literature reviews). The Team will also analyse the consistency and robustness of the key assumptions made in the financial analysis compared to the dataset developed in the previous task. Attention will be given to the prevailing credit market conditions, investor and financial institutions’ requirements.

The analysis of the main assumptions used in the financial analysis, will encompass a thorough assessment on the followed aspects:

• Macroeconomic conditions (especially interest rates, inflation, other economic parameters, etc.); • Type of investments (physical characteristics, duration and relevant typologies of risks); • Benchmark values on main assumptions; • Sector and Country / Region (specific framework, legal context, etc.); and • EU guidelines.

As an example of the EU guidelines, the table below shows the useful check-list to conduct a financial analysis.


Table D2: Project appraisal check-list, according to EU guidelines

The Team will perform a critical assessment of the assumptions that can directly influence financial performance as detailed below focused on the following main aspects:

Total investment costs and schedules

Investigation areas - Investment costs;

- Duration of the construction period and schedule of the investment;

- Potential risks.

In the first step, the team will perform a detailed analysis of the different costs affecting the construction phase. We will identify the typology of the project, the physical characteristics, the duration of the construction period and the relevant typologies of risks that could affect the financial performance of the project during construction phase8.

The Team will run a detailed analysis of the different costs to be borne for the implementation and realisation of the projects. We will then consider the financial costs related to the use of debt in order to finance the construction of the infrastructure on the basis of hypotheses (i.e. financing conditions9).

8 Infrastructure project investment costs generally include:

- Fixed investments: in general the largest component of total investment. They can include land, buildings, equipment, extraordinary maintenance, etc.

- Start–up costs: this class of costs includes research and development costs, costs related to the implementation phase (including the feasibility study), costs of advisory services, etc.

- Changes in working capital over the entire time horizon.

FINANCIAL ANALYSIS - Have depreciation, reserves, and other accounting items which do not correspond to actual flows been

eliminated in the analysis? - Has the determination of the cash flows been made in accordance with an incremental approach? - Is the choice of the discount rate consistent with the Commission’s or Member States’ own guidance? If not,

why? - Is the choice of the time horizon consistent with the recommended value? If not, why? - Has the residual value of the investment been calculated? - In the case of using current prices, has a nominal financial discount rate been employed? - In the case of revenue generating projects, has the ‘amount to which the co-financing rate applies’ been

identified in accordance with - EU regulations (Art. 55 Reg. 1083/2006)? - Have the main financial performance indicators been calculated (FNPV(C), FRR(C), FNPV(K), FRR(K))

considering the right cash-flow categories? - If private partners are involved, do they earn normal profits as compared with some financial benchmarks?


We will analyse the original Feasibility Study and the project schedule during the construction period. The investment scheduling will be reviewed and compared with the evidence of the analysis carried out in Task 2b.1, as one of the critical issues in infrastructure projects is the additional budget due in case of delay in construction period.

The Team will also review, at a high level, the construction contracts and agreements that could affect the financial performance of the project identifying potential risks in this phase.

Total operating costs and revenues

Investigation areas - Operating and Maintenance Costs

Operating Costs

Routine Maintenance

Major maintenance/renewal

Other costs

Financial Costs

- Other revenues.

The Team will perform a detailed analysis of the different costs affecting the operational phase of the project. Generally, in infrastructure project, the assumptions related to operating costs are categorised as:

• Direct production costs: consumption of materials and services, personnel, maintenance, general production costs;

• Administrative and general expenditures; and

• Other costs.

In particular, we will focus principally on the main relevant costs in an infrastructure transportation project, i.e. maintenance costs (frequency, values, etc.) and labour costs. The Team will also investigate the evolution of each typology of costs throughout the period of analysis and analyse the trends in EBITDA and EBIT. The main assumptions (for example inflation factor or frequency of maintenance costs) will be compared with the result of the market and benchmark analysis.

In addition, the Team will use its experience in negotiating with financial institutions in order to verify the verifiability of the financial cost assumption considered in the financial analysis. Finally, the Team will consider in the analysis additional revenues not covered in Task 2.a.

9 The main debt service costs considered will be (list not exhaustive):

- Term sheet contains pricing; - Interest rate; - Hedging costs; - Fees – Upfront fee, commitment fee and agency fee; - Base Case cover ratios need to be met – debt sizing; - Repayment costs.


Sources of financing:

Investigation areas - Public Funding

European Union

National/Regional

- Private Funding

Equity

Revenues

- Funding gap method

The Team will review the different funding sources used to finance overall investment costs. The analysis will take into account the operative, technical, and geographical specificities of the project. Within the framework of EU-co-financed projects10, the main sources of financing are:

• Community assistance (the EU grant);

• National public contribution (grants or capital subsidies at central, regional and local government level);

• National private capital (i.e. private equity under a of Public Private Partnership (PPP) scheme); and

• Other resources (i.e. EIB loans, loans from other lenders).

In the current macroeconomic scenario, and taking into consideration the actual “infrastructure gap” in Europe, the use of Public Private Partnership (PPP) in infrastructure projects is considered a potential source of benefits and an important tool for financing investment projects when there is appropriate scope to involve the private sector.

Figure D1 Different types of delivery models

10 Guide to Cost Benefit Analysis of Investment Projects, July 2008


The Team will use its expertise in PPP and the results of market and benchmark analysis in order to identify good practice in the implementation of PPPs.

During the review, the Team will analyse the accuracy in the application of the funding gap method. We will review the assumptions used to define the residual value of the investment and calculation of the eligible expenditure.

The table below presents the main characteristics of the debt financing instruments.

Table D3 Characteristics of debt financing instruments

Debt financing instruments Characteristics

Corporate Loan facilities and/or bonds issued to capital markets; on balance sheet with full recourse; simple.

Leveraged Debt on balance sheet with recourse; structuring can include senior, subordinated and mezzanine tranches.

Project Project finance which can be on or off balance sheet with limited recourse.

PPP/PFI Project finance whereby the underlying asset is underpinned by contracts between public and private entities.


Cash flow analysis:

Investigation areas - Incoming flows

- Outgoing flows

- Temporal distribution of the flows

- Financial sustainability

The cash flow analysis is a critical point in infrastructure projects. In order to be sustainable, a project must have sufficient cash available to cover all stages of its lifecycle. It is essential to identify and secure the ongoing source of funds for the maintenance and operation of infrastructure. If this is not done there is a risk that the significant cash investment in building the infrastructure will effectively be wasted.

It will also be necessary to identify and estimate all cash flows, inward and outward, over the lifecycle of the asset. The incoming flows can include:

• Equity capital; • Revenues such as tolls or fares; • Loans; • Public Funding (grant); • EU funds; and • National / Regional funds.

The outgoing flows are related to:

• Investment costs (development and construction); • Operating and maintenance costs; • Renewal of components of the infrastructure with a shorter useful life than the asset as a whole; • Reimbursement of loans and interest paid; • Taxes; and • Other disbursements (i.e. dividends, retirement bonus, etc.).

The AECOM and KPMG Team will verify that the projects are in fact financially sustainable over its full lifecycle by considering all of these aspects. The risk of cash flow imbalances will be addressed.

Financial and performance indicators analysis

Investigation areas - Financial Net Present Value (FNPV)

- Financial Internal Rate of Return (FRR)

- Bankability index


Based on the previous steps, the Team will review and comment on the main financial and performance indicator of the financial analysis. As foreseen by EU guidelines and procedures, a particular emphasis will be placed on two financial indicators used to test the project’s financial performance:

• Financial Net Present Value (FNPV); and

• Financial Internal Rate of Return (FRR).

Financial indicators will be considered respectively in terms of return on investment cost (showing the overall financial performance of the investment independently of the sources or methods of financing) and return on national capital.

The Team will evaluate the methodology and the overall assumptions used for the financial return determination (according with EU guidelines). Particular attention will be given to the determination of the financial discount rate. To ensure the consistency of the rate, the Team will compare the rate used in the financial analysis with: (i) benchmark values; (ii) macroeconomic conditions; (iii) the nature of the investment; (iv) the sector; (v) the rates applied in similar projects located in the same Region/Country. In addition, the Team will focus on the main bankability indices generally used in evaluating main investment projects11.

Task 2b.3: Sensitivity and Financial risk analysis

Based on the operative hypotheses of the projects and the main issues derived from the market and benchmark analysis, the Team will review the sensitivity analysis of each project in order to verify whether the main variables, i.e. those that could have the greatest impact on a project’s financial performance have been appropriately “flexed”. Then we will verify the value and variability of the main financial and bankability indicators related to the variation of the key selected assumptions.

The identification of critical variables is the main task in a sensitivity analysis and it is important to take into consideration the characteristics of the project itself. Table D4 illustrates some examples for the identification of the variables used in the financial analysis.

11 They include (list not exhaustive):

- Annual Debt Service Cover Ratio (ADSCR); - Loan Life Cover Ratio (LLCR); and - Debt repayment period.


Table D4. Identification of critical variables

Categories Examples of Variables

Price Dynamics Rate of inflation, growth rate of real salaries, energy prices, change in prices of goods and services

Demand data Population, volume of traffic, specific consumption

Investment costs Duration of construction (delays in realisation), hourly labour cost, cost of land, cost of transport, cost of concrete aggregate, cost of rentals

Operating costs Prices of goods and services used, hourly cost of personnel, price of electricity, gas and other fuels

Quantitative parameters for the operating costs

Specific consumption of energy and other goods and services, number of people employed

Prices of outputs Tariffs, sales prices of products, prices of semi-finished goods

Quantitative parameters for the revenues

Hourly (or other period) production of goods sold, volume of services provided, productivity, number of users, percentage penetration of the areas served, market penetration

The Team will review the financial risk analysis and will analyse the probability distribution of the financial and bankability indicators of the projects. For each category of risks, the Team will verify the possible solutions proposed to mitigate them.

Generally, cost overrun risks in the construction phase and demand/traffic related risks in the operation phase will be the main typologies of risks in transportation infrastructure projects.

The Team will verify simulation in terms of the use of different typologies of funds in order to finance the infrastructure. Since the selected projects are completed and operational, the investment costs are not subject to variations. Hence, it is not necessary to perform a risk analysis.

In this phase we will use the results of the market and benchmark analysis (Task 2b.1), as well as the literature review and EU guidelines, in order to ensure that all potential risks have been taken into account. As an example, the table below shows an outline check- list for the review of a financial analysis.

Table D5: Project Appraisal Check List

RISK ASSESSMENT - Is the choice of the critical variables consistent with the elasticity threshold proposed? - Has the sensitivity analysis been carried out variable by variable and possibly using switching values? - Has the expected value criterion been used to evaluate the project performance? - Have ways to minimise the level of optimism bias been considered? - Have risk mitigation measures been identified?


Typical project risks are derived from four different areas:

• Construction/Delivery: Planning/consents, design, technology, ground conditions, protestor action, construction price, construction program, performance/availability;

• Operations: Operating cost, operating performance, maintenance cost/timing, raw material costs, insurance premiums/availability, vandalism;

• Macroeconomic: Interest rates, inflation, tax exposure; and • Revenue: Output volume, usage, toll levels, accidents, competition, force majeure.

At the end of the review process, the Team will furnish the European Commission with a critical judgment on the quality of the financial analysis, including the verifiability of assumptions.

In particular the Team will:

• Establish an Excel database of the information in the list of projects and the summaries established on each project;

• Make an assessment of the use of different kinds of Community financing for the projects (European Regional Development Fund, Cohesion Fund and TEN-T Programme);

• Make an assessment of the effects of user charges for the projects in terms of the revenue raised the effect on use of the projects; and

• Make an assessment of the effectiveness of the use of national funding.

D5 Task 2c: Horizontal analysis of assumptions - Methodology The primary role of Task 2c is to organise the evidence collected under Task 2a and 2b in order to describe and analyse patterns across the selected projects. Common patterns will be classified in macro categories and will be identified for Member States and for each sector, as well as across all Member States. Issues to be covered will include: • Demand analysis; • Revenues (traffic, availability payment, etc.); • Financial aspects (source of financing, EU funds, funding gap method, financial structure, etc.); and • Capital expenditures.

The Team will collect available evidence on “what works” and “what does not” in project analysis and compare the main evidence with the practices and results of the market and benchmark analysis conducted in the previous tasks. In this phase, an overview of the possible individual and systematic methodological or approach errors will be identified and will be used as basis for the discussions with the project’s managers/officials in the different Member States, which will take place in Tasks 3 and 4. As an example, a typical error in transport projects relates to difficulties in the demand analysis and the definition of the traffic volumes. According with the EU guidelines, “estimates of the financial viability of transport projects are heavily dependent on the accuracy of transport demand forecasts. Future demand is also the basis for economic and environmental appraisal of transportation infrastructure projects. The accuracy and reliability of data regarding traffic volumes, spatial traffic distribution and distribution between transport modes is crucial for assessing project performances. […] Estimating future travel demand entails forecasting not only the key macro drivers influencing the total demand (population, personal income and GDP) but also sectorial developments, since each sector contributes


to the total demand according to its specific characteristics. […] The price of the service provided is not the only determinant of travel demand”. Another important aspect referred to within the guidelines is the funding gap method and the patterns of financing a scheme. In particular, the Team will investigate the main typologies of funds used to finance the selected projects, what part of the expenditure is generally co-financed by the EU grants and in which phase of the project life (construction or operation) the EU grant occurred. The Team will highlight where errors have occurred in the financial analysis due to a process not being carried out in accordance with the legal and regulatory provisions. The Team will organise the evidence and the results of the analysis and will present patterns across the projects (analysed in Task 2a and 2b). Evidence will be identified for: • Member States, in order to identify geographical trends; and

• Sectors, in order to identify specific issues across different sectors.

The overall review of projects will be accompanied by a systematic organisation of information to allow them to be efficiently used in later stages of the evaluation. A statistical analysis will be conducted on the main evidence. The aim of the analysis is to provide a collective knowledge of how Cohesion Policy works in the transport sector. In this context, identifying errors could enable adequate preventive and corrective measures to be identified. Finally, the Team will analyse the results and draw preliminary findings on the available evidence. These findings will be refined further in Tasks 3 and 4.

Section E: Task 3 Project Case Studies


E1 Introduction As outlined above, Tasks 2 will identify the key assumptions and theoretical foundations for demand and finance analyses within major projects. The hypotheses established as to the main causes of observed bias and forecast inaccuracies will be explored further and in more depth through the use of the 10 Major project case studies. These case studies will be of projects that have been fully implemented and that are operational, to ensure that evaluation can consider the full implementation and initial maintenance life cycle stages.

The case studies will be used to identify the outturn costs and delivery programmes of operational projects, thereby verifying and/or challenging financial assumptions and methodologies. Using project level case studies will allow the Team to identify what happened in reality, explore reasons for observed variance and discuss potential solutions with key stakeholders. They will therefore complement the largely desk-based analysis undertaken in previous Tasks, providing a more in-depth evaluation.

Another key benefit of adopting the case study approach is the ability to determine the precise context for project delivery and operation. This will ensure that contextual changes, one potential explanatory factor for variance between forecast and outturn financial values, is accounted for within the evaluation. The key information sources include policy documents, programme documents, other publicly available written information sources (e.g. articles, news) beneficiary database and stakeholder interviews. Within our methodology outlined below the following delivery approach is proposed:

Figure E1: Approach to Project Case Studies

2 Pilot Selection

Task 3.1 Case Study Selection

Task 3.2 Template Preparation

Task 3.3 Member State Representatives

Task 3.4 Additional Information

Task 3.5 Discussion Guides

Task 3.6 Case Study Interviews

Task 3.7 Case Study Analysis

Task 3.8 Reporting

Feedback from Pilots to inform 8 case studies

Output: Recommended Case Studies

Output: Recommended Pilots

Output: Discussion Guides

Deliverable 4: Case Study Pilots

Deliverable 5: Interim Report

Section E: Task 3 Project Case Studies


E2 Task Methodology

Task 3.1 Project Case Study and Pilot Selection We will propose 10 major projects to the Commission for use as case studies, from the list of 20 used within Task 2. In selecting the 10 case studies we will seek to maintain a representative sample, reflecting the key factors: Member State coverage; size of projects, transport sector etc. As noted in the Specification and in Task 2, the 10 proposed case studies will all be fully implemented and operational. This will generate a potential constraint on the ability to select a sample that is fully representative of the major projects commissioned. We consider that an element of compromise will be required during the selection process, prioritising with the Commission the importance of:

• Member state coverage: reflecting the prominence of Poland and the Baltic states; and

• Transport priority theme: and the requirement for case studies that are not rail or road related.

We also recognise the importance of exploring particular or very specific assumptions/hypotheses that may emerge from Task 2. An example of this would be where the financial assumptions adopted at the ex-ante evaluation varied substantially from the market average, supporting further in-depth exploration. Similarly, the following criteria will also be used in selecting the 10 case studies:

• Where case studies have complex financial structure i.e. blending of:

o Different EU funds;

o PPP structures;

o Innovative sources of revenues; and

o Cross financing.

• Where there are observed small (best practices) or large differences (worst practices) between initial

hypotheses and actual figures in terms of:

o Capital Costs;

o Operating Costs ;

o Traffic;

o Interest rate; and

o Tariffs.

The case study task will start with the selection of 2 pilot case studies in agreement with the Commission. We support the adoption of a pilot approach, as this will ensure that the most efficient and effective methodology is developed. From the shortlist of 10 projects we will propose two to be completed as pilots. We will prepare two page summaries for these which will be provided to DG REGIO in support of our proposed choice. These two page summaries will be based on the data template presented in Task 3.2 and will provide a clear and defendable justification for the selection of each candidate project.


Task 3.2 Case Study Template Central to undertaking robust and focused case studies will be the preparation of a template, which will define the data and information to be collated. This will build on the project application documentation reviewed within Task 2 and ensure that data is recorded consistently across the 10 case studies. An initial template is presented below which was reviewed and discussed with the Commission at the kick-off meeting. Each case study template will follow a defined standard structure that will be central for the further case study operations. The template’s structure reflects the fundamental elements of Cohesion Policy, focusing on the key transport priority themes, indicators and financial data. As such it will be based on a double-view content: • First content is useful to understand the context in which the projects are implemented with a direct focus on the

involved sector(s); and • As the Tender specification requires, the second section will be concerned with the financial analysis of the

project. In particular, we will compare the initial assumptions and results of the financial analysis with the new analysis provided.

Case Study Template

Reference Number Unique reference number for each Case Study

Name Short name for each Case Study

Definition Summary of Case Study

Transport Sector

Motorway TEN-T Motorway Railway TEN-T Railway Mobile Rail National Roads Regional/Local Roads Urban Transport

Priority Theme Code 16 – 32, 52

Member State

Cohesion Fund/ERDF Cohesion Policy Support + Member State contribution

Managing Authority/Beneficiary

Financial Analysis

Feasibility Study:

Cost-Benefit Analysis: Data Sources

Financial Plan:


Ex-ante Assumption Ex-Post Value

Planned Investments Costs:

Operating Costs:

Maintenance Costs:

Financial costs:

Other Revenues:

Residual Value:

Eligible expenditure:

Financial Data

Funding gap rate:

Financial Net Present Value:

Financial Internal Rate of Return:

Financial indicators

Additional bankability index (as example DSCR, LLCR, etc.)

Task 3.3 Member State Representatives The third element of Task 3 will be the identification of key professionals and stakeholders to be interviewed for each case study. We will identify a range of professionals relevant to each case study to: • Ensure that viewpoints and opinions are obtained from different stakeholder perspectives on individual issues

(see discussion on triangulation under Task 3.6); and

• Address the potential risks of non-availability within individual Member States and/or case studies. As noted within the Tender Specification, a significant risk to this task will be the availability of individuals for interview, given the high turnover of staff in some Member States and the timing of project implementation within the 2007-2013 period.


A list of key individuals will be prepared for each of the case studies, reflecting the specific Member State, institutional structure and type of project delivered. In establishing case study lists we will utilise our extensive network of Member State staff, working in partnership with the Commission to identify the most relevant individuals. The type and number of representatives identified is function of each project’s features. We have assumed a total of 4 interviews will be undertaken for each Major project case study.

Task 3.4 Request Additional Information

To complement the ex-ante data and information reviewed in Task 2, and in preparation for the in-depth interviews, we will request the most recent project information for each case study from the individuals defined in Task 3.3. The template will be distributed to a single representative of the beneficiary, with a request to complete defined sections; the ex-ante information will already be inserted by our team. The request will also seek to obtain any additional project reports, such as interim evaluations, monitoring data and particularly financial updates to verify the data presented.

In particular, this activity will seek to obtain financial data on the outturn costs of case study implementation and, importantly, ongoing maintenance costs. This may also include documentation on the re-financing of projects, and the consideration of options; we consider this an important consideration given the economic turmoil experienced during the 2007-2013 period. The information received will be reviewed by the members of the core evaluation team visiting each project, to enhance the design of the discussion guide in Task 3.5.

Task 3.5 Discussion Guides

Our experience of undertaking in-depth interviews has demonstrated the benefits of developing detailed discussion guides. This includes ensuring that the interview process is adopted consistently between case studies and interviewees. Discussion Guides will be developed for each interviewee for each of the case studies, to ensure that they are specific to the proposed discussion. This will enable our evaluation and Member State experts to explore particular issues, assumptions or changes in project context in detail, through focused face-to-face meetings. We propose to structure the guides using the principles of the DG REGIO case studies12, for example mapping the requirements of Task 3 (as set out in the Specification) against the given areas of a complex case study.

Case Study Expectation Task 3 Requirement

Demonstrate how policy is decided • Projects as part of larger transport strategy

Demonstrate how policy is coordinated • Overview of the selection mechanisms

Demonstrate how policy is implemented • Institutional factors critical to reliable financial analysis • Impacts of financial aspects on project implementation • Solutions implemented to ensure financial sustainability

Determine the effects of policy

• Compare planned and actual financial figures • Reliability of assumptions • Financial sustainability of investment

12 Case studies in the framework of ex post evaluation, 2000-2006: Expectations and experiences of the DG for regional policy, DG REGIO 2009


Determine the reasons for defined effects

• Systematic bias • Implementation difficulties

AECOM’s head of Social and Market Research, Jo Christensen, will review all discussion guides prior to use, to ensure that they are of a very high standard; topic and question logic, flow, content and wording will be reviewed. An outline generic discussion guide is presented below to highlight the key issues that will be covered in the interviews. This is presented according to requirements of the specification.

Discussion Guide Section Example Questions

Is the project part of larger transport strategy

• What is the policy context for case study implemented within the individual Member State?

• What is the longer term strategy for transport within the Member State and case study region?

• What are the main transport sectors within the strategy and what is the relative balance in terms of investment?

• How is the case study project designed to contribute to achieving the longer term objectives?

• What other factors in the case study region are considered likely to have influenced the outcomes of the investment, and why?

Overview of the selection mechanisms and factors leading to project selection

• What are the selection mechanisms adopted to prioritise transport investment? • What were the options considered in regard to the case study and why were

they excluded? • Why was the case study selected for investment? • How well did the selection process work, and what improvements would be

adopted?

Institutional factors critical to reliable financial analysis

• What are the institutional factors inherent to financial analysis and forecasting? • Which of these are considered critical to the reliability of financial analysis? • Which factors influence the reliability and accuracy of demand forecasting and

analysis?

Compare planned and actual financial figures

• What were the planned and actual financial figures for the case study? • What were the main causes of any observed variance? • What was the perceived accuracy of assumptions adopted, and how can these

be enhanced for subsequent schemes? • What was the actual financial sustainability of investment? • How does this compare with the level of sustainability assumed and forecast at

the ex-ante stage? • What were the main reasons for observed variance?

Identify systematic bias

• What were the main causes of identified systematic bias in the calculation of financial forecasts?

• How have these been address (or not) by the Managing Authorities? • What impact did such bias have on the accuracy of the financial analysis

undertaken?


Implementation difficulties

• What were the main implementation difficulties experienced? • At which stages of case study delivery were these occurring and why? • What mitigation measures were adopted and how effective were they? • What were the impacts of these difficulties on the cost and programme of the

case study?

Impacts of financial aspects on project implementation

• How did the financial analysis and approach to finance management impact on the implementation of the case study?

• What were the main impacts on the case study? • How did the use of alternative funding sources influence the programme and

synergy of implementation?

Solutions implemented to ensure financial sustainability

• What solutions have been implemented to ensure financial sustainability of the case study?

• What are the initial views on how well these measures are working? • How accurate was the ex-ante forecast of financial sustainability and what are

the main reasons for variance?

Task 3.6 Case Study Interviews The core evaluation team, in association with our Member State experts, will arrange face-to-face interviews with the identified representatives. Interviews for each case study will be coordinated into a single day (or two days where the list of representatives is long) to maximise the efficiency of time, whilst being careful not to overburden staff. Face-to-face interviews are intensive activities and no more than four interviews will be arranged for each day. Three members of our team will attend each interview:

• A member of AECOMs transportation and evaluation team;

• A member of KPMGs transport and financial analysis team; and

• The relevant Member State expert.

We believe that this combination of staff will provide the right blend of expertise and knowledge through which to unpick the complex assumptions and project delivery issues. We propose to interview case study organisations separately. This will ensure that the findings can be triangulated between interviewees, and responses and opinions can be compared/contrasted. This is an important evaluation technique to ensure that conclusions are drawn using consensus and not the opinion of dominant interviewees. Our expertise in designing, undertaking and analysing qualitative data will be central to delivering this Task robustly.

Each interview will be recorded using digital recorders, but only with the prior approval of interviewees. The interviews will be transcribed for future reference. In addition key themes and facts emerging in the interviews will be identified, grouped and quantified to the extent possible. e.g. the extent to which certain options are held or problems are identified by a range or stakeholders will be quantified. This will be done by analysing the text of each interview using a “coding framework” identifying the key standard statements that might be made by stakeholders. This framework will be devised based on a review of the interview material and agreed with the Commission.

Task 3.7: Case Study Analysis The analysis of the project case studies will revolve around the testing of hypotheses established in Tasks 1 and 2. These will include the assessment of the main systematic errors in financial and demand assumptions, alongside


the main challenges in project implementation. Data from all sources collated will be analysed, and cross referenced, to address the defined evaluation questions:

• How reliable were demand and financial assumptions made during project development? • What are the main causes of systematic bias in financial analyses? • What are the main institutional factors in preparing robust and reliable financial analyses? • How do the above factors vary according to project context and the presence of a wider transport strategy? • What were the main causes of change in financial or demand assumptions? Evaluation questions that are not addressed fully through the in-depth-interviews, or where some uncertainty remains, will be identified; recognising that these remain as important as those questions that can be addressed in full. Based on the review of the demand and financial analysis (Task 2a and 2b), on common patterns identified for Member States and for each sector (Task 2c) and the results of the interviews in the previous sub-tasks, we will identify for each selected project:

• the correctness and the verifiability of the demand assumptions; • the correctness and the verifiability of the financial assumptions; • the financial sustainability and profitability; and • the main causes related to financial difficulties and possible solutions. We will recalculate the financial analysis based on the new assumptions provided by the market and benchmark analysis, interviews with main stakeholders, legal and regulatory provision. The new analysis will be provided in order to ensure the reliability of the results of the financial analysis. If necessary, we will be able to analyse and propose new solutions that will be put in place to ensure the financial sustainability of the project. We will also recalculate key indicators of the project (performance or bankability index). In this phase, the Team will focus on the implementation of the funding gap method according to the Commission’s working document13. The recalculation of the financial analysis will also include sensitivity analysis on critical variables identified in Task 2a and 2b. The new assumptions and results of the financial analysis will be compared with the initial analysis and will be summarised in the case study template (Task 3.2). The Team will analyse the main errors that occurred in financial analysis of the selected projects. We will identify factors (institutional or others) that are critical to producing reliable financial analysis. One of the most important categories of Cohesion-related errors is eligibility errors. These include: • Errors in the selection of the projects, when some criteria or rules on the procedure to be applied were not

observed; • Intangible costs, when costs presented in financial analysis not corresponding to the eligibility rules were

declared (i.e. cost categories not eligible, costs incurred outside the eligibility period, costs related to projects other than the co-financed ones, costs incurred in regions/areas which were not eligible and double declaration of the same costs); and

• Other eligibility errors involved, as example, incorrect calculation of the co-financing rate or declaration of recoverable VAT.

13 Commission’s Working Document No.4 “Guidance on the methodology for carrying out cost/benefit analysis”.


Another category of errors linked, as anticipated in Task 2, relates to the complex issue of revenue generating projects. According to the regulations, estimated revenues should be deducted when deciding the eligible expenditure to be co-financed. Errors occurred when revenues were not calculated or were deducted incorrectly and thus the co-financing rate was too high. The analysis of data will include the preparation of graphical summaries of key financial trend data, process mapping and also the preparation of case study narratives. Although we recognise the challenges in generalising findings from across the 10 case studies we will seek to draw conclusions on the types of systematic bias across Member States and the potential impacts on project implementation.

Task 3.8: Reporting The reporting of each of the 10 case studies will use the following outline table of contents: Section Contents

Introduction • Summary of case study major project

Project Description

• Description of the Major project o Scope o Location o Route sections + context o Function (speed limits for highway schemes) o List of scheme elements/design

• Funding source and levels

Project Development and Design

• Details of project development and delivery programme, including key design stages

• Presence and approach to national programme and prioritisation approach

Financial Analysis (Inputs) • Comparison of forecast and actual costs (implementation and

maintenance) • Reasons for variance in financial forecasts

Scheme Delivery (Outputs) • Summary of the delivered scheme

Scheme Benefits (Results) • Collation of evidence available on scheme results • Collation of evidence from interviews on contribution of scheme to

observed results The reporting will focus on addressing the defined evaluation questions and hypotheses being assessed, to ensure that it remains focused. It will combine descriptions of the outturn context and conditions of project implementation, with the detailed evaluation of the causes of bias or poor forecasting. A critical assessment of adopted or potential solutions will also be presented. The reporting will therefore combine qualitative narratives with supporting quantitative analysis. The reporting will be led by the core evaluation team, with support from the Member State experts to ensure that the presentation and analysis of local contextual issues are accurate.

The reporting of the 10 project case studies will be split into two elements. The first, within five months of the project inception, will be the presentation of the two pilot case studies. This will enable the Commission to confirm both the content and format of the case study summaries, whilst also confirming the detailed approach to the remaining 8 project case studies. The remaining 8 case studies will be reported within the Second Interim Report, to be delivered within 8 months of the project inception.

Section F: Task 4 Catalogue of Challenges


F1 Introduction This task consists of assembling a “Catalogue of Challenges” that sets out descriptions of the problems that beneficiaries, Managing Authorities and other agencies in Member States have encountered in the financial analysis of major projects in the field of transport infrastructure. More importantly, this Catalogue will also set out solutions to these problems. These solutions will be drawn from best practice observed during the study or developed by the study team as part of their work. The steps involved in completing this task are:

• Identify, understand and assess challenges arising in the 20 major projects for which the financial analysis was reviewed in Task 2;

• Document the results of this step by preparing the first draft of the “Catalogue”;

• Refine the understanding of these challenges based on the detailed case studies of 10 major projects carried out in Task 3. Identify any steps taken by the Member State to address the challenge and any changes in procedures and approach adopted by the Member State as a result of lessons learnt from the project in question;

• Update the draft Catalogue based on these findings, recording the nature of the challenge and observed best practice in addressing the challenge;

• For certain key challenges it will be useful to discuss their nature and the appropriate response in a structured way with national administrations as a group. The draft catalogue will be used as a basis to discuss these challenges in the Seminar organised as Task 6 of this study; and

• Document the results of this work in a final draft of the “Catalogue of Challenges” for presentation to the Commission.

Each of these steps is described in more detail below.

F2 Task Methodology Task 4.1 Initial Identification of Challenges Task 2 of this study will be a comprehensive review of the financial analysis carried out for 20 major transport projects. The review will highlight challenges that arose in the financial analysis of these projects. An example of a challenge that can arise in the financial analysis of major transport projects is the proper implementation of the Railway Directives. As a result of this reform of railway markets, railway infrastructure will be owned and operated by independent “Infrastructure Managers”. Train Operators will use this infrastructure to provide railway transport services to end users, in return for “track access charges” paid to the Infrastructure Manager. The calculation of these track access charges is also governed by the railway Directives. Estimating the future level of these charges and the treatment of these charges in an application for funding by an Infrastructure Manager are complex, relatively new issues to be addressed in the financial analysis of a railway infrastructure project.

Task 4.2 Initial Draft of Catalogue The problems identified as a result of Task 2 will be captured in a first draft of the Catalogue of Challenges. Box F.2 below sets out a possible format for the Catalogue. The headings are illustrated using the challenge encountered in the financial analysis of a major Polish rail project referred to in Task 4.1 above.

As indicated in the specification, the proposed format includes, for each Challenge:

Section F: Task 4 Catalogue of Challenges


• A non-technical summary of the Challenge. If the Catalogue is to be a useful tool for Member State agencies potential users will have to be able to quickly understand the nature of the Challenges described in the Catalogue, and so find the solutions that the Catalogue offers;

• A technical, and more detailed, summary of the Challenge. Having identified a Challenge in the catalogue that is relevant to their situation a user will need access to a fuller description of the Challenge, based on the analysis carried out in other projects;

• Potential warning sign for the Challenge in question to allow agencies to have an “early warning” of potential challenges;

• Implications of the challenge in question for the financial analysis of the project, drawing out how the financial analysis can appropriately address the challenge; and,

• Implications of the challenge in question for the future financial sustainability of the project, and how to reflect these in the analysis of the project.


Box F.1: Format of Catalogue of Challenges

Challenge

Heading 1. Rail Track Access Charges

Non-technical summary

EU Railway reform means that track infrastructure will be owned by independent Infrastructure Managers who will charge Train Operators for use of the infrastructure. These Infrastructure Managers will be the beneficiaries of funding for new rail infrastructure. The new rail infrastructure will generate future income for the Infrastructure Manager. The financial analysis of the project to build new infrastructure will have to reflect this in the calculation of the funding gap and the consideration of the economic sustainability of the project.

Technical Summary

Summary briefing for Managing Authorities, beneficiaries etc. on the relevant terms of the rail Directives covering:

• The required market structure for Member State rail sectors; • The role of Infrastructure Managers and their relationship with Member States; • The rules governing the calculation of Track Access Charges

Warning Signs The key warning sign that this challenge might be relevant is the existence of a proposal for a major rail investment in a Member State that is at a relatively early stage of implementing the Railway Directives.

Implications for financial analysis

Guidance drawn from existing best practice, including examples:

• How to estimate the access charge for a piece of rail infrastructure based on cost estimates and use forecasts; • How to incorporate forecast track access charge revenue into the financial and economic appraisal of a rail infrastructure

project; • How to calculate allowable expenditure and funding claimed taking this income into account

Implications for the future financial stability of the project

Guidance on calculating the future operation, maintenance and renewal cost of rail infrastructure and the likely contribution of realistic Track Access Charges to these costs.


Task 4.3 Refine the Understanding of the Challenge based on Case Studies A Challenge will initially be identified in the Task 2 review of the financial analyses of 20 major projects. The understanding of a Challenge, and the appropriate response to a Challenge, will be significantly deepened if the project in question goes on to be the subject of one of the Task 3 case studies. In fact the existence of a particularly relevant Challenge in a particular project could be a strong argument for including that project in the Task 3 case studies.

If a challenge is part of a Case Study project the study team will have the opportunity to speak to Member State officials with personal experience of the context for the challenge and how it was addressed. This will increase the amount and quality of guidance that can be included in the draft Catalogue entry outlined in Box F.2 above.

Task 4.4 Update the Draft Catalogue The additional insights gained from case studies will be captured by expanding the relevant entries in the draft Catalogue of Challenges. Task 4.5 Discuss Key Challenges in Task 6 Seminar At this stage the draft Catalogue of Challenges will contain sufficient material to form the basis of a constructive, efficient consideration at the Task 6 Seminar. The draft Catalogue, or the relevant parts could be circulated in advance of the Seminar and the Seminar could be used to agree and disseminate best practice on key Challenges. Task 4.6 Final Draft Catalogue of Challenges At this stage it will be possible to provide DG REGIO with a “final draft” of the Catalogue of Challenges. Once it has been reviewed and discussed with the relevant parts of DG REGIO, including the relevant Member State Desk Officers, the comments and suggestions of DG REGIO can be incorporated in the draft. A similar consultation of JASPERS could also be relevant. At this stage a comprehensive and authoritative Catalogue will be available which the Commission could provide to Member States as useful guidance. We could indicate that we will give an overview of the steps necessary to meet the challenge e.g.

• Desk officers to reinforce the solution with MS

• Incorporation in Technical Budgets

• Change in EU or national guidance;

• Improved guidance and or training by JASPERS; and

• Change in EU or national legislation.

Section G: Task 5 Member State Case Studies


G1 Introduction Task 5 will use Member State case studies to explore in detail the contribution of Cohesion Policy to national and EU transport policy. It is recognised that the availability of results data may be very limited, with many projects still to be fully implemented. The case studies will therefore explore, through in-depth interviews, the considered contribution of Cohesion Policy funding towards intended results. Data from Tasks 1-4 will be utilised in the design and delivery of this task, particularly the hypotheses generated in Task 1 on the contribution of Cohesion Policy to generating a comprehensive transport network.

G2 Task Methodology

Task 5.1: Member State Case Study Selection

We will propose 6 Member States to the Commission for use as case studies, from the list of 15 used within Task 1. Using this initial sub-sample will provide connectivity between Tasks 1 and 5, enabling the triangulation of findings across evaluation questions. In selecting the 6 case studies we will again reflect the key factors of the size of funding and transport sectors covered.

We recognise that the 6 case studies represent a relatively small sub-sample of Member States (just over 20% coverage) and therefore it may be challenging to ensure that all evaluation issues can be addressed to the same level of detail. The selection process will therefore balance the importance of each evaluation question, with the above selection criteria. We will also highlight any gaps in coverage or areas of evaluation that may be covered to a lower level of rigour.

Task 5.2: Evaluation Framework and Logic Mapping The inputs and outputs of Cohesion Policy will be determined in Task 1. Task 5 will seek to build on this and explore the contribution of Cohesion Policy to EU and national level transport policy. This will be achieved through holding in-depth face-to-face interviews with key Member State experts, to explore progress towards results realisation and recommendations for how to focus future Cohesion Policy investment.

To provide a framework for this Task, policy level logic mapping will be prepared for each Member State case study. This will record the policy objectives, Member State Operational Programmes and hypothesised causal pathways leading to anticipated results. The in-depth interviews with Member State experts will be used to consider progress achieved towards stated objectives and the considered contribution of Cohesion Policy. Unanticipated results will also be discussed, alongside considered challenges for the 2014-20 funding period. An example logic map is presented below, which was prepared by AECOM for the evaluation of the ERDF funded White Rose Way project in the UK. It is noted that the focus of Task 5 will be on outputs and results, and the impacts shown in the figure overleaf are unlikely to be realised in the evaluation.

Section G: Task 5 Member State Case Studies


Task 5.3: Case Study Template As per Task 3, central to undertaking robust and focused case studies will be the preparation of a template, which will define the data and information to be collated. An updated template for use in Task 5 is presented below. This focuses on the key issues of Cohesion Policy contribution to transport investment and transport policy achievement. The key information to be collated as part of the initial task includes Member State transport policy documentation including the stated objectives, transport strategy documents by sector outlining the detailed investment plans for the 2007-13 period, the range of funding inputs used in each case study Member State, the transport projects/schemes delivered within each sector, and any evidence on the results achieved.

Member State Case Study Template

Reference Number Unique reference number for each Case Study

Member State

EU Transport Policy A summary of the elements of the Common EU Transport Policy that has been targeted as part of individual Member State’s National Policy, to support in the analysis of the influence of CTP on National Transport Policy.

Member State National Transport Policy

A summary of the Member State’s National Transport Policy, including information such as: • National Transport Policy documents • National Investment Programmes (which may include transport) • National Development Strategies These sources will allow identification of a list of policy objectives for each Member State

Transport Sector(s) Assisted by Cohesion Policy

Road Rail

Urban Transport

Airports Ports/Inland

Waterways


/ / / / /

Transport Sector(s) Assisted by of EU funding / / / / /

Transport Sector(s) Assisted by Member State Funding

/ / / / /

Road Investment Summary

• Spend across 2007-13 period • Funding sources and % contribution of Cohesion Policy considering the

relative contribution and importance of Cohesion Policy assistance • Road specific policy objectives • Outputs achieved (by each funding source), including Core Indicators and

additional Member State specific indicators considering the relative contribution and importance of Cohesion Policy assistance

• Evidence of results • Evidence of contribution of Cohesion Policy to observed results

Rail Investment Summary • As above

Urban Transport Investment Summary • As above

Airports Investment Summary • As above

Ports/Inland Waterways Investment Summary • As above

Contribution Analysis

Evidence of Member State Results

It is recognised that the availability of data on results will be relatively sparse. Available data and information on Member State objectives and results will be collated, addressing issues such as: • Accessibility (eg. journey times, mode coverage) • Quality of the transport network (passenger perceptions) • Efficiency of the transport network (journey time variability) • Safety (accidents, severity using the Community database on Accidents on the

Roads in Europe (CARE)14) • Environment (CO2 emissions)

Alternative Explanations of Policy Objective Achievement

Alternative explanations of Policy Objective achievement will be recorded as identified through document reviews and updated following depth interviews. The link between investment from Cohesion Policy and outputs will be determined in Task 1, showing the relative contribution of Cohesion Policy to delivery. In the anticipated absence of detailed ex-post project or programme evaluations it will be necessary to collect more qualitative evidence from experts on the considered explanations for observed change on transport accessibility, efficiency or safety for example. Confounding factors such as reduced travel demand during the economic recession would also be included in the assessment of a) the extent to which transport policy objectives in Member States has been achieved, and b) the

14 http://ec.europa.eu/idabc/en/document/2281/5926.html


contribution of Cohesion Policy.

Member State Challenges

Funding • Level • Competition within Member State for funding • Securing a share of the total amount of EU funding available

Priorities • Transport sectors • Member States • TEN-T or non-TEN-T

The case study template and analysis approach has been developed based on the principles of the contribution evaluation methodology15. This focuses on the following six steps to evaluate the contribution of a policy or programme to observed results:

1. Set out the cause-effect issue to be addressed (logic mapping).

2. Develop the postulated theory of change and risks to it.

3. Gather existing evidence.

4. Assemble and assess the contribution story.

5. Seek out additional evidence through stakeholder interviews.

6. Revise and strengthen the contribution story.

Task 1 will provide documentation and data for points 1-4 of the above, collating data on outputs of Cohesion Policy across Member States and undertaking an initial assessment of contribution. Task 5 will update such data and gather the views of Member State experts through which to test and verify emerging conclusions on the contribution story.

Task 5.4: Member State Representatives The fourth element of Task 5 will be the identification of key experts within the case study Member States to be interviewed. A list of key individuals will be prepared for each of the case studies, reflecting the specific Member State, institutional structure and type of projects delivered. An example of the types of individuals required is presented below, for Romania:

• Representative of the Ministry of Transport – as commonly Contracting Authority for Cohesion Fund projects;

• Representative of other Ministries e.g.

o Ministry of Regional Development and Public Works;

o Ministry Responsible for European Funds;

o Ministry for Economics/Finance; 15 Mayne J (1999) Addressing Attribution Through Contribution Analysis: Using Performance Measures Sensibly Office of the Auditor General of Canada. Also see Wimbush E and Beeston C (2010) Contribution analysis: What it is and what does it offer impact evaluation? The Evaluator, Spring 2010 issue, pp 19-24


o Ministry for Tourism;

o Ministry for Regional Development, Ministry of Environment;

• JASPERS – as heavily involved in the technical development of projects;

• DG REGIO – to provide a strategic policy perspective across all 6 case study Member States;

• Freight industry representatives – to highlight specific policy issues and opportunities; and

• Trade Bodies i.e. Inward Foreign Investment.

In establishing lists we will utilise our extensive network of Member State staff, working in partnership with the Commission to identify the most relevant individuals. We also note within the Specification the need to review and update, where necessary, information and data on inputs, outputs and results collated through Tasks 1-4. Requests will be made to key Member State experts, particularly Ministries and other central bodies, for data to enhance and verify previous information.


Task 5.5: Discussion Guides The detailed discussion guides will again be developed for each interviewee for each of the Member State case studies, to ensure that they are specific to the proposed discussion. We propose to structure the guides around the five key evaluation questions set out in the Specification. AECOMs head of Social and Market Research, Jo Christensen, will again peer review all discussion guides prior to use, to ensure that they are of a very high standard; topic and question logic, flow, content and wording will be reviewed. An initial list of in-depth questions under each of the five evaluation questions is presented below, to be developed in partnership with the Commission during the course of the project. Should Cohesion Policy have supported all the transport sectors which received assistance? What alternative funding sources have been used for each transport sector? How important do you consider this investment to be in achieving the National and EU Transport Policy Objectives? Which projects or programmes would/not have progressed without Cohesion Policy support? What was the relative contribution of Cohesion Policy for each sector in delivering outputs and results against National Transport Policy objectives? How did Member States prioritise Cohesion Policy investment by transport sector? Which transport sectors should have received more/less Cohesion Policy support and why? Are the investments made in the area of transport by Cohesion Policy the ones that fulfil its objectives? What progress has been made towards delivering anticipated National and EU Transport Policy objectives? What is the anticipated contribution of Cohesion Policy in individual Member States to achieving EU and National Transport Policy objectives? What has been actual role and contribution of Cohesion Policy to Transport Policy objectives? In which sectors has Cohesion Policy contributed significantly to achieving Transport Policy objectives? What should be supported by the Cohesion Policy in the area of transport? What are Member State priorities for the 2014-20 period in terms of transport? What are the risks and opportunities within each transport sector? What are the alternative funding sources within each sector? What are the challenges for the Cohesion Policy in the area of transport in the next 10 years? What is the policy direction within the Member State? What are the trends in transport sectors and what influence will this have on Cohesion Policy investment? What is the perceived dependence of Member States on Cohesion policy investment? What is the considered role of Cohesion Policy in the area of transport to support the achievement of the 2014-20 thematic objectives, such as supporting a low carbon economy, promoting climate change adaptation and promoting sustainable transport. Has the Common EU Transport Policy influenced the national transport policy and in what way? What changes have been adopted in National Transport Policy as a direct result of the Common EU Transport Policy? This includes the influence of objectives such as internal economic markets, TEN-T implementation, Europe’s international connectivity, as well as making a safer and more efficient transport system. What changes have been adopted in National Transport Policy as an indirect result of the Common EU Transport Policy?


Task 5.6: Member State Case Study Interviews

The core evaluation team, in association with our Member State experts, will arrange face-to-face interviews with the identified experts. This will be coordinated with a request for additional or updated information relating to this Task, and for collation into the template. Interviews for each case study will again be coordinated to maximise the efficiency of time, whilst being careful not to overburden staff. As per Task 3 we propose to interview each case study expert separately. This will ensure that the findings can be triangulated between interviewees, and responses and opinions can be compared/contrasted.

Each interview will be recorded using digital recorders, but only with the prior approval of interviewees. The interviews will be transcribed and coded, using a coding framework agreed with the Commission. (See Task 3 for explanation of a coding framework.) As outlined in the Organisation of Work section of this technical proposal, we again propose that three members of our team will attend each interview:

• A member of AECOMs transportation and evaluation team; • A member of KPMGs transport and financial analysis team; and • The relevant Member State expert.

Task 5.7: Case Study Analysis A narrative will be prepared for each Member State case study, utilising key evidence collated through all tasks completed to date. This will build on the presentation of updated factual data on inputs, outputs and, where possible, results, and the logic mapping that has been discussed with the Member State experts. The narrative will highlight consensus among Member State experts on areas of Member State and EU transport policy investment where progress towards results is considered evident, and the contribution of Cohesion Policy. As outlined above, we will adopt the principles of contribution analysis16, developed specifically for use in complex interventions, and where no one piece of evidence may provide a convincing explanation of change or progress towards change.

The case studies will detail the considered role of Cohesion Policy in each Member State, the transport sectors that have benefited from such investment and the future role/function of Cohesion Policy, including recommendations on which sectors of transportation should be supported in the future. We will also analyse the contribution of Cohesion Policy against key areas of evaluation, as defined by Evalsed:

• Relevance: considering the ongoing validity of Cohesion Policy vis-a-vis addressing the key challenges of social and economic integration. This will also consider the ongoing relevance of Cohesion Policy and the relative contribution and placement of transport within this;

• Utility: carefully considering the views of experts on wider society and economic trends in their Member State; and

• Sustainability: a key consideration of the durability of results, particularly the financial sustainability of Operating Programmes.

16 Mayne J (1999) Addressing Attribution Through Contribution Analysis: Using Performance Measures Sensibly Office of the Auditor General of Canada. Also see Wimbush E and Beeston C (2010) Contribution analysis: What it is and what does it offer impact evaluation? The Evaluator, Spring 2010 issue, pp 19-24


Task 5.8: Reporting The reporting of the 6 Member State case studies will be incorporated into Deliverable 5 the Second Interim Report, and will use the following outline table of contents for each case study:

Section Contents

Introduction • Summary of Member State Cohesion Policy programme and investment

Cohesion Policy Overview

• Description of the Member State Cohesion Policy Investment o Road o Rail o Urban transport o Aviation o Ports/inland waterways

• Funding source and levels

National Transport Achievements

• Summary of actual/perceived results of National Programmes and Operational Programmes (2007-13)

• Views on areas of where National Policy objectives have not been achieved

Contribution Analysis

• Qualitative assessment of the contribution of Cohesion Policy to achieving National the EU Transport Policy objectives

• Quantitative analysis of the contribution of Cohesion Policy to achieving National the EU Transport Policy objectives

• Presentation of stakeholder views on those transport sectors where Cohesion Policy has been effective, and where future investment should be focused

Challenges for 2014-20 • Collation of stakeholder and expert opinion on the main challenges of adopting Cohesion Policy investment in transport

The reporting will focus on addressing the defined evaluation questions and hypotheses being assessed, to ensure that it remains focused. It will combine descriptions of the policy context and conditions of Cohesion Fund investment, with the evaluation of any available results and stakeholder opinion on progress towards realising results. The reporting will again combine qualitative narratives with supporting quantitative analysis. The reporting will be led by the core evaluation team, with support from the Member State experts to ensure that the presentation and analysis of local contextual issues are accurate.

Section H: Task 6 Seminar


H1 Introduction The primary role of Task 6 is to discuss and test the emerging findings and recommendations from the analysis of individual major projects carried out for Task 3, and the analysis of the operation of Cohesion Policy in the transport sectors of Member States carried out for Task 5. In addition is will be a useful opportunity to test and refine the “Catalogue of Challenges” being prepared as Task 4.

Our overall approach to this Task is summarised in the diagram below:

AECOM and KPMG will prepare material to brief the participants in the Seminar and to form the basis for discussions during the Seminar. This material will summarise the emerging findings and recommendations from the Major project case studies and from the Member State case studies, and the proposed content of the “Catalogue of Challenges”. AECOM and KPMG will also identify participants, and facilitate the proceedings of the seminar leading discussions, writing up the event and consolidating the findings into the draft of the final report.

After the Seminar in Brussels, the Team will incorporate the refinements and additions to the findings and recommendations from the work to date in the draft final report of this study. The findings from the Seminar will be incorporated into the “Catalogue of Challenges” as appropriate.

H2 Task Methodology

Task 6.1. Prepare for the seminar and invite attendees

Task 6.1 Task 6.2 Task 6.3 Task 6.4

The seminar participants will be identified with the specific aim of providing the necessary experience and knowledge to discuss the emerging findings of the project case studies and Member State case studies, prepared in Tasks 3 and 5 respectively). Potential attendees were outlined in the Specification for this work and discussed further during the Inception work. The Seminar attendees will include:

• Representatives of the national administrations responsible for coordination of transport support under Cohesion Policy;

• Representatives of the Managing Authorities responsible for the projects examined by the 10 project case studies;

• Representatives of the beneficiaries for these major projects, i.e. the road administrations, Railway Infrastructure Managers and Local Authorities etc responsible for implementation of the projects.

• External experts focused on economic assistance in the field of mobility transport (including the experts interviewed for Task 5);

• Member State officials involved in the development, and ideally, operation of transport projects as identified during our case study work; and,

• Relevant Commission officials.

Section H: Task 6 Seminar

Task 6.1 Prepare for the Seminar and invite

attendees

Task 6.2 Formulate draft findings,

validation and conclusions

Task 6.3Conduct seminar in Brussels

Task 6.4 Finalization and reporting


In order to ensure the availability and presence of the attendees, the Team will contact them during the implementation of the previous tasks. The Team will expect to start the organisation of the seminar in parallel with Task 3. The proposed list of attendees will be provided to the Commission for approval.

The timing of the Seminar will have to be finalised with care. The project management of this Work Package and of the evaluation as a whole requires that the seminar be completed by October 2015. In addition the Seminar will have maximum value if it is carried out when findings are emerging from other Tasks. This suggests that it should take place in the month of October. A series of open days organised by DG REGIO is foreseen between 12th and 15th October during the 13th European Week of Regions and Cities. With this in mind the 9th of October is suggested as the date for the Seminar.

The attendants having confirmed their presence will be provided with the agenda of the seminar and draft materials will be provided to the attendants who have already confirmed their presence. These draft materials will be based on the results of work done up to this point as explained in Task 6.2 below.

Task 6.2. Formulate draft findings and recommendations for distribution to Seminar participants


The Seminar will take place as findings emerge from other workstreams, the first and second interim reports of our will have been delivered to DG REGIO at this point. The Seminar represents an excellent opportunity to refine and develop the findings from this work. In particular it is an ideal opportunity to test the findings from: the analysis of Major projects (Task 2), the Major project Case Studies (Task 3) and the Member State Case Studies (Task 5). In addition it provides an opportunity to refine the Catalogue of Challenges as described in the methodology for Task 4.

A short (approximately 10 page) briefing document will be prepared for Seminar participants to brief them on emerging results and to allow this testing and development to take place. This document will be a summary of key points from the Interim Reports already developed at this stage. It will cover:

• A listing of the 20 Major projects studies in Task 2, and the 10 of these for which case studies have been prepared in Task 3;

• The main findings and recommendations emerging from this project specific work;

• A listing of the 15 Member States considered as part of Task 1 and the 6 Member States for which case studies were prepared in Task 5;

• The main findings and recommendations emerging from this project specific work; and,

• A summary of the emerging “Catalogue of Challenges” emphasising any potential solutions to these challenges identified at this stage.

A draft Agenda for the seminar and a list of invitees will also be circulated. The Agenda and the techniques to be used in running the Seminar itself are described further in Task 6.3 below.


Task 6.3. Conduct seminar in Brussels


The seminar will take place in Brussels, organised by using support from Work Package 1 (“WP1”) of the ex post evaluation (“Synthesis”). The Team will contact and meet the contractor of WP1 in order to define a prior planning and organisation of the seminar. Initial contact has been established with the WP1 consultants, and we will meet with them to finalise arrangements for the Seminar and their participation.

The contractor for WP1 is a team of consultant led by Applica. This body will cover travel and accommodation costs for participants and a fee appropriate to the level of expertise for participants not from public authorities.

A draft outline Agenda for the Seminar is set out overleaf.

The Team will identify the key participants involved in the conduction of the Seminar. The speakers at the Seminar will include:

• REGIO representative to provide the welcome and opening overview of the ex-post evaluation scope and purpose;

• AECOM/KPMG Task 3 leader Richard Redfern;

• AECOM/KPMG Task 5 leader John Finnegan;

• AECOM/KPMG core team members to present and facilitate breakout sessions.

In addition to the speakers, we propose to introduce the role of the seminar facilitator who will:

• Introduce the speakers;

• Act as a moderator ensuring that seminar discussions are confined to the specific areas where clarification is needed; and

• Guide the discussion to identify consensus among the participants.

The format of the seminar will be designed to facilitate and encourage open and frank discussion. There is a risk that the attendees will feel constrained to represent institutional positions rather than engaging in a full an frank discussion of the findings from this study. This risk will be mitigated through the choice of attendees and in the design of activities on the day itself.

During the interviews for the Major project case studies the study team will make contact with the beneficiaries who were involved in the detailed work of developing and implementing the projects. Every effort will be made to encourage representatives of these bodies to attend the seminar in addition to officials of managing authorities and government ministries. Representatives of Managing Authorities and Ministries may have less to offer in a detailed discussion of the lessons to be learnt from specific projects, and will be more constrained to defend organisational or even national positions in any discussion.


Time Activity Tasks9.00 Registration

9.30 Opening Presentation by Facilitator • Introductions• Agenda for the Day

9.45 Opening Presentation by Study Team • Context for Work Package 5• Approach Adopted and State of Completion

10.00 Breakout Session 1 – Lessons from Individual Projects

• Recap of emerging findings and recommendations from project case studies

• Identify additional findings and recommendations

• Prioritise findings• Test recommendations

11.30 Coffee Break

11.45 Breakout Session 2 - Lessons from Member States

• Recap of emerging findings and recommendations from Member State case studies

• Identify additional findings and recommendations

• Prioritise findings• Test recommendations

1.15 Lunch Break

14.30 Breakout Session 3 – “Catalogue of Challenges”

• Recap of the emerging Catalogue of Challenges

• Identify any missing challenges and suggest solutions

• Prioritise Challenges• Test proposed solutions

16.00 Plenary Session

• Report back from Session 1• Discuss conclusions from Session 1• Report back from Session 2• Discuss conclusions from Session 2• Report back from Session 3• Discuss conclusions from Session 3

17.00Closing remarks by facilitator and DG REGIO


The time available for the Seminar is, inevitably, limited, and will have to be used efficiently and effectively. AECOM and KPMG have extensive experience in the conduct of such working sessions. They are an important element of many aspects of our work. If the seminar is run as a simple exercise in delivering and discussing papers, it will be difficult to achieve much analysis or generate new ideas and insights in the time available. A traditional format such as this will also increase the risk that the attendees simply place the “official” views of their organisations on the record rather than engaging in dialogue with their peers and the consultants, and bringing the fruits of their experience to the group. The materials for the day will be designed when more of the results of the earlier Tasks are available, however in outline some of the techniques and approaches that could be used include:

• The breakout sessions referred to in the draft Agenda will be run by splitting the attendees into a number of smaller groups of approximately 10 people. The attendees from individual Member States will be split between these groups, and each small group will be assigned a specific area of the material to discuss.

• A member of the study team will join each small group to ensure that the discussion moves forward and covers all the relevant topics. A range of simple, yet effective, techniques such as capturing the discussion on a flip chart as it progresses rather than traditional note taking are very effective in this respect.

• The facilitators will establish and impose ground rules for the small group discussions to ensure that they are run as “brain storming” sessions rather than formal meetings where the participants represent their employers.

• The breakout sessions on project case studies may have a large amount of material to cover and it will probably be necessary to split the material between the groups. The composition of the groups and the split or material can only be finalised closer to the date but, for example, one might split the small groups by mode and have them discuss the cases for that mode. This would focus discussions on the issues of specific relevance to individual modes, such as the different challenges in demand forecasting for road and rail projects.

• At the end of each breakout session, each small group will pick a member to report back the group’s findings to the plenary session in the afternoon;

• In the breakout sessions, and in the plenary session it will be necessary to rank certain findings and recommendations, or even to pick short lists to be carried forward. From Task 3 this would include the identified solutions to enhance ex-ante activities, and from Task 5 this could include the challenges for Cohesion Policy for the period 2014-2020. This can be done effectively by such simple techniques as writing the items up on large sheets of paper, and allowing the attendees to vote by placing sticky notes next to individual items.

• Many of these techniques (splitting in to small groups, capturing discussions on flowcharts, reporting back to plenary sessions, and voting systems based on sticky notes) have the important secondary effect of moving the attendees around and keeping them energised through a day of intensive discussions.

Task 6.4. Finalization and reporting


Following the seminar, key findings and conclusions will be incorporated into the preparation of the draft final report. In effect the final report will be based on the interim reports as expanded and refined by this consultation and analysis process. The results of the Seminar will also be communicated to DG REGIO. DG REGIO may also wish to attend the Seminar. The Catalogue of Challenges will also be refined and completed based on the results of the Seminar.

Section I: Task 7 Reporting


I1 Introduction The final Task will be the preparation of a Draft Final Report within 11 months of the contract signature, with a Final Report being prepared within 12 months.

I2 Task Methodology

Task 7.1: Draft Final Report

Our core evaluation and financial team will meet following the Task 6 Seminar to confirm the structure and preparation responsibilities for the Draft Final Report. Building on the required contents outlined in the Specification we propose the following structure of the report.

The two interim reports already provided will document the full extent of the work undertaken and results obtained. The key goal of the Final Report is not to repeat this material but to analyse this material and draw out conclusions and recommendations for the future. The volume and complexity of the material involved will require the use of the full range of available techniques to communicate the results of our work in a useful way. Potential techniques available to the study team include:

• Use of GIS to map key data and findings;

• Use of photography and video where findings are best presented in this way; and,

• Graphic design expertise, including the preparation of “infographics” to present the analysis of complex data.

It is also important to remember that the most important audience for the results of this work is the public at large and our reporting must easily feed in to the information and communication activity of the Commission. Also, many of the key decision makers relying on the report will not be transport or evaluation specialists. The results of our work will only be useful if they can be communicated to a non specialist audience.

Section Contents

Executive Summary (max 200 words)

• Overview of the evaluation • Key conclusions and recommendations for 2014-2020

Non-Technical Presentation of Findings

• Summary of main findings and recommendation written for a non-specialist audience. This would include the key graphics, maps and other visual aids from the report proper.

Introduction • Summary of the evaluation objectives and approach

Cohesion Policy Analysis

• Using the findings of Tasks 1 and 5, and additional information from Tasks 2, 3, 4 and 6 to present an analysis and synthesis of evidence on: o The contribution of Cohesion Policy to the delivery of outputs, such as the

number of people potentially served by an urban public transport service, km’s of highway and rail line.

o The contribution of 2007-13 investment in achieving transportation objectives

Section I: Task 7 Reporting


o The identification of those transport sectors that have contributed most to achieving these objectives.

Transport SWOT Analysis

• Strengths: of Cohesion Policy in supporting EU investment in transport and contributing to social and economic objectives.

• Weaknesses: challenges and gaps in delivery investment, addressing objectives and achieving enhanced connectivity.

• Opportunities: in the wider investment context. • Threats: to future Cohesion Policy effectiveness and delivery efficiency.

Financial Sustainability Analysis

• Detailed analysis of the financial and demand assumptions adopted in 2007-13 projects.

• Identification and exploration of individual errors in financial analyses. • Identification and exploration of systematic bias factors. • Analysis of the financial sustainability of projects and the effectiveness of

mitigation measures adopted. • Recommendations on the methodological approach to financial analysis for

the 2014-20 period.

Catalogue of Challenges • A final set of key challenges and suggested solutions • Incorporating the final outputs from Task 4, as refined in the Seminar

Conclusions and Recommendations for 2014-20

• Key conclusions on evaluation findings. • Recommendations for 2014-20 Cohesion Policy period.

The individual report sections will be drafted by the relevant core team members who will have responsibility for the management and delivery of individual Tasks. As outlined in our Organisation of Work section we have identified core team staff members who will lead on Tasks 1 and 5, considering Member State delivery and case studies, and staff to lead on Tasks 2, 3, 4 and 6 considering project delivery and case studies. This will ensure that the technical content of the report is accurate and of a very high quality.

Both AECOM and KPMG are highly experienced in this type of drafting. In addition, we have noted DG REGIOs recommendation that we consult the Commissions own guidance on drafting and generally accepted style guidance such as that used and published by “The Economist” magazine.

The Draft Final Report will, as outlined in our Quality Control Measures section, be checked by our Project Manager John Finnegan. John has extensive technical experience in the field of Cohesion Policy, financial analysis and project implementation and will have overall responsibility for confirming the accuracy of the report’s contents and the analysis carried out to reach its conclusions and recommendations.

The Draft Final Report will subsequently be reviewed by Bernard Feeney, one of AECOMs Technical Directors and Peer Reviewer for this commission. Bernard will verify that the report meets the full requirements of the Specification. Finally, the Report will be reviewed by our Project Director, Richard Redfern, for approval for issue to DG REGIO.

Task 7.2 Discussion of Draft with Steering Group

The draft Final Report will be discussed with the Steering Group for this meeting at the meeting scheduled for 20th November, 2015.


Task 7.3: Final Report

Following the first presentation of findings and detailed comments from DG REGIO we will prepare the Final Report. This will follow the same structure as the Draft Final Report and will address all comments provided by DG REGIO and those issues from other stakeholders as agreed with DG REGIO. The Final Report will proceed through the same quality control gateways as the Draft Final Report, before being approved for issue by our Project Director.

Section J: Project Management and Quality Control


J1 Introduction This Section of the Inception Report describes: • The team structure that has been established to carry out this study and the allocation of work between

members of this team; and, • The work programme for completion of this study.

J2 Team Structure and Work Allocation

The scale of this study and the range of expertise required means that AECOM and KPMG have assembled a relatively large team for the work drawing on the extensive resources and geographic scope of both firms. The study requires that we synthesise the findings of what will necessarily be wide ranging work, and reach a set of key conclusions and recommendations. As a result a small core team of experienced staff will have to be familiar will all of the work done and be able to work together to devise these conclusions and recommendations. AECOM and KPMG have devised a team structure and allocation of responsibilities that meets both of these requirements. This team structure is summarised in Figure J1 below. The key elements of this structure are: • A central project management team that controls and takes responsibility for all areas of the work; • A core team of experienced transport, economic and financial professionals who carry out the extensive

research and analysis work involved in this study; • An “Expert Board” of senior professionals from AECOM and KPMG who will provide quality control and

technical leadership to the members of the core team. This is a normal part of the Quality Assurance processes of AECOM and KPMG for a study of this scale and is in addition to the three external experts specified in the Terms of Reference for this study.

• A set of country coordinators based in each Member State, who will provide local knowledge, linguistic support and facilitate contacts between the core team members and stakeholders in Member States;

• A support team of experts in specific technical issues who can assist the core team as and when these arise in our research;

• A dedicated EU office to provide a physical and practical link to DG REGIO; and • The panel of three independent external experts agreed with DG REGIO to play the role of “critical friend” in

assuring the quality of our work. The composition and role of each of these components of the team is described in more detail below.

Section J: Project Management and Quality Control


Figure J1: Team Structure

J2.1 Project Management Team The Specification highlighted the need for strong co-ordination, guidance and quality control to ensure the successful delivery of this contract. To this end, we have assembled a strong Project Management Team consisting of senior experienced personnel assigned to the roles of Project Director and Project Manager. These will be supported by experienced specialists in finance and transport respectively. This team will adopt a pro-active approach to the ongoing management and coordination of the commission, characterised by:

• Paying due care and attention to specific tasks and confirm a realistic and achievable programme of work and resourcing plan;

• Regular and timely updating of the programmes, forecasts to completion and financial monitoring data;

• The early establishment of project risks with clear and effective mitigation processes;


• Early notification of the occurrence of factors that potentially jeopardise task programmes and budget; and

• The adoption of a ‘no surprise’ philosophy in which the need for task changes is communicated quickly to the client and changes are managed to minimise impact on task programme and cost.

The Project Director, (Richard Redfern) will be responsible for the overall financial and contractual control and will be the first contact for any client grievance. He will attend the main formal client meetings He will have an overview of the delivered services in order to ensure the appropriateness of the services being undertaken prior to submission of reports to the client.

The Project Manager (John Finnegan) will be the main point of contact on our side for the project. He will deal with day-to-day management tasks such as project planning, timing and budget control, handling information requests, dealing with administrative matters and for managing any staff-related issues and ensuring continuous service and the required progress of the project. He will be responsible for overseeing project delivery and quality control of the services delivered.

Both of them will have key roles in this project and will work in close co-operation with the Core Team. They are familiar with all the management requirements and ensure seamless provision of services throughout the contract. They will ensure that the following management activities are properly performed:

• Ensure that tasks are appropriately allocated;

• Planning the work, monitoring the progress and keeping the project on schedule, to ensure the team can anticipate issues that may arise in order to resolve them quickly and efficiently;

• Mobilising team members and facilitating regular meetings/updates;

• Following up on meetings, targets and deliverables; and

• Based on the input from the team, reporting to the Steering Group.

J2.2 Core Team

The Core Team, a pool of evaluation and policy experts, will be in charge of delivering the services. This team will form the heart of the technical work to be performed. All team members have practical knowledge of quantitative and qualitative research methods. They are familiar with evaluation projects and have conducted impact evaluation studies before; while at the same time all members have experience with Cohesion Policy and most of them have significant transport experience and knowledge. All of the team members are used to performing their work in English language, and team members master English language on native level.

Table J.1 provides an overview of the key team members who will manage each of the four Tasks, and identifies the core technical team that will be responsible for delivering each activity. We have selected Task leaders and core team members that have the technical expertise and experience to deliver the work, as well as the experience of specific Managing Authorities


Table J.2: Allocation of Task to Core Team

Task Task Leader Core Team Members • Kevin Brady 1.a John Finnegan • Susie Crawford

• Craig Bell • Nick Woollett • Tim McCann 1.b • Bernard Feeney • Jacopo Signorile • Fulvio Pastorelli • Francesco Patrizi • Luigi Micarelli

1

John Finnegan

• Anda Berènyi • David Arthur 2.a John Finnegan • Nick Woolett • Luca Longo 2.b Jacopo Signorile • Luigi Micarelli • Luca Longo • Francesco Patrizi

2

2.c Jacopo Signorile • Fulvio Pastorelli • John Finnegan • Nick Woolett • David Arthur

3 Richard Redfern

• Tim McCann • Nick Woollett 4 John Finnegan • Tim McCann • Richard Redfern • Susie Crawford • Marco Serifio • Fulvio Pastorelli • Francesco Patrizi

5 John Finnegan

• Luigi Micarelli • Jacopo Signorile • Anda Berènyi 6 Richard Redfern • John Finnegan • Richard Redfern • Bernard Feeney • Alessandro Guiducci • Jacopo Signorile • Fulvio Pastorelli • Francesco Patrizi

7 John Finnegan

• Luca Longo


J2.3 EU Coordinator The EU coordination and support team will provide Brussels-based support throughout the project execution. They will act as personal contact points for the Commission and support arrangements for the Seminar to be conducted in Brussels. They provide quality assurance to the deliverables both in terms of language and the coherence between the Core Team’s and the Country Coordinators’ work – ensuring at the same time that the input from the “Expert Board” of senior AECOM and KPMG staff is duly included in our outputs.

J2.4 Country Coordinators Separate Country Coordinators have been set up for each country in the scope of this evaluation. In the organisational chart we present each team member who will be in charge of leading the local teams carrying out the empirical work in their respective countries. This involves contacting local stakeholders and collecting relevant and comparable local data for the analysis (as described under Tasks 1, 3 and 5), collecting and reviewing existing country literature only available in the local language and providing local knowledge and expertise in the execution of the analysis in close cooperation with the Core Team.

J2.5 Independent External Experts A panel of three distinguished academic experts have agreed to act as independent external experts and will review all our deliverables and participate in the work of the Steering Group. Emile Quinet of the Paris School of Economics, Dan Graham of Imperial College London and Roger Vickerman of Kent University have agreed to take part in this panel.

J.3 Work programme The timetable for the deliverables in this study was agreed at the kick off meeting, and subsequent discussions with DG REGIO. The dates agreed at this meeting are set out in Table J.2 below:

Table J.2 Timetable for Deliverables Inception Report 10th February 2015

Steering Group & Experts 18th February 2015

Progress Report By the 10th of each month

Proposal for two pilot case studies 10th March 2015

First Interim Report 10th April 2015

Progress meeting 17th April 2015

2 pilot project case studies 11th May 2015

Second Interim Report 10th August 2015

Steering Group & Experts 7th September 2015

Seminar 9th October 2015

Draft Final report 10th November 2015

Steering Group & Experts 20th November 2015

Final report 10th December 2015

Figure J.1 below sets out the timeline and work schedule to meet these deadlines.


Figure J.1: Project timeline Month Task 0 1 2 3 4 5 6 7 8 9 10 11 12

Task 0: Inception M0

R0 M

Task 1: analysis of achievements

R2

Task 2: review of financial analyses

R2 P

Task 3: project case studies

R1 R3 R4

Task 4 catalogue of challenges

R4 R5

Task 5: case studies on CP contribution to EU and national transport policy

R4 M

Task 6: Seminar R5

Task 7: Final report

M R6

M0 – Kick-off meeting

P – Progress Meeting with DG REGIO

M – Meeting with the Steering Group The following reports will be delivered as part of this evaluation. Inception Report within one month of contract signing; • R0- Inception Report

• R1- Proposal for the two pilot case studies

• R2 - First Interim Report within four months of contract signing;

• R3 – Pilots for project case studies

• R4- Second Interim Report within eight months of contract signing

• R5 - Draft Final Report and the final “Catalogue of Challenges” within eleven months of contract signing; and

• R6 - Final Report within twelve months of contract signing. The delivery of the Inception Report, the Second Interim Report and the Draft Final Report will be associated with a meeting of the Steering Group. The minutes from all such meetings will be prepared by the evaluation team for approval by the client team, seven days after each meeting.

Appendix

AECOM/KPMG Ex post evaluation of Cohesion Policy 2007-2013 – Work Package 5 Transport 83 Capabilities on project: Transport

Evaluation Economics

Appendix: Long List of Major Projects

Appendix : “Long List” of Major Projects

Original Commission Long List

Update on completion by AECOM/KPMG during Inception

Country

Title

Category Total investments (in m. EUR)

Decision date selected for

WP 5 / completion

Status Proposed for Tasks 2 and 3 (Y/N)

2010BG161PR002 Balgarija Completion of Trakia Motorway, Lots 2, 3 and 4 Motorways (TEN-T) 363 21/12/2010

X 1 Complete 29/07/2013 Y

2013BG161PR003 Balgarija North Speed Tangent from km 0+000 to km 16+540 National roads 120 04/06/2014

? 3

2011BG161PR003 Balgarija Modernisation of the Railway Section Septemvri-Plovdiv – Part of Trans-European Railway Network

Railway (TEN-T) 322 16/01/2012end 2015 2 Incomplete Y

2010BG161PR003 Balgarija Burgas Integrated Urban Transport Project Urban transport 62 16/08/2011 end 2015 2 Incomplete

2008CZ161PR008 Ceska Republika D1,stavba 0135 Kroměříž-Říkovice a R55,stavba 5503 Skalka-Hulín

Motorways (TEN-T) 496 31/10/2011X 1 Complete

2010CZ161PR010 Ceska Republika Rychlostní silnice R4 křižovatka I/20 - Nová Hospoda National roads 57 16/05/2013

X 1 Complete

2009CZ161PR010 Ceska Republika Elektrizace trati Zábřeh-Šumperk Railways 74 19/09/2011 X 1 Complete

2009CZ161PR013 Ceska Republika Modernizace tratě Votice - Benešov u Prahy Railway (TEN-T) 351 27/04/2012

X 1 Complete Y

2008DE161PR004 Deutschland

Neubau der Bundesautobahn A 26, Projektabschnitt I - südöstl. Stade bis Horneburg (K 36n), Projektabschnitt II, Horneburg (K 36n) – AS York (K 26)

Motorways 190 19/07/2010

end 2014 1 Complete Y

2012DE161PR009 Deutschland Neubau von Teilabschnitten der A 71 zwischen AK Erfurt (A 4) und AD Südharz (A 38)

Motorways (TEN-T) 214 18/06/2013end 2015 2 Incomplete

2011DE161PR006 Deutschland Bundesstraße B 6n - Modul 1 (OU Bernburg bis OU Köthen) National roads 142 19/04/2012

end 2015 2 Incomplete

2009DE161PR009 Deutschland City-Tunnel Leipzig (Module 5 und 6) Railway (TEN-T) 178 29/07/2011

X 1 Complete

2012DE161PR008 Deutschland City-Tunnel Leipzig (Modul 7)

Railway (TEN-T) 86 10/04/2013X 1 Complete

2009EE161PR004 Eesti Reconstruction of Väo-Maardu section of E20 Tallinn-Narva Road (km 9-17)

Motorways (TEN-T) 142 19/03/2010bridge project 1

Partially Complete

2009EE161PR009 Eesti Reconstruction of Ülemiste Juncton in Tallinn Regional/local roads 81 13/09/2010 X 1 Complete Y

2013GR161PR012 Ellada

ΒΕΛΤΙΩΣΗ/ΑΝΑΒΑΘΜΙΣΗ ΤΗΣ ∆ΥΤΙΚΗΣ ΕΣΩΤΕΡΙΚΗΣ ΠΕΡΙΦΕΡΕΙΑΚΗΣ Ο∆ΟΥ ΘΕΣΣΑΛΟΝΙΚΗΣ ΑΠΟ ΠΕΡΑΣ ΚΟΜΒΟΥ Κ16 (ΛΑΧΑΝΑΓΟΡΑ) ΕΩΣ ΚΟΜΒΟ Κ5 (ΝΟΣΟΚΟΜΕΙΟ ΠΑΠΑΓΕΩΡΓΙΟΥ)

Motorways 155 08/08/2014

? 3 Incomplete

2010GR161PR007 Ellada Αποπεράτωση Οδικού Άξονα «ΕΛΕΥΣΙΝΑ – ΚΟΡΙΝΘΟΣ – ΠΑΤΡΑ»

Motorways (TEN-T) 2025 16/12/2013end 2015 2 Incomplete Y


Εργασίες Αποκατάστασης προβληματικών τμημάτων της Ε.Ο. Τριπόλεως – Καλαμάτας στο τμήμα Παραδείσια – Τσακώνα

National roads 203 28/02/2013

? 3


ΟΛΟΚΛΗΡΩΣΗ ΝΕΑΣ ∆ΙΠΛΗΣ ΣΙ∆ΗΡΟ∆ΡΟΜΙΚΗΣ ΓΡΑΜΜΗΣ ΥΨΗΛΩΝ ΤΑΧΥΤΗΤΩΝ ΤΙΘΟΡΕΑΣ – ΛΙΑΝΟΚΛΑ∆ΙΟΥ - ∆ΟΜΟΚΟΥ

Railway (TEN-T) 690 22/08/2011

bridge project 1 Incomplete

2013GR161PR003 Ellada ΠΕΡΙΦΕΡΕΙΑΚΗ Ο∆ΟΣ ΚΑΤΕΡΙΝΗΣ Regional/local roads 66 22/01/2014 end 2015 2 Incomplete

2012ES161PR003 España AUTOVÍA DEL OLIVAR

Motorways 995 16/09/2013end 2015 2

2011ES161PR004 España TREN DE LA BAHÍA DE CÁDIZ Railways 205 19/09/2014 ? 3

2009ES161PR012 España

LÍNEA DE ALTA VELOCIDAD MADRID – CASTILLA LA MANCHA – COMUNIDAD VALENCIANA – REGIÓN DE MURCIA TRAMOS: MADRID – CUENCA - FUENTES, ALBACETE – VALENCIA, ACCESO VALENCIA Y ACCESO ALICANTE/ELCHE

Railway (TEN-T) 1455 08/10/2009

? 3 Y

2012FR161PR002 France Route des Tamarins - Tranche 2 Motorways 1091 28/11/2012

? 3 Complete Y

2007FR162PR001 France

Improvement of the rail links Rennes-Brest and Rennes-Quimper Railways 306 16/12/2009 ? 3 Complete

2012FR162PR005 France

REALISATION DE LA TROISIEME PHASE DU TRAMWAY DE L'AGGLOMERATION DE VALENCIENNES Urban transport 183 30/04/2013 ? 3 Complete Y

2013HR161PR003 Hrvatska Okucani to Novska Railway Rehabilitation and Upgrade Railway (TEN-T) 43 25/08/2011

? 3 Incomplete*

2008IT161PR003 Italia

Completamento Autostrada Siracusa-Gela: lotto 6+7+8 "Ispica-Viadotti Scardina e Salvia - Modica"

Motorways 340 12/04/2012

? 3 Incomplete


Itinerario Agrigento - Caltanissetta - Adeguamento a 4 corsie della SS 640 "di Porto Empedocle" nel tratto dal km 9+800 al km 44+400


? 3 Unknown


Completamento delle Opere Civili e realizzazione delle Opere Tecnologiche della Linea 1 della Metropolitana di Napoli – Tratta Dante (esclusa)/Municipio (inclusa)/Garibaldi (inclusa)/Centro Direzionale Railways 1577 18/12/2009 ? 3 Complete Y

2009IT161PR012 Italia Raddoppio Palermo-Messina: Tratta Fiumetorto-Ogliastrillo Railway (TEN-T) 3269 06/05/2011

? 3 Incomplete Y


S.S. 268 "del Vesuvio". Lavori di costruzione del 3° tronco compreso lo svincolo di Angri Regional/local roads 139 08/12/2011 ? 3 Incomplete

2009LV161PR002 Latvija Construction of E22 section Riga (Tinuži) to Koknese Motorways (TEN-T) 145 17/06/2010

end 2015 2 Complete

2009LV161PR005 Latvija

Modernisation of Riga Suburb Railway Passenger Transport System and Renewal of Diesel Rolling Stock Railways 379 01/03/2011 ? 3 Incomplete

2009LV161PR001 Latvija Construction of Second Track Skriveri – Krustpils (Riga-Krustpils section)

Railway (TEN-T) 113 13/11/2009end 2014 1 Incomplete

2013LT161PR001 Lietuva Construction of the Second Track on the Vilnius Bypass, section Kyviskes - Valciunai

Railway (TEN-T) 79 07/05/2014? 3 Complete

2009HU161PR003 Magyarország Southern Section of M0 Budapest Ring Road (between M1-M5)

Motorways (TEN-T) 490 18/12/2009end 2015 2 Unknown

2013HU161PR009 Magyarország

Construction of development of highway no. 62 sections between M8 (new Danube-bridge) and Székesfehérvár


? 3 Unknown

2008HU161PR003 Magyarország Budapest Metro Line 4, Section I

Promotion of clean urban transport 1418 02/09/2009

bridge project 1 Unknown Y

2010HU161PR003 Magyarország

Budapest-Esztergom vasútvonal rekonstrukciója I.ütem Railways 418 04/10/2012 end 2015 2 Unknown

2013HU161PR002 Magyarország Reconstruction of the Szajol (excl.) – Püspökladány (incl.) Railway (TEN-T) 557 28/03/2014

? 3

2013PL161PR028 Polska

Zakup kolejowego taboru pasażerskiego do obsługi połączeń dalekobieżnych -Etap I Mobile rail assets (TEN-T) 492 19/09/2014 ? 3

Implementation timetable for trains; delays in the delivery

of trains Y

2012PL161PR049 Polska Budowa drogi ekspresowej S8 na odcinku węzeł Walichnowy - Łódź (A1)

Motorways 1238 20/06/2013? 3

Construction


Budowa autostrady A1, odcinek Toruń - Stryków

Motorways (TEN-T) 3335 27/04/2011

end 2014 1

Motorway in use;

passengers service area

under construction Y


Kontynuacja budowy Drogowej Trasy Średnicowej ZACHÓD odcinek Zabrze-Gliwice, Podprojekt 3 - odcinek G2 w Gliwicach


? 3

construction


Modernizacja linii kolejowej Warszawa - Łódź, etap II, Lot A - odcinek Warszawa Zachodnia - Miedniewice (Skierniewice) Railways 517 20/11/2012 ? 3

construction

2011PL161PR001 Polska Modernizacja linii kolejowej E30/C-E 30, odcinek Kraków - Rzeszów, etap III

Railway (TEN-T) 1129 19/11/2012? 3

construction; delays in

construction Y


Kontynuacja budowy Drogowej Trasy Średnicowej ZACHÓD odcinek Zabrze-Gliwice, Podprojekt 3 - odcinek G2 w Gliwicach Regional/local roads 349 24/10/2012 ? 3


Zintegrowany system miejskiego transportu publicznego w Lublinie Urban transport 131 31/01/2013 ? 3

under implementation Y

2013PT161PR002 Portugal

Linha da Beira Baixa - Modernização do troço Castelo Branco/Covilhã/Guarda (1ª fase) Railway (TEN-T) 108 29/10/2014

? 3

A modernização do troço Castelo

Branco - Covilhã foi

concluída em 31DEZ2014;

2008RO161PR002 România Motorway Construction on TEN-T 7 , Cernavodă - Constanța Section

Motorways (TEN-T) 848 06/07/2009X 1

Completed Y

2009RO161PR048 România

Reabilitare DN6 Alexandria - Craiova


end 2014 1

Partially completed

Y

2011RO161PR008 România

Rehabilitation of the railway line Brasov-Simeria, component of the IV Pan-European Corridor, for the trains circulation with a maximum speed of 160 km/h, Section Sighisoara - Coslariu

Railway (TEN-T) 1124 22/01/2013

? 3

Construction on-going

2008SI161PR002 Slovenija Motorway A4; Slivnica - Gruškovje: Slivnica - Draženci Motorways (TEN-T) 278 12/11/2009

X 1 Yes

2013SI161PR002 Slovenija Poljane by-pass of Škofja Loka (R1-210 stretch 1087, R1-210 stretch 1110)

National roads 52 11/09/2014? 3 Yes Y

2012SI161PR001 Slovenija

Implementation of digital radio system GSM-R on Slovenian Railway Network Railways 150 14/03/2013 end 2015 2

2011SI161PR003 Slovenija

Reconstruction, electrification and upgrading of the railway line Pragersko-Hodoš for 160 km/h, modernisation of level crossings and construction of subways on railway stations – Phase 1

Railway (TEN-T) 330 15/05/2013

? 3

2009SK161PR001 Slovenska Republica

R1 Žarnovica - Šášovské Podhradie Motorways 216 01/03/2010

? 3

Y

2007SK161PR002 Slovenska Republica

DIALNICA D1 SVEREPEC - VRTIZER Motorways (TEN-T) 380 18/02/2009

? 3

ex post evaluation of cohesion policy 2007-2013 –...

Documents