the assessment of the added value of leader/clld as...

The assessment of the added value of LEADER/CLLD as improved social

capital in the LAG Prealpi & Dolomiti (Italy)

Good Practice Workshop: “Showing the added value of LEADER/CLLD through evaluation”

17-18 May 2018 Helsinki, Finland

Elena Pisani – TESAF Dep. – University of Padova

Matteo Aguanno – Director of the LAG Prealpi & Dolomiti

Outline 1. Introduction 2. Background and needs3. Process and methods 4. Evaluation findings, and lessons learned from the findings5. Reflections on the evaluation approach

2

LAG Prealpi & Dolomiti• Key information: The LAG Prealpi & Dolomiti was firstly

established in 1997 in a large, but fragmented, mountain territory in the North-East of Italy. It now includes 23 municipalities, covering a total surface of 1,344 km2where 142.803 inhabitants are residents. The partnership is composed of 23 members (11 public authorities and 11 private actors)

• Working with LEADER: LEADER II (1994-1999), LEADER + (2000-2006), LEADER 2007-2013, LEADER 2014-2020

• CLLD strategy approved in October 2016, total budget is of 8.9 millions euros, total number of projects n° 106 and total n° of implemented projects 83 by March, 2018

• Organizational features: The LAG is a private not-for profit association and the delivery mechanisms is via the mono-fund approach selected by the Veneto Region

3

https://www.galprealpidolomiti.it

Background and evaluation needs

Good Practice Workshop: “Showing the added value of LEADER/CLLD through evaluation" - 17-18 May, Helsinki 4

Key questions Background and evaluation needsWhy? (i) Different critical points emerging from the external evaluation of the LEADER Axis

(2007-2013) in the Veneto Region pointing on the limited value added of the LEADER Approach

(ii) Opportunity of a research project done by the University of Padova on Social Capital and Local Development: from theory to practice recently published by Palgrave MacMillan

Who initiated? LAG’s director and staff together with Elena Pisani of the TESAF Dep. of the University of Padova have jointly initiated the assessment of these different forms of added value

How? The initial evaluation need has been now transformed into a LAG’s self-assessment

Who supported? Italian Rural Network, TESAF Dep. UNIPD, regional network of the LAGs of the Veneto Region, and local community

Which stage in the assessment?

Programming period 2007-2013: completed! Programming period 2014-2020: first round of data collection

https://www.palgrave.com/de/book/9783319542768https://www.galprealpidolomiti.it/organigramma/https://orcid.org/0000-0002-8918-2781https://www.galprealpidolomiti.it/monitoraggio-e-valutazione/

The evaluation approach • Social capital can be defined as “networks together with shared norms, values and under-

standings that facilitate co-operation within or among groups”• This definition is used in the Guidelines Evaluation of LEADER-CLLD (2017) and OECD (2001)

5

SOCIAL CAPITAL

STRUCTURAL FORM

Context

Network actors

Horizontal structure of the network

Trasparency and accountability

Reputational power

NORMATIVE-COGNITIVE FORM

Trust & reciprocity among actors

Trust in institutions

Quality of networkQuality of participation

Shared valuesConflits

RELATED ASPECTS OF GOVERNANCE

Decision-making processes

Efficiency and effectiveness

Organisational culture and capacityVertical structure

The network approach: using networks to measure and show the social capital in LEADER

6

Indicators based on questions to the director, members and a

representative sample of beneficiaries

Indicators based on SNA indexes of specific relationships among

LAG members (information sharing, collaboration and trust

relationship)

Social capital

3 Forms

15 Dimensions34 Sub-

dimensions

78 Indicators

Information sharing among LAG members

0 1

Node size

NormativeCognitive

Related aspects of governance

Structural

The network approach to assess the social capital in LEADER-CLLD: How does it work? (1)

7

8

KEY POINT SPECIFICATIONSThe approach can be used for the the self-assessment of the added value of LEADER in terms of improved social capital.

Baseline, ongoing (in itinere), and final data collection and the consequent elaboration of indicators allow to estimate if changes in social capital indicators are correlated with the changes in output, results and outcome indicators of the local development strategy.

The network approach to assess the social capital of LEADER-CLLD: How does it work? (2)

Example of evaluation elements used in the network approach

9* Indicators are computed with SNA

EVALUATION QUESTIONS

JUDGEMENT CRITERIA

INDICATORS

To what extent did the HORIZONTAL STRUCTURE OF THE NETWORK lead to the generation of added value?

Internal participation has increased

C1 Average annual rate of attendance of LAG members at Board of Directors’ meeting

C2 Average annual rate of attendance at the LAG Assembly

C3 Regular attendance of LAG members at formal meetings

Level of openness of the LAG has increased

C5 Implementation of communication channels by the LAG

C6 Outreach to new actors by the LAG

Density of relations in the LAG has increased

C7* Density of the information network of the LAG

C8* Density of the collaborative network of the LAG

Public-private relations internal to the LAG has increased

C9* Proportion of private actors in the centre of the network compared to proportion of private members in the Assembly

C10* Collaborative relationships among public and private actors in the LAG

C11* Public–private relations over total number of relations of the beneficiary

Pro-activity of the LAG  has increased

C12* Bridging capacity of the LAG among members and beneficiaries

C13* Role of the LAG in building relationships among members

C14 Level of synergy between beneficiaries and the LAG

10

Step Description Who does what1. Setting up the

purpose and plan Define the evaluation questions Agree on roles and responsibility

LAG manager and LAG staff LAG members External researcher NRN

2. Conceptualizing the approach

Reviewing the CLLD strategy objectives Define the indicators to be used among the 78 proposed Prepare and test the data collection tools (questionnaires)

LAG manager External researcher (supervision)

3. Collecting data Carrying out interview, focus groups, etc. External actor

4. Analysing data Data entry in the excel file and on the SNA software Elaboration of indicators and their graphical representation

External researcher (supervision) External actor

5. Visualizing, discussing, and interpreting findings

Display findings in graphs and online tools Arrange meetings with LAG members to diffuse Adopt consequent actions or improve the existing ones in

order to capitalize the lessons learnt

IT or communication expert Facilitator of interactive group

works LAG manager Executive Board President of the organization

The network approach to assess the social capital in LEADER-CLLD: How does it work? (3)

Evaluation Findings

11

Related aspects of governanceStructural social capital Normative-cognitive social capital

INDICATOR A3: value 0.01Beneficiaries’ level of indirect

knowledge of projects supported by the LAG

https://www.galprealpidolomiti.it/capitale-sociale/indicatore/

INDICATOR H5: value 0.07Innovative capacity of the LAG as

perceived by beneficiaries


INDICATOR O2: value 0.12Understanding the role of the LAG in

the territory


https://www.galprealpidolomiti.it/capitale-sociale/indicatore/https://www.galprealpidolomiti.it/capitale-sociale/indicatore/https://www.galprealpidolomiti.it/capitale-sociale/indicatore/https://www.galprealpidolomiti.it/capitale-sociale/indicatore/https://www.galprealpidolomiti.it/capitale-sociale/indicatore/https://www.galprealpidolomiti.it/capitale-sociale/indicatore/

Lesson learnt and follow-up actions

12

Related aspects of governanceStructural social capital Normative-cognitive social capital

Example of action: training courses

Example of action: dedicated tool

Example of action: innovative agriculture

https://www.galprealpidolomiti.it/corsi-di-formazione/https://www.galprealpidolomiti.it/corsi-di-formazione/https://www.galprealpidolomiti.it/progetto/pc-1-da-lago-a-lago-lungo-il-piave/https://www.galprealpidolomiti.it/progetto/pc-1-da-lago-a-lago-lungo-il-piave/https://www.galprealpidolomiti.it/incontri-misura-16-cooperazione/https://www.galprealpidolomiti.it/incontri-misura-16-cooperazione/https://www.galprealpidolomiti.it/corsi-di-formazione/https://www.galprealpidolomiti.it/progetto/pc-1-da-lago-a-lago-lungo-il-piave/https://www.galprealpidolomiti.it/progetto/pc-1-da-lago-a-lago-lungo-il-piave/https://www.galprealpidolomiti.it/incontri-misura-16-cooperazione/

Reflections on the evaluation approach Challenges encountered in assessing the added value of LEADER: Cultural challenge Simpler definition of social capital for communication purposes

Elements that worked out well, and could be transferred to other Member States: Training of the staff and interviewer Graphical representations (e.g. network graph, spider diagram, etc.)

Overall ability of the approach to show the added value of LEADER: Number of indicators is the real added value of the approach Social capital becomes “tangible”

Possibility to upscale the approach at RDP level: Require strong coordination between the LAGs and the MA

Critical elements to consider: The SNA method has direct costs and indirect costs

How to connect social capital with improved results


Thank you

Elena Pisani, Dep. TESAF University of Padova, [email protected]://www.palgrave.com/de/book/9783319542768https://orchid.org/0000-0002-8918-2781

Matteo Aguanno, LAG Prealpi e Dolomiti, [email protected]://www.galprealpidolomiti.it/monitoraggio-e-valutazione/


mailto:[email protected]://www.palgrave.com/de/book/9783319542768https://orchid.org/0000-0002-8918-2781mailto:[email protected]://www.galprealpidolomiti.it/monitoraggio-e-valutazione/

The assessment of the added value of LEADER/CLLD as improved social capital in the LAG Prealpi & Dolomiti (Italy) �Outline LAG Prealpi & DolomitiBackground and evaluation needsThe evaluation approach �The network approach: using networks to measure and show the social capital in LEADERThe network approach to assess the social capital in LEADER-CLLD: How does it work? (1)Slide Number 8Example of evaluation elements used in the network approachThe network approach to assess the social capital in LEADER-CLLD: How does it work? (3)Evaluation FindingsLesson learnt and follow-up actionsReflections on the evaluation approach Thank you

the assessment of the added value of leader/clld as...

Documents