mapping administrative data - world...
TRANSCRIPT
![Page 1: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/1.jpg)
Capacity Building workshop on Impact Evaluation of Employment Programs
Mapping Administrative Data
Matteo Morgandi, Gdańsk, February 22, 2017
![Page 2: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/2.jpg)
Basic setup of a randomized evaluationRandomization methodsIndividuals/Clusters Multiple/Cross-cutting treatments
3. Random Assignment
Treatment group
Control group
NEET Youth 15-29
1. Enrollment of potential
beneficiaries
2. Baseline data
Youth training program offered by poviat labor office
7. Decision
on program based on
results
4. End-line data
5. Data Analysis
![Page 3: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/3.jpg)
National Power Program (ESF)
Regional Labor Office
Programs for 15-29
Labor fund
Poviat Labor office
Employment programs
Regional Operational Program
Marshall Office
Regional Labor office
Poviat Labor office
Employment programs
Implementation of ESF funded labor programs
![Page 4: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/4.jpg)
National Power Program (ESF)
Regional Labor Office
Programs for 15-29
Labor fund
Poviat Labor office
Employment programs
Regional Operational Program
Marshall Office
Regional Labor office
Poviat Labor office
Employment programs
Possible avenues for Impact Evaluation of ESF funded labor programs
MR resources for
impact evaluation
1 2
3
ROP resources for
impact evaluation
![Page 5: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/5.jpg)
Main actors involved in the design and execution of an impact evaluation
Regional Labor office
Delivery Agency
Poviat Labor Office
Research Institution
![Page 6: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/6.jpg)
Main actors involved in the design and execution of an impact evaluation: roles
Regional Labor office
Delivery Agency
Poviat Labor Office
Research Institution
Has funding for IE. !
Contracts out evaluation
Carries out the Analysis of the data at baseline and endline
Can collect new data
Implements the program
Can collect new data on beneficiaries
Assigns beneficiaries to the program
Supervises implementation of program
Data collection and management
![Page 7: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/7.jpg)
Data for program management vs data for program evaluation
Individual Personal Data Demographic Information Baseline data Intervention
Endline data
Participantnumber
Name Last Name
PESEL Phone Number
Address
Age Education
Sex Work status
Earnings
Treat/ Control
Workstatus
Earnings
1
2
3
4
5
6
7
8
9
![Page 8: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/8.jpg)
Data for program evaluation can be analyzed anonymously
# Individual Personal Data
1
2
3
4
5
6
7
8
9
# Demographic Information
Baseline data Endline data
1
2
3
4
5
6
7
8
9
Anonymization
![Page 9: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/9.jpg)
Database for program evaluation: Syriusz• Operational database for Poviat Labor Offices
• All necessary variables for program management (personal , demographics)
• Collects key variables for program evaluation (participation in measure, unemployment duration, labor market status in connection with ZUS database)
• Can be operated and analyzed by Poviat labor office only
• Designed and connected with central database in Ministry of Famly, Labor Social Policy (Cezar) and ZUS database (Broker)
![Page 10: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/10.jpg)
Information system to monitor Labor Market Measures
Central Application:MFLSP database (non-anonymized personal
data)
Syriusz: Poviat LO database
Checking the current, past and future labor market situation of the registered unemployed
CEZAR(anonimized data)
PESEL, participation in the programs, even information from profiling
Viator: Regional LO database
MFLSP statistics
BROKER: MFLSP
platform
ZUS database
![Page 11: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/11.jpg)
Other data collection options
• SL 2014 Database (Ministry of Development): • Personal variables for beneficiaries identification, can obtain matched
information on labor market outcome with ZUS database • Limited number of outcome and demographic variables, no customization
possible• Data can be exported legally to authorized evaluation agency to conduct
follow-up survey• Data entry can be done by any authorized entity (e.g. service provider)
• Phone / Online / In person Surveys • Flexible tool to collect additional variables at different points in time • More Expensive than administrative data• May require prior consent of beneficiaries to collect data in future
![Page 12: Mapping Administrative Data - World Bankpubdocs.worldbank.org/en/932191500920430881/PT4-Mapping... · Main actors involved in the design and execution of an impact evaluation: roles](https://reader034.vdocuments.site/reader034/viewer/2022051906/5ff90aa88ba53f724f4145a9/html5/thumbnails/12.jpg)
Group exercise: adapting the impact evaluation process to Polish institutional reality
• Divide into 6 groups of maximum 7 people each:
• Identify:• a measure that you want to evaluate
• the indicators that you need to track to measure impact
• detailed steps to conduct the evaluation across time
• Clarify along the process “who is doing what”
• Data tool that would be used to collect and analyze the data
• Map on a flip chart the process