Evaluation of NSRF measures on the development of public

education

Gábor Balás Hétfa Research Institute

30 April 2013

Goals and focuses of the evaluation

Goal: Evaluation of measures of the 2007-13 period for public education• Education programme development,• teachers’ training,• equipment stock development,• real estate development from 5 aspects:• relevance (targeting),• linkage with the institutional integration,• factors affecting demand and fund absorption,• results, effects,• synergies.

Additionally• Some other (e.g. EEOP) calls, • Kindergartens only in case of real-estate developments

Background and methodology of the evaluation

Three important background process from 2007:• „PISA shock”• reduction of funding• transformation of the institutional system of the education

Recent changes relevant to the recommendations:-regulations of the new public education act-establishment of the Klebelsberg Institution Maintanace Centre

Mixed qualitative and quantitative methodology: • econometric analysis (EMIR-competencies assessment-TeIR), • 16 case studies of schools, • 18 expert interviews, • document analysis

Evaluation of relevance

Strategic relevance before 2010:• The controversial basis: the medium term development

strategy (Ministry of Education 2003);• 2007-2008 prolonged negotiation process (Ministry of

Education and Culture, HRP-MA) • However: the evaluated calls fit to the strategy

Strategic relevance after 2010:• remarkable paradigm shift (public education act);• The most recent calls follow this, yet with the mitigation of

the risks;• The change in management cause some, yet manageable

risks

Assessment of targeting – 1.

Where project selection exists (e.g. real-estate):•The development aspects besides public education objectives make an effect (sustainability territorial cohesion vs. competencies results, equipment supply)

Variables Applied Rejected from applicants

With existing contract from applicants

the sponsoring organisation 0,2256 -0,0224 -1,8590economic power per capita -0,0577 0,0826 5,4826change in the number of population aged 3-5 0,0078 0,1827 15,5673The institution is in a larger villages 0,3776 -0,2553 2,6154The instititution is in a county seat city -0,4045 0,2117 21,0628LAMR developed with a complex programme 0,3276 -0,2619 1,4801micro-region with small villages 0,1580 -0,2653 9,0678N 14150 790 790

Probit coefficients: Real-estate development, in case of convergence regions

Source: EMIR - competencies assessment – TeIRCoefficients marked grey are significant

Assessment of targeting – 2.

In case of „Sprinkle" funding (teachers, equipment, content):•The applicant activity and the capacity are the decisive factors,•The institutions with weaker capacities are left out

Variables Applied Rejected from applicants

With existing contract from applicants

economic power per capita -0,1188 0,1542 -0,1159The institution is in a village -0,4604 0,1968 -0,0985The institution is in a larger village -0,3577 0,4764 -0,8154The instititution is in a county seat city 0,1425 0,1594 -0,0549Settlements with low tax capacity -0,1722 -0,2110 0,1190Settlements with high unemployment -0,1731 0,3514 -0,1783N 14150 406 406

Probit coefficients: Teachers’ training measures, in case of convergence regions

Source: EMIR - competencies assessment – TeIRCoefficients marked grey are significant

Demand analysis

Relationship with the institutional integration:•There is no linkage between them, but •We have not even found such intention, however•It has helped the integration (e.g. common trainings etc.).

Factors affecting popularity:•Applying pressure for funds, but adjustable for own needs

– Broad scope of activities allowed– Large project size (interoperability, flexibility)– Fitting to the existing (funding searching) programmes

Factors and importance of the demand•The demand affects more, than the NDA’s intention•The educational performance affects only via territorial aspects•Application consultants are important, the larger schools built their own capacities

Demand assessment – Illustration 1.

Geographical distribution of grants for equipment stock development per school-age inhabitants, and distribution of successful applications

Geographical distribution of grants for teachers’ training awarded per school-age inhabitants and geographical distribution of granted applications

Demand assessment – Illustration 2.

Distribution of real-estate development grants awarded per school-age inhabitants and geographical distribution of granted applications

Demand assessment – Illustration 3.

Geographical distribution of grants targeting quality improvements of education programmes per school-age inhabitants and geographical distribution of granted applications

Demand assessment – Illustration 4.

Impacts

The time scale of the databases are too short to conduct data analysis, but

According to our case studies there are positive effects, e.g. on:– the organisational structure of the institutions, – the popularity of subjects, – the decrease of number of the absents without leave,

Which can have an impact on the competencies results in the future

Questionable:• How quickly they will be transformed into competencies

results;• Will they be sustained on the long term (e.g. the retirement of

the teacher is a negative risk).

Evaluation of Synergies 1.

Linkage between the calls: • The interoperability between the OPs grew;• The typical co-applications are not ones which the

authorities planned

Synergies on the institutional levelThe institutions are trying to build the developments on each otherThe granting scheme does not help, yetThere are examples of success;• The main motivation however is the continuation of existing

educational program, among the scarcity of funds

Evaluation of Synergies 2.

example: correlation of the applicant groups in the convergence regions

Equipment stock

Educational programme

Teachers’ training

Real-estate stock

Other SROP EEOP Other

Equipment stock

1,0000

Educational programme

0,2460 1,00000,0000

Teachers’ training

0,0183 0,2424 1,00000,2563 0,0000

Real-estate stock

0,3781 0,2466 0,1049 1,00000,0000 0,0000 0,0000

Other SROP 0,0354 0,2849 0,3026 0,1525 1,00000,0282 0,0000 0,0000 0,0000

EEOP 0,1318 0,2043 0,1400 0,1245 0,2062 1,00000,0000 0,0000 0,0000 0,0000 0,0000

Other 0,1993 0,2670 0,1935 0,3881 0,2452 0,1463 1,00000,0000 0,0000 0,0000 0,0000 0,0000 0,0000

Recommendations 1.

The reasons behind the lack of efficiency of the developments:-The high costs of coordination as obstacles for synergies-The lack of information about the development needs

The appearance of the central government as direct sponsoring institution in the public education means new possibilities, but it is also a challenge.

Recommendation:• The design and implementation of the developments should

be more strongly integrated into the system of the education policy;

• The synergies should be strengthened by the support of interoperability and not by the system of parallel calls;

• The methodological key projects supporting developments should be the ones started first;

• Detailed demand and capacity assessment should be started along the goals of public education system, but in a comprehensive manner;

• Instead of granting– For the funding of compulsory tasks,– On the basis of these problem maps,– Need indicator based funding should be introduced, – with the supervision of school-district authority;

• For the non-compulsory tasks, there should remain grants which are rewarding creativity, innovation, and in the same time maintain the motivation of schools and strengthen their governance capacities.

Recommendations 2.

Thank you for your attention!

Hétfa Research InstituteH-1051 Budapest

Október 6. utca 19.www.hetfa.hu