· web viewthe participatory approach should be central in the evaluation approach, ensuring the...

TERMS OF REFERENCE FOR SERVICE CONTRACTING

Assignment Evaluation of UNICEF-supported programme “Ma’An Together Towards a Safe School Campaign” with its sub-components ‘Tarbiyeh: towards behavioural transformation in schools’; the school advocacy groups (SAG); school-level activities and monthly survey.

Programme area and specific project involved

This study is part of the Annual Work Plan 2016 signed between Ministry of Education and Child Protection programme of UNICEF. It is under the Intermediate Output 5.1: Children and families are mobilized to prevent and reduce violence and harmful practices.

Location Country-based (Jordan)

Duration 5 months

Estimate number of working days

113

Reporting to Chief Child Protection and Chief of Planning, Monitoring & Evaluation

of 18

1. BACKGROUND

The Jordan Ministry of Education banned corporal punishment in 1981. However, despite the ban, children still experience high levels of emotional and physical abuse at school, and also at home, as revealed in a UNICEF-supported national survey in 2007 (UNICEF, 2007)1. The survey found that more than two thirds of children were subjected to verbal abuse by teachers and school administrators (71 per cent) as well as by their parents/legal guardians (70 per cent). More than a third of children (38 per cent) reported being subjected to verbal abuse by adults in the community. Around 57 per cent of children reported physical abuse by teachers and school administrators, 34 per cent by parents/legal guardians and 13 per cent by adults in the community. The study also revealed wide acceptance of corporal punishment among families, who saw it as an effective tool for changing children’s behaviour.

To reduce this violence, in 2009 MoE, UNRWA and Military Education Schools started the implementation of Ma’An (Together) Towards a Safe School campaign with UNICEF support, in close collaboration with the Office of Queen Rania Al Abdullah as advocate for non-violence, This nationwide campaign aims to reduce of violence against children by educators in all schools, it promotes new disciplinary methods in schools, advocates to end societal tolerance of violence in schools and supports media coverage to spread the message nationwide. The campaign’s initial design include:

Administrative mobilization and advocacy: Through meetings and personal letters, the Ministry of Education promotes the campaign and non-violent discipline to all principals, teachers and school and municipal administrative staff.

Interpersonal communication: Each school has an advocacy group comprised of the principal, the counsellor, two teachers who use positive disciplinary measures, two parents from the Parent-Teacher Association and four students from the Student Council. The members of this group promote the new disciplinary methods among teachers and principals through monthly discussions based on the results of the school surveys. The school counsellors have been trained on the Ma’An campaign principles and approaches, and they train the other members of the advocacy group. Huge capacity building component is designed to support teachers and counsellors.

Media campaign: Three times a year, a three-week media campaign is organized on radio, television, newspaper, social media and the Ma’An website. (has not been implemented in the last two years)

Community mobilization: Community and religious leaders from municipalities, districts and governorates, trained by the Ministry of Interior and Ministry of Awqaf and Islamic Affairs on advocacy skills and the principles of the Ma’An campaign, conduct meetings with opinion and tribal leaders. The goal is to promote a culture of zero tolerance of violence against children and support schools in their efforts to reduce violence.

Within the campaign School Advocacy Groups were established to effectively support the over-all Ma’An activities including developing, implementing and monitoring school based activities to reduce violence in schools such as implementing the monthly violence survey and presenting recommendations and taking corrective measures to address gaps when necessary.

1 UNICEF, Violence Against children in Jordan, Khalil Elayan, 2007.

of 18

In 2011, an assessment of the campaign was conducted revealing that teachers felt they were incapable of managing their classrooms without resorting to violence. This revealed a clear need to equip all teachers with classroom management skills. In addition, some of the strategies used by the Ma’An campaign were seen as less suitable for older children. Based on these results, UNICEF implemented a programme for behaviour management and introduction of classroom management tools in 6 pilot schools across the Kingdom. Based on positive outcomes of the pilot phase, this ‘Tarbiyeh’ programme was scaled up to 50 schools. Tarbiyeh is a School Transformation Programme reinforces positive social behaviours among teachers and students through positive house system rewards, through development of tools and computerized system to provide and monitor hands-on training and coaching of counsellors and principals, advocacy and team building events.

Latest figures show from the online MoE school survey2 significant reductions of verbal and physical violence rates over school years from 2012-2015. On the other hand, the percentage of positive disciplinary alternatives also dropped.

Table (1): percentages of violence and positive disciplinary alternatives in total schools participating in survey for school years 2012-2015School year Verbal violence (%) Physical violence

(%)Positive disciplinary alternatives (%)

2012-2013 33.4 20.9 54.82013-2014 25.1 15.6 47.02014-2015 20.4 12.8 43.22015-2016 18.38% 10.88%

Due to the slow reduction in violence rates in schools, UNICEF started in July 2013, a comprehensive transformative behavioural program to reduce violence against children in schools ‘Tarbiyeh’ which equips teachers with skills and tools to manage student's behaviour in the classroom and schools at large has been implemented in six selected schools (boys/girls- Government public and UNRWA schools), scaled up to 50 schools in 2014-2015. The program is designed for a whole scholastic year (2013-2014) and also focuses on equipping students with social skills and intensives to encourage self –discipline. The program includes a computerized system that helps schools track the activities and progress made.

Implementing partners of the Ma’An Campaign (Ministry of Education, Sharek Academy, Jordan River Foundation, UNRWA, Military Education Schools and Save the Children) have reported across the implementation time on progress made. For the School Advocacy Groups and Tarbiyeh components of the Ma’An campaign, in the school year 2014-2015, 12,407 (7,572 female, 4,835 male) members of School Advocacy groups were reached and 11,487 (4,320 female, 7,167 male) students through the Tarbiyeh programme. In addition, school counsellors were trained on the inter-agency Standard Operating Procedures and referral pathways for child protection (CP) and gender-based violence (GBV) (1,808 counsellors reached, 60% female) and the case management manual for CP and GBV (trainings for core team to be provided in November-December 2015).

2 MoE, Online School Survey Data, 2016

of 18

In 2016, UNICEF started the implementation of Tarbiyeh in six schools in Za’atari camp to test the methodology of Tarbiyeh and its successes in reducing violence in schools in a camp settings. Attached is focus group report that highlights the issues related to violence in schools as seen by children, teachers and families.

Given the scale-up of the different components (Tarbiyeh, Advocacy Groups, survey, school-level activities, etc.) of the Ma’An campaign from 2010 and the resources dedicated, UNICEF is conducting an evaluation of the scale-up of the key components of the Ma’An campaign to generate evidence based knowledge of the impact of the campaign on the reduction of violence in schools.

2. PURPOSE, OBJECTIVE AND UTILITIES

The main purpose of the consultancy is to determine the relevance, efficiency, effectiveness and impact of the Ma’An Campaign interventions on reducing violence against children in schools in Jordan. The evaluation will look at alternative implementation, future strategy and lessons learnt of the campaign.

The findings of the evaluation will valuable to redirect and adjust the government, UNRWA and UNICEF response to violence in schools. The evaluation will look at the Ma’An campaign as a whole to identify good practices and areas for improvements, this will guide the way forward for Ma’An campaign and guide the scale up of certain components of the campaign that is most effective while ensuring a holistic, comprehensive programme informed by evidence. The evaluation will be an important opportunity to document effectiveness and outcomes/impact of the Tarbiyeh programme in the schools that it is piloted in.

The evaluation will be mainly used by the Ministry of Education, UNRWA and Military Education Schools to improve their strategies in reducing violence. It is expected to be used by UNICEF Jordan Country Office to adjust it support to the campaign and guide the rollout of the Ma’An campaign. Regional Office, global Child Protection working group and other CP actors will use the evaluation as a case study for programmes that managed to reduce violence in schools. Secondary audiences include donors and media, as the findings will also be used for reporting and advocacy purposes.

More specifically, implementation of this evaluation will aim for the following key specific objectives:

A. To assess the relevance of the Ma’An Campaign and in particular the Tarbiyeh, Monthly School Survey, and School Advocacy Groups components to national priorities and strategies to combat violence from the viewpoint of policy makers, educators, and its relevance to the needs of children, families and communities, taking into account the appropriateness for and social acceptability by school management, children, teachers, caretakers and communities.

B. To assess the effectiveness of the Ma’An Campaign and in particular the Tarbiyeh, Monthly School Survey, and School Advocacy Groups components and to measure to what extent the programme has achieved its set results, including behaviour change among students and teachers in the schools targeted. To identify the value of the Tarbiyeh sub-component for achieving the Ma’An overall objectives. The research will compare the effectiveness of the campaign in the various settings, boys/girls; camps/host communities; double shift/single shift, different age groups of students to inform the development of targeted strategies for reducing violence in these settings. The evaluation will conduct a comparison between the implementation of the campaign in different contexts to analyse supporting and hampering factors that are in these contexts.

C. To assess the efficiency of the Ma’An Campaign and in particular the Tarbiyeh, Monthly School Survey, and School Advocacy Groups components and to what extent the programme has used

of 18

resources (human, financial and others) in an efficient manner, and do the achieved results justify the resource? To assess the cost of the response per unit of aggregation as compared to the cost being incurred by other CP programmes implemented in Jordan.

D. To assess the impact of the Ma’An Campaign and its components in capturing changes (positive, negative, intended and unintended) and differences the programme may have brought including a gender perspective. What has been the impact/effect of the programme on teachers, students, parents, other groups?

E. To assess the sustainability of the Ma’An Campaign and in particular the Tarbiyeh and School Advocacy Groups components and its results, considering the likelihood of programme sustenance in the absence or reduction of ongoing UNICEF and donor support. This is to be done by identifying the degree to which the Ma’An campaign has built on existing institutional and local capacities, and a potential exit strategy.

F. To document lessons learnt and good practices that will inform future programming and propose recommendations to deliver the Ma’An campaign activities in a more effective, efficient and sustainable way, and possible scale up of the campaign, suggesting different options for UNICEF, Ministry of Education and partner organizations

3. SCOPE OF WORK

The evaluation will cover, as a main focus, all aspects of the Ma’An campaign, and more specifically the Tarbiyeh, School Advocacy Groups components, school-level activities and monthly survey, the media and community mobilization activities at the startup of the campaign. The time period under evaluation will be from the start-up of Ma’An campaign in 2010 to end of the scholastic year 2015/2016 for Ma’An campaign and this includes in particular the programme period for the EU funded scale-up of Tarbiyeh and School Advocacy Groups components (project period 25 March 2014 – 28 February 2016).

Since the Ma’An campaign is covering all schools in Jordan from grade 4 till grade 10, the School Advocacy Groups programme covers 2,000 schools and the scale-up of Tarbiyeh covers 56 schools, the scope of the evaluation is expected to cover all these categories with sufficient representation for each component.

The participatory approach should be central in the evaluation approach, ensuring the voices and views of all stakeholders are reflected. While all stakeholders are important, special emphasis will be placed on girls and boys and teachers and their perceptions around the programme in their school environment. Within the selected schools to be part of the evaluation, different target groups will be included in the exercise, including male and female students, school counsellors, teachers, principals and parents (mothers and fathers) and other stakeholders such as community leaders, Imams, media personnel.

The evaluation will focus on all aspects of the Ma’An campaign targets and as such is not an evaluation of humanitarian action. However, there will be dedicated attention to the situation in double-shifted schools as well as camp schools.

4. EVALUATION CRITERIA AND TAILORED EVALUATION QUESTIONS

of 18

The evaluation will focus on the expected results of the Programme, and provide evidence-based analysis in order to answer the following questions that are clustered by evaluation criteria below. It should be noted that the questions suggested are for guidance only and they are not meant to be exclusive. The applicants applying for this consultancy can modify the questions to respond to the purpose and objectives of this evaluation. Questions will be finalized in the inception phase.

Programme relevance /appropriatenessKey questions:

a) How relevant is the Programme strategy with regards to the overall national priorities? To what extent is the Ma’An campaign and its components are in line with the relevant national strategies and programmes?

b) ?How relevant and appropriate are Programme interventions in responding to 1) demand and needs of students (differentiated by gender) and including all refugees in camps and host communities; 2) demand and needs of school counsellors, teachers and other government actors? 3) demand and needs of parents of students. To which extent did the Programme address particular/differentiated needs of targeted/reached different school types, i.e.; UNRWA, double-shifted (Jordanian/Syrian), girls/boys schools, different grades, camps/host?

Programme efficiencyKey questions:

a) To what extent the Programme’s governance was efficient in ensuring the achievement of Ma’An results?

b) How effective and efficient were the coordination mechanisms at the country level (i.e. coordination with Government, including within Ministry of Education, UNRWA, Queen Rania Teaching Academy and other implementing partners, other stakeholders (other UN agencies, NGOs, donors, etc.)? If noticeable gaps are evident, how can they be addressed?

c) To what extent were the Programme activities coordinated with other sectors within UNICEF such as Education and communications sections?

d) To what extent UNICEF’s implementing partners were able to implement the interventions? What capacity / expertise is in place or does it require additional / different capacity or

resources?e) How adequately did UNICEF programmatically respond to unforeseen contingencies over the

course of the Programme’s implementation?f) How adequate are the developed monitoring and reporting indicators to provide

information in a timely manner?g) How often were outcomes and outputs measured? Was data sufficiently disaggregated to

identify all target groups?h) To what extent were funds used in a cost efficient manner in order to optimise Programme

outcomes? How systematically have the funds been allocated/utilised (across administrative levels

and programme strategies/activities) to realise Programme objectives? If there are delays/deviations in fund allocation/utilisation, how were these justified and

what are the implications for attaining Programme objectives? What lessons and recommendations can be drawn for the future?

Programme effectiveness

of 18

Key questions:a) Has the main goal of Ma’An campaign been met?b) To what extent has the Programme contributed to decreasing violence levels in schools and

enabling transformative behaviour change? Has the programme been able identify and reduce the causes of violence in schools; sociocultural norms; teachers status; over crowdedness; violence at home and communities; and gender inequality in access and permanence in schools? To what extent did the Programme contribute to increasing knowledge and service providers’ practices in addressing violence and promoting transformative behaviour change?

c) What are the major achievements contributable to and lessons learned from the Programme; which strategies have yielded good results? What are the gaps that need to be addressed? Review of the theory of change for Ma’An campaign and identify which changes that didn’t contribute to the desired results

d) Examine the campaign contribution to the decline in alternative positive measures and teachers’ perception’s around it. Were the stakeholders targeted in the Programme activities the most relevant and how

were selection criteria developed and used?

e) What are the key successes achieved of the Programme? Are the key successes well documented and disseminated? Has the knowledge acquired through this programme was put to use to improve

programme implementation and decision-making process (i.e., adjusting the planned results/targets, shifting programme focus)

To what extent and in what ways partnerships were strengthened amongst various parties including Government, NGOs and others to reduce violence i.e. religious leaders; community members?

To what extent has the computerized system impacted the planning process for the Advocacy group?

Map the various programmes/interventions at the school level to support the reduction of violence

f) To what extent was the Programme successful in using key elements of result-based planning and management?

g) What strategies and approaches that were part of the Tarbiyeh and School Advocacy Group scale-up have had a demonstrable result in impacting violence levels and how? The suitability of Tarbiyeh strategies to students of all age groups/grades and gender.

Role of the media and community mobilization on Ma’An implementation To what extent was the media (whether social media or otherwise) influencing the

implementation of the campaign and in which direction? Was the campaign successful in changing deeply rooted beliefs about child disciplining? To what extent has the programme managed to engage with the media, opinion leaders and

Imams?

Programme impacta) What lasting changes can be identified in the students and teachers’’ behaviors, schools

principals and school environment,b) Has the programme impacted student learning achievements and schools attendance? How

does the Programme support children’s learning through influencing the quality of teaching and learning and creating a positive learning environment?

Programme sustainability

of 18

The evaluation will assess administrative, institutional, technical and financial sustainability and explore possible opportunities for expansion of the Ma’An campaign activities:

a) What are major factors that influence the achievement or non-achievement of sustainability of the programme outcomes? What conclusions can be drawn regarding sustainability and expansion/replicability of the

approaches used? What are the issues and options related to the feasibility for replication and expansion

To what extent the Government and UNRWA were engaged and owns the Ma’An campaign initiative at the national and sub-national levels?

How likely the government and particularly MoE and UNRWA can sustain and replicate the programme interventions without direct UNICEF technical and financial support?

What is the ministry plan to sustain the campaign? What are the regulation and policies alignment to sustain the campaign? Need to look at financial; human resources, etc.

What measures have been taken to document the lessons learned to allow for sharing with government and donors for further scaling up and replication?

b) To what extent have programme decision making bodies and implementing partners undertaken the necessary decisions and course of actions to ensure the sustainability of the effects of the programme?

c) What is the contribution of the Programme to putting in place systems at school, directorate and national level for monitoring and following up on violence in schools? To what extent did the programme contribute to the capacity of all stakeholders at

community, school, directorate and organizational (including ministerial) level to address the issue of violence in schools effectively?

Additional Evaluation Questions (Cross-cutting issues)Human rights-based approach in programming (HRBAP) and gender equality:

a) How successfully were the key principles of HRBAP applied in planning and implementing the Programme? In particular:

To what extent were the national and local context (knowledge, beliefs, gender and cultural differences) taken into account when the Programme was designed?

b) To what extent has the Programme contributed to equity, including gender equity? Which groups of children benefited and which did not? Why? Were there differences in programme outputs related to sex, different groups (i.e. Syrian,

Bedouin, urban, etc.), economic status, and geographic location? What specific activities have taken place across the Programme by UNICEF and other

partners to facilitate the inclusion of children with disability? To what extent gender equality existed in participation, decision making and access

throughout the program cycle?

5. METHODOLOGY

The evaluation should use a range of different methods to analyse the Ma’An campaign. It should be as participatory as possible to be able to get information from various stakeholders, including girls and boys. A specific and detailed evaluation design should be presented to UNICEF by the evaluation team based on the scope of the evaluation mentioned above. The key evaluation questions should be unpacked into sub-questions as appropriate, and should be answered, validated and triangulated through appropriate methods of data collection and analysis.

of 18

A representative selection of schools for grades (4-10) (targeting principals; teachers; counsellors; students and their families) will be made to ensure that the sample is clustered into different geographic locations (north, middle and south of Jordan; urban, rural, Bedouin areas), type of schools (boys/girls, MoE, UNRWA and Military Education Schools, double-shifted (Syrian and Jordanian), camps), different grades represented to assess the programme’s effectiveness across the board. Data analysis should also include the difference that the Programme had on the schools that were most impacted by the Syria refugee crisis in the country, such as the double-shifted schools as well as the schools located in the Za’atari refugee camps. As a control group, a number of schools that are not part of the scale-up of Tarbiyeh will be included in the evaluation exercise.

Key informants for the evaluation will include representatives of Ministry of Education, UNRWA and Military Education Schools school management, UNICEF, NGOs, media, Imams and implementing companies and partners.

UNICEF will provide necessary information including all available data, reports, and the mapping of all targeted schools. Available data for the evaluation includes UNICEF and partner reports, the 2007 violence study, monthly survey reports (from September to December and from February to May each year since 2011 onward), two focus group reports, Tarbiyeh pre and post evaluations and programme documents (Programme Cooperation Agreements with MoE, UNRWA and Sharek Academy, Concept Note, Project Plans, and Plans of Actions). The evaluation team is expected to propose relevant methods for data collection and analysis to respond adequately to the evaluation questions listed in the ‘Evaluation Criteria’ section. The proposed methodologies include:

a) Desk Review of secondary data and documentsb) Interviews with key informantsc) Surveyd) Field observatione) Focus group discussions with children, boys and girls, of different agesf) Specific FGD with male and female teachers and parents/caretakers of childreng) Structured and semi-structured interviews with relevant stakeholdersh) Case-studies based on the most significant change technique

The list above is for reference only. A selection of participants and locations should be made based on representative sampling methods as per the criteria mentioned before.

As part of methodology, it is essential to articulate how ethical considerations will be taken into account in the design of the evaluation and throughout the evaluation process. As stipulated in UNICEF’s relevant procedure for ethical standards, the proposal must identify actual or potential ethical issues, as well as measures and methods adopted to mitigate against these issues.

6. EXPECTED DELIVERABLES & DELIVERY DATESA proposed time frame for the evaluation is provided below. The evaluation is expected to be completed within about five months. This might be subject to change depending on the prevailing situation on ground at the time of the evaluation and as per mutual agreement between the evaluation team and managers.

of 18

http://www.unicef.org/jordan/VAC_Study_English_FOR_SCREEN.pdf

http://www.unicef.org/jordan/VAC_Study_English_FOR_SCREEN.pdf

# Deliverables Type and language

Delivery Date

1 Detailed inception report Arabic and EnglishThe inception report will be submitted within two weeks after signing the contract and initial briefing with evaluation manager and/or evaluation reference group. The report will be maximum 15 pages in length and should be in line with UN standards for inception report. The report should outline the detailed scope, theory of change, evaluation framework, methodology, sampling, field visit schedule and timing, data collection methods and tools, timeline for activities and submission of deliverables. It should also include initial data

Electronic in English and Arabic

10th

September

of 18

Activity Duration Tentative Due Datekick-off meeting One day 25 AugustReview of literature related to the Ma’An campaign. Conduct initial review of available data to identify gaps and data collection needs. Presenting to UNICEF a detailed Inception report describing the assessment design, including, detailed work plan, methodology, the outline of the final report in addition to the inception report in English and Arabic and tools and protocols for data collection

Two weeks 10th September

Review of inception report by UNICEF and Reference Group

10 days 20th September

Second review of the revised inception report 4 days 25th SeptemberTesting and revision of the tools, data collection training as required,

15 days10th October

Data collection in the field including FGD 30 days 10th NovemberData analysis and writing of first draft of the report in English and Arabic

15 days 25th November

Review of first draft by UNICEF and Reference Group

10 days 5th December

Incorporation of comments and produce second draft


Conduct validation workshop with key stakeholders to review the analysis and develop recommendations


Finalization and submission of the final report, along graphs; PowerPoint in Arabic and English with raw data


and findings based on the desk review and final evaluation design/plan. A draft of the inception report will be shared in advance for comments and will be approved by the evaluation reference group.Data collection tools and methodologies especially related to interviewing children and the ethical considerations impeded.


10th

October

2 Draft report Arabic and EnglishThe draft final report will be in line with UNICEF-Adapted UNEG Evaluation Reports Standards and contain the same sections as the final report. It will be 50 to 70 pages in length (excluding annexes). It will also contain an executive summary of no more than 3 pages that includes a brief description of the programme, its context and current situation, the purpose of the evaluation, its methodology and its main findings, conclusions and recommendations. The draft report will be shared with the evaluation reference group to seek their comments and suggestions.


25th

November

3 Final ReportThe report should be in line with the UNICEF-Adapted UNEG Evaluation Reports Standards should be between 50 to 70 pages in length excluding an executive summary and all annexesThe executive summary of no more than 3 pages should include a brief description of the programme, its context and current situation, the purpose of the evaluation, its methodology and its major findings, conclusions and recommendations.

Electronic and hard-copy in English and Arabic

20th

December

4 PowerPoint PresentationPresentation based on the finalised evaluation report summarising the entire process, purpose, objectives, methodology, findings and recommendations and conclusions.


20th

December

5 InfographicsAt least one infographic, up to two pages each summarizing the quantitative and qualitative findings of the evaluation using text, charts and diagrams. The info graphics will be used for advocacy purposes and less technical terminology should be used. Info graphics can be drafted and discussed in English first and the final version translated into Arabic.


20th

December

The evaluation team should propose a timeline to submit the deliverables in their proposals.Necessary and adequate time (at least two weeks) should be allocated for review and quality assurance processes of the deliverables by the evaluation reference group.

7. DESIRED QUALIFICATIONS, SPECIALIZED KNOWLEDGE OR EXPERIENCE AND TEAM COMPOSITION

The evaluation is planned to be conducted by an institution or by a registered consultancy group/firm. The team will be comprised of a team leader and team member(s), ensuring gender

of 18

balance with qualifications, skills and experience stated below. If the evaluation is carried out by an international firm, the team leader will ensure that a national expert is included in the team.

The Team Leader should have at least a Master’s Degree in Education, Sociology, Anthropology, Social Sciences, statistics or a related field and at least 10 years’ experience in programme evaluation and must have completed at least two high quality programme evaluations in that period, at least two of them being related to education or violence related response. Production of sample work is required.

The qualifications and experience required for the evaluation team are as follows: Each member should have at least 7-10 years of experience in leading/conducting of similar

scale and level of complex evaluations. (CVs required) Combination of advanced degree in education, protection, international development, C4D

and/or an associated field; Experience with and strong skills in quantitative and qualitative research and approaches; Evaluation team leader should have relevant past experience in leading evaluation of similar

scale and complexity. Experience in evaluating child protection programme preferable. Evaluation team should include members with more than 5-year working experience

(international and Arabic speaker) in the area of child protection, and more than 2 year working experience (international and Arabic speaker) in the area of education.

Experience in the use of participatory appraisal techniques in data collection, sensitive to gender issues

Knowledgeable of institutional issues related to development programming and humanitarian action (including funding, administration, the role of the UN system, partnerships, human rights, sustainable development issues)

Experience in working with government counterparts is desirable The team must include experts on gender issues and have expertise in gender analysis of

programmes to ensure the evaluation is responsive to gender aspects of entire interventions.

Excellent analytical, report writing and communication skills in English and Arabic for some key members of the evaluation team;

Familiarity with child protection and inter-sectoral issues of violence in school issues either as researcher/evaluator or programme manager

Profiles and/or CVs of the evaluation team as well as references/links to two most recent evaluations should be provided with the proposal.

8. OFFICIAL TRAVEL INVOLVED

It is expected that the Evaluation team would make a visit to the country (if located outside) including areas for field work as per methodology finalised for this evaluation. All travel costs (international and local) should be planned properly in the technical proposal and included in the financial proposal. Please note that if selected, the contract can be a supporting document to obtain entry visa (if necessary). UNICEF will be unable to secure travel visas.

9. ESTIMATED DURATION OF THE CONTRACT AND PAYMENT SCHEDULES

It is expected that evaluation will be conducted from August to December 2016. The contract would cover the entire duration of the evaluation. Proposed and estimated timeframe for deliverables is listed under each deliverable in the Section 6. The evaluation team should propose a timeline to

of 18

submit the deliverables considering necessary and adequate time (at least two weeks) to be allocated for review and quality assurance processes of the deliverables by the evaluation reference group and regional Monitoring and Evaluation advisor.

Payment is contingent on approval by the evaluation manager and will be made in three instalments.

20 % of the total contract will be paid upon clearance of the inception report by the ERG; 30 % of the total contract will be paid upon submission of the draft report of the evaluation; Remaining 50 per cent will be paid upon clearance of final deliverables by Steering

Committee as spelled out in the TOR

All interested institutions or group of consultants are requested to include in their submission detailed costs including:

a) Daily rate including hours per dayb) Expenses (please include all costs that are to be charged to UNICEF) to be agreed prior to

commencing projectc) Any additional requirements needed to complete project or that might have an impact on

cost or delivery of productsd) The consultants would be required to use their own computers, printers, photocopier etc.

10. PERFORMANCE INDICATORS FOR THIS ASSIGNMENTProposed timelines for completion of activities are met and deliverables submitted on time with good quality and as per the standards described in the TORs as well as UNICEF/UNEG global standards. The evaluation team should conduct evaluation and develop deliverables in line with the UNEG Evaluation Standards and Norms, UNICEF Procedure on Ethics in Evidence Generation, UNEG Standards for Inception Report, and UNICEF UNICEF-Adapted UNEG Evaluation Reports Standards. Overall performance at the end of the contract will be evaluated against the following criteria: timeliness, responsibility, initiative, communication, and quality of the products delivered.

11. ROLES AND RESPONSIBILITIES IN THE EVALUATION PROCESSUNICEF as commissioner takes the accountability of the final evaluation and designate Chief of Child Protection Programme as supervisor for this evaluation. Managerial function for this evaluation will be implemented jointly by the Child Protection Chief and Planning, Monitoring & Evaluation Chief at Jordan Country Office.

a) Evaluation managers will have the following responsibilities:• Lead the management of the evaluation process throughout the evaluation (design,

implementation and dissemination and coordination of its follow up)• Convene the evaluation reference group meetings• Facilitate the participation of those involved in the evaluation design• Coordinate the selection and recruitment of the evaluation team by making sure the

lead agency undertakes the necessary procurement processes and contractual arrangements required to hire the evaluation team

• Safeguard the independence of the exercise and ensure the evaluation products meet quality standards

• Connect the evaluation team with the wider programme unit, senior management and key evaluation stakeholders, and ensure a fully inclusive and transparent approach to the evaluation

of 18

• Facilitating the evaluation team’s access to all information and documentation relevant to the intervention, as well as to key actors and informants who should participate in interviews, focus groups or other information-gathering methods

• Provide the evaluators with overall guidance as well as with administrative support• Oversee progress and conduct of the evaluation, the quality of the process and the

products• Approve the deliverables and evaluate the consultant’s/team’s work in consultation

with Evaluation reference group and will process the payments after submission of the deliverables that respond to the quality standards.

• Take responsibility for disseminating and learning across evaluations on the various programme areas as well as the liaison with the National Steering Committee

• Disseminate the results of the evaluation

b) Evaluation Reference Group (ERG) will comprise the representatives of the major stakeholders including Evaluation managers, programme specialist from UNICEF or other UN or NGO], and a representative from the Ministry of Education, UNRWA, Military Education Authority and NCFA. Regional Monitoring and Evaluation advisor will also provide a quality assurance of ToR, inception report, draft and final evaluation report and participate at ERG meetings as necessary. The ERG will:

• Provide clear specific advice and support to the evaluation manager and the evaluation team throughout the whole evaluation process

• Review the ToR, inception report and draft evaluation report and ensure final draft meets the UNICEF-Adapted UNEG Evaluation Reports Standards.

• Review and provide comments and feedback on the quality of the evaluation process as well as on the evaluation products (comments and suggestions on the TOR, draft reports, final report of the evaluation).

• Ensure the evaluation adheres the ethical principles and addresses potential ethical concerns.

c) The evaluation team will report to Evaluation Manager and conduct the evaluation by:• Fulfilling the contractual arrangements in line with the TOR, UNEG/OECD norms and

standards and ethical guidelines; this includes developing of an evaluation plan as part of the inception report, drafting and finalising the final report and other deliverables, and briefing the commissioner on the progress and key findings and recommendations, as needed.

12. CALL FOR PROPOSALS

A two stage procedure shall be utilized in assessing the proposals, with assessment of the technical proposal being completed prior to any price proposal being compared. Applications shall therefore contain the following required documentation:

1. Technical Proposal :

Applicants shall prepare a proposal as an overall response to ToR ensuring that the purpose, objectives, scope, criteria and deliverables of the evaluation are addressed. The proposal shall include detailed breakdown of inception phase and data collection methodology, coverage and the approach and proposed sampling to be used in the evaluation. It should

of 18

also include a brief explanation of data collection, analysis and report writing phases. Draft work plan and timeline for the study should be included. The Technical Proposal shall also include updated profiles/CV and Personal History Forms (P11) of the expert(s) to be part of the evaluation, and electronic copies/links of two most recent and relevant evaluations led by the proposed evaluation team leader.

2. Financial Proposal :

Offer with cost breakdown: Consultancy fees, international (economy class) and internal travel costs, Daily Subsistence Allowance (DSA), required translations and other costs. The Financial Proposal shall be submitted in a separate file, clearly named Financial Proposal. No financial information should be contained in the Technical Proposal.

Travel expenses shall be based on the most direct route and economy fare. Quotations for business class fare will not be considered.

13. EVALUATION AND WEIGHTING CRITERIA OF THE PROPOSALSSubmitted proposals will be assessed using Cumulative Analysis Method. All request for proposal will be weighed according to the technical (60%) and financial considerations (40%). Financial proposals will be opened only for those application that attained 70% or above on the technical part. Below are the criteria and points for technical and financial proposals.

a) Technical proposals1) Overall Response (10 points)

- General adherence to Terms of Reference and tender requirements- Elaborated and articulated understanding of scope, objectives and overall

assignment and requirement for this evaluation, demonstrated knowledge and experience in evaluation of similar scope and scale, and completeness and coherence of response

2) Proposed methodology and approach (35 points)

- Deliverables are addressed as per ToR- Articulated understanding of the results chain or theory of change for the

programme that is being evaluated.- Evaluation questions are unpacked into sub-questions- Proposed approach/methodology and management control system respond to the

scope of the evaluation and the nature of the evaluation questions- Methodology is attentive to the availability of and limitations around the available

data and sensitive to the different roles, mandates and background of stakeholders- Proposed Implementation Plan, i.e. how the bidder will undertake each task, and

maintenance of project schedules demonstrates timelines will be met- Process and procedures for ethical review are elaborated and integrated into the

evaluation process- Alternative proposals are made on methodology, implementation plan and

timeframe as per the sounds understanding of the scope of the assignment and with reasonable justifications

of 18

3) Technical capacity (30 points)

- Key team members especially the team leader meets the defined qualifications such as minimum years of proven and relevant experience The evaluation team has previously conducted similar work with evidence The agency proposing a team with relevant and justified capacity and skillsets Range and depth of experience with similar evaluation

- Strong analytical skills and qualitative and/or quantitative statistical data processing applications

- Excellent report writing and language skills- Company is properly registered a/o has required certifications, memberships, etc.

Technical score: XX % of XX points = XX points

b) Financial ProposalOnly those financial proposals will be opened which have been technically accepted according to the above criteria. Financial proposal will be weighted based on the clarity and appropriateness.

Total Financial XX

The Contract shall be awarded to a bidder obtaining the highest combined technical and financial scores. Proposals not complying with the terms and conditions contained in this ToR, including the provision of all required information, may result in the Proposal being deemed non-responsive and therefore not considered further.

14. EQUITY, GENDER, HUMAN RIGHTS, INCLUDING CHILD RIGHTSThe TOR indicates both duty bearers and rights holders (particularly women and other groups subject to discrimination) as primary users of the evaluation and how they will be involved in the evaluation process. It spells out the relevant instruments or policies on human rights, including equity issues, child rights and gender equality that will guide the evaluation process.The TOR includes an assessment of relevant human rights, including child rights and gender equality, aspects through the selection of the evaluation criteria and questions.The TOR specifies an evaluation approach and data collection and analysis methods that are human rights based, including child rights based and gender sensitive, and for evaluation data to be disaggregated by sex, ethnicity, age, disability, etc.The ToR should clearly lay down the equity dimension of evaluation through inclusion of issues like describing the nature and causes of inequity, assessing the impacts of development policies and programs on vulnerable [excluded] groups Identifying policy priorities for enhancing equity.The TOR defines the level of expertise needed among the evaluation team on gender equality and human rights, including child rights, equity and their responsibilities in this regard and calls for a gender balanced and culturally diverse team that makes use of national/regional evaluation expertise.

15. ETHICAL PRINCIPLES AND PREMISES OF THE EVALUATIONThe evaluation process will adhere to the United Nations evaluation norms and standards available at: http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4 and ethical guidelines for evaluation http://www.uneval.org/papersandpubs/documentdetail.jsp?doc_id=102

of 18

http://www.uneval.org/papersandpubs/documentdetail.jsp?doc_id=102

http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4

The assignment to be carried out according to the ethical principles, standards and norms established by the United Nations Evaluation Group (UNEG).a. Anonymity and confidentiality. The evaluation must respect the rights of individuals who

provide information, ensuring their anonymity and confidentiality.b. Responsibility. The report must mention any dispute or difference of opinion that may have

arisen among the consultants or between the consultant and the commissioner of the evaluation in connection with the findings and/or recommendations. The team must corroborate all assertions, or disagreement with them noted.

c. Integrity. The evaluator will be responsible for highlighting issues not specifically mentioned in the TOR, if this is needed to obtain a more complete analysis of the intervention.

d. Independence. Evaluation in the United Nations systems should be demonstrably free of bias. To this end, evaluators are recruited for their ability to exercise independent judgement. Evaluators shall ensure that they are not unduly influenced by the views or statements of any party. Where the evaluator or the evaluation manager comes under pressure to adopt a particular position or to introduce bias into the evaluation findings, it is the responsibility of the evaluator to ensure that independence of judgement is maintained. Where such pressures may endanger the completion or integrity of the evaluation, the issue will be referred to the evaluation manager and, who will discuss the concerns of the relevant parties and decide on an approach which will ensure that evaluation findings and recommendations are consistent, verified and independently presented (see below Conflict of Interest)3.

e. Incidents. If problems arise during the fieldwork, or at any other stage of the evaluation, they must be reported immediately to the evaluation manager. If this is not done, the existence of such problems may in no case be used to justify the failure to obtain the results stipulated in these terms of reference.

f. Validation of information. The consultant will be responsible for ensuring the accuracy of the information collected while preparing the reports and will be ultimately responsible for the information presented in the evaluation report.

g. Intellectual property. In handling information sources, the consultant shall respect the intellectual property rights of the institutions and communities that are under review. All materials generated in the conduct of the evaluation are the property of UNICEF and can only be used by written permission. Responsibility for distribution and publication of evaluation results rests with the Country Office. With the permission of the agency, evaluation consultants may make briefings or unofficial summaries of the results of the evaluation outside the agency.

h. Delivery of reports. If delivery of the reports is delayed, or in the event that the quality of the reports delivered is clearly lower than what was agreed, the penalties stipulated in these terms of reference will be applicable.

In line with the Standards for UN Evaluation in the UN System, all those engaged in designing, conducting and managing evaluation activities will aspire to conduct high quality and ethical work guided by professional standards and ethical and moral principles.

16. UNICEF RECOURSE IN CASE OF UNSATISFACTORY PERFORMANCEIn case of unsatisfactory performance the payment will be withhold until quality deliverables are submitted. If the firm is unable to complete the assignment, the contract will be terminated by notification letter sent 30 days prior to the termination date. In the meantime, UNICEF will initiate another selection in order to identify appropriate candidate.

3 UNEG Ethical Guidelines for Evaluation

of 18

17. INDICATION OF HEALTH STATEMENT AND CERTIFICATE OF GOOD HEALTH HAS BEEN RECEIVED PRIOR TO SIGNING THE CONTRACT

For consultants/individual contractors traveling with UNICEF or working in UNICEF Office

18. INDICATION THAT THE CONSULTANT/INDIVIDUAL CONTRACTOR HAS RECEIVED A COPY OF THIS DOCUMENT OR. ALTERNATIVELY, AN EXPERT OF RELEVANT PROVISIONS INCLUDING THOSE CONCERNING LEGAL STATUS, OBLIGATIONS AND TITLE RIGHTS.

For consultants/individual contractors traveling with UNICEF or working in UNICEF Office

19. CONDITIONS- The contractor will work on its own computer(s) and use its own office resources and materials in

the execution of this assignment. The contractor’s fee shall be inclusive of all office administrative costs

- International and Local travel and airport transfers (where applicable) will be under responsibility of the contractor in accordance with UNICEF’s rules and tariffs.

- Flight costs will be covered at economy class rate as per UNICEF policies.

of 18

· web viewthe participatory approach should be central in the evaluation approach, ensuring the...

Documents