Project External Evaluation Consultancy

Terms of Reference

Final Evaluation

Saving lives now and in future through education, inclusion and through enhancing local capacity in targeted areas of South Sudan

Geographical coverage: global; region; country(ies)-please specify

Jonglei state: Pibor

Central Equatoria state: Juba Country

Number of people targeted:

12,800 ALP learners

8,400 Functional Adult Literacy young and adult women

1,150 skills development for youth

Program/project lifespan

January 2018 to December 2021

Program/project budget (TOC3 only):

3,143,298 DKK

Donour

Danida

Evaluation commissioning manager

Education Program Manager

Evaluation manager

Programme quality Coordinator

1. *Background

The political instability from December 2013 engulfing South Sudan have led to an unmitigated humanitarian crisis afflicting 7.3 million people, and severe food insecurity impacting an estimated 4.9 million people across the country. The protracted nature of the conflict has considerably curtailed the delivery of social services, agricultural production and destroyed key basic infrastructure; schools, roads, health facilities, leaving behind a trail of disrupted livelihoods, whilst eroding the coping abilities of local communities. Today at least 1 in 2 women live below the $1.25/$2 per day poverty line and only 27% aged 15 years are literate (40% male and 15% females). This is intensified by the volatile situation caused by the manmade crisis, where the abuse of rights such as gender-based violence (GBV) and child recruitment is widespread.

The education sector suffers from the prolonged crisis with an estimated 2.4 million children not receiving an education - the highest proportion of out of school children in the world[1] and with approximately 75% girls not enrolled in primary school.[2] Needs in the education sector were already high before the crisis intensified in 2016–in 2015 when only 41% of teachers were qualified and officially recognised, and 1/3 of primary classrooms were open air or not permanent structures.^[3] 90% of schools in South Sudan have active Parent Teacher Associations (PTAs) and/or School Management Committees (SMCs). Shortages of teachers are reported as a result of insecurity, and student attendance declining as a result of ongoing violence and consequent displacement, as well as lack of adequate infrastructure. The education context, analysed in a study by Plan, is marked by long distances to school, lack of funding for school fees, and lack of learning materials being top reasons mentioned for boys; and negative gender stereotypes (such as those minimising the value of education for girls and preferring boys), early marriage, household chores, inadequate Menstrual Hygiene Management (MHM), as well as long distance, being top reasons mentioned for girls.[4] The gender disparity among teachers as well as the low number of teachers (156 in total) is another particularly worrying statistics with 13% (20) of the teachers being female and 87% (136) male.[5] Adolescent girls face additional barriers to education such as: the fear of parents that schooling may expose their daughters to premarital sex or making them “strong headed” or less traditional, which would decrease their chances of getting married or fetching a high dowry upon marriage.[6] It is even harder for adolescent girls already married and with children to access education - due to lack of child care and unavailability of adult classes, extra demand for domestic chores or not being allowed by their husbands.[7]

Most education actors are involved in implementing formal education which include provision of support to Early childhood education, Primary and secondary school education. Others are involved in supporting construction of schools and long-term teachers training.

Oxfam in South Sudan has implemented a four-year project, titled, “Saving lives now and in future through education, inclusion and through enhancing local capacity in targeted areas of South Sudan”. This project is funded as part of the Strategic Partnership agreement between Oxfam IBIS and Danida, Danida Humanitarian Funding for South Sudan (TOC2) (Juba and Pibor). The project is being implemented with the following partners at different stages of implementation: SALT, HACT[8], YWCA, CDI, and DARD.

Specific objective: Improved access to and utilization of quality education by conflict-affected children, youth and women in the targeted areas

Outcome 1: Children, youth and women with improved knowledge, skills and opportunities.

Outcome 2: Enhanced capacity of local civil society organizations and community alliances to ensure strengthened local leadership towards the delivery of education.

Outcome 3: Promoting the implementation of gender and conflict sensitive education to local authorities and other actors

Main project recipients/beneficiaries include: Marginalised children, youth and adults without access to quality education, including girls and women and internally displaced persons (IDPs). Other target groups include: Government officials, Community based organization (CBOs), Civil Society Organisations (CSOs), Learners, Teachers, Parent Teacher Associations (PTAs), and Community members.

For further information, the results framework for the project is attached (Annex I)

2. Evaluation purpose and scope **

The purpose or intended use of the evaluation is to help Oxfam and its partners to conduct an end of project evaluation to assess progress of the project against its goal and objectives and to learn from what works well and less well. The evaluation will provide input to upcoming discussions concerning the preparation of a new strategic phase of Danida programming, as well as benefit other similar programmes.

The final evaluation will be looking mainly at the Danida Humanitarian project components implemented in both Juba and Pibor which includes: Accelerated Learning Program (ALP), Functional Adult Literacy (FAL), skills development, working with PTAs and CSOs; Teacher Education and Professional Development (TEPD), research and studies as well as advocacy on education issues in South Sudan. An additional small component will be added looking at the Danida CIV project in Rumbek including functional literacy and support/training of PTAs as well as civil society strengthening, including partnership with National Education Coalition (NEC).

If needed, the scope of the evaluation may be further elaborated by the evaluator in the inception report.

1. Evaluation objectives and questions

The evaluation shall conform to OECD/DAC’s Quality Standards for Development Evaluation.

The objective of this evaluation is to evaluate the relevance, effectiveness, efficiency, impact and sustainability of the project and formulate input to upcoming discussions around similar interventions.

(i) Relevance

· To what extent were the project interventions responding to recipients/target beneficiaries and national needs and priorities, and have they continued to do so if/when circumstances have changed?

· How were the different needs of conflict-affected women and men and young people integrated into the design and implementation of the project and to what extent has gender been mainstreamed? What is the experience and views of youth of working with Oxfam (opportunities/challenges)?

· How compatible has the project been with other interventions in South Sudan, sector or within Oxfam where it is being implemented?

· Was the project design logically sound? Were the activities and outputs of the interventions consistent with the overall goal and the attainment of its objectives? Were they also consistent with the intended outcomes and impacts?

(ii) Effectiveness

· To what extent has the project achieved its objectives, and its results, including any differential results across different groups (IDPs/Host communities)? What factors have contributed to achieving or not achieving intended project outcomes and set objectives?

· To what extent have the interventions been gender and conflict sensitive?

· Have the M&E system delivered robust and useful information that could be used to assess progress towards outcomes and contribute to learning?

· Has the accountability system ensured participation, regular feedback/complaint from the community and provided a timely response?

· Assess how the partner portfolio and the mix of different types of partners have contributed to the objectives of the strategy.

(iii) Efficiency

· To what extent has the project delivered results in an economic and timely way?

· Was the project designed and/or amended throughout the implementation period to provide the best value for money?

· Was this project the most cost effective as compared to similar projects?

· Were the resources for running all the activities available, adequate and was this the best use of resources to achieve results?

· To what extent has effective coordination and collaboration with existing interventions and partners been addressed and achieved?

(iv) Impact

· To what extent has the project or programme generated, or is expected to generate, significant positive or negative, intended, or unintended changes in the lives of recipients and in their environment?

· How well did the project succeed in involving women and men, especially the most vulnerable?

· Assess to which extent the implementation (so far) of the program/project has achieved outcomes related to women’s empowerment? And the reasons behind the achievement (or not).

· To what extent did different groups, including conflict affected young people/children and other vulnerable groups, benefit in different ways from the intervention?

· What are the challenges and opportunities to strengthen or “scale up” “nexus” approaches at country level? What does this require in terms of financing, project design, technical support? (What are the implications for this in terms of how Oxfam IBIS is structured to be able to deliver on “nexus” approaches?)

(v) Sustainability

· To what extent can the activities and the benefits of the project continue after external funding has ceased? To what extent do project recipients and/or partner country stakeholders have ownership, capacity and resources to maintain the activity results after external funding ceases?

· Are any areas of the project clearly unsustainable? What lessons can be learned from such areas?

· What were the major factors that influenced the achievement or non-achievement of sustainability of the intervention?

(vi) *Evaluation approach and methods

A gender responsive approach/methodology, methods, tools and data analysis techniques should be used. It is therefore expected that the evaluators, in their application, present a methodology and methods for data collection that create space for reflection, discussion and learning between the intended users of the evaluation. The evaluation methodology will be presented for approval to the Evaluation Reference Group.

The methodology should use a combination of quantitative and qualitative research methods and a desk review of project overview should be done. It should be utilisation focused, gender responsive and explore the possibility of utilising participatory methods for developing case studies. Data should be disaggregated by sex and according to other relevant project-specific parameters.

Some of the data collection tools expected to be used during the evaluation are:

Desk review: The evaluator will consult all available documentation in preparation for the review, including Programme documents, baseline report, minutes of meetings; quarterly reports, annual reports and programme implementation and research reports, implementing partners reports and documentation, technical training reports etc.

Survey: Design and implement a survey which includes participation of project primary stakeholders with ALP learners, FAL learners, and skills development learners, and partners. In addition, the methodology should spell out how stakeholders will participate at each level during the end line survey.

Interviews with Key Informants: The team are expected to conduct a range of interviews with key informants and stakeholders and will visit and interview relevant Ministries and government agencies, local partners, community leaders, teachers of ALP/FAL/Skills development Tutors, recipients/ beneficiaries, etc.

Focus group discussions: The team will conduct focus group discussions with direct and indirect beneficiaries of the project (including but not limited to schools SMCs, PTAs as well).

Human Interest Stories: During the interview, the evaluators will support beneficiaries of the project to document their stories on how the programme has impacted on their lives.

The evaluators shall specify how quality assurance will be handled by them during the evaluation process.

(vii) *Evaluation deliverables and time schedule

It is expected that a time and work plan is presented in the application and further detailed in the inception report. The evaluation shall be carried out between 3^rd January and 18^th March 2022. The timing of any field visits, surveys and interviews will be agreed upon during the inception phase.

The inception report will form the basis for the continued evaluation process and shall be approved by Commissioning Manager before the evaluation proceeds to implementation. The inception report should be written in English and cover evaluability issues and interpretations of evaluation questions, present the evaluation approach/methodology, methods for data collection and analysis as well as the full evaluation design. The data collection tools will also be provided for approval. All limitations to the methodology and methods need to be made explicit and discussed. A specific work plan, including number of hours/working days for each team member, for the remainder of the evaluation should be presented.

The final report shall be provided in English and based on an agreed upon structure. The evaluation approach/methodology and methods for data collection used shall be clearly described and explained in detail, including all limitations to the methodology. Findings based on the data analysed and evidence presented will support the conclusions. Evaluation findings, conclusions and recommendations should reflect a gender analysis/an analysis of identified and relevant cross-cutting issues. Recommendations and lessons learned should flow logically from conclusions. Recommendations should be specific, directed to relevant stakeholders and categorised as a short-term, medium-term and long-term. The report should be no more than a maximum of 40 pages excluding annexes (including Terms of Reference and Inception Report).

The table below lists key deliverables for the evaluation process. Deadlines for deliverables may be suggested by the evaluator.

Deadlines

Deliverables

Working days

3^rd Feb 2022

Start-up meeting

½ working day

4^th – 10^th Feb 2022

Draft inception report and inception meeting

5 working days

11^th – 24^th Feb 2022

Data collection

10 working days

25th Feb 2022

Debriefing after data collection

1 working day

28^th Feb – 11^th Mar 2022

Data analysis, report writing and quality assurance; deliverable Draft evaluation report

10 working days

14^th March 2022

Validation workshop

½ working day

15-18^th March 2022

Final evaluation report based on comments received

4 working days

Payment schedules will be defined in the agreement

1. 20% upon submission of an inception report, within two weeks (10 working days) after signing of the contract. This report will outline in detail the key scope of the work and detailed study methodology; a work plan/schedule of tasks designating a team member with the lead responsibility for each task and deliverable (output); sources of data; and a data reporting plan.

2. 30% A draft report will be presented to Oxfam by the research team during a stakeholder workshop at a date to be agreed. The evaluation team will facilitate the workshop, which will be aimed at discussing preliminary findings and conclusions of the study.

3. 50% upon submission of a final evaluation report in the agreed upon structure.

Note: Please take into account that South Sudan deducts 10% Withholding Tax on Consultancy Fees.

(viii) *Evaluation management

This evaluation is commissioned by Oxfam in South Sudan. The commissioning manager is the Education Program Manager from the Oxfam South Sudan Country Team. The consultancy will also be coordinated with Yasmine Wahba/Irene Fredriksson (Oxfam IBIS) and Oxfam South Sudan MEAL Team who will constitute part of the Evaluation Reference Group overseeing this consultancy. The reference group will approve the inception report and the final evaluation report and interaction with the committee will be at specific points such as the start-up meeting of the evaluation, as well as debriefing/validation workshop where preliminary findings and conclusions are discussed.

(ix) Confidentiality and data protection **

The consultant firm shall not produce these materials in part or whole and any form (electronic, hard copies, etc) to a third party without written permission from Oxfam South Sudan.

(x) *Roles and Responsibilities

The consultant should:

· Lead on the evaluation process and submit all deliverables as per the agreed upon workplan

· Cover all out of country costs (flight, visa)

Oxfam will:

· Facilitate the work as per the specified activity and timeframe in the outline above

· Provide relevant documentation to the consultant

· Provide needed letters (invitation and facilitation letters)

· Provide costs of enumerators and translators as per agreed upon methodology at inception

· Effect payments according to the agreement. Apart from consultancy fee, Oxfam shall cover the in-country travel costs related to the field work (to cover accommodation, meals and local transportation during the duration of field travel) as per Oxfam procedures.

· Provide timely feedback on the evaluation report from all relevant staff members.

(xi) *Evaluation Team Qualifications

The consulting firm/team leader should have experience and conversant in conducting evaluations and leading evaluation teams, including communication and facilitation skills. The evaluation team must be independent from the evaluation object and evaluated activities and have no stake in the outcome of the evaluation. The evaluation team shall include the following competencies:

· Higher university degree in Humanitarian/Development studies, education, economics, statistics, social sciences, gender or any other related fields.

· Proven experience in conducting quality evaluations and leading evaluation teams

· Extensive experience working with international organizations globally and specifically in South Sudan.

· Knowledge/experience of education in emergencies projects/programmes

· Strong analytical and research skills

· Fluency in written and spoken English

· Good facilitation skills

APPLICATION REQUIREMENTS **

Interested candidates in the position are expected to provide the following documentation:

· A technical proposal with detailed response to the TOR, with specific focus on addressing the scope of work and methodology to be used, as well as including a workplan

· A financial proposal: Submit a financial proposal that indicates all-inclusive costs for conducting the final evaluation detailing the daily rate expected[9], transportation costs, accommodation costs, etc. The consulting firm/consultant shall bear all costs associated with the preparation and submission of the proposal. The consultant shall submit ONE (01) copy of technical and ONE (01) copy of financial proposal in electronic form or in a single envelope. The technical and financial proposals should be clearly marked and should include the name and detailed contact of the consultant/ firm. All costs should be quoted in USD and will remain valid up to sixty days (60) from the day of proposal submission

· Sample of previous evaluation conducted

· Consultancy firm profile or CV including a minimum of 3 traceable, recent and relevant references to this task

· A CV for each team member shall be included in the call-off response. It should contain a full description of relevant qualifications and professional work experience.

[1] UNICEF report found at https://www.unicef.org/southsudan/media_21715.html

[2] Global Initiative on Out of-School Children South Sudan Country Study UNICEF, May 2018, Global partnership for Education, UNESCO, Ministry of General Education and Instruction

[3] South Sudan Crisis Sensitive Education Sector Analysis, 2015

[4] Needs Assessments Report, Education & Child Protection, Administrative areas of Pageri and Pibor. Plan International, November 2017

[5] ibid

[6] “The Journey of Adolescent Girls” during and after Armed Conflict in South Sudan, Plan International

[7] Needs Assessments Report, Education & Child Protection, Administrative areas of Pageri and Pibor. Plan International, November 2017

[8] HACT was a partner with Oxfam South Sudan 2020, however is no longer a partner

[9] Please take into account that South Sudan deducts 10% Withholding Tax

How to apply:

Deadline for submission of applications is 26^th January 2022. Interested Applicants should send soft copies of the requested application documents addressed to JubaQuotations@oxfam.org.uk.