Save the Children Turkana OFDA Nutrition Project End term Evaluation Terms on Reference

Evaluation of the ‘Emergency Nutrition Assistance for Drought Affected Vulnerable Communities of Turkana County of Kenya’
Project funded by USAID’s Office of U.S. Foreign Disaster Assistance (USAID/OFDA)

Terms of Reference (ToR)

1. Introduction and background information:

Turkana County covers about 77,000 square kilometres of the country’s arid north western region. The county is classified as arid and semi-arid lands of Kenya (ASAL) and is historically drought prone area that experiences frequent, successive and prolonged dry periods. It has two rainy seasons, the long rains (April–June) and the short rains (October–December) but rainfall is often unreliable. The 2011 drought that affected the Horn of Africa region exposed many pastoralist communities, including those living in Turkana County to severe food insecurity which eventually led to high rates of acute malnutrition amongst children under five years and pregnant and lactating women (Turkana nutrition survey – May 2011).

The subsequent scaling up of nutrition interventions has resulted in improvement of nutritional status of children and pregnant and lactating women.  
As part of the general scaling up of nutrition activities, Merlin since April 2013 has been implementing an OFDA funded emergency nutrition support project in Turkana County. The project ended on 31st January 2014. Since February 1st 2014 Merlin has merged with Save the Children to create a world-class humanitarian health force for children and their families living in some of the most challenging places in the world.
Project Objectives and Expected Results
The overall goal of the project was to contribute to the reduction of levels of morbidity and mortality amongst drought affected vulnerable communities, particularly under-five children and pregnant and lactating women, in Turkana Central, Loima, Turkana South, and Turkana North districts of Northern Kenya.
Main highlights of the project strategy for implementation of this action included:Integration of nutrition and health issues like the prevention and management of diseases like diarrhea, that negatively impact on nutritional status and vice versaStrengthening of hygiene practices to reduce the incidence of diarrheal disease associated with contaminated water in the household including health education to educate the community on domestic treatment of drinking water and effective hand washing (soap/ash) and use of latrines/toilets. In particular, a focus on longer-term strategies with a behaviour change communication (BCC) component that would help to translate knowledge to practice.Implementation of community-led infant and young child feeding practices program with the involvement of mother-to-mother peer support groups.Distribution of micronutrients and de-worming.Community and household level screening together with approaches that will avoid at risk children deteriorating, especially in the lean seasonal period. One approach will be the linking of households with children recently discharged from SFP programme to GFD (FFA), to reduce the relapse cases, due to household food insecurity.Management of moderate and severe acute malnutritionSupporting district health authorities to developing joint emergency preparedness and response plans The project was also based upon the partnership framework between the MoH, UNICEF and other partners to support the delivery of essential nutrition services in Kenya. This partnership framework envisaged that general principles of support to MoH should include:- Full integration of a comprehensive package of high impact nutrition interventions. Results based orientation to programming. Full commitment to coordination. Strategic development of capacity at all levels of service delivery to enable the MoH function well with minimal or no support from partners. Direct involvement in service delivery only in exceptional casesFocus on mentoring, supportive supervision and on-the-job training.2. Overall objective of the Evaluation:

As the ‘EMERGENCY NUTRITION ASSISTANCE FOR DROUGHT AFFECTED VULNERABLE COMMUNITIES OF TURKANA COUNTY’ grant has ended, Save the Children wishes to undertake an external evaluation of the project – focusing on achievements, successes, lessons learnt and best practices. 

These will be assessed against the OECD-DAC criteria that are most relevant to the purpose of this evaluation.

3. Specific Objectives:

i. Outputs and outcomes: Outputs and outcomes generated by the program in relation to the stated goal, objectives and desired results. In particular, the evaluation will assess the effects of the program activities on the targeted beneficiaries in OTP, SFP and community members reached with the various services.

ii. Relevance: whether the program interventions met needs of the beneficiaries; the appropriateness of results in relation to the needs of the communities, national policies and priorities.
iii. Effectiveness: In particular assess the extent to which program interventions achieved the desired outcomes, factoring in issues of program management including decision making processes, risk management, institutional arrangements and partnerships with MOH and other agencies and their effects on the program results.
iv. Efficiency: the relationship between the quantity, quality, and timeliness of program inputs, including personnel, training, office equipment. In addition, determine the quantity, quality, and timeliness of the outputs generated and whether the resources were spent as economically as possible.
v. Sustainability: assess the readiness of partner (MOH) and other stakeholders to sustain program interventions, in particular assess the infrastructure and systems of partners, resources available to sustain the activities and services, collaborative links and referral networks with other service providers, and the level of community ownership.
vi. Program Improvement areas: capture key successes, best practices lessons learnt, implementation challenges, constraints, strengths and weaknesses.
vii. Determine the extent of complementarity of this project with other related health and nutrition projects currently supported by UNICEF in Turkana County.
viii. Role of beneficiary consultation in programme design and implementation, and areas for improvement.

4. Methodology

The following methodologies shall be used for the evaluation:

Evaluation Design

Both quantitative and qualitative methods may be used for the study.The qualitative method will for example employ focus group discussions with target groups, employ interviewer administered structured questionnaires to stakeholders - government, beneficiaries, MoH, WFP and UNICEF, as well as in-depth interview of key informants and  document review, both primary and secondary. The quantitative method will review existing data related to the outcome indicators supplied by Merlin/MoH, and verify sample of same in the field. The consultant will use information both from primary and secondary sources which are directly related to the project. For triangulation purposes, the consultant is expected to refer to documents from Save the Children / MoH reports and national and sub-national statistics, research findings relevant to this project and donor guidance and associated documents. These sources will be used to validate the results and triangulate successes and failures for lessons documentation.

Key evaluation questions
1.1. How far has the implementation and delivery of the project activities, especially the choice of beneficiaries and organisational modalities, facilitated the achievement of specific objectives?

1.2. To what extent has the project addressed the emergency nutrition needs identified in the target communities as reflected in nutrition surveys conducted prior to commencement of project implementation.
1.3. What is the level of integration of health and nutrition activities in the MOH facilities in areas of Merlin operation and what lessons can be derived from current implementation approach?
1.4. What is the level of sustainability for the project results?  What linkages have been created and or exploited by the project to sustainably address malnutrition among the target communities?
1.5. What are the costs and effectiveness of implementing the health and nutrition activities in Turkana? Are the project implementation approaches cost effective in relation to practical realities on the ground?
1.6. What lessons have been learnt through the implementation of this project?

Study Area

The evaluation will be conducted in locations determined by the evaluator(s) in consultation with Save the Children, but has to be within Save the Children’s geographical area of operation.

Data collection and analysis

The evaluation should be participatory and focus on obtaining both qualitative and quantitative information.Quantitative data obtained from Save the Children on key indicators to triangulate with baseline data.Qualitative data obtained from FGDs, key informant interviews and structures questionnaires. Project document review, individual interview of project staff, government, UNICEF, WFP and other partners.5. Responsibility of the consultant / deliverables
The Consultant is expected to undertake the following tasks:Design and develop appropriate tools/mechanisms that will enable the analysis of the supplied information.Analyse collected information and write up a report detailing findings using Merlin’s evaluation policy and templates. Submit draft report for reviewPresent final report to Save the Children.Submit a final hardcopy report  as well as soft copies in word and PDF formatsPreparation of a PowerPoint of the evaluation methodology and key findings, for further dissemination, and a delivery of that presentation to key project staff. The evaluation process must be impartial, independent and must provide information that is credible and useful, enabling the incorporation of lessons learnt, and identify the strengths, weaknesses, opportunities and threats to inform future decision-making process within Save the Children, MoH and donors.

6. Key outputs of the evaluation are

A report completed on Save the Children reporting template (final hardcopy report  as well as soft copies in word and PDF formats);A PowerPoint of the evaluation methodology and key findings;Presentation of key findings to project staff and county representatives prior to drafting the evaluation report; An evaluation report completed with recommendations7. Required Qualification and Experience
The consultant should have the following qualifications and experience:Relevant academic and professional background in humanitarian interventions in complex environments. Extensive practical experience in conducting evaluations of humanitarian interventions in the ASAL regions of Kenya with preference given to significant emergency nutrition programming. Technical research skills, including qualitative and quantitative approaches.Fluency in both written and spoken English and Kiswahili. Knowledge and experience of working in Turkana is preferred.Submission of Expression of Interest
All interested consultants/firms are requested to write an expression of interest following the attached EOI format ONLY by email to: Kenya.jobapplications@savethechildren.org  by 30th April 2014 COB.  Please include the Tiltle of the TOR on the subject line.

The evaluation criteria are based on technical and financial responsiveness which also includes a valid licence and tax registration. 

The key technical evaluation criteria are:Known reliability in delivery of timely and quality servicesRelevant field/country experienceRelevant sector specific technical experience and qualificationsRelevant evaluation experience Presentation and writing skills evident from the proposalDemonstrated understanding of the work proposed in the TORClarification of methodologyCost/budget9. Ownership of the deliverables

Save the Children shall have full ownership of the report, research, working papers and other results of the Consultant’s work under this assignment. 

The Consultant shall not publish nor otherwise use any portion of the report without the written authorization of Merlin. The consultant shall waive any copyright to the work produced as a result of the contract.

10. Timeline

In consultation with project staff, the consultant shall design the review work plan and develop the evaluation instruments including appropriate questionnaires. 

The review is expected to take two weeks and additional six paid days have been catered for report writing. The study is expected to apply participatory approaches along with quantitative and qualitative methodologies.
Format for the evaluation report:

The final report shall at least consist of the following sections:

a) Executive summary
b) Project description
c) Context of the evaluation
d) Evaluation methodology
e) Findings
f) Lessons learnt, best practices, and recommended actions for the review of primary users
g) Annex (es) as necessary

10. Logistics/Transport/Accommodation

Save the Children will facilitate meetings with stakeholders and support the consultant in conducting interviews and other activities. Save the Children will also take care of the consultant’s accommodation while in the field.