Kenya: MID TERM INDEPENDENT EXTERNAL EVALUATION

The MENTOR Initiative, Building Sustainable and Effective Malaria Control for

Vulnerable Communities in North Eastern Province, Kenya

PERIOD: October – November 2012 (maximum 20 days)

I. Background

The MENTOR Initiative (MENTOR) is a not-for-profit organization registered in UK and Kenya which is focused on reducing death and suffering due to malaria and vector borne diseases in emergencies.

Up until 2010 the organization’s work in Kenya focused on implementing short-term emergency malaria control programmes, during and after severe flooding in the Western and North Eastern Provinces in collaboration with Division of Malaria Control (DOMC), other Ministry of Public Health and Sanitation and Ministry of Medical Services. However, after a comprehensive programme development phase, a four year Comic Relief Special Initiative (SI) commenced in January 2010 with the aim of building local capacity to provide sustainable malaria surveillance and control and epidemic response solutions in an area of semi-arid North East Kenya.

This Special Initiative is now due for a periodic review.

II. MENTOR/ Comic Relief, Kenya Malaria Programme

In January 2010, The MENTOR Initiative launched a 4 year special initiative, with Comic Relief, to achieve systemic change in malaria across the three districts of Garissa, Fafi and Lagdera.

Objectives of Programme

To have >80% of all sick people seeking treatment have access to differential diagnosis and confirmatory malaria test and treatment by 2013.To build capacity of the DHMT and communities in the larger Garissa district to detect malaria epidemics, preparedness and respond to epidemics within 7 days of onset by 2013.To ensure that >85% of communities have access to malaria education and improved malaria control services.

Specific activities in this 4 year long Term Special Initiative include: • Improve local capacity at district level to plan and co-ordinate malaria control responses in semi arid areas. • Improve malaria case management through technical capacity building of health workers to accurately diagnose and treat malaria. • Increase speed and accuracy of malaria incidence and health facility stock reporting by establishment of a robust and "real time" ARGOS early warning system (EWS), and local climate change monitoring, comparing advance technology with paper based systems. • Improve commodity supply chain to health facilities, and potentially in the future also to community health workers, through access to regular stock updates. • Weekly monitoring of EWS data and climate predictions at central level to identify early indicators of an epidemic and mount timely responses. • Increase community ownership to deliver malaria prevention activities through supply of LLINs specifically adapted for nomadic communities, technical and material support to implement indoor residual spray (IRS) campaigns to communities exposed to the risk of epidemics and, generally speaking, deliver key messages about why and how to access effective services. • Increase community understanding of malaria, demand for and usage of effective malaria control services and participation in malaria control programming. • Increase understanding of malaria epidemiology in NEP by all partners, including collaboration with KEMRI to implement standardised annual base line malaria surveys for plasmodium faciparum malaria, and the implementation of studies to actively identify plasmodium vivax cases and measure their relative burden. • Improve understanding of the benefit of the standardised use of malaria RDTs and how this may contribute to health system strengthening, especially in relation to the rationalisation of ACT usage and the detection and case management of other febrile illnesses.

III. Evaluation Goal Comic Relief has been one of the funding partners of The MENTOR Initiative since 2010, this mid-term evaluation forms part of the requirements of the programme grant. The goal of this evaluation is to assess and critically analyse the progress of the programme, the methods used, planned strategies and timeframe to identify assumptions and lessons learnt in order to provide inputs for further implantation of the project..

IV. Evaluation Questions

The process of the evaluation should answer the following questions by using participatory instruments;

Programme Effectiveness.
• Outputs and objectives; What progress has been made towards achieving the planned outputs and objectives? Specifically towards the Monitoring and Evaluation plan and learning questions. • Programme Coverage; Has the programme reached all the participants and beneficiaries in the programme’s selection criteria? • Programme Participation; Has the programme engaged sufficiently and appropriately with project stakeholders, in particular project participants and beneficiaries? • Programme Capacity; Is there sufficient capacity to implement and manage the project effectively?

So far, what difference has the programmemade to people’s lives (what, how, who, where, when)? • To what degree have the project outcomes been achieved? Have there been any unexpected outcomes? • Who has benefited (women, men, girls and boys) and in what ways? • Are those changes (outcomes) relevant to people’s health needs? • Are they likely to be sustainable in the long term? • Have there been changes to policies, practice and attitudes of decision and policy makers to benefit the project’s target groups? • To what extent has the project contributed to the achievement of broader Malaria national and international policies, conventions, targets etc in Kenya? • To what extent has the achievement of the changes/ outcomes been influenced by external context and other factors?

To date, how has the project made this difference? Approaches used by the project and implementing organisations: • Has the overall theory of change for this project been effective in starting to bring about lasting change? Were there any gaps? • What have been the most effective methodologies and approaches the organisation used to bring about changes to people’s lives? What has worked and what has not? What lessons have been learned? Who have they been shared with? • How has the type of organisations funded (e.g. user-led, social enterprise, national or international NGO), both UK and local, helped or hindered the delivery of lasting change? • How have relationships between partners throughout the relationship chain (looking at UK organisation-local partner(s)-target groups) helped or hindered the delivery of change /outcomes? • How effective have the project’s management, monitoring, learning and financial systems been? How have they helped or hindered the delivery of lasting change? • Has the project been cost effective?

Approaches used by Comic Relief: • How have Comic Relief’s grant making policies and processes (e.g. how we define our programme strategies and outcomes, how we assess applications) helped or hindered the delivery of lasting change? • How has Comic Relief’s approach to grant management (e.g. individual work with grant holders, and learning activities with other funded organisations) helped or hindered the delivery of lasting change? • How has the way Comic Relief used its organisational assets helped or hindered the delivery of change (e.g. use of the media, access to decision makers)? • Are there any other ways in which Comic Relief has helped or hindered the delivery of change?

V. Project Review and Methodology

It is expected that the evaluator would review and make an analysis of the relevant documents such as project proposals and project reports.It is expected the evaluator will apply a participatory methodology throughout the assessment, engaging all stakeholders including Comic Relief grant staff and MENTOR HQ staff.

VI. Project Area

The Field implementation area is Greater Garissa and includes the districts of Garissa, Balambala, Fafi and Lagdera. The Programme workbase is in Garissa. All pre and post field trip write up must be conducted from the evaluators own premises.

VII. Documentation The available documents related to The MENTOR Initiative project are as follows: 1. Project Proposal. 2. Project Reports. 3. Study protocols. 3. Published paper.

VIII. Qualifications of Evaluator 1. The evaluator should have 3-7 years of experience of working with health projects preferably with a malaria component in emergency or insecure environments. 2. The evaluator should have prior experience of working with Somali communities, and nomadic communities. 3. The evaluator should provide a 3-5 page proposed evaluation plan to undertake this assignment. 4. The evaluator should submit 1 sample of a previous evaluation report.

The evaluator should be aware that North Eastern Province is classed as an insecure environment and all travel within the implementation area will be dependent on the current security situation and assessment by MENTOR’s Kenya Programme Director. As a contractee of The MENTOR Initiative the evaluator will be expected to closely adhere to and respect The MENTOR Initiative’s Code of Conduct including sexual harassment guidelines and security regulations.

IX. Timeline and Deliverables

The final evaluation will be undertaken and completed between October to November of 2012. (For a maximum of 20 days) 1. 2 days for preparation and review of documentations. 2. 1-2 days for inception report. 3. 8-10 days for field implementation. 4. 3 days for data analysis and report writing. 5. 1 day for draft report discussion. 6. 2 days for finalizing evaluation report.

X. Reporting

The evaluator should provide final evaluation report in English that is jargon free and no more than 30 pages in length, consisting of the following items.

Executive summary.Introduction and background to the evaluation that is relevant to the report’s analysis and conclusionsMethodology usedContext analysis that include findings, conclusions and assessment. Lesson Learned. The evaluator is expected to consider on how good practice has been incorporated in the project. Specific Actionable and Prioritized recommendations. This includes short term recommendations and long term recommendations and details of how actions can be readily implemented.Guidance and road map of how the document will be disseminated and findings shared.Annexes. • Terms of reference for the Evaluations. • Profile of the Evaluation Team. • Evaluation Schedule. • Documents and technical details consulted during the evaluation. • List of informants. • Field data used during the evaluation, including baselines and focus group stories and data. • Bibliography. • Background information if required.

Draft report should be discussed and presented to MENTOR Programme Managers and MENTOR Kenya prior to finalization.

XI. Funding • This mid-term evaluation is being funded by The MENTOR Initiative.

View the original article here