South Sudan: Consultant (team) for end evaluation of humanitarian joint response in South Sudan

1. Project background

South Sudan

The Republic of South Sudan, the world’s youngest nation, gained full independence in 2011. With an estimated population of 8.2 million, South Sudan performs poorly across all developmental indicators, with high levels of malnutrition and food insecurity, and weak service provision. The humanitarian situation in South Sudan was further compounded by an outbreak of conflict in December 2013 between the SPLA in government and SPLA in Opposition (SPLA-IO). Since December 2013 the combination under-development and the conflict has displaced 1. 46 million people, leaving an estimated 4.6 million people in urgent need of aid, with the States of Unity, Upper Nile and Jonglei particularly badly affected. [1]

DRA context

In response to the challenges of the humanitarian system and the growing gap between humanitarian needs and humanitarian funding, the Dutch Minister for International Trade and Development Co-operation set-up of a Dutch Relief Fund (DRF) to increase effectiveness of Dutch humanitarian aid. The DRF is a pilot funding window for relief operations for the period 2015 – 2017, with a total budget of €570 million of which €120 million has been reserved for Dutch NGOs.

The Dutch Relief Alliance (DRA) is an alliance of 12 Dutch NGOs which respond to chronic crises as well as acute crises, for which they receive funding from the DRF. The DRA currently funds nine joint responses, of which the South Sudan Joint response is one.

The specific objectives of the Dutch Relief Alliance as included in the MoU DRA are:

1. Deliver fast humanitarian aid in major crises;

2. Deliver humanitarian aid linked to needs and gaps in response to major crises in a timely, appropriate, effective and efficient manner;

3. Generate synergies and cooperation between the Members aimed at increasing efficiency and effectiveness in providing humanitarian aid in crisis situations;

4. Increase the visibility of this Dutch contribution towards the Dutch constituency, Parliament and in-country.

5. Work together, also with other parties, to tackle the major bottlenecks in the humanitarian practice through co-created innovation, joint learning and research;

Joint Response South Sudan

A one year programme beginning on the 1^st January 2015 and ending on the 31^st of December 2015 with a budget of € 10,985 216, the SSJR is an emergency lifesaving, humanitarian program targeting both the Internally Displaced Persons and Host communities who are chronically food insecure and affected by ongoing conflict. The SSJR operates in the following states in South Sudan : Abyei area (disputed), Western Bahr-el-Ghazal (WBeG), Equatoria (Eq) Central, Eq East, Eq West, Jonglei, Unity, Upper Nile, Warrap States.

The SSJR consortium is led by Save the Children International (SC) and includes CARE, Cordaid, Dorcas, HealthNet-TPO, ICCO & Kerk in Actie, OXFAM-Novib, PLAN, Save the Children, Stichting Vluchteling/IRC, World Vision, ZOA. The thematic areas of the activities of all partners are listed in Annex 2.

The projects are implemented in the clusters Food Security and Livelihoods (FSL), Health, Non-food Items and Emergency Shelter (NFI), Nutrition, Protection and Water, Sanitation and Hygiene (WASH) (see Annex 2 for more information). The identified results and locations are in line with the strategic priorities of the Crisis Response Plan 2014 (CRP). To accommodate the efficient planning and monitoring by individual agencies some flexibility vis-à-vis the CRP has been built in at the level of describing activities and output indicators.

2. Purpose and scope of the evaluation

The SSJR has a contractual obligation towards the donor, the Dutch Ministry of Foreign Affairs, to ensure the realisation of an evaluation report. The purpose for this evaluation is twofold. On the one hand , the evaluation report will assess the performance of the SSJR and ensures accountability towards the Dutch Government and public. On the other, it offers a learning aspect for all stakeholders. The findings will be used as input for phase 2, which will already have started on 1^st January 2016. Lessons learned and recommendations for programme implementation and design should be part of the evaluation report.

The scope of this evaluation is reflected in the following thwo objectives:

SSJR Strategic Objective 1: Effective implementation of humanitarian aid projects in priority sectors and locations after 12 months.

SSJR Strategic Objective 2: Added value generated for key stakeholders (MoFA, UNOCHA, SSJR members, target populations) through the collaboration between the partners in the Joint Response.

SSJR Strategic Objective 1: Effective implementation of humanitarian aid projects in priority sectors and locations after 12 months

The first objective refers to the success and quality of the response. Success refers to whether and to what extent the objectives of the programme have been achieved, considering any changes in context or needs over the life of the response, while ensuring quality through globally accepted paradigms for quality relief in humanitarian contexts, such as CHS or Sphere. The logical framework of the SSJR programme shows activities and output indicators for each Strategic Objective set by UNOCHA SRP in September 2014. The JRE log frame is included in Annex 1 and the thematic targets are in Annex 2.

Scope: The evaluation should assess the following OECD-DAC evaluation criteria:

I. Quality: The evaluation should assess the quality of the response in terms of meeting humanitarian needs, timeliness and adherence to humanitarian standards (CHS and Sphere);

II. Appropriateness: The situation changed after the approval of the funds and over the course of the response. Therefore, the evaluation should address to what extent the JRE was able to adapt and provide appropriate response to changing local needs and the priorities of the people, taking into account the specific needs of women and vulnerable groups, and to what extent the JRE partners involved beneficiaries in project design and implementation;

III. Relevance: to what extent does the SSJR programme connect to, and contribute to the fulfillment of, the UN HRP 2014 and the changing environment as described in the UN HRP?

IV. Effectiveness: The logframe for the JRE was based on UN OCHA’s Overview of Needs and Requirements and included broad Strategic Objectives and set result areas. The evaluation should assess to what extent the planned outputs were reached and to what extent the JRE activities contributed to the results and the Strategic Objectives; what were the major factors influencing the achievement or non-achievement of the objectives of individual agencies and the response as a whole; have there been any unexpected positive or negative side-effects on beneficiaries as a result of the aid that was provided under the SSJR and how was this dealt with by implementing partners?

V. Efficiency: How efficient was the delivery of aid within SSJR per organization and overall as a consortium? What would be opportunities within SSJR to reach more beneficiaries with the available budget or to reduce costs while reaching at least the same number of beneficiaries? Concrete evaluation questions on efficiency under DRA objective 2 added value are described in Annex 4.

VI. Connectedness: How did SSJR work on including elements of sustainability in their program? What are opportunities to include more sustainable elements of recovery in the next phases of SSJR (ranging from materials and structures to participative methods and innovative ideas)?

i. Coverage: coverage involves determining who was supported by humanitarian action, and why? What were the main reasons that the intervention provided or failed to provide major population groups with assistance and protection, proportionate to their need?

VII. Coordination: During the period of the SSJR activities, a wide range of actors was active in responding to the crisis in the region. The evaluation should include to what extent the activities of the SSJR organisations were complementary to the work of other stakeholders, prevented duplication, and contributed to the larger response activities in the countries. (N.B. Annex 4 under DRA objective 3 (harmonization & complementarity) also gives attention to duplication and joined complementarity but within the SSJR only)

SSJR Strategic objective 2:Added value generated for key stakeholders (MoFA, UNOCHA, SSJR members, target populations) through the collaboration between the partners in the Joint Response

The joint humanitarian response for South Sudan (SSJR) aims to contribute to more effective delivery of humanitarian aid by Dutch government and humanitarian organisations. By implementing their own projects while collaborating under the umbrella of the joint response, the 11 participating agencies aim to provide added value for several actors in the Netherlands and in South Sudan. The value added ranges from efficiency gains for the Ministry of Foreign Affairs by funding humanitarian response through one humanitarian lead agency; to improved learning and innovation by humanitarian agencies through exchanging resources and methodologies. We can distinguish added value on the Dutch level, for the Ministry and at Head Quarters level of Dutch DRA member organization, as well as country level added value for joint response partners in country. Added value on both levels need to be assessed in this evaluation.

Scope: The evaluation should:

I. Assess if added value targets as stated in the logframe and proposal have been met. Annex 1 shows the logframe. Annex 3 lists the targets for added value as described in the proposal for the SSJR.

II. Assess the added value of cooperation within this SSJR, including any opportunities which have arisen for added value over the life of the programme. Annex 4 lists a range of evaluation questions to measure the added value of the SSJR under the DRA. As Lead of this SSJR, Save the Children Netherlands wishes to fully use this tool that was developed by the Monitoring and Evaluation & Added Value Working Group (DRA MEAV WG) to measure the added value of the DRA as a new mechanism to structure Dutch emergency responses and the added value of partnering in a joint response. That’s why all questions need to be asked to respondents. Perhaps not all questions can be answered yet and perhaps not all answers yield much information; we like to receive feedback from the consultant on how applicable and feasible the tool is.

III. Identify key lessons learned from SSJR cooperation and recommendations for future cooperation within Joint Responses;

IV. Discuss how the SSJR members perceived the added value of the Joint Response for their own organisation.

V. The outcomes of this exercise can be seen as a baseline for measuring added value within the DRA mechanism, as an innovative mechanism to fund humanitarian programmes in the Netherlands. This baseline would give a proper insight in the inception phase of the DRA (2015 – 2017) and the added value learning curve in the next three years, which will be investigated in the mid-term and end-term evaluation of DRA.

3. Methodology

The methodology will be developed by the consultant as well as all relevant tools and presented in the inception report. The evaluation should be based on combined quantitative and qualitative research methods. All Strategic Objectives under which activities are implemented should be assessed. The data collection strategy should include the use of a number of tools to gain a deeper understanding of the outcomes of the project, including:

• Desk review of background documents
• HH surveys
• Choice for geographical area’s and thematic areas to be discussed with consultant
• Field visit to project sites (urban as well as rural areas; security permitting)
• Focus group discussions
• Key informant interviews in South-Sudan and The Netherlands
• Participative workshops and presentation during learning events in South Sudan and The Netherlands

4.Deliverables

The consultant is expected to lead, accomplish and submit the following deliverables within the agreed timeframe and budget:

1.An inception report, which will serve as an agreement between parties on how the evaluation will be conducted. Items to address:-Understanding of the issues and questions raised in the ToR
-Data sources; how to assess the questions in the ToR
-Research methodology, including sampling procedure and suggested sample size
-Team composition
-Schedule of activities and traveling (timeline)
-Proposal for a learning event
-Detailed budget
-Appropriate validated draft data collection tools (survey questionnaire, KII/FGD guidelines)

2.A 25-50 page draft and final report (in MS Office and PDF for final), excluding annexes and in English, in the following format at a minimum, to be submitted to Save the Children Netherlands. Preferably as little text as possible, use of graphs, tables and a dashboard with results. The report should consist of:

a) Executive Summary in bullets (max. 2 pages)

b) Introduction c) Methodology, including sampling and limitations

d) Analysis and findings of the evaluation. The analysis should consist of three parts:

SSJR Strategic Objective 1

SSJR Strategic Objective 2

Feedback on the use of the DRA Added Value Evaluation tool

e) Address concerns, lessons learned and comments from SSJR partners f) Some cases of success stories and quotes from respondents g) Conclusions for each of the three parts and for all of the evaluation questions
h) Recommendations on SSJR strategic objectives and the DRA Added Value Evaluation tool
i) Annexes:
Relevant maps and photographs of the evaluation areas
Bibliography of consulted secondary sources
Finalized data collection tools
List of key informants
Raw data of the research in MS Excel format
Powerpoint presentation of findings and recommendations, validation and feedback sessions with key stakeholder and SC South-Sudan

3.Presentation of findings to SSJR partners and a Learning and evaluation meeting in Juba. The evaluators will be required to present preliminary findings before the publication of the final report to SSJR partners and lead a half day workshop once a final draft has been agreed in Juba. The structure and activities of the learning and evaluation meeting will be agreed with Save the Children. We expect attendance of 20-30 people in Juba.

4.Presentation of findings to SSJR partners (and the Dutch Ministry of Foreign Affairs if possible) and a Learning and evaluation meeting in The Hague. The form of this session is to be developed by the Consultant. A presentation and some workshop sessions would be an option. Logistical support (ie. venue) is available. We expect attendance of 15-20 people in The Hague.

5. Indicative timescales

21 Dec 2015 Draft inception report, presented to SSJR partners (in NL) for comment (1 workday)

5 Jan 2016Deliverable 1: final inception report including budget, quantitative and qualitative research tools, approved by SSJR Lead (4 workdays)

5 Jan 2016 Desk review (3 workdays)

7 Jan- 5 Feb 2016 Field visits SS and interviews in SS and the Netherlands (30 workdays)

8 Feb 2016 Presentation of initial findings in NL (3 workdays)

29 Feb 2016 Draft Evaluation Report, for comment by SC-SS, SSJR partners (20 workdays)

7 Mar 2016Deliverable 2: Final Evaluation Report (5 workdays)

8 Mrt 2016Deliverable 3: Learning session in The Netherlands (4 workdays)

10 Mrt 2016Deliverable 4: Learning session in Juba

Total: 70 workdays ** Number of workdays as if for one consultant at senior rate, however lumpsum to be divided over a team of researchers, preferably 2-3 people.

6. Roles and responsibilities

Lead Consultant

• Develop an inception report, detailing the methodology-stakeholders to be interviewed, tools to be developed, time frame for the evaluation and budget
• Holds the overall management responsibility of the evaluation, including designing and carrying out the evaluation, drafting the final report and debriefing the project team and key stakeholders.
• Liaise with Save the Children staff throughout the process, providing weekly updates and seeking their input and advice where necessary.
• As a condition of entering into a consultancy agreement the evaluator and research assistants must sign the SCI Child Safeguarding Policy and abide by the terms and conditions thereof.

Save the Children South Sudan

• Provide all required background materials to the consultant in a timely manner.
• Read and provide comments on the proposal plans submitted by the consultant (especially the proposed research methodology, the information gathering techniques used and the suggested target dates);
• Provide guidance throughout all phases of execution, approving all deliverables, and facilitating access to any documentation (or any person) deemed relevant to the evaluation process.

Save the Children Netherlands (Contract holder)

• Review Evaluation consultant’s qualifications or specialized knowledge or experience required.
• Oversee the service provider by managing the consultancy contract; monitor adherence to specified deadlines; facilitating access to required information.
• Review and comment on analysis and draft report submitted by the evaluator i.e. preliminary reports and the final report
• Providing feedback to draft data collection tools and reports

SSJR partners

• Providing data/information for desk review
• Arrange logistics and planning of the field research, supporting the evaluation team during field work in their areas, act as guides and bring research team to the beneficiaries

7.Qualifications and experience

Required

• Academic degree in International Development Studies, Humanitarian Action or a related field
• Demonstrated experience in humanitarian response and knowledge of humanitarian standards (CHS, Sphere, Code of Conduct)
• Demonstrated experience of leading evaluations of humanitarian response programmes
• Demonstrated experience with quantitative and qualitative research, data base management and statistical data analysis
• Experience of working in South Sudan/East Africa
• Experience of evaluating consortia/joined responses
• Proven record of communicating with beneficiaries, also with children using child friendly methods
• Demonstrated knowledge of Dutch development and funding streams and access to the Dutch Ministry for an interview
• Ability to assess and further develop a conceptual evaluation tool
• Relevant subject matter knowledge and experience regarding the thematic areas of this SSJR
• Ability to deal with hardship and remote area field work
• Advanced English writing skills

Preferred

• Strong understanding of humanitarian and evaluation ethics and a commitment to ethical working practices
• Deep understanding of CHF and UN Strategic Response Plan
• Experience in and understanding of measuring the added value of partnerships/ cooperation
• Experience of working in insecure environments in South Sudan and managing security risks
• Action-oriented and evidence based approach and strong drive for results

• Highly developed self-management, and communication skills

• Guiding Principles and Values

Adherence to Save the Children Code of conduct, Child Safeguarding practices and confidentiality when interviewing or photographing children.

Interested Parties can contact Esther Ten Hoorn, MEAL Advisor of Save the Children Netherlands, at esther.ten.hoorn@savethechildren.nl for a full technical document and Annexes of this ToR.

Interested Parties are requested to submit a proposal explaining their comprehension of the proposed consultancy, and how they would approach this assignment with a summary of their methodology especially in terms of how the party plans to meet the objectives. Additionally, they should submit one or two examples of a similar evaluations conducted previously. The application should include a team composition with Lead Consultant and at least 2-3 experienced evaluators. The application should include minimum three CVs of the persons to be involved in the assignment, relevant experience, a detailed budget in Euros and time availability. All proposals should be received by 15 december 09.00 (CETime) by email to Esther Ten Hoorn, MEAL Advisor of Save the Children Netherlands at esther.ten.hoorn@savethechildren.nl