Railway Children Africa is looking for an independent consultant or firm
to conduct a high quality external evaluation of a 3 year project
funded by DFID's Global Poverty Action Fund, implemented in Mwanza,
Tanzania.
Railway Children (RC)
Railway Children Africa is registered in Tanzania as a Limited Company
(77698) under the Companies Act, with NGO compliance (1563) under the
Non-Governmental Organisations Act. Railway Children Africa is a
subsidiary of Railway Children. Railway Children is registered in the UK
as both a Charity (1058991) and a Company Limited by Guarantee
(3265496).
Railway Children exists to help vulnerable children in grave
circumstances. Our objective is to provide relief to children and young
people who are in conditions of need, hardship or distress and in
particular to those living on the streets.
As an organisation our primary aim is to enable and create positive
sustainable change in the lives of children living alone and at risk on
the streets. Our ‘3 Step Change Agenda’ embodies our vision of real
change being achieved through success in three interwoven, inextricable
areas:
- Meeting the immediate needs of children on the streets
- Shifting local perception
- Holding governments responsible
Railway Children is receiving funds from DFID's Global Poverty Action
(GPAF) Fund for a 3 year period - January 2013-March 2016 - and the
external evaluation outlined in this ToR will aim to verify and learn
about impact achieved in implementing the project supported by GPAF in
Mwanza, Tanzania.
The Global Poverty Action Fund (GPAF)
DFID provides significant funding to civil society organisations (CSOs)
annually in line with its overall strategy to alleviate poverty and
promote peace, stability and good governance. The Programme Partnership
Arrangement (PPA) and Global Poverty Action Fund (GPAF) are two of
DFID’s principal funding mechanisms and have provided £480 million to
approximately 230 CSOs between 2011 and 2013. The current political
climate and results-based agenda demand a rigorous assessment of the
effectiveness of funds disbursed to ensure that they are managed to
provide value for money.
TripleLine is the Fund Manager for the GPAF and is responsible for
assessing performance of grantees at the project level. Coffey
International Development is the Evaluation Manager and is responsible
for assessing the performance of the funding mechanisms as a whole. For
more information on the fund level valuation, please see the 2012
Evaluation Strategy.
Railway Children Project Description
Project title: ‘Provision of support, reintegration and referral
services for 1,200 children and youth on the streets of Mwanza,
Tanzania’
This project has aimed to create sustainable support structures for
children on the streets in Mwanza; the major urban centre for a radius
of several hundred kilometres and the destination for the rural
population migrating to the city including children escaping poverty and
abuse. Government agencies have few resources to respond to the needs
of children and local agencies are under resourced in terms of funds
available and appropriate skills. We are developing quality practice
across agencies in the city which work with children on the streets,
which includes contact, short term care and effective reintegration with
families for children already on the street and support for vulnerable
families to prevent their children going to the street. We provide
opportunities for homeless youth to learn new skills to enable them to
improve their livelihoods and leave the street. The project is also
working with local government to help build effective linkages with
service providers to encourage sustainable structures and processes.
Purpose of the independent final evaluation for GPAF grantees
The evaluation report will ultimately be submitted by Railway Children
to DfID, and will be used to inform the GPAF Fund Manager’s
understanding of Railway Children's performance at the project level and
will also be used to inform the Evaluation Manager’s assessment of
performance at the GPAF fund level. The independent final evaluation
report needs to be a substantial document that (a) answers all the
elements of the Terms of Reference (ToR); (b) provides findings and
conclusions that are based on robust and transparent evidence; and (c)
where necessary supplements the grantee’s own data with independent
research.
Key objectives of the evaluation
The evaluation has two explicit objectives that are explained below:
1 -
To independently verify (and supplement where necessary), Railway
Children's record of achievement as reported through its Annual Reports
and defined in the project logframe;
2 - To assess the extent to which the project was good value for money, which includes considering:
- How well the project met its objectives;
- How
well the project applied value for money principles of effectiveness,
economy, efficiency in relation to delivery of its outcome;
- What has happened because of DFID funding that wouldn’t have otherwise happened; and
- How well the project aligns with DFID’s goals of supporting the delivery of the MDGs.
Verification of grantee reporting
The first task of the final evaluation is to verify Railway Children's
achievement. The record of achievement will be presented in past Annual
Reports and progress against the project logframe. This exercise could
include verifying information that was collected by the grantee for
reporting purposes and possibly supplementing this data will additional
information collected through primary and secondary research. Verifying
the results from the project log frame will begin to capture what the
project has achieved. However, there will be other activities and
results that occur outside of the logframe that may require examination
in order to respond to the different evaluation questions. Verifying
reporting will also necessarily include a review of the data and systems
that were used to populate results.
Assessment of value for money
The evaluation should assess the extent to which the delivery and
results of the project are good value for money. Value for money can be
defined in different ways, but at minimum the evaluation report should
include an assessment against:
- How
well the project applied value for money principles of effectiveness,
economy, efficiency in relation to delivery of its outcome;
- What has happened because of DFID funding that wouldn’t have otherwise happened.
Evaluation questions
To ensure comparability across the final evaluation reports, the
evaluator(s) should respond to the questions below. Please note that the
attention given to each evaluation question may vary depending on the
objectives of certain projects and the availability of data, so the
independent evaluator(s) should use his/her discretion in the level of
effort used to respond to these questions.
The evaluator is encouraged to structure their research questions
according to the OECD-DAC criteria of relevance, effectiveness,
efficiency, sustainability and impact.
GPAF's generic Evaluation Questions:
Relevance
- To what extent did the Railway Children support achievement towards the MDGs, specifically off-track MDGs?
- To what extent did the project target and reach the poor and marginalised?
- To
what extent did the project mainstream gender equality in the design
and delivery of activities (and orother relevant excluded groups)?
- How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time?
Effectiveness
- To what extent are the results that are reported a fair and accurate record of achievement?
- To what extent has the project delivered results that are value for money? To include but not limited to:
- How
well the project applied value for money principles of effectiveness,
economy, efficiency in relation to delivery of its outcome;
- What has happened because of DFID funding that wouldn’t have otherwise happened; and
- To what extent has the project used learning to improve delivery?
- What are the key drivers and barriers affecting the delivery of results for the project?
Efficiency
- To what extent did the grantee deliver results on time and on budget against agreed plans?
- To what extent did the project understand cost drivers and manage these in relation to performance requirements?
Sustainability
- To what extent has the project leveraged additional resources (financial and in-kind) from other sources?
- What effect has this had on the scale, delivery or sustainability of activities?
- To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends?
Impact
- To what extent and how has the project built the capacity of civil society?
- How many people are receiving support from the project that otherwise would not have received support?
- To what extent and how has the project affected people in ways that were not originally intended?
Railway Children's additional questions
- CSO
capacity building: To what extent did Railway Children succeed in
building the institutional and programmatic capacity of the implementing
organisations? What were the conditions in each partner and/or our
support that led to success or acted as a barrier?
- To what extent did emotional and relational support provided by staff have a positive impact on families and children?
- How
well have we balanced the pressure to deliver on project plans against
the individual needs of beneficiaries - and what have we learnt from
this.
- Youth:
To what extent has the economic empowerment been successful (grants /
placements) in realising youth working in their selected field? What are
the main changes that the youth have experienced as a result of our
intervention?
- Economic
empowerment across the programme: Analysis of failure and success rates
across the programme: What were the factors or conditions caused these
differences?
- Government
and stakeholder collaboration (output 4) - What have been the barriers
of the partnership in building sustainable government collaboration and
referral pathways with other organisations?
METHODOLOGY
The consultant will be responsible for designing the methodology, with
input from Railway Children. To fulfil the objectives of this exercise
both quantitative and qualitative assessment methods should be employed.
We require a participatory methodology whereby the work engages all key
stakeholders including beneficiaries; children, youth,
parents/guardians, community members, partner staff and Railway Children
staff.
Selected consultants will be expected to provide an overview of the
proposed methodology as part of the tendering process including
evaluation tools to be used, progress reports and debriefing meetings
and a detailed work plan should be established before the work begins.
Evaluation Outputs
A report should be produced that documents findings of the evaluation in
relation to the key questions outlined and be consistent with the
reporting guidelines below. The draft report will be shared with Railway
Children and respective partner organisations for their comments prior
to finalising the report.
REPORTING
The report will be prepared within 20 working days following the end of
the evaluation and is not expected to be more than 40 pages, excluding
appendices.
Report structure
Front and cover and title page
- Title of the evaluation.
- Project location
- Name of the partner organisations.
- Names of the evaluation team and any support staff
- Dates and duration of the evaluation
- Date report completed
Acknowledgements
- Beneficiaries, Communities, advisers, team members, funders etc.
Contents page
Executive summary
- A summary of the most important parts of the report
- Who was involved in the evaluation
- The purpose and objectives of the evaluation
- How the evaluation was carried out, where and when
- Key findings
- Key recommendations
Background information
- Only information directly relevant to the report's analysis and conclusion
- The
context of the project - social, economic, political background along
with the existing legal framework and particular problems faced by the
target communities
Evaluation Purpose and Methodology
- What methods were chosen and why?
- Which stakeholders were involved and why were they chosen?
- How was the information collected and by whom, and which methods were used?
- How reliable and valid did the methods prove to be?
- Include any evaluation timetable or schedule in an appendix.
Findings and Analysis, {only information directly relevant to the report's analysis and conclusion}
The report will include:
- Key findings – Responses to each of the key questions
- Challenges and learning
Lessons Learnt and Recommendations:
This section should reflect and recommend on the project design and
outline any improvements to this implied by the findings of the
evaluation.
Appendices
This should include technical information referred to in other sections,
for example details of methods used, questionnaires, timetables and
schedules, a list of informants and the team's work schedule.
SCHEDULE:
A schedule should be provided with the tender application detailing time
spent for each part of the assignment that fits broadly within the
following framework:
- Consultant selected and TOR signed - Before 27th November 2015
- Design, methodology and tools are finalised - By end of December 2015 In consultation with RC.
- Field work – carrying out surveys. January 2015 Mwanza - Tanzania
- Data
compilation, analysis & reporting February 2015 First draft of the
report is ready to be sent to partners and RC for feedback and comment.
- Sharing
of report and finalisation March 2015 Railway Children and partners
review the report Incorporation of feedback from all stakeholders
PROJECT FINAL EVALUATION TEAM
The composition of the evaluation team will be agreed in advance by Railway Children.
PROFESSIONAL FEES AND LOGISTICS
Full details of all Professional Fees and additional costs in order to
complete the assignment should be in the tender document. The total
budget for this assignment is £15,240.
RC will pay 30% advance payment at the time of commissioning of the
exercise and 30% after the completion of the field work. The remaining
40% would be paid on the completion of the evaluation process and the
submission of the report.
CONFIDENTIALITY
The engagement will be carried out confidentially. The consultants will
neither use, nor appear to use, information acquired during the course
of this engagement for either personal advantage or the advantage of a
third party. The consultants will have to abide by Child Protection
Policy of Railway Children and the partner organisations.
