1.0 RIGHT TO PLAY
Right To Play is a global organization committed to improving the lives of children and youth affected by conflict, disease and poverty. Established in 2000, Right To Play has pioneered a unique play-based approach to learning and development which focuses on quality education, life skills, health, gender equality, child protection and building peaceful communities. With programming in over 20 countries, Right To Play transforms the lives of more than one million children each week using play and sports, both inside and outside of the classroom. In addition to our work with children, Right To Play advocates with parents, local communities, and governments to advance the fundamental rights of all children. Right To Play is headquartered in Toronto, Canada and has operations in North America, Europe, the Middle East, Africa and Asia. Visit us at www.righttoplay.com
Given the challenging COVID 19 context, including limitations on international travel and uncertain opening and access to public schools in Ghana, Rwanda and Mozambique, Right To Play is seeking consultancy services for a multi-county education mid-term evaluation that will operate under a remote management model. In practice, the selected firm will not travel to the required countries and will rely on Right To Play Offices to subcontract local firms for data collection according to field protocols. The selected firm will be fully responsible for the end product. The technical oversight of data collection through local firms is retained remotely so as to assure quality and coherence across the three countries. Services include:
- revision of the existing baseline evaluation framework design;
- the review, refinement and/or development of instruments for midterm needs;
- Remote planning and monitoring on data collection in Ghana, Rwanda and Mozambique (Field protocols, tools, mobile data collection systems, etc)
- Design of remote fieldwork administration and logistics plan;
- Guidance for Right To Play Ghana, Rwanda and Mozambique on management of local data collection to be undertaken by a firm to be subcontracted by Right To Play Offices
- review and data cleaning;
- data analysis; reporting; and dissemination documents. This will include the writing of any and all accompanying reports as specified in the key deliverables.
2.0 PROGRAM OVERVIEW
The Gender Responsive Education and Transformation (GREAT) program aims to directly address the learning needs of children and youth through a five-year quality education program in three countries: Ghana, Mozambique, and Rwanda. From 2018 to 2022, the GREAT program endeavours to improve quality education for girls and boys at the primary school level through a sustainable and replicable child-centred, play-based learning model. Right To Play will contribute to this ultimate outcome through focused interventions at the school, community and policy level that will result in:
- Increased integration of gender-responsive play-based learning by teachers into teaching practice
- Increased engagement of parents and caregivers in addressing gender-specific learning needs of girls and boys, including at-risk children
- Increased adoption of gender-responsive play-based learning in in-service and pre-service teacher training
In 2020 the Logic Model has been reviewed to accommodate for complementary COVID19 activities (indicators).
GREAT is currently in the third year of implementation. The main aim of the Mid-Term Evaluation will be to systematically analyze GREAT´s progress and achievements so far.
Following the initial baseline undertaken in 2018, the findings of this mid-term exercise will inform the further implementation of the project, allowing to fine-tune and – where needed – redirect current strategies to maximally achieve aspired impact.
The specific objectives of the Mid-Term Evaluation are:
- Determine the project’s impact on quality education outcomes from baseline to midterm
- Develop concrete recommendations for the next years of project implementation
- Stimulate learning and reflection among RTP and key beneficiaries, including from what has or hasn’t worked so far
- Check-in against baseline recommendations.
The consultancy firm will propose specific research questions according to the DAC criteria in view of the exceptional COVID 19 circumstances. There was a temporary halt of the GREAT intervention and introduction of COVID 19 activities. Questions to be considered include: What is the progress made towards or contribution towards the ultimate, intermediate and immediate outcomes? What are the likely COVID 19 unintended effects on the project? What adaptations to the TOC can be made to increase the programme’s relevance in the COVID 19 contexts where it operates?
4.0 EVALUATION OVERVIEW
The midterm study for the GREAT program will be conducted within the anticipated timeline of September 2020 to June 2021.
The initial part of the midterm study will involve a comprehensive review of all existing GREAT program, monitoring, and evaluation plans including the logic model, performance measurement framework, and measurement tools.
Simultaneously it will involve the review of the existing baseline evaluation framework. The continued use of the OECD/DAC criteria for evaluating development assistance is required. This includes relevant research questions, a rigorous sampling methodology and review, refined and/or draft measurement instruments to be piloted during the training of data collectors and enumerators. The review and adjustment of the existing baseline evaluation framework design, including the analytical framework to meet midterm needs is requested, while ensuring coherence for the final end line evaluation; the review, refinement and/or development of reliable and valid quantitative and qualitative research instruments for mid-term measurement needs. This includes designing, piloting, and calibrating EGRA and EGMA assessments so as to generate information on impact on learning between baseline and midterm.
The sampling methodology should take into account both baseline and endline sample size calculations to enable meaningful statistical testing at the endline, taking into account expected sample attrition rates. Remote planning of fieldwork/data collection administration and logistics plan will be developed and implemented through Right To Play in the three countries of implementation, who will be subcontracting local firm/enumerators for data collection. At baseline, ODK-based data entry platform called KoboToolBox was used and there is a continued preference for such applications (possibility of entering data from any device while offline, and the reduction of data entry mistakes by virtue of its user-friendliness). Specific instructions on system to be used for data entry and submission are expected as well as coordination with Right To Play Country Office and selected local firm/enumerators to ensure timely quality data submission.
No international travel will be required from the remote consultancy team who will then be required to develop a robust remote data collection strategy in the three countries. They will provide guidance to Right To Play Offices on requirements for local data collections so that Right To Play sources, hires such local services.
The selected firm will train Right To Play MEL Specialists and provide remote support in training (as needed) enumerators from the locally subcontracted firm. Right To Play staff are expected to lead the procurement of services for local data collection, management of consultancy services, coordination between lead international consultants and local consultancy firm.
The final part of the midterm study process will consist of the review, cleaning and analysis of the data collected and drafting of the multi-country and country-level reports. The inputs from the validation workshops will be conducted in each of the assigned countries will need to be accounted for in the last version of the Report. Lastly, the consultancy team must also produce relevant dissemination documents (presentation slides) and a dissemination plan at the global and country-levels.
For further details please reference the key deliverables section.
The evaluation framework design process is expected to be carried out at the consultants’ base, with significant collaboration with the GLOBAL Right to Play Monitoring, Evaluation and Learning team. No travel is expected. For the field work and data collection, only in-country travel is expected in the three implementing countries (Ghana, Mozambique and Rwanda). This is will assured by the local consultancy services/enumerators and under the technical guidance of the international firm.
The first part of the consultancy will involve a document review of the GREAT measurement framework and its accompanying tools; desk research and literature review; and engagement with the program team in order to produce a quasi-experimental evaluation design/framework that will be used for the midterm and that will continue to serve as the key reference for the endline evaluation, particularly with regards to sample size. Any revisions to the performance measurement framework must feed into the evaluation design/framework and where possible, sample sizes will be calculated using a 95% confidence level and confidence interval of 5. The sample should allow robust results between the treatment and comparison group.
As the evaluation will adopt a quasi-experimental difference in difference design to determine the project’s impact. The evaluation will track the same girls and boys who were sampled at baseline. Alignment with baseline is expected which adopted a mixed-methods, hypothesis-driven quasi-experimental design (Difference-in-Differences technique). Baseline sample sizes were calculated based on power analysis and a required power of 0.8, and a 0.25 effect size in learning gains over and above a control group. Through this method, it was determined that the minimum sample size should be a total combined of 506 cases. Adding an expected 30% attrition buffer between evaluation years this yielded a baseline sample size of 658 children. Given that RTP wishes to generalize for boys and girls separately in the impact DID model, the sample size was doubled following Turner’s (2003). This yielded a minimum final estimate of 1,316 per country (total of 3,948 boys and girls).
At Midline the evaluation team will be required to track the same participants and support the in-country partner with implementing tracking processes and closely adhering to replacement protocols.
**PLEASE VISIT OUR WEBSITE (https://righttoplaydiag107.blob.core.windows.net/rtp-media/documents/RFP_-Gender_Responsive_Education_and_Transformation__GREAT-MidTerm-_July_2020.pdf)TO VIEW SAMPLE SIZES FOR THE ORIGINAL COHORT TO BE TRACKED.***
*In terms of qualitative tools, at baseline a total of 223 FGDs and KIIs were organized across the three countries.
The midterm sample sites, samples and groups proposed should enable calculation of statistical significance at endline (should account for potential future attrition as these same sample characteristics will be used at the endline).
The second part of the consultancy will involve the planning and implementation of the midterm study that will enable the program to understand progress towards outcome so far, taking into account the specific COVID 19 circumstances where there was temporary closure of schools and hence GREAT intervention. As the field work will occur in the countries of implementation, the consultancy is expected to collaborate significantly with Right To Play country teams in order to successfully provide technical guidance for remote guidance on how to ensure quality field work.
Prior to field work, the international team will be required to conduct a pilot exercise in each country to identify literacy and numeracy tests of comparable levels of difficulty to those used at baseline. Data from the pilot will be analyzed to calibrate literacy and numeracy subtasks to ensure the tools are fit for purpose. This process will result in final literacy and numeracy tools for midline and for endline that are of the same levels of difficulty as those used at Baseline.
The final part of the consultancy is the analysis of the data collected and the production of the midterm reports with the validation workshop findings and appropriate dissemination documents, incorporating and integrating feedback from relevant Right To Play staff.
Through significant, collaborative engagement with relevant staff at Right To Play, particularly the Monitoring, Evaluation and Learning team, it is expected that the consultancy will strengthen the organizations’ M&E capacity, particularly in the development and implementation of sound, rigorous research activities. As such, the consultancy will be expected to share and contribute to the Right To Play’s knowledge-base while deepening the capacities of the organization to more effectively conduct research in the future.
Data Collection Tools
The evaluation design framework will consist of a toolkit with relevant, appropriate tools to measure progress on the outcomes selected and to adequately answer the research questions posed. Existing tools will undergo a critical review and revision while new ones will be developed. The following is a (non-exhaustive) list of tools that will need to be revised/developed as needed:
· Student Literacy and Numeracy Assessments
· Household survey (HHS)
· Child’s survey (CS) – above 7 years old measuring various PMF components
· Teacher’s survey
· School-wide assessment
· Parent council’s survey
· Classroom observations (teaching practices)
· Key-informant interviews (stakeholders and beneficiaries)
· Key-informant interviews (KII)
· Focus group discussions (FGD) (children, parents, community members, other stakeholders)
The research planning process will entail a remote training of Right To Play staff, enumerators and data collectors which will include piloting of the instruments to ensure satisfactory reliability and validity. Tools will be subsequently refined after training and prior to going to the field. High consideration will be given to firms who have implemented data collection using mobile tools and software.
Data will be analyzed both quantitatively and qualitatively and will cover all areas described in Section 7 below “Areas of Focus.” Moreover, the revised analytical framework will be submitted as part of the evaluation design process which will detail the specific analytical methodology that will be used to produce the result for each indicator measured and each research question posed. The consultant will engage significantly with the Right To Play team to determine and agree on these.
The consultant is encouraged to utilize analytical software to analyze both the quantitative (e.g. Excel, SPSS, STATA, R), and qualitative (e.g. Nvivo) data. Outputs from data analysis will be submitted as part of the deliverables, as well as the scripts (or list of commands) with clear notes/guidance, particularly for quantitative data analyzed (in SPSS, STATA, R).
1.0 AREAS OF FOCUS
**A Performance Measurement Framework can be shared with shortlisted applicants**
2.0 GENERAL CONDITIONS OF THE CONSULTANCY
Steering Committee **
A steering committee of key implementation stakeholders (i.e. Right To Play country and global staff) will be formed to guide and inform the research process. They will help to inform the relevance and appropriateness of the evaluation design framework; the data collection tools and the analytical framework. They will also help to ensure that the research planning and data collection processes are sound, culturally appropriate and contextually relevant to Right To Play’s programmatic needs and to the needs of all relevant stakeholders (i.e. beneficiaries, community members and partners).
The measurement plan adheres to the Global Affairs Canada GREAT program performance measurement framework (PMF) and uses a results-based management approach for tracking the progress of outputs and outcomes against targets.
· The consultancy firm/group will have an orientation to Right To Play’s program delivery model;
· The consultancy firm/group will submit a detailed work plan and time frame for the completion of the midterm research;
· The data analysis and draft reports will be shared with Right To Play with sufficient time to allow for Right To Play to review and provide feedback, which will be incorporated into subsequent work;
· After the completion of any research reports, a presentation will be given to Right To Play staff and appropriate stakeholders to share both results and recommendations;
· Throughout each phase, the consultancy firm/group will be expected to maintain regular communications with Right To Play regarding progress;
· No Travel is envisaged
· The consultancy firm/group must follow OECD-DAC principles for evaluation;
· All materials, processes, methodologies, reports, plans and other works provided to the consultancy firm/group or developed by the consultancy firm/group on behalf of Right To Play remain the property of Right To Play;
· All data must be stored in a safe and secure location, allowing full access to Right To Play staff during the evaluation process;
· Upon completion of the research, all raw data must be submitted to Right To Play.
3.0 ROLES & RESPONSIBILITIES (SUMMARY)
The consultancy firm/group will report directly to the Director, Monitoring, Evaluation and Learning at Right To Play International (with copy to the GREAT Monitoring, Evaluation and Learning Manager), and all deliverables should be submitted to her according to the schedule outlined in the “Key Deliverables and Timeline” section below.
The consultancy firm/group’s roles and responsibilities include, but are not limited to, the following:
· Background research to familiarize themselves with the GREAT program and Right To Play’s approach (i.e. program documents provided by Right To Play);
· Revision of the existing baseline evaluation framework design in collaboration with the Right To Play International Steering Committee;
· Review, refinement and/or development of instruments for midterm needs;
· Remote planning and monitoring on data collection in Ghana, Rwanda and Mozambique (Field protocols, tools, mobile data collection systems, etc)
· Design of remote fieldwork administration and logistics plan;
· Guidance/responsibility for Right To Play Ghana, Rwanda and Mozambique on management of local data collection to be undertaken by a firm to be subcontracted by Right To Play Offices;
· Development and implementation of a remote fieldwork/data collection administration and remote logistics plan in the three countries of implementation;
· Data cleaning, and analysis;
· Guidance of validation workshop and inclusion of data validation and interpretation through internal review of findings;
· Completion of final reports incorporating stakeholder´s feedback;
· Preparation and presentation of findings and recommendations for the final multi-country integrated quality report.
4.0 KEY DELIVERABLES AND TIMELINE (NEGOTIABLE AND TO BE FINALIZED WITH RTP STAFF)
PLEASE VISIT OUR WEBSITE ( https://righttoplaydiag107.blob.core.windows.net/rtp-media/documents/RFP_-Gender_Responsive_Education_and_Transformation__GREAT-MidTerm-_July_2020.pdf )FOR DETAILS ON PROPOSED DELIVERABLES AND TIMELINES
How to apply
1.0 PROPOSAL APPLICATION SUBMISSION
Interested organizations are requested to submit proposals including the following documents:
- Cover letter including the consultant’s/firm’s suitability for the assignment and current contact information;
- CV including contact details of 2 referees. If the lead consultant is planning to recruit sub-contractors for conducting this assignment to support the data collection from various sites, data entry and dataset development, please state this in your application and provide details (cost, expertise, expected role of each consultant, etc.); **
- Detailed technical response to TOR, with specific focus addressing the purpose and objectives of the assignment, methodology to be used and key selection criteria (max. 2 pages);
- Detailed budget breakdown based on expected daily rates and initial work plan;
- Initial draft of the proposed work plan in Gantt chart style.
- Two writing samples, ideally reports the firm/organization/group has lead authorship on
- The Proposal must be submitted no later than Monday 10th August , 2020 to Patricia Oliveira, Monitoring, Evaluation and Learning Manager at: [email protected],
- To help us track our procurement effort, please indicate in your email where (vacanciesincanada.ca) you saw this tender/procurement notice.
Proposals will be accepted on a rolling basis and will be reviewed as soon as they are received. Early submissions are encouraged and Right To Play reserves the right to select a consultancy before the proposal submission date noted above.
While we thank all applicants for their interest, only those selected for interviews will be contacted.
Right To Play is a child-centered organization. Our recruitment and selection procedures reflect our commitment to the safety and protection of children in our programs.
To learn more about how we are and what we do, please visit our website at www.righttoplay.com.
- Address 18 KING ST EAST, 14TH FLOOR TORONTO, ONTARIO $ Month