ToR Consulting Services for An Evidence Gap Map and Systematic Review of Energy Sector Intervention in Developing countries

Procurement
Closing date
13 Feb 2023

I. Background 

Sustainable Energy for All (SEforALL) is an international organization working with leaders in government, the private sector and civil society to drive further, faster action toward the achievement of Sustainable Development Goal 7 (SDG7), which calls for universal access to sustainable energy by 2030, and the Paris Agreement, which calls for reducing greenhouse gas emissions to limit climate warming to well below 2° Celsius.

Achieving these goals will require a radical rethink of the way we produce, distribute, and consume energy. SEforALL is at the heart of this foundational shift to ensure no one is left behind. Drawing on data and evidence, we identify the critical path to success in achieving SDG7. The eighth UN Secretary-General Ban Ki-moon launched Sustainable Energy for All as a UN initiative in 2011. The initiative was instrumental in ensuring energy was at the heart of the 17 Sustainable Development Goals (SDGs) that were adopted by UN members in 2015. 

The ambitions of SDG7 are extraordinary. Aiming to achieve them in the context of the Paris Agreement on climate change involves transformation at a scale never undertaken before. Swift action must be taken by leaders in governments, private sector companies, institutions, financiers, development banks, unions, communities, entrepreneurs, and civil society. As we enter the final decade to achieve SDG7, SEforALL has strategically chosen to strengthen global agenda-setting while expanding its activities to an engagement model that prioritizes data-driven decision-making, partnerships with high-impact countries and implementation on the ground. The current SEforALL three-year business plan (2021-2023) outlines “Results Offers” that scope out our planned interventions and demonstrate to our funding partners the impact of their support. 

In 2016, SEforALL was established as a Quasi-International Organization (QIO) under Austrian law. The organization currently employs approximately 95+ full-time professionals and now operates out of its headquarters in Vienna and satellite offices in New York City and Washington DC USA; it also has a country office in Abuja, Nigeria. Additional team members are based in Freetown, Sierra Leone; Accra, Ghana; Nairobi, Kenya; and Jakarta, Indonesia, with an expectation of more country-based assignments to occur in the future. 

As an independent organization, SEforALL maintains close links with the UN, including through a ten-year (2016-2026) relationship agreement with the Executive Office of the Secretary-General, partnerships with several UN agencies and through its CEO’s appointment as the UN Secretary-General's Special Representative (SRSG) for Sustainable Energy for All and as Co-Chair of UN-Energy. The SRSG role includes advising the UN Secretary-General and his staff on issues relating to sustainable energy and the implementation of SDG7 and supporting the coordination of sustainable energy issues in the United Nations System.   

II. Project Objectives and Goal 

The overall aim of the evidence gap map and systematic review will be to enable decision-makers and practitioners to explore the findings and quality of the existing evidence on the impacts of diverse energy sector interventions and to ultimately facilitate evidence-based decision-making with respect to the design and implementation of energy sector programmes in the future. The final output may also inform further work areas by SEforALL that we may want to consider in the next business planning cycle (2024-2027).

This consultancy assignment is supported by the Monitoring, Evaluation and Learning (MEL) unit of the SEforALL. MEL is requesting applications for a consulting firm to develop an energy sector intervention evidence gap and systematic review focusing on developing countries.  

One challenge in taking stock of evidence in energy sector interventions in developing countries is the absence of a causal relationship between energy interventions and overall results (outcomes or impact). Overall impacts can arguably be diverse but for the evidence map, we are specifically interested in interventions that deal with energy sector intervention including energy access and energy transition programmes and interventions and their causal impact on various possible result areas including access to energy, change in energy mix, wellbeing, health, education, and greenhouse gas emissions including gender disaggregation when possible. For the purposes of this review, we are looking for a consulting firm that understands EGMs and high-quality studies that include counterfactual identification designs and is able to develop the EGM and undertake a thorough synthesis of the key findings. The consultant will also be expected to incorporate to a limited extent the institutional structure of the organizations that are implementing the energy sector interventions, the local context of where these programmes are being implemented and also incorporate process evaluations to provide additional context for implementation fidelity-related topics.

 The consultant is expected to undertake three specific activities. 

  1. Develop an energy sector intervention evidence gap review by populating the gap map framework with studies of energy sector interventions and their outcomes/impacts in developing countries
  2. Tailor the framework such that it is relevant to the SEforALL institutional perspective while also incorporating the institutional, local and process-related contexts in a limited way (as mentioned above);
  3. Supplement the EGM with a thorough and detailed systematic review of the different studies. The consultant will also provide an annotated bibliography with keywords, an assessment of quality, and recommendations for possible future meta-analyses. 

The gap map accompanying report will address the following common key questions: 

  • What is the existing evidence that takes stock of energy sector interventions in the context of SDG7 and climate change mitigation and adaptation in alignment with the Paris Agreement? 
  • How do these attributes differ by scale and levels of implementation? 
  • To what extent has organizational context, made a difference, including implementation processes?
  • Given these inputs, what type of lessons can be drawn from the emerging evidence gap map for future meta-analyses? 
  • How does SEforALL’s role in energy sector interventions compare with the mandates of different agencies in the context of global policy? 

 In this respect, the consultant will build an evidence gap map through the following steps:

  • Become familiarized with the categories of energy sector interventions and outcomes/impacts for developing countries, including possibly developing a theory of change if assessed to be feasible and relevant;
  • Develop a systematic review protocol and a search protocol; o Using an evidence gap map framework, build a comprehensive protocol by mapping the population, interventions, comparison and outcomes and study designs (PICOS) of studies focused on measuring the impacts of energy sector interventions;  
    • Screen, review and code studies, including some relevant grey literature including those on institutional local and process contexts to a limited extent, as useful. 
    • Develop inclusion and exclusion criteria;
    • Update the search protocol for an extensive and exhaustive search of the literature and indicate the databases where this will take place
  • Undertake a comprehensive review of the literature for primary studies and systematic reviews of relevant studies in developing countries;
  • Populate the evidence gap map framework based on theory-driven intervention and outcome/impact criteria and sub-criteria;
  • Review existing strategic documents of key international organisations (including UN-Energy, IRENA, WB, IEA, ESMAP, WHO, GOGLA, Clean Cooking Alliance, Africa Mini-Grid Developers etc.) on energy sector programmes in the context of SDG7 and climate change mitigation and adaptation (e.g, GCF, AF, CIF);
  • Deliver a high-quality, well-written, well-thought-out evidence gap and systematic review report, policy brief and detailed PowerPoint slide deck;
  • Provide a forward-looking review of possible future meta-analyses. 

III.  Deliverables and Timelines  

The scope of this task should be completed within five (5) months, preferably earlier, from the date of contract signing which is tentatively planned for March 2023. The deliverables for this EGM and systematic review are structured in three stages: Inception and Planning Stage (March – April 2023), data collection, cleaning and analysis (May - June 2023) and delivering the final report (July 2023). The scope of this review may be modified in discussion with the selected consultant.

Inception and planning: This phase include starting the engagement with SEforALL MEL and its stakeholders by the selected team to complete the first draft of the Approach/Inception Paper, receiving feedback on this document and finalizing the approach. The main deliverable of the inception and planning phase will be a draft Approach/Inception Paper that includes Inclusion and exclusion criteria, along with a search protocol developed in conjunction with the MEL team; a database search based on criteria and protocol. The due date for this deliverable is end of March 2023

Data collection and analysis: The selected team will finalize studies to be included through a title search, abstract search and full-text screening. The studies are analysed and synthesized in a thorough and rigorous manner. There are two main deliverables under this phase:  

  1. The selected team submit mid-term report including completed data collection and analysis. The due date for this deliverable is May 2023 
  2. Delivery of the zero-draft EGM and systematic review report – including all Annexes – except for the final Conclusions, Recommendations, and Executive Summary. The due date for this deliverable will be June 2023 

Reporting: The final report will be completed in this phase. It will be shared, socialized, and reviewed for quality. The selected team is required to keep track of the feedback in a systematic and logical format. The socialisation process will include the continuous sharing of lessons with SEforALL, followed by the finalisation of the draft, a presentation slide deck, and the sharing and wider use and utilization of the report. The main deliverable of this phase includes analysis of all remaining data collected during the assignment, the final systematic review report after addressing all comments, a 4-pager learning brief and a detailed PowerPoint presentation. The due date for this deliverable will be on or before July 2023 

Key Deliverables and Tentative Timeline   

Description  

Timeline

Kick-off meeting between SEforALL and the consulting firm  

1 March 2023

Submit work plan and inception report to for review and approval by SEforALL  

 

31 March 2023

Submit mid-term report including data collection and analysis to SEforALL for review. 

15 May 2023

Submit zero-draft EGM and systematic review report – including all Annexes – except for the final Conclusions,

Recommendations, and Executive Summary

 

15 June 2023

Incorporate Observations and Comments of SEforALL 

1 July 2023

Submit final report including Executive Summary and Recommendations; 

 15 July 2023

Develop supplementary communications materials highlighting key messages and conclusions including PowerPoint presentation as well as a 4-pager summary brief 

31 July 2023

IV.  Approach, Travel, and Reporting

Approach: The consulting firm will work for the successful completion of the engagement, including all its deliverables, as outlined in the scope of work above in close coordination and continuous exchange with the SEforALL team. This will be ensured by organising weekly meetings with the consulting firm. The selected consulting firm will maintain close contact with a designated SEforALL focal point throughout the assignment who will facilitate access to documentation, relevant SEforALL staff and other partners as needed. 

The consulting firm must make a concerted effort to mainstream gender considerations throughout this assignment. If possible, the firm is also expected to implement at least a 40:60 female to male ratio in their team and to submit a copy of any gender policies they follow.

The final product of this project will be jointly owned by SEforALL and the consulting firm. SEforALL MEL will provide overall leadership and technical vision. The consulting firm will be hired under SEforALL procurement procedures and the study will be co-authored with the task managers from MEL and consulting firm who will continue to provide conceptual and technical guidance.

Travel: There is no expectation of travel for the assignment. 

Reporting: The selected consulting firm shall report to SEforALL’s MEL Programme Manager for this assignment, with additional input and consultations with the SEforALL technical experts. 

V.  Expertise

A qualified team of maximum three members (one lead consultant and two supporting consultants) requires the following attributes to undertake this evidence gap and systematic review successfully:

Lead Consultant Requirements:  

  • Master’s degree or its equivalent, or higher, in economics, statistics, evaluation, public policy, or related disciplines. At least 10 years of experience in energy sector, climate change or evaluation in general.
  • Extensive, demonstrated experience in evidence gap maps, and inclusion and exclusion criteria, meta-analyses methods, and counterfactual and as-if counterfactual methods.
  • Demonstrated experience with systematic reviews and meta-analyses methods.
  • Demonstrated expertise and linkages with multi-sector evidence, especially in the areas of energy, climate, health, poverty and environment.
  • Extensive, demonstrated experience working in developing countries.
  • Demonstrated experience in database search and access to relevant databases. High level of proficiency in statistical analysis and data management tools. Advanced use of statistical or spreadsheet software required. Familiarity with econometric and database software a plus. 
  • Strong analytical skills. Track record of well-regarded publications such as reports, reviews, academic articles on the topics of evaluation of development activities and/or energy projects.
  • Excellent written and verbal communication skills in English.

Supporting Consultant(s) Requirements:  

  • Master’s degree or its equivalent, or higher, in economics, statistics, evaluation, public policy, or related disciplines. At least 5 years of experience in energy sector, climate change or evaluation in general.
  • Demonstrated experience in evidence gap maps, and inclusion and exclusion criteria, meta-analyses methods, and counterfactual and as-if counterfactual methods.
  • Demonstrated experience with systematic reviews and meta-analyses methods.
  • Demonstrated expertise and linkages with multi-sector evidence, especially in the areas of energy, climate, health, poverty and environment.
  • Demonstrated experience in database search and access to relevant databases. High level of proficiency in statistical analysis and data management tools. Advanced use of statistical or spreadsheet software required. Familiarity with econometric and database software a plus. 
  • Extensive, demonstrated experience working in developing countries.
  • Strong analytical skills. Track record of well-regarded publications such as reports, reviews, academic articles on the topics of evaluation of development activities and/or energy projects.
  • Excellent written and verbal communication skills in English.

Consulting firm requirements: 

  • Minimum seven years’ experience in delivering comprehensive evidence gap maps or systematic reviews of the type specified in the ToR, with a special emphasis on the energy and climate change sectors
  • Extensive, demonstrated experience working in developing countries.
  • Teams should have qualified database search specialists and access to relevant databases;
  • Teams should be able to commit that they will be able to produce a highly credible, well-written evidence gap and systematic review report in the period requested. The report will be jointly owned by incoming consulting firm and SEforALL MEL unit.
  • The organisation must legally be able to provide business services within the European Union.  All personnel must have a high-level command of the English language. 

The assessment of the team will be based on the criteria below. Following this, the team is welcome to support their proposal with evidence of evaluations that are relevant to this assignment. 

Table 1. Technical assessment criteria (TAC) for EGM and systematic review 

 

Criteria

Subscore

Score

 

1

Technical

 

20

 

 

Expertise and experience of the organisation in providing strong, continuous and dedicated support for the evidence gap map and systematic review, based on the present Terms

of Reference, with specific emphasis on learning, organizational reviews and policies, through innovative approaches and a focus on arriving at strategic lessons, conclusions and recommendations for informing the SEforALL future in a formative way. 

20

 

Minimum seven years’ experience in delivering comprehensive evidence gap maps and systematic reviews of the type specified in the ToR, with a special emphasis on the energy and climate change sectors (10); Teams should have qualified database search specialists and access to relevant databases (4);  Team experience of working in international settings demonstrated in CVs (3);

Team experience working in developing countries demonstrated in CVs (3)

2

Methodology and Workplan

 

30

 

  

The scope, magnitude, urgency and challenges of the overall task are fully understood and reflected in the technical proposal, which broadens the ToR in important aspects if necessary.

15

 

The technical proposal demonstrates an elaborated version of the ToR with a nuanced understanding of the SEforALL energy sector interventions, the review scope and objectives including references to other global energy investments and research syntheses as well as potential risks

(10)

  

The technical proposal, tools and work plan are well defined and are relevant and correspond to extending the ToR, in particular in terms of measuring causal changes, and strategic thinking. 

15

 

The proposal has articulated a normative standard and analytical framework for the EGM and systematic review (5)

The team has critically commented on the ToR showing a

sophisticated understanding of SEforALL requirements (5)

3

Personnel

 

30

 

  

A proven Team Leader with the capacity to directly, energetically and creatively lead and organize the process under tight time constraints and manage teams and reviews. This includes extensive EGM and systematic review experience, academic and implementation experiences, capacity, time availability, and demonstrable ability to motivate all members that are proposed as part of the team

12.5

 

Experience of the team leader in the energy sector and climate change within an international

setting demonstrated within CVs

(5)

Peer-reviewed publications by the team leader on the energy sector and climate change and

development in general (7.5)

 

Demonstrated excellent communication skills, ability to work to tight deadlines and history of timely delivery of practical, strategic documents that add value

5

 

Team members have worked with diverse institutions in a diverse environment (2.5)

Gender Balanced Team and a

sound Gender Policy (2.5)

 

Demonstrated relevant experience of team members and support structure:

  • The proposed team is appropriately composed, including the expertise and experience of key team members, the structure of the team, as reflected in gender balance, knowledge of the energy sector, good knowledge of complexity, the ability to work across languages, and relevant academic qualifications
  • Demonstrated ability of key staff to work and analyse quantitative and qualitative data by using wellrecognized methods, especially as related to statistical methods, meta-analysis, synthesis, theories of change, logic models, while recognizing biases and behavioral science methods  

12.5

 

Data analysis skills of team members particularly in quantitative methods applying econometric

approaches shown in CVs (4)

Peer-reviewed publications of team members on the energy sector and climate change with a focus on

innovation in methods and analysis

(4)

Energy sector understanding and ability to think innovatively for the energy and climate change sector as shown in CVs (4.5). 

  

TOTAL

  

80

 

VI.  Bidding Process

Proposals can be submitted using your own format; however, the proposal must include the following:

  • Technical Proposal:
    • Cover page, including the organisation’s name, address, and contact information.
    • Brief background about your organisation as well as details of relevant work experiences. 
    • Portfolio of similar projects and works that are similar to the elements within the scope of work for this engagement.
    • The consulting firm’s understanding of the assignment, including any proposed changes to the scope of work, timeline and engagement approach. Identification of key risks, and potential mitigants, should be outlined.
    • A project plan that includes planning of activities, needs (meeting schedules, documentation, etc.), approach to progress tracking and reporting as well as a Gantt Chart with clear timelines that adhere to the duration that has been stipulated in the Terms of Reference (i.e., five months). 
    • Three (3) organizational references from the last three years
    • Key personnel CVs of a gender-balanced team, with CVs preferably no more than two pages each.
    • Gender Policy.
  • A Financial Proposal, outlining the lump sum costs associated with carrying out the scope of work, including:
    • Labour costs (personnel, daily rates, number of days)
    • Other costs
    • All costs must be in USD 

 Evaluation, Conflict of Interest, and Terms of Payment

  • Bidders are requested to disclose and explain any potential conflicts of interest 
  • Terms of payment at SEforALL follow a deliverable based framework. The breakdown of the milestone payments will be prescribed in consultation with the selected firm.
  • The evaluation of proposals will be based on an 80 – 20 split for technical proposal and financial proposal, respectively. The technical assessment is divided between Comprehension of the Project, Technical Capacity, Experience, and Personnel (team), as well as Approach and Methodology. 

Pre Bid Call

SEforALL is hosting a 1-hour long Pre-bid call on February 6, 2023 11:00 AM Vienna. You can register in advance for this meeting using this link:

https://seforall.zoom.us/meeting/register/tZUscOCqqjojE9KGjh_3_JTAmByPk41fVm2z
 
After registering, you will receive a confirmation email containing information about joining the meeting.

How to Apply 

Please submit your proposal by 13 February 2023, 17:00 Central European Time (CET) to  procurement@seforall.org  

Any questions, should be submitted in written format directly to procurement@seforall.org