Template_Project Completion Review



Project Completion ReportReview Date:October 2019Title: Pollution Management and Environmental Health (PMEH)Programme Code: n/aStart Date:30th November 2015End Date:31st October 2019If programme was closed >1 month early please describe in a few words why.BEIS funding could not be effectively spent within the scope of the business case Summary of Programme Performance Year2016201720182019BEIS Programme ScoreABBBBEIS Risk RatingModerateMajorMajorSevereFinancial PositionOriginal programme value BEIS ICF contribution: ?12.5mTotal UK contribution: ?20.1mExtensions/ amendmentsRecommended for early closure.BEIS logframe revisions Original logframe: 2015/16Revision 1: Dec 2017: minor language changes, milestone slightly revised for Impact Indicator 1.1, baselines revised in output 1.3 and 3.2. Revision 2: April 2018: interim logframe accounted for misalignment of PMEH with the original business case. Indicators from this version will be used throughout this document. Total BEIS programme spend?1.65m Follow up actions required following closureDFID and BEIS have already communicated their intention to withdraw from the PMEH programme at the last Steering Committee meeting and have notified the PMEH Secretariat and the other PMEH donors with an official letter dated 08 July 2019. World Bank regulations require donors to provide a minimum of three months’ notice to withdraw from any World Bank-managed Trust Fund. Following discussions with the PMEH Secretariat and other donors, it was agreed that DFID and BEIS’s withdrawal date from the PMEH Trust Fund will be 31 October 2019, in order to minimise disruption to the programme. This will also enable the PMEH Secretariat to provide the final financial reports covering the month of October 2019. On receipt of this financial report, a Promissory Note to the value of ?10.85m is to be cancelled by the World Bank. DFID will use the financial report to make a final determination on whether to claim back the DFID pro-rata share of any residual funds to be spent on programme activities by 31 December 2019.A. Introduction and ContextDevTracker Link to Business Case: Business CaseDevTracker Link to Log frame: Logframe 2017 Outline of programme and what it has achievedThe Department of Energy and Climate Change (DECC) (now the Department for Business, Energy & Industrial Strategy (BEIS)), committed ?12.5m Capital Departmental Expenditure Limit (CDEL) funding to the World Bank’s Pollution Management and Environmental Health (PMEH) programme in November 2015. The Department for International Development (DFID) provided a further ?7.6m Resource Departmental Expenditure Limit (RDEL) contribution. Norway (NORAD) and Germany (BMU) also donated to the PMEH programme, bringing the total Fund size to USD $47.8m. The PMEH?programme?originally had?the following components:Pollution abatement and remediation activities in the air, on land and in the marine environment respectively (although management of the latter has since been transferred to the PROBLUE Trust Fund). ?A component funded solely by DFID, which was research-based and focused on generating new academic and scientific knowledge, as well as researching and developing?technology-based solutions to tackle and reduce pollution levels in developing countries.?Communications support for the portfolio.?BEIS’s contribution?was provided to?principally?fund activities under Component 1 by purchasing full-scale air quality monitoring (AQM) equipment to monitor particulate matter, greenhouse gases (GHGs) and short-lived climate pollutants (SLCPs). This equipment would produce evidence for policymakers to use in climate mitigation efforts, and to mobilise finance from other sources. Through a global awareness programme, the data could also raise public awareness of the scale of air pollution, and the linkages between air pollution and climate change. During programme delivery, a number of challenges were encountered. Limited monitoring, oversight and forward planning in the early stages of delivery meant that procurement windows were missed, protracting the purchase of equipment. In addition, further scoping exercises identified that most target countries required technical assistance, rather than equipment. Although the Secretariat was restructured and processes strengthened, the resulting proposals to redirect BEIS funding represented a deviation from the original business case for BEIS’s contribution.BEIS funding was committed through a Promissory Note, of which ?1.65m was encashed by the World Bank in December 2015. An encashment schedule was provided in the Administration Arrangement, with the final balance expected to be drawn down in March 2018. Given the lack of opportunities to utilise BEIS funding, no further encashments were made by the Bank. At project closure, no further funding (beyond the initial ?1.65m) has been encashed. Following the identification of most target countries already having suitable AQM networks, DFID and BEIS worked closely with the PMEH Secretariat and other donors to agree a new work plan. However, limited reporting from the Secretariat and delays meant the programme fell behind schedule. Consequently, DFID and BEIS announced a joint intention to withdraw from the programme 15 months early.B: PERFORMANCE AND CONCLUSIONSOverall Outcome Assessment At programme inception, BEIS’s expected programme outcome was: “improved access to evidence of the level and scale of air pollution and their linkages to GHG emissions amongst policymakers and the general public in participating countries”. This was revised to two programme outcomes in the 2018 annual review after it became apparent that the original outcome could not be met through CDEL provision. The first revised outcome expected air quality management to be established and improved to strengthen mitigation efforts in participating countries, cities and regions (driven by equipment funded by PMEH). Secondly, awareness of the issue of pollution among policymakers and the public was expected to increase, and international networks would be strengthened. The PMEH Secretariat commissioned an independent analysis of the situation and needs of their target countries. It found the provision of a global knowledge hub to be duplicative, and so this was deprioritised. The analysis also helped them adapt to the needs of recipient countries, which resulted in a much stronger focus on technical assistance. As BEIS funding could only be spent on capital goods, steps were made to redirect BEIS funding to activities within the scope of the business case. However, it was not possible to identify suitable proposals for the majority of BEIS’s funding. Where there was an opportunity to improve existing AQM networks in target countries, missed procurement windows further delayed plans. The PMEH Secretariat and BEIS worked closely together to overcome these delays. For example, the PMEH Secretariat proposed demand-led programming and holding a call-for-proposals whereby PMEH could add air quality elements to existing World Bank projects. However, this shift in the direction of the Fund meant that it was unfeasible to spend the majority of BEIS’s contribution without significant revision to the business case. With the proposed programme activities now unaligned with the original BEIS objectives and logframe, BEIS worked with the PMEH Secretariat to identify activities that could be monitored while a new work plan for BEIS CDEL was agreed. This covered some of the successes of the programme’s technical assistance, which continues to be funded by Germany and Norway. An ‘interim working logframe’ was produced and used for the 2018 annual review. It will be used for the purposes of this report as discussions of a refreshed workplan effectively halted and so no further changes were agreed. Although this programming may deliver the revised outcomes over time, it will not result in capital spend for which the promissory note was laid. Discussions were held at the Steering Committee meeting in December 2018 and continued until June 2019 between donors to amend the Administration Arrangements to take these delays into account. A clear workplan for the programme’s extension was not produced. BEIS and DFID decided not to agree to the extension of the Administration Arrangements and instead proceeded with withdrawing from PMEH. This meets the BEIS definition of an ‘exceptional termination event’ as defined for more recently approved programmes in that a) the PMEH programme has failed to meet to a material extent its commitments under the Arrangements and b) it has persistently failed to deliver on the objectives of the programme. However, by June 2019, the World Bank had produced clear plans for the remaining BEIS portion of the encashed Promissory Note – this would support the procurement of capital goods in three countries (subject to contract). It was decided that CDEL funding for these procurements ($750,000) would remain within the Trust Fund and that only the uncashed, uncommitted portion (?10.85m) would be returned to BEIS. Lessons and how these have been sharedBEIS had few avenues for the capital funding to be used, which stalled progress in the programme as a whole. More rigorous, earlier testing of the PMEH Secretariat’s needs assessment may have uncovered this to some extent. For example, the Desert Research Institute was commissioned to scope country-by-country need for technical equipment and laboratory facilities. If this work had been carried out during the programme design stage, rather than during the programme implementation stage, the business case could have considered interventions in other countries or the provision of RDEL rather than CDEL. The business case recommended the use of a MDTF for its ability to coordinate need with donor finance. While it managed this well between the UK, Norway and Germany, the PMEH Secretariat could have used more levers in-country to map activities of other donors. Greater engagement with British Embassies and other donor countries may have also identified this. The PMEH Secretariat missed opportunities to add to the World Bank in-country team procurement plans during the first year of delivery. This issue could have been managed by working with them to ensure they were aligned with wider World Bank initiatives. The management of the Fund has since been strengthened.The original target countries were chosen to reflect the progressive range of activity on air pollution and could demonstrate the intervention across a range of contexts. However, this approach requires increased engagement to secure country buy-in and reduces the flexibility of the programme. Programme managers should consider whether demand-led intervention is better suited to achieving impacts.C: DETAILED OUTPUT SCORINGOutput Title Established or improved ambient air quality monitoring network.Output number per LF1Output Score ARisk ratingMajorImpact weighting (%)35Risk revised since last AR? NoImpact weighting % revised since last AR? NoIndicator(s)Milestones (2019) Progress 1.1 – # of projects collecting samples for laboratory analysis according to the standard methodology0 (2016)0 (2017)3 (2018)5 (2019)7 (2020) Achieved – six cities in China, India, Ghana, Egypt, Vietnam and South Africa are collecting samples for laboratory analysis according to standard methodologies.1.2 – # of local technicians who received on-site training to conduct continuous air monitoring, sampling and chemical analysis0 (2016) 0 (2017) 20 (2018) 30 (2019) 50 (2020)Achieved – 70 people trained in China, Nigeria, India and Vietnam, although it is not clear what training was provided in India. Key PointsThis output was introduced in the 2018 annual review. It replaced the following outputs as their milestones could not be achieved:The installation of air quality monitoring equipment The installation of laboratory equipment The development of a global knowledge hub Instead, indicators 1.1 and 1.2 focused on technical assistance activities to improve the use of existing monitoring and laboratory equipment in-country.The PMEH Secretariat reports that six cities have collected samples for analysis according to standard methodologies. In Ghana, PMEH is directly supporting analysis of samples collected in six sites in Accra – covering commercial and residential sources. In several countries, PMEH support is in earlier stages. In South Africa and Egypt, protocols and data-acquisition plans have been completed and will now inform standard operating/ quality management systems. Indicator 1.1 monitors samples collected for laboratory analysis, regardless of whether the equipment collecting the samples is supported by PMEH. In the case of Delhi, advanced air quality monitoring networks are already in place. PMEH recommendations have encouraged the Delhi Pollution Control Committee to enhance laboratory capacity. A protocol and methodology for chemical composition analysis has been developed. It is unclear when it will be applied. Overall, evidence for indicator 1.1 suggests that PMEH support for laboratory analysis using standard methodologies is underway but not yet well established in most target countries.Over the programme’s lifetime, 70 local technicians have been reported as trained in four countries (40 more since the 2018 annual review). Missions to the Redeemer University in Ede, Nigeria are enabling technicians to expand their research capacity to cover chemical composition analysis. Training, analysis and evaluations activities in China support the national reference laboratory in Xian, the municipal research laboratory in Shijiazhuang province, and other laboratories managed by the Hebei Research Academy for Environmental Sciences. In Vietnam, PMEH are improving capacity of technicians at the Hanoi EPA laboratory, the Northern Centre for Environment Monitoring laboratory, and the Finnish Meteorological Institute (FMI) laboratory.Further training was conducted in India.The opportunity for this training to bring about transformational change is dependent on the depth of the training, the ability of attendees to enact change, and whether learning can be effectively disseminated. BEIS does not have information available to judge this. Summary of responses to issues raised in previous annual reviews Updated work plans for each country have since been produced by the Secretariat. Programming should continue to discuss SLCPs as well as PM2.5. This continues to be a recommendation, except for Ghana where SLCPs have been well supported (e.g. through the Desert Research Institute and University of Nevada’s research into gaps in SLCP monitoring).Recommendations for future programmesFuture programming could consider the idea of a ‘centre of excellence’ laboratory to showcase best practise.In Sub-Saharan Africa, off-grid solar could be considered as an energy source for monitoring equipment in lieu of reliable grid capacity, and to reduce the carbon footprint. Output indicators should be phrased to identify activities that are directly attributable to programme intervention.Output Title Completed analytical steps to improve understanding of the source structure and develop full-scale AQM plans which integrate health and climate considerations to identify cost-effective mitigation measures. Output number per LF2Output Score CRisk ratingSevereImpact weighting (%)35Risk revised since last AR? NoImpact weighting % revised since last AR? NoIndicator(s)? Milestones (2019)Progress??2.1?–?# of projects which have built or improved air pollutant emissions inventory through full or partial PMEH support.?0 (2016)?0 (2017)?2 (2018)?4 (2019)?6 (2020)??Not achieved – although work has started in some target countries, it has yet to be completed. In South Africa, it is reportedly being supported by German development agency GIZ. 2.2 – # of projects which have conducted source apportionment and modelling based on collected filter samples, to identify and quantify PM source contributions.?0 (2016)?0 (2017)?1 (2018)?4 (2019)?7?(2020)?Not achieved – work is underway but is largely in the early stages of implementation. 2.3 – # of projects which have conducted integrated analysis to identify cost-effective abatement options for air pollution reduction.?0 (2016)?0 (2017)?1 (2018)?2?(2019)?5?(2020)?Not achieved – analysis is not due to be completed in China until Spring 2020. Analysis in Nigeria is still in development. Although a cost-effectiveness study looked at transport pollution in Egypt, it utilised satellite data rather than samples. Key PointsThis output was added to the 2018 annual review to support BEIS in monitoring the wider programme while a workplan for the BEIS funding was identified. Technical assistance has been core to PMEH’s programming as there has been stronger demand from recipient countries. There has been no demonstrated progress on the emissions inventories in Ghana or Nigeria since the last annual review and limited evidence of any further progress in Egypt, India and South Africa. In China, a further capacity building workshop was held in June 2019. The 2018 annual report notes that an emissions inventory was due to be finalised in October 2019, although this is now expected in early 2020. Efforts to understand and model the different sources of air pollution are underway in Egypt, India, Ghana and China. While progress has been made, some analysis and modelling are yet to be completed.Initial rounds of analysis have been performed in China, although it’s not clear whether this included modelling.According to country updates from South Africa, this activity is currently at a stage of developing Terms of References (TORs) for recruiting technical services to support the activity and has not started yet.PMEH have partnered with FMI in Vietnam to analyse PM2.5 in the larger Hanoi Metropolitan Area. The delivery, placement and installation of PM2.5 samplers were scheduled for August 2019 to start the sampling and source-apportionment analysis. This equipment was not funded by BEIS. Six sites will sample PM10 in Accra through a one-year source apportionment analysis. PMEH is also supporting EPA Ghana to use black carbon monitors as part of this analysis due to be completed in November 2020.The World Bank has worked with the Egyptian Environmental Affairs Agency (EEAA) to complete an Analysis of air pollution sources, Policy Options Study, Cost of Environmental Degradation Study, and a Health Impact Assessment.A transport pollution analysis was conducted in Egypt using satellite data. It provided cost-benefit analysis for potential interventions. It should be noted that this analysis did not utilise air quality monitoring data. In China, the cost effectiveness of air quality monitoring will be analysed using the GAINS model which explores cost-effective emission control strategies. This is not due to be completed until Spring 2020. In Nigeria, PMEH plans to develop a report on cost-effective air quality management interventions in consultation with the Government and stakeholders. This will inform a policy roadmap and implementation plan.Summary of responses to issues raised in previous annual reviews Evidence shows that target countries are using, or plan to utilise, the GAINS model, which ensures that policy interventions on air pollution do not increase GHG emissions. This should be actively encouraged as an Official Development Assistance (ODA) intervention. Recommendations for future programmesPMEH held workshops with a delegation from China to support the development of the GAINS model and followed up with top-up training 7 months later. This is good practice and should be followed by other target countries as they move towards implementation of this activity. Output Title??Improved awareness (among policy makers and other stakeholders) of air pollution and related AQM issues (including linkages to climate and health) in participating countries.?Output number per LF?3?Output Score??ARisk ratingMajorImpact weighting (%)20%?Risk revised since last?AR???YesImpact weighting % revised since last AR????NoIndicator(s)? Milestones (2019)Progress??3.1 –?# of participants who have increased awareness?about AQM issues by attending the events organized by PMEH.??100 (2016)?200 (2017)?300 (2018)?800 (2019)?1000 (2020)?Exceeded – over 1000 people have attended air pollution and health events, including the National Clean Air Program conference and the WHO first Global conference on Air Pollution and Health.3.2?# of Bank publications and articles published in peer-reviewed journals which production was supported by PMEH programme and/or used information and data generated by the AQM component.?0 (2016)?1 (2017)?2 (2018)?3 (2019)?5 (2020)?Not achieved – two peer-reviewed articles have been published. PMEH have contributed to a further article but it has not been published. 3.3??#?of products?produced for public communication purposes.?3 (2016)?10 (2017)?15 (2018)?20?(2019)?25?(2020)?Exceeded – The World Bank reported 43 products last year. Since then, they’ve produced a series of technical guides, publications and a presentation. Key PointsIndicator 3.1 reflects a significant number of participants at events organised, or co-organised, by PMEH. 1260 people reportedly attended in the 2018 annual review, although it was unclear whether their awareness was raised. Events over the lifetime of the programme include:the National Clean Air Program conference and associated events (500 people participated from 20 states) The WHO first Global conference on Air Pollution and Health (900 participants from more than 100 countries) 300 that attended the launch of the Accra AQM plan in 2018 and a further 119 attended a stakeholder workshop in Ghana. Workshops in Egypt to build awareness about AQM issues led to a formal request for an investment project loan on air pollution.It is unclear how many invitees were members of the general public. As PMEH were involved in the organisation of these events and they include climate, air pollution and/or health impact information, they have been included in the indicator. PMEH experts have been guest editors in Atmospheric Environment, which included one article on improving measurements in low- and middle-income countries. Another article for the Journal is due to be published on satellite measurements and ground level monitoring. A further article has been published in the Environmental Research Journal which considers the health impacts of ambient air pollution. Two working papers have also been produced and published for the World Bank’s Policy Research Working Papers Series. They both review whether pollution hinders urban competitiveness. It should be noted that this research is funded by DFID’s research component of the programme and cannot be attributed to BEIS. PMEH had already surpassed the expected milestone for indicator 3.1 last year. Examples of public engagement include media articles that mention PMEH, presentations in Ghana and China on toxic sites, and presentations at the World Bank. However, in the last year, evidence supplied is much more technical. 10 technical guides for China were produced and 4 articles including ‘the effect of particulate air pollution shocks on acute respiratory diseases in Egypt’. These are is unlikely to engage lay audiences.Summary of responses to issues raised in previous annual reviews PMEH has become more proactive in its communications since the last annual review. However, a stronger focus should be given to products that support the general public’s better understanding of air pollution and why it should be managed. Recommendations for future programmesConsider how improved communications will deliver the intended outcome and how this can be reflected in the output indicators. What type of communication, which audience, what key messages do you need to get across? Consider how the impact of these communications can be monitored. Attendance at events does not necessarily equate to improved knowledge. Feedback on the level of information or how it can be used by the attendee could provide more robust evidence. More specific language in the logframe could help to capture this during the annual review process.Output Title??Improved collaboration between the Bank and participating countries as well as among participating countries on AQM issues.?Output number per LF?4?Output Score??ARisk rating MinorImpact weighting (%)10%?Risk revised since last?AR???NoImpact weighting % revised since last AR???NoIndicator(s)? Milestones (2019)Progress??4.1?–?#?of technical workshops on AQM, hosted (or co-hosted) by the participating country (or countries), inviting other participating countries.?0 (2016)?1 (2017)?2 (2018)?3 (2019)?4 (2020)?Achieved – There have been two exchanges between China and Nigeria. There was also a University of Iowa and Chinese delegation to India. 4.2?–?#?of people who have taken the PMEH e-learning course on AQM.?0 (2016)?0 (2017)?100 (2018)?200?(2019)?300?(2020)?Exceeded – the World Bank reports that 385 people have taken the course. Key PointsIndicator 4.1 was met ahead of schedule. PMEH has hosted other events that shared knowledge but did not count these towards the indicator as they were hosted by developed countries. In 2017, supported by the PMEH programme, a senior Chinese delegation of air pollution control officials visited Austria, Germany, and the Netherlands. The delegation learnt how the iron and steel sector achieved stringent emission standards through technology investment and management upgrades in Germany. In September 2018, delegates from the China Research Academy of Environmental Sciences visited the International Institute for Applied Systems Analysis in Austria. The team consisted of experts on modelling, inventory, scenario analysis, coding, and policy development. The training focused on hardware and software installation and testing, and a practical introduction to the application of the GAINS model.PMEH also supported a study tour to Rome in May 2018 where high-level Lagos state government officials actively participated in technical discussions on institutional and technical practices on AQM, health surveillance and impact assessment.The PMEH programme launched an e-learning course on AQM in April 2018. This course is open to the public, and its objective is to introduce key concepts to air quality management. The target audience is government environment officials, World Bank staff and policy makers. The 2018 annual review noted that 105 were registered to take the course, although the World Bank were not able to provide data on how many of them had completed the modules. This year, the World Bank report that 385 people have taken the course, although no other data was supplied. Summary of responses to issues raised in previous annual reviewsIt is still recommended that more South-South learning occurs, incorporating lessons learnt from the visits to China and Nigeria. PMEH flagged the e-learning platform in the 2017 annual report. Further promotion of the e-learning platform within the Bank, and with counterparts in target countries, could support them to better integrate air quality considerations into their work. Recommendations for future programmesConsider how online resources can be best disseminated, perhaps through links on social media or incorporated into a wider training programme.Access to e-learning monitoring information will help to analyse how effective it is. This data could indicate which modules were most viewed or provide qualitative information through a participant feedback process.D: VALUE FOR MONEY & FINANCIAL PERFORMANCE Assess value for money compared to the proposition in the Business Case, based on the past year and during the lifetime of the programme The programme deviated significantly from the proposition agreed in the business case. BEIS’s investment was originally intended to enable: 1) the introduction, upgrade and enhancement of?particulate matter?(PM2.5)?monitoring in?selected?cities;2)??the introduction of?equipment for the?monitoring of additional GHGs and SLCPs;3)?monitoring data to be able to?construct?baselines across?this?range of pollutants/ emissions; 4)?monitoring?data to?assess population exposure,?assist the?validation?of?emissions?inventories and AQM?plans that?assess cost-effective policy for both?air?pollution?mitigation?and?GHG emissions reductions; 5) the use of this emissions data to raise public and policy awareness of?the scale of air pollution,?as well as?the linkages between?air?pollution and climate change,?and the available actions for mitigation?through a range of?transparent and?accessible dissemination?tools?and?a?global awareness programme.After the business case was agreed, the scope of the programme changed to reflect a higher need for technical assistance in-country, and to account for some early delivery challenges. With the programme now unaligned with the business case, BEIS worked closely with the Secretariat to revise the workplan. Ultimately, there was no high-quality, evidence-based proposals for refocusing the majority of BEIS funding within the timeframe. The programme was due to run until December 2020 – there was a discussion to extend this further in order to bridge procurement difficulties but there was not enough evidence that this would represent value-for-money. Instead, DFID and BEIS announced a joint intention to withdraw from the programme 15 months early.However, PMEH is in the process of procuring capital goods in Egypt, South Africa and Nigeria. Although the value-for-money for this equipment is somewhat reduced by the delayed procurement, it was still considered good value for money to continue with the investments while BEIS withdrew. In light of this, only the uncashed, uncommitted portion on BEIS funding will be returned. BEIS is confident that the remaining funding will support PMEH’s wider programming. PMEH is a co-mingled fund but an equivalent amount to BEIS’s contribution is being spent in the Greater Cairo Region, where they are in the process of procuring filter-sampling equipment and laboratory equipment to support chemical analysis. Lab supplies and consumables for 18 months of sample collection and analysis will be provided. This will result in a minimum of a 12-month continuous period of valid, high-quality data generation to aid analysis and inform a subsequent report. PMEH have provided a cost estimate for this equipment and a country update in July 2019 stated that procurement was due to complete Q3 2019.Where air quality monitoring equipment in target countries was already in place and not attributable to PMEH activities, GHG emissions reductions could not be attributed to UK ODA. The business case also reasoned that investing through PMEH would increase donor coordination, which in turn would increase value for money. In practice, PMEH did coordinate donor funding well within the Trust Fund (Norwegian, German and DFID funding directed at the research and technical assistance elements of the programme and BEIS intending to fund the capital investments). Further coordination with other donors and wider efforts in-country would have benefitted the programme. Quality of financial management during the programmeBEIS’s approved budget was ?12.5m, representing 62% of the UK’s contribution (BEIS and DFID) and 34% of the total Fund size. This was expected to be fully drawn down via the Promissory Note by March 2018. Expenditure and commitments under the project have remained extremely low, with only $750,000 due to be spent on assets at programme close. This low level of expenditure, coupled with the lack of an evidence-based needs assessment, resulted in low encashments by bined administrative and management costs totalled ?875,000 for the programme lifetime, or 7% of BEIS’s funding. This includes the 2% standard fee for membership of the MDTF and the 5% variable fee for managing and operating costs. This was a very competitive rate. The new standard fee at the Bank for MDTFs is 17%.World Bank Trust Fund regulations state that all funding from donors must be co-mingled, especially for programme wide or cross-cutting expenditure lines. However, the proportion of each donor’s allocation can be worked out using a pro-rata calculation system, which allows for apportionment of expenditure to each donor proportional to its financial contribution to the overall program budget. As an organization the wider World Bank has amply demonstrated, over a number of decades, its ability to deliver high quality programming. However, the lack of clarity around financial reporting in the PMEH program, particularly during early delivery, made it difficult to understand where funding had been committed and how it related to GHG emissions reductions. BEIS still considers the World Bank’s wider procurement, fiduciary risk assessment and management systems to deliver strong value for money, as well as providing a suitable level of protection and safeguards for BEIS’s investment. After the UK’s withdrawal, PMEH will continue its operations with a focus on a revised workplan that is more aligned with the priorities of the WB and the remaining donors.Date of last narrative financial report21/09/2019Date of last audited annual statement30/09/2016E: RISK Quality of risk management over the life of the programmeThe overall risk rating for the programme has moved from moderate in the first annual review to severe. This acknowledged no clear route through which to spend BEIS’s contribution. To date, BEIS managed this risk by seeking solutions through the Steering Committee and by working closely with the PMEH Secretariat, as agreed in the business case. This would have been further mitigated by more effective engagement between the Secretariat and in-country World Bank teams to scope procurement routes and identify needs. Other risks included a potential lack of ambition from recipient countries, which has been monitored by country plans provided by PMEH. Moving forwards, well-targeted technical assistance may reduce this risk by helping to build capacity to act. Again, the possibility of demand-led programming may also reduce this risk.BEIS has full confidence in the ability of the PMEH Secretariat to deliver the equipment funded by the $750,000 remaining in the Trust Fund. PMEH provided assurance in September 2019 that the procurement of goods in Egypt, South Africa and Nigeria is already underway. By withdrawing from the program, BEIS and DFID have relinquished direct oversight of the delivery and maintenance of these goods, which remains a risk. The UK’s withdrawal aimed to cause minimal disruption to the programme. The Secretariat provided evidence that BEIS funding would be spent on capital goods; clawing back the full contribution would have limited the programme’s pipeline and affected World Bank relationships with recipient countries.F: COMMERCIAL CONSIDERATIONS Delivery against planned timeframeThe timeframes set out in the business case have not been met. Although this has reduced value for money of BEIS’s investment, BEIS has worked closely with the PMEH Secretariat to improve oversight of remaining activities. Upon the UK’s withdrawal from the programme, the programme will focus on a revised workplan that is more aligned with the priorities of the World Bank and the remaining donors. The Desert Research Institute recommended that the Global Knowledge Hub workstream should be discontinued as similar initiatives already existed. Therefore, the objectives set out in the business case could not be met within the timeframe. Ultimately, BEIS decided not to extend the programme in order to reach out to more countries over a longer time period.Performance of partnershipBEIS’s relationship with the Secretariat improved in August 2018 by agreeing a Terms of Reference for the Steering Committee, which clarified the role of donors in decision-making. Changes in leadership also affected the flow of communication to donors. The Director and Senior Director for Environment at the World Bank made much-appreciated interventions to resolve donor concerns at key points and facilitate discussions about improving the partnership. Partnerships with the other donors were positive and collaborative throughout the programme and enabled us to bring common positions to official meetings with the PMEH Secretariat. Despite our decision to withdraw from the programme, positive relationships have been maintained with donors and with the World Bank more widely.Asset disposal and value obtained by BEIS Assets were expected to be purchased by, or ultimately be owned by, beneficiary countries. This recognised that effective interventions secure local ownership of assets and approaches. Although PMEH has been delayed in procuring equipment, they have now secured partnerships with three developing countries. In this instance, BEIS CDEL will be used to purchase the equipment by the Bank and then transferred to the local authority at the end of the contract.G: MONITORING & EVALUATION Evidence and evaluationThe assessments and conclusions made in this document were based exclusively on the review and assessment of verifiable evidence available at the time of the Project Completion Review. This includes a full report from the PMEH Secretariat on the 23rd September 2019, country progress updates from the PMEH Secretariat shared with BEIS in July 2019, as well as historical analyses, and interviews and calls with previous BEIS programme managers. As with previous reviews, DFID and BEIS are conducting interlinked reviews that are written in parallel but cover separate components of the PMEH programme. Where possible, findings from review of the evidence submitted for this review have been shared with DFID colleagues and common conclusions have been drawn.In the business case, BEIS’s investment included ?300,000 - ?400,000 for an independent evaluation. This was not considered value-for-money as there were no capital goods to monitor and so no funding was spent. Instead, the programme was evaluated through annual reviews, as per standard BEIS ODA processes.Monitoring progress throughout the programmeBEIS’s business case was approved in November 2015, with annual reviews in 2016, 2017 and 2018. Evidence for these documents was generated through country progress documents, publicly accessible information, and calls with GIZ, the German development delivery agency, who provided on-the-ground assessments of progress. The UK expected access to the Donor Portal Center, which would provide real-time financial information on the programme. BEIS could only access the Donor Portal Center after several technical problems were resolved by the PMEH Secretariat. While a useful tool, data from this source sometimes differed to data produced in Steering Committee meetings and during results collection, which caused some confusion for the donors. German counterparts are still unable to access it at the time of programme closure. The provision of more timely and detailed information may have improved if the Secretariat had been better resourced.Learning Lessons learned have been described throughout this report. In addition, some broader lessons learned are summarised below. These lessons will be shared with other BEIS and DFID ICF programme managers through a debrief exercise. This will consider the whole programme cycle, and how lessons can be incorporated into the design of new programmes. Programme design and alignment of goals between the World Bank and the donors are crucial for ensuring programme effectiveness. Design flaws, coupled with divergences between the goals and expectations set by the PMEH Secretariat and the PMEH donors, have had a negative impact on the overall programme.There should be clear protocols for interactions between donors and the PMEH Secretariat. The lack of clear protocols led to situations where BEIS and DFID representatives becoming involved in activities that would normally lie within the responsibility of the Bank (e.g. the review of Terms of Reference for consultancy services prepared by the World Bank’s task teams). In previous years, the lack of protocols also led to situations where the BEIS and DFID representatives attended meetings that breached the common rules of engagement and led to uncomfortable interactions and ineffective dynamics.All activities for future MDTFs should be clearly linked with World Bank products, especially development policy lending, investment project financing and Programs for Results. These should be made clear to all donors by the relevant World Bank programme team at the inception stage of the programme, in order to avoid these issues.Technical programmes, such as PMEH, should have clearly defined cross-cutting components and impact strategies. The design of the PMEH programme involved the delivery of operational activities to reduce air pollution on-the-ground, as well as the funding of analytical and research work and the development of technology-based solutions. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download