EUROPEAID/ 119860/C/SV/multi



SPECIFIC TERMS OF REFERENCE

End Term Review of the Sector Policy Support Programme

“Periodic Maintenance of Trunk, Main and District Roads”

financed by the 9th EDF in Zambia

FWC BENEFICIARIES 2009 - LOT 2: Transport and Infrastructures

EuropeAid/127054/C/SER/multi

1. BACKGROUND

1.1) General

The 9th European Development Fund (EDF) National Indicative Programme (NIP 2002-2007) signed by the Government of the Republic of Zambia (GRZ) and the European Union (EU) on 11th July 2002, reflects the EU’s willingness to support the Transport Sector in Zambia as a means to develop economic growth, regional and national integration and alleviate poverty and is the response to Zambia’s Poverty Reduction Strategy Paper (PRSP) of 2002 in which improvement to the road sector has been identified as the single most important means to address and combat poverty.

Within this framework, the focus of interventions is related to the (i) periodic maintenance of trunk, main and district (TMD) roads and (ii) the rehabilitation of trunk and feeder roads.

The EU interventions on periodic maintenance for TMD roads and the rehabilitation measures on rural feeder roads are implemented under the sector budget support (SBS) financing modality. This SBS is implemented under a sector approach through a sector policy support programme (SPSP) of a total of € 96M for the period 2005- 2009. This amount comprises € 88.5M for SBS (funds released directly to the Treasury) and € 6.13M for capacity building measures and supplies to be financed under classical EDF project funding.

The purpose of the SPSP EDF9 is to contribute to the establishment of a safe, reliable and sustainable core road network with an improved provision of basic access for social services and food security achieved by the end of ROADSIP II (Road Sector Investment Plan).

The main expected results are: (i) the elimination of backlog of periodic maintenance in all priority and essential roads by the end of ROADSIP II, together with increased capacity in the management of road maintenance and improved management of the road fund revenues in the National Road Fund Agency (NRFA) and (ii) the direct and significant improvement in accessibility and increased capacity of Road Development Agency (RDA) at provincial level and of District Councils (Road Authorities), to plan and manage their road networks.

It is important to note that Zambia has been the forerunner of sector budget support in the road transport sector in the framework of the EDF9. Therefore, this End Term Review (ETR) will provide essential information and lessons learnt to national sector stakeholders, Cooperating Partners (CPs), EU Delegation (EUD) and EU Headquarter Services.

Implementation of the SPSP EDF9 started at the end of 2005 and accompanied the sector reform progress mainly taking off in 2005 (based on Public Road Acts in 2002) with regard to the establishment, operation and functioning of three autonomous Road Agencies, the Road Development Agency (RDA) for the management of the entire public road network, the National Road Fund Agency (NRFA) for financing the sub-sector and the Road Traffic and Safety Agency (RTSA).

The SBS component of the SPSP in its original scope of 2005 comprised 3 tranches to be released against nine indicators measuring sector performance on network management, sector financing and institutional progress. The total amount was € 70M including the three tranches of a total of some € 63M for releases in 2005, 2006 and 2007. The remainder was programmed for capacity building measures and supplies to be financed under classical project funding as mentioned below:

1) Technical assistance (TA) programme to assist the 2005 newly established road agencies RDA and NRFA in fulfilling their roles and mandates.

2) Annual Road Condition Surveys (RCS) on the TMD road network for the period 2006-09.

3) International supplies of vehicles, IT equipment etc.

A first addendum to the Financing Agreement (FA) was signed in 2006 in response to the increasing need to support “basic access issues” to the rural poor increased the total amount of SBS by € 20M to € 90M and included a component on the rehabilitation of rural roads (mainly feeder roads). In this context, two additional tranches on top of the existing three for releases in 2008 and 2009 were included.

The additional and specific accompanying capacity building was designed to provide support to the rural authorities and provincial RDA offices for the management of the rural road network and also included a sub-component for the support of the local construction industry with focus on the small and medium sized enterprises.

The entire rural roads component was in coherence and complementarity to the EDF9 financed food security programme with regard to target districts and is designed similar to DANIDA and KfW funded interventions in other provinces under aspects of aid effectiveness and alignment of interventions to achieve a maximum of synergies for the beneficiaries.

A second addendum to the FA was signed in August 2007 and included a financial increase by another € 6M to a total of € 96M. The additional financing was to be used to support ROADSIP II in periodic maintenance interventions for 2008 on TMD roads. The latter will bridge the gap for support to such interventions until EDF10 sector budget support is envisaged to materialise.

A mid-term review including an assessment of the state of play in the seven key areas relevant for the application of SBS as financing modality was carried out in the 2nd quarter 2008.

1.2) Implementation

1.2.1) Disbursement Overview SBS-SPSP I

|TRANCHE and INDICATIVE release|Amount in Euro |Release in % |Date of Release|Comment |

|calendar | | | | |

|Tranche 1-fixed |25,400,000 |100 |Dec 2005 | |

|3rd qrt 05 | | | | |

|Tranche 2–Variable |21,000,000 |100 |Sep 2006 | |

|3rd qrt 06 | | | | |

|Tranche 3–Variable |21,400,000 |99 |May 2008 |Claim offset: deduction of 76,288.58 Euro -> total |

|3rd qrt 07 | | | |payment 21,323,711.42 |

|Tranche 4 | | | | |

|Fixed-1st qrt 08 |8,000,000 |100 |May 2008 |- 8 M Euro added by Addendum II to FA |

| | | | |- Released together with 3rd variable tranche |

|Variable-3rd qrt 08 |8,500,000 | |pending |- On hold since April 2009 |

|Tranche 5-Variable |4,200,000 | |pending |- On hold since September 2009 |

|3rd qrt 09 | | | | |

|Subtotal Tranches |88,500,000 | | | |

|Total FA |96,000,000 | | |Incl. FA Addenda I (20 M) and II (8 M) |

The disbursement of tranches is subject to a (i) positive assessment of the macro-economic framework; (ii) positive review of the Public Finance Management situation and (iii) satisfactory assessment by GRZ on a list of nine indicators (except for tranche 1).

These review and assessment are carried out jointly by the EUD and the NAO.

1.2.2) TA services

The TA component provides capacity building services to the RDA and NRFA in the broad field of road management and financial management.

Following an international restricted tender services, Roughton International Firm was contracted by the National Authorising Officer (NAO) in Zambia in September 2005 to provide a technical assistance team (TA) to RDA and NRFA. The assignment commenced in October 2005 and the total duration was of 4 year and 4 months (including Addendums 1 and 2, see below).

The purposes of this TA were (i) to increase the effectiveness and transparency of NRFA to manage the road fund and (ii) to increase the effectiveness of RDA to manage and maintain the road network.

The results to be achieved were the following:

1) To enhance the efficiency of the NRFA to manage the effective disbursement of the road sector funds by:

• Improved preparation of road maintenance budgets

• Improve reporting, monitoring and auditing of the road fund.

2) To enhance the efficiency of the RDA to implement road maintenance programmes by:

• Increased transparency in prioritization and planning of road maintenance programme

• Improved contract procedure and administration

• Improved quality of construction and maintenance standards.

The original TA team was composed by 5 long-term full-time and 2 short-term experts:

- TA I Planning - Team Leader (RDA) – 3 years

- TA II Implementation (RDA) – 2 years

- TA III Highway Management System (HMS) Specialist (RDA) – 2 years

- TA IV Social/Poverty Specialist (RDA) – 3 years

- TA V Financial Specialist (NRFA) – 3 years

- HMS System and Software Specialist (RDA) – 4 man/months

- RMMS Specialist (RDA) – 29 man/months

As mentioned previously, 2 contracts variations were done:

A) Addendum 1 (May 2007)

The purposes of this addendum to the TA were to:

(i) Help the RDA to ensure successful implementation of the District and Feeder road rehabilitation and repair programme in Central and North Western Provinces;

(ii) Identify, consult, recommend and assist in the implementation of capacity building in regional organisation and coordination to ensure the sustainability of local capacity in the public and private sectors for the management and maintenance of roads;

(iii) Support RDA in executing (i) annual workshops on the progress of the sector reform on rural roads management and the appointment of road authorities and (ii) programme area wide annual workshops on specific programme issues;

(iv) Assess socio-economic impacts of the programme interventions.

The assignments of TA I and IV were respectively extended by 6 and 4 months and 2 long-term full-time TA were added to the team:

- TA VI Coordinator Rural Road Programme (based in Central Province) – 31.5 months

- TA VII Rural Road Programme (based in North-Western Province) – 31.5 months

B) Addendum 2 (October 2007)

This addendum was to extend by 4 month the services of TA III HMS. The purposes of these additional services were to: (i) help the RDA to ensure successful implementation of the HMS and BMS systems; (ii) ensure future sustainability of the HMS and BMS Systems; (iii) finalise the first year’s Road Needs Report.

The Final report of these completed TA Services was presented during a stakeholders' workshop in May 2010 and was approved in August 2010. During this workshop, the stakeholders expressed the need to carry out an independent end-term review (ETR) of these TA services in order to draw lessons and to better formulate any further new capacity building programme for the sector.

This formulation of any new capacity building programme has also to take into account the findings of the recent (March 2010) financial, technical and procurement audit carried out by the Office of the Auditor General (AG) in Zambia of roads projects procured under national procedures from January 2006 to September 2009 which are managed by the RDA (see section 1.3).

1.2.3) Socio-economic assessment of EDF interventions under the SPSP

The above TA capacity building component provides specific support to the RDA on the matter. The socio-economic assessment aimed to measure the impact of road transport interventions on poverty reduction and includes district profiling, route based assessments starting with those interventions financed under the SPSP tranches (origin destination survey etc. and case studies).

1.2.4) Technical Audits

Two independent technical audits under EDF9 funding for the civil works under SBS were carried out in 2007 and 2008 to assess (i) the procurement process with regard the application of internationally accepted standards and transparency of procurement and (ii) quality of civil works.

In 2009, three experts were financed by the SPSP EDF9 to support the Office of the Auditor General (AG) in Zambia in carrying out the procurement audit of roads projects managed by the RDA.

1.2.5) Supplies

A range of supplies for IT equipment, vehicles etc. under EDF9 procedures has been carried out so far to support the operational capacity of the NRFA and RDA.

1.2.6) Financial Overview by 30/09/2010

Global commitment: € 96,000,000.00

Committed: € 95,085,642.28 (99%)

Paid: € 81,601,952.39 (85%)

1.2.7) Review of ROADSIP II and Transports Sector Policy (2002)

A mid-term review of ROADSIP II has been carried out in 2009. However, this document needs to be updated to make it "bankable" by taking into account

- the findings and recommendations of the AG audit and PAC's reports 2009 (see section 1.3),

- the road reclassification study of 2009

- and the last road condition survey report available.

This exercise has been during the first quarter 2011 and should be finalised in June 2011.

A review of the 2002 transport sector policy has started in June 2010 and the MCT has requested a support from SSATP to finalise this exercise.

1.2.8) Steering Function

Both project and sector budget support modalities are an integral part of the EU’s sector dialogue as lead donor in Zambia in support of ROADSIP II. The steering of the SPSP is mainly embedded in quarterly ROADSIP II committee meetings between CPs and key stakeholders of the sector in view of reducing transactions costs.

1.3) Sector context

Since the launch of the AG audit 2009, the EU has put a hold on its sector budget support. The findings of this audit can be assessed on the Auditor General's website (.zm) while observations and recommendations of the Public Accounts Committee (PAC) at the National Assembly website (.zm).

In response to the AG's audit 2009 and the PAC's report, the GRZ has started to implement in August 2010 short-term corrective actions. Medium and long-term ones have been started to be implemented since January 2011 which should allow the CPs envisaging to release new funds to support the sector.

Besides to the issues listed above in the road sector, a financing agreement has been signed in July 2009 between the GRZ and the EU for a new SPSP EDF10. The total amount is € 77M comprising € 69.3M for SBS and € 6.5M for capacity building measures (TA and trainings) under classical EDF project funding. For the capacity building component, Draft Terms of Reference (ToRs) are available since July 2009 but appear now outdated due to the needs to be addressed in the sector. In addition, planned initiatives from other CPs have also to be taken into account to ensure coordination and complementarity between all CPs' funded interventions.

Within the above context, the GRZ and the EU have jointly express the need to carry out an external end-term review of the SPSP EDF9 even if 2 tranches of the SBS are yet to be released.

The beneficiaries of this exercise are the National Authorising Officer (NAO) – Ministry of Finance and National Planning (MFNP) and other sector stakeholders such as the Ministry of Works and Supply (MWS), the Ministry of Communications and Transport (MCT), the RDA, the NRFA and the RTSA.

2. DESCRIPTION OF THE ASSIGNMENT

2.1) Global objective

The global objective of the study is to provide the Government of Zambia with an independent and reliable assessment about the overall performance of the SPSP EDF9 in supporting the sector programme ROADSIP II, paying particularly attention to the impact of the programme actions against its objectives.

2.2) Specific Objectives

The ETR of the SPSP EDF9 will provide the decision-makers in the GRZ, the EU Delegation to Zambia and Comesa (EUD) and the wider public with sufficient information to:

a. Whether the design of the SPSP and its components have been appropriate in response to the evolving sector programme ROADSIP;

b. Make an overall independent assessment about the past performance of the SPSP EDF9, paying particularly attention to the impact of the component actions against its objectives and the expected results;

c. Identify key lessons and to propose practical recommendations for follow-up actions;

d. Propose practical recommendations for reformulation and implementation of the SPSP EDF10 including the planned capacity building programme.

2.3) Requested services

The requested services respond to the requirements of the project cycle and the practical guide on EU Support to Sector Programmes.

The consultant shall verify, analyse and assess in detail the issues outlined in Annex 2. The list of issues is not intended to be exhaustive. The outline of the final report thus delivers the scope for the detailed assessment. The issues refer to the five evaluation criteria endorsed by the OECD-DAC (relevance, effectiveness, efficiency, sustainability and impact), and to the EU-specific evaluation criteria (EU added value and coherence).

The ETR of the SPSP EDF9 will coincide with the updating of ROADSIP II bankable document and probably with the institutional review study in the road sector led and financed by the WB and the review of the 2002 National Transport Policy, all led by MCT. Results of either a preliminary or final nature of the latter studies will have to be taken into account by the Consultant. A close consultation with the TA services for the latter is therefore required and also with any on-going TA in support to the road sector institutions. The results of the AG's audit 2009 and the PAC's report will have to be taken into account.

The study comprises the following services to be carried out:

2.3.1) End Term Review (ETR) of the SPSP EDF9 according to OECD-DAC criteria (relevance, efficiency, effectiveness, impact and sustainability of the SPSP in support of the ROADSIP II sector programme);

The consultant shall when relevant propose alternative or complementary set of evaluation questions justifying their relevance.

The consultant shall also evaluate the socio-economic poverty assessment methods applied. The consultants are requested to verify, analyse and assess the integration and impact of cross cutting issue (Gender and Good Governance in particular) in the project. The consultants are required to use their professional judgement and experience to review all relevant factors and to bring these to the attention of the GRZ and the EUD.

For methodological guidance the consultant will refer to the EuropeAid's Evaluation methodology website: and also to the Europeaid Aid Delivery Methods PCM - Project Approach Guidelines available in .

Methodological guidance for the evaluation of integration of cross-cutting issues (environmental sustainability, gender, good governance and human rights) may be found in the same weblink above.

For methodological guidance, the consultant will refer to the Guidelines to European Support to sector programmes available in , as well as to complementary source materials to be made available during the briefing session by the EUD. The mission will take due account of EC formats, rules and procedures.

2.3.2) Proposal of modifications of the FA SPSP EDF10 either by an addendum or an exchange of letter in response to the outcomes of 2.3.1 above incl. issues of capacity building, indicators, disbursement mechanism etc.

2.3.3) Proposal of revised capacity building programme component of the SPSP EDF10. The Consultant will have to reformulate a meaningful programme for the entire sector under a holistic approach, taking issues of Governance and tertiary university education in relation with Road Transport into consideration and also other planned intervention from other CPs like JICA to ensure complementarity and coordination. This formulation will have to be in line with principles of the EC backbone strategy for Technical Cooperation of March 2009 (see and relevant documentation). Part or full of this programme will be supported by the EUD up to the amount available under the 10th EDF SPSP II. The Consultant will propose ToRs for this EDF10 supported Capacity Building Programme. These ToRs will have to respect the EDF template available in .

2.3.4) Preparing and implementing a Stakeholders' Seminar (presentations, working dossiers) on the findings of the ETR and the proposed reformulation of the SPSP EDF10.

During the assignment in Zambia, there will be a need for field visits from the experts. These field visits shall mainly focus on the rural roads components and the management of the rural road network through Road Authorities. Missions are foreseen to the target provinces North-Western and Central where TA services were provided to the regional RDA offices and the Road Authorities (District Councils). Since the AG carried out a very comprehensive technical audit between August 2009 and March 10, there will be no specific need to re-visit civil works interventions unless particularly agreed upon between stakeholders, the Supervisor and the Contracting Authority during project meetings.

Since the SPSP ED9 FA has an emphasis on assessing road transport interventions impact on poverty alleviation, the expert on socio-economic poverty impact assessment (expert 4, see point 5 of this TOR) will have to carry out field visit to a representative sample of target areas to be decided with the Supervisor and agreed by the Contracting Authority. The selection shall be carried out in close liaison with any on-going TA services in place to the NRFA and the RDA.

2.4) Required outputs

- Aide-mémoire

- Draft and Final ETR Reports.

- Proposal of an addendum to the FA SPSP EDF10

- Proposal a capacity building program for the Road Sector in Zambia

- Proposal of Revised draft ToRs for the components of the Capacity Building Programme to the Road Sector to be supported by the EDF10 SPSP.

3. MANAGEMENT AND STEERING OF THE ETR

The Contracting Authority for this contract is the EU Delegation to the Republic of Zambia and Comesa (EUD) on behalf of the Zambian National Authorizing Officer (NAO).

The Supervisor is the Ministry of Communication and Transport (MCT) – Department of Planning.

The Contracting Authority and the Supervisor will identify one officer who will be the day-to-day contact person for this institution. This contact person will be introduced to the Consultant during the inception meeting (see below).

The Supervisor will arrange introductory letters to facilitate the experts’ access to Ministries, public organisations, authorities and agencies etc. whose activities and role are relevant to the assignment.

The Supervisor will also arrange for an inception meeting and regular progress meeting with the presence of representatives of all relevant core stakeholders.

During the assignment of the expert, NAO, MWS, MCT, NRFA, RTSA and RDA will make available all documents and information essential to the services to be performed.

The Consultant in conjunction with the supervisor will keep regularly the core stakeholders and the Contracting Authority fully informed of all developments of his assignment. In particular, any problems of timely availability or completeness of documentation must be formally advised to the Contracting Authority.

The ETR will be monitored by a Reference group chaired by the MCT-Department of Planning. This reference group will consist in representatives of the core stakeholders of the sector including MFNP, NAO, MWS, RDA, NRFA, RTSA, NCC, EIZ and the University of Zambia-Department of Engineering.

The reference group member's main functions are:

• To ensure that the evaluation team has access to and has consulted all relevant information sources and documents related to the project/programme.

• To validate the Evaluation Questions.

• To discuss and comment on notes and reports delivered by the evaluation team. Comments by individual group members are compiled into a single document by the evaluation manager and subsequently transmitted to the evaluation team.

• To assist in feedback of the findings, conclusions, lessons and recommendations from the evaluation.

4. EXPERTISE REQUIRED

4.1) Number of requested experts per category and number of man-days per expert or per category

The mission will be composed of three (3) senior experts: one Transport Economist, one Institutional development specialist, one civil Engineer, and one (1) Junior expert socio-environmentalist.

|No |Experts |Category |Number of working days |

|1 |Transport Economist / Team leader |Senior |37 |

|2 |Institutional Development Specialist |Senior |36 |

|3 |Civil Engineer |Senior |20 |

|4 |Socio-environmentalist |Junior |15 |

The number of working days foreseen for the experts is as follows:

Experts 1: 37 working days split up into 1 working day for preparatory work at home country, 1 working-day for travel and mission to Brussels, 2 working days for international travels to and from Lusaka, 25 working days for assignment in Zambia, 4 working days for drafting Draft ETR Report and 4 working days for drafting final ETR report. The duration of the assignment in Zambia for the expert is spread over a total of 6 calendar weeks.

Experts 2: 36 working days split up into 1 working day for preparatory work at home country, 1 working-day for travel and mission to Brussels, 2 working days for international travels to and from Lusaka, 25 working days for assignment in Zambia, 4 working days for drafting Draft ETR Report and 3 working days for drafting final ETR report. The duration of the assignment in Zambia for the expert is spread over a total of 6 calendar weeks.

Experts 3: 20 working days split up into 2 working days for international travels to and from Lusaka, 15 working days for assignment in Zambia, 2 working days for drafting Draft ETR Report and 1 working day for drafting final ETR report. The duration of the assignment in Zambia for the expert is spread over a total of 4 calendar weeks. The presence of expert 3 is not required during the Stakeholders' Seminar.

Experts 4: 15 working days split up into 2 working days for international travels to and from Lusaka, 10 working days for assignment in Zambia, 2 working days for drafting Draft ETR Report and 1 working day for drafting final ETR report. The duration of the assignment in Zambia for the expert is spread over a total of 3 calendar weeks. The presence of expert 4 is not required during the Stakeholders' Seminar.

4.2) Profile per expert required

Common Minimum requirement to all experts:

• University degree or equivalent

• Solid and diversified experience in the specific field of expertise needed, including experience in evaluation and/or monitoring of capacity building projects/programmes;

• Relevant experience of at least 3 years in Developing countries;

• Hands-on experience with the management and operations of road agencies in Developing countries;

• At least one of the experts is fully conversant with the principles and working methods of project cycle management and EC aid delivery methods;

• At least one of the experts will have hands-on experience with socio-economic impact assessments and gender integration analysis;

• Full working knowledge in English;

• Excellent computer skills with at least Microsoft offices usual softwares (Word, Excel, MS-Project, Power Point) or equivalent.

Additional requirements:

Expert 1- Transport Economist (Team Leader)

- Relevant University degree or equivalent experience in transport economics. A relevant post-graduate for this assignment will be favoured;

- Extensive experience in designing and/or implementing SPSP/Sector Wide Approaches;

- Specific experience in Eastern and/or Southern Africa will be favoured.

Expert 2 – Institutional development specialist

- Relevant University degree or equivalent experience in institutional development. A relevant post-graduate for this assignment will be favoured;

- Hands-on experience with the management and operations of road institutions in Sub-Saharan Africa;

- Specific experience in Eastern and/or Southern Africa will be favoured.

Expert 3 – Civil Engineer

- Relevant University degree or equivalent experience in Civil Engineering. A relevant post-graduate for this assignment will be favoured;

- Hands-on experience with management and evaluation of road projects in Sub-Saharan Africa;

- Specific experience in Eastern and/or Southern Africa will be favoured.

Expert 4 – Socio-environmentalist

- Relevant University degree or equivalent experience in monitoring and evaluating social/poverty impacts for road sector development programmes. A relevant post-graduate for this assignment will be favoured;

- Hands-on experience with socio-economic impact assessments, gender integration analysis;

- Specific experience in Eastern and/or Southern Africa will be favoured.

The composition of the team of experts should be balanced to allow for complete coverage of the different aspects of the programme identification as set out in these terms of reference, including cross-cutting issues (environmental sustainability, gender equity, good governance, and human rights).

4.3) Working language

The working language of the assignment is English.

5. LOCATION AND DURATION

5.1) Starting period and duration

It is expected that the assignment shall commence as soon as possible after the signature of the specific contract but no later than 11/07/2011.

The overall duration for the assignment is spread over 11 calendar weeks (77 calendar days = total performance period) and allows for a maximum of 37 working days for experts 1 (=longest duration), 36 working days for expert 2, 20 working days for expert 3 and 15 working days for expert 4. This overall duration shall cover desk study during preparatory stage, consultation with relevant stakeholders and preparation and implementation of a Stakeholders' Seminar, as well as time dedicated to preparing Draft and Final ETR reports.

It is expected that all Experts commence their assignment in Zambia the same day.

Detailed timelines and table of activities required from the experts, as well as their contribution in working days over duration of the assignment are shown in section 5.2.

The date on which the Contracting authority formally approves or rejects the Final ETR report following formal opinion from the supervisor will be considered the end date for the purpose of this contract. This approval/rejection shall be communicated to the Consultant in writing and shall occur within 21 calendar days from the receipt of the consolidated final ETR report.

5.2) Planning

At the beginning of the ETR, the relevant available documents should be reviewed and analysed.

Before going to Lusaka, experts 1 & 2 will have to interview in Brussels, relevant EC officials who participated previously in Zambia in the formulation and implementation of the SPSP I EDF9. They will also be briefed on the EC backbone strategy for Technical Cooperation principles.

A kick-off meeting with the Reference Group will be hold the 1st day of assignment in Lusaka and first interviews will be done with the major stakeholders when relevant. On the basis of the information collected the ETR team should:

• Describe the development co-operation context.

• Comment on the logical framework.

• Comment on the issues / evaluation questions suggested (see annexe 2; section3) or, when relevant, propose an alternative or complementary set of evaluation questions justifying their relevance. Develop the evaluation into sub-questions, identify provisional indicators and their verification means, and describe the analysis strategy.

• Present an indicative methodology to the overall assessment of the programme.

• Propose the work plan for the finalisation of the field phase.

• Submit its detailed work plan with an indicative list of people to be interviewed, surveys to be undertaken, dates of visit, itinerary, and name of team members in charge. This plan has to be applied in a way that is flexible enough to accommodate for any last-minute difficulties in the field. If any significant deviation from the agreed work plan or schedule is perceived as creating a risk for the quality of the evaluation, these should be immediately discussed with the evaluation manager.

During the inception phase an inception report shall be prepared (see section 5).

Once the inception report is submitted, the ETR team should:

• Ensure adequate contact and consultation with, and involvement of, the different stakeholders; working closely with the relevant government authorities and agencies during their entire assignment.

• Use the most reliable and appropriate sources of information and will harmonise data from different sources to allow ready interpretation.

• Carry out site visits to interview local stakeholders who benefited from the TA in Central and North-Western Provinces.

• To prepare an aide-mémoire presenting the key findings, conclusions and recommendations of the ETR.

• To prepare and implement a Stakeholders' Seminar in Lusaka where the aide-mémoire will be presented. The purpose of the Seminar will be to check the factual basis of the ETR, and to discuss the draft findings, conclusions and recommendations, including a reformulated proposed SPSP EDF10 Capacity Building programme.

On the basis of comments made by participants, and collected by the experts, the ETR team will then prepare a draft ETR report.

15 working days after submission of the draft ETR report, the experts will receive consolidated comments to be integrated in the final ETR report. Comments requesting methodological quality improvements should be taken into account, except where there is a demonstrated impossibility, in which case full justification should be provided by the ETR team. Comments on the substance of the report may be either accepted or rejected. In the latter instance, the ETR team is to motivate and explain the reasons in writing.

The quality of the final ETR report will be assessed by the Contracting Authority using a quality assessment grid (see annexe IV). The explanation on how to fill this grid is available on the following link: .

The various stages of the study are proposed to be undertaken as follows:

Home based desk study (1 working day for experts 1&2 to review relevant documentation listed in Annex I and draft initial remarks and/or comments.

Travel and Mission to Brussels (1 working day for experts 1&2): The Experts will have to meet key EC Officials (see Annex VI) EC officials who participated previously in Zambia in the formulation and implementation of the SPSP I EDF9. They will also be briefed on the EC backbone strategy for Technical Cooperation principles.

Mission to Zambia: the experts will start their mission to Zambia with a 1/2 working day kick-off meeting in the Supervisor's offices in Lusaka to be organised by the Supervisor with the Reference group. The purpose of this meeting will be to discuss the assignment in detail, to ensure that there is agreement on the meaning of the ToRs, to collect documents and to establish contacts with the relevant stakeholders etc.

During their assignment in Lusaka, the experts will prepare a Stakeholder's Seminar with all core sector stakeholders to present their aide-mémoire. Experts 1 and 2 will have to conduct this Seminar.

At the end of the Stakeholders' Seminar and before the departure of the experts, a debriefing meeting with the Reference Group will take place in the Supervisor's offices.

Following that debriefing meeting, the experts will return to their home base to prepare the draft ETR report.

Once received the consolidated comments submitted by the stakeholders within 15 working days after submission of the draft ETR Report, the consultant will have 4 working days to submit the final ETR report and any other relevant documents.

The dates mentioned in the table may be changed with the agreement of all parties concerned.

|Activity |Place |Duration |Expert 1 |Expert 2 |Expert 3 |Expert 4 |

| | |(working days) | | | | |

|Review of relevant documentation |Home country |1 |1 |1 |0 |0 |

|Travel and mission to Brussels |Brussels |1 |1 |1 |0 |0 |

|Travel home country to Lusaka |- |1 |1 |1 |1 |1 |

|Briefing meeting with Reference Group |Lusaka, Central and |1/2 |1/2 |1/2 |1/2 |1/2 |

| |North-Western | | | | | |

| |Provinces | | | | | |

| |/ Zambia | | | | | |

|Interviews with relevant stakeholders and Preparation and | |4 |4 |4 |4 |4 |

|Submission Inception report | | | | | | |

|Presentation Inception report to Reference Group | |1/2 |1/2 |1/2 |1/2 |1/2 |

|Interviews with relevant stakeholders, sites visit preparation of | |18 |18 |18 |15 |5 |

|stakeholders' seminar, preparation and submission of Aide-mémoire | | | | | | |

|Implementation of stakeholders' seminar, Debriefing meeting with | |2 |2 |2 |0 |0 |

|Reference Group | | | | | | |

|Travel from Lusaka to home country |- |1 |1 |1 |1 |1 |

|Preparation and submission of draft ETR report |Home country |4 |4 |4 |2 |2 |

|Comments from Stakeholders on draft ETR report |- |15 |- |- |- |- |

|Review & process comments from stakeholders and preparation and |Home country |4 |4 |3 |1 |1 |

|submission of final ETR report | | | | | | |

|TOTAL | |52 |37 |36 |20 |15 |

5.3) Location of assignment

Home office, Lusaka, Central and Northern Provinces (Zambia), Brussels (Belgium).

6. REPORTING

The consultant will submit the following reports in English:

- Inception report

- Short minutes of meetings

- Aide-mémoire

- Stakeholders' Seminar report including PowerPoint (or equivalent software) presentation(s)

- Draft ETR report

- Final ETR report

The reports must match quality standards. The text of the report should be illustrated, as appropriate, with maps, graphs and tables.

6.1) Content

6.1.1 Inception report of maximum 12 pages. In the report the consultant shall describe, in addition to his programme of work and staff mobilization, the first findings of the study, the foreseen difficulties in collecting data, other encountered and/or foreseen difficulties, their proposed methodology to deliver the services expected, the actual work done so far as a way of forecasting what quality of conclusion may be reached at the end of the consultancy.

6.1.2 Short minutes (max 2 pages per meeting) of each of the meetings held with the Reference Group.

6.1.3 Aide-mémoire the key findings, conclusions and recommendations of the consultancy.

6.1.4 Stakeholders' Seminar report (max 15 pages). The report will contain the proceedings and conclusions of the Stakeholders' Seminar and will include as annexe the PowerPoint (or equivalent software) slides presented by the Consultant.

6.1.5 Draft ETR report (of maximum 50 pages main text, excluding annexes) using the structure set out in Annex 2.

6.1.6 Final ETR report with the same specifications as mentioned under 6.1.5 above, incorporating any comments received from the concerned parties on the draft ETR report. The consultant will include as an Annex the DAC Format for Evaluation Report Summaries (see Annex 5).

All documents, reports, or other material acquired during the mission and relevant to the Road Sector will be submitted to the Contracting Authority and the Supervisor at the end of the mission, and will remain available for further missions and/or programmes.

6.2) Language

The language of the reporting is English.

6.3) Submission/comments timing

The inception report should to be produced after 4.5 working days from the start of the assignment in Zambia.

The Short minutes of each of the meetings held with the Reference Group should be produced within one working day following the meeting.

The Aide-mémoire should be sent to all Stakeholders invited to the Stakeholders' Seminar at least 2 working days before the day of the Seminar.

The Stakeholders' Seminar report shall be provided within two working days after the Seminar.

The draft ETR report shall be submitted 5 working days after departure of the Consultant from Zambia.

After submission of the draft ETR Report, the Stakeholders will have 10 working days for providing their comments to the Contracting Authority. The Contracting Authority after consultation with the Reference Group will then have 5 working days to send the consolidated comments to the Consultant.

The final ETR report shall be submitted five working days after the deadline for receiving consolidated comments from the Contracting Authority and should give account on how comments have been reflected in the report. The contracting authority will inform the contractor whether the Final Reports has been accepted within twenty-one (21) calendar days from its receipt.

6.4) Number of reports copies

All reports should be submitted in ten (10) paper copies and one (1) electronic copy to the Contracting Authority which will make circulated them to all main stakeholders. The paper copies should be printed on both sides (except for specific technical annexes where necessary).

7. ADMINISTRATIVE INFORMATION

7.1) Interviews if necessary indicating for which experts/position

Phone interviews of the proposed experts could be held by the Evaluation Committee before finalising the evaluation of offers received.

7.2) Language of the specific contract

The language of the specific contract is English.

7.3) Reimbursables

• Per diems to Brussels and Zambia;

• International travels to/from Brussels, to/from Zambia;

• Local travels between Lusaka and Central and Northern Provinces

• Logistics for Stakeholders' Seminar (hands-out for the participants, conference room, catering, coffee breaks and lunches only). A maximum of 50 participants is expected. The workshop will last maximum 2 working days and will be organised in Lusaka.

7.4) Others

The Supervisor will arrange introductory letters to facilitate the experts’ access to Ministries, public organisations, authorities and agencies etc. whose activities and role are relevant to the assignment.

MFNP (NAO), MCT, MWS, RDA, NRFA and RTSA will avail all the documents and information essential to the services to be performed.

The EUD and the MCT will identify one officer who will be the Consultant's day-to-day contact person. They will be formally presented to the consultant during the briefing meeting. The consultant will keep these officers informed in a detailed way of all developments of the study. These officers will assist in all meetings organized within the frameworks of this contract

ANNEXES

Annex I - Key documents for the ETR

Annex II - Layout, structure of the Final ETR Report

Annex III - Methodological observations

Annex IV - Quality assessment grid

Annexe V - The Standard DAC Format for Evaluation Report Summaries

Annexe VI – Key contact persons

Annex I: Key documents for the etr

• TRANSPORT POLICY OF THE REPUBLIC OF ZAMBIA (2002)

• Road Sector Policy of the Republic of Zambia (2003)

• ROADSIP II and its addenda

• Country Strategy Papers Zambia and Indicative Programmes for 2002-2007 and 2008-2013

• Financing Agreement SPSP EDF9 & SPSP EDF10

• SPSP EDF9 identification and formulation studies

• SPSP EDF10 Formulation study

• Mid-term review of the SPSP EDF9 & 7 key assessment areas 2008

• Final Report EDF9 TA to NRFA & RDA

• EC’s Result Oriented Monitoring Reports of the SPSP EDF9

• Auditor General's Audit Report 2009 on RDA

• PAC's report of July 2010 on Auditor General's Audit Report on RDA

• GRZ Matrices of short, medium and long-term corrective measures following Auditor General's Audit Report 2009 on RDA and PAC's recommendations

• Draft ToRs for the planned EDF10-funded TA

• Last version available of the Road Sector management Plan agreed between GRZ and CPs

Note: The evaluation team has to identify and obtain any other document worth analysing, through its interviews with people who are or have been involved in the design, management and supervision of the project / programme. Resource persons to collect information and data are to be sought in the EU services, implementing body and / or public service in Zambia.

Annex II: Layout, structure of the ETR Report

THE FINAL ETR REPORT SHOULD NOT BE LONGER THAN APPROXIMATELY 50 PAGES. ADDITIONAL INFORMATION ON OVERALL CONTEXT, PROGRAMME OR ASPECTS OF METHODOLOGY AND ANALYSIS SHOULD BE CONFINED TO ANNEXES.

The cover page of the report shall carry the following text:

‘’This end-term review is supported and guided by the Government of the Republic of Zambia and the European Union and presented by [name of consulting firm]. The report does not necessarily reflect the views and opinions of the Government of the Republic of Zambia and the European Union’’.

1. Executive Summary

A tightly-drafted, to-the-point and free-standing Executive Summary is an essential component. It should be short, no more than five pages. It should focus mainly on the key purpose or issues of the ETR & 7 key assessment areas, outline the main analytical points, and clearly indicate the main conclusions, lessons learned and specific recommendations. Cross-references should be made to the corresponding page or paragraph numbers in the main text that follows.

2. Introduction

A description of the SPSP EDF9 in the overall development cooperation context, the consistency with EU programming and in the context of the sector programme ROADSIP II and the ETR, providing the reader with sufficient methodological explanations to gauge the credibility of the conclusions and to acknowledge limitations or weaknesses, where relevant.

3. Answered questions/ findings

A chapter presenting the ETR questions and conclusive answers, together with evidence and reasoning.

The organization of the report should be made around the responses to the ETR questions which are systematically covering the DAC evaluation criteria: relevance, effectiveness, efficiency, impact and sustainability, plus coherence and added value specific to the EU. In such an approach, the criteria will be translated into specific questions. These questions are intended to give a more precise and accessible form to the ETR criteria and to articulate the key issues of concern to stakeholders, thus optimising the focus and utility of the ETR.

3.1) Problems and needs (Relevance)

The extent to which the objectives of the SPSP EDF9 in support of the ROADSIP II sector programme are consistent with the sector programme itself, the transport policy and the overarching PRBS/FNDP as well as consistency with EU policies on Effectiveness of Aid, the PARIS Declaration on Aid Effectiveness and DAC Guidelines on Sector Wide Approaches, and the European Consensus on Development, 2006.

The analysis of relevance will focus, but not limited to, on the following questions in relation to the design of the project:

• the extent to which the SPSP EDF9 has been consistent with, and supportive of, the policy and programme framework within which the SPSP is placed, in particular the EC’s EDF9 Country Strategy Paper and National Indicative Programme, the GRZ PRBS/FNDP an the 2002 Transport Policy and derived 2002 Road Acts 11,12 and 13;

• the quality of the analyses of lessons learnt from past experience such as ROM missions, Mid-term review of 2008 and technical audits, etc.;

• the SPSP EDF9 coherence with current/on going initiatives from GRZ, EU and other CPs particularly regarding the response to the concept of rural roads management and the periodic maintenance/rehabilitation of the Public Core Road Network in a national and regional context;

• the quality of the problem analysis and the SPSP EDF9 intervention logic and logical framework matrix, appropriateness of the objectively verifiable indicators against ROADSIP II indicators;

• the extent to which stated objectives correctly address the identified sector issues, the impact of the road transport interventions on poverty reduction matters such as basic access to the rural poor, clarity and internal consistency of the stated objectives;

• the appropriateness of the taken approach of a rather targeted sector budget support with interventions to specific (as specified in the Financing Agreement) interventions of periodic maintenance of trunk, main and district roads and rehabilitation of rural roads (feeder roads) against the concept of untargeted support of not identifying the specific utilisation of tranches and leave it to the demand driven sector needs under ROADSIP II defined in Annual Workplans;

• the extent to which the nature of the problems originally identified have changed during the sector reform process mainly taken off in 2005 with the establishment of the three autonomous road agencies;

• the extent to which objectives have been updated in order to adapt to changes in the context;

• the degree of flexibility and adaptability to facilitate rapid responses to changes in circumstances for physical interventions, capacity building and road condition survey (TA services), technical audit, socio-economic assessment of physical interventions under sector budget support tranches, supplies to support the operation of the NRFA and RDA and support to the Axle Load Control Programme which was specifically requested for during implementation;

• The appropriateness of the TA concept of long-term capacity building to the NRFA and RDA in view of the sector reform process and establishment of the new autonomous agencies RDA and NRFA in 2005 against different TA concepts such as short term support on a periodic basis (demand driven and owned by GRZ) for upgrading systems, developing specific skills, in country-training, workshops etc.;

• The appropriateness of TA services and supplies to be delivered under EDF procedures (project approach under SPSP framework) in comparison to providing the latter under sector budget support tranches utilising national procurement;

• The appropriateness of the design of TA services for road conditions survey under EDF project funding with regard to scope;

• the quality of the identification of key stakeholders and target groups (including gender analysis and analysis of vulnerable groups) and of institutional capacity issues of mainly the RDA and NRFA to fulfil their mandates according to the respective 2002 Road Acts;

• the stakeholder participation in the design and in the management/implementation of the SPSP particularly for the rural roads component, the level of local ownership, absorption and implementation capacity at NRFA, RDA, District Councils (Road Authorities);

• the quality of the analysis of strategic options (selection of modality), of the justification of the recommended implementation strategy, and of management, coordination and steering arrangements (ROADSIP Steering committee meetings, Joint Donor Fora etc.);

• the realism in the choice and quantity of inputs (financial, human and administrative resources) in view of the sector reform progress and the establishment of road agencies;

• the analysis of assumptions and risks;

• the appropriateness of the recommended monitoring and ETR arrangements;

3.2) Achievement of purpose (Effectiveness)

The effectiveness criterion, concerns how far the SPSP EDF9 results (i) in support of ROADSIP II and (ii) in relation to capacity building were attained so far, and how far the SPSP EDF9 specific objective(s) have been achieved, or are expected to be achieved.

The analysis of Effectiveness will therefore focus, but not limited to, on such issues as:

• whether the planned benefits in support of ROADSIP II to achieve its targets have been delivered and received, as perceived by all key stakeholders from GRZ, CPs and the vulnerable groups of beneficiaries;

• whether the planned benefits to create capacity in the newly established road agencies RDA and NRFA have been delivered and received, as perceived;

• whether the planned benefits through support to the Axle Load Control Programme have been delivered and received, as perceived;

• whether the planned benefits from the provided services for annual road condition survey have been delivered and received, as perceived in view of the fact that not all roads of the public core road network are covered;

• whether intended beneficiaries participated in the intervention particularly with regard to the rural roads component;

• with regard to support the institutional reform progress through TA capacity building services, whether behavioural patterns have changed at NRFA and RDA national and regional offices, Road Authorities and how far the changed institutional arrangements and characteristics have produced the planned efficiency improvements;

• if the assumptions and risk assessments at results level turned out to be inadequate or invalid, or unforeseen external factors intervened, how flexibly the NAO and EU Delegation have adapted to ensure that the results would still achieve the purpose; and how well has it been supported by the NRFA and RDA;

• whether the balance of responsibilities between the various stakeholders was appropriate, which accompanying measures have been taken by the partner authorities in support of the SPSP EDF9;

• how unintended results have affected the benefits received positively or negatively, directly and indirectly, and could have been foreseen and managed;

• whether any shortcomings were due to a failure to take account of cross-cutting or over-arching issues such as gender, environment and poverty during implementation up to date;

• how the SPSP EDF9 contributed to aid effectiveness on such issues as harmonization, joint reporting and missions, co-financing arrangements etc. with the view to reduce transaction costs?

3.3) Sound management and value for money (Efficiency)

The efficiency criterion concerns how well the various activities in the sector programme supported by the SPSP transformed the available resources into the intended results, in terms of quantity, quality and timeliness. Comparison should be made against what was planned.

The assessment of Efficiency will therefore focus, but not limited to, on such issues as:

• whether the same results with regard to physical interventions could have been achieved at lower costs particularly with regard to the application of output performance based contracts (OPRC) against conventional contracts;

• in the same light of the above, whether the long-term TA capacity building approach in support of the newly established road agencies RDA and NRFA as accompanying measure of the sector reform process was appropriate in comparison to other possible options (see also 1 on the issue) better responding to demand;

• whether the procurement and delivery of capacity building services would have been better carried out and managed under the sector budget tranches using national procurement rather using EDF procedures with regard to ownership and transaction costs (see also 1 on the issue);

• In the same light of the above the supply provided for the operation of the RDA and NRFA agencies;

• quality of sector dialogue and relations with local authorities, institutions, beneficiaries and other CPs from the Delegations perspective;

• whether management of risk has been adequate, i.e. whether flexibility has been demonstrated in response to changes in circumstances;

• the quality of information management and reporting from RDA, NRFA, TA services, and the extent to which key stakeholders have been kept adequately informed of project activities (including beneficiaries/target groups);

• respect for deadlines; timely submission of request for release of tranches in line with the budget cycle and the preparation of the Annual Workplan of the RDA;

• Extent to which the costs of the SPSP EDF9 have been justified by the benefits and the performance of ROADSIP II whether or not expressed in monetary terms in comparison with similar projects or alternative approaches under project financing modality from other CPs supporting ROADSIP II under a sector approach, taking account of contextual differences and eliminating market distortions;

• RDA, NRFA, MFNP, MCT, MWS contributions (such as offices, counterpart-staff, reports, tax exemptions, local purchase orders etc.): have they been provided as planned?

• EC HQ/Delegation inputs (e.g. TA support, supplies, support to Axle Load Control Programme, training/capacity building under Programme Estimates): have they been provided as planned?

• TA for RDA and NRFA and Road Condition Survey: how well did it help to provide appropriate solutions and develop local capacities to define and produce results?

• quality of monitoring: its existence under ROADSIP II, accuracy and flexibility, and the use made of it; adequacy of baseline information;

• quality of technical and financial audit;

• did any unplanned outputs arise from the activities so far which could have short and medium repercussion on the continuation of SBS in support of ROADSIP II (to be also seen in the light of the assessment of the seven key areas, part “B”);

• the assessment to which “additionality” was achieved at sector level for ROADSIP II target achievements in comparison to the “without SPSP” approach bearing in mind that GRZ reduced treasury contributions to an extent of the magnitude of SPSP tranches (to be confirmed from MOFNP, NRFA and RDA).

3.4) Achievement of wider effects (Impact)

The term impact denotes the relationship between the SPSP specific and overall objectives.

At Impact Level the ETR will make an analysis of the following aspects:

• Extent to which the objectives of the SPSP EDF9 have been achieved;

• Extent to which ROADSIP II objectives and Annual Sector Workplans have been achieved through the support and contribution of the SPSP EDF9;

• Extent to which the capacity building component has built “capacity” at the road agencies RDA and NRFA;

• Extent to which the supplies under the SPSP EDF9 could facilitate efficiency improvement at the agencies;

• Whether the expected socio-economic impact has so far been achieved through road interventions supported under the SPSP EDF9;

• whether the effects of the project:

a) have been facilitated/constrained by external factors

b) have produced any unintended or unexpected impacts, and if so how have these affected the overall impact.

c) have been facilitated/constrained by project/programme management, by co-ordination arrangements, by the participation of sector stakeholders

d) have contributed to economic and social development through infrastructure access/connectivity improvements

e) directly/indirectly created employment and supported the diversification of the economy e.g. for providing access in areas for tourism such as Eastern province

f) have contributed to poverty reduction

g) have made a difference in terms of cross-cutting issues like gender equality, environment, good governance, conflict prevention etc.

3.5) Likely continuation of achieved results (Sustainability)

The sustainability criterion relates to whether the positive outcomes of the SPSP EDF9 and the flow of benefits are likely to continue under ROADSIP II if support to the latter through SBS as financing modality would end.

The ETR will make an attempt to assess the prospects for the sustainability of benefits at this stage on basis of the following issues:

• the ownership of objectives and achievements, e.g. how far all stakeholders were consulted on the objectives from the outset, and whether they agreed with them and continue to remain in agreement;

• policy support and the responsibility of the beneficiary institutions such as RDA and NRFA, e.g. how far EC policies and national policy are corresponding, the potential effects of any policy changes such as the transfer system of funding from the Treasury to the Road Fund which has changed beginning of 2007; how far the relevant national, sectoral and budgetary policies and priorities are affecting the project positively or adversely; and the level of support from governmental, public, business and civil society organizations.

• institutional capacity, how far good relations with existing institutions such as RDA and NRFA have been established; whether the institution appears likely to be capable of continuing the flow of benefits after the SPSP EDF9 ends (is it well-led, with adequate and trained staff, sufficient budget and equipment); whether counterparts have been properly prepared for taking over, technically, financially and managerially;

• if the approach taken for the capacity building to Road Authorities, Regional RDA Engineers Offices and support to the local construction industry through training of contractors is likely to deliver sustainable results and if the coordination arrangements with other CPs like Danida and KfW delivering similar programmes under project financing modality are regarded as sustainable;

• the adequacy of the SPSP EDF9 budget for its purpose incl. physical interventions, TA capacity building support and support to the axle load control programme;

• socio-cultural factors, e.g. whether the project is in tune with local perceptions of needs particularly with regard to the provisions of basic access to the rural poor, reduction of overload to preserve the road assets etc.; whether it respects local power-structures, status systems and beliefs, and if it sought to change any of those, how well-accepted are the changes both by the target group and by others; how well it is based on an analysis of such factors, including target group/beneficiary participation in design and implementation; and the quality of relations between the external project staff and local communities and institutions such as RDA and NRFA.

• financial sustainability, e.g. whether the physical interventions provided are/will be placed under affordable maintenance schemes financed under the National Road Fund within the annual budget of the annual workplans of sector agencies (to answer the question whether the provided physical road interventions meet their service delivery functions and whether they can be adequately maintained under annual sector budgets to leave the vicious cycle of built – collapse – re-built through adequate safeguards); and economic sustainability, i.e. how well do the benefits (returns) compare to those on similar undertakings once market distortions are eliminated.

• technical (technology) issues, e.g. whether (i) the technology, knowledge, process or service introduced or provided fits in with existing needs of the beneficiary institutions and local Road Authorities also with regard to culture, traditions, skills or knowledge; (ii) alternative technologies are being considered such as labour based methods whenever feasible; and (iii) the degree in which the beneficiaries have been able to adapt to and maintain the capacity acquired without further assistance.

• Wherever relevant, cross-cutting issues such as gender equity, environmental impact and good governance; were appropriately accounted for and managed from the outset of the project.

3.6) Mutual reinforcement (coherence)

The extent to which activities undertaken allow the EU to achieve its development policy objectives without internal contradiction or without contradiction with other EU policies. Extent to which they complement partner country's policies and other CPs' interventions.

Considering other related activities undertaken by GRZ or other CPs, at the same level or at a higher level:

• likeliness that results and impacts will mutually reinforce one another;

• likeliness that results and impacts will duplicate or conflict with one another;

Connection to higher level policies (coherence)

-> Extent to which the SPSP EDF9 (its objectives, targeted beneficiaries, timing, etc.):

• is likely to contribute to / contradict other EU policies

• is in line with evolving strategies of the EU and its partners

3.7) EU value added

Connection to the interventions of institutions of Member States such as DANIDA and KfW active in the Road Transport Sector. Extent to which the project/programme (its objectives, targeted beneficiaries, timing, etc.) is:

• complementary to the intervention of EU Member States in the country;

• co-ordinated with the intervention of EU Member States in the country;

• creating actual synergy (or duplication) with the intervention of EU Member States;

• involving concerted efforts by EU Member States and the EU to optimise synergies and avoid duplication.

4. Visibility

The consultants will make an assessment of the SPSP EDF9 strategy and activities in the field of visibility, information and communication, the results obtained and the impact achieved with these actions in both the beneficiary country and the EU countries.

5. Overall Assessment

A chapter synthesising all answers to ETR questions into an overall assessment of the SPSP EDF9 and how it has supported the sector programme ROADSIP II so far incl. capacity building issues and Road Conditions Survey as well as the support to the axle load control programme. The detailed structure of the overall assessment should be refined during the ETR process. The relevant chapter has to articulate all the findings, conclusions and lessons in a way that reflects their importance and facilitates the reading. The structure should not follow the ETR questions, the logical framework or the seven ETR criteria.

The main results will also be reflected in part “C” of the report.

6. Proposal for modifying the financing agreement of the SPSP EDF10

Taking into account the lessons learnt from the ETR SPSP EDF9 and described in the previous chapters, and the sector context, the consultant will have to propose, if relevant, necessary changes to the SPSP EDF10 in order to ensure a successful implementation of this programme. These changes will be translated into a draft addendum to the financing agreement SPSP EDF10 to be attached in the technical annexe.

7. Proposal for a sector Capacity Building Program

Taking into account the lessons learnt from the ETR SPSP EDF9 and described in the previous chapters, the medium and long-term needs expressed by the Beneficiaries and coming from the findings of AG's and PAC's reports and the other planned and on-going CPs' interventions, the consultant will have to propose a revised meaningful Capacity Building programme for the entire sector under a holistic approach, taking issues of Governance and tertiary education related to Road Transport into consideration. This programme will be partly or fully financed by the SPSP EDF10. Revised draft Terms of reference of the TA financed by SPSP EDF10 in charge of supporting the full or part implementation of this programme will be proposed in technical annexe to this report.

8. Conclusions and Recommendations

8.1) Conclusions

This chapter introduces the conclusions relative to each question. The conclusions should be organised in clusters in the chapter in order to provide an overview of the assessed subject.

Note: the chapter should not follow the order of the questions or that of the ETR criteria (effectiveness, efficiency, coherence, etc.)

It should features references to the findings (responses to the ETR questions) or to annexes showing how the conclusions derive from data, interpretations, and analysis and judgement criteria.

The report should include a self-assessment of the methodological limits that may restrain the range or use of certain conclusions.

The conclusion chapter features not only the successes observed but also the issues requiring further thought on modifications or a different course of action.

The ETR team presents its conclusions in a balanced way, without systematically favouring the negative or the positive conclusions.

A paragraph or sub-chapter should pick up the 3 or 4 major conclusions organised by order of importance, while avoiding being repetitive. This practice allows better communicating the ETR messages that are addressed to the GRZ and the EU.

If possible, the ETR report identifies one or more transferable lessons, which are highlighted in the executive summary and presented in appropriate seminars or meetings so that they can be capitalised on and transferred.

8.2) Recommendations

They are intended to improve or reform the project/ programme in the framework of the cycle under way, or to prepare the design of a new intervention for the next cycle.

Note: the recommendations must be related to the conclusions without replicating them. A recommendation derives directly from one or more conclusions.

The ultimate value of an evaluation depends on the quality and credibility of the recommendations offered. Recommendations should therefore be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the project, and of the resources available to implement them both locally and in the Commission.

They could concern policy, organisational and operational aspects for both the national implementing partners and for the EY; the pre-conditions that might be attached to decisions on the financing of similar projects; and general issues arising from the evaluation in relation to, for example, policies, technologies, instruments, and institutional development, and regional, country or sector strategies.

Recommendations must be clustered and prioritised, carefully targeted to the appropriate audiences at all levels, especially within the EU structure.

Technical annexes

- Technical information and data as required substantiating the proposal

- The proposed addendum to the FA SPSP EDF10

- The proposed revised ToRs for the TA to the road sector to be financed under SPSP EDF10

- The aide-mémoire

- The Stakeholder's Workshop report

- Map of project area, if relevant

- Page DAC summary, following the format in Annex V.

Administrative annexes

- ETR methodology (including: options taken, difficulties encountered and limitations. Detail of tools and analyses /work plan (2-5 pages)

- Itinerary (1-2 pages)

- List of persons/organisations consulted (1-2 pages)

- List of documentation consulted (1-2 pages)

- Curricula vitae of the consultants (1 page per person).

- Terms of Reference of the ETR and 7 key assessment areas study

Annex III - Methodological observations

The ETR team should refer to the programme’s logical framework.

It is suggested that the ETR team carries out:

• a rapid appraisal through a field visit and a series of interviews

• a questionnaire survey involving a sample of beneficiaries

• a series of focus groups involving beneficiaries and non-beneficiaries

• a series of case studies

The proposal in response to these terms of reference should identify any language and/or cultural gap and explain how it will be bridged.

The programme is to be judged more from the angle of the beneficiaries’ perceptions of benefits received than from the managers’ perspective of outputs delivered or results achieved. Consequently, interviews and surveys should focus on outsiders (beneficiaries and other affected groups beyond beneficiaries) as much as insiders (managers, partners, field level operators). The proposal in response to these ToRs, as well as further documents delivered by the ETR team, should clearly state the proportion of insiders and outsiders among interviews and surveys.

A key methodological issue is whether observed or reported change can be partially or entirely attributed to the programme, or how far the programme has contributed to such change. The ETR team should identify attribution / contribution problems where relevant and carry out its analyses accordingly.

It must be clear for all ETR team members that the ETR is neither an opinion poll nor an opportunity to express one’s preconceptions. This means that all conclusions are to be based on facts and evidence through clear chains of reasoning and transparent value judgements. Each value judgement is to be made explicit as regards:

• the aspect of the programme being judged (its design, an implementation procedure, a given management practice, etc.)

• the evaluation criterion is used (relevance, effectiveness, efficiency, sustainability, impact, coherence, EU value added)

The ETR report should not systematically be biased towards positive or negative conclusions. Criticisms are welcome if they are expressed in a constructive way. The ETR team clearly acknowledges where changes in the desired direction are already taking place, in order to avoid misleading readers and causing unnecessary offence.

Annex IV - Quality assessment grid

*THIS GRID IS ANNEXED TO THE TORS FOR INFORMATION TO THE CONSULTANT

The quality of the final report will be assessed by the Contracting Authority using the following quality assessment grid where the rates have the following meaning:

1 = unacceptable = criteria mostly not fulfilled or totally absent

2 = weak = criteria partially fulfilled

3 = good = criteria mostly fulfilled

4 = very good = criteria entirely fulfilled

5 = excellent = criteria entirely fulfilled in a clear and original way

|Concerning the criteria and sub-criteria below, the evaluation report is rated: |1 |2 |3 |4 |5 |

|b) Does the report clearly cover the requested period of time, as well as the target groups and| | | | | |

|socio-geographical areas linked to the project / programme? | | | | | |

|c) Has the evolution of the project / programme been taken into account in the evaluation | | | | | |

|process? | | | | | |

|d) Does the evaluation deal with and respond to all ToR requests. If not, are justifications | | | | | |

|given? | | | | | |

|2. Appropriate design | | | | | |

|b) Is the evaluation method clearly and adequately described in enough detail? | | | | | |

|c) Are there well-defined indicators selected in order to provide evidence about the project / | | | | | |

|programme and its context? | | | | | |

|d) Does the report point out the limitations, risks and potential biases associated with the | | | | | |

|evaluation method? | | | | | |

|3. Reliable data | | | | | |

|b) Are the sources of information clearly identified in the report? | | | | | |

|c) Are the data collection tools (samples, focus groups, etc.) applied in accordance with | | | | | |

|standards? | | | | | |

|d) Have the collected data been cross-checked? | | | | | |

|e) Have data collection limitations and biases been explained and discussed? | | | | | |

|4. Sound analysis | | | | | |

|b) Is the analysis clearly focused on the most relevant cause/effect assumptions underlying the| | | | | |

|intervention logic? | | | | | |

|c) Is the context adequately taken into account in the analysis? | | | | | |

|d) Are inputs from the most important stakeholders used in a balanced way? | | | | | |

|e) Are the limitations of the analysis identified, discussed and presented in the report, as | | | | | |

|well as the contradictions with available knowledge, if there are any? | | | | | |

|5. Credible findings | | | | | |

|b) Is the generalisability of findings discussed? | | | | | |

|c) Are interpretations and extrapolations justified and supported by sound arguments? | | | | | |

|6. Valid conclusions | | | | | |

|b) Does the report reach overall conclusions on each of the five DAC criteria? | | | | | |

|c) Are conclusions free of personal or partisan considerations? | | | | | |

|7.Useful recommendations | | | | | |

|b) Are recommendations operational, realistic and sufficiently explicit to provide guidance for| | | | | |

|taking action? | | | | | |

|c) Do the recommendations cater for the different target stakeholders of the evaluation? | | | | | |

|d) Where necessary, have the recommendations been clustered and prioritised? | | | | | |

|8.Clear report | | | | | |

|b) Is the report well structured and adapted to its various audiences? | | | | | |

|c) Are specialised concepts clearly defined and not used more than necessary? Is there a list | | | | | |

|of acronyms? | | | | | |

|d) Is the length of the various chapters and annexes well balanced? | | | | | |

|Considering the 8 previous criteria, what is the| | |

|overall quality of the report? | | |

|Evaluation type: Efficiency, effectiveness and |Date of report: |Subject of evaluation : |

|impact. | | |

|Language : English |N° vol./pages : |Author : |

|Programme and budget line concerned : |

|Type of evaluation : |( ) ex ante |(x ) intermediate / ongoing |( ) ex post |

|Timing : |Start date : |Completion date : |

|Contact person : |Authors : |

|Cost : Euro |Steering group : Yes/No |

Annexe VI – Key contact persons

|LIST OF RELEVANT STAKEHOLDERS |

|ENTITY |CONTACT PERSON |EMAILS |

|MINISTRY OF FINANCE AND NATIONAL PLANNING |MR TEMWANI CHIHANA – COORDINATOR |TCHIHANA@.ZM |

|(MFNP) - NATIONAL AUTHORISING OFFICE FOR THE | | |

|EDF (NAO) | | |

|MINISTRY OF COMMUNICATIONS AND TRANSPORT |MR GREEN MBOZI - DIRECTOR |WILL BE COMMUNICATED TO THE CONSULTANT WHEN |

|(MCT) - DEPT OF PLANNING | |STARTING THE ASSIGNMENT |

|MINISTRY OF WORKS AND SUPPLIES (MWS) – DEPT |MRS BUPE KAONGA - DIRECTOR |NALUHANGA@ |

|PLANNING & MONITORING | | |

|MINISTRY OF LOCAL GOVERNANCE AND HOUSING |MR PETER LUBAMBO - DIRECTOR |LUBAMBOP@.ZM OR LUBAMBOP@ |

|(MLGH) - DEPT OF INFRASTRUCTURE AND SUPPORT | | |

|SERVICES | | |

|ROAD DEVELOPMENT AGENCY (RDA) |MR LUAMBE MONDOLOKA – BOARD CHAIRMAN |luambem@.zm; luambemmondoloka@iconnect.zm|

| |Mr –Director & CEO |mmulenga@.zm |

|National Road Fund Agency (NRFA) |Mr Rabbison M. CHONGO – Board Chairman |chongorabbi@ |

| |Dr Anthony MWANAUMO – Director | |

| | |amwanaumo@.zm |

|Road Traffic and Safety Agency (RTSA) |Mr Kaitano CHUNGU – Board Chairman |k.chungu@ |

| |Mr Frederik MWALUSAKA - Director |fmwallsk@ |

|National Construction Council (NCC) |Mr Levi ZULU – Board Chairman |lzulu@zamnet.zm |

|Engineering Institute of Zambia (EIZ) |Mr Benjamin MUSONDA |bjmusonda@ |

|University of Zambia (UNZA) - School of |Dr Mundia MUYA |Mundiamuya2000@yahoo.co.uk |

|Engineering - Dept of Civil Engineering | | |

|Association of Building and Civil Engineering|Mr Phesto MUSONDA – President |pnm@zamnet.zm |

|Contractors (ABCEC) |Mr Paul GOLSON - Secretary |abceczam@ |

|EU Delegation to Zambia & Comesa – |Mr Sigvard BJORCK – Head of Section |sigvard.bjorck@eeas.europa.eu |

|Infrastructure Section |Mr Mehdi MAHJOUB – Engineering Advisor |mehdi.mahjoub@eeas.europa.eu |

|European Commission (EC) Directorate General |Mr Paolo CICCARELLI |paolo.ciccarelli@ec.europa.eu |

|(DG) DEVCO-EuropeAid |(previously Head of Infrastructure section in the | |

| |EUD to Zambia Sep 2001- Aug 2005) | |

|EC DG DEVCO-EuropeAid |Mr Jürgen KETTNER (previously Head of |juergen.kettner@ec.europa.eu |

| |Infrastructure section in the EUD to Zambia Jan | |

| |2006- July 2010) | |

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download