EUROPEAID/ 119860/C/SV/multi



SPECIFIC TERMS OF REFERENCE

Evaluation of EU support to the reforms in the Georgian education sector

FWC BENEFICIARIES 2013 - LOT 9: Culture, Education, Employment and Social EuropeAid/132633/C/SER/multi

1. BACKGROUND

Systematic and timely evaluation of its programmes and activities is an established priority[1] of the European Commission (further referred to as 'Commission')[2]. The focus of evaluations is on the assessment of achievements, the quality and the results of interventions in the context of an evolving cooperation policy with an increasing emphasis on result-oriented approaches[3]. Evaluations should provide an understanding of the cause and effects links between activities and results.

The main objectives of the evaluation are:

• to identify key lessons and to produce recommendations in order to improve the current and future European Union's strategies, programmes and actions in the field of education in general. In particular, the evaluation should provide lessons in order to inform the preparation of the programmes to be launched by the EU in Georgia, under the new programming period (2014 – 2017);

• to provide the relevant external co-operation services of the European Union and the wider public with an overall independent assessment of the European Union's past and current cooperation with Georgia.

The evaluation should take into account and aim to feed recommendations into the future cooperation programing in the field of education (bilateral, regional and thematic) with Georgia as part of the Eastern Partnership region.

This evaluation is to be carried out under ENPI-East Decision 2009/021067 for the VET SPSP (contract 227-540) and the Twinning Technical Assistance Facility (TWG/TA FA 2008 2008/019-751).

1.1 Reforms in the Education Sector in Georgia

The scope and pace of Georgian education reform since 2003 are unique in the region. As a result of reforms implemented since becoming a Bologna signatory country in 2005, higher education in Georgia has moved decisively from a centrally-controlled, supply-driven, public-funded, state system, to a more complex, increasingly market-oriented system, with considerable cost-sharing through student fees and a strongly emerging private university sector. The extent to which higher education has moved forward in these new directions, and the swiftness of the pace with which it has done so, makes Georgia unique in relation to other countries in the region.

In particular, reforms of financing and governance of educational institutions, that other countries have been grappling with for years, have been initiated at a stroke with the per capita financing principle of ‘money follows the student’ in both general and higher education. The state undertakes to provide twelve years of free general education, and primary, basic and general schools (which have been converted from local-government budget organizations to autonomous Legal Entities of Public Law or LEPLs) are funded directly from the Ministry of Education and Science, receiving an amount per pupil (a voucher) which varies only according to the location of the school (highest for those in highlands, lowest for those in cities) and covers current but not capital expenditures.

A similar formula-funding model for teaching activities has been applied to higher education institutions (HEIs), with three voucher rates depending on the student’s specialization. A courageous accreditation process has also been followed, reducing the number of authorized institutions in two steps from 240 in 2004 to 63 in 2013; HEIs (also LEPLs) can set their own tuition fees; uniform grants to a relatively small number of students have been replaced by grants on a sliding scale to a larger number, but still merit-based; and a student loan scheme has been initiated in cooperation with commercial banks. In line with Georgia’s admission to the Bologna process, a three-cycle degree system has been adopted, and the heads of HEIs are no longer appointed by the President but elected by each institution’s Academic Council (the highest representative body, including elected professors and student representatives, with a managerial role). Currently there are 67 HEIs recognised by the state (accredited and newly licensed): 18 public and the remaining 49 private. Only 18 HEIs are located outside the capital, which means that 73% of HEIs are in Tbilisi. The total number of students in all HEIs enrolled is 149,819 (January 2014).

A crucial contribution to reduction in corruption and reform of higher education admission has been made by the introduction of unified entrance examinations, held annually since 2005, using sophisticated testing methods across a range of subjects. These are administered by an agency, the National Examinations Center,[4] under the governance of but at arm’s length from the MoES. Wide support for the new system was found throughout the country, and a large majority of test takers, parents and administrators felt confident that it would help to eliminate corruption in university admission (Transparency International 2006).

Similar agencies have been set up in other areas important to quality assurance and control. The accreditation of higher education institutions, already described, was carried out by the Legal Entity of Public Law (LEPL) "National Center for Educational Quality Enhancement" (EQE),[5] which was founded on 1 September, 2010 under the Law of Georgia on Educational Quality Enhancement. The EQE is the legal successor of the "National Center for Educational Accreditation", established in October 2004. It provides accreditation to educational institutions on all educational levels, and is developing criteria for this purpose as well as encouraging a process of self-evaluation. The National Curriculum and Assessment Center, established in April 2006, has introduced new curricula, designed to encourage active learning rather than mechanical transfer of knowledge. Authors have developed new textbooks in response to the new curricula: a textbook rental scheme has been piloted but not implemented nationally. Since 2008 the Teachers’ Professional Development Center, aims to develop standards and qualification requirements for teachers, to conduct (in collaboration with the NEAC) a process of accreditation of teacher training and retraining programs and to introduce a system of teacher certification. After many years of attention to in-service training, reform of pre-career training is regarded as the greater challenge. This trend also includes implementation of the objectives of the Bologna Process, which is a top priority of the Ministry of Education and Science (MoES) of Georgia. The recent Tempus study on the state of play of the Bologna process shows that in Georgia: The three-cycle HE system has been implemented. Bachelor, Master and doctoral programmes have already been introduced in all accredited higher education institutions (HEI). All students below doctoral level are enrolled in the two-cycle degree system (except for certain specific specialisations such as medicine). More than 75% of Georgian Higher Education institutions and programmes use ECTS for both transfer and accumulation purposes. Allocation of ECTS is based on learning outcomes and student workload. Bachelor programmes cannot comprise of less than 240 ECTS credits whereas Master programmes comprise of 120 ECTS and doctoral programmes, of 180 ECTS.

Georgia is among the most advanced Tempus Partner Country issuing the Bologna Diploma Supplement to graduates at the end of their studies, in the vast majority of HE institutions and study programmes. In Georgia, the transition towards an independent quality assurance system is not yet finalised. At the moment, the established National Centre for Educational Quality Enhancement still acts as a government-dependent body, with the objective of ensuring the authorisation and accreditation processes. Georgia is very advanced in the process of establishing a National Qualifications Framework. The National Qualifications Framework for higher education has been formally adopted and implementation started. In Georgia, the compliance with NQF became, in 2011, one of the criteria for the accreditation of programmes.

1.2 Cooperation between EU and Georgia in the sector of education

In June 2004, Georgia was included among the countries benefiting from the European Neighbouring Policy (ENP). This latter opened a new partnership prospective, by intensifying political, security, economic and cultural relations with the EU. In November 2006, an Action Plan (AP) for Georgia was signed which covers a 5 year timeframe, and its implementation contributes to deepen economic integration and political co-operation with the EU. In 2014 it will be replaced by the Association Agenda.

Supporting the implementation of education reforms is one of the priorities under the ENP AP. The long-term impact according to the Country Strategy Paper (CSP) under the heading "5.2.3. Sub-priority 2.3: Education, science and people-to-people contacts/exchanges" aimed to improve education levels, access to jobs, economic competitiveness and social stability in Georgia. Its specific objective is to reform and upgrade education and training systems, including through exchange programmes, and to further Georgia's integration into the European Higher Education (EHEA) and Research Areas (ERA). Expected results were:

1. Improved quality, capacity and relevance to the needs of society and the economy of education, vocational education and training systems and their convergence with EU standards and practices, including greater participation in relevant exchange programmes;

2. Enhanced commercialisation of research results;

3. Greater capacity of research structures (human and material resources) with the focus on innovation and scientific excellence;

4. Stronger links between scientific and research communities in the EU and Georgia, including educational institutions and networks fostering business matchmaking in the context of trade support. An improved connection of the Georgian National Research and Education Network (GRENA) to the European backbone GÉANT2 brings benefit to the entire Georgian science and education sector.

Indicators of achievement were identified on the long-term impact level on social and economic development (unemployment rate, enrolment ratios, etc.), for specific objectives level it is:

a) Improved education and training systems (as demonstrated by studies assessing progress of reforms and convergence with EU standards and practices),

b) Positive independent reports of decreasing impact of corruption in the education system,

c) Increased numbers of Georgian nationals and education institutions participating in exchange programmes,

d) Greater integration of Georgia into the European Research Area (as demonstrated by a higher number of joint activities and projects, including Private-Public Partnerships in technology licensing and transfer).

The interventions supported in the area of education and training are implemented in close coordination with the Tempus programme, which is funded out of the ENPI-wide programme. The dissemination, exchange and enhancement of best practice in these areas need to be assessed and its impact on national and institutional level evaluated.

1.2.1 EU Support to the Reform of Higher Education in Georgia

Since 26 May 2005 Georgia joined the Bologna process and committed itself to the implementation of the respective principles into their legislation and university practice for becoming a full-fledged member of the European Higher Education area. As a long-standing partner of Georgia in the higher education reforms the European Union supported these efforts through its thematic programmes (TEMPUS, ERASMUS MUNDUS) as well as through bilateral technical assistance in a TWINNING project or piloting TACIS projects. This evaluation should assess the impact of all these programmes and projects on the national (legislation, policy) as well as institutional level (Ministry of Education and Science, National Agency for the Education Quality Enhancement and higher education institutions) and draw some lessons learned from the application of the different instruments.

1.2.1.1 Participation of Georgia in Tempus IV (2007-2013)

Since 1995 Georgia is participating in the TEMPUS programme. The main aim of the Tempus programme is to support the modernisation of higher education in Partner Countries outside the European Union. Georgia is currently involved in 48 projects, representing 33% of all accepted projects of the six Eastern European countries (Georgia, Armenia, Azerbaijan, Belarus, Moldova and Ukraine) to date under Tempus IV. The total budget committed so far for Georgia in Tempus IV (First to Sixth Call), of more than 17 million Euro. For the 5th call, the budget augmentation from 1.4 to 4.4 million Euros resulted in an increase in the number of selected projects from 5 to 12, involving a Georgian partner. Given the significant increase of budget allocation for the 6th and last Tempus IV Call the number of selected projects with Georgian participation grew further to 19 projects (The regional budget allocation for ENPI East increased approximately from 13 million Euro in 2010 to 30 million Euro in 2011 and to 42.5 million in 2012 for the on-going call).

The number of applications involving a Georgian partner is very high, representing significant 18% of all the applications involving partners from the Eastern partnership countries. There was a remarkable increase in the number of submitted applications for the 5th Call, from 49 to 70 applications. For the last call (deadline was in March 2013) the number of submitted applications involving a Georgian partner increased again to the impressive number of 103 applications out of which 12 proposals were coordinated by a Georgian applicant institution. The selection results for 6th call under Tempus IV that have been published on 22 October 2013 indicate 19 successful projects with involvement of Georgian higher education institutions.[6]

Besides all the quantitative increase of funded TEMPUS projects in Georgia it is interesting to learn about the qualitative impact on the policy level (Ministry of Education and Science and the National Agency for Education Quality Enhancement) as well as on the institutional level (higher education institutions) internally (governance structures, student involvement, curriculum reform, employability, internationalisation) as well as on the horizontal level (national and international cooperation, internationalisation of faculty, etc.). Of special importance is the role the National TEMPUS Office played in Georgia as an important intermediary between Georgian and EU HEIs.

Tempus has made significant contributions towards the reform and development of the higher education systems and institutions in Georgia. In particular, the programme has created a solid basis for the successful implementation of Bologna principles. Since 1995, Tempus projects in Georgia supported quality assurance, institutional and programme accreditation systems, university-industry links, lifelong learning, international relations, university management and curricular reform. Most of the Tempus projects in Georgia have been implemented in the field of curriculum reform. However, during Tempus IV, greater priority was given to governance reform projects. Two joint-degree Master programmes have been developed at the Tbilisi State University in the framework of Tempus projects: 'Social Work', with University of Sheffield Hallam (UK) and 'Medical Molecular Biology', with University of Westminster (UK). More joint-degree programmes are being developed by ongoing projects.

UPDATE ON THE PARTICIPATION OF GEORGIA IN TEMPUS IV (2007-2013)

|Applications (publication date) |First call |Second call |Third call |Fourth call |Fifth call |Sixth call |Total |

| |(01/2008) |(01/2009) |(11/2009) |(10/2010) |(11/2011) |(12/2012) |(all calls) |

|Number of applications involving the Eastern |210 |273 |222 |280 |374 |463 |1822 |

|European Countries (including Russian Federation) | | | | | | | |

|Number of applications in which a Georgian |37 |31 |35 |49 |70 |103 |325 |

|institution(s) was/were involved as partner (% from| | | | | | |(18%) |

|the Eastern European countries) | | | | | | | |

|Number of applications co-ordinated by a Georgian |4 |4 |8 |7 |10 |12 |45 |

|institution | | | | | | | |

|Selected projects |  |  |  |  |  | |  |

|Projects recommended for funding in the six Eastern|20 |15 |14 |15 |36 |46 |146 |

|European countries (not including the Russian | | | | | | | |

|Federation) | | | | | | | |

|Projects recommended for funding which included a |3 |4 |5 |5 |12 |19 |48 |

|Georgian institution (% from the six Eastern | | | | | | |(33%) |

|European countries) | | | | | | | |

|Projects recommended for funding which included a |0 |0 |1 |1 |2 |2 |6 |

|Georgian co-ordinator | | | | | | | |

|Selected Joint Projects (all priority themes) |3 |3 |4 |4 |10 |14 |38 |

|Selected Structural Measures (all priority themes) |0 |1 |1 |1 |2 |5 |10 |

|National Projects |0 |0 |0 |0 |1 |2 |3 |

|Multi-Country Projects |3 |4 |5 |5 |11 |17 |45 |

|Budget committed (M euros) |  |  |  |  |  | |  |

The introduction of quality assurance (QA) as a key element of the Bologna process both at national and international level has been supported by five Tempus projects in Georgia. Tempus also supported the development of National Qualifications Framework for Higher Education, through the Tuning project, “Application of Tuning Approaches in Georgian Higher Education System'. Considering the results achieved, interest to Tempus projects is growing and more and more Georgian higher education institutions, notably from the regions, are applying to Tempus, building national, regional and cross-regional networks (Central Asia, Southern Neighbouring Area) of higher educational institutions. At the moment 28 Georgian universities from 9 cities as well as the Ministry for Education and Science, the National Agency for Education Quality Enhancement, several NGOs, professional associations, the Georgian and Adjarian Tourist Agencies and others participate in 37 Tempus projects.

1.2.1.2 Participation of Georgia in Erasmus Mundus

Erasmus Mundus is a cooperation and mobility programme in the field of higher education that aims to enhance the quality of European higher education and to promote dialogue and understanding between people and cultures through cooperation with countries outside the EU. Erasmus Mundus (EM) provides scholarships to individual students, researchers, scholars or staff for studies or research/teaching carried out as part of an Erasmus Mundus project. It cannot provide scholarships for courses or research which are not part of Erasmus Mundus. Students from any country in the world are eligible for scholarships to take part in Erasmus Mundus Joint Courses (Action 1), organised by consortia of mainly EU universities. There are currently 138 Masters and 43 Doctorates offering offer EU-funded scholarships or fellowships to students and scholars.

71 Georgians Masters students, six EM scholars have been selected over the ten annual selections up to 2013. Three selections of Doctoral candidates have been made since 2010. So far, two Georgian fellows were selected to follow an Erasmus Mundus Joint Doctorate course. One should consider that the selection criterion for action 1 students is excellence and that these students are in competition with students from all over the world.

Erasmus Mundus Partnerships (Action 2) bring together HEIs from Europe on the one hand and from a particular region in the world on the other. Together the partnerships manage mobility flows between the two regions for a range of academic levels: undergraduate, masters, doctorate, post-doctorate – and for academic staff. [7]

838 Georgian in total took part in mobility offered through EM Action 2. Considered the size and population of the country, Georgia can be considered an active participant. The total number of places also depends on the budget available under each lot, which is an EU policy decision. Georgia’s participation has been boosted since 2011 with the allocation of an additional budget to neighbourhood countries. In 2013, students and staff from Georgia will receive 264 scholarships to attend a European higher education institution under nine new Erasmus Mundus partnerships selected for the EaP region in 2013.

We are especially interested in the institutional impact of the Erasmus Mundus Partnerships (Action 2), which are developing very well over the last years. It started with one consortium in 2008 and in 2011 a second consortium was selected. In 2012 finally nine consortia with Georgian HEIs were selected and in 2013 another nine, so that there are now 18 ongoing Erasmus Mundus Action 2 mobility programmes.[8]

1.2.1.3 Support to the Institute for European Studies at Tbilisi State University

In support of the Bologna process the establishment of an interdisciplinary Institute for European Studies at Tbilisi State University (IES) with links to regional universities in Batumi and Telavi a special TACIS pilot project (1 Mio. Euros) was launched in April 2006, with the aim to identify selected domains for professional skills (EU law, economics, history and political science) and research, elaborate most consistent and pertinent training courses in EU Studies, train university teachers in the selected domains as well as the administrative body in coherence with the principles of the Bologna Process, define criteria for the selection of students, indicators for academic quality assurance and approaches to assessment and elaborate Master’s degree in European Studies. Another 300,000 Euros were allotted to establish a library and necessary pedagogical support in terms of hard- and software. The project implementation was finalized in December 2009.

To build up the Georgian graduates’ professional skills in research in order to enable better and reflected understanding of European Union policies and to improve relationships with the European Union in December 2012 an additional project "Establishment of a PhD programme in European Studies at Tbilisi State University" (450,000 Euros) was launched via a direct grant agreement with the management of the university. The ongoing project shall increase the research capacity at the IES on European Union (e.g. research papers, scholarly articles in peer-reviewed journals) and EU expertise nationally and internationally through joint research projects or other forms of cooperation with foreign universities. Secondly, the project aims to provide PhD students with an opportunity to apply their knowledge and skills in contributing to further approximation of Georgia towards EU in different fields relevant to the Georgian labour market. Thirdly, the project will help to update and improve the syllabi and contents of the existing IES MA programme and have a tangible impact on marketing and teaching activities of IES. The PhD students will be actively involved in teaching courses and providing a better understanding of research design through colloquia. Finally, the project will help to introduce the Bologna principles in the Georgian third cycle (PhD programmes) in practical terms. Thus it will enhance the overall interdisciplinary research capacities at Tbilisi State University and contribute to the planned establishment of a Graduate School of Excellence.

1.2.1.4 Participation of Georgia in Jean Monnet

The Jean Monnet programme aims at stimulating teaching, research and reflection in the field of European integration studies at the level of higher education institutions within and outside the European Community. European integration studies are defined as the analysis of the origins and evolution of the European Communities and the European Union in all its aspects. They cover both the internal and external dimension of European integration, including the European Union's role in the dialogue between peoples and cultures and the European Union's role and perception in the world.

Launched in 1989, the programme is now present in 72 countries throughout the world. Between 1990 and 2011, the Jean Monnet Programme has helped to set up approximately 3,700 projects in the field of European integration studies, including 165 Jean Monnet European Centres of Excellence, 879 Chairs and 2,139 permanent courses and European modules. These projects bring together 1.500 professors, and reach approximatively 500.000 students every year. Since 2001, the Jean Monnet Action operates on a world-wide basis. Higher education institutions from all countries in the world are eligible and encouraged to apply for Jean Monnet projects such as the creation of Jean Monnet teaching Modules, Jean Monnet Chairs, Jean Monnet Centres of Excellence and Jean Monnet conference projects on the European Union.

At this moment, we have a Jean Monnet chair financed in 2008 in European Law at the Tbilisi State University. However the chair holder Prof. Gaga GABRICHIDZE has left the University and the Chair is vacant waiting the Rector to appoint a new coordinator. No projects received in 2012.

For the 2013 call, two proposals have been filed by Georgian institutions/associations and the following has been selected and will be funded:[9]

The Association European Studies for Innovative Development of Georgia - Oleg Shatberashvili has filed a project for organising a conference and events on the theme: EU Regional Innovation Policy as a model for Eastern Partnership Country Regions. The Association European Studies for Innovative Development of Georgia (ESIDG) gathers more than 40 professors and researchers from a number of universities and research organizations who have recently taken part in public debates on the Research, Development & Innovation system reform. The Association’s objective is to foster the European integration process of Georgia and in particular the elaboration of an innovative policy and the introduction and development of the methods of state regulation and research in industry, education and other spheres.

1.2.1.5 Capacity Enhancement for Implementing the Bologna Action Lines

Between June 2009 and March 2011 the Ministry of Education and Science (MoES) received support through the first Twinning project implemented in Georgia: "Capacity Enhancement for Implementing the Bologna Action Lines in Georgia" (CEIBAL). In close cooperation with European partners this Twinning project intended to support Georgia’s Higher Education System and its key institutions in increasing the effectiveness of implementing the Bologna principles for its better access into the European Higher Education Area (EHEA). The project addressed the important dimension of the institutional and professional capacities of the MoES of Georgia and other key institutions, namely the National Agency for Education Quality Enhancement (NAEQE). Through consultations, working groups, training courses, visits of short term experts' diverse but interrelated issues like strengthening of the policy dialogue, legislative framework, human and administrative capacities were addressed. It should have enabled the MoES to capitalise new expertise and ensure the harmonisation with European standards and requirements. The CEIBAL project represented a first example of coordinated action among relevant stakeholders in Georgia.

1.2.1.6 Marie Curie Actions

All Marie Curie Actions (MCAs) of the PEOPLE specific programme (FP7) are open to the international dimension under different forms and different eligibility conditions for researchers and organisations. Almost a third of the PEOPLE programme budget (FP7) is allocated to the "International dimension" through the "World Fellowships" i.e. International Outgoing Fellowships (IOF), International Incoming Fellowships (IIF) and the International Research Staff Exchange Scheme (IRSES). In addition, the host driven actions, namely Initial Training Network (ITN) and Industry-Academia Partnership and Pathways (IAPP) are also open to the participation of third country research organisations and researchers, as well as the Co-funding of national, regional or international fellowships (COFUND). Third country nationals can participate in all of the 8 Marie Curie Actions in accordance with the specific application requirements. The participation of third country organisations is further a mandatory requirement for two of the MCAs: International Research Staff Exchange Scheme (IRSES) and International Outgoing Fellowships (IOF). So far in FP7, 7 Georgian researchers have benefited from a Marie Curie Action, 11 organisations located in Georgia participated, receiving funding of € 1.1 million.

1.2.2 Vocational Education and Training

Though the system of professional education was relatively well functioning during the Soviet times it quickly crumbled after the Union’s collapse and introduction of a market economy in newly emerging democracies, and among them, in Georgia. The Government of Georgia (GoG) started reforms in the Vocational Education and Training Sector (VET) in 2005-2007 through adoption of a new legal basis and investments in part of public VET infrastructure. The National Medium Term VET Strategy 2009-2012 was approved in 2009. In the end of 2009 the EU and the GoG signed the Financing Agreement aimed at supporting the VET Sector in Georgia through a VET Sector Policy Support Programme (SPSP). The overall objective of SPSP is to enhance credibility and attractiveness of the VET system. SPSP covered the period 2009-2012 with a budget of Euro 19 million, composed of Euro 17 million disbursed as budget support and Euro 2 million as technical assistance, which came to an end in October 2013. The National Vocational Education and Training (VET) Strategy 2009-2012 adopted by the Government of Georgia in November 2009 was the basis for this Programme and a new strategy for the period 2013-2017 was approved in December 2013 thus paving the way for further support to this sector.

The overall objective of the Programme was to support development of the sector of vocational education and training (VET) in Georgia and to enhance the credibility and attractiveness of the VET system. The programme was designed to contribute to the establishment of important building blocks of a modern VET and systematise policies, but it was clear from the beginning that further sector support would be necessary in a follow-up phase to deepen the reform and consolidate its achievements. The programme will come to an end in December 2013 with the completion of the technical assistance. The final budget support disbursement took place in July 2013 for an amount of € 5.5 million. 98% of the total amount envisaged for the tranches was disbursed.

Initially, it was planned to phase out EU interventions in this sector in 2013, mainly due to lack of interest for genuine sector reform by the previous administration. However, since October 2012, the new government has demonstrated a strong renewed interest in VET and employment issues. It foresees a strong public role, thus reversing the trend towards privatisation of VET colleges. In particular, structural reforms are under way focusing on better synergies between the VET system and the needs of the labour market, improved labour market information and active employment initiatives. A National VET strategy for the period 2014 -2020 is expected to be approved in September/October 2013.

Against this background, it was proposed to use the EaPIC 2013 allocation (approx. € 22 million) for a new programme which tackles the nexus between VET, skills and the labour market, thus enhancing employment prospects and productivity. The new programme has just been launched and will propose a combination of budget sector support, technical assistance and a grant component.

Therefore an evaluation in the VET sector should pay attention on how the previous budget support programme including its technical assistance component contributed or contributes to the Government’s education reform programme and the three inter-related sector policies (Labour Market Strategy; Strategy for the development of Vocational Education and Training 2013-2020 and the Agriculture Sector Strategy 2012-2022), whose mutual goal is to eradicate poverty and stimulate sustainable and inclusive socio-economic development.

The MoES has reinforced its capacities for dealing with VET. The VET department was restructured and additional staff recruited. The First Deputy Minster is responsible solely for the VET that sows the importance of this sector for the government.

Therefore it would be helpful to have an evaluation of the EU support in the VET sector so far in line with the above mentioned indicators to look at the policy and institutional level for the impact of EU support in this sub-sector.

Continuous support on VET and adult education was provided to all relevant stakeholders in Georgia by the European Training Foundation (ETF). This is an EU agency that helps transition countries like Georgia since 2007 to harness the potential of their human capital through the reform of education, training and labour market systems in the context of the EU's external relations policy. The expertise of ETF in the preparation of the above mentioned SPSS programme was crucial, but it also ran smaller programmes e.g. focusing on adult education strategy development or policy learning, social dialogue in VET.

1.2.3 Civic Education

The third and last sub-sector, where the EU provided substantial support over the last few years, was in civic education mainly funded under the EIDHR programme. There were two major regional EIDHR projects on civic education: "Democracy Starts With You! – Improving Political Participation of Young People from Minority and Rural Areas in the South Caucasus Region" implemented by Konrad Adenauer Foundation in Armenia, Azerbaijan and Georgia (1,46 Mio. Euro for 42 months, October 2009 - April 2013); the other one "Regional Children Action For Participation" implemented by Safe the Children addressed children rights through peer group mobilisation (810,047 Euro for 18 months, December 2010 – June 2012). Two still ongoing projects under the Eastern Partnership regional culture programme are also dealing with educational issues at school level and are interacting with the Ministry of Education and Science of Georgia on school standards, curricula reform and teacher training for the history and civic education subjects. The first one is "Let‘s Talk about Films in South Caucasus" implemented by the Czech Agency People in Need (653,470 Euro for 24 months, finishing in February 2014). Its overall objectives are 1) to strengthen regional linkages and communication in the South Caucasus region and 2) to make practical use of documentary film as a tool for fostering democratic dialogue in the region. The other project, "Strengthening Capacities in Cultural Education" (700,000 Euros for 36 months, December 2011 – December 2014), brings together history, heritage and citizenship education professionals from Armenia, Azerbaijan, Georgia, Moldova and Ukraine aims at raising awareness for innovative and responsible approaches in cultural education, which enhance democracy and intercultural dialogue, creating sustainable national and cross border networks and strengthening the professional capacities of their individual members. In addition there were several smaller projects that addressed educational issues on the local as well as the policy level under the European Instrument for Democracy and Human Rights (EIDHR), whose impact should be measured in a consolidated way.

1.2.4 Educational elements in capacity building instruments

Several Twinning projects implemented under the ENP Instrument that were completed so far had educational elements. For instance, the Twinning with the National Bureau of Enforcement (NBE) contained the drafting of textbooks and other work with academia. Another Twinning project between the Georgian National Museum and the Prussian Cultural Heritage Foundation worked on the elaboration of educational materials for their exhibitions. The given evaluation should assess the efficiency of such kind of educational elements in institution building instruments, their consistency with other educational programme approaches and draw some key lessons and recommendations from them, in order to improve current and future actions in line with the lifelong learning approach.

2. DESCRIPTION OF THE ASSIGNMENT

The main objectives of the evaluation are to provide the relevant external co-operation services of the European Union, the partner government and, when appropriate, the wider public with:

– an overall independent assessment of the past performance of the interventions in the above mentioned educational sectors (higher, civic as well as vocational education and training), paying particularly attention to the consolidated impact of the results of different support instruments and methodologies against the objectives of the CSP 2007-2013;

– key lessons and recommendations in order to improve current and future actions, esp. on how bilateral, regional and thematic EU cooperation programmes can create synergies to increase its impact on the structural level as well as on how the efficiency of educational elements in EU institution and capacity building actions can be improved.

Primary users of this evaluation will be the cooperation partners in Georgia, the Ministry of Education and Science, its executive agencies and educational stakeholders on the one hand and the EU Delegation in Georgia, DG Development and Cooperation and DG Education and Culture for the European Union. The evaluation should help to identify best practices from the past cooperation relying on a wide variety and provide recommendations on support instruments for future programming.

Secondary users are all the educational stakeholders that are planning to contribute to the cooperation in the field of education between Georgia and the EU e.g. in the programmes of ERASMUS+ for higher education institutions or eTWINNING at school levels. It includes also civil society actors monitoring and contributing to the development of the educational sector in Georgia including teachers associations and trade unions.

2.1. Scope

2.1.1 Legal scope

The overall engagement with Georgia should be taken into consideration in the analysis, including agreements, the co-operation framework and any other official commitments. This includes notably: the Partnership and Cooperation Agreement (PCA), EU – Georgia ENP Action Plan, the Eastern Partnership road map and Platform 4 Work Programme and the following instruments: The European Neighbourhood and Partnership Instrument (ENPI), the Instrument for Stability (IfS), the European Instrument for Democracy and Human Rights (EIDHR) and different thematic programmes under the Development Cooperation Instrument (DCI).

Interventions funded by the European Commission Humanitarian Office (ECHO) and/or the European Investment Bank (EIB) are not part of the evaluation scope. However, the coherence and complementarity between these interventions and the EU cooperation strategy evaluated must be examined and reported, if there are special points observed.

Changes in the European Union institutional set-up with the creation the European External Action Service (EEAS) should be taken into account only, if they are relevant for cooperation in the education sector.

2.1.2 Temporal scope

The evaluation covers the European Union's co-operation strategy with Georgia and its implementation during the period 2007-2013 in the sector of education.

2.1.3 Thematic scope

The evaluation should take into account the education sector in bilateral cooperation with Georgia as outlined above under 2.2. However, in order to keep the scope of the evaluation manageable, the evaluators should focus their analysis on the following areas of cooperation:

• Higher Education (implementation of Bologna principles in Georgia);

• Vocational Education and Training (implementation of VET reform strategy and its contribution to employment);

• Civic Education (contribution to the development of civic awareness and human rights).

The main reasons why these three sub-sectors above have been prioritised is that they are likely to remain an important focus of the 2014 – 2020 programming of EU bilateral, regional and thematic cooperation with Georgia.

The evaluators are also requested to assess the coherence and coordination between the support provided under the bilateral areas of cooperation with Georgia on one hand and the support provided through the regional and thematic components on the other. They should in particular assess if the different types of cooperation are/were mutually reinforcing each other, and if not, why this is the case.

Based on the evolving EU cooperation framework, political and policy dialogue in these areas should be carefully assessed. The State Ministry on European and Euro-Atlantic Integration, which is facilitating the EU-Georgia cooperation for the Government of Georgia, will be closely involved in the evaluation exercise and represented in the Reference Group.

2.2. Issues to be studied / Evaluation Questions

The evaluation has to be strategic. It should therefore assess whether and to what extent the various projects and programmes financed, did contribute or are still contributing to the achievement of the strategic objectives of the EU Cooperation with Georgia related to education. Therefore, the evaluators are not asked to produce a series of small scale project or programme evaluations and they should keep in mind the strategic focus of the evaluation.

The evaluation must identify the factors that hindered and those that helped the effectiveness of the EU cooperation in the past, in order to be able to draw lessons and provide recommendations that will inform the design of new programmes. The identification of influential structural factors will be of crucial importance.

The evaluation will assess the project/programme using the standard 5 DAC evaluation criteria, namely: relevance, effectiveness, efficiency, sustainability and impact. It will focus on the impact of the diverse EU support measures at national level on the Ministry and its executive agencies in terms of policy formulation, strategic management and organisational development on the one hand and on the institutional level on major Georgian higher education institutions, VET centres and other education providers and NGOs for their participation and implementation of quality education. The inclusiveness of those processes on national and institutional level will be of interest.

In addition, the evaluation will assess

– the added value of the EU projects/programmes, both regarding its design and quality of implementation in the field of education;

– the coherence of the projects and programmes, with the EU strategy for the field of education in Georgia and with Member States' and other donors' interventions;

– to what extent have technical assistance and capacity building actions in Georgia contributed or been replicated in other programmes or sectors (e.g. the application of results from TEMPUS programme under ERASMUS MUNDUS programme);

– to what extent EM mobilities were granted in a transparent way thus contributing successfully to encourage for an increased exchange and development of the candidates (e.g. best practices of announcement and selection on university and inter-university level) as well as the extent of recognition of their studies abroad (and vice versa of EU citizens in Georgia) or did the EM selection process contribute to the establishment of transparent selection procedures at Georgian HEIs;

– to what extent the participating universities increased the cooperation with their partners into other fields;

– to what extent were vulnerable groups (minorities, IDPs) participating in one of those programmes, projects;

The EU supported the education reforms on national, institutional and sectorial levels to contribute to a greater compliance with the respective EU policies in the fields of higher education (Bologna principles), VET (strategy, curricula), Life Long Learning and also provided various capacity building actions for state and non-state actors in the educational sector.

The evaluation team should also consider whether the following cross-cutting issues like gender equality, inclusiveness (minorities, socially, physically or mentally disadvantaged), capacities for policy dialogue and policy formulation as well as transparency and accountability were taken into account in the identification documents and to which extent they have been adhered to during the implementation of the projects and programmes concerned and afterwards.

The evaluation criteria are translated into specific evaluation questions. These questions to be studied are indicative; the Reference Group may discuss them with the experts during the Inception Phase. However, once agreed the evaluation questions are contractually binding. Here is a preliminary selection of potential evaluation questions:

1. To what extent did the EU support to the education reforms in Georgia contribute to the policy formulation and strategy elaboration at the level of the Ministry of Education and Science of Georgia? What where the main factors that helped/ hampered these processes?

2. To what extent did the EU support contributed to improve the capacity of educational entities like the Ministry, different executive agencies for education quality enhancement, curricula development or teacher training? What where the main factors that helped/ hampered these processes?

a) In how far has the support / cooperation contributed to enhance the skills and qualifications of higher education staff working on the institutions’ and in particular in the external relations offices so that they can contribute actively towards improvement of quality of institutional services?

b) In how far has the support / cooperation contributed to build the capacity of the administration and public and private sector by participation of their staff in higher education mobility activities?

c) In how far have the relevant actors, involved in the decision making process and/or the implementation of the reforms, benefited from the support, training and mobility actions? (the issue of turnover in public administration should be considered)

3. Did the beneficiaries use the EU support in a strategic way for the further development of their organisation or institution (Ministry, agency, HEIs, VET centres, etc.)? If so, why is that? Did they have had sufficient absorption capacity for cooperation and sustainable implementation during and after the end of the project or programme?

4. What were the interventions assessed by the beneficiaries as the most effective ones in supporting education reform in Georgia? Why do they think this is the case? What do they recommend for future cooperation with the EU?

5. Did TEMPUS and ERASMUS MUNDUS programmes support an increased and sustainable cooperation in the area of higher education between Georgia, the EU and other TEMPUS members?

• How do the partners assess the cooperation, exchange of know-how and best practices between sending and hosting institutions / EU and Georgian HEIs?

• Have these programmes provided academics, decision-makers and other professionals in Georgia with opportunities generated by the development of networks of EU and Georgian higher education institutions and resulted in teaching and research reforms / modernisation?

• Have these programmes contributed to improve the transparency and recognition of studies and qualifications, in particular building on the achievements under the Bologna process in this area?

• Have these programmes contributed to develop transparent selection procedures?

• Have these programmes contributed to enhance the international cooperation capacity of universities in Georgia?

14. How did the management of Georgian HEIs apply the assistance provided for their own institutional or organisational development along Bologna principles?

15. To what extent has the EU support to the VET sector reform contributed to increased quality and recognition of professional education?

16. Is there an increased number of applications, awareness of national and EU policies or strategies and educational methodologies and concepts by educational stakeholders (university staff, teachers, students, NGOs, etc.)? [10]

17. What would be the most important systemic impact the EU provided so far to the educational development in Georgia according to the educational stakeholders?

The methodology used will be based on the DEVCO Evaluation methodology for projects/programmes as set out on the website



(See for example "The main achievements of the Tempus Programme in Eastern Europe, 1993-2013, Issue 16 – July 2013 Report, ).

The basic approach to the assignment consists of three main phases, and encompasses several methodological stages. Deliverables in the form of reports[11] and slide presentations should be submitted at the end of the corresponding stages.

All reports will be written in English. Only the executive summary must be translated in Georgian. The reports must be written in Arial or Times New Roman minimum 11 and 12 respectively, single spacing. The Inception report will be delivered only electronically. The draft final and the final report will also be delivered in hard copies The executive summaries as well as the photo (free of any copy right, free of charge) used on the cover page will be delivered separately in electronic form. The electronic versions of all documents need to be delivered in both editable and not editable format.

The table below summaries these links:

|Evaluation phases: |Methodological stages: |Deliverables[12]: |

| |Inception: Structuring of the evaluation |Slide presentation |

|1. Desk phase | |Inception report |

| |Data collection analysis |Formulation of hypotheses |

|2. Field phase (Mission to |Data collection |Slide presentation |

|Georgia) |Verification of the hypotheses | |

| |Preliminary findings | |

|3. Final report phase |Analysis |Draft final report |

| |Final Judgements |Slide presentation |

| | |Final report |

The desk phase

The desk phase comprises two components: the inception stage covering a presentation and the delivery of the inception report and a second stage which requires a preliminary document review.

Presentation of the Intervention Logic & Evaluation Questions (inception meeting)

The assignment will start with the team leader's mission to Tbilisi for a briefing session.

After that, the contractor shall prepare a slide presentation including logical diagram(s), the evaluation questions and when possible judgement criteria.

For this presentation, the main work consists in:

➢ Identifying and prioritizing the key co-operation objectives in the education sector, as observed in relevant documents regarding the European Union’s co-operation with Georgia and translate these specific objectives into intended results.

➢ Reconstructing the intervention logic of the EU in the framework of its co-operation with Georgia. The reconstructed logic of the EU intervention will be shaped into one logical diagram and accompanying narrative. The reconstructed intervention logic should include:

o the whole logic from intervention rationale through to impact;

o to the extent possible, the assumptions made at the time of decision-making, revealing why x was foreseen to lead to y;

➢ Defining the Evaluation Questions. The reconstructed intervention logic should help to identify key evaluation questions. This intervention logic and evaluation questions must be elaborated in close consultation with the members of the reference group and with the persons that have been involved in the programming exercise. A first set of indicative evaluation questions is included in these terms of reference (section 4.2). In light of the intervention logic, the evaluators should propose adjustments to these questions.

The contractor will carry out a preparatory visit to Georgia with the following main objectives:

1) To discuss main motivations of the evaluation with key stakeholders; and

2) To clarify/reconstruct the faithful intervention logic;

3) To discuss the focus of the evaluation questions. This visit should not exceed one week.

An inception meeting will be held with the reference group in Tbilisi at the end of this visit to discuss via a slide presentation:

– the draft reconstructed intervention logic;

– the evaluation questions (and when possible, judgement criteria).

During that meeting an agreement on their content (intervention logic and evaluation questions) should as far as possible be reached.

The Inception report

Taking into account the outcome of the Inception meeting, the contractor must deliver an inception report which should contain the following elements:

• the education background/context and the key features of the cooperation context between the European Union and Georgia in the field of education. This part should be concise and focus on the context features that have an implication on the proposed evaluation focus;

• the intervention logic of the European Union's cooperation in the sector of education with Georgia (diagrams + narrative spelling out the rationale of the cooperation in those areas and to the extent possible, any assumptions made during the programming phase);

• the validated evaluation questions (upon validation by the EU Delegation assisted by the DEVCO evaluation unit, the evaluation questions become contractually binding); a limited number of appropriate judgment criteria (sub-questions) per evaluation question and a limited number of quantitative and/or qualitative indicators related to each judgment criterion;

• a proposal outlining a concrete strategy for data collection and analysis, indicating any limitations; this proposal should clearly indicate the approach the evaluators intend to use in order to respond to each of the evaluation questions (i.e. by collecting which data and by applying which analytical tools?)

• a detailed work plan for the next phases.

If necessary, the report will also suggest modifications to contractual provisions inter alia for the following points:

• the final composition of the evaluation team; and

• the final work plan and schedule.

Field phase

The fieldwork shall be undertaken on the basis set out in the Inception report. The work plan and schedule of the mission will be agreed in advance (in principle at least three weeks before the mission starts). If it appears necessary to substantially deviate from the agreed fieldwork approach and/or schedule, the contractor must ask the approval of the Evaluation manager before any changes can be applied. At the conclusion of the field mission the contractor will present the preliminary findings of the evaluation to the Reference Group at the EU Delegation to Georgia, during a de-briefing meeting with respective DGs from Brussels involved in educational cooperation via video conference with the support of a slide presentation.

Final Report phase

The contractor will submit the Draft final report in conformity with the structure set out in Annex 2. Comments received during de-briefing meetings with the Delegation and the Reference group must be taken into consideration. The draft final report will be discussed with the reference group during a half-day meeting in Tbilisi. Following the meeting with the Reference group, the contractor will make appropriate amendments to the draft final report based on the consolidated comments sent by the evaluation manager.

The in-country seminar

The approved draft final report will be presented at a seminar in Georgia, Tbilisi, using a slide presentation. The purpose of the seminar is to present the findings of the evaluation to the national authorities, the Delegation and other stakeholders (EU Member States, representatives of civil society organisations, other donors etc.) and obtain their reactions.

For the seminar, 30 hard copies of the main report (see annex 2 of the ToR) have to be produced and delivered to the EU Delegation in Georgia (the exact number of reports and delivery date will be specified by the evaluation manager). The electronic version of the report (including the annexes) will be provided to the evaluation manager.

The contractor shall submit the minutes of the seminar. These minutes and the updated slide presentation will be included as an annex of the final report.

The final report

The contractor will prepare the final report taking into account the comments expressed during the seminar. The Final report must be approved by the evaluation manager before it is printed. The executive summary should be translated in (Georgian) and included into the final main report.

30 hard copies of the final main report (without annexes) as well as 2 copies of annexes must be sent to the EU Delegation. An electronic support (CD-ROM) should be added to each printed final main report (PDF format).

Besides the provisions of the article 7.5 of the Global Terms of Reference, the quality of the final report will be assessed by the evaluation manager of the EU Delegation using the "Quality Assessment Grid" (see annex VII-h) to be sent to the contractor for information before publication.

The evaluation is managed by the Delegation of the EU to Georgia, Operation Section with the assistance of a Reference group consisting of members of DG DEVCO Evaluation Unit, B.4 and DG EAC C.4 as well as one representative from the State Ministry on European and Euro-Atlantic Integration of Georgia as the main interlocutor of EU-Georgia cooperation for the Government of Georgia under the coordination of the Monitoring and Evaluation manager at the Delegation, who oversees the evaluation on behalf of the Commission.

The reference group member's main functions are:

• To facilitate contacts between the consultants and the services.

• To ensure that the evaluation team has access to and has consulted all relevant information sources and documents related to the project/programme.

• To define and validate the Evaluation Questions.

• To discuss and comment on notes and reports delivered by the evaluation team. Comments by individual group members are compiled into a single document by the evaluation manager and subsequently transmitted to the evaluation team.

• To assist in feedback of the findings, conclusions, lessons and recommendations from the evaluation.

3. EXPERTS PROFILE or EXPERTISE REQUIRED

The evaluation team will consist of three members with a team leader to be an expert of category I (Senior), while the other two experts are Category II. All of them will have a relevant degree in higher education. The team composition should be justified. The team coordination and complementarity should be clearly described. The team is expected to possess expertise as listed below.

1. Team leader (Senior, Category I, 50 WD)

– University Master's degree relevant for this sector;

– Professional experience in the field of evaluation methods and techniques in general of at least 12 years. It is highly desirable that at least the team leader is fully familiar with the Commission's methodological approach (cf. Evaluation Unit’s website: ).

– Professional experience in the field of evaluation in education in the field of external relations or development cooperation will be considered an advantage.

– Team Management skills of at least 12 years in the field of education and/or evaluations;

– Professional expertise in at least one of the following fields: capacity building assessment, policy formulation and dialogue, strategic and organisational development of ministries of education in the fields of Higher education, Vocational Education and Training, and/or Civic Education in formal (schools) or non-formal environment;

– Working knowledge of the following language: English;

– The knowledge of the Georgian language as well as previous relevant expertise in Georgia will be considered an advantage.

2. Senior expert on higher education or VET (Category II, 50 WD)

– University Master's degree relevant for this sector;

– Professional experience in the field of evaluation methods and techniques in general of at least 6 years.

– Professional experience in the field of evaluation in education in the field of external relations or development cooperation will be considered an advantage.

– Professional expertise in at least one of the following fields: capacity building assessment, policy formulation and dialogue, strategic and organisational development of ministries of education in the fields of Vocational Education and Training;

– Working knowledge of the following language: English;

– The knowledge of the Georgian language as well as previous relevant expertise in Georgia will be considered an advantage.

3. Senior expert on civic education in formal or non-formal environment (Category II, 50 WD)

– University Master's degree relevant for this sector;

– Professional experience in the field of evaluation methods and techniques in general of at least 6 years.

– Professional experience in the field of evaluation in education in the field of external relations or development cooperation will be considered an advantage.

– Professional expertise in at least one of the following fields: capacity building assessment, policy formulation and dialogue, strategic and organisational development of ministries of education in the fields of Civic education in formal (schools) or non-formal environment;

– Working knowledge of the following language: English;

– The knowledge of the Georgian language as well as previous relevant expertise in Georgia will be considered an advantage.

The team members must be independent from the interventions evaluated. Should a conflict of interest be identified in the course of the evaluation, it should be immediately reported to the evaluation manager for further analysis and appropriate measures.

The team must have excellent writing and editing skills. The Contractor remains fully responsible for the quality of the report. Any report which does not meet the required quality will be rejected.

During the evaluation of the offers, the contracting authority reserves the right to interview by phone one or several core members of the evaluation teams proposed. For this purpose, the contractor could be asked to provide the contact details of the experts to be interviewed.

4. LOCATION AND DURATION

Location of the assignment will be Tbilisi with site visits to Kutaisi, Batumi and Telavi.

Provisional start and end of the assignment: April 2014 – October 2014

Maximum duration of the assignment: 150 calendar days.

The indicative timetable is as follows:

|Activity |Location |Indicative Duration |Indicative Dates |

|Inception phase | | | |

|Briefing Session |Tbilisi |1 WD |April 2014 |

|Reference Group (RG) Meeting – Kick off Meeting |Tbilisi |1 WD |April 2014 |

|Preparation of the inception report |Home Base |8 WD |April 2014 |

|Submission of the Report |Tbilisi |1 WD |April 2014 |

|RG Meeting |Tbilisi |1 WD |April 2014 |

|Field Phase | | | |

|Missions to Tbilisi/Georgia |Tbilisi |21 WD |May 2014 |

|Interviews of educational stakeholders (ministry, | | | |

|agencies, universities, VET centres, teacher unions, | | | |

|NGOs, schools, etc.) | | | |

|Visits of universities and VET centres in Georgia | | | |

|Debriefing |Tbilisi |1 WD |May 2014 |

|Synthesis Phase | | | |

|Drafting provisional final report |Home base |11 WD |June 2014 |

|Meeting with Reference Group- Presentation of |Tbilisi |1 WD |June 2014 |

|report | | | |

|Finalization report |Tbilisi |1 WD |June 2014 |

|Submission of the final report |Tbilisi |1 WD |June 2014 |

|Dissemination Phase | | | |

|Preparation of dissemination documents … |Tbilisi |1 WD |July 2014 |

|One dissemination seminar |Tbilisi |1 WD |July 2014 |

|TOTAL (maximum) |3x |50 WD | |

5. REPORTING

The reports must match quality standards. The text of the report should be illustrated, as appropriate, with maps, graphs and tables; a map of the project’s area(s) of intervention is required (to be attached as Annex). The consultant will submit the following reports:

| |Number of Pages |Main Content |Timing for sub-mission |

| |(excluding annexes) | |(See section 4 for |

| | | |timetable) |

|Inception report |10 pages |Evaluation questions |End of Inception phase |

| | |First findings of the study | |

| | |Encountered and anticipated difficulties | |

| | |Detailed evaluation approach & work plan | |

|Draft Final report |60 pages |Cf. detailed structure in Annex 2 |Final Report phase |

| | |Answer to the evaluation questions | |

| | |Synthesis of all findings & conclusions into an overall | |

| | |assessment | |

|Final report | |same specifications as above, incorporating any comments | |

| | |received from the concerned parties on the draft report | |

All reports will be in English using Font Arial or Times New Roman minimum 11 and 12 respectively, single spacing. Each report will be submitted first in electronic version as a draft. For each report/output, the contracting authority will submit comments within the period of 15 calendar days. The Final Report, incorporating any comments received from the concerned parties on the draft report, shall be submitted within 15 calendar days from the date of receipt of the comments. The evaluation team should provide a separate document explaining how comments have been integrated or the reason for non-integration of certain comments. The Final Report (final version) will be provided in 30 paper copies and in electronic version (with annexes on electronic CD-Rom). The executive summary will be translated into Georgian.

6. INCIDENTAL EXPENDITURE

➢ The offer will include three trips to Georgia (nine return flights) incl. per diems.

➢ The offer will include interpretation and translation services (from/to English and Georgian).

➢ The offer will include local travel from the capital for the three experts to visit Batumi, Kutaisi and Telavi (respective educational institutions like universities, VET centres, schools etc.).

➢ The offer will include a dissemination seminar with a meeting room for approximately 25 participants, accommodation for 6 of them (from regional educational institutions visited), travel arrangements for these six considering the cost of the travel e.g. from Batumi, Kutaisi and Telavi to the capital, catering for 25 participants in the form of standing buffet and non-alcoholic drink.

➢ In Georgia all EU-funded programmes and projects are exempted from VAT. The Framework contractor is invited to obtain information from the Contracting authority concerning reimbursement procedures or possible ways of exemption from VAT in Georgia in advance of the implementation.

7. MONITORING AND EVALUATION

The quality of the implementation of the evaluation contract as reflected in the final report will be assessed by the evaluation manager using the quality assessment grid given in Annex III to this contract.

Annexes I - III

Annex I: Information that will be provided to the evaluation team

Indicative list to be adapted / expanded where appropriate:

• Legal texts and political commitments pertaining to the projects and/or programmes in the sector of education

• Country Strategy Paper Georgia and National Indicative Programmes (and equivalent) for the period 2007 – 2013

• Governmental national and sector policy documents (e.g. Vocational Education and Training Development Strategy for 2013-2020. Ministry of Education and Science of Georgia, 2013)

• Project identification study

• Project feasibility study

• Project financing agreement and addenda

• Project’s Global and Annual Operational Plans

• Project’s quarterly and annual progress reports, and technical reports

• EC’s Result Oriented Monitoring (ROM) Reports, and eventual other external and internal monitoring reports of the projects and/or programmes

• Project’s mid-term evaluation report and eventual other relevant evaluations audit reports. The evaluation team should not repeat the points already covered by such documents but use them and go beyond them.

• Other sources of information, e.g. base-line surveys, specific studies or analyses of specific issues/groups, relevant country, sector, thematic and project evaluations, whenever available, works/supplies/services contracts, etc.

• Relevant documentation from national/local partners and other donors

• Relevant policy and planning documents from national/local partners and other donors

• Education, Audiovisual and Culture Executive Agency. Overview of the Higher Education Systems in the Tempus Partner Countries: Eastern Europe, A Tempus Study, No 11, November 2012, EACEA, Brussels.

• Education, Audiovisual and Culture Executive Agency. 'Human Resource Management in Public Higher Education in the Tempus Partner Countries', A Tempus Study, No 10, June 2012, EACEA, Brussels.

• Education, Audiovisual and Culture Executive Agency. 'State of Play of the Bologna Process in the Tempus Partner Countries (2012)', A Tempus Study, No 9, April 2012, EACEA, Brussels.

• Education, Audiovisual and Culture Executive Agency. 'Regional Seminars on University Governance in the Tempus Partner Countries (2010-2011) - Conclusions', A Tempus Study, No 8, October 2011, EACEA, Brussels.

• Education, Audiovisual and Culture Executive Agency. The main achievements of the Tempus Programme in Eastern Europe 1993 – 2013, 2013,

• Education, Audiovisual and Culture Executive Agency. Erasmus Mundus in Eastern Partnership countries, September 2013

• Adult Education Association of Georgia (AEAG), Adult Education Strategy in the context of Lifelong Learning, AEAG, Tbilisi (January 2009)

• Economic Policy and Research Center (EPRC), Economic Security of Georgia: Challenges and the Best Ways to Address Them, Tbilisi (October 2012)

Geostat; World Bank;

NOTE: THE EVALUATION TEAM HAS TO IDENTIFY AND OBTAIN ANY OTHER DOCUMENT WORTH ANALYSING, THROUGH ITS INTERVIEWS WITH PEOPLE WHO ARE OR HAVE BEEN INVOLVED IN THE DESIGN, MANAGEMENT AND SUPERVISION OF THE PROJECT / PROGRAMME. RESOURCE PERSONS TO COLLECT INFORMATION AND DATA ARE TO BE SOUGHT IN THE EC SERVICES, IMPLEMENTING BODY AND / OR PUBLIC SERVICE IN THE PARTNER COUNTRY.

Annex II: structure of the Executive Summary & Final Report

The final report should not be longer than the number of pages indicated. Additional information on overall context, programme or aspects of methodology and analysis should be confined to annexes.

The cover page of the report shall carry the following text:

‘’ This evaluation is supported and guided by the European Commission and presented by [name of consulting firm]. The report does not necessarily reflect the views and opinions of the European Commission’’.

The main sections of the evaluation report are as follows:

Executive Summary

A tightly-drafted, to-the-point and free-standing Executive Summary is an essential component. It should be short, no more than five pages. It should focus on the key purpose or issues of the evaluation, outline the main analytical points, and clearly indicate the main conclusions, lessons to be learned and specific recommendations.

1. Introduction

A description of the project/programme and the evaluation, providing the reader with sufficient methodological explanations to gauge the credibility of the conclusions and to acknowledge limitations or weaknesses, where relevant.

2. Answered questions/ Findings

A chapter presenting the evaluation questions and conclusive answers, together with evidence and reasoning.

3. Overall assessment (optional)

A chapter synthesising all answers to evaluation questions into an overall assessment of the project/programme. The detailed structure of the overall assessment should be refined during the evaluation process. The relevant chapter has to articulate all the findings, conclusions and lessons in a way that reflects their importance and facilitates the reading. The structure should not follow the evaluation questions, the logical framework or the seven evaluation criteria.

4. Conclusions and Recommendations

4.1 Conclusions

This chapter introduces the conclusions of the evaluation. The conclusions should be organised in clusters in the chapter in order to provide an overview of the assessed subject.

A paragraph or sub-chapter should pick up the 3 or 4 major conclusions organised by order of importance, while avoiding being repetitive. This practice allows better communicating the evaluation messages that are addressed to the Commission.

If possible, the evaluation report identifies one or more transferable lessons, which are highlighted in the executive summary and can be presented in appropriate seminars or.

4.2 Recommendations

They are intended to improve or reform the project/ programme in the framework of the cycle under way, or to prepare the design of a new intervention for the next cycle.

Recommendations must be clustered and prioritised, carefully targeted to the appropriate audiences at all levels, especially within the Commission structure.

5. Annexes of the report

The report should include the following annexes:

• The Terms of Reference of the evaluation

• The names of the evaluators and their companies (CVs should be shown, but summarised and limited to one page per person)

• Detailed evaluation method including: options taken, difficulties encountered and limitations. Detail of tools and analyses.

• Intervention logic / Logical Framework matrices (original and improved/updated)

• Map of project area

• List of persons/organisations consulted

• Literature and documentation consulted

• Other technical annexes (e.g. statistical analyses, tables of contents and figures)

• Detailed answer to the Evaluation questions, judgement criteria and indicators (evaluation matrix)

Annex III: Quality assessment grid

The quality of the final report will be assessed by the evaluation manager using the following quality assessment grid where the rates have the following meaning:

|To be filled in by Evaluation Manager |Grade |Comments |

| |1 - Poor | |

| |2 - Acceptable | |

| |3 - Good | |

| |4 - Very Good | |

| |5 - Excellent | |

|1. Meeting needs: Does the evaluation adequately address the information needs | | |

|of the commissioning body? Does the evaluation deal with and respond to all ToR| | |

|requests? If not, are justifications given? | | |

|2. Relevant scope: Does the evaluation fully examine the project/ programme | | |

|rationale, cause-effect relationships, impacts, policy context, stakeholders' | | |

|interests, etc.? | | |

|3. Defensible design: Is the evaluation design appropriate and adequate to | | |

|ensure that the full set of findings is made accessible to answer the main | | |

|evaluation questions? Does the report point out the limitations, risks and | | |

|potential biases associated with the evaluation method? | | |

|4. Reliable data: To what extent are the primary and secondary data selected | | |

|adequate? Are they sufficiently reliable for their intended use? | | |

|5. Sound analysis: Is the analysis appropriate and systematic so that | | |

|evaluation questions are answered in a valid way? Are inputs from most | | |

|important stakeholders used in a balanced way? | | |

|6. Credible findings: Are the findings derived from the data and analyses? Are | | |

|interpretations and extrapolations justified and supported by sound arguments? | | |

|7. Useful recommendations: Are the recommendations consistent with the | | |

|conclusions? Are recommendations operational, realistic and sufficiently | | |

|explicit to provide guidelines for taking action? Are the recommendations | | |

|drafted for the different target stakeholders of the evaluation? Have the | | |

|recommendations a true added value? | | |

|8. Clear report: Is the executive summary relevant and concise? Is the report | | |

|well written, well structured and understandable by the different project’s | | |

|stakeholders? | | |

-----------------------

[1] EU Financial regulation (art 27); REGULATION (EC) No 1905/200; REGULATION (EC) No 1889/2006; REGULATION (EC) No 1638/2006; REGULATION (EC) No 1717/2006; COUNCIL REGULATION (EC) No 215/2008

[2] SEC(2007)213 "Responding to Strategic Needs: Reinforcing the use of evaluation"

[3] COM (2011) 637 final "Increasing the impact of EU Development Policy: an Agenda for Change"

[4]

[5]

[6] Details can be found here:

[7] Further information about the programme can be found on this webpage:

[8] Complete list of projects can be found here: or here for Georgia:

[9]

[10] Possible benchmarks could be: a) applications to calls (Tempus, Erasmus Mundus, Jean Monnet), b) actual exchanges (mobility scholarships) and partnerships with EU universities, c) sustainability of the partnerships / networks after the end of the projects, d) set up of new or updated curricula.

[11] fiwz|}ƒ¯°ºÀÌÍÎÏÐÑÜÝ9 óéÝÑŹ󭥠”Źʼn}q}fZRA h[h[CJPJaJnH tH

h€[pic]CJaJh§sÑh€[pic]5?CJaJh•Hh€[pic]CJaJh•Hh[C5?CJaJh•Hh€[pic]5?CJaJh•Hhã5îCJaJh•Hh[5?CJaJ h[5?haÓh[5For each Report a draft version is to be presented. For all reports, the contractor may either accept or reject through a response sheet the comments provided by the Evaluation manager. In case of rejection the contractor must justify (in writing) the reasons for rejection. When the comment is accepted, a reference to the text in the report (where the relevant change has been made) has to be included in the response sheet.

[12] The contractors must provide, whenever requested and in any case at the end of the evaluation, the list of all document reviewed, data collected and databases built.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download