List of Abbreviations - UNESCO-UNEVOC



The Diploma in Technical and Vocational EducationTracer StudyNovember 2008Research Team:Margaret Nare, FCTVEMontlenyane Selma-Robertson, FCTVETebogo Angel Moreetsi, FCTVEGeorge Herd, TAKlavs Dahl Christensen, TACandy Mbongwe, TASepako Seosenyeng, TATable of Contents TOC \o "1-3" \h \z \u 1List of Abbreviations PAGEREF _Toc215398751 \h 42Acknowledgements PAGEREF _Toc215398752 \h 53Introduction PAGEREF _Toc215398753 \h 63.1The Diploma in Technical and Vocational Education (DTVE) PAGEREF _Toc215398754 \h 63.1.1Aims of the programme PAGEREF _Toc215398755 \h 73.1.2Outcomes of the Programme PAGEREF _Toc215398756 \h 83.1.3The DTVE programme structure. PAGEREF _Toc215398757 \h 93.1.4The facilitation of the programme. PAGEREF _Toc215398758 \h 104Policy background PAGEREF _Toc215398759 \h 114.1Research Team PAGEREF _Toc215398760 \h 115Methodology PAGEREF _Toc215398761 \h 125.1Project Implementation Plan (PiP) PAGEREF _Toc215398762 \h 125.2Purpose and Objectives of the Survey PAGEREF _Toc215398763 \h 125.2.1The research areas PAGEREF _Toc215398764 \h 125.2.2Target Groups for the Data Collection PAGEREF _Toc215398765 \h 135.2.3The research questions PAGEREF _Toc215398766 \h 145.2.4Detailing the research questions PAGEREF _Toc215398767 \h 155.3Contact details of the former DTVE Students PAGEREF _Toc215398768 \h 155.4Development of Data Collection Instruments PAGEREF _Toc215398769 \h 165.5Development of Templates for Registration of Data PAGEREF _Toc215398770 \h 175.6Data Collection Procedures PAGEREF _Toc215398771 \h 175.6.1Data from DoE lecturers PAGEREF _Toc215398772 \h 175.6.2Data from former DTVE student-teachers not employed in brigades/TCs PAGEREF _Toc215398773 \h 175.6.3Data from brigades/TCs PAGEREF _Toc215398774 \h 175.7Quality Aspects PAGEREF _Toc215398775 \h 185.7.1Size of Samples PAGEREF _Toc215398776 \h 195.7.2Reliability PAGEREF _Toc215398777 \h 215.7.3Validity PAGEREF _Toc215398778 \h 216Data presentation PAGEREF _Toc215398779 \h 236.1Questionnaire for DoE lecturers. (Annex D) PAGEREF _Toc215398780 \h 236.1.1Relevance of the content of the DTVE PAGEREF _Toc215398781 \h 236.1.2The structure of the DTVE programme PAGEREF _Toc215398782 \h 256.1.3Portfolio Assessment PAGEREF _Toc215398783 \h 276.2Questionnaire for former DTVE student-teachers (Annex C) PAGEREF _Toc215398784 \h 296.2.1Relevance of the content of the DTVE and its use by DTVE graduates. PAGEREF _Toc215398785 \h 296.2.2The structure of the DTVE programme PAGEREF _Toc215398786 \h 336.2.3Mode of facilitation used in the DTVE programme. PAGEREF _Toc215398787 \h 356.2.4Quality of facilitation by DoE lecturers. PAGEREF _Toc215398788 \h 376.2.5DTVE student-teacher perception on portfolio assessment PAGEREF _Toc215398789 \h 396.2.6Effectiveness of the DoE support system for the DTVE programme PAGEREF _Toc215398790 \h 406.3Comparing responses from DoE lecturers with responses of the DTVE student-teachers. PAGEREF _Toc215398791 \h 446.3.1DTVE programme content PAGEREF _Toc215398792 \h 44Table 7 Average relevance ratings for content of modules DN01, DP01 and DP02 by DoE lecturers and DTVE student-teachers. PAGEREF _Toc215398793 \h 446.3.2DTVE programme structure PAGEREF _Toc215398794 \h 466.3.3DTVE portfolio assessment system PAGEREF _Toc215398795 \h 476.3.4DTVE programme support system PAGEREF _Toc215398796 \h 486.4Structured focus group interview: management and supervisors of DTVE graduates PAGEREF _Toc215398797 \h 486.5Structured focus group interview: learners taught by DTVE graduates PAGEREF _Toc215398798 \h 506.6Completing the DTVE programme PAGEREF _Toc215398799 \h 516.7Concerns raised. PAGEREF _Toc215398800 \h 547Conclusions and Recommendations PAGEREF _Toc215398801 \h 587.1Curriculum PAGEREF _Toc215398802 \h 58Integrated Key Skills (KS) PAGEREF _Toc215398803 \h 58Facilitation/Facilitators PAGEREF _Toc215398804 \h 59The Programme Structure/Portfolio Assessment PAGEREF _Toc215398805 \h 60Learning support to students PAGEREF _Toc215398806 \h 61Teaching Placement (TP) PAGEREF _Toc215398807 \h 628Literature Reference PAGEREF _Toc215398808 \h 648.1Exit Survey – DTVE Diploma in Technical and Vocational Education Programme PAGEREF _Toc215398809 \h 648.2BOTA survey PAGEREF _Toc215398810 \h 648.3Consultancy to Conduct a Tracer Study of Graduates of the Botswana Technical Education Programme, 2007 PAGEREF _Toc215398811 \h 65Table of tablesTable 1: DTVE Enrolment 2001 - 2008-11-247Table 2: DTVE modules and credit values9Table 3: Response rate19Table 4: Representation of categories among respondents20Table 5: Representation of cohorts among respondents20Table 6: Frequencies of responses on item 2.1636Table 7: Average relevance ratings for content of modules DN01, DP01 and DP02 by DoE lecturers and DTVE student teachers44Table 8: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers45Table 9: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers46Table 10: Comparing ratings of related to programme structure by DoE lecturers and DTVE graduates47Table 11 Comparing of average ratings on items related to portfolio assessment by DoE lecturers and DTVE graduates47Table 12 Colleges and brigades visited and number of respondents in the focus group interviews with management / supervisors of the DTVE graduates48Table 13: Colleges and brigades visited and number of learners participating in the focus group interview50Table 14: Data on withdrawal, supplementation, pass and fail. DTVE cohorts 1-4.52Table 15: Reasons for withdrawing from DTVE programme52Table 16: Pass/fail rates as percent of completers53Table of diagrams TOC \h \z \t "Quote" \c Diagram 1: Bar chart illustrating the relevance rating (scale 1 to 5) by DoE lecturersrs PAGEREF _Toc215398833 \h 24Diagram 2: RELEVANCE of module content as perceived by DoE lecturers PAGEREF _Toc215398834 \h 25Diagram 3: RELEVANCE of module content as perceived by DoE lecturers PAGEREF _Toc215398835 \h 26Diagram 4: Structure of the DTVE programme PAGEREF _Toc215398836 \h 27Diagram 5: Portfolio assessment mode and process as perceived by DoE lecturers PAGEREF _Toc215398837 \h 29Diagram 6: Relevance and use of modules compared. PAGEREF _Toc215398838 \h 31Diagram 7: Modules content relevance and use as perceived by DTVE graduates PAGEREF _Toc215398839 \h 32Diagram 8: Modules content relevance and use as perceived by DTVE graduates PAGEREF _Toc215398840 \h 33Diagram 9: Structure of the DTVE programme views of DTVE graduates PAGEREF _Toc215398841 \h 35Diagram 10: Facilitation modes used in the DTVE programme rating of PAGEREF _Toc215398842 \h 36effectiveness and frequency of use PAGEREF _Toc215398843 \h 36Diagram 11: Pie chart illustrating views on variety in facilitation PAGEREF _Toc215398844 \h 37Diagram 12: Pie chart displaying student-teachers views on constructivist, PAGEREF _Toc215398845 \h 38learner-centred facilitation modes used by DoE lecturers. PAGEREF _Toc215398846 \h 38Diagram 13 Former DTVE student-teachers views on the quality of their facilitators PAGEREF _Toc215398847 \h 39Diagram 14: Bar chart of average agreement ratings on items on portfolio assessment mode nad process as perceived by the former DTVE student-teachers PAGEREF _Toc215398848 \h 40Diagram 15: Bar chart of average agreement ratings on support related issues in the DTVE programme as perceived by former DTVE students PAGEREF _Toc215398849 \h 42Diagram 16: Average agreement ratings on items related to support during supplementation and support ro distance learners as perceived by former DTVE students PAGEREF _Toc215398851 \h 44List of AbbreviationsAPPAnnual Performance PlanATTCAutomotive Trades Technical CollegeBOTABotswana Training AuthorityBTEPBotswana Technical Education ProgrammeCTVECollege of Technical and Vocational EducationDoEDepartment of EducationDTVEDiploma in Technical and Vocational EducationDTVETDepartment of Technical and Vocational Education and TrainingFCTVEFrancistown College of Technical and Vocational EducationGTCGaborone Technical CollegeICTInformation and Communication TechnologyJTeCJwaneng Technical CollegeMTCMaun Technical CollegeNCCNational Craft CertificateNCQFNational Credit and Qualification FrameworkNDP #National Development Plan #ODeLOpen, Distance and eLearningOBEOutcomes Based EducationPaTeCoPalapye Technical CollegePiPProject Implementation PlanRNPERevised National Policy on EducationSPTeCoSelebi Phikwe Technical CollegeTATechnical AssistantTCTechnical CollegeTECTertiary Education CouncilTPTeaching placementTSTracer StudyTVETTechnical and Vocational Education and Training VBAVisual Basic ApplicationsAcknowledgementsThe tracer study group would like to express sincere gratitude to everyone who contributed to the collection of data for this study. We are highly indebted to the Brigade Coordinators and Principals of the Technical Colleges (TCs) who assisted by facilitating and organising the visits of the data collectors to their institutions. We especially appreciate all learners and staff who contributed by either completing the lengthy questionnaire - the former students on the Diploma in Technical and Vocational Education (DTVE), or who participated in a focus group interview - institutional managers and learners. We thank them all for their time and input.IntroductionThe main objective of this Tracer Study (TS) is to provide data and information to support a review of the Diploma in Technical and Vocational Education (DTVE) which is the responsibility of the Department of Technical and Vocational Education and Training (DTVET) and the management of the Francistown College of Technical and Vocational Education (FCTVE). The study is timely, in view of the significant changes envisaged for the Technical and Vocational Education and Training (TVET) sector of Botswana.DTVET will become responsible for the management and funding of the 41 Brigades to support the expansion and standardisation to the Botswana Technical Education Programme (BTEP) qualification framework. A significant staff development requirement will arise since many of the brigade instructors do not have formal teacher training qualifications and are unfamiliar with the BTEP.The Tertiary Education Council (TEC) is in the process of taking management and funding responsibility for the TVET sector with significant operational implications for the TCsThe National Credit and Qualification Framework (NCQF) is under development in an attempt to define and standardise the qualification levels of all education sectors in Botswana.National Craft Certificates (NCC) units are under reviewProposals for a certificate and an advanced certificate in Teacher training, within the BTEP framework, have been presented to DTVET. The study involved the tracing of former students of the DTVE programme from 2001 onwards to gather data on their experiences, both during the programme, and their subsequent involvement within the TVET sector. This report presents the data gathered, and suggests conclusions and recommendations.The Diploma in Technical and Vocational Education (DTVE)The DTVE is an 18 month, full-time teacher training programme offered by the Department of Education (DoE) at FCTVE. The entry requirement is normally a diploma or first degree in a vocational field. The programme is modularised, credit and outcomes-based.Until the end of 2007 The DTVE programme was originally offered by the autonomous institution – The College of Technical and Vocational Education (CTVE) located in Gaborone. The first cohort of students was enrolled in 2001. At the end of 2007, the college relocated to Francistown to become the DoE at FCTVE.The primary aims of the original college were defined by the RNPE Government Paper Number 2 of 1994 recommendation 62 (a), (see NDP 8 chapter 15, paragraph 15.138) as:to provide pre-service training for teachers / instructors in the TVET sector to provide in-service ,staff-development training for the persons working in the TVET sectorNDP 9 in section 15.122 describes the major task of the DoE at FCTVE asto provide lecturers with professional teaching skills using both full-time course and distance/e-learning deliveryCurrently, the only programme offered by the DoE is the DTVE full-time programme in affiliation with the University of Botswana (UB).Since 2001, the enrolment numbers are as follows:Table 1: DTVE Enrolment: 2001 - 2008Sept 2001 - May2003Cohort 1 July2003 - Dec2004Cohort 2Jan2005 – July2006Jan 2005 – May 2008Cohort 3Sept2006 - June2008Cohort 4TOTALFull time programmeFull time programmeFull timeBlock releaseFull time programmePre-service In-servicePre-serviceIn-servicePre-serviceIn- service123271828 Pre23 In15331715456650176*The cohort of 80 students enrolled in 2008 and currently studying is not included in the table and was not used to gather information for this TS.A total of 174 students, in 4 cohorts have started the DVET programme. *The above total is 176 - 2 students starting in cohort 2 left due to illness and continued in cohort 3.Aims of the programmeThe DTVE curriculum aims to provide opportunities for student-teachers to:a) become reflective and critical professional practitioners in the vocational education and training sector;b) acquire knowledge of relevant educational theory related to the system in which they already work or plan to enter following the programme;c) develop a wide range of strategies, tactics and expertise necessary for planning, preparing, implementing, assessing and evaluating teaching and learning sessions for the subjects and groups of learners or trainees with which they will be working;d) integrate in the facilitating of learning, studies relating to inter-personal relationships and the development of key skills associated with their intended or actual work role;e) acquire knowledge and skills to respond to the needs of their learners;f) identify, select and use a range of resources to support learning sessions, including ICT;g) acquire confidence in their professional knowledge and problem-solving skills, developing their own personal philosophy of education and commitment to and critical awareness of their professional situation; h) commence the development of a positive and professional attitude to the philosophy of life long learning and professional development;i) gain access to a ladder of awards, each of which matches their training and professional needs and those of their employers and learners;j) identify barriers faced by learners in education and training such as disability, age, race and gender and promote professional practice that does not exclude groups of learners but recognises and values their diversity;k) study teaching and training situations within the social, psychological and philosophical context and to relate these to changes in industry and to Government policy.Outcomes of the ProgrammeOn successful completion of the DTVE Programme the student-teacher will:a) exhibit non-discriminatory beliefs, values and behaviour, in respect to disability, race, gender, sexual orientation, culture or ethnicity in all written work, teaching practice, work based experience and their day to day involvement with the Programme tutors, peers, learners, and others with whom they work or study;b) have developed skills and understanding in relation to the use and application of information and communication technologies, sufficient to carry out their intended or expected role in technical/vocational education and training;c) have demonstrated appropriate skills in researching, analysing and using information gathered from a variety of sources such as libraries, internet, intranet, world wide web, CD-ROMs, newspapers, journals and educational texts such as syllabuses and unit specifications;d) be able to design, plan, implement, assess and evaluate learning programmes which take into account the diverse needs of the vocational learner or trainee and the specialist area in which they will teach;e) demonstrate appropriate skills and competence in relation to supporting their own learners, monitor progress and provide effective guidance which enhances learning;f) evaluate their own teaching effectiveness and establish their own development targets;g) understand the role of assessment in relation to the evaluation of learning, the vocational curriculum and the award of national qualifications related to their own teaching area;h) understand the factors which influence the development and structure of the vocational education and training curriculum and play an effective part in the planning implementation and evaluation of this curriculum, in relation to their own specialist teaching field;i) understand the role of vocational education and training in relation to the Botswana Government, economic, social and cultural policy, in relation to their own professional practice;j) be able to plan and execute work based projects aimed at enhancing the quality and effectiveness of the vocational curriculum, through appropriate levels of enterprise and collaboration with peers, mentors and managers;k) evaluate their own learning during the programme and contextualise this with their own values and beliefs in relation to their own professional role as a teacher or trainer;l) develop the knowledge, understanding and skills, needed for effective and unsupervised work in a wide range of contexts in vocational education and training.The DTVE programme structure.To achieve the above stated aims and outcomes an 18 month modular programme was designed and piloted with a group of 15 student teachers beginning September 2001. Teaching Placement (TP) at an vocational institution i.e. attachment to a brigade or TC offering BTEP, forms an important, integral part of the programme of which 60% is college based and 40% is TPThe programme structure consists of 10 modules. Each module has a credit value which is an indication of the time an average student-teacher will need to complete the module.1 credit corresponds approximately to 30 hours of study time. This includes contact hours with the tutor, with peers, study in library, work in computer room, individual study, etc. The table below gives an overview of the modules in the programme.Table 2: DTVE modules and credit valuesModuleCredit valueDN01 Assess learners’ needs6DP01 Plan for effective learning6DP02 Prepare resources to support learning6DF01 Facilitate learning8DS01 Provide learners with support4DA01 Assess learning4DA02 Evaluate learning sessions6DE01 Reflect upon roles of a teacher6DE02 Improve own professional practice6DE03 Undertake an action research8TOTAL60The complete programme carries a credit value of 60, being equivalent to approximately1800 hours of study. The following “key skills” or generic skills are fully integrated into the 10 modules:personal and interpersonal skills;information gathering and processing skills (Information and communication technology, ICT);communication skills;numeracy skills;quantitative data processing skills (numeracy);improving own professional performance.These skills are developed by ensuring they are applied to activities in each and every module. The Key Skill level of each student is assessed at start of the programme and, whenever necessary, student-teachers are assisted and provided with (online) resources to practice and upgrade their skills.The facilitation of the programme.The DTVE is a practical, activity-based programme founded on the general principles of adult education. Module activities and tasks are related the vocational area of specialisation of each student-teacher where possible. It is intended to be facilitated in a mode exemplary of that specified by the BTEP i.e. learner centred, based on a constructivist view of learning, supported by appropriate use of Information and Communication Technology (ICT).The evidence for the outcomes, again similar to some BTE programmes, requires portfolio building. The portfolios are assessed in line with the Quality Assurance and Assessment Unit (QAA) following the procedure: unit assessment, internal verification and external verification. The only difference from DTVE is that the external verification is not done by QAA but by UB. In line with all BTE programmes the DTVE programme has assessed work placement periods. DoE refers to these work placement periods as Teaching Placement (TPThe DTVE programme is flexible and individualised. Student-teachers, within limits, can determine the pace of their own progress and the pace at which evidence are gathered for each module portfolio. For this to be successful, an effective support and mentoring system is in place. A college based personal tutor is assigned to each student teacher to provide support and guidance the student-teacher throughout the programme. During TP each student-teacher is attached to a mentor from the institution concerned.Policy backgroundThe Government is committed to providing quality technical and vocational education and training NDP9, section 15.112). The emphasis is on equipping learners with the knowledge, skills and attitudes to contribute to the socio-economic and technological advancement of the country. The Government in its drive to diversify the economy seeks to create a competent, innovative and internationally competitive workforce. To provide quality education one needs quality teachers / instructors. DTVET’s Strategic Plan (2005 – 2009), FCTVE’s Strategic Plan (2007 – 2009) and FCTVE’s Annual Performance Plan, (APP) 2008-2009incorporated the Government’s objectives mentioned above. One of the initiatives mentioned in the APP, as a quality improvement initiative, is to conduct a tracer study of DTVE graduates. The DTVE programme has been running, with few changes, since its inception in 2001. A review of the programme is required to establish whether the stated aims and objectives are being achieved, and are still relevant to the current needs of DTVET and the vocational sector as a whole.In a meeting with DoE in February 2008 called by the FCTVE Management attended by the Technical Assistance (TA) Team, the FCTVE Management instructed DoE to run a tracer study (TS) of former students of the DTVE programme with the purpose of informing the review of the programme, which has been offered unchanged since September 2001. The DoE was instructed to present the TS report not later than 30 June 2008. At the same meeting the TA Team committed itself to support DoE in its undertaking of the TS by developing a plan for implementation of the TS including survey instruments and assistance with analysis and reporting. The TAs estimated the DoE staff input – in order to meet the deadline stated by the FCTVE Management - to the TS to be 2 lecturers ?-time for 3 months. It was also underlined by the FCTVE Management that the TA assistance should be seen as capacity building of DoE staff with the objective of enabling DoE staff to undertake future similar surveys. Carrying out surveys of tertiary education institutions, like FCTVE, is an expectation that has been expressed by Tertiary Education Council (TEC) members during their visit to FCTVE in relation to accreditation of the college and its programmesResearch TeamFollowing the meeting with the FCTVE Management, DoE nominated the following lecturers from their Department to be responsible for the planning and carrying out of the tracer study:Margaret NareMontlenyane Selma-RobertsonTebogo AngelaMoreetsiTo assist DoE the following TAs were nominated:Jan DeurwaarderGeorge HerdKlavs Dahl ChristensenThe three DoE lecturers and the three TAs formed the TS task force. The task force was in September 2008 joined by the following short term TAs:Candy MbongweSepakoSeosenyengMethodologyThis tracer study used quantitative as well as qualitative data. The main study is based on a survey questionnaire to two groups: 1. former DTVE student-teachers and 2.lecturers in the DoE at FCTVE. Qualitative data were collected through structure interviews with two groups 3.management and supervisors of the DTVE graduates employed in brigades and TCs and 4.Learners now being taught by the graduates. Interview guidelines were developed for groups 3 and 4. The established TS task force had its first meeting in March 2008. Following that, several meetings were planned. However, some meetings had to be cancelled due to high workload of the DoE members.A separate folder was created on the FCTVE Moodle platform to cater for the communication among the task force members. All reference material, ideas, suggestions and decisions were uploaded to the platform. Due to a high workload on the part of the DoE members of the TS task force, nearly all the work was carried out by the TAs. This meant that the request formulated by the FCTVE Management about capacity building among the DOE members of the TS task force did not occur.Project Implementation Plan (PiP)The first output from the task force was a PiP (Annex A), outlining tasks to be undertaken, responsibility and timeframe. The PiP took its point of departure in the request formulated by FCTVE Management in February 2008 that the survey report should be presented by June 30, 2008. This was due to the inability of DoE members of the task force being unable to meet the various deadlines and contributions stated in the PiP, resulting in a delays in completion of the TS report.Purpose and Objectives of the SurveyFollowing the development of the PiP, the task force focused on defining the purpose and objectives of the survey. The purpose of the survey was already defined by FCTVE Management as:Provision of information for review of the DTVE programmeThe research areasIn order to meet the stated purpose the task force identified the areas that should be the target for the survey.The relevance of the content of the DTVE programme for the teachers / instructors in order to competently deliver their vocational subject area should obviously be addressed in the survey. But the content should not be assessed in isolation. The content only becomes meaningful if the graduates are able and willing to implement the knowledge, skills and attitudes developed during their study. Consequently, the survey would also investigate to what extent the DTVE graduates were actually applying the skills and knowledge in their daily work as lecturers or instructors.The quality of any face-to-face programme as perceived by the learners also depends on the quality and mode of facilitation. This is especially the case, when the objective of the programme is to equip the student teachers-with planning, learning resource development, facilitation, assessment and evaluation skills, using an outcome based, and learner centred approach. The mode of facilitation of the DTVE programme should be exemplary (do what you preach). The learning resources and facilities available to the student-teachers are important in order to enhance their learning in a technological enhanced environment. Then, when in employment in colleges, they can act as examples of good teaching practice to colleagues on the use of technology in learning and facilitation.The assessment mode of the DTVE programme is through continuous portfolio building for each of the learning modules. Portfolios contain the evidence of having achieved the learning outcomes and document the process towards their achievement. This mode of assessment is unfamiliar to many of the DTVE student-teachers more used to end examinations. It is a mode of assessment that requires continuous support from the DoE lecturers and which serves at the same time as an example to them, as they must be able to use portfolio assessment when they join TCs teaching BTE programmes at advanced and diploma level.The support system in place for student-teachers was also identified as an area for evaluation, being a key aspect of any educational programme. The views of student teachers on the support system of the DTVE programme form an important aspect of the data collection. Views of those that completed the programme successfully (obtaining their DTVE), those that left before the normal end of the programme, and of those that completed the programme but failed to achieve the DTVE are all relevant for evaluation of the support system. The final research area identified, focused on what happened to learners on the DTVE programme after they completed or left the programme i.e. post course destinations. Did the newly qualified student-teachers find employment? What happened to those who failed or left the programme early?In conclusion it was decided by the TS task force that in order to meet the purpose of the survey that data should be gathered in the following seven areas:Content, its relevance and extent to which it is used by graduates, and programme structure Mode of facilitation of the programmeQuality of facilitationLearning resources and environmentMode of assessmentSupport systemPost course destinationsHowever, an issue was raised concerning item 4 above. With the exception of the groups, cohort 4 and cohort 3 (block release), none of the other graduates of the programme would have experience of FCTVE facilities, its learning resources and environment, since they completed their programme at CTVE in Gaborone. Cohort 3 (block release) and cohort 4 were at FCTVE for a relatively few months only, and during a time when the campus was not fully operational. Therefore, it was decided to exclude the above item 4 from the survey as student-teachers would not be in a position to give views on the facilities and resources a full operational FCTVE would offer.Target Groups for the Data CollectionDiscussions in the TS task force indicated that there was a need to collect data not only from those who completed the DTVE programme successfully – the graduates, but also those who did not. Consequently, the student-teacher population targeted consisted of all student-teachers enrolments to the DTVE programme since its inception. In this document this group is referred to, not quit appropriate, as DTVE graduates.Another group targeted to provide data, were current and former lecturers in the DoE (formerly CTVE). With their experience in facilitating the programme as well as their theoretical background they would be in a position to provide useful information related to the relevance and usefulness of programme content, its mode of facilitation and assessment, and the student support system.The purpose of the DTVE programme is to produce skilled teachers / instructors for the TVET system in Botswana. It can therefore be expected that former DTVE graduates will be working in a brigades, government TCs or other vocational training institutions as instructors/lecturers. In order to establish whether DTVE graduates use the knowledge, skills and attitudes acquired in the DTVE programme two groups were identified to provide data.a. learners in brigades/TCs currently taught by the DTVE graduates b. the management of the institution in which the graduates are working, including their immediate supervisorsThis data would focus on the relative performance of DTVE graduates compared with teaching staff with a non DTVE background.In summary, the target groups for the survey were:Former students enrolled on the DTVE programme since its inception (referred to as DTVE graduates)Lecturers from DoE (current and/or former)Learners in brigades/TCs currently taught by the graduatesSupervisors in brigades/TCs of the graduates The research questionsThe TS posed the following the questions:a. How relevant is the content of the DTVE programme to the work of facilitators of BTEP/ other vocational programmes within the vocational sector of Botswana?b. Do the graduates apply the skills and knowledge which they have developed?c. How do graduates perceive the structure of the programme (Duration, workload, level, sequencing)and the content of modules? Where the modes of facilitation used in the programme conducive to learning?b. How do those taught by the graduates rate the quality of their facilitation?c. How do managers/supervisors in brigades and TCs rate the quality of the graduates? a. How do graduates perceive the portfolio assessment system?b. How do DTVE facilitators perceive the portfolio assessment system?What are graduates views on the support received during / after the programme?a. What is the completion rate / pass rate?b. What are reasons for dropping out from the programme before completion?c. Where are graduates employed after completion?d. What is the time gap between completion and employment?Detailing the research questionsFor each of the research questions a key performance indicator (KPI) with a standard was formulated. The data source / respondents to supply data and the data collection instrument to be used to collect the data were identified. The person(s) responsible for collecting these data were identified. To ensure that the questions included in the data collection instruments address the objective a pro forma document “justifying the questions” was designed (Annex B). The DoE members were to complete these ‘justification’ documents with the Department as a form of validation that the correct questions were asked and ‘nice to know’ questions, not related to the set objectives were not included. Time constraints did not allow this part of validation of the questions to take place.Contact details of the former DTVE StudentsThe records of former students of the DoE were requested. However, the lists did not include the latest contact details for graduates or their completion status (passed with or without supplementation, failed). There was also no indication of DTVE student teachers that withdrew from the course before the normal end date. As graduates tend to be mobile, contact details current when the graduates left the DoE become quickly outdated. Those not in employment within the TVET system were therefore particularly difficult to trace. Considerable energy and time was therefore needed to trace each individual. Strategies for generating a complete list of DTVE graduates were as follows:Records of the DoE on the 4 cohorts were the starting point, though they were found subsequently to be incomplete. Each student teacher admitted to the DTVE programme receives a unique ID number. The list of former student-teachers submitted to the tracer study group had some IDs omitted.Admission records for cohorts 1 to 3 from a former DTVE lecturer were examined to supplement, where appropriate, the records by DoE management. The Deputy Principal resources at FCTVE (former HoD in the DoE) provided a fairly comprehensive for the student teachers in cohort 4.The above strategy produced a list of 174 graduates. The list has a few ‘missing IDs’ i.e. ID numbers without a name attached. However the missing IDs could be accounted for as it was found that for some cohorts (especially 3 and 4) IDs were given to all student-teachers offered admission to the DTVE programme. Some having been offered a place on the programme did take up the offer i.e. never started the programme resulting in ‘empty’ IDs. The TS working group is confident that all DTVE graduates were however properly identified. The next task was to trace the current location and contact details of the graduates.a. Strategy for tracing graduates working in the TCs or brigadesStaff records from the TCs and the database containing the data of all persons working in the brigades were used to identify the graduates currently working in these institutions. This allowed 116 (66%) of the graduates to be located, 77 in brigades and 39 in TCs.A snowball technique was used whereby graduates who had been located were asked for assistance in tracing other graduates from their year or other cohorts. A further 36 graduates were traced.22 graduates (13% of the total) could not be traced.In summary, a total of 152 (87%) graduates were traced and contacted.Development of Data Collection InstrumentsThe following data collection instruments were developed|:a. A questionnaire for former DTVE student-teachers (graduates). (Annex C)b. A questionnaire for former and/or current lecturers in the DoE facilitating the DTVE programme. (Annex D)c. Guidelines for a structured focus group interview with managers / supervisors of DTVE graduates. (Annex E)d. Guidelines for a structured focus group interview with learners currently taught by DTVE graduates. (Annex E)The steps in developing the instruments were the following:a. Documents detailing the research questions in each of the six areas included a section with draft / suggested questions that should be answered by the identified populations in order to collect data for answering the research question. . These 6 documents were made available to members of the TS workgroup for scrutiny and review, together with a pro forma checklist to justify each question (See annex B) suggested for inclusion in the data collection instruments. The workgroup members based in the DoE were tasked with presenting the documents within the Department and include Departmental feedback and review. This is a low level form of establishing the validity of the data collection instrument. No feedback was received and hence it was assumed that the documents could be used for the next stage without any change.b. The suggested questions in the 6 documents were collated into 4 separate documents: the first draft of the 2 questionnaires and the 2 guidelines for the structured focus group interview. These draft documents were made available, through the Moodle platform to the members of the TS workgroup for review and dissemination in the DoE, together with two checklists. The first checklist was to be used to check each questions, whether it was correctly formulated, short, simple language, not a leading question, etc. The second checklist addresses the layout and structure of the whole questionnaire e.g. where similar questions grouped together, does it start with relatively straight forward questions, etc. No feedback was received from the DoE, but feedback was received from TA members in the TS workgroup.c. The TA members of the group produced, based on the feedback, a final version of the documents. These were presented to the principal FCTVE for approval, and to the HoD of the DoE for discussion within the Department and approval. Approval was received from principal and HoD of DoE.Development of Templates for Registration of DataOnce the questionnaires were developed, templates for entering of the quantitative data were developed. In order to construct the two dimensional XY (Scatter) charts a Microsoft Visual Basic Applications (VBA) macro was developed.Data Collection ProceduresFour of the TS workgroup members were involved in the collection of the data working in pairs. To ensure that data were collected in the same way by each pair of data collectors, guidelines for the structured focus group interview with management/supervisors and learners of the DTVE student-teachers were developed (Annex E). Data collection visits to TCs and brigades were structured in the same way to increase the reliability of data collected. Data from DoE lecturersThe developed and approved questionnaires (Annex D) were distributed to all 14 lecturers in the DoE at FCTVE. Five former lecturers were contacted by e-mail and requested to complete the questionnaire.Of the 14 DoE lecturers 9 (75%) returned completed questionnaires. Three completed questionnaires were received from former DTVE lecturers. Relatively, this is a fair representation of DTVE lecturers involved in facilitating the DTVE programme.Data from former DTVE student-teachers not employed in brigades/TCs58 former student teachers (33%) are not working in the TVET system (TC or brigade). Of these 58 former DTVE student teachers 22 could not be traced (13% of the total 174 former DTVE student-teachers). The 36 former DTVE student teacher traced were approached by phone and/or e-mail and questionnaires were dispatched to them through fax, e-mail or by hand. All 36 received the questionnaire. 22 (61%) were received back, some after numerous follow ups through telephone and/or e-mail. A few were collected by visiting the persons at their workplace / house. The collection of these 22 questionnaires turned out to be a tedious and time consuming exercise.Data from brigades/TCsA data collection plan was drawn up by the TS task force, including visits to all TCs and brigades. There are 20 brigades with 39 DTVE graduates. 13 (85%) were visited and data collected from the former DTVE student-teachers management / supervisors and learners. A time plan, scheduling visits to two institutions per day was produced. A standard programme for the visits was drawn up. All institutions were contacted by telephone, informed about the purpose of the survey and the programme for the visit. Once the details had been agreed on, the programme was faxed to the institution. Generally, all institutions contacted were very cooperative, and interested in receiving the data collection teams.The 7 brigades not visited by the data collection teams were contacted by phone and fax. Questionnaires for the former DTVE student-teachers were supplied through fax. The completed questionnaires were to be faxed back. A total 34 (87%) completed questionnaires were obtained from DTVE student-teachers in the brigades and 49 (64%) from DTVE graduates working in the colleges. Former DTVE studentsAt the brigades/TCs the graduates of the DTVE programme were addressed as a group. A brief introduction explaining the purpose and the objectives of the data collection exercise was given. Following that, the questionnaires were handed out to the instructors/lecturers for completion. The time for completing the questionnaire was approximately 1 – 1? hour. This is generally too long for a questionnaire, however as the DTVE former student-teachers were sitting together, discussing issues, recalling good and bad experience during their stay at the college and given the time by management to complete the questionnaire in all but one instance this worked well. In GTC the former DTVE student-teachers refused to complete the questionnaire as they had, according to them, raised issues with DTVET to which no response was received and hence they felt that there was no need for them to complete the questionnaire as ‘nobody will attend to what we say’ and ‘we completed a questionnaire when exiting the college, so this is a duplication’.With 49 (64%) of the 77 former DTVE student-teachers currently employed in the TCs completing the questionnaire, sufficient data were collected. After, and during completion of the questionnaires the respondents were encouraged to raise issues not covered in the questionnaire. A number of concerns related to recognition of the DTVE, BTEP phase training, the BTEP, continuous staff development, and other were brought forward. These are presented in section 7 of this report.Management, supervisors of the instructors/lecturers in the brigades/TCsWhile the instructors/lecturers completed their questionnaires, the research team interviewed the supervisors. In the brigades the supervisors comprised the brigade coordinator, the training coordinator and in some cases HoDs. In the TCs the supervisors generally were the HoDs. In several instances members of the brigade/college management participated, e.g. the centre contact, deputy principal academic. The structured interview was conducted, following the guidelines produced (Annex E). One member from the data collection team would read the questions and both researchers would write down answers for later comparison and compilation. Purpose of the interview was explained to the managers / supervisors emphasising that in answering the questions they should make comparison between those of their staff who had followed the DTVE programme and those who had not. The interviews lasted approximately 45 minutes.Learners in the brigades/TCs currently being taught by DTVE graduatesAfter interviewing the supervisors, learners taught by the DTVE graduates were gathered and interviewed. On average 3-4 learners of each DTVE graduate were requested to participate in the focus group interview. In institutions with many graduates the number of learners per. graduate could be smaller to keep the total number of learners in around the 15 range for a manageable group and ensure participation of all the learners were briefed on the purpose of the interview and its confidence. The names of the DTVE graduates were mentioned to them and they were requested in their answers to keep these instructors / lecturers in mind and compare them with those of their instructors/lecturers who had not completed the DTVE programme. During the interview learners raised issues not covered in the guidelines and/or not directly related to the facilitation of the DTVE graduates. Yet some of the concerns are relevant to the TVET system in general and hence included in section 7 of this report.Quality AspectsIn this section issues of sample sizes, reliability and validity will be discussed Size of SamplesThe sample sizes measured by respondents as compared to the populations are as follows:Table 3: Response rateTarget groupPopulation sizeNumber of respondents%DoE lecturers Lecturers in DoE: 14Former lecturers traced 59375%60%Former DTVE students17410560%Learners taught by DTVE graduatesNot known195-Management / Supervisors of DTVE graduatesNot known59-A 60% response rate from all the DTVE former students and a 69% response rate from those traced and contacted is high for tracer studies. The data collected are more than sufficient to draw conclusions as they may be considered representative for the whole population.The tables 4 and 5 present the data on how the different categories in the population of 174 DTVE former students are represented in the sample of 105 respondents.Table 4 Representation of categories among respondentsPopulationNumberNumber respondentsMale9458 (62%)Female8047 (59%)In service7556 (75%)Pre service9949 (60%)Full time15993 (58%)Block release1512 (80%)In the total population of 174 DTVE graduates three categorisations can be usedBy gender: male or femaleBy background: in service (employed in the TVET sector and send for teacher training) or pre service (mainly following completion of diploma course in a vocational area but without work experience in the TVET system)By mode of studying: in full-time face to face mode or in block release modeFull-time face to face student-teachers train for 18 months at the DoE. The block release student-teachers (all in-service) train at DoE for one academic term (with a front load of two terms) and work at their institution the next term. It takes these student-teachers 3 years to complete the programme.Table 4 gives the data on the number of respondents in each category as a percent of that category in the population. With a representation of about 60% to 70% in the respondent groups all categories are well represented.Table 5 Representation of cohort among respondentsCohortNumber in populationNumber respondingC1155 (33%)C243(actual 45, but2 transferred to C3)24 (56%)C35132 (63%)C3 block1512 (80%)C45032 (64%)Total174105 (60%)Table 5 gives the percent of each cohort among the 105 respondents. Cohort 1 is not surprisingly least represented as they are most difficult to trace. 6 (40%) could not be traced of the 9 traced and supplied with the questionnaire 5 (56%) responded. The other cohorts have a 60% representation, with the block release group 80%. As this group left in May 2008 and an exit structured focus group interview was carried out with this group, data on the group were available. In addition all student-teachers in this group were in service and hence could be easily contacted in their workplace in a brigade or a TC.From the above data one might conclude that the 105 respondents are a good representation of the population of 174 on the variable mentioned.ReliabilityA reliable data collection instrument is one that would give the same result if repeatedly used with the same or an identical group.In order to assess whether the data collection instruments were reliable the data collection instruments were piloted at Palapye Technical College and Selebi Phikwe Technical College. The pilot did not reveal any conceptual problems with the instruments. When the data collection instruments were rolled out to other colleges and brigades, no significant difference between the responses from the two colleges and the responses from the two pilot colleges were observed.More systematically the internal reliability of the two questionnaires was estimated by applying Cronbach?s as an estimator that quantifies the reliability of a score of several questions in a questionnaire. The estimator will take values between 0 and 1. In most cases an acceptable minimum score is 0.7. The estimator was calculated for all sections in the two questionnaires with numerical data. The estimator provided values to underline that the two questionnaires can be regarded as having acceptable reliability.For the calculated means (average rating) of the scores on a nominal 1 to 5 scale the 95% confidence intervals were computed. These intervals will with 95% certainty contain the true value in the population.ValidityIt is only after establishing the reliability of the instrument that validity is to be considered.Validity of a data collection instrument refers to the question “Is the instrument or measure used measuring what it is intended to measure?” To assess the validly of the data collection instruments the method of “consensual members checks” was used.The data collection instruments were presented by the DoE members of the task force for feedback, comments and alterations to all DoE staff members, together with “the question justification document”(Annex B) Each question in the instrument was to be justified: why is the question asked? To what research question is it attempting to find an answer to? As no feedback was received it was assumed that the instruments were valid. As professional educators staff in the DoE wants to collect valid data from the DTVE student teachers in order to review content, mode of facilitation, assessment and support of the programme offered. The lecturers in DoE being the experts on the DTVE programme are therefore the most knowledgeable persons to assess the content validity of the data collection instruments.The data collection instruments do allow to collect data on the relevance and use of the DTVE programme content, its mode of facilitation, its facilitators, its mode of assessment and the support provided as perceived by stakeholders. This is one aspect of the DTVE programme. The aspect NOT addressed is how the DTVE programme compares to vocational teacher training programmes offered in the region and internationally. Is the programme in line with current developments and thinking related to the training of teachers for the TVET sector in the 21st century? This is an important but different aspect. This however, raises the question whether the survey actually is providing answers to all the relevant questions. An independent researcher with relevant background in pedagogical training of vocational lecturers might need to look at the DTVE programme from these different angles. This is further elaborated under Recommendations.Data presentation In this section the data collected are presented.Questionnaire for DoE lecturers. (Annex D)The facilitators in the DoE have on average 3.3 years of experience with the programme. 33% of the lecturers have facilitated 6 or more of the 10 modules in the DTVE programme.Relevance of the content of the DTVEOn a scale from 1 (very irrelevant) to 5 (very relevant) the relevance of each content aspect of the programme has an average rating from 4.3 to 5.0. As key performance indicator was set the value 4. Hence the content of the DTVE programme is considered to be relevant for student-teachers to become competent and efficient facilitators.The average relevance rating for the 10 modules is in the range 4.6 to 5.0. Lecturers view the 10 modules as very relevant.The lowest overall rating is scored on the integrated key skills (average score 4.3, closer to the ‘partly relevant’ score of the relevant rating scale.Diagram 1 illustrates lecturers? ratings of the relevance of each of the 10 modules and of the incorporated key skills (KS)Diagram 1: Bar chart illustrating the relevance rating (scale 1 to 5) by DoE lecturersrsThe relevance rating of the elements of content of each of the modules, and of integrated key skills is presented in diagram 2 and diagram 3Diagram 2: RELEVANCE of module content as perceived by DoE lecturersDiagram 3: RELEVANCE of module content as perceived by DoE lecturersThe lecturers identified in their individual comments 4 content areas that, in their opinion, need to be included and/or strengthened in the programmeIntegrating of HIV/AIDS related issuesUse of modern technology in facilitation and learning (e-learning)Underpinning knowledge related to adult learning principles and psychology of learningAspects of school management and administration (professional conduct, academic rules) The structure of the DTVE programmeLecturers consider the DTVE programme as a very relevant teacher training programme. However, as for the structure of the programme on several statements the key performance indicator level of 4 (on the 1 to 5 scale) was not attained. The average rating on the 1 to 5 agreement scale (from strongly disagree to strongly agree) are displayed in diagram 4Diagram 4: Structure of the DTVE programmeJust half of the responding lecturers feel that the programme is well planned. The weakness of the programme reported is time related or support related. In relation to time the majority of lecturers stated that Time to complete the programme by learners is inadequateWorkload for learners and lecturers is too highTime for teaching placement is insufficientIn relation to support the majority of lecturers feel that the support for student teacheron the programme is inefficientfailing the programme (and having to complete in distance mode) is not effectiveThe items related to the above all score 3 or below. Time is a major issue in the programme (work overload) and there is need when reviewing the programme to look for ways to make the programme more ‘time’ friendly. Portfolio AssessmentThe views of the 12 DoE lecturers related to the portfolio assessment mode and process indicate that the lecturer consider the portfolio assessment system to be (average scores 4.1 to 4.8)the most appropriate for OBEencouraging self directed learning and reflective practicemotivatingvalidfair in demonstrating the progress towards the achievement of theLOstransparent to student-teachersDiagram 5 illustrates the agreement rating on the statements in the questionnaire on the 1 to 5 agreement scale.Diagram 5: Portfolio assessment mode and process as perceived by DoE lecturersThe majority consider the portfolio assessment mode more time consuming than other more traditional ways of assessment of learning. About one third of the lecturers doubt whether the assessment is facilitator independent and can ensure authentic of the work produced by learners.The above responses are amplified in the responses to the open questions in this section of the questionnaire. The motivating aspect, quality assurance (IV/ EV) process and the learner centred nature of portfolio assessment are mentioned as the strength of this form of assessment.As weaknesses are mentioned that portfolio assessment is time consuming, brings in issues related to authenticity of the documents placed in the portfolio (plagiarism) by the student-teachers, and for it to function w needs lecturers very familiar with OBE and its assessment.To combat the weaknesses it is suggested that more, well trained teacher trainer, familiar with OBE should be employed and staff development workshops should be facilitated for new staff not familiar with OBE portfolio assessment. Questionnaire for former DTVE student-teachers (Annex C)104 former DTVE student-teachers returned the questionnaire which was made available to 154 (80%) of the DTVE former student that could be traced from 174. The questionnaire was lengthy and as a result most of the questionnaires returned missed responses to some of the questions. The data as collected are summarised in annex C . Relevance of the content of the DTVE and its use by DTVE graduates.The content of the DTVE programme was rated by respondents on its relevance and its use in the work situation. As one might expect the ”USE” ratings are all lower than the ”RELEVANCE” ratings. The content of the DTVE programme is considered by DTVE graduates as (highly ) relevant all content being rated above the performance indicator (4) set. The relevance rates are in the range 4.2 – 5.0.The use frequencies were rated on a nominal scale from never used (1) to very often (5) and fall in the range from 2.5 to 4.8. Diagram 6, 7 and 8 display relevance and use ratings of contentDiagram 6: Relevance and use of modules compared.Diagram 6 clearly illustrates the lower rating on use than on relevance of the content of the 10 modules and the in-cooperated keys kills.The DTVE graduates ratings on relevance and use of content was analysed across the following three categorisations of the populationgender: male or femalebackground: in service (employed in the TVET sector and send for teacher training) or pre service (mainly following completion of diploma course in a vocational area but without work experience in the TVET system)mode of studying: in full-time face to face mode or in block release modeAverage ratings for each group in each category were computed in Excel and a t-test to test for significant difference at the 95% confidence level was carried out (Annex H). No significance differences were found. The expressed views and given ratings on aspects of the DTVE programme are independent of gender, mode the programme was taken by the student-teacher or whether the student teacher was in service or pre service.Diagram 7: Modules content relevance and use as perceived by DTVE graduatesDiagram 8: Modules content relevance and use as perceived by DTVE graduatesThe USE of following content was rated 3 or belowuse of manipulatives in learning sessionsuse of PowerPoint presentationskeeping of personal developmental logaction researchThe low use of PowerPoint presentations is mainly due to lack of the appropriate technology in the TVET institutions, as reported during interviews.Content that was mentioned as to be included or strengthened can be place in the following categoriesUnderpinning theory / knowledgeCurriculum developmentPsychology (adult educational)Learning theoriesBotswana’s educational laws and policies Key skillsStress managementTime management (2)Data analysis (2)Presentation skillsICTSoftware use such as PowerPoint, Excel (6)Internet use and search (3)Video productionIntroducing and training of vocational area specific software (e.g. AutoCad)Management and administration in educationPrinciples of educational administration and managementGuidance and counsellingSchool resource managementUse of modern technology in educationE-learningVideo conferencingE-assessmentBTEP phase 1 - 3Full coverage (and recognition by DTVET) of BTEP phase trainingThree of the above areas, underpinning knowledge, use of modern technology and management/administration in education, were also mentioned by the DoE lecturersThe structure of the DTVE programmeThe responses to the items in the questionnaire (item 1.63 – 1.78) related to the structure of the DTVE programme indicate that the respondents agree that the programme is enjoyable, interesting, motivating, well planned, of high quality. The average score on the agreement scale on these items was in the range 3.2 to 4.3. Diagram 9 illustrated the strength of agreement to the items in the questionnaire. Diagram 9: Structure of the DTVE programme views of DTVE graduatesTwo items score below the neutral point of 3. Both are time related. Workload is seen as not acceptable and time to complete LOs as insufficient. In the general comments (question 1.79) this is amplified. None of the cohorts completed in the 18 months scheduled and a number of months were added to each course.In the general comments, remarks related to the structure of the programme restate in many different forms the two low rated statementsThe workload is too heavy andTime is insufficientit can argue that both statements express the same. The time is insufficient toFully benefit from TP as TP is too much evidence collection focussed instead of developing classroom management and facilitation skills in a range of situationsTo ‘learn’, the focus is on evidence collections and document production with insufficient time to ‘digest’ the underpinning theory. This results in copying / plagiarism / ‘buying’ of evidence from other groups and/or cohortsOne suggestion forwarded was to make all the learning resources for each module available online (e-learning) so student-teachers can work at own pace.Mode of facilitation used in the DTVE programme.The items 2.01 to 2.17 requested the former student teachers to give their opinion on the facilitation mode used. The DTVE programme is designed to do what it preaches: using learner centred, blended approaches to supporting student-teachers to acquire knowledge, skills and attitudes. Items 2.01 to 2.11 ask learners to rate effectiveness of various methods used and to rate the frequency these methods are used.Diagram 10 shows a bar chart displaying the average ratings on effectiveness and frequency of use.Diagram 10: Facilitation modes used in the DTVE programme rating of effectiveness and frequency of useThe methods mentioned are all considered to be (very) effective with ratings in 4.1 to 4.6 range. The uses of the various methods range from 2.8 to 4.2. Least used (rate 2.8) are video based sessions. As the method used will depend on the content it is not surprising that certain methods are used less frequent than others.Average ratings across gender, programme mode, and in/pre service entry were checked for significant differences using t-test. No significant differences at the 95% confidence level were found.Respondents prefer mainly co-operative learning approaches and dislike teacher centred, lecure type of facilitation mode.The variety in facilitation mode is found to be fine by 82% of the respondents. The pie chart displays the responses.Diagram 11: Pie chart illustrating views on variety in facilitationAs for the mode of facilitation catering for the preferred learning style of the student-teacher, the majority (about 60%) feels that that is true to some extent. The results are in tableTable 6: Frequencies of responses on item 2.16Mode of facilitation catered for preferred learning styleFrequencyPercentNot at all22%To some extend6159%Very well4039%The mode of facilitation used by DoE reflects very well (70%) a constructivist, learner centred approached according to the student-teachers. Diagram 12 illustrates in a pie chart student-teachers views.Diagram 12: Pie chart displaying student-teachers views on constructivist, learner-centred facilitation modes used by DoE lecturers.Views of former DTVE student teachers on variety in facilitation, link between mode of facilitation and own preferred learning style and whether facilitators use a learner centred constructivist approach do not differ significantly across gender, learning mode (FT or BL) or across in/pre-service student teachers. Quality of facilitation by DoE lecturers.Section 3 of the DTVE graduates questionnaire collected data on the views of the former students as to the quality of the facilitation they received. ‘Customer’ satisfaction surveys are important for the service provider in order to find out whether or not customers are satisfied. The information obtained can guide well informed improvement plans. In the educational setting the student-teachers are the customers and the lecturers the service providers.The rating on a scale from very poor (1) to very good (5) are displayed in the bar chart in diagram 13.Diagram 13 Former DTVE student-teachers views on the quality of their facilitatorsStudent teachers? average rate on each item is in the adequate to very good range. Student teachers generally express the opinion that their lecturers are doing a professional job. Student teachers are least satisfied with the turn over time of assignments (rated 3.6). Across the categories used in this survey the average ratings are not significantly different (Annex H).In the open question space student-teachers were positive about the facilitation of their lecturers. Positive critical observations were clearly (i) referring to isolated cases (ii) aimed to improve the learning environment and interaction between student-teachers and lecturers. Issues mentioned by more than one student-teacherGive prompt feedback *6Ensure timely feedback – well before submission deadline *2Give constructive feedback, not destructive *3Treat student teachers with respect / as adults *5Employ qualified lecturers familiar with the DTVE programme / OBE *7Employ DTVE graduates as SDFs *6Lecturers need to come to sessions well prepared *4Lecturers should be familiar with / inducted into the DTVE programme *7Lecturers should be open to criticism *2Inconsistency in assessment among lecturers need to be addressed *3Lecturers should also be specialist in a particular vocational area *4Address authenticity / plagiarism *2Act professional, avoid bias, favouritism DTVE student-teacher perception on portfolio assessmentDTVE is assessed continuously through portfolio building. Particularly for teacher trainin programmes, portfolio assessment is very appropriate, provided it is used effectively and efficiently. The DTVE portfolios should contain the evidence for the learning outcomes and document the process that lead to the final evidence. Student-teachers, were to rate their agreement with statements related to portfolio assessment. Diagram 14 illustrates the average rating on the portfolio assessment related itemsDiagram 14: Bar chart of average agreement ratings on items on portfolio assessment mode nad process as perceived by the former DTVE student-teachersStudent teachers were in agreement (score 4 of above) with the statements that portfolio building as used in the DTVE programme is most appropriate, ensures all learning outcomes are covered, gives responsibility to learners for own learning, enhances reflective practice and critical thinking, is motivating, transparent and effective to monitor own progress. These aspects are mentioned in the open items 4.17 and 4.19 where as strength of the portfolio assessment system were mentioned:Continuous nature of portfolio assessment, the cycle: submit evidence – receive feedback – implement feedback – resubmitPortfolio assessment (PA) allows to monitor growth and progressPA encourages reflective and critical thinkingPA places responsibility for learning and evidence collection with the student teacherPA ensures all LOs are coveredPA process is transparent as the portfolio verification document describes in detail the evidence to be submittedThe below 4 agreement ratings scores are on exactly the same aspects as identified by the DoE lecturers:ReliabilityAuthenticityTime consuming modeThe three aspects being problematic are described in a variety of ways in item 4.18Reliability of assessmentMarking is ‘subjective’Different lecturers use different criteria (You ‘pass’ with lecturer A, but will have to resubmit with lecturer B)Authenticity of submitted evidenceCopying from other students (as work will be submitted to different lecturers, hence copying / plagiarism goes unnoticed in nearly all instances)Plagiarism from web‘buying’ portfolios from previous cohortsFocus on evidence production / assessment – at the cost of learning – encourages copying due to time pressure and deadlines to be metTime consuming nature of PAToo much work within a short time‘not achieved’ for minor short comings [could be resolved using oral evidence as in BTEP]The expressed views of the DTVE student teachers on portfolio building are independent (at 95% significance level) of gender, whether they are in- or pre-service student teachers and whether they studied in the full time face to face mode or block release mode (See annex H).Effectiveness of the DoE support system for the DTVE programmeIn all modes of programme delivery the support available and given to learners to achieve the learning outcomes is crucial. One might produce the most engaging, motivating learning materials but without effective and efficient support to learners the materials by itself will not produce the desired outcome i.e. that learner will learn and achieve the set learning outcomes. Within the DTVE programme the student teacher may aspect support and guidance from (i) the module facilitator on academic aspect and evidence production for the module (ii) his/her personal tutor to receive guidance and support in a holistic way. The (weekly) meetings with the personal tutor will discuss not just a single module but look at the overall picture of the programme and set attainable targets on weekly basis. Action planning is a key feature, as is the addressing of (personal) issues that might negatively impact on achieving set goals. (iii) The TP mentor, the experienced lecturer / instructor in the institution where the student teacher is placed for TP. The mentor inducts supports and assists the student teacher throughout the TP period by observing the student-teacher, giving feedback and supporting in evidence collection.The questionnaire items 5.01 – 5.12 addressed the above issues. Diagram 14 displays in a bar chart the average agreement ratings on the items related to the availability and receiving of support.Diagram 15: Bar chart of average agreement ratings on support related issues in the DTVE programme as perceived by former DTVE students Average agreement ratings on the item seeking view on the support available and given to the DTVE student teachers falls within the 3.4 – 4.2 range. The lowest average agreement rating (3.4) is on the item relating to a personal guidance/counseling, a system to address personal problems negatively affecting the performance of the student teacher. This is not surprising as the DoE has for long been depending on guidance and counselling officers NOT located within the Department. With DoE at FCTVE the current student-teachers can make use of the Guidance and Leaner support system in place within FCTVE. Overall student teachers perceive the support system available and the support given by their lecturers as adequate for their needs. Among strengths are mentioned (item 5.19):Facilitators are very supportive throughout the programme *18Facilitators are committed and readily available for support *3Strong support given before and during TPClose cooperation between student-teachers and facilitators, and constructive feedback ensures effective evidence collection *2Counselling available when needed (‘if the going was tough’) *2Personal tutor system effective to overcome academic problemsLearning resources accessible and available *6As weaknesses in the support system student-teachers mentioned:Facilitators are unskilled/less experienced in OBE *6Feedback providedNot timely *8Inconsistent across facilitators (biased /reflecting favouritism of lecturers) *6Plagiarism / copying goes undetected *2Insufficient briefing and support for student-teacher placed in ‘distance learning mode’Supplementation rates of DTVE students are high. Few pass after final submissions of their portfolios. The supplementation rate for the first cohort was 33%, for the second cohort 10%, while for cohort 3 and 4 the supplementation rate was 100% - no student teacher passed ‘on first attempt’. Those who supplement are provided with extensive feedback on what is to be done in order to meet the evidence requirements; this is part of the support system. Those failing the programme are referred to ‘distance mode’ – they can complete the programme in distance mode. For those having failed and continuing in distance learning mode a support system is crucial, without it they will not be in a position to successfully complete the programme. Diagram 16 is a bar chart displaying the average agreement rate of DTVE former students that were supplementing and/or referred to the distance learning mode.Diagram 16: Average agreement ratings on items related to support during supplementation and support ro distance learners as perceived byformer DTVE studentsThe average agreement ratings by supplementing DTVE students on the questionnaire items 5.13 – 5.15 are between 3.5 and 4. This is in the partly agree region. It suggests that the support system during supplementation needs to be critically reviewed to make it more effective and efficient.This corresponds to the rating of the DTVE facilitators on the statement “2.17. The programme has efficient support systems in place” with an average agreement rating of 3.0 – half of the responding facilitators (strongly) disagreed with the statement.DTVE student-teacher that failed the programme and were placed in ‘distance mode’ rated the items related to support in the range 1.8 to 2.0. In other words they perceive that there is no effective operational system in place to support them. 50% of the facilitators concur with that perception (item 2.18 in the lecturers’ questionnaire score average rating of 2.7)The expressed views of the DTVE student teachers on the DTVE programme support system are independent (at 95% significance level) of gender, whether they are in- or pre-service student teachers and whether they studied in the full time face to face mode or block release mode (Annex H).Comparing responses from DoE lecturers with responses of the DTVE student-teachers.The questionnaire for lecturers and for DTVE graduates had many identical items. This allows the perceptions of the two groups to be compared and tested for significant differences. (Annex H) DTVE programme contentThe average relevance ratings of content and modules in general by DoE lecturers and DTVE graduates were compared. A t-test was applied at 95% confidence level – no significant difference were found. The ratings of DoE lecturers and of DTVE student-teacher are not in any way different. Table 7 compares the average relevance rating by DoE lecturers and DTV graduates.Table 7 Average relevance ratings for content of modules DN01, DP01 and DP02 by DoE lecturers and DTVE student-teachers.DTVE graduatesDoE lecturers1.01 Diagnose Learners? Needs4.651.02 Determine Learning Style4.54.81.03 Methods for different needs and learning styles4.751.05 Interpret LU specifications4.651.06 Prepare scheme of work4.851.07 Prepare session plans4.751.09 Different activities to meet LOs4.74.61.10 Written LM to support learning4.94.91.11 Audio/visual LM to support LOs4.751.12 Visual learning aids to support LOs4.84.81.13 Manipulatives to support Los4.54.31.14 OHT to support Los4.64.51.15 PowerPoint to to support LOs4.64.51.16 Internet/Web based resources to support LOs4.74.6The differences in ratings are very small and not significant at 95% confidence level (Annex H) The content of the modules is rated as (very) relevant by both student-teachers and DoE lecturers. Diagram 16 in a double bar chart illustrates that ratings of the two groups are extremely close to each other or identical in some instances.Table 8: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers.DTVE graduatesDoE lecturers1.18 Group discussions in facilitation4.84.81.19 Cooperative learning spproaches4.651.20 Experimental learning approach4.751.21 Individualised learning approach4.64.81.22 Range of motivational techniques to support LOs4.751.23 Manage conflict situations4.74.71.24 Effective questioning techniques4.84.91.25 Promotion of equal opportunities4.84.51.27 Induction programme4.84.91.28 Support to meet LOs4.74.61.29 Personal support and guidance4.64.51.31 Different formative assessment activities4.851.32 Feedback to support learning 4.951.33 Keeping record of learners achievements4.94.91.34 Fair and unbiased assessment4.851.36 Evaluate own facilitation4.64.91.37 Evaluate teaching methods4.64.91.38 Evaluate learning resources4.64.91.40 Being a reflective practitioner4.24.61.41 Keeping personal develoment log4.04.41.43 Evaluate own practice using action plans4.44.51.44 Effective working relationships4.74.51.45 Adhering to high professional standards4.84.51.46 Championing teaching and learning methods4.74.91.47 Enterprise activities with institutions4.24.71.48 Proactivity on own further development4.74.81.50 Undertake action research4.44.91.52 Using Word4.84.71.53 Use speadsheet, e.g. graphing4.84.61.54 Use power point4.84.51.55 Use data analysis techniques4.84.61.56 Use Web search4.74.51.57 Use APA referencing4.54.51.58 Use discussing skills (participating)4.84.51.59 Use discussion skills (leading)4.84.41.60 Use presentation skills to audience4.94.7Note that on the key skills content the average ratings by DoE are ALL less than the average ratings of the student teachers.The overall relevance rating of each module by DoE lecturers and TVE student-teachers are displayed in table 9. Table 9: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers.DTVE graduatesDoE lecturersDN01 Assess Learners? Needs4.64.7DP01 Plan for effective Learning4.85DP02 Prepare Resources to support Learning4.84.9DF01 Facilitate Learning4.84.9DS01 Provide Learners with Support4.74.6DA01 Assess learning4.75DA02 Evaluate learning Sessions4.64.8DE01 Reflect upon Roles of a Teacher4.44.7DE02 Improve own professional Practice4.74.7DE03 Undertake Action Research4.34.9Key Skills4.84.3The two differences, although not significant, to note are on DE03 and KS. Student-teacher are not strongly convinced about the relevance (and use) of module DE03, the action research module, while lecturers consider it highly relevant. For the integrated key skills it is the other way around. Student teachers find key skill very relevant and mentioned (see section 5.2.1) that more attention and support should be given to key skills. DTVE programme structureIdentical questions in the DoE lecturers and the DTVE graduates questionnaires allow to compare ratings on programme structures. The ratings and the items are tabulated in table 10. Table 10: Comparing ratings of related to programme structure by DoE lecturers and DTVE graduatesItemDTVE graduatesDoE lecturersEnjoy programme3.94.0Programme well planned3.23.5Good sequencing of Units3.34.0Appropriate Content3.44.4Time for Completion adequate2.62.0Workload acceptable2.22.3High Quality programme4.24.3Well prepared for TP4.14.1Integration of TP and college work4.14.4Programme is flexible3.53.1On most items DoE lecturers give a slightly higher rating. The problem of too heavy workload and the insufficient time given to complete the programme is shared between lecturers and student-teachers. DTVE portfolio assessment systemBoth DoE lecturers and DTVE student-teachers rated items related to the portfolio assessment. They are tabulated in table 10 with the average agreement rates. Diagram 20 illustrates the data in the table in a double bar chart. The difference in rating are not significant at 95% confidence level as checked by using a t-test (Annex H).Table 11 Comparing of average ratings on items related to portfolio assessment by DoE lecturers and DTVE graduatesPortfolio assessment related itemsDTVE graduatesDoE lecturersMost appropriate for OBE4.34.6Ensures evidence for all LOs4.44.5Encourages self directed learning4.54.8Demonstrates progress towards identified LOs4.34.3Encourages reflective practice4.34.1Provides opportunities to demonstrate competence4.34.1High validity4.34.4Fair way to document progress4.54.4Motivating4.44.3Reliable method. Lecture independent3.83.5Ensures authenticity of evidence3.73.9Transparent method of assessment4.14.1Encourages critical thinking4.43.3NOT more time consuming than other modes of assessment3.22.5DoE lecturers and DTVE student-teachers share the perception that using portfolio assessment is more time consuming than other forms of assessment. They share the view that portfolio assessment is most appropriate for the outcome based DTVE programme, ensures all learning outcomes are covered, gives responsibility to learner for own learning, enhances reflective practice and critical thinking, is motivating, transparent and effective to monitor progress.DTVE programme support systemIn the student-teacher questionnaire a section was focusing on the support system of the DTVE programme. In the DoE lecturer questionnaire only two items (2.17 & 2.18) referred to the support system. The average agreement score on these items is low, 3.0 and 2.7 respectively. Both the support system during the running of the DTVE programme and the support system to support student-teachers that failed (and are moved to the distance learning mode) the programme are perceived as inefficient. The student-teachers share this opinion as outlined in section 6.2.6.The DTVE programme structure, its content, portfolio assessment system and support system are perceived in the same way by DoE lecturers and student teachers. Any differences in average rating of corresponding items in the two questionnaires are not significant. Strengths and weaknesses are shared ground and hence form a solid basis for implementation of any change in the programme that might be needed. There is no disagreement between DoE lecturers and their former students as to the strong and weak aspect of the DTVE programme.Structured focus group interview: management and supervisors of DTVE graduates Structured focus group interviews were conducted with management/supervisors of the DTVE graduates with in 6 TCs and 13 groups in the Brigades. A total of 59 respondents were involved 27 in the TCs and 32 in the brigades. The details are in table 12. Table 12 Colleges and brigades visited and number of respondents in the focus group interviews with management / supervisors of the DTVE graduatesInstitution: College / BrigadeManagement / supervisorsATTC4GTC5JTeC4MTC7PaTeCo2SPTeCo5Total TCs27Barolong VTC2KRDA4Marapong3Marobela2Naledi6Ngethu3Okavango2Ramatea1Shashe1Tlokweng1Tswelelopele2Tutume5Total Brigades32The purpose of these structured focus group interviews (Annex E) with management / supervisors of the DTVE graduates was to find out: How do managers / supervisors of DTVE graduates rate the quality of their facilitation as compared to facilitators without this teacher training background? The summary of the responses of the 19 structured focus group interviews with management/supervisors of former DTVE students is found in Annex H. The most frequent mentioned strengths of the DTVE graduates were:Competent and confident in learner centred methodsEffective use of teaching/learning aidsEffective use of a variety of teaching/learning methodsEffective planning and preparation of lessonsPreparing effective schemes of workSupporting management/Co operative Supporting learners in achieving the learning outcomesDemonstrating professional behaviour/ Good work relationship with learnersContribute ideas and take initiative in the institutionProfessional interaction with learnersThe responses are based on classroom / workshop observations by management / supervisor taking place at least once every term and the level of participation in staff meetings observed. Management / supervisors in brigades reported a stronger impact of the DTVE graduates on the institution than management / supervisors in the TCs. DTVE graduates in the brigades on completion of the DTVE programme were given more responsibilities, promoted and/or given positions in the management. In 10 of the 13 interviews with brigade management / supervisors it was mentioned that the DTVE graduates had been running workshops for the instructors in the institution varying in length from 1 day to 2 weeks (during term break). This dissemination of knowledge and skills within the institution was NOT mentioned in any of the 6 TCs visited. In discussion weaknesses and suggestions for improvement of the DTVE programme management and supervisors mentioned issues not directly related to the quality of facilitation of the DTVE graduates. These will be covered in the next section under concerns and issues raised Structured focus group interview: learners taught by DTVE graduatesA structured focus group interview (Annex E) was carried out with learners currently taught by the DTVE graduates working in the TCs or Brigades. The procedure followed is described in section 5.6. Data were collected through interview sessions with 91 learners in 6 TCs and 104 learners in 13 Brigades. The details are in table 13.Table 13 Colleges and brigades visited and number of learners participating in the focus group interviewInstitution: College / BrigadeLearnersATTC10GTC14JTeC14MTC17PaTeCo15SPTeCo21Total TC’s91Barolong VTC15Ghanzi6KRDA8Marapong8Marobela4Naledi5Ngethu6Okavango8Ramatea5Shashe8Tlokweng6Tswelelopele17Tutume8Total Brigades104The purpose of the interviews was to answer the research question: How do learners of graduates rate the quality of the facilitation by DTVE graduates? Learners took the opportunity provided to raise various issues related to their education and their educational environment. Some of these issues, reported in the section “Concerns raised”, did not immediately relate to the focus question: What is the quality of the facilitation of the DTVE graduates? A report was produced for each of the 19 focus interviews and these were summarised (Annex J). The data relating to the perceived quality of facilitation by the DTVE graduates by the learners is summarised here. At the surface level the facilitation of DTVE graduates does not differ much from the facilitation of lecturers / instructors without DTVE. All lecturers in TCs and brigades facilitate, assess and support their learners. However by looking more carefully the learners mentioned differences. Learners in the brigades pointed out two areas in which the DTVE graduates differ from the non DTVE trained lecturers / instructorsDifferences in practice. DTVE graduatesUse more variety in teaching methods. They ensure the teaching method used is aligned to the learning outcome to be coveredPredominantly use learner centred approaches. Cooperative learning methods, especially small group work, are frequently used. Non teacher trained lectures / instructors tend to use teacher centred lecture methods. Plan each session carefullyScheme the whole course. In contrast non teacher trained lecturers / instructors are poor in scheming or do not scheme. This leads to situations that work is unequally divided over the term, with extreme pressure towards the end or even having to extend the official set length of the course.Demonstrate a professional approach to their workProvide prompt and constructive feedback Set very regular formative testsTreat all learner equalDifferences in attitude. DTVE graduatesDemonstrate confidence. “They seem to know what they are doing”Are less authoritative / more flexible in taking learners’ suggestion on boardAre very supportive in assisting learners to achieveAre approachable in and outside sessionsDTVE graduates in the brigades were singled out as good role models for other lecturers / instructors in the great majority of cases. The differences mentioned by learners are too a large extend similar to those reported in the previous section as the strengths observed by management / supervisors.Learners in the TCs saw little or no difference in the practice of the DTVE graduates and their other lecturers. This is understandable as most staff in the TCs are experienced and a number of them have some teacher training background. In the brigades qualifications of instructors is lower and majority has not gone through any form of teacher training. The learners in the TCs did mention noticeable difference in attitude, similar to the once mentioned by learners from the brigades.Learners in both brigades and TCs, although emphasising the positive aspects of their DTVE trained facilitators, did express that in their view ALL lecturers / instructors in their institution, including the DTVE graduates, did not approach them as adult learners. Learners mentioned numerous situations in which they felt that they were treated without sufficient respect, felt belittled, insulted. “They all treat us like kids”Completing the DTVE programmeThis section presents data to answer the research questionsa. What is the completion rate / pass rate?b. What are reasons for dropping out from the programme before completion?c. Where are graduates employed after completion?d. What is the time gap between completion and employment?Data to calculate withdrawal, completion, pass and fail rates of student-teacher in the DTVE programme were collected from the DoE. Different documents give slightly different data for the 174 student teacher that went through (part of) the programme. Table 14 summarises the data as available to the TA tracer group and mainly based on the compiled student list (Annex K). Table 14 Data on withdrawal, supplementation, pass and fail DTVE cohorts 1 to 4.CohortPre serviceIn serviceWithdrawn from courseSupplement ingPassedFailedC112305 (33%)9 (60%)6(40%)C227189 (20%)4 (11%)32 (71%)4 (9%)C3 FT28233 (6%)48 (100%)43 (84%)5 (10%)C3 Block-151 (7%)14 (100%)5 (33%)9 (60%)C4331710 (20%)40 (100%)33 (66%)7 (14%)Withdrawal rates for cohort 2 and 4 are 20% i.e. 1 out of each 5 student teacher that started the programme withdrew from the programme before completing. Reasons for withdrawing from the course or not completing were given by 15 respondents and can be categorized as in the table 15.Table 15: Reasons for withdrawing from DTVE programmeCategoryReason as stated in responseMedical (13%)Due to illnessMedical reasonsTaking up employment opportunity (20%)I was appointed at Department of SuppliesI found an employment opportunity and so I went for itOffered employmentCourse related reasons (33%)The programme was taking longer than the expected 12 months and was caught up in a financial crisisHad not finished all the modules when the time elapsed(f=3)There were many reasons, e.g. Files not being marked in time, Teaching Placement wrongly placed and facilitators personalizing things.Personal reasons (33%)I had personal / family / business issues to take care of(f=5)Supplementation rate All student-teachers in cohort 3 and 4 had to supplement i.e. nobody passed the programme on 1st submission of the portfolios. No data are available to explain the 100% supplementation rate. It is clearly related to the way the programme is structured i.e. the workload and time pressure mentioned by both DTVE student-teachers and DoE lecturers.Pass rates are calculated as a percent of those starting the course. Pass rate for the block release group is very low, only 1 out of every three student teachers passed in this mode. If one calculates the pass rate as a percent of those completing the course (leaving out the withdrawals) the percent pass / fail are as in table 16.Table 16. Pass / fail rates as percent of completersCohortPassedFailedC19 (60%)6(40%)C232 (89%)4 (11%)C3 FT43 (90%)5 (10%)C3 Block5 (36%)9 (64%)C433 (82.5%)7 (17.5%)Employment data for DTVE former student teachers are incomplete – data on the 22 non traceable previous students are missing. Assuming that the 22 not traced students are employed (if in the TVET system they would have been traced) then 8% of the former DTVE students are unemployed (all except 1, from the last cohort finishing in July 2008). Employed in the TVET sector (TCs or brigades) are 67% of the 174 DTVE student-teachers. Hence about 25% is employed outside the TVET system (some will be self employed or studying).17 (16%) of the respondents did not find employment immediately after completion. Reported waiting times vary from a few months to a year.Reasons stated for not being immediately employed are:Still waiting for DTVET to post me“They said I am a foreigner so I can only be employed when DTVET advertises, then I can apply just like any other foreigner”Have not completed all modules (f=2)I completed my studies in 2006 and had to wait for DTVET to employ me. I got employment in February 2007 with the Department. I don’t know why I had to wait so long to get employment looking at the fact that I was a qualified lecturerAs a pre-service trainee I was not automatically placed by DTVETI applied at various Institutions and did not get an offer(f=2)It took time for the Ministry/Department to allocate me to a college(f=5)I had to wait for 6 months reason being that DVET was still looking for posts No idea why DTVET did not employ meTook 3 months to find Temporary Full Time employment in a TCIt is made clear to student-teachers entering the programme as pre service candidates that employment on completion is not guaranteed. Yet with the shortage of trained vocational teachers, one would expect that all successful graduates from the DTVE programme could be immediately absorbed into the TCs. It needs further investigation to find out why successful graduates from the DTVE programme are not immediately employed.Concerns raised. Inevitably, there were issues raised during the data collection that were outside the scope of the study. Listed here are those thought to be important for a future review of the programme, or a wider review of vocational teacher training. DTVE graduatesRecognition of DTVE1. Graduates working in the TCs expressed disappointment that DTVET does not recognise the DTVE as worthy of a salary increment. They are at same scale point as those having a vocational diploma but with no teacher training qualification. This could act as a deterrent to some, since there is no financial incentive to attend or achieve.2. Comments were made regarding some college managers, not fully appreciating the value of DTVE and therefore reluctant to permit staff to enrol for the course. 3. Articulation routes are apparently not well defined. The lack of an internationally recognised qualification caused difficulty for some students attempting to enrol on Bed programmes abroad. This could be addressed by seeking formal arrangements through memorandums of Understanding between FCTVE and other institutionsBTEP training1. DTVE graduates are required to go through BTEP phase 1 / 2 /3 training when placed in colleges. The content of these training workshops is covered by the DTVE programme and was an unnecessary duplication. The DTVE graduates complained that they often knew more about BTEP, than those delivering the phase training!DTVE facilitators at DoE1. It was mentioned that there has been ‘an inflow of secondary school teachers’ as facilitators of the outcome based DTVE programme. Graduates felt that the background of these lecturers made them unsuitable to deliver the outcome based DTVE programme without themselves first being thoroughly trained on OBE in general and the DTVE programme in particular and familiarising themselves with vocational training (work attachment in industry was suggested and/or attachment for at least one term as lecturer in a vocational area in a TC i.e. facilitating BTEP.Staff developmentThe graduates expressed concern that there was no programme for continuous professional development upon completion of the DTVE. There would be a demand for:training in project supervisionportfolio building (several BTEP units require learners to build portfolios)the use of technology (e-learning and eAssessment).Gap between competencies achieved and what can be implemented1. The lack of resources in some of the TCs, and in most of the Brigades was mentioned by graduates as a major problem. The lack of data projectors, and network connectivity in the classroom prevented the use of PowerPoint presentations and internet resources in the classroom.Follow up of ‘non-completers’1. There are a number of former DTVE students that did not complete all the modules. They reported that they did not receive clear information as to how they could continue beyond the normal end of programme, to complete them. Some reported that attempting to clarify this with DoE had not been successful.Feedback on outcomesMany respondents made the comment that after having giving their time and input to the tracer study they will never receive any feedback or information on the outcomes.The Teaching Placement component of the DTVE1. It appears that students on TP are reluctant or unwilling to take responsibility for a group of learners on a timetabled basis. They want access to a class group only while being observed for external assessment.2. The consequence of the above: TCs are reluctant to employ DTVE graduates as they have not really seen the graduates ‘performing’ within the college.2. TP supervisors suggested student-teachers to (i) follow experienced teacher for about 1 week (ii) take full responsibility for a group of learners for the next weeks. This allows feedback from supervisors.3. The TCs and brigades are not sufficiently informed beforehand by DoE about the upcoming TP. When DoE lecturers are at the TCs and Brigades they don’t consult or interact with the lecturers/instructors of that institution. 4. External supervision (DoE at FCTVE) could be greatly reduced by training the workplace based mentors and, after training, allowing these to assess student on TP. Currently TP supervisors / mentors feel sidelined: their views are not considered nor asked munication DoE with TCs and brigades1. Management at both TCs and brigades expressed that “we do not know much about the content of the DTVE programme”. A need was expressed for workshops/ briefing meetings by DoE for management / HoDs on the programme.Staff development (SD)1. Management at both TCs and at brigades identified the need for continuous staff development to upgrade staff and management. The question was raised in some of the brigades “What happened to the BRIDEC short training courses?” These were felt to have been very useful for instructors to improve their teaching skills. As these courses were offered (phase 1 -3) during term breaks they fitted well in the programmes of the Brigades. Similar short courses were suggested for staff development.2. SD activities related to using of modern technology in teaching, understanding blended/flexible learning approaches, setting formative assessments, classroom management skills – were mentioned as some areas a staff development unit in the DoE should provide.3. Teachers should also be upgraded in vocational areas. Currently some of the graduates cannot teach all units in their vocational area as they lack the content knowledge for certain units. During the teacher training attention should be paid to the vocational content knowledge to ensure the graduate feels confident with all units at the foundation and certificate BTE programme level. Learners taught by DTVE graduatesLearners are concerned abouta. the BTEP1. The assessment mode is not challenging, same assessment instrument(s) used repeatedly.2. The long time it takes to receive the BTEP certificate after completion. Learners in TCs stated that learners– since the start of the programme – had not received a Certificate (going back to 2004 / 2005).3. Employers do not value the BTEP as it shows A only (achieved, misinterpreted as a grade A).4. Learners prefer a system of assessment/reporting that does distinguish between learner levels of achievement, as they feel this could enhance employment opportunities.5. Lack of understanding of the BTEP by senior college management.6. Lack of understanding of BTEP by employers – who prefer the graded NCC.7. Poor planning of some of the programmes – a programme can be extended by some months because some LOs are not covered or they ‘forgot’ to offer an elective – leading to extension of the programme.8. Key skill departments in TCs received much criticism from learners. They felt the departments are poorly organised, lecturers authoritative and lacking knowledge. It was suggested by some learners that the lecturers in their vocational department ‘could do a better job’ and cover the key skills as integral part of other units.b. Access to resources1. Opening hours, access to computers and library was felt to be insufficient.2. More computers and fast internet access was felt to be needed in connecting with project report research and writing3. Access to computers frequently only for learners in ICT programmes, access for other vocational areas was considered to be insufficient in relation to work they had to do on the computers (e.g. project work and reports)c. Resources1. Brigade learners mentioned lack of resources: library books, computer (if present – outdated hard and software), data projector2. In some vocational areas it was said that the number of teaching staff was insufficient resulting in the few staff available being overloaded and having insufficient time to support learners individually.d. Support1. Non ICT related vocational areas felt that they were not supported enough in ICT during the very limited time (at times 1 or 2 hours a week) they had access to computers. Nobody was around to give them support and they felt belittled by ICT lecturers who hardly tolerated them in “their” lab.e. The learners? views1. Learners in TCs and brigades are concerned about the weak vocational knowledge of many lecturers / instructors. It was stated that it is not uncommon that instructors / lecturers with an NCC are teaching NCC. Learners expressed that there is a need for vocational upgrading of lecturers / instructors.2. Due to lack of vocational knowledge instructors/lecturers tend to copy from books but fail to explain what they copied.3. Learners in TCs observed that the content presented frequently lacked relationship to the real work situations. In their opinion this was due to the lack of industrial experience of the great majority of lecturers. They, the lecturers, have a theory background but lack experience in industry.4. Learners in TCs observed that in their opinion there was no difference in approach used by lecturers when delivery foundation, certificate or advanced certificate BTEP. In their view the different levels need different approaches and ways of interaction with the learners.Conclusions and RecommendationsThe recommendations are based upon the combination of the data collected from the all the target groups:Curriculum Both DoE lecturers and graduates are satisfied with the curriculum content of the modules. The DoE lecturers are marginally more satisfied with the module content. Conversely, the graduates? satisfaction rating of the integrated Key Skills was higher than the lecturers.Some comments from the graduates, the DoE lecturers and management supervisors suggest that they would have liked additional input on the following topics:psychologyadult learning theoriesthe context of vocation education in Botswanaeducational policy and managementintegrating of HIV/AIDS related issuesuse of modern technology in facilitation and learning (e-learning)The content of the DTVE has not changed significantly since 2001 during which time there have been major changes in the vocational sector, with more to come over the next few years. While there is a general satisfaction with the content of the DTVE, from the surveyed stakeholders, a wider stakeholder review e.g. including TEC, BOTA and private colleges, could provide useful input to a review of the design and content of the programme. Equally, comparison with similar programmes from other countries, both regionally and further afield, could be considered. Recommendation 1That DTVET initiate a review of the DTVE curriculum by external consultants and with the involvement of a wider set of stakeholders. This should include comparisons with similar programmes offered both regionally and internationally. Integrated Key Skills (KS)The DTVE programme was designed to include key skills at BTEP level 4. There are no stand alone modules; all of the desired competencies are included in the other individual modules. This is in line with current international trends ensuring that students develop and improve key skills throughout all levels study. The concern is however, that while the criteria for achievement of the key skill is embedded, how is the skill to be evidenced, how is it to be taught and by whom? The average rating of the relevance of key skills by DoE lecturers (4.3) - lower than the average rating (4.8) of the graduates. This is not statistically significant in itself. However when combined with comments from the graduates concerning the strengthening of key skills in the programme (see section 6.2), this may indicate that the teaching practices of DoE lecturers should be reviewed for effectiveness as far as the key skills component is concerned. Very few of the current DoE staff were involved in the original design of the programme, and some of the original aims and objectives may need to be re-emphasised, especially in view of the later recommendation concerning new staff induction. Further, very few of the current DoE lecturers have previous experience with an outcomes based programme and the integration of Key Skills.Recommendation 2It is recommended that FCTVE management initiate an internal review of the practices being adopted by staff to teach the embedded key skills components in the taught modules. The review should consider the effectiveness of the practices and consider ways to enhance and improve student opportunities that enable them to practice their key skill competencies to the required level. The review should further take into consideration the diverse background of students by acknowledging prior learning. Sharing of identified best practice could be included in new staff induction processes.Facilitation/FacilitatorsGenerally speaking the DTVE graduates are satisfied with the mode of facilitation used to deliver the DTVE. They are also positive about the general facilitation of the programme by the DoE lecturers who are considered professional in their approach and demonstrate good quality in their work. However, an observation made by some graduates indicated that when being taught by staff new to DoE, it was obvious that they had not been sufficiently well inducted to the programme facilitation and assessment procedures. This was confirmed by informal comments by DoE staff, who recently joined the department. It is good practice to ensure that new staff are properly inducted to both the institution and the specific department in which they will work. It is usually carried out by a group of experienced practitioners, each contributing a particular area of expertise to the induction programme. It should involve appropriate verifiable activities.Some comments indicated that the real-world business and industry knowledge of some lecturers could be improved. As is the case for vocational lecturers, short industrial placement is an important and appropriate way to keep up-to-date with modern practices and trends and could be considered by DoE management and staff alike.Recommendation 3That the FCTVE management initiate an internal review of the continuous professional development of DoE staff. This should include vocational placements and the induction processes for new teaching staff. The following should be considered:Include an initial training needs analysis for new teaching staff in relation to the content, facilitation and assessment of the DTVE programmeEnsure that appropriate induction and continuous professional development activities are created to meet individual staff needs. The use of ICT could usefully be explored here e.g. online induction coursesCreate opportunities e.g. short industrial placement, or attachment to vocational departments in TCs, for staff to become familiar with the vocational (BTEP) areas of their students in order to contextualise DTVE content to student teachers vocational areasThe Programme Structure/Portfolio AssessmentThere is general agreement that the structure and sequencing of the programme is effective. The portfolio assessment is considered appropriate, fair and valid. However, concerns were raised by both DoE teachers and the graduates. The overall workload is too heavy for the time availableInsufficient time available to develop and practice new skillsIssues concerning plagiarismAssessment reliabilityGathering evidence is time-consumingA heavy workload can have the effect of reducing the quality of facilitation e.g. delays in providing feedback, the quality of feedback. To save time, students may be tempted to copy evidence generated by other students (plagiarism). One frequently suggested way to address this issue was to increase the number of staff in the DoE. This might address the workload of the DoE staff, but is not guaranteed to resolve the issue for the students. It might be expected that the curriculum review recommended earlier would consider this issue in some depth; however, there may be some useful efficiencies that could be considered by the department.integrating related content from different modules so that it is taught once and the same evidence used across different module portfolios investigate other possibilities for cross referencing of evidence across module portfolios to reduce the amount of evidence which needs to be gatheredreview timetabled staff allocations e.g. a single lecturer teaching the same module to all student groups in a single cohort. This would reduce preparation time and reduce the time required for internal verification (since the overall number of portfolios to be internally verified would be reduced). It could also be expected to improve the reliability of assessment.use more flexible / blended learning approaches to reduce face-to-face contact timeConsider stricter use of deadlines and the use of student progress filesintroduce Accreditation of Prior Learning (APL) to recognise the prior experience and knowledge of students, many of whom already have extensive teaching experience. This could lead to exemption from certain modules and/or learning outcomes.investigate how modern technology can be used to make more effective and efficient use of the time available for the programme e.g. electronic submission and tracking of evidenceThe issue of plagiarism is a growing concern internationally. Student use of ICT makes the copying and sourcing of others work, much easier than before. It is obviously a concern that individual students obtain the diploma by their own efforts and study and not through some form of academic deceit. However, the issue is arguably more serious from an institutional point of view. Educational institutions have a duty of due diligence to ensure that the status or perceived value of its qualifications remains high. A qualification which may be obtained simply by purchasing a portfolio from a previous student, is of little value. Any institution which ignores the trends is failing in its duty.This is an urgent issue which needs to be given a high priority by the DoE. Effective approaches could be:Consider the use of computer software, or subscription to online plagiarism detection servicesRegularly change the nature or form of evidence to be gathered across student cohorts and groupsStudent induction activities to emphasise the nature of plagiarism as academic fraud and clearly outline the measures taken by the department to combat it, and the penalties attached FCTVE Plagarism Policy made available to students during inductionRecommendation 4While awaiting the outcome of Recommendation 1, the DoE should internally review its assessment practices, timetabling and the use of modern technology to seek efficiencies which reduce the work load of staff and students. Recommendation 5That FCTVE management initiate an investigation to determine the extent of plagiarism on the DTVE programme and to propose effective methods and procedures which may be applied to combat it.Learning support to studentsAn important component of an educational training programme is the learning support system available to the learners. This becomes even more important, for the new flexible and blended learning approaches expected to be developed at FCTVE using e-learning and distance learning. The support available and given to student teachers during the DTVE programme is rated by student-teachers as adequate in meeting their needs. However, both DoE lecturers and DTVE graduates feel that it is inefficient. As might be expected, amongst those who needed support beyond the normal end of the programme, the satisfaction rating of the support system is very low (see section 6.2). A good departmental learner support system will generally include:An extensive programme of inductionAssigning a named support/programme tutorFirst line pastoral careAppeals and other support for student advocacyRegular measure of student satisfaction across a range of indicatorsRegular in-course progress guidance using progress filesExit guidance at normal end of programmePost-course guidance when requiredPublished post-course reviews and performance indicatorsThe following table is copied from section 6.6CohortPre serviceIn serviceWithdrawn from courseSupplement ingPassedFailedC112305 (33%)9 (60%)6(40%)C227189 (20%)4 (11%)32 (71%)4 (9%)C3 FT28233 (6%)48 (100%)43 (84%)5 (10%)C3 Block-151 (7%)14 (100%)5 (33%)9 (60%)C4331710 (20%)40 (100%)33 (66%)7 (14%)Learning support is now internationally recognised to be a holistic strategy aimed at ensuring a quality experience for learners in support of their optimal achievement. The cost to government of providing financial support to learners, maintaining and staffing institutions is high. Retention and pass rates are an important key performance indicator which helps to assess the cost effectiveness of educational policy and government spending. Institutions need some means by which they can be assessed for cost-effectiveness and performance against national benchmarks. Quality frameworks and measures of performance are readily available for comparison from other countries and equivalent teacher-training programmes. Recommendation 6It is recommended that DoE initiate a review of the DTVE learner support system currently in place. Best practice from similar regional and international institutions should be consulted. It would also be appropriate to maintain records of student retention and pass rates for current and future student cohorts.Teaching Placement (TP)TP is an integrated part of the DTVE programme. The students go for 3 months on TP during their study. In-service student teachers return to their institutions for TP while for the pre service student teachers a place is found in a TC or a brigade that offers programmes in the vocational area of specialization of the DTVE student. During TP the student-teachers collect evidence for various portfolios. Some of the evidence is obtained through observation – DoE lecturers use observation checklists during classroom sessions.Some concerns about TP were raised during interviews with the management/supervisors group and relate mainly to the communication, coordination and effectiveness of TP. Mentioned were:No regular consultation or meetings with institutions to coordinate TP, leading sometimes to inconvenience and inappropriate timing.Student teachers concentrate on evidence collection and consider the placement period too short to effectively practice their facilitation skillsStudent teachers should fully participate in the institution during TP, taking over the full responsibilities of a lecturer / instructor under close supervision of the mentorLimited or no resources for student teachers to use, but needed for evidence collection e.g. use of data projector, OHT etc.With training, institution based mentors could be used for some observational assessment activities thus reducing the workload of the DoE lecturers and reducing costs overall.With the relocation of the DoE to FCTVE new opportunities for student-teachers to collect evidence at the college itself e.g. establishing learners? needs, giving academic support to learning, running a PowerPoint session with a group of on-campus learners could all be done on campus. Each student teacher could be assigned to a lecturer in the vocational Departments. Opportunities for the collection of portfolio evidence could be identified e.g. teaching single topics to groups of learners. This would create time for ‘full-time facilitation’ during TP at other institutions.Recommendation 7It is recommended that the planning, coordination and arrangements for TP are reviewed, taking into consideration that the DTVE is now delivered within a technical college with a wide range of vocational departments. This could permit regular interaction with the vocational departments, its lecturers and learners. The review should consider the feasibility of: Involving experienced instructors and lecturers, after training in the assessment of the student-teachersGradually giving a full observation/teaching load to the student teacher during TP under the supervision and guidance of the institutional based mentor.Literature ReferenceExit Survey – DTVE Diploma in Technical and Vocational Education ProgrammeThe survey was carried out in January 2008 by the TA Team with a group of 8 DTVE block release students shortly before their completion of the programme. Block release DTVE programme mode implied that the student-teachers would be full time on campus for the first two trimesters (July – December 2005) and after that alternate between working as instructors in their Brigade for one trimester (started first trimester 2006) and being full time student-teachers on campus for the following semester. As the DTVE programme is a 18 month programme for full time student-teacher, this made the programme for the block release group to last 3 years. The purpose of the survey was to gain understanding of the learning experiences of the participants in the block-release DTVE programmeThe main findings wereThe content of the programme was seen as relevant and useful (applicable) to the work as an instructor in a brigade.The portfolio assessment mode was seen as an appropriate way to assess the learning outcomesGreat dissatisfaction with the quality of the facilitators of the programmeSupport perceived to be of good standard during the initial block deteriorated later in the course (progress grids, personal tutor system cancelled; great delays – up to two years – in feedback) Lack of communication between DoE and brigade management on the nature, content and mode assessment of the DTVE programmeFor the full report see Annex LThe findings of the exit survey are in line with and support the following findings in this report The content of the DTVE programme is relevant and applicable in the work situationPortfolio assessment is an appropriate assessment mode for the DTVE programmeWeaknesses in the portfolio assessment system (time consuming, authenticity, reliability of assessment, turn over time and nature of feedback) need to be addressedThe need for strengthening the underpinning theories and philosophies .The support available and received from DoE lecturers is of good quality.Some weaknesses in the support system need to be addressed e.g. support to student-teachers placed in the distance learning mode BOTA surveyBota carried out a tracer study for vocational training graduates (phase 1) in 2005. The main aim of the survey out this was to track down vocational training graduates in order to identify and address mismatches between former learner’s jobs and the training received.The main conclusion from the tracer study is that the vocational training system does not produce readily employable people mainly because of the absence of practical experience in the training programme.This conclusion is identical to concerns expressed by learners, DTVE graduates and management. This relates to confusion over the purpose of the Foundation and Certificate programmes: are these programme an exit or bridging qualification. With the Government Grant Loan Scheme now applicable to learners in the TVET system for certificate – advanced certificate and diploma level, the three perceived as an entity ,it can be deduced that exit into industry / self employment should be at diploma level. However with no BTEP advanced certificate and/or diploma holders produced as yet it is still to be established whether BTEP effectively prepares learners for the world of work i.e. whether or not they are (self) employable. The delays in offering the BTEP Advanced Certificate and Diploma levels contributed to negative perceptions of BTEP among learners, management and DTVET graduates. The stronger work experience component, which is a feature of both the Advanced Certificate and DiplomaBTEPs, will also help to combat the negative image of the BTEPs.Consultancy to Conduct a Tracer Study of Graduates of the Botswana Technical Education Programme, 2007The overall objective of this tracer study on BTEP graduates (2002 to 2006) was to find out about the graduates1. employability in terms of course relevance and quality for the world-of-work; 2. employment or self employment. Some of the findings werelack of BTEP awareness by employers who may be prefer to employ an NCC or other better known qualification.There is need to re-consider the programme structure of BTEP, especially with regard to the balance between theory and practical assignments as both employers and graduates identified this as an area of weaknessRespondents also identified problems with the quality and education levels of instructors. A serious concern for BTEP is that half of the total graduate respondents are still unemployed or economically inactive.These findings are identical to the concerns mentioned in section 6.7 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download