Program Evaluation Rubric - Weebly



Program Evaluation RubricAlison ChateauneufAET 570February 23, 2015Randy HowellProgram Evaluation RubricThis program evaluation will take a look at measuring the success of a current training in a call center environment. The program focused on for this evaluation will be homeowner’s insurance training. This module style training runs for a three day timespan and walks participants through cost evaluators and homeowners insurance products with the overall goals of making them proficient in navigating the systems we use to quote, and accurately assess replacement value on a property that our policyholder is looking to insure. This program will be deemed successful based on whether it can meet certain concepts of Kirkpatrick’s four levels of evaluation. Kirkpatrick (2008) describes the four levels of evaluation as follows:Reaction: A measure of the satisfaction of the participants who attended the program (p. 486).Learning: The extent to which participants increased their knowledge, learned or improved present skills, or changed their attitudes (p. 486).Behavior: The extent to which participants applied what they learned when they returned to their jobs (p. 486).Results: The improvement of morale, the increase in sales or production, the reduction in turnover, the increase in customer satisfaction, the return-on-investment (ROI), and any other benefits that came from attending the program (p. 486).In keeping with this method of thought, the following rubric has been designed to give an overall evaluation of this program.Level 1: ReactionUnsatisfactorySatisfactoryExemplaryCommentsTraining DesignWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 0-3.4 when asked about the course objectives, topic, pace and level of difficultyWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 3.5-4.4 when asked about the course objectives, topic, pace and level of difficultyWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 4.5-5 when asked about the course objectives, topic, pace and level of difficultyThe level 1 evaluation is sent directly to participants and consists of several questions in each category that will average together in order to provide the overall results. The rating scale is 0-5.InstructorWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 0-3.4 when asked about the instructors overall performance, knowledge, time management, and responsivenessWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 3.5-4.4 when asked about the instructors overall performance, knowledge, time management, and responsivenessWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 4.5-5 when asked about the instructors overall performance, knowledge, time management, and responsivenessThe level 1 evaluation is sent directly to participants and consists of several questions in each category that will average together in order to provide the overall results. The rating scale is 0-5.ExercisesWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 0-3.4 when asked if the exercises were helpful and meaningfulWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 3.5-4.4 when asked if the exercises were helpful and meaningfulWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 4.5-5 when asked if the exercises were helpful and meaningfulThe level 1 evaluation is sent directly to participants and consists of several questions in each category that will average together in order to provide the overall results. The rating scale is 0-5.ApplicationWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 0-3.4 when asked if the information provided during this training program was relevant to their jobWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 3.5-4.4 when asked if the information provided during this training program was relevant to their jobWhen provided a formal level 1 evaluation the average participant score on the level 1 evaluation ranged between 4.5-5 when asked if the information provided during this training program was relevant to their jobThe level 1 evaluation is sent directly to participants and consists of several questions in each category that will average together in order to provide the overall results. The rating scale is 0-5.LogisticsWhen provided a formal level 1 evaluation the participant gave all did not meet expectations ratings when asked about facilities, seating, and lighting during the trainingWhen provided a formal level 1 evaluation the participant gave all met expectations ratings when asked about facilities, seating, and lighting during the trainingWhen provided a formal level 1 evaluation the participant gave all exceeded expectations ratings when asked about facilities, seating, and lighting during the trainingThe level 1 evaluation is sent directly to participants and consists of several questions in each category that will average together in order to provide the overall results. The rating scale is 0-5.Level 2: LearningUnsatisfactorySatisfactoryExemplaryCommentsFormal Level 2 EvaluationAverage participant score on the 100 question criterion-referenced level 2 learning evaluation ranges from 0-79.9%Average participant score on the 100 question criterion-referenced level 2 learning evaluation ranges from 80-89.9%Average participant score on the 100 question criterion-referenced level 2 learning evaluation ranges from 90-100%Passing score on this CRT is 80%Level 3:BehaviorUnsatisfactorySatisfactoryExemplaryCommentsSkills ApplicationWhen asked to provide level 3 feedback a majority of managers advise that skills learned in the training are not being applied in the workplaceWhen asked to provide level 3 feedback a majority of managers advise that skills learned in the training are being applied in the workplace most of the timeWhen asked to provide level 3 feedback a majority of managers advise that skills learned in the training are being applied in the workplace consistentlyBased on questionnaire handed to management and site leadership Performance EnvironmentWhen asked to provide level 3 feedback a majority of managers advise that the training has not improved employee performance or the workplace environmentWhen asked to provide level 3 feedback a majority of managers advise that the training has improved employee performance or the workplace environment in some waysWhen asked to provide level 3 feedback a majority of managers advise that the training has improved employee performance or the workplace environment consistently.Based on questionnaire handed to management and site leadershipDirect ObservationThrough process confirmations, it has been observed that employees who have attended the training are not using what they were taughtThrough process confirmations, it has been observed that employees who have attended the training are using what they were taught some of the timeThrough process confirmations, it has been observed that employees who have attended the training are consistently using what they were taughtFeedback observed by site and call center management as well as trainersLevel 4: ResultsUnsatisfactorySatisfactoryExemplaryCommentsBusiness MetricsTrained employees are not meeting the new call handle time goals of 6:58 within 30 days for this call type on the jobTrained employees are meeting the new call handle time goals of 6:58 within 30 days for this call type on the jobTrained employees are exceeding the new call handle time goals of 6:58 within 30 days for this call type on the jobMetric information based upon call center daily/weekly metric report for this categoryTrained employees are not meeting the new after call work goal of 1:13 within 30 days for this call type on the jobTrained employees are meeting the new after call work goal of 1:13 within 30 days for this call type on the jobTrained employees are exceeding the new after call work goal of 1:13 within 30 days for this call type on the jobMetric information based upon call center daily/weekly metric report for this categoryReferencesBrinkerhoff, R. O., & Mooney, T. P. (2008). Chapter 30: Level 3: Evaluation ASTD handbook for workplace learning professionals. Alexandria, VA: ASTD Press.Coscarelli, W., & Shrock, S. (2008). Chapter 29: Level 2: Learning-Five Essential Steps for Creating Your Tests and Two Cautionary Tales ASTD handbook for workplace learning professionals. Alexandria, VA: ASTD Press.Kirkpatrick, D. L. (2008). Section VI: Measuring and Evaluating Impact - Luminary Perspective: Evaluating Training Programs ASTD handbook for workplace learning professionals. Alexandria, VA: ASTD Press.Kristiansen, N. (2008). Chapter 28: Level 1: Reaction Evaluation ASTD handbook for workplace learning professionals. Alexandria, VA: ASTD Press.McCain, D. V. (2008). Chapter 31: Level 4: Results ASTD handbook for workplace learning professionals. Alexandria, VA: ASTD Press. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download