The Methods Used to Evaluate the NextEra Energy Instructor ...



The Methods Used to Evaluate the NextEra Energy Instructor Continuing Training Program FormSherry CoxWalden UniversityAuthor NoteEDIT 6130-4, Program Evaluation.The Methods Used to Evaluate the NextEra Energy Instructor Continuing Training Program FormA necessary component of performing a program evaluation is determining what data is needed to meet the project objectives and how that will be evaluated. Using a systematic methodology, the evaluation methods used for the evaluation of the NextEra Energy Instructor Training Program Continuing Training form are considered.Evaluation MethodsThis program evaluation is narrowly focused on the form used to collect and validate all information required to meet the annual continuing training requirements for an instructor of any of the accredited training programs. To meet this evaluation scope, this project will target the inputs and outputs of the Instructor CT form and determine if all required information is collected and validated at the time designated by the NextEra Fleet Instructor Training Program Description, TR-AA-101. Five evaluation methods were considered for use in the design of this evaluation; advantages and disadvantages for each method were considered (see table 1). A mixed model approach comprised of the consumer, program, and participant-oriented approaches was selected. This selection was based upon the advantages of these models and the ability to minimize the disadvantages of each as they relate to the evaluation questions. Data Collection Design and Sampling StrategyThe design for this evaluation project will use a mixed data collection methodology; combining a review of data and face-to-fact interviews. A review of completed Instructor CT forms will be reviewed to determine the following: Completeness and Pre-approvals & approvals of inputs at the specified times. The LMS will also be queried to determine if any instructors were statused as Unqualified due to failure to complete continuing training requirements. Face-to-face interviews will be performed to include a minimum of two-thirds of all qualified instructors (n=30) and all training supervisors (n=4); interviews will be semi-structured in nature. The interviews will include the evaluation questions listed in figure 2 and will allow for unstructured questions, as deemed fitting by the interviewer, to gather information required to satisfy the purpose of this evaluation. If the data review reveals any incomplete forms or unqualified instructors under the conditions specified above, additional questions may be included for those interviews involving affected individuals.The evaluation design will satisfy the needs of all stakeholders. The instructors and training supervisors, who provide the overwhelming majority of the inputs and outputs of this form, will have the opportunity to provide input of its effectiveness. These two groups, being the most intimate with the form, will provide the best indication of its value to act in its intended purpose. As the designers of the form; the ITs will not be interviewed; information pertinent to this stakeholder group, will be gathered through the review of the data.There is minimal bias anticipated during the implementation of this evaluation plan. Interviewers should begin each interview by reminding the interviewers that the goal of this project is to ensure the Instructor CT form is the best vehicle to gather the required data; the interviewee should also be reminded that the collection of relevant continuing information satisfies Federal Code of Regulations, Academy/Accreditation and procedural requirements.Interview Questions. To effectively evaluate the use of the Instructor CT form and ensure it is meeting the needs of all stakeholders; this evaluation will answer five questions that to determine the effectiveness of the form and make any necessary adjustments, five questions will be asked during this project. The evaluation questions, proposed by this project, are shown in figure 2.Two primary stakeholders should be included in the determination of which questions should be included in this evaluation; the instructors and the training supervisors. Both groups of stakeholders are vested in this form being accurate and easy to use. An Instructor CT form that is difficult to use and does not collect all of the required information will lead to both instructors and supervisors creating workarounds which could lead to noncompliance of the Training Program Description.The Instructional Technologist/Instructor Trainer (IT) at each NextEra Energy nuclear site should also be included in the question selection. As the Program Administrators for the Instructor Training Program at their respective sites, they use this form to enter completion of continuing training requirements, by the instructors in their program, into the SuccessFactors Learning Management System (LMS). The IT has to know, with certainty, that the information contained on the form is accurate and compete. Representatives from the above mentioned stakeholder groups should be asked to review the proposed questions put forth by this evaluation plan and will be asked to validate that the questions will satisfy the needs of this project.References BIBLIOGRAPHY Fitzpatrick, J. L., Sanders, J. R., Worthen, B. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines. Upper Saddle River, NJ: Pearson Education.TR-AA-101, Fleet Instructor Training Program Description [NextEra Energy Fleet Training Procedure]. (2015, March 9). Revision 3, Approved by W. Goodes for C. Sizemore TablesTable 1Table of Evaluation ModelsEvaluation ModelAdvantagesDisadvantagesEXPERTISE AND CONSUMER-ORIENTED APPROACHESEXPERTISEThe expertise method leverages the experience of the field’s experts.The expertise method allows for the evaluation of more complex programs and products.CONSUMER-ORIENTEDThe consumer-orientated approaches are best used to evaluate products, services or organizations.Helpful in determining what to buy.The evaluation is unbiased.The consumer of the product or service is the beneficiary of the evaluationEasy to use and understandEXPERTISEThe expertise method is subject to the prejudices and politics of the experts on the evaluation panel.CONSUMER-ORIENTEDMay be narrowly focusedNot open to debate PROGRAM-ORIENTED EVALUATION APPROACHESCan be used to explain the program outcomeAvoids unknowns in outcomeHeavily concentrated on research and less on stakeholdersMay over emphasize outcomesDECISION-ORIENTED EVALUATION APPROACHESMore comprehensive than other modelsUses a systematic approachSensitive to the needs of those who will use the evaluationThere is an assumption that decision making is rational and predictableThe focus is on the needs of program managersPARTICIPANT-ORIENTED EVALUATION APPROACHESThis approach pluralistic; it focuses on description and judgementThe model has multiple uses; it can be used to evaluate an organization or individual learningThere is a potential to be cost and labor intensive.Higher potential for less knowledgeable stakeholders to lead the study inappropriatelyLesser ability to replicateExplain your choice of model for your program evaluation: The model employed for this program selection is a mix of the consumer, program, and participant-oriented approaches. The decision to use a mixed-model approach over a single method is to leverage the advantages of each method and minimize the disadvantages of all in order to produce a strong and meaningful product (Fitzpatrick, 2011).FiguresFigure 1. Example of the NextEra Instructor Continuing Training and Development Form (Revision 3,2015).Figure 2. Proposed Evaluation Questions 097790Are all required inputs obtained at the time specified?What are the barriers to obtaining this information?How is the instructor’s technical continuing training validated?Are there any difficulties in determining the technical requirements? (for Operations Training Instructors only)Does the format of the instructor continuing training form allow for capture of all required information necessary to document all continuing training requirements as outlined in TR-AA-101, Fleet Instructor Training Program?Does the design of the continuing training form create any barriers to prevent capture of the instructor continuing training requirements?Do instructors and supervisors understand how to use the form?What errors have been identified on forms submitted for entry into LMS?Are any changes, to the form, needed to ensure it is meeting its intended use?Do any of the stakeholders have suggestions for improving the Instructor CT form?Are all required inputs obtained at the time specified?What are the barriers to obtaining this information?How is the instructor’s technical continuing training validated?Are there any difficulties in determining the technical requirements? (for Operations Training Instructors only)Does the format of the instructor continuing training form allow for capture of all required information necessary to document all continuing training requirements as outlined in TR-AA-101, Fleet Instructor Training Program?Does the design of the continuing training form create any barriers to prevent capture of the instructor continuing training requirements?Do instructors and supervisors understand how to use the form?What errors have been identified on forms submitted for entry into LMS?Are any changes, to the form, needed to ensure it is meeting its intended use?Do any of the stakeholders have suggestions for improving the Instructor CT form? ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download