North Carolina New Teacher Support Program

[Pages:100]Consortium for Educational Research and Evaluation? North Carolina

North Carolina New Teacher Support Program

Final Race to the Top Evaluation Report

Authors:

Kevin Bastian and Julie Marks Education Policy Initiative at Carolina, The University of North Carolina at Chapel Hill

August 2015

NC NTSP: Final RttT Evaluation Report August 2015

Table of Contents

Executive Summary ........................................................................................................................ 3 Introduction..................................................................................................................................... 6 Data and Analyses........................................................................................................................... 8

Evaluation Sample ...................................................................................................................... 8 Data Sources ............................................................................................................................... 9 Analysis..................................................................................................................................... 10 Findings......................................................................................................................................... 11 Implementation ......................................................................................................................... 11

To What Extent was the NC NTSP Implemented as Intended? ........................................... 11 To What Extent did the NC NTSP Reach its Target Population? ........................................ 11 Knowledge, Skills, and Attitudes.............................................................................................. 13 How do Teachers Perceive the Impact of NC NTSP Components on their Knowledge, Skills, and Attitudes about Teaching? .................................................................................. 13 How do Teachers Perceive the Impact of NC NTSP Components on Self-Efficacy and Job Satisfaction? .......................................................................................................................... 15 Teacher Effectiveness ............................................................................................................... 16 To What Extent does the NC NTSP Impact Teacher Effectiveness, as Measured by Teacher Value Added to Student Achievement? ................................................................................ 16 To What Extent does the NC NTSP Impact Teacher Effectiveness, as Measured by the North Carolina Educator Evaluation System (NCEES) Teacher Evaluation Ratings? ........ 18 Retention ................................................................................................................................... 20 To What Extent does the NC NTSP Impact the Retention of Novice Teachers a) in the Same Schools or LEAs, and b) in the State? .................................................................................. 20 Conclusions................................................................................................................................... 22 Summary of Findings................................................................................................................ 22 Implementation ..................................................................................................................... 22 Teacher Knowledge, Skills, and Attitudes............................................................................ 22 Teacher Effectiveness ........................................................................................................... 22 Teacher Retention ................................................................................................................. 22 Limitations ................................................................................................................................ 23 Recommendations and Next Steps............................................................................................ 23 Appendix A. Evaluation Sample................................................................................................... 25 Appendix B. Data Sources ............................................................................................................ 30 Appendix C. To What Extent was the NC NTSP Implemented as it was Intended? ................... 42

Consortium for Educational Research and Evaluation?North Carolina

NC NTSP: Final RttT Evaluation Report August 2015

Appendix D. To What Extent did the NC NTSP Reach its Target Population?........................... 44

Appendix E. How do Teachers Perceive the Impact of the NC NTSP Components on their Confidence, Knowledge, and Skills in Teaching? ........................................................................ 52

Appendix F. How do Teachers Perceive the Impact of the NC NTSP on their Self-Efficacy and Job Satisfaction? ........................................................................................................................... 58

Appendix G. To What Extent does the NC NTSP Impact Teacher Effectiveness as Measured by Teacher Value-Added to Student Achievement?.......................................................................... 61

Appendix H. To What Extent does the NC NTSP Impact Teacher Effectiveness as Measured by the North Carolina Educator Evaluation System (NCEES) Teacher Evaluation Ratings? .......... 82

Appendix I. To What Extent does the NC NTSP Impact the Retention of Novice Teachers to the Same School, LEA, and the State? ............................................................................................... 90

Consortium for Educational Research and Evaluation?North Carolina

2

NC NTSP: Final RttT Evaluation Report August 2015

NORTH CAROLINA NEW TEACHER SUPPORT PROGRAM: FINAL RACE TO THE TOP EVALUATION REPORT

Executive Summary

Overview

The North Carolina New Teacher Support Program (NC NTSP) was developed to provide induction supports to beginning teachers in North Carolina's lowest-achieving schools and to meet two high-priority needs identified by the state's Race to the Top (RttT) grant: (1) helping teachers to succeed during their initial years in teaching; and (2) retaining qualified teachers, particularly in high-need schools. The NC NTSP aims to improve the instructional knowledge, skills, attitudes, effectiveness, and retention of participating teachers through the provision of three support components: institutes (multi-day trainings); instructional coaching; and professional development. This report reflects findings from an independent external evaluation of the NC NTSP conducted by the Education Policy Initiative at Carolina as part of the statewide RttT evaluation undertaken by the Consortium for Educational Research and Evaluation?North Carolina (CERE?NC).

Data and Methods

This final evaluation report draws upon the following data sources: (1) participation records from each of the components of the NC NTSP; (2) Perception of Success Inventory for Beginning Teachers (PSI-BT) survey responses; and (3) student demographics and test scores, classroom rosters, teacher evaluation ratings, EVAAS scores, certified salaries, school personnel, and school characteristics files provided by the North Carolina Department of Public Instruction. With these data sources, the Evaluation Team used a comparison group design to examine: levels of participation in NC NTSP components; NC NTSP teachers' perceptions of the impact of program components on their confidence, knowledge, and skills for teaching; teachers' perceptions of their self-efficacy and job satisfaction; and the impact of the NC NTSP on teacher value-added, teacher evaluation ratings, and teacher retention.

Summary of Findings

This report reflects evaluation findings for the two full years of program implementation spanning the 2012-13 and 2013-14 school years. The study design and analysis were developed to address evaluation questions across four overarching areas: implementation; teacher knowledge, skills, and attitudes; teacher effectiveness; and teacher retention.

Implementation

To what extent was the NC NTSP implemented as intended, and to what extent did it reach its target population? Over the course of the four-year grant period, the NC NTSP was developed, staffed, and implemented from the ground up and has grown to scale, serving over 1,100 teachers in 114 schools as of the 2013-14 school year. There were substantial differences in program participation and implementation across the four NC NTSP regions. As the NC NTSP doubled in

Consortium for Educational Research and Evaluation?North Carolina

3

NC NTSP: Final RttT Evaluation Report August 2015

size in 2013-14, there was a drop in attendance at institutes and professional development sessions and fewer instructional coach visits per teacher.

Teacher Knowledge, Skills, and Attitudes

How do teachers perceive the impact of NC NTSP components on their confidence, knowledge, skills, and attitudes toward teaching? There was a statistically significant difference in the proportion of NC NTSP evaluation sample respondents who felt the program components had a positive impact on their teaching, compared with similar services provided by their own school. This significant difference also existed between NC NTSP respondents' perceptions of program utility and comparison sample perceptions of analogous school-provided services.

Teacher Effectiveness

To what extent does the NC NTSP impact teacher effectiveness as measured by teacher valueadded to student achievement (EVAAS) and the North Carolina Educator Evaluation System (NCEES) teacher evaluation ratings? Overall, NC NTSP evaluation sample teachers had significantly higher EVAAS estimates than comparison sample teachers in fifth and eighth grade science. When assessing results by cohort, positive and significant EVAAS results were concentrated within NC NTSP Cohort 1 teachers, while NC NTSP Cohort 2 teachers were generally no more or less effective. Regarding teacher evaluation ratings, there were no significant differences between NC NTSP evaluation sample teachers and comparison teachers in overall models. By cohort, NC NTSP Cohort 1 teachers had significantly higher evaluation ratings on four standards in 2013-14.

Teacher Retention

To what extent does the NC NTSP impact the retention of novice teachers in the same schools or local education agencies (LEAs) and in the state? Overall, NC NTSP evaluation sample teachers were significantly more likely to return to teaching in North Carolina public schools, to the same LEA, and to the same low-performing school. NC NTSP teachers from both cohorts were significantly more likely than comparison sample teachers to return to the same low-performing school.

Limitations

Two primary limitations should be taken into account when interpreting the findings in this report: (1) the ability to isolate the impact of the NC NTSP is diminished due to the lack of a comparison group of teachers working in schools similar to those served by the NC NTSP in all ways other than participating in the program; and (2) given the time required for program development and scaling up, the length of the evaluation period is not adequate to assess the effectiveness of a consistent program model implemented as intended for first-, second-, and third-year teachers over the course of three years.

Consortium for Educational Research and Evaluation?North Carolina

4

NC NTSP: Final RttT Evaluation Report August 2015

Recommendations and Next Steps

There are three overarching recommendations for the NC NTSP after the close of the RttT grant:

1. The impacts of the NC NTSP on teacher value-added to student achievement and teacher retention support a recommendation to sustain the program beyond the end of the RttT grant;

2. Findings from this evaluation should be used to explore the disparities in program implementation by region, and moving forward, implementation fidelity should be formally monitored; and

3. Strategies should be explored to counter the decline in program participation and effectiveness seen in 2013-14, such as requiring participating schools and Local Education Agencies (districts) to enforce mandatory participant attendance, and/or concentrating resources toward instructional coaching (the most intensive program component).

Consortium for Educational Research and Evaluation?North Carolina

5

NC NTSP: Final RttT Evaluation Report August 2015

Introduction

North Carolina's $400 million, four-year Race to the Top (RttT) grant was built upon a comprehensive plan to strengthen the education workforce, with the goal of having a great teacher in every classroom and a great principal leading every school. With this end in mind, the North Carolina Department of Public Instruction (NCDPI) identified the state's highest-priority needs, including the recruitment and preparation, equitable distribution, professional development, and induction and retention of high-quality teachers.1

Data from the 2010 Teacher Working Conditions Survey illustrated the need for induction supports prior to RttT, showing that while nearly all new teachers (93%) are assigned a mentor, almost half (47%) do not have time during the day to meet with their mentors, and one in eight indicate that they never received any additional support as new teachers.1,2 These supports are particularly needed in high-need schools, where novice teachers are concentrated and where the teacher turnover rate is often greatest.

To help teachers succeed during their early-career years and persist in the state's highest-need schools, NCDPI partnered with the UNC General Administration (UNC-GA) to develop and implement the North Carolina New Teacher Support Program (NC NTSP). The NC NTSP is a comprehensive induction program for early-career teachers designed to increase competency in goal-setting, backwards planning and assessment, data-driven decision-making, classroom management, and strategies for success in the school and community. The goal of the NC NTSP is to help participating novice teachers acquire the knowledge and skills necessary to increase the quality of their instruction, raise student achievement, and persist in teaching.

The NC NTSP drew from the successful induction model utilized by Teach For America to create a three part program comprised of 1) institutes (multi-day training sessions); 2) intensive face-to-face and virtual instructional coaching; and 3) professional development sessions held throughout the academic year. Implementation of these beginning teacher supports is organized through a central NC NTSP office and regional anchor sites located at four UNC system institutions--East Carolina University (ECU), the UNC Center for School Leadership Development (UNC-CSLD), UNC Charlotte (UNCC), and UNCG Greensboro (UNCG).

As part of the statewide evaluation of RttT conducted by the Consortium for Education Research and Evaluation?North Carolina (CERE?NC),3 the Education Policy Initiative at Carolina (EPIC) was tasked with conducting an independent external evaluation of the NC NTSP from the 201112 to the 2013-14 school years. Over this time period, EPIC has collected and analyzed data on program implementation and participation, short-term and intermediate outcomes, and summative impact findings to address the following evaluation questions:

1 NC RttT Proposal, 2010 2 Teacher Working Conditions Survey, 2010 3 The Consortium for Educational Research and Evaluation?North Carolina () is a partnership of: the SERVE Center, University of North Carolina at Greensboro; the Education Policy Initiative at Carolina, University of North Carolina at Chapel Hill; and the Friday Institute for Educational Innovation, North Carolina State University.

Consortium for Educational Research and Evaluation?North Carolina

6

NC NTSP: Final RttT Evaluation Report August 2015 Implementation

1. To what extent was the NC NTSP implemented as it was intended? 2. To what extent did the NC NTSP reach its target population?

Teacher Knowledge, Skills, and Attitudes

3. How do teachers perceive the impact of NC NTSP components on their confidence, knowledge, and skills in teaching?

4. How do teachers perceive the impact of NC NTSP components on their self-efficacy and job satisfaction?

Teacher Effectiveness

5. To what extent does the NC NTSP impact teacher effectiveness as measured by teacher value-added to student achievement?

6. To what extent does the NC NTSP impact teacher effectiveness as measured by the North Carolina Educator Evaluation System (NCEES) teacher evaluation ratings?

Teacher Retention

7. To what extent does the NC NTSP impact the retention of novice teachers in the same schools or LEAs and in the state?

Consortium for Educational Research and Evaluation?North Carolina

7

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download