2016 Minnesota English Language Arts Standards and Multi ...

2016 Minnesota English Language Arts Standards and Multi-Tiered System of Supports Implementation Survey

JULY 2016

Overview

The English Language Arts standards and Multi-tiered System of Supports (MTSS) status report provide a picture of state-wide implementation.1 MTSS refers to a framework that provides schools with an integrated system of high-quality, standards-based instruction and interventions that are matched to students' academic, social-emotional, and behavioral needs. This report was designed to provide a status update on state-wide implementation of MTSS as it applies to English Language Arts and lessons learned in implementing MTSS over time. An important lesson from this report is that full implementation of the standards and MTSS requires a systemic approach that unfolds over many years.

The survey has been conducted annually to inform stakeholders of changes in the statewide implementation of MTSS and focus technical assistance and support. Wilder Research was commissioned by the Minnesota Department of Education to conduct the annual surveys starting in 2013.

A total of 710 schools in 2013, 749 schools in 2014, 623 schools in 2015 and 721 schools in 2016 completed the survey, representing 46 percent, 49 percent, 39 percent and 47 percent of all elementary, middle, high schools and charter schools in 2013, 2014, 2015 and 2016, respectively.

Executive Summary

The results of the annual ELA standards and MTSS Implementation Survey make it clear that implementation does not occur at a constant rate of growth. Rather, it occurs through a multi-year process of improvement that may see "stagnant" or even "backwards" progress at times.

First, schools are making slow progress in implementing the 2010 English Language Arts Standards (ELA) and an MTSS system. Results show that at the current rate of growth, both the 2010 ELA standards and MTSS can be projected to be fully implemented by all schools in 2064. Adding to the challenge is the push to implement multiple complex initiatives simultaneously. For example, open

1

MTSS and Response to intervention are viewed as similar concepts by the Minnesota Legislature and Department of Education; however, since 2012, education

leaders have witnessed a systematic movement away from RtI toward MTSS.

1

item responses indicate schools are implementing Positive Behavioral Interventions and Supports, Math standards, and Social-Emotional Learning. Second, high schools are implementing the ELA standards and MTSS at a slower rate than elementary and charter schools. High schools rate themselves at the installation stage while elementary and charter schools are at the partial implementation stage. This is the third year in a row that high schools have rated themselves at an earlier stage of implementation than other grades. Third, schools are at a more advanced stage of implementing Leadership Supports and Supplemental and Intensive Interventions than Core Instruction and the ELA standards. For example; Items related to fully implementing a system of assessments to be used for instructional decision making are rated as being further along in implementation than items related to knowing the standards, how to design instructional sequences, and how to differentiate instruction based on student needs. Fourth, turnover in key leadership and content positions is pervasive; over 40 percent of key leaders changed between 2015 and 2016. This is the third year in a row with turnover in the double digits. A continuous rate of turnover is likely to present a challenge for increasing and sustaining full implementation. Finally, survey respondents in 2016 indicated that lack of access to training and system capacity as top barriers in implementing MTSS at their school. A similar response from schools has come up in every year the survey has been administered. The results of this report will be used for action planning at the state (MDE), district, and school levels (for more information on what the Minnesota Department of Education is doing see page 26). One of the most significant areas to focus on will be the development of sustainable training and coaching infrastructure supports. A focus on training and coaching supports will mitigate potential impacts of turnover and increase access for school teams. A range of topics will need to be addressed in training, some of which may include developing communication and collaboration structures, selecting and using evidence-based practices, and measuring and using fidelity data for increasing implementation.

2

What's covered in this report?

Overview................................................................................................................................................. 1 Executive Summary ................................................................................................................................ 1 What's covered in this report?................................................................................................................. 3 What is MTSS? ....................................................................................................................................... 4 What does the survey measure? .............................................................................................................. 4

Leadership Supports............................................................................................................................ 4 Core Instruction (Tier 1) ..................................................................................................................... 5 Supplemental and Intensive Intervention (Tiers 2 and 3) ................................................................... 5 English Language Arts (ELA) Standards ........................................................................................... 5 How are the items rated?..................................................................................................................... 6 Results from the 2016 survey ................................................................................................................. 7 Demographics of Responders ............................................................................................................. 7 Implementation Result by Grades Served........................................................................................... 7 Some Schools are at Full Implementation .......................................................................................... 9 Implementation has Increased Since 2014.............................................................................................. 9 Persistent Patterns in Implementation............................................................................................... 11 Portrait of Implementation Across Time: Results from Schools Completing the Survey from 20142016....................................................................................................................................................... 12 Analysis of Each Area of Focus............................................................................................................ 14 Leadership Supports.......................................................................................................................... 15 Core instruction................................................................................................................................. 18 Supplemental and intensive interventions ........................................................................................ 22 English Language Arts Standards ..................................................................................................... 24 Minnesota Department of Education's Work from 2015-2016 ............................................................ 26 Additional resources for training and managing implementation......................................................... 27 Appendix A: Pairwise comparisons of focus areas............................................................................... 28 Appendix B: Item level responses for changes between 2015 and 2016.............................................. 32

3

What is MTSS?

MTSS is a school-wide framework that relies on multiple tiers of instruction that work together as a safety net to prevent school failure. The critical features of the framework include screening, progress monitoring, and data-based decision making for instruction and movement of students within the system of supports. School leaders and teachers use data obtained through the framework process, to improve organizational supports, instruction, and make decisions about students at risk of not meeting grade-level expectations. Effective implementation of the framework ensures that all students receive instruction that is evidence-based and leads to proficiency in areas to which it is applied. In this survey, we focus on the Minnesota ELA standards.

What does the survey measure?

The survey was designed to examine the school-wide implementation of the core features of MTSS as applied in the context of the Minnesota ELA standards (MDE, 2015)2. The survey measures leadership supports and implementation of core instruction (Tier 1) and supplemental and intensive intervention (Tiers 2 and 3). These core features are described below. The survey items within these core features can be found at the end of this report.

Leadership Supports

Effective leadership is essential to the development and continuing improvement of any organization. The survey items are designed to represent different levels of action that leadership teams provide during implementation efforts. The items are organized into four subscales:

Commitment. Leadership commits to holding sustained attention and guiding commitment from the staff. Infrastructure Supports. Leadership ensures that each school has financial, material, and programmatic resources adequate to provide each student an equitable opportunity to learn and achieve success. Data-based Decision. Leadership communicates expectations and holds staff accountable for using data to make decisions and continuously challenge all students with a rigorous, culturally relevant curriculum. Systematic Improvement. Leadership reviews and analyzes the efficiency and effectiveness of curriculum, instruction, assessment, and professional development on student performance and makes systematic improvements.

2 The survey items were developed in collaboration with an advisory group in 2013 and 2014 and the grouping of the items for MTSS core features were adjusted for 2016 by the Minnesota Department of Education to provide schools with clearer understanding of the framework and to support schools in their action planning process.

4

Core Instruction (Tier 1)

Core instruction aims to meet the needs of all students and lead all students to make progress and achieve proficiency in the state standards. The survey items are designed to represent the presence of infrastructure supports that must be in place for core instruction to fulfill its purpose. The items within core instruction are organized into the following subscales.

Curriculum and Instruction. Standards-based curricula and instruction are articulated and well understood by all staff who teach and reinforce student learning, are designed and delivered at the appropriate level of depth and rigor, are grounded in culturally responsive practices, and are Universally Designed and differentiated based on student needs. Assessment. A system of assessments is used to inform instruction and programmatic improvements that include multiple measures useful for determining how students are responding to core curriculum and instruction. Also included in this section are questions on training for accurate administration and interpretation of data, data systems that support timely access, and communication of results. Collaboration. Time is protected and sufficient for staff with different specialties to collaborate on interpreting data, designing instruction, and solving problems. Data-based Decision. A process of inquiry and procedures for using the data for all students are in place in core instruction.

Supplemental and Intensive Intervention (Tiers 2 and 3)

Supplemental and intensive intervention must be designed to support students in closing gaps in understanding of and achievement in the subject area. The critical features within this area include:

Intervention. Interventions are evidence-based, matched to student needs, aligned with standards, and effective for students receiving them. Assessment. A system is in place and consistently used to review student progress. Collaboration. Time is established and protected, so staff with different specialties and parents area able collaborate in problem-solving, support efficient implementation, and communicate effectively . Data-based Decision. Defined procedures for problem solving and decision making are known and consistently implemented.

English Language Arts (ELA) Standards

Items measuring the implementation of ELA standards are embedded within the three MTSS core features. Implementation of the ELA standards is cyclical, and the process follows a sequence from understanding to aligning curriculum, assessments, and instructional practices to making iterative improvements using student performance data. All the steps in the process are articulated in the Mapping the Journey section of the Minnesota ELA Standards Implementation Toolkit. Survey items follow steps in this process.

5

Common Understanding. Staff members have a common understanding of what the standards are, the depth and rigor required, as well as access to a curriculum scope and sequence that makes it possible to cover all the benchmarks in a grade. In Practice. Staff can put this understanding into practice and review student progress. Continuous Improvement. Staff members are gathering and using data to make improvements. How are the items rated? School principals were strongly encouraged to have the team responsible for supporting MTSS be involved in rating the survey as a team. Teams rated items along a continuum measuring the depth of implementation. The definitions of each rating are below. Not in Place: No action is occurring in this area. This item is not a priority at this time. Exploring: At this stage, the school spends time identifying needs and exploring possible solutions: increasing knowledge, building awareness, communicating intentions, or developing plans. School representatives may attend training with the express purpose of bringing information back. Installing Infrastructure: At this stage, the school acquires data systems, assessments, and/or trains staff on selected practices. Students are not yet receiving the benefits of what teachers have been trained to do. Partial Implementation: At this stage, a few or some staff are implementing their training and making use of the infrastructure supports on a daily basis. Some but not all practices and processes are being used school-wide. Partial implementation applies to the consistent use of the practices with a few grade levels when the intention is for all staff to use them. Full Implementation: At this stage, the school has successfully moved to deepening understanding and refining use of data and practices; implementation of practices and use of data has become ingrained as a way of being for the staff. Leadership continues to plan training and coaching to prevent erosion and promote sustainable implementation. Accountability and monitoring fidelity of practices are the norm.

6

Results from the 2016 survey

The survey has been conducted annually to inform stakeholders of changes in the statewide implementation of MTSS and focus technical assistance and support. There are five critical findings from the 2016 survey. Each will be discussed in turn with additional data.

1. Schools are making slow progress in implementing an MTSS system with the English Language Arts Standards (ELA). The progress of high schools lags behind Elementary, Middle, and Charter Schools.

2. Schools are at a more advanced stage of implementation in Leadership Supports and Supplemental and Intensive Interventions than with Core Instruction and the ELA standards.

3. The percentage of turnover in key leadership positions is high and likely presents a challenge for increasing and sustaining full implementation

4. Schools are simultaneously implementing multiple complex initiatives without achieving full implementation and fidelity with MTSS and ELA.

Survey respondents in 2016 indicated that lack of access to training and system capacity as top barriers in implementing MTSS at their school. A similar response from schools has come up across the years of the survey.

Portrait of Responders

A total of 721 schools responded to the survey in 2016; 53 percent were elementary, 42 percent middle, 40 percent high school, and 39 percent of them were charter schools. The majority of survey responses were completed by a single person (467 of 721 respondents), most often the principal (399 of 467). When responses were inclusive of team members, those members included academic dean or dean of students, RtI coach, reading specialist, general and special education teachers, school psychologists and school counselors.

Percentage of Schools Completing the Survey Out of All Schools

Grade level

2013 Survey Percent Total N

2014 Survey Percent Total N

2015 Survey Percent Total N

2016 Survey Percent Total N

Elementary/K-12

47%

825

51%

819

43%

816

53%

735

Middle School

37%

206

47%

205

38%

209

42%

222

High School

46%

355

42%

352

36%

368

40%

378

Charter (all grades)

52%

163

52%

163

32%

194

39%

206

Source: 2016 Minnesota English Language Arts Standards and Multi-Tiered System of Supports Implementation Survey Findings, Wilder Research, July 2016

Implementation Result by Grades Served

In general, results of the 2016 statewide survey show participating Minnesota schools have "partially implemented" MTSS features. Analyses of grades served, reveals differences in stage of implementation. For

7

the third year in a row, elementary, middle, and charter schools averaged partial implementation, while the high schools averaged installing infrastructure stage in all areas.

a The results here are presented based on the average score of items in the respective area. Stages of implementation in each area are categorized as: "not in place" (mean score: 0.0-1.49), "exploring" (mean score: 1.50-2.49), "installing infrastructure" (mean score: 2.50-3.49), "partial implementation" (mean score: 3.50-4.49), and "full implementation" (mean score: 4.50-5.0). an ELA items are distributed across the focus areas (leadership supports, core instruction, and supplemental and intensive intervention) but are also pulled out into its own focus area.

Looking at the data from a different perspective we see that implementation by focus shows some variation; however, the biggest story is that no focus areas are at full implementation. Supplemental and intensive instructional supports are more available in elementary and charter schools than middle and high schools, a trend mirrored in national data. Implementation of core instruction tends to be higher than the ELA standards focus area because core instruction includes assessment, using data, providing feedback, and infrastructure supports. The ELA focus area only includes items that relate to the implementation of the standards. Therefore, it is possible for schools to rate their implementation of core instruction higher than the ELA standards even though the standards are embedded within the Core Instruction cluster. Another takeaway is that the summative data masks some of the unique challenges that show up when the focus areas are broken down. Example findings to be looking for later in the report include:

? Leadership commitment is more prevalent than use of data to make timely adjustments and continuous improvements over time.

? Some schools report significantly higher levels of implementation of supplemental and intensive interventions than their implementation of core instruction.

8

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download