Investigating the Relevance and Importance of English ... - ERIC

Research Memorandum

ETS RM?16-08

Investigating the Relevance and Importance of English Language Arts Content Knowledge Areas for Beginning Elementary School Teachers

Michelle P. Martin-Raugh Clyde M. Reese Geoffrey C. Phelps

Richard J. Tannenbaum Jonathan H. Steinberg Jun Xu

August 2016

ETS Research Memorandum Series

EIGNOR EXECUTIVE EDITOR

James Carlson Principal Psychometrician

ASSOCIATE EDITORS

Beata Beigman Klebanov Senior Research Scientist

Heather Buzick Research Scientist

Brent Bridgeman Distinguished Presidential Appointee

Keelan Evanini Research Director

Marna Golub-Smith Principal Psychometrician

Shelby Haberman Distinguished Presidential Appointee

Anastassia Loukina Research Scientist

Donald Powers Managing Principal Research Scientist

Gautam Puhan Principal Psychometrician

John Sabatini Managing Principal Research Scientist

Matthias von Davier Senior Research Director

Rebecca Zwick Distinguished Presidential Appointee

Kim Fryer Manager, Editing Services

PRODUCTION EDITORS

Ayleen Gontz Senior Editor

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Memorandum series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Memorandum series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.

The Daniel Eignor Editorship is named in honor of Dr. Daniel R. Eignor, who from 2001 until 2011 served the Research and Development division as Editor for the ETS Research Report series. The Eignor Editorship has been created to recognize the pivotal leadership role that Dr. Eignor played in the research publication process at ETS.

Investigating the Relevance and Importance of English Language Arts (ELA) Content Knowledge Areas for Beginning Elementary School Teachers

Michelle P. Martin-Raugh, Clyde M. Reese, Geoffrey C. Phelps, Richard J. Tannenbaum, Jonathan H. Steinberg, and Jun Xu

Educational Testing Service, Princeton, New Jersey August 2016

Corresponding author: Michelle P. Martin-Raugh, E-mail: mmartin-raugh@ Suggested citation: Martin-Raugh, M. P., Reese, C. M., Phelps, G. C., Tannenbaum, R. J., Steinberg, J. H., and Xu, J. (2016). Investigating the relevance and importance of English language arts content knowledge areas for beginning elementary school teachers (Research Memorandum No. RM-16-08). Princeton, NJ: Educational Testing Service.

Find other ETS-published reports by searching the ETS ReSEARCHER database at

To obtain a copy of an ETS research report, please visit

Action Editor: Heather Buzick Reviewers: Leslie K. Nabors Olah and Kevin Petway

Copyright ? 2016 by Educational Testing Service. All rights reserved. ETS and the ETS logo are registered trademarks of Educational Testing Service (ETS). MEASURING THE POWER OF LEARNING is a trademark of ETS. All other trademarks are the property of their

respective owners.

M. Martin-Raugh et al.

Investigating the Relevance and Importance of ELA Content Knowledge Areas

Abstract The purpose of this report is to explore the content-related validity evidence supporting the English language arts (ELA) components of the ETS? National Observational Teaching Exam (NOTE) assessment series, a kindergarten through 6th-grade teacher licensure assessment. To establish the content knowledge required for the effective teaching of ELA in elementary school, we (a) identified content knowledge categories through the use of an expert panel and (b) surveyed a sample of 279 educators to verify that this body of content knowledge is indeed necessary and reasonable for the effective practice of beginning elementary school teachers teaching ELA. We report information regarding the importance and relevance of ELA content knowledge areas for both elementary school teachers and faculty members who prepare elementary school teachers. Implications of this work for the ELA components of the NOTE assessment series are discussed.

Key words: English language arts, content validity, teacher licensure, elementary school teaching, content knowledge for teaching

ETS RM-16-08

i

M. Martin-Raugh et al.

Investigating the Relevance and Importance of ELA Content Knowledge Areas

Acknowledgments Some of the content that appears in this report also is used in a companion report, entitled Investigating the Relevance and Importance of High-Leverage Mathematical Content for Beginning Elementary School Teachers (RM-16-10) by Clyde M. Reese, Michelle P. MartinRaugh, Heather Howell, Richard J. Tannenbaum, Jonathan H. Steinberg, and Jun Xu.

ETS RM-16-08

ii

M. Martin-Raugh et al.

Investigating the Relevance and Importance of ELA Content Knowledge Areas

Table of Contents Page

Content-Related Validity Evidence ................................................................................................ 2 Establishing a CKT Framework for ELA.................................................................................... 3

Method ............................................................................................................................................ 6 Sample ......................................................................................................................................... 6 Administration Procedure............................................................................................................ 8 Survey Instrument........................................................................................................................ 8

Table 1. Background Information--Overall Sample (N = 279) ..................................................... 9 Analysis......................................................................................................................................... 10 Results........................................................................................................................................... 10

Overall Agreement Concerning Relevance and Importance Judgments ................................... 10 Group Differences in Relevance and Importance Ratings ........................................................ 11 Table 2. Summary of Importance Judgments for Content Knowledge for Teachers (CKT) Areas for Teachers, Faculty, and Overall............................................................................................. 12 Table 3. Summary of Importance Judgments for Content Knowledge for Teachers (CKT) Areas by Current Grade Level Taught ................................................................................................. 13 Table 4. Summary of Importance Judgments for Content Knowledge for Teachers (CKT) Areas by Race/Ethnicity....................................................................................................................... 14 Table 5. Summary of Importance Judgments for Content Knowledge for Teachers (CKT) Areas by Geographic Region ............................................................................................................... 15 Table 6. Summary of Least and Most Important Judgments for Content Knowledge for Teachers (CKT) Areas............................................................................................................................... 16 Table 7. Top English Language Arts (ELA) Content Knowledge for Teachers (CKT) Areas and Practices by Relative Importance............................................................................................... 17 Discussion ..................................................................................................................................... 17 Conclusion ................................................................................................................................. 18 References..................................................................................................................................... 20 Appendix. English Language Arts (ELA) Content Knowledge for Teaching (CKT) Areas ........ 23 Notes ............................................................................................................................................. 30

ETS RM-16-08

iii

M. Martin-Raugh et al.

Investigating the Relevance and Importance of ELA Content Knowledge Areas

The purpose of this report is to explore the content-related validity evidence supporting the English language arts (ELA) content knowledge for teaching (CKT) components of the ETS? National Observational Teaching Exam (NOTE) assessment series. NOTE is a kindergarten through 6th-grade licensure assessment developed in a collaboration between Educational Testing Service (ETS) and TeachingWorks (). The NOTE assessment series is designed to measure a prospective elementary school teacher's ability to translate his or her knowledge of content and of teaching into effective teaching practice. The NOTE assessment series includes two components. One component includes standardized performance assessments that focus on three high-leverage practices (HLPs) for teaching: modeling and explaining content, evaluating student thinking, and leading a classroom discussion. TeachingWorks defines HLPs as "the basic fundamentals of teaching. These practices are used constantly and are critical to helping students learn important content. These high-leverage practices are used across subject areas, grade levels, and contexts" (2016b, para. 2). The NOTE assessment series assesses high-leverage content, defined as the specific topics, practices, and texts that have been put forward by TeachingWorks as foundational to the K?12 curriculum and crucial for beginning teachers to be able to teach (TeachingWorks, 2016a). This content, organized by subject area and grade level, is rooted in national and state standards for student learning that have been crafted with the involvement of key professional groups.

The second component focuses on CKT. CKT is a theory derived via job analysis and based in practice that outlines the content knowledge required for teaching a subject (ETS, 2011). Each component must include tasks that identify CKT necessary for the effective teaching of ELA. According to the Standards for Educational and Psychological Testing (American Educational Research Association [AERA], American Psychological Association [APA], National Council on Measurement in Education [NCME], 2014), a critical element of licensure assessments that focus on beginning teaching proficiency in subjects such as ELA is valid frameworks that define the ELA CKT domains.

The remainder of this report is organized as follows. The first section provides a brief overview of the role of content-related validity evidence in licensure assessment. The second section describes the process we used to generate the ELA CKT framework. The third section describes the study design, methods, and results. Finally, we conclude by discussing the implications of this work for the ELA components of the NOTE assessment.

ETS RM-16-08

1

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download