PDF Leading a Classroom Discussion: Definition, Supporting ...

Research Memorandum

ETS RM?16-09

Leading a Classroom Discussion: Definition, Supporting Evidence, and Measurement of the ETS? National Observational Teaching Examination (NOTE) Assessment Series

Margaret Witherspoon Gary Sykes Courtney Bell

August 2016

ETS Research Memorandum Series

EIGNOR EXECUTIVE EDITOR

James Carlson Principal Psychometrician

ASSOCIATE EDITORS

Beata Beigman Klebanov Senior Research Scientist

Heather Buzick Research Scientist

Brent Bridgeman Distinguished Presidential Appointee

Keelan Evanini Research Director

Marna Golub-Smith Principal Psychometrician

Shelby Haberman Distinguished Presidential Appointee

Anastassia Loukina Research Scientist

Donald Powers Managing Principal Research Scientist

Gautam Puhan Principal Psychometrician

John Sabatini Managing Principal Research Scientist

Matthias von Davier Senior Research Director

Rebecca Zwick Distinguished Presidential Appointee

Kim Fryer Manager, Editing Services

PRODUCTION EDITORS

Ayleen Gontz Senior Editor

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Memorandum series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Memorandum series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.

The Daniel Eignor Editorship is named in honor of Dr. Daniel R. Eignor, who from 2001 until 2011 served the Research and Development division as Editor for the ETS Research Report series. The Eignor Editorship has been created to recognize the pivotal leadership role that Dr. Eignor played in the research publication process at ETS.

Leading a Classroom Discussion: Definition, Supporting Evidence, and Measurement of the ETS? National Observational Teaching Examination (NOTE) Assessment Series

Margaret Witherspoon, Gary Sykes, and Courtney Bell Educational Testing Service, Princeton, New Jersey August 2016

Corresponding author: Margaret Witherspoon, E-mail: mwitherspoon@ Suggested citation: Witherspoon, M., Sykes, G., & Bell, C. (2016). Leading a Classroom Discussion: Definition, supportive evidence, and measurement of the ETS? National Observational Teaching Examination (NOTE) assessment series (Research Memorandum No. RM-16-09). Princeton, NJ: Educational Testing Service

Find other ETS-published reports by searching the ETS ReSEARCHER database at

To obtain a copy of an ETS research report, please visit

Action Editor: Heather Buzick Reviewers: Heather Howell and Michael Kane

Copyright ? 2016 by Educational Testing Service. All rights reserved. ETS and the ETS logo are registered trademarks of Educational Testing Service (ETS). MEASURING THE POWER OF LEARNING is a trademark of ETS. All other trademarks are the property of their

respective owners.

M. Witherspoon et al.

LCD: Definition, Supporting Evidence, and Measurement of ETS? NOTE Assessment Series

Abstract This paper provides a description and rationale for a performance assessment of a teaching practice--leading a classroom discussion (LCD)--included in the ETS? National Observational Teaching Examination (NOTE) assessment series. In this assessment, candidates interact with a small class of virtual students represented by avatars in a computer-based, simulated classroom. The five avatars are enacted by a single simulation specialist who has been trained and certified on the particular task presented, either in elementary English language arts or mathematics. The paper defines and describes the construct of LCD, then provides a review of the research and scholarly literature that supports the importance of this practice for effective teaching, and finally describes how the construct is measured in the NOTE assessment.

Key words: high-leverage teaching practices, teaching performance assessments, teacher licensure, measurement of teaching

ETS RM-16-09 i

M. Witherspoon et al.

LCD: Definition, Supporting Evidence, and Measurement of ETS? NOTE Assessment Series

Acknowledgments Some of the content in this report is used in the following companion reports: Eliciting Student Thinking: Definition, Research Support, and Measurement of the ETS? National Observational Teaching Examination (NOTE) Assessment Series (RM-16-06) by Yi Qi and Gary Sykes; and Modeling and Explaining Content: Definition, Research Support, and Measurement of the ETS? National Observational Teaching Examination (NOTE) Assessment Series (RM-16-07) by Leslie Stickler and Gary Sykes.

ETS RM-16-09 ii

M. Witherspoon et al.

LCD: Definition, Supporting Evidence, and Measurement of ETS? NOTE Assessment Series

Table of Contents Page

A Performance Assessment ............................................................................................................ 1 The National Observational Teaching Examination (NOTE) .................................................... 2 LCD and the License to Teach.................................................................................................... 2

Construct Definition, Rationale, and Explication ........................................................................... 6 Rationale for the Construct ......................................................................................................... 6 Explication of the Construct ....................................................................................................... 8

Measurement of Discussion in the Research Literature ............................................................... 15 Discussion and Reasoning Skills .............................................................................................. 15 Academic Achievement ............................................................................................................ 16 Reading Comprehension ........................................................................................................... 17 Disciplinary Knowledge and Practices ..................................................................................... 18 Engagement in Learning ........................................................................................................... 19 Discussion and Other Outcomes ............................................................................................... 19 Issues in Measuring Discussion ................................................................................................ 20

Measurement Approach for the LCD Performance Assessment .................................................. 22 Delivery Mode .......................................................................................................................... 22 Discussion Content ................................................................................................................... 24 Structural Features .................................................................................................................... 25 Scoring Criteria ......................................................................................................................... 26

Supporting Evidence for Measurement Approach........................................................................ 27 Aspect 1: Prompting Participation From All Students ............................................................. 27 Aspect 2: Steering Discussion Toward the Learning Goal(s)................................................... 28 Aspect 3: Representing the Content.......................................................................................... 29 Aspect 4: Concluding the Discussion ....................................................................................... 30

Conclusion .................................................................................................................................... 32 References..................................................................................................................................... 34 Appendix. Leading Classroom Discussion (LCD) Sample Task.................................................. 44 Notes ............................................................................................................................................. 47

ETS RM-16-09 iii

M. Witherspoon et al.

LCD: Definition, Supporting Evidence, and Measurement of ETS? NOTE Assessment Series

This paper describes and provides support for an assessment of a critical practice of teaching--leading a classroom discussion (LCD). This practice recently has risen to prominence among scholars of teaching (Resnick, Asterhan, & Clarke, 2015) but has deep roots dating back to the Socratic dialogues (Haroutunian-Gordon, 1991, 2010); forward to philosophical underpinnings provided by Buber (1970), Burbules (1993), Hawkins (1974), and others in the dialogic tradition; and in the sociocultural tradition of learning supplied by such scholars as Bakhtin (1981, 1986) and Vygotsky (1978).

Teachers promote learning in many ways that include lectures, recitation, work on projects, and others. But among instructional practices that teachers employ, discussion is an important, even critical, method. Prominent scholars today advocate discussion-based teaching because it creates opportunities for students to practice important skills such as argumentation, critical thinking, and collaboration (Gall & Gillett, 1980; Hadjioannou, 2007; Larson, 1996; Sun, Anderson, Lin, & Morris, 2015; Walshaw & Anthony, 2008). Further, discussion is a vital form of participation in the democratic way of life, as students are encouraged to find their voice, express and defend their views, function in communities of inquiry, and learn from and respond to the ideas and opinions of their classmates (Nystrand, 1997; Nystrand, Wu, Gamoran, Zeiser, & Long, 2003; Parker, 2006; Reisman, 2015; Resnick, Michaels, & O'Connor, 2010).

A Performance Assessment Before describing discussion-based teaching in greater detail, we provide a brief preview of the LCD assessment component of the ETS? National Observational Teaching Examination (NOTE) assessment series, developed by Educational Testing Service (ETS), for the reader to keep in mind. The LCD assessment engages the candidate in a simulated classroom environment that is delivered via computer. Prior to the assessment, the candidate is provided with curriculum materials and a lesson goal with time allotted for study and preparation. Then, the candidate conducts a short discussion of some 15 minutes duration with five "students" represented by avatars in the computer "classroom." The avatars are controlled by a trained and certified simulation specialist such that the candidate is engaged in an interaction with the simulated students around the lesson content that is provided. The simulation specialist can see and hear the candidate. The candidate's performance is videotaped and scored based on a rubric applied by trained and certified raters. The simulation specialist's actions and words are standardized to be similar across candidates, and the specialist's performance--that is, the actions and words of the

ETS RM-16-09 1

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download