Computers & Education

Computers & Education 86 (2015) 137e156

Contents lists available at ScienceDirect

Computers & Education

journal homepage: locate/compedu

Use of physics simulations in whole class and small group settings:

Comparative case studies

A. Lynn Stephens*, John J. Clement

TECS Department, College of Education, University of Massachusetts, Furcolo Hall, Amherst, MA 01003, USA

a r t i c l e i n f o

a b s t r a c t

Article history:

Received 19 July 2014

Received in revised form

25 February 2015

Accepted 28 February 2015

Available online 9 March 2015

This study investigates student interactions with simulations, and teacher support of those interactions,

within naturalistic high school classroom settings. Two lesson sequences were conducted, one in 11 and

one in 8 physics class sections, where roughly half the sections used the simulations in a small group

format and matched sections used them in a whole class format. Unexpected pre/post results, previously

reported, had raised questions about why whole class students, who had engaged in discussion about the

simulations while observing them projected in front of the class, had performed just as well as small

group students with hands-on keyboards. The present study addresses these earlier results with case

studies (four matched sets of classes) of student and teacher activity during class discussions in one of

the lesson sequences. Comparative analyses using classroom videotapes and student written work reveal

little evidence for an advantage for the small group students for any of the conceptual and perceptual

factors examined; in fact, if anything, there was a slight trend in favor of students in the whole class

condition. We infer that the two formats have counter-balancing strengths and weaknesses. We

recommend a mixture of the two and suggest several implications for design of instructional simulations.

? 2015 Elsevier Ltd. All rights reserved.

Keywords:

Cooperative/collaborative learning

Improving classroom teaching

Pedagogical issues

Simulations

Teaching/learning strategies

1. Introduction

The purpose of this study is to investigate student interactions with simulations, and teacher support of those interactions, within

naturalistic1 high school physics classroom settings. We ask what differences there might be between whole class and small group

discussions during use of simulations. Constructivist educators have stressed the importance of learning by doing, which, in our

experience, has been interpreted by many teachers to mean that students must have their own hands on the keyboards. However, we

have noticed that students may misinterpret, or simply miss, important information in a simulation. Because simulations are intended

to convey dynamic visual information, teachers may be tempted to believe that simulations are automatically effective in communicating complex models to students. However, research such as Lowe (2003) has shown that comprehension of animations is dependent

on appropriate prior knowledge structures. We have observed interesting student reasoning and interesting teacher support moves

designed to promote reasoning and comprehension during use of simulations in both whole class and small group contexts; this has led

us to look more deeply at what is occurring during these discussions. Our motivating question is, does one format have strengths that

the other does not? Though this is a complex question, we can begin to address it by comparing several factors at work in the class

discussions.

* Corresponding author. Present address: Center for Knowledge Communication, School of Computer Science, 140 Governors Drive, University of Massachusetts, Amherst,

MA 01003-9264, USA. Tel.: ?1 413 545 2744, fax: ?1 413 545 1249.

E-mail address: lstephen@umass.edu (A.L. Stephens).

1

We mean ¡®naturalistic¡¯ as opposed to ¡®in the laboratory.¡¯ We tried to interfere with the classroom environment as little as possible because our interest is in what can

happen in a real-world classroom.



0360-1315/? 2015 Elsevier Ltd. All rights reserved.

138

A.L. Stephens, J.J. Clement / Computers & Education 86 (2015) 137e156

We observed classes using simulations in one of two formats, a small group hands-on format or a whole class discussion format during

which a single computer was used to project the simulation in front of the class. We will review the pre/post results from Stephens (2012)

and then focus on two broad issues: the extent to which students engaged with conceptual issues and the extent to which visual features

were recognized, used and supported. Comparative case study analyses of four matched sets of classes identify differences and similarities

between the class sections in each matched set, as revealed in classroom videotapes and student written work.2

2. Theoretical background

Studies have investigated the effects of instructional guidance for simulations when guidance was provided within the learning materials

or by the teachers (review by Cook, 2006; Reid, Zhang, & Chen, 2003) and have recommended such actions as providing interpretive support

and minimizing cognitive load. The de Jong and van Joolingen (1998) review of simulation use in discovery learning contexts cited the

importance of structuring and supporting students' work in ways to prevent dif?culties. However, there do not appear to be many studies

that address the question of how best to provide instructional guidance for simulations and animations in the context of whole class

discussion.

Researchers have studied the use of simulations and other digital tools by small groups and by individual students (Adams et al., 2008a;

Buckley, 2000; Linn, 2003; Williams, Linn, Ammon, & Gearhart, 2004; Windschitl & Andre, 1998; and Zietsman & Hewson, 1986). Among the

potential advantages described for these tools are that they can increase engagement, that teachers can use them to ¡°help students make

their thinking visible,¡± and that much of this software provides students the opportunity to customize their own modeling tools. Another

potential bene?t is that animated graphics can show changes over time (review by Cook, 2006) although these can also produce cognitive

overload and actually hinder novice learning (Lowe, 2003; Tversky, Morrisson, & Betrancourt, 2002). Therefore, novices may need to be cued

to details of motion in animated graphics (Rieber, 1990).

Hands-on activity afforded by small group work would appear to offer students a more active learning experience with simulations than

would a whole class format. In the context of think-aloud interviews, for example, Adams et al. (2008a), indicate that simulations can be

highly effective, but only if the student's interaction is directed by the student's own questioning. This kind of self-directed interaction with a

simulation would seem to require individual or small group work with hands-on-keyboard opportunities. Considering this, and the fact that

the teachers in our study have stated they prefer to allow students to work with simulations in small groups and feel experienced teaching in

that format, it might be expected that the small group format would work better for them than would a whole class format. On the other

hand, studies have reported a variety of issues concerning the effective use of small group discussions in science classes (some of which used

simulations), including that students can exhibit a low level of engagement with tasks (Bennett, Hogarth, Lubben, Campbell, & Robinson,

2010), in contrast to ?ndings cited above.

Good practices for teacher response in whole class discussion have been and continue to be informed by the work of Chin (2006), Hammer

(1995), Hogan and Pressley (1997), and McNeill and Pimentel (2009) among others. In general, these recommend that the teacher play a role

in 1) drawing out student reasoning and 2) scaffolding certain kinds of reasoning where students have dif?culty. However, these studies did

not focus on use of interactive simulations (but see Raghavan, Sartoris, & Glaser, 1998, for a counterexample). What kinds of teacher responses are optimal during class discussion may be affected by use of an interactive simulation that has been designed to provide feedback

and to serve as an expert voice. Some believe we know very little about how to use animation effectively in instruction (Jones, Jordan, &

Stillings, 2001). Principles suggested by theory and by laboratory work with students using simulations (Lowe, 2003; Mayer & Moreno,

2002) would appear to need further validation in science classroom contexts (Cook, 2006), and may well have to be modi?ed to be usable by teachers employing available simulations with available, frequently limited, classroom hardware.

The present study is of classrooms engaged in model-based learning in science. Studies of expert scientists and of science students

conclude that the ability to generate and evaluate mental models appears to be a crucial aspect of scientists' thinking (Clement, 2008;

Darden, 1991; Nersessian, 1995) and of student thinking (Clement & Ramirez, 2008; Gentner & Gentner, 1983; Nunez-Oviedo & Clement,

2008). The pedagogical approach of the teachers in this study can best be characterized as guided inquiry (Bell, Smetana, & Binns, 2005;

Hammer, 1995), in which students are supported by the teacher in lessons that are neither pure inquiry nor pure lecture but somewhere

in between. Studies of model-based instruction (Hestenes, 1987; Krajcik, McNeill, & Reiser, 2008; and Schwarz et al., 2009) emphasize that

complex scienti?c models can be constructed using prior knowledge ideas and reasoning resources of students, but that this usually requires

scaffolding from external supports of various kinds. Minstrell and Kraus (2005), Williams and Clement (2015), and Windschitl, Thompson,

Braaten, and Stroupe (2012) have recently identi?ed many interesting strategies for teachers to use in dealing with conceptual dif?culties

encountered during the learning of complex models in science. Findings from social constructivism (Hogan & Pressley, 1997) have led to a

belief that classroom discussion that includes teacher-student or studentestudent exchanges can be an important and helpful component of

model-based learning; we note that these can occur in either small group or whole class contexts.

Little work has been done comparing small group vs. whole class formats. Wu and Huang (2007) compared a single teacher-centered and a

single student-centered class using physics simulations where the classroom formats were similar to the whole class and small group

formats we describe. They found no overall difference in pre/post conceptual gains between the two groups, although they found qualitative

differences in cognitive and behavioral engagement. The present study takes a somewhat different tack; rather than focusing on engagement, we focus on students' ability to make use of the visuals in order to address certain dif?cult concepts and the extent to which discussions dealt with this. Smetana & Bell (2009) compared the use of chemistry simulations in a single small group class and a single whole

class discussion and found no signi?cant difference in pre/post gains; they suggest that future research involving more varied populations

and additional teachers and classrooms is needed. The present study, along with a related study of a second lesson sequence (Stephens,

2012), aims to contribute to that goal and to investigate more deeply some of the underlying factors at work in the two modes.

2

Levels of physics courses included in this study, from least to greatest dif?culty: CP ? College Preparatory, HP ? Honors, AP ? Advanced Placement.

A.L. Stephens, J.J. Clement / Computers & Education 86 (2015) 137e156

139

3. Research questions

For this investigation into potential differences between whole class and small group discussions during use of simulations, we focus on

the following Research Questions:

 To what extent were crucial physics concepts dealt with by teachers and students in the whole class and small group conditions?

 To what extent were key visual features of the simulation recognized and used in the two conditions?

4. Material and methods

4.1. Rationale for mixed methods approach

Our primary purpose for planning videotape and transcript analysis was to increase our understanding of teaching and learning processes. More particularly, with respect to the eight class sections discussed here, we wished to investigate possible differences between

whole class and small group teaching and learning modes that were invoked to support conceptual learning during science classroom

discussions. Our primary method was to conduct intensive video analysis in qualitative case studies. We wanted to supplement this with the

results of the pre/post tests for these sequences. In general, pre/post test analyses could be completed quickly for all observed classes, while

the more intensive videotape analyses on selected lesson sequences took much longer to complete. When designing the larger study, our

plan to implement video analysis precluded large samples. This issue points up a tension we have always felt when designing such studies,

the tradeoff between the need for a sample size small enough to be manageable for qualitative analysis and the need for very large numbers

of classes for rigorous quantitative analysis. Our solution in designing this series of studies (along with a companion set, Stephens, 2012) was

to focus on the qualitative analysis with a manageable sample size and to use the results of quantitative analysis for unusually narrow

purposes. The primary purpose was not to attempt to project the quantitative results to a larger population outside the study. Rather, for

each of the studies discussed here, the results of the pre/post analysis yielded quantitative information about the sample inside the study,

and served to motivate, inform, and constrain the design of the main qualitative study of the sample. Thus we decided on a rather narrow

use of quantitative measures as part of a mixed methods design.

4.2. Participants and setting

The analyses to be described here involve class sections from two physics teachers at a high school in a suburban college town. The

teachers were purposefully selected; they had to be willing to teach model-based lessons and to foster discussions in both whole class and

small group settings, and they had to be willing and able to use computer simulations as part of their lesson plans. Class sections taught by a

given teacher were purposefully selected for analysis according to whether they ?t the following criteria, in which case they were

considered to form matched sets of classes. The teacher must have been teaching at least two comparable sections in a given semester and

conducted the lesson sequence in at least one section in a whole class format and in at least one other section in a small group format.

Teachers' evaluations and records were relied upon to determine that the sections within a set had students comparable in terms of age and

demonstrated levels of aptitude for the content of the course as evidenced by their prior work in the course. In addition, the class sections in

each set must have been provided similar levels of preparedness for the lesson, as indicated by the teachers' records of their lesson plans.

Finally, the lesson sequence as taught in the two formats must have been similar (see Materials and Procedure below) and the class sections

must have been allowed similar amounts of time on the lessons and the pre- and post-tests. Fifteen lesson sequences were observed; seven

sequences (and one teacher) were dropped from analysis because they did not meet the above criteria, leaving eight sequences from two

teachers to be subjected to analysis.

Once it was determined that class sections were matched, they were assigned to the whole class (WC) or small group (SG) condition for

the lesson sequence according to practical logistical considerations, such as how much time there would be before and after the class to

rearrange equipment. Class sections within each matched set met in the same rooms. One or both authors observed all lesson sequences.

The authors conducted follow-up interviews with the teachers.

These eight class sections comprised four matched sets as indicated in Table 1, N ? 150. Teacher A taught this as a two-day sequence

while Teacher B taught it as a one-day lesson. Therefore, 10 videotapes were collected for this lesson sequence from the eight class sections.

The intention is not to draw comparisons between different teachers but to compare each teacher's small group lesson to the same teacher's

whole class lesson of the same matched set.

4.3. Materials and procedure

Although materials varied slightly for each level of physics, within each matched set (as described above), the teacher used identical

materials in the two conditions but varied the way in which the simulation was used. In the whole class condition, each teacher used a single

Table 1

Data sources.a

Year

Year

Year

Year

1

1

1

2

Honors Physics

College Preparatory Physics

Advanced Placement Physics

Advanced Placement Physics

Teacher

Teacher

Teacher

Teacher

A

B

B

B

1

1

1

1

SG

SG

SG

SG

1

1

1

1

WC

WC

WC

WC

a

1 SG and 1 WC indicate one class section taught in small group format and one in whole class format. From least to greatest dif?culty were College Preparatory, Honors,

and Advanced Placement classes.

140

A.L. Stephens, J.J. Clement / Computers & Education 86 (2015) 137e156

Fig. 1. The teachers chose PhET Energy Skate Park for the lesson, shown with the dotted GPE Reference Line and the Energy Bar Graph turned on. ().

computer to project the simulation onto a screen in front of the class and facilitated a whole class discussion as students worked through the

activity sheets. In the small group condition, multiple computer stations were used with 2e4 students to a computer; they were allowed to

engage in hands-on exploration and small group discussion guided by the same activity sheets as in the whole class condition while the

teacher circulated among the groups. In both conditions, the teacher began by introducing the computer activity to the whole class, though

the extent of this introduction varied. In both conditions, the teacher was available for questions the entire time the simulation was in use.

Other than the constraints provided by the technological set-up, the activity sheets, the simulation condition (whole class or small group)

and the data-collection needs of the study, teachers were free to conduct their classes as they saw ?t and were encouraged to use the best

teaching strategies they could devise for each situation. Control for time on task was implemented by using the same activity sheets and the

same number of class periods to cover the material within each matched set. The lesson plans and activity sheets were developed by the

teachers and reviewed by the authors. (Though early versions of the materials were inspired by sample lesson plans from the simulation

website, , the ?nal lesson plans and activity sheets were largely the construction of the teachers who participated

in this study.) The pre/post tests were developed jointly by the teachers and research team and consisted of transfer questions that were not

directly addressed during instruction; this was to minimize the possibility of the teachers' teaching to the test and also because the desire for

these studies was to measure conceptual rather than rote learning. These tests were administered immediately before and after the

instructional portion of the lesson sequence.

4.4. Gravitational potential energy lesson sequence

The teachers selected a simulation ahead of time from freely available online sources, Energy Skate Park at

(Perkins et al., 2006). See Fig. 1. This is a sophisticated simulation developed through a series of formative evaluation trials. The track

can be added to or reshaped, the skater placed anywhere in the scene and released, and the simulation run to see how the skater would

respond under the in?uence of gravity (with or without friction). A sample page from an activity sheet and a sample pre/post test are

provided in Appendices A and B.

After the whole class introduction, the lesson plan included ?ve minutes of free exploration of the simulation (either in whole class or in

the small groups) before students began work on their activity sheets. The activity sheets then supported students as they engaged in an

exploration of the skater's motion, the changes in his/her potential, kinetic, thermal, and total energy with time, and the relationships

between those changes (2nd order relationships). Depending on the level of the physics class, some of the activity sheets asked students to

write their predictions for what would happen if the simulation were run for certain speci?ed scenarios. All of the activity sheets asked

whether the total energy of the skater could ever equal zero and requested a written explanation for the answer. All activity sheets explicitly

included the instruction to turn on the Gravitational Potential Energy (GPE) Reference Line (see Fig. 1) and to move it around. Also included

was an instruction to turn on the animated Energy Bar Graph, which showed clearly when the potential energy of the skater took on

negative values. Late in the lesson were an exploration of a pre-set track con?guration that included a full loop and an exploration of an

¡°outer space¡± setting of the simulation that set gravity to zero. The classes ended with the post test.

5. Exploratory quantitative results

An exploratory study (Stephens, 2012) of learning gains via identical pre/post tests served to raise questions that we will attempt to

address in the qualitative analyses that constitute the bulk of the paper. Scores were tabulated from multiple-choice and short answer

questions that addressed conceptual issues. ANOVAs3 were used to compare the pre/post gains of the whole class and small group students

within each matched set.

3

ANCOVAs completed recently yielded similar results.

A.L. Stephens, J.J. Clement / Computers & Education 86 (2015) 137e156

141

Table 2

Gravitational PE multiple choice/short answer transfer question pre/post gains (Stephens, 2012).

WC Gains

CP

HP

AP

AP

SG Gains

N

Mean

SD

N

Mean

SD

11

20

23

21

0.26

0.22

0.10

0.09

0.20

0.21

0.12

0.16

14

19

21

21

0.25

0.09

0.02

0.07

0.24

0.15

0.11

0.10

t-Value

Sig.

Cohen's d

0.097

2.221

2.368

0.506

0.924

0.033a

0.023b

0.616

0.04

0.71

0.71

0.16

Boldface indicates the larger mean gain within each matched set. In order of increasing dif?culty were CP, HP, then AP course levels.

a

Signi?cant difference in gains in favor of the whole class condition. This row is shaded to indicate the possibility that unanticipated events may have had a disproportionate

effect on the small group condition; those students were encouraged, but not required, to ?nish their activity sheets in a single period while students in the whole class

condition were not given this explicit suggestion. Both classes actually spent two days on the activity sheets.

b

Signi?cant difference in gains in favor of the whole class condition.

Seven of eight class sections that met the criteria for matched-set analysis for the Gravitational Potential Energy sequence had statistically signi?cant pre/post gains on multiple choice and short answer transfer questions, p < 0.025, effect sizes ranging from d ? 0.45 (small)

to d ? 1.40 (large).

The results of ANOVAs comparing the pre/post gains of the two conditions are summarized in Table 2. In order of increasing dif?culty

level, there were College Preparatory Physics (CP), Honors Physics (HP), and Advanced Placement Physics (AP) classes. Gains are expressed as

percentages of a perfect score.

As can be seen in Table 2, the effect sizes for the differences between whole class and small group performance on transfer questions

were negligible in two of the comparisons, while in two others, the differences in gains reached signi?cance in favor of the WC condition

with a medium effect size. In one of these comparisons, there was a possible confounding factor with a slight variation in instructions that

the teacher gave the two classes, but in the second comparison there was no obvious reason for the lower scores in the SG condition other

than difference in condition. Because of this and the presence of signi?cant effects in only some of the classes, we will be conservative in our

description of the overall result: no advantage detected for students in the SG condition. But this result was still surprising given that both

teachers had said at the time that the small group students appeared to be learning more.

These results from use of a sophisticated simulation agree with exploratory results from a related study on a Projectile Motion lesson

sequence in which students used a very simple simulation and animations (Stephens, 2012). In that study, also, small group students showed

no pre/post advantage over students who had experienced the materials solely in the context of whole class discussion. (See Fig. 2). The

results are also consistent with two smaller studies of which we have since become aware, each of which showed no signi?cant difference

within a single pair of classes with lesson formats similar to the present study (Smetana & Bell, 2009; Wu & Huang, 2007).

5.1. Discussion of exploratory results

These results raised questions for us and for the teachers. At a follow up meeting with the teachers after the second year when they were

shown the pre/post results, they expressed astonishment that the small group students had not outperformed the whole class students.

Given the nature and sizes of the samples, we did not attempt to conduct a comparison across all eight Gravitational PE classes but only to

compare each small group Gravitational PE class with its matched whole class discussion class; in other words, to conduct four comparative

analyses. The pre/post results have motivated us to conduct in-depth qualitative studies of these classes using videotape and activity sheet

analyses. There are many lenses that could be applied to such analyses. Our general aim in undertaking the studies is to investigate student

interactions with simulations and teacher support of those interactions; this constitutes our main focus. In particular, a review of observation notes and discussions with teachers suggested that there might have been interesting differences in student and teacher responses

to, and interactions with, certain visual features of the simulation.

6. Qualitative analysis and results

6.1. Introduction to the qualitative study

The overall motivating question for these studies is whether there might be different strengths and weaknesses in the whole class and

small group formats regarding student and teacher interactions with simulations. In view of the apparent lack of advantage with respect to

pre/post gains for students who had used the gravitational PE simulation hands-on as compared with students in the matched whole class

discussions, we also wish to shed light on this question: Why did the whole class format produce gains as strong as those of the small group

format in these classes? Our hope was that videotape and activity sheet analysis would enable us to generate viable hypotheses to address

these related questions. Because we wanted to approach this analysis as free as possible from preconceived ideas about what was happening

in the classroom discussions, we employed elements of grounded theory to develop fresh analytical methods, as described below.

We used two types of data sources to investigate each matched set. The results of videotape analysis of the classroom discussions allow

us to develop a picture of what an individual hypothetical student could have been exposed to in each class. In this analysis the video camera

can be viewed as a proxy for an individual student; that is, the camera took the viewpoint of a hypothetical student in that classroom and

recorded what an individual student might have seen and heard. In whole class discussion, the camera took a ?xed position in the classroom

and so captured some of the limitations likely to be experienced by student participants in that mode; at any position, some comments from

fellow students were likely to be inaudible and accompanying hand gestures could be dif?cult or impossible to see. In classes in which the

small group format was used, at the point at which the students moved into small groups, the camera moved to one of the groups also.

Although fewer students were visible on camera than in the whole class condition, the videotape again recorded what an individual student

in that group could have seen and heard, including occasional interactions with students from other small groups and with the teacher. In

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download