1shep



[pic]

-----------------------

[pic]

Scott S. Critzer and Jeffrey S. Garrett

EDLP 713

Dr. Sokol and Dr. Clayton-Prince

STAR Power

[pic]

STAR Power 1

Place logo or

company name here

Reading is a skill that is fundamental to all other areas of education, both in and out of school; and the teaching of effective reading skills extends beyond the primary grade levels. One of the key aspects of helping students continue to grow their reading skills after they have mastered basic reading is to keep them reading on a consistent basis. Renaissance Learning Incorporated’s Accelerated Reader (AR) program is designed to do this by helping motivate students to engage in active and meaningful reading experiences on a daily basis throughout their educational careers. However, the question becomes just how effective the program is at helping motivate students to continue to read across the entire spectrum of their educational careers. With that in mind, this program evaluation was commissioned by the administration of Sokol County Public Schools (SCPS) to ascertain to what extent the AR program in Sokol County is succeeding at motivating students to read.

Program Context

Sokol County Public Schools serves a rural population in a socio-economically depressed locality in Southside Virginia. Like many divisions in rural settings where education is not a top priority for the majority of households, Sokol County has a vested interest in the promotion of programs that enhance the most fundamental of student skills—reading. The program chosen by SCPS to help address this issue is the AR program. While there are literally thousands of school divisions in the United States that use the AR program and countless pages of literature surrounding the program, there continues to be a raging debate as to its effectiveness. Review of literary artifacts related to the AR program and the pedagogical foundation on which it is based, suggest that as a tool designed to utilize progress monitoring best practice, AR appears to be an appropriate use of the progress monitoring design that is noted to achieve gains in student reading participation and comprehension, particularly at the lower grade levels. In contrast, however, there is literary evidence that suggests that AR is not effective in changing long-term attitudes and habits towards reading, that the rigor and quality of the AR assessments themselves may not be adequate to prepare students for secondary-level reading, and that the use of extrinsic motivation and tests may cause students to rebel against reading.

Recommendations

Ultimately, we found that the AR program, in and of itself, is relatively benign. Although there are some students who indicate that they do not participate simply because they do not like to read, most did not express particularly strong feelings about particular elements of the program itself. While the primary motivating factor indicated by the students was an increase in rewards and prizes for their efforts, most students indicated that the program did not motivate them. As was discussed earlier, this lack of motivation can be traced back to an inconsistency in program implementation on the part of the division.

To combat this problem, the team recommends that the division institute guidelines that require that all schools implement the program in keeping with the recommendations of the company. As part of the new division guidelines, the secondary levels should consider ways that they can successfully execute the program in keeping with the changing needs of their student population. This may include increase time allotment or schedule changes. In addition, the data indicated that grades may play a part in motivating students to participate in the program, so replacing some reading activities with graded AR assignments, essentially killing two birds with one stone, may help alleviate some time constraints while increasing AR participation.

“Read, Lead, Succeed”

“Type your callout text here.”

Evaluation Design

To answer the three key evaluation questions, the team recognized that the measure of the program’s effectiveness would be rooted in the values, needs, and perceptions of those using it. As a result, they approached the evaluation from the perspective of a Responsive model within the Values Branch of research. This led them to conduct their data collection using a quasi-experimental, sequential, mixed-methods design represented in the diagram below in which they gathered qualitative data from focus groups, surveys, interviews, and ongoing conversations with program stakeholders over a three month period and used that data as the foundation for creating a student survey that addressed important questions of the AR program’s motivational effectiveness as it pertained to the student reading.

[pic]

STAR Power

An Executive Summary of Sokol County’s Use of the

STAR and AR Program at the Secondary Level

S |M |T |W |T |F |S | |1 |2 |3 |4 |5 |6 |7 | |8 |9 |10 |11 |12 |13 |14 | |15 |16 |17 |18 |19 |20 |21 | |22 |23 |24 |25 |26 |27 |28 | |29 |30 | | | | | | | | | | | | | | |

2008

Place logo

or logotype here,

otherwise

delete this.

Delete text and place photo here.

Delete text and place photo here.

Data Collection

The evaluation began with focus groups consisting of secondary English teachers, librarians, and students to get preliminary insight into perceptions of the program. The team then met with the Superintendent and, using their focus groups findings as a basis for the conversation, narrowed the focus of the evaluation to the effectiveness of the AR program to motivate students to want to read. The team then created a survey for all librarians to ascertain how the program is currently being implemented in their schools and their perceptions of motivational issues with the program methods.

A series of e-mail follow-up questions between the team and the librarians provided the team deeper insight into the values and perceptions that needed to be explored. The findings from these two groups were then discussed with the Director of Instruction, who added insight into what the prescribed implementation of the program is and concerns that she recognized.

The team then utilized all of the qualitative data collected during these steps to create a mixed-methods student survey that addressed the stakeholders’ questions and concerns about the program’s motivational effectiveness. The survey incorporated a mixed-methods design in which some questions were simply scored on a Likert Scale while others allowed students the opportunity to include their opinions and perceptions. The survey was given to a population of students at the middle and high schools using a stratified cluster sampling method in which a class was chosen at each grade level based on its diverse demographic make-up in order to give a sampling that was as representative of all levels of the student population as possible.

company

name

STAR Power 10

STAR Power 9

STAR Power 8

[pic]

STAR Power 4

STAR Power 6

STAR Power 5

STAR Power 7

[pic]

[pic]

STAR Power

Conclusions

Ultimately, the most important skill that a school division can teach its students is the ability to read. It is the gateway to all other learning, and, like most other skills, can only be perfected through practice and repetition. Schools around the country and around the world are looking for answers to the question of how to get students to continue to read and to continue to want to read, and the AR program appears to be a viable tool for achieving this end when used appropriately.

What “appropriately” means may be different for each division and may change at each grade level, so it is incumbent upon localities such as Sokol County to look deeply into how the program is being used and what steps can be taken at each level to make it the most effective for all students involved.

Further Evaluation

The process of conducting this evaluation opened other areas of consideration that SCPS may want to explore through further evaluation. The first consideration is to perform a more in-depth Process Evaluation into the AR program as a whole. Since the evaluation findings indicate that the identified issues with the AR program may have less to do with the make-up of the program than with its implementation, the division will want to explore more deeply how the program is being used within each individual school, what factors are affecting the proper use of the program, and whether these factors are interfering with the success of the program in achieving its goal of motivating students to read. A primary consideration in this evaluation should be the feasibility of proper use of the AR program according to the guidelines of Renaissance Learning, Inc., specifically whether it is feasible for schools within the division to allot the appropriate amount of in-class reading time for the program to be effective as designed. In addition, student data indicated that incentives were a major factor in their motivation to read, is it feasible for the division financially to afford the types of incentives identified to meet those needs?

In addition, even though this evaluation focused on the broad question of overall student motivation, the data collection centered primarily on the secondary level. In order to gain a deeper insight into the program as a whole, the division may choose to perform further evaluation of how AR is utilized and the perceptions of its motivational capabilities at the elementary level. Since all three elementary schools implement the program differently, an evaluation across all three schools would give an interesting comparison of motivational ability related to implementation. This may help the division more clearly understand the needs and perceptions of those who use the program at the lower grades and help them make adjustments in the program implementation in order to make it more responsive to those factors.

STAR Power

[pic]

STAR Power

[pic]

[pic]

Data Findings

In examining the data findings, the team looked at the data in two phases. The first was the qualitative data collection that included the secondary-level student and teacher/librarian focus groups, the surveys and follow-up interviews of all SCPS librarians, and the interview of the division’s Director of Instruction. Several themes and topics emerged from this data collection. Overall, the team found that while there is a program provided by Renaissance Learning that allows teachers to set goals based on students’ reading levels and the amount of time they are given in class to read, schools were not using this uniformly. Not all teachers or librarians were using the program to set student goals, making them arbitrary and inappropriate at times. In addition, there was a difference in who set the goals, with most schools having individual teachers create them while others had the librarians establish them.

When asked about their perceptions of how motivational the program was, most respondents indicated that it was at least minimally effective. The majority noted that the program seemed to be most effective for students who already liked to read, were competitive, or who had teachers who emphasized the program, while they noted a lack of motivational ability for those who did not already like to read, those whose teachers did not emphasize the program, and those in the upper grades. The latter group was noted as being less motivated not by the program but by other time constraints such as more homework and increase activities outside of school.

All of these themes were incorporated into a student survey for students at the middle and high schools. This survey served to further enlighten the evaluators on the students’ perception of the program. In general, the student responses supported the suppositions of the previous groups. Overall, the majority of students indicated that AR was not a motivating factor in their reading and that this became even more so as they progressed in grade. In addition, of the students who said it was motivational to them, the majority of those said that they already liked to read. Both of these supported the notion that AR’s motivational capability decreases as students get older and that it is most motivational for those who already like to read.

STAR Power 3

STAR Power 2

Day Year

[pic]

Looking more deeply into the causes of this loss of motivation, the student data supported two major themes. The first was the theme of time. Most students who responded to the questions of why their AR participation had diminished said that time and outside elements such as homework and extra-curricular activities that encroached on that time were an issue. This was even more prevalent at the high school level, where students have more time conflicts and AR is used as a grade in several classes. The second theme was interest. Many students indicated that a lack of interest or an inability to find interesting reading material caused a lack of motivation for them.

Finally, when asked what aspects of AR students did find motivational, there were, again, two main themes. Several students indicated that interesting reading material and better reading material were motivational to them. However, the overriding theme was prizes and rewards. The majority of students who responded about what motivated them in the program or how they would improve it suggested that prizes were a motivating factor and that larger and more frequent prizes would help to increase that. Interestingly, this theme was consistent through both the middle and the high school responses.

[pic]

[pic]

Evaluation Focus

It is this latter point of student motivation that is the focus of this evaluation. Using the SCPS Superintendent’s input as the foundation, the research team decided to focus the assessment on determining if the program was effective in motivating students to read and what aspects of the program did or did not contribute to this motivation. To answer this question, the evaluation was centered around three key questions:

1) Is the program effective in motivating students to read?

2) What program factors contribute to the reading motivation, or lack thereof, of the students?

3) How is the program being implemented within SCPS?

The AR program evaluation will provide the school division with a clear understanding of how the program is currently being implemented and utilized at the secondary level, the program’s perceived effectiveness in motivating students to read, and what factors contribute or detract from that effectiveness. The evaluators will present these findings to the group.

Data Discussion

Upon review of the data findings, we found that motivation or lack thereof appears to have little to do with the AR program itself. When comparing the student data with the qualitative data collected from the program administration, the primary issue with the motivational capabilities of the AR program at the secondary level has to deal with how it is implemented in the secondary grade levels in SCPS. Overall, there is a lack of fidelity of implementation to the program. It is implemented differently from school to school and teacher to teacher within the division. This causes a lack of continuity, but, more importantly, it is often not in keeping with the recommendations of the program designers.

The primary issue is the goal setting in accord with the amount of time allocated to read in class. The findings indicate that students are not given enough consistent in-class time to read to meet their goals, either because the time is not allotted or the goal is set too high for the time provided. This relegates achievement of AR goals to outside of class work, homework, and increases the risk that only those students who like to read or who have support at home will do it. This negates a large amount of the intended benefit of the program, as it then does not reach all levels of students effectively. This problem becomes even more profound as students get older.

The data indicates a trend in student responses that time and the increased amount of outside of class conflicts such as activities and increase homework load play a major role in a lack of interest and stress surrounding AR. Given proper implementation, this should not be an issue because the program should be completed during school hours. However, as the students get busier, the program becomes even less responsive to their fuller schedule. Thus, the students become even less likely to read.

Results coincided with identified themes. Student specific perceptions of themes identified.

Quantitative

Student Survey

(Mix Methods)

Results directed survey question development

Qualitative

Literature Review,

Staff and Student Focus Groups, Client Interview, Librarian Interview, Director of Personnel Interview

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download