Table of Contents

Table of Contents

Executive Summary ...................................................................................................................................... 2 Analysis of Year 4 (2006-07) Achievement Outcomes for TN Elementary Charter Schools....................... 4

School Descriptions ................................................................................................................................... 4 Circles of Success Learning Academy (COSLA). ................................................................................. 4 Smithson-Craighead Academy (SCA). .................................................................................................. 4 Star Academy. ....................................................................................................................................... 4 Promise Academy.................................................................................................................................. 4 Southern Avenue Charter School for Academic Excellence. ................................................................ 4

Method .......................................................................................................................................................... 5 Design and Variables ................................................................................................................................ 5 Sample restriction ...................................................................................................................................... 6 Control Group Selection ............................................................................................................................ 6 First Grade. ............................................................................................................................................ 6 Second Grade. ....................................................................................................................................... 6 Third Grade. ........................................................................................................................................... 7 Fourth Grade.......................................................................................................................................... 7 Fifth Grade. ............................................................................................................................................ 7 Nashville Metro .......................................................................................................................................... 7 Third Grade. ........................................................................................................................................... 7 Fourth Grade.......................................................................................................................................... 7 Achievement Measures ............................................................................................................................. 7 The Tennessee Comprehensive Assessment Program. ....................................................................... 8

Results .......................................................................................................................................................... 8 Second Year Schools ................................................................................................................................ 8 Promise Academy.................................................................................................................................. 8 Southern Avenue Charter School for Academic Excellence. ................................................................ 9 Third Year School .................................................................................................................................... 10 Star Academy. ..................................................................................................................................... 10 Fourth Year Schools ................................................................................................................................ 11 Circles of Success Learning Academy (COSLA). ............................................................................... 11 Smithson-Craighead Academy. ........................................................................................................... 13

Summary and Conclusions ......................................................................................................................... 14 References .................................................................................................................................................. 15

List of Tables

Table 1. Elementary Math Regression Outcomes ..................................................................................... 16 Table 2. Elementary Math Regression Means, Standard Deviations, & Effect Sizes ............................... 18 Table 3. Elementary Reading/Language Arts Regression Outcomes ....................................................... 19 Table 4. Elementary Reading/Language Arts Regression Means, Standard Deviations, & Effect Sizes . 21

1

Executive Summary

This report presents student-level achievement results for Tennessee charter schools that serve elementary grades. The present achievement report supplements a report (Ross, McDonald, & Bol, 2008) on the implementation progress made by the charter schools, encompassing school climate, classroom teaching methods, and perceptions by teachers, principals, parents, and students.

The schools had different start-up years as follows: (a) Circles of Success in Learning Academy and Smithson-Craighead Academy (2003-04); (b) Star Academy (2004-05); and (c) Promise Academy and Southern Avenue (2005-06). These three "cohorts" were therefore completing their fourth, third, and second academic years respectively at the time the achievement tests were administered in the spring of the 2006-07 school year.

Overview of Schools

The following table provides an overview of the elementary charter schools.

School Circles of Success Learning Academy (COSLA)

Smithson-Craighead Academy (SCA)

Star Academy

Promise Academy Southern Avenue Charter School

for Academic Excellence

Cohort 1 1 2 3

3

Level Elementary Elementary Elementary Elementary

Elementary

Grades (06-07)

K-5 K-4 K-4 K-1

K-2

Enrollment (06-07)

120

Location Memphis

174

Nashville

200

Memphis

115

Memphis

103

Memphis

Methodology

To examine student achievement outcomes, there was no student-level matching of control students. Instead, all students from non-charter, Title I schools in Memphis City Schools (MCS) and Nashville Metro in 2nd through 4th grades who took both the 2005-06 and 2006-07 TCAP Achievement Test and all 1st grade students who took the 2006-07 TCAP Achievement Test were included in the analysis as controls. The Nashville Metro students were utilized for the Smithson-Craighead comparison, whereas the MCS students were used as the comparison group for the Memphis-area charter schools. Student-level matching was not conducted because most students who attended these charter schools would not have attended a previous school, and would therefore have no pre-charter school test data to use for matching purposes. The primary variable of interest in the analysis is charter school membership, which if statistically significant, would constitute a reliable predictor of students in general performing higher or lower on TCAP: (a) based on their enrolling or not enrolling in a charter school and (b) after controlling for their prior achievement (where available), gender, and lunch status.

We encourage readers to interpret these results cautiously given that (a) due to student choice and other constraints, we were unable to conduct a randomized experimental study that eliminated family interest or involvement as an influential factor; and (b) we did not have a true "pre-test" measure for the elementary students, which greatly limits our ability to make causal statements regarding differences between charter and comparison students.

Achievement Measures

The Tennessee Comprehensive Assessment Program: Achievement Test (TCAP/AT) Normal Curve Equivalent (NCE) scores in Reading/Language Arts (Reading/LA) and Math were analyzed for students in grades 1 and 2. The number correct scores in Reading/Language Arts and Math were analyzed for students who were in grades 3 through 5 during the 2006-07 school year.

2

Summary and Conclusions To summarize the achievement outcomes obtained in this study, a brief achievement profile of each of the four schools is provided below: Second-Year Schools

Promise Academy. Control students had slightly better performance in Reading/LA while Promise students had slightly better performance in Math. It is important to note, however, that charter school membership was not a significant influence on achievement in 2006-07.

Southern Avenue. 1st grade students at Southern Avenue had better performance in both Reading/LA and Math, while controls had better performance in both subjects in 2nd grade. Charter school membership was only a significant influence on 2006-07 achievement in 2nd grade Math. Third-Year School

Star Academy. Suggestive positive effects were found in both Reading/LA and Mathematics at all grade levels. Charter school membership did not significantly influence achievement in 2006-07. Fourth-Year School

Circles of Success in Learning Academy. Suggestive positive effects were evidenced in both Reading/LA and Mathematics in 3rd and 4th grades. Controls had better performance in both subjects in 5th grade. Charter school membership was a significant influence on 2006-07 achievement in 4th grade Reading/LA and for both subjects in 5th grade.

Smithson Craighead Academy (SCA). In third grade, Smithson students had lower average scores in both Reading/Language Arts and Math. Charter school membership was significant in Math. The average number correct for 4th grade Smithson-Craighead students was lower than controls in both Math and Reading/Language Arts. Charter school membership, however, was not significant.

3

Analysis of Year 4 (2006-07) Achievement Outcomes for TN Elementary Charter Schools

This report presents student-level achievement results for charter schools that served elementary grades. The present achievement report supplements a report (Ross, McDonald, & Bol, 2008) on the implementation progress made by the charter schools, encompassing school climate, classroom teaching methods, and perceptions by teachers, principals, parents, and students.

The schools had different start-up years as follows: (a) Circles of Success in Learning Academy and Smithson-Craighead Academy (2003-04); (b) Star Academy (2004-05); and (c) Promise Academy and Southern Avenue (2005-06). These three "cohorts" were therefore completing their fourth, third, and second academic years respectively at the time the achievement tests were administered in the spring of the 2006-07 school year.

School Descriptions

Circles of Success Learning Academy (COSLA). COSLA is an urban school located in Memphis that emphasizes literacy and fine arts. COSLA enrolled a total of 120 students in kindergarten through fifth grade. All were African American, and most (80%) were eligible for free or reduced price lunch. COSLA's staff consisted of six fulltime teachers, one part-time teacher, a curriculum coordinator, and five teacher assistants. The student to teacher ratio was 10:1 (including assistants). COSLA follows the Memphis City Schools calendar.

Smithson-Craighead Academy (SCA). Smithson-Craighead Academy is an urban school located in Nashville. The school is housed in an older building which previously served as an elementary school within the Nashville Metro District School System. Approximately 174 students in grades K ? 4 attended the school. The students were predominantly African American (99%) and eligible for free or reduced price lunch (97%). The school maintained a staff of ten full-time teachers, two part-time teachers, and 10 paraprofessionals. The student to fulltime teacher ratio was 18:1. The mission of the school is to meet the academic and social needs of at-risk children through mastery of basic academic skills, student self control, obedience, and diligence. Smithson follows the same calendar as the district and continues to offer a summer school session. The school also offers several popular after-school programs and a variety of extra-curricular activities.

Star Academy. Star Academy is located in northeast Memphis. The school is housed in a former preschool building on the grounds of a Baptist church. Approximately 200 students in grades K ? 4 attended the school. The student population was predominantly African American (99%), with approximately 88% qualifying for free or reduced price lunch. The school maintained a staff of 10 fulltime teachers, one part-time teacher, and four paraprofessionals. The student to fulltime teacher ratio was 20:1. The primary focus of Star Academy is "back to basics," with an emphasis on literacy and mathematical competencies. The school operates from 7:30 ? 3:00 three days per week and 7:30 ? 4:00 on two days.

Promise Academy. Promise Academy is located in Memphis and currently housed in St. Paul's Episcopal Church. Of the 115 students served in grades K ? 1, 97% were African American and 90% were eligible for free or reduced price lunches. Six fulltime teachers served the students, with a student to teacher ratio of 20:1. The school's mission is "to prepare children to excel in the nation's most rigorous high schools. Our central and only work is to teach and inspire the mind, body and spirit of our children so that they can succeed in any academic or cultural setting." The educators at Promise Academy want their students to learn and excel and want to provide that opportunity in a unique atmosphere that the students would not experience in most elementary schools. The school day is from 7:30 ? 4:30.

Southern Avenue Charter School for Academic Excellence. Southern Avenue Charter School for Academic Excellence is located in Memphis and housed at Greenwood Christian Methodist Episcopal Church, near the University of Memphis campus. The school served 103 students in grades K

4

? 2. In addition to the basic curriculum, "kaleidoscope" activities such as art, music, ballet and tae kwon do are offered. The school also offers Spanish once a week. The school calendar and school days are extended at Southern Avenue, with "kaleidoscope" classes beginning at 3:00 in the afternoon. The school supplements the students' education with reading and math tutorials during an after school program, which runs from 4:00 pm to 6:00 pm.

Method

Design and Variables

Given the different grade levels served and curricular objectives emphasized by the charter schools, the results for each were examined separately. Student-level matching was not conducted to examine achievement outcomes because most students who attended these schools would not have attended a previous school, and therefore would have no pre-program achievement data to use for matching purposes.

Instead, ordinary least squares multiple regressions were used to determine the effects of charter school enrollment, lunch status, gender, and prior achievement (where applicable, in grade 2 and above) on current year (2006-07) achievement to determine the extent to which: (1) all three or four variables combined explained variability (variance) in achievement, and (2) which individual variables, if any, had a unique, significant influence on current year achievement.

The standardized regression coefficient, Beta, was used to assess the strength of each independent variable that was significant. In addition, the R-square values are discussed for each analysis and reported in the tables to indicate the proportion of variance the independent variables explained in the outcome variable. The higher the variance explained, the better the regression model. In these analyses, the primary variable of interest is charter school enrollment, which if statistically significant, would constitute a reliable predictor of students in general performing higher or lower on TCAP: (a) based on their enrolling or not enrolling in a charter school and (b) after controlling for their prior achievement (where available), gender, and lunch status.

Effect sizes were computed by a formula called "Cohen's d." Each effect size (or d) indicates the number of standard deviations by which the charter school student mean differs from the control student mean. Thus, an effect size of +0.50, would indicate a half of a standard deviation advantage. Generally, in education, effect sizes exceeding +0.20 would be considered meaningful and fairly strong when obtained for a whole-school intervention. In all cases, Cohen's d effect size was computed as the mean difference (treatment ? control) divided by the pooled standard deviation.

When conducting a regression analysis, one potential problem is the issue of multicollinearity. This occurs when the independent variables (in the case of this analysis: Charter school enrollment, lunch status, gender, and prior achievement) are highly correlated with each other as opposed to the dependent variable (i.e., 2006-07 Reading/LA and Math results). For example, two independent variables may measure the same construct, thereby fighting over shared variance and making it difficult to determine which variable is actually producing the effect on the outcome measure. One result can be a lack of statistical significance for individual independent variables while the overall model may be strongly significant. Multicollinearity "may also result in wrong signs and magnitudes of regression coefficient estimates, and consequently in incorrect conclusions about relationships between independent and dependent variables" (Multicollinearity in Logistic Regression, n.d.). The variance inflation factor (VIF) is one way of determining if multicollinearity exists. If any VIF exceeds 10, multicollinearity may be present (Stevens, 2002, p. 92). Preliminary examination of the results indicated there was no multicollinearity in the data (all variance inflation factors were less than 1.2).

5

Sample restriction

All Memphis City Schools and Nashville Metro students from non-charter, Title I schools in 2nd through 5th grades who took both the 2005-06 and 2006-07 TCAP Achievement Test and all 1st grade students from non-charter, Title I schools who took the 2006-07 TCAP Achievement Test were initially selected as control students. The Nashville Metro students were utilized for the Smithson-Craighead comparison, whereas the MCS students were used as the comparison group for the Memphis-area charter schools. Charter school or control students that fell into any of the following six categories were not included in the analyses:

1. The grade level tested did not match the grade level in which the student was enrolled (e.g., a fourth grade student who took the 3rd grade level test), or

2. Not continuously enrolled1 in the 2006-07 year, or

3. Special education students in the 2006-07 year, or

4. ELL students in the 2006-07 year, or

5. Retained students in the 2006-07 year (i.e., the student was in the same grade in both 2005-06 and 2006-07), or

6. Missing data, including NCE Math and Reading (for 1st and 2nd grades), Number Correct Math and Reading/LA (grade 3 and above), gender, lunch status (Free, Reduced, or Full Pay), grade level, or test level for either the 2005-06 or 2006-07 years (excluding the 2005-06 data for the students in 1st grade).

In addition, any control students who attended an Optional School or were not in a Title I school in the 2006-07 school year were also dropped.

NRT data for 1st grade students were not available for Star Academy and Circles of Success. Therefore, no analyses were conducted for first graders at these two schools. Additionally, Nashville Metro schools, including Smithson-Craighead, did not test students in grades 1 or 2 during the 2006-2007 academic year.

Control Group Selection

Separate control groups were established using Memphis City and Nashville Metro students. The development of the grade level control groups is discussed below.

Memphis City Schools

First Grade. Of 2545 1st grade control students available, 719 were dropped. Of these, 334 were not continuously enrolled, 175 were special education students, 158 were ELL students, 51 had been retained from the previous year, and 1 was missing the 2006-07 test data. Therefore, 1826 control students in 1st grade were included in the analyses.

Second Grade. Of the 8058 2nd grade control students available, 5945 were dropped: 1110 were not continuously enrolled, 741 were special education students, 480 were ELL students, 159 had been retained from the previous year, 10 were missing the 2006-07 test data, and 3445 were missing the 2005-06 test data. Thus, 2113 control students in 2nd grade were included in the analyses.

1 Defined as being enrolled at the school since the first 20-day attendance reporting period until the test administration. Schools are only held accountable under No Child Left Behind (NCLB) for making Adequate Yearly Progress (AYP) based on students who are defined as "continuously enrolled" at their particular school. This is also known as the student's "membership" status.

6

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download