ELEMENTARY SCHOOL COUNSELING DEMONSTRATION ACT



ELEMENTARY SCHOOL COUNSELING DEMONSTRATION ACT

MILE HIGH ELEMENTARY SCHOOL COUNSELING PROJECT

PROJECT STATUS REPORT

2002-2003

I. OBJECTIVES

Measure progress towards goals and objectives.

Measure the effectiveness of program strategies.

Measure student outcomes.

Measure increased capacity of school and staff to deliver programming and manage classroom behavior.

Generate and share knowledge that can be used among partnership members for ongoing quality improvement, potential replication and obtain future funding.

II. DATA COLLECTION

The evaluation team (Dr. Daniel Hettleman and Kristi McCollum, MA) have collected data for the third year to assess the effectiveness of project strategies. Data collection was significantly improved this year through the addition of:

• Completed measurements and assessment of change in students’ reading grades, allowing detection of changes in reading abilities that may have been impacted by this project.

• Pre-treatment and post-treatment ratings of children’s social skills and reading grades, allowing more accurate and structurally valid assessments of change.

• Focus groups with staff from two elementary schools where character education programs were offered.

The rest of the data collection, in all 8 schools, was conducted the same as last year. Data include:

• Teachers’ lists of the three children most concerning behaviorally, at the beginning and end of the year. This also included whether children were referred for “social skills”, “anger management”, or “both”.

• Each served child’s number of ART groups attended; number of individual therapy sessions; number of family sessions, home visits, parent contacts, and referrals to outside agencies.

• Each served child’s number of office referrals and suspensions as a result of aggressive-defiant, “educationally interfering”, or self-destructive behaviors. To accurately assess change, these totals were tallied for two time periods: (1) 20 school days in October 2002 (near the beginning, but not the exact beginning of the school year), and (2) 20 school days in April 2003 (near the end, but not the exact end of the school year). These time periods were chosen due to their expected sensitivity to potential change in behavior resulting from this project.

• Reading grades in the 1st and 3rd grading quarters (grades from the 4th quarter were not chosen because too many extraneous variables potentially affect grades as summer nears). Grades were coded on a 4-point GPA scale commonly used to quantitatively measure grades in other school settings (high schools, colleges).

• Teachers’ ratings of each child’s social skills (see appendix 1). This was a twelve-item measure directly derived from the author of the ART (Aggression Replacement Training) program, Arnold Goldstein, Ph.D. Ratings were applied on a 5-point scale, measuring how often the teacher observed the child engaging in the skills (almost never, seldom/rarely, sometimes, frequently, almost always). The measure was chosen due to its use in past research demonstrating benefits of ART programming, as well as its solid psychometric properties. A few items were slightly changed by the evaluation team following discussion with the 9 school counselors, taking into account their assessments of what they were targeting most with their children. As mentioned above, this questionnaire was filled out by teachers in September and in May, to better assess children’s changes over the course of this school year.

• Children’s ratings of their own social skills (see appendix 2). This measure was exactly parallel to the teacher rating form, but was administered only to 3rd, 4th, and 5th grade students, who were thought to be better self-raters than younger children.

• It is important to note that due to a computer error, data collection from one half of one school was uninterpretable, and therefore not included in analyses. Sum totals of services provided, as described below, are therefore likely to be underestimates by approximately 6.25%. For instance, where it states that 192 children were referred for ART at the beginning of the school year, it is more likely that the true number of children referred was approximately 204 [(192 x .0625) + 192]. Mean estimates of grades, social skills, and behavior problems are not likely to be affected, given the large sample size of data collected.

III. QUANTITATIVE RESULTS

A. Number of students identified as needing individual or group counseling

On the teachers lists of the three children in their classrooms most concerning behaviorally, the following results were obtained:

• 192 students were referred for ART at the beginning of the school year

• 70 more students were referred for ART during the rest of the school year for similar reasons, even though they had not been identified as a “top 3” needs child by individual teachers in September.

• Of the 262 total ART participants, 40.8% were referred for “social skills training”, 18.7% were referred for “anger management training”, 29.0% were referred for “both”, and 11.4% were referred for “other” reasons.

• Of the 262 total ART participants, 172 had 15 lessons or more; 223 had 10 lessons or more.

• Of the total ART participants, 76% completed their expected amount of time in ART; 14.4% left school before completion, and 3.4% were removed from ART for behavioral or other reasons.

• 43.8% of the students who were on the initial “students of concern” lists were no longer on that list by the end of the school year. Although this data is not conclusive, it is possible that this shift reflects considerable progress for many of those children – reflecting either behavioral improvement, change in teachers’ perceptions of those children, or other unmeasured explanations (e.g. other children emerged as even more difficult).

• 246 students received individual counseling sessions; 128 had 5 sessions or more, and 37 had 15 sessions or more.

B. Number of families receiving counseling or referred for services

This was compiled by each school counselor charting the families participating in counseling sessions, groups or Parent Empowerment Nights.

• At least 244 students’ parents had contact with the counselors; 111 had more than two contacts with the counselors.

• 77 students’ parents attended Parent Empowerment Groups; 33 attended more than two sessions.

D. Number and severity of office referrals

This was tallied by each school counselor for the 8 schools on a master data sheet that included 3 categories of disruptive behavior: “Aggressive-Defiant”, “Educational Interference”, and “Self-Destructive”. A comparison between Fall and Spring office referrals was made in order to assess one aspect of the impact of the interventions. This comparison can be seen in the table below.

| | Fall Office Referrals |Spring Office Referrals |

| | | |

|Total |93 |91 |

| |(67 children) |(60 children) |

|Aggressive-Defiant |78 |75 |

| |(53 children) |(46 children) |

|Educational Interference |15 |14 |

| |(14 children) |(13 children) |

|Self-Destructive |0 |2 |

| |(0 children) |(1 child) |

• As can be seen in this table, the number of office referrals very slightly declined between the two measurement periods. There is no control or comparison group to know if this slight decline is better, equivalent, or worse than that for children not treated with ART.

• Fewer children (60 versus 67) did get office referrals in the Spring than in the Fall, reflecting some positive change. Almost all of this change was accounted for by 7 fewer children getting office referrals for aggressive-defiant behavior.

• The data suggests that children referred for either “social skills” or “anger management” deficits showed a greater decline in office referrals than children referred for “both” or “other” reasons, although the difference was not statistically significant. It is possible that children rated as “both” were more difficult to treat with ART.

E. Number of suspensions

This was tallied in the same manner as described in section D.

| |Fall Suspensions |Spring Suspensions |

| | | |

|Total |80 |111 |

|Aggressive-Defiant |67 |83 |

| |(40 children) |(50 children) |

|Educational Interference |13 |27 |

| |(10 children) |(21 children) |

|Self-Destructive |0 |1 |

| |(0 children) |(1 child) |

• As can be seen in this table, the number of suspensions significantly increased between the two measurement periods. Again, it is not known whether non-treated children had the same type of increase or not. Children referred for either “social skills” or “anger management” deficits again seemed to show more improvement than children referred for “both”. Children with the lowest social skill ratings in the Fall showed far greater increases in suspensions between Fall and Spring, compared with children with the highest social skill ratings in the Spring, who showed some decline in suspensions. These differences approach but do not quite reach statistical significance.

F. Changes in Social Skills

Teachers were asked to rate students' social skills in the Fall and Spring, allowing a more sensitive detection of change over the course of the year. The ratings were comprised of twelve items (see Appendix 1), measuring listening, anger control, on-task behavior, self-control, avoiding trouble, emotional regulation, being able to identify feelings, empathy, handling teasing, avoiding fights, following directions, and making friends. Third through fifth-grade students were asked the same questions. Results are described in the table below.

|SOCIAL SKILLS |Average score in the Fall |Average score in the Spring |

| |(5-point scale) |(5-point scale) |

| | | |

|Teacher ratings |2.78 |3.13 |

|Student self-ratings |3.49 |3.64 |

|(grades 3 through 5) | | |

• As can be seen in this table, teachers saw significant improvements, on average, in ART-treated children’s social skills between the Fall and the Spring. Children detected improvements in themselves as well, although not as strongly.

• The magnitude of change in Teacher ratings of social skills equals approximately 1/3 of a standard deviation. In measurement of psychological change, this is considered a “moderate gain”. The magnitude of change in children’s Self-ratings of social skills is considerably smaller, closer to 1/10 of a standard deviation, and is considered a “small gain”.

• The change in teacher ratings is quite significant, given that it can reflect (1) more observable gains, and (2) changes in children’s reputations from the teacher’s perspective. It is also noteworthy that the overall average score increased from below the “middle” point of a 5-point scale (2.78), to above that middle point (3.13).

• The strongest gains in teacher ratings occurred for the items “can identify his/her own feelings”, “shows sensitivity for others’ feelings”, and “knows what to do if someone is teasing him/her”.

• The strongest gains in self-ratings occurred for the items “I know my own feelings”, and “I know how to keep out of fights”.

• There was a mild correlation between Fall reading grades and amount of improvement in teacher-rated social skills, such that children with higher reading grades in the Fall demonstrated greater improvement.

• Conversely, children with the lowest social skill ratings in the Fall showed by far the greatest improvement in teacher ratings over the course of the year (p ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download