Association of American Colleges & Universities

4/30/2019

The Scope of BTtoP Research: Design and Findings from the Demonstration Project | Association of American Colleges & Universities

Association of American Colleges & Universities

A VOICE AND A FORCE FOR LIBERAL EDUCATION IN THE 21ST CENTURY

Give to AAC&U

Home > Publica ons Research > Periodicals

Peer Review

The Scope of BTtoP

Resource Hubs

Research: Design and

Findings from the

Demonstration Project

By: Ashley Finley and Lynn E. Swaner

Summer 2007, Vol. 9, No. 3 Buy Print Copies

Current Issue

The Bringing Theory to Prac ce (BTtoP) project seeks the advancement of knowledge and the establishment of best prac ces centered in the BTtoP triangularity of engaged learning, student mental health and wellbeing, and civic development. Since its incep on, BTtoP has established an aggressive research agenda through systema cally evalua ng funded projects that inten onally address this triangularity. The most substan al research effort in this vein is the BTtoP demonstra on project, for which this ar cle discusses the development and design of a research approach, encapsulates findings to date, and describes future research direc ons.

Search Periodicals

Search

TABLE OF CONTENTS

Overview From the Editor Examining the Outcomes of the Bringing Theory to Prac ce Project

Conceptual Framework

Extensive review of the literature (Swaner 2007) iden fied sugges ons as to linkages between engaged learning, student mental health and wellbeing, and civic development. First, at the outcomes level, correla ons have been iden fied between elements of engaged learning (e.g., involvement in group projects and interac on with faculty) and selfreport of be er emo onal health (As n 1993), and between student par cipa on in prosocial ac vi es (such as community service) and lower rates of heavy drinking (Wechsler et al. 1995, Jessor et al. 1995, Fenzel 2005). On a theore cal level, research on



1/17

4/30/2019

The Scope of BTtoP Research: Design and Findings from the Demonstration Project | Association of American Colleges & Universities

Engaged Learning, Student Mental

stress in academic environments has demonstrated that while

Health, and Civic Development: Can moderate levels of environmental stress can lead to op mal

We Demonstrate the Rela onship?

performance, extreme levels of stress can lead to "anger,

Engaged Pedagogies, Civic Development, and Student WellBeing within a Liberal Learning Context

Engaged Learning through Curriculum Infusion

fa gue, anxiety, fear, depression, or boredom" (Whitman, Spendlove, and Clark 1986). This would suggest that if engaged learning can op mize stress levels for students, be er emo onal health may result. Addi onally, because students' level of moral development has been nega vely correlated with substance abuse and other selfinjurious behaviors (Berkowitz 2000),

Assessing the Impact of Engaged

engagedlearning experiences that promote moral development

Learning Ini a ves for FirstYear

may help reduce these behaviors. Finally, in a developmental

Students

The Scope of BTtoP Research: Design and Findings from the Demonstra on Project

Terms of Engagement

view of students' health behaviors, both depression and substance abuse can result from developmental overchallenge posed by the college environment (Rivinus 1992). Thus, counterbalancing challenges with support may improve students' health, as might equalizing levels of freedom and responsibility by increasing students' "social responsibili es

through community work" (Schulenberg and Maggs 2001, 33).

Given these poten al linkages, the project began to formulate its central research goal to explore and describe the rela onships between engaged learning, student mental health and wellbeing, and civic development. It was understood at the project's incep on that engaged learning would most likely not cons tute a silver bullet for either depression or substance abuse, but that there was also enough preliminary evidence to consider engaged learning as a promising communitylevel approach worthy of systema c inves ga on. The central challenge of this effort is to conduct inquiry that is mul variate, contextual, and me sensi ve in nature.

Mul variate Inquiry Due to the complexity of BTtoP's triangularity, project research necessarily extends beyond the tradi onal focus on one or two research variables to a mul plicity of variables that are psychological (mo va on, selfconcept, and selfesteem), affec ve (empathy and caring), valuesrelated (moral and civic), and social, among others. Addi onally, there is the largely unanswered ques on of whether and how these variables actually influence student behavior.



2/17

4/30/2019

The Scope of BTtoP Research: Design and Findings from the Demonstration Project | Association of American Colleges & Universities

This is par cularly true in the case of mental health and well being, as much is s ll unknown about the interplay between gene c, psychological, and environmental factors in students' experiences of depression and substance abuse in college. This includes whether students have any history of problems or previous diagnoses and whether students experience "collateral events" that may impact outcomes. For example, joining a fraternity or sorority has been correlated with higher levels of binge drinking, and the disrup on of interpersonal rela onships can lead to an increase in depressive symptoms.

Thus, the mul variate nature of the research necessitates an equally complex research strategy, one that can determine whether observed changes or lack of changes "in educa onal performance, or psychological func oning, or other outcomes are due to the program under study or to confounding life events" (Waterman 2003, 80). And that moves beyond one or two univariate instruments that "are not designed to capture the full range of poten al impacts of a complex, individual program" (Furco 2003, 15).

To account for these issues, BTtoP research involves a two pronged approach: first, to develop mul variate instrumenta on that allows for the iden fica on of broad correla ons between variables; and second, to employ a diverse range of data collec on methods and instruments, thereby crea ng a composite picture of students' experiences that enables a deeper understanding of the rela onships between variables, and, ideally, to advance research into the realm of causality. This approach is in keeping with what Johnson and Onwuegbuzie (2004) describe as "mixed methods" research, which seeks to integrate both quan ta ve and qualita ve paradigms into a single study of complex problems. As the authors explain, "Today's research world is becoming increasingly interdisciplinary, complex, and dynamic.... [A] mixed posi on allows researchers to mix and match design components that offer the best chance of answering their specific research ques ons" (15).

Contextual Inquiry BTtoP research must take into account the convergence and



3/17

4/30/2019

The Scope of BTtoP Research: Design and Findings from the Demonstration Project | Association of American Colleges & Universities

variance of at least two specific contexts that affect research phenomena: the forms of engaged learning in which students par cipate and ins tu onal culture. As Hecht (2003) describes, service learning is not "a specific program with iden fiable characteris cs... [but] an approach to teaching and learning that is given meaning by the school or organiza on where it is based.... Studies [should] account for the tremendous variability across and even within programs" (107). Thus, BTtoP research goes beyond mixed methods to describe across contexts the nature of the programs under study and the campus cultures in which they are situated.

The granddesign approach (Furco 2003) is one specific methodology that is promising for mul site studies of engaged learning, and service learning in par cular. First, a set of both quan ta ve and qualita ve measurements are selected and used across all par cipa ng sites to measure various outcomes. Then, in addi on to this common group of instruments, a "second set of protocols that allows the researcher to inves gate each unique program site in fuller detail" (26) is also developed. According to Furco, this approach "strives for comprehensiveness as well as for universality" and is therefore "applicable and relevant" across diverse programs (25). The use of this approach has been documented at the secondary school level in the use of the Evalua on System for Experien al Educa on, a package of ten qualita ve and quan ta ve instruments assessing outcomes (academic, social, personal, career, ethical, and civic) of servicelearning par cipa on. The BTtoP research design adapts and extends the granddesign approach to assess engaged learning at the postsecondary level.

TimeSensi ve Inquiry Finally, BTtoP research is necessarily me sensi ve. First, the effects of specific forms of engaged learning may not become evident either during or immediately following student exposure. The impact of an engaged learning experience may extend beyond the actual experience (see As n, Sax, and Avalos 1999), or it is also possible students may not recognize the value of an intense learning experience un l a er its conclusion and they have had me to reflect on it. Secondly, matura onal effects during college are important to consider, as a significant



4/17

4/30/2019

The Scope of BTtoP Research: Design and Findings from the Demonstration Project | Association of American Colleges & Universities

number of students "mature out" of binge drinking over the course of four years (Rivinus 1992).

BTtoP research uses two methods to address this issue. First, it monitors student change using longitudinal data collec on at mul ple points of students' program par cipa on and then analyzes data from each point as well as aggregate data. Second, it employs a quasiexperimental design that uses a comparison group comprised of adequately similar students not par cipa ng in the engagedlearning experience, with the assump on that changes observed in the program groups above and beyond normal matura on (as witnessed in the comparison groups) are a ributable to the engagedlearning experience.

Research Design

To address the ques on of what rela onships exist between engaged learning, student mental health and wellbeing, and civic development, the BTtoP Demonstra on project involves a two ered, simultaneous approach to research using quan ta ve and qualita ve methods at the campus level and crosssite level:

Campus level: All seven campuses submi ed local research protocols and designated a local evaluator prior to project commencement. Campus research plans were required to involve longitudinal evalua on, forma ve and summa ve data collec on, and adequate comparison groups. The range of quan ta ve instruments employed among the seven sites includes campusdeveloped pretest and pos est measures; mental health measures (i.e., Brief Symptoms Inventory), engagement measures (i.e., College Student Expecta ons Ques onnaire), civic engagement scales (i.e., Scale of Service Learning Involvement), and exis ng ins tu onal data sets from na onal surveys (i.e., Coopera ve Ins tu onal Research Program Freshman Survey). Qualita ve methods include clinical interviews, focus groups, and analysis of reflec on journals and course evalua on feedback.

Crosssite level: A na onal, crosssite evaluator worked with each demonstra on site to implement a set of



5/17

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download