Short -Term Study Abroad Programs - Free Website Builder ...



Short -Term Study Abroad Programs

[pic]

Psychology 600: Introduction to Measurement & Statistics

Kelly Detrick, Mary Frier, Marshall Greenleaf & Dan Murphy

[pic]

JMU Office of International Programs

[pic]

Short-Term Study Abroad Programs

Introduction

The problem:

There is no supporting data to show that the objectives of the short-term (3-6 weeks) study abroad programs are being met as effectively as the semester/year long study abroad programs, resulting in the possibility of reduced funding and resources for short-term study abroad programs.

The Program(s):

James Madison University offers a wide variety of semester and short-term study abroad programs. The semester programs include: Antwerp, Florence, London, and Salamanca. This year, JMU will offer about 40 different short-term study abroad programs. These programs vary from year to year depending on resources and the interest of students and faculty. The majority of short-term programs occur during summer break, with one or two scheduled during winter break.

Short-term study abroad programs have dramatically increased in popularity over the past decade. This is largely due to the convenient options they provide. They rarely conflict with a student’s major or course of study, and they are a cheaper option than semester or year long programs. A listing of the short-term programs for 2006 is attached in Appendix A.

One of the greatest challenges that we face when gathering data from trips revolves around immersion into a culture. On many of the short-term trips, immersion into the culture is not possible. Students usually attend these trips as a group and spend the majority of their time interacting with the students from their home institution. This allows for considerably less interaction between our students and native students to the region in which they are traveling. Immersion would have a significant effect on the data collected in regards to cultural norms, traditions, and language. Within that, there are still challenges with short-term immersion.

The language of a region will play a large part in the learning of the student and the effect of the program. Students that spend time in English speaking countries have an easier time assimilating into the new culture while students in non-English speaking countries face a greater difficulty in transition. Students studying in English speaking countries have the benefit of English speaking instructors, tutors, and community members. This allows the student to focus primarily on gaining knowledge in content areas and curriculum. Conversely, those students in non-speaking countries have to concentrate more on understanding others before they can retain content. This has potential to affect the data collected in terms of the history, politics and religion of the region and in their academic course of study.

It would be important in collecting data to pay attention to the scores on the foreign language exams for students traveling between English- and non-English speaking countries. Generally speaking, students traveling to a country that speaks the same language as their native country would not show as much growth in the area of language acquisition.

The Purpose of the Evaluation:

By evaluating objectives of the short-term study abroad programs, we can determine their effectiveness in increasing students’ knowledge in the following areas:

Cultural Assumptions

Foreign Language Skills

Knowledge of other cultures and traditions

Knowledge of international politics, history, religion, and current events

International experiential learning

By the conclusion of our study, we hope to address the following questions:

▪ Is there a significant difference between the students who participated in a short-term study abroad program and those who did not in the ability to identify cultural assumptions?

▪ Is there a significant difference between the students who participated in a short-term study abroad program and those who did not in the ability to speak a foreign language at a conversational level?

▪ Is there a significant difference between the students who participated in a short-term study abroad program and those who did not in the ability to identify cultural norms and/or traditions?

▪ Is there a significant difference between the students who participated in a short-term study abroad program and those who did not in the knowledge of history, politics, religions, and current events of other countries?

▪ Is there a significant difference between the students who participated in a short-term study abroad program and those who did not in gaining experiential credits that cannot be gained in the traditional campus classroom?

The Rationale for the Evaluation:

The Office of International Programs at James Madison University has received a grant to assess the effectiveness of the semester and year long study abroad programs. However, there are currently no evaluation measures in place to assess the effectiveness of the short-term study abroad programs. Given the amount of students and faculty who participate in the short-term study abroad programs compared to the semester long programs (about 5 times as many), the assessment of the short-term programs is essential.

In addition, recent criticism of short-term study abroad programs has arisen nationwide. Do these programs truly provide educational benefits? Are they more than a vacation? Do they improve foreign language skills? Do they help students understand other cultures and interact with people of other nationalities more effectively? Most importantly, are they long enough to make a difference? The evaluation of the short-term study abroad programs is essential to their continued existence in higher education. Without it, resources and funding for these programs may decrease.

Literature Review

Study abroad is an extremely popular program throughout the country aimed at increasing students’ awareness and knowledge of another culture. The terms awareness and knowledge encompass a very broad range of skills and information, from learning the language of the foreign country to obeying the cultural norms and guidelines to classroom knowledge of the historical background of the country and culture of one’s origin of study.

Study abroad programs are available in differing time lengths, from year long to semester long to summer study’s and even two-three week experiences. The cross-cultural competence gained in these various length experiences will vary but steps can be taken to enhance learning while abroad despite the length of time. Helpful preparation steps include “provide background and context for the experience abroad, and opportunities to develop language skills, research skills, and interdisciplinary research projects” (Lewis & Niesenbaum, 1).

The authors’ of “The Benefit’s of Short-Term Study Abroad” surveyed students they worked with in a short-term study abroad experience in Costa Rica. Their interviews reached four main conclusions: the experience enhanced the participants willingness to take courses outside their area of study when back at their home college; almost half of those who participated traveled or studied abroad again; “most of the students demonstrated increased interest in interdisciplinary studies;” and all who participated expressed an increase on their “perceptions of the costs and benefits of globalization” (Lewis & Niesenbaum, 2). Overall the students who participated in Lewis and Niesenbaums program “reported that it had made them question their assumptions, gather and interpret data, and use the data to reach a better understanding of their own role in a globalized society” (2).

Study abroad programs are often assessed to determine their success at meeting their learning objectives. Kyushu Lutheran College in Japan has a mandatory study abroad program for its students. The purpose and goal of this program is:

Providing awareness of global and ecological issues will be done by the experience of living in a foreign environment. This awareness will grow by (a) physically crossing borders and entering architectural spaces different from home; (b) personally engaging with people using a foreign language for purposes of friendship, study, and daily needs; (c) effectively functioning in programs that are not designed by us but provided by partner institutions. These physical, interpersonal, and institutional experiences are utilized to expand awareness (Ingulsrud et/al, 474).

A team of culturally experienced individuals were chosen to develop an assessment tool for the Kyushu study abroad program.

Several assessment tools are already in existence to measure cross-cultural competence. Byram and Morgan “focus on three factors: knowledge, empathy, and behavior in their assessment” (475). The tests are in both oral and written format. Survey has been the most popular method of assessment to date. Multiple choice tests can also be used to determine cross-cultural competence as evidenced by Seelve (1991), although even Seelve (1991) himself writes about the poor reliability of such an instrument in assessing this aspect of learning. Valette (1977, 1986), another assessment guru, “suggests instruments that are open-ended, involving reactions to statements or visual stimuli” (476). A fourth approach to cross-cultural competence assessment is the portfolio. The portfolio allows for the inclusion of a versatile degree of instruments and material and it is more holistic in its approach.

Upon researching the assessment tools available the Kyushu team opted for the portfolio option. With the portfolio students are actively engaged in “selecting examples of their work that exhibit both their performance level and growth” (477). Items that can be included vary from research papers, audio/visual samples of language proficiency, foreign language writing sample, experiential journals etc. “The main advantage of portfolio assessment is that it allows an individual student’s growth to be demonstrated. It is a personalized approach and takes a longitudinal view of the learning process” (479). One significant disadvantage of this method is that comparisons can not as easily be made between study abroad participants and non-participants. The criteria are also subject to reviewer’s bias even though its contents must meet uniform guidelines.

There are several reasons to assess and several means to go about obtaining the feedback one desires. It is essential to determine clear and measurable outcomes for the program and then tailor the assessment tools accordingly.

Methodology

For our evaluation design, we will be administering pre and post-tests for the first four research questions. Our control group will consist of an equal number of non-study abroad students matched by academic performance to our short-term study abroad participants. Our subjects will consist of JMU short-term study abroad participants vs. JMU non-study abroad students. Procedures for data collection include pre-tests for the study abroad participants which will be administered during their study abroad orientation.

Pre-tests for the non study abroad students will be randomly administered in a computer format when students register for fall classes on e-campus (March). By participating in the study, students will be eligible for a drawing for early registration. Post-tests will be administered on the last day of classes for study abroad participants. Post-tests for non study abroad students will be administered when they go to register for spring classes (October).

We will be using a variety of instruments. To test growth in cultural assumptions, we will administer the Cross Cultural Adaptability Inventory. In order to test our second research question that focuses on foreign language skills, students will participate in the James Madison University computerized foreign language placement exam (jmu.edu/onebook/ index.shtml).

To measure knowledge of other cultures and traditions, students will be tested in their ability to identify and list 3 cultural norms and/or traditions of a country other than the United States. Knowledge of international politics, history, religion, and current events will be measured with a multiple choice pre/post test.

For international experiential learning, there is no instrument required (1=yes, 2=no). Students participating in the short-term study abroad program will receive a score of one. Students not participating in a short-term study abroad program receive an automatic score of two.

Proposed Analysis and Hypothesis

After careful consideration of our research questions and methodology, we have determined the following means of analysis to be the most appropriate and effective way to assess results of our study. We have also developed hypothesis for each research question. In our study, the independent variable is participation on a short-term study abroad trip.

The first question that we plan to address is if there is a significant difference between the students who participated in short term study abroad trips and those who did not in the ability to identify cultural assumptions. In order to analyze this we will be using a Chi Square. Our hypothesis is that students who participate in short term trips will be significantly more able to identify cultural assumptions versus students who did not participate in a trip. The null hypothesis for the first research question would be that there is no difference in the ability to identify cultural assumptions between students that participated in trips and those that did not.

The second question that we are addressing is if there is a significant difference between students that participated in trips and those that did not in the ability to speak a foreign language at a conversational level. We believe that students who do participate in a short-term study abroad trip will be able to speak a foreign language at a conversational level. Students who do not participate will not be able to speak a foreign language at a conversational level. The null hypothesis for the second research question is that there is no difference between participants and non-participants in the ability to speak a foreign language at a conversational level. Results from the second research question will be analyzed using an Independent T-Test.

The third question that we are assessing is if there is a significant difference between the students who participated in a short-term study abroad trip versus those who did not in the ability to identify cultural norms and/or traditions. We believe that students who did participate will be better able to identify cultural norms/traditions than students that did not participate. The null hypothesis for this question is that there is no difference between the two groups in the ability to recognize cultural norms or traditions. This question will be analyzed using a Chi Square.

The fourth research question that we are addressing is if there is a significant difference between the students who participated in a short-term study abroad program and those who did not in the knowledge of history, politics, religions, and current events of other countries. We hypothesize that students who participate in short-term study abroad trips will have a greater knowledge in these areas, while non-participants will not have as much knowledge. The null hypothesis is that there is no difference between the two groups in the knowledge of history, politics, religions, and current events of other countries. The data from this question will be analyzed using an Independent T-Test.

The fifth and final question that we are researching is whether or not there is a significant difference between students who participated in short-term trips and those that did not in the acquisition of experiential credits that cannot be gained in the traditional classroom. Our hypothesis is that participants in the short-term study abroad program will gain experiential credits while non-participants will not. The null hypothesis is that there is no difference in experiential credit acquisition between short-term study abroad participants and non-participants.

Discussion

Our research is targeting students that attend short-term study abroad trips offered through James Madison University. Short-term trips last between three weeks and a couple of months. Long-term trips last either a semester or a whole year. There are some limitations to the research that we wish to conduct.

In terms of data collection, a limitation could be the way that information is being collected. Most of our instruments would utilize multiple-choice instruments, which are not always the most convincing form of data collection. It will sometimes diminish the value of the data being collected or not necessarily be the most effective form of data collection.

Some programs include a portfolio project as a part of their assessment. Portfolios assist the student with assimilation of material into their personal experience as well allowing for a more subjective approach to assessing the student’s learning. In choosing multiple-choice instruments, we are creating a more objective approach to our assessment experience. We may be sacrificing some of the personal development that students may experience abroad. The multiple-choice instruments restrict personal expression and do not provide as holistic of an approach that more qualitative assessments would allow.

Another confounding variable to account for could be the self-selection that takes place for study abroad trips. The cost difference between a semester abroad and a semester at the native institution can be a factor. Therefore, many students traveling abroad may come with a lot of resources not available to all students. With the resources available to them, they have always been encouraged to perform well in school and are able to afford opportunities that poorer students may not be able to afford. Since not everyone can go on these trips abroad, in many cases, the students attending may already be students in good standing with the university. They may have a proven successful track record of academic performance that would affect our data regarding academic course of study.

Students choosing to go on the trip may show little growth in the area of academic performance because they already perform well. To account for this, we plan to match participants in the experimental group with students in control group based on academic performance.

As always, the world is in constant change and over the past few years, it has become more difficult for Americans to travel freely throughout the world. The way current events are playing out around the world; there is a host of variables that might affect student performance or their overall experience on the trip. For instance, in cases of terrorism, students studying abroad are usually taken to their embassy and supported by predominantly American service personnel. This can take away from the cultural experience of the trip.

On the flip side, following tragedy, many countries experience extreme patriotism following terrorist events. Many times, the new culture is very different from the culture of the region prior to the attacks. This could create confusion for students studying abroad or allow them to see aspects of a culture that do not normally exist.

A significant part of the study abroad experience is living in a different value system. There is nowhere that a different value system is more apparent than in terms of the use of drugs and alcohol. Many countries abroad have legal drinking ages significantly younger than America and some countries even allow the legal use of substances that are illegal within the United States. Many times, students find themselves in this new value system, unprepared to handle the use of these substances and they begin to play a considerable role in their abroad experience. Excessive abuse of these substances could affect the data collected in terms of content retention. It may also facilitate dependence of substances that would require attention upon arrival back in the United States.

The great thing about study abroad experiences is the personal growth that takes place for the participants. It is sometimes difficult to extract all of the data that researchers are looking for because a lot of the personal growth that takes place is intangible. Other areas of further research could include student confidence as a result of their study abroad experience. Student’s self-efficacy could also be studied as well as ability to manage stress as a result of their study abroad experience. There is a lot of room for growth with the evaluation and assessment of study abroad programs.

Within the assessment that we are performing there are many practical implications for the programs we are studying. First and foremost is the ability to justify the existence of short-term study abroad trips. There is a lot of criticism that short-term trips do not provide the same growth as long-term trips. Through this assessment we plan to solidify the base of short-term trips offered by James Madison University. It would also set the stage for further growth and expansion of short-term programs to new countries, new schools, and for more people. It would also be our hope that the results of our study would encourage the administration to consider additional fiscal resources to be put towards short-term study abroad trips.

We will also be able to see if students are really learning what we expect them to learn in the experience. If students are returning and there is no significant growth in the targeted areas then we will need to reevaluate the content and curriculum of these experiences abroad. The Office of International Programs will also be able to see if there are specific sites abroad that students learn better in compared to others. Finally, the study could provide some support for re-entry programs offered through Study Abroad. We may gain some insight into what it is like to return to America after some time in a foreign country and what those students may need in terms of support.

Works Cited

Ingulsrud, J., Kai, K., Kadowaki, S., Kurobane, S., & Shiobara, M. (2002). The assessment of cross-cultural experience: measuring awareness through critical text analysis. International Journal of Intercultural Relations. 26. 473-491.

James Madison University Office of International Programs. Retrieved November 11, 2005 from jmu.edu/international.

James Madison University Foreign Language Placement Exam. Retrieved November 13th, 2005 from jmu.edu/onebook/index.shtml.

Lewis, T., & Niesenbaum, R. (2005). The Benefits of Short-Term Study Abroad. The Chronicle of Higher Education. 51 (39), B20.

Short-Term Study Abroad Opportunities. Retrieved October 21, 2005 from .

Wang, Felix. Office of International Programs, JMU. Personal Interview. October 11, 2005.

Appendix A

Short -Term Programs - Tentatively Planned for 2006

|Africa |Europe |

|Ghana: History and Culture |Altamura: Fresco Preservation and Field |

|Kenya: Field Study |Archaeology |

|Zambia: Zambian Arts and Culture |Belgium: Marketing and Economics |

| |Germany: ISAT |

| |Germany: Music |

| |Madrid: Appreciation of Spanish Art |

| |Malta: ISAT |

| |Malta: Health and Human Services in Malta |

| |Paris: Language and Culture |

| |Perugia: Cross-Cultural Psychology |

| |Prague: Business Environment of Central Europe |

| |Prague: Literature and Culture |

| |Rome: Teacher Education Practicum |

| |Russia: Language and Culture |

|Asia | |

|China: Chinese Business Studies | |

|China: Chinese Culture | |

|India: Contemporary India | |

|Philippines: Geography: The Global Dimension | |

|Australia | |

|Melbourne: Teacher Education Practicum | |

|Tasmania: Biology Minimester | |

|Britain |Latin & South America |

|Ireland: Geology Field Camp |Argentina: JMU in Argentina |

|Ireland: Writing in Ireland |Dominican Republic: Service Learning |

|London: Shakespeare's Politics |Galapagos: Natural History Studies |

|London Dance: Contemporary Theatre and Dance |Guatemala: International Environment Management |

|London Institute: Colonialism and Culture |Martinique: JMU in Martinique |

|London Institute: British Cinema and Media |Nicaragua: International Energy Studies |

|London Science: Physics, Chemistry, and the Human Experience | |

|Scotland: Cultural History | |

|Canada |USA |

|Montréal: Music and Culture |American Southwest: Archaeological Field School |

|Montréal: Intercultural Communication | |

Appendix B

Cross-Cultural Adaptability Inventory (CCAI)

Assessment of International Programs (Research Question 1)

The Cross Cultural Adaptability Inventory is an assessment tool designed to measure an individual’s ability to succeed in a cross cultural environment. This assessment is given in pretest/post-test format. The test asks fifty questions that are scored on a six point Likert Scale ranging from definitely true to definitely not true. The post-test and pretest are exactly the same.

The fifty questions of the CCAI measures seven basic skills and mental states related to intercultural communication skills. This includes Emotional Resilience, Flexibility and Openness, Perceptual Acuity, and Personal Autonomy. Each category has 10-15 questions that are used to measure each student’s score in each respective category. The pretest is given before departure on an international program, and the pretest is given towards the end or immediately upon return.

The CCAI was developed by Colleen Kelley and Judith Meyers. Each of these psychologists had substantial experience in international organization development, and the pilot testing phase for the CCAI was conducted through the International Society for Intercultural Education Training and Research. Professor Lynn Brown of Northern Arizona University sites this broad experience base as the strength of this assessment tool. The CCAI is also sited as being an adequately valid tool.

The sample group used in testing the CCAI, however, does flaw the validity of the instrument. No racial or ethnic demographics were collected in the testing of the CCAI, and the balance of U.S. Citizens vs. International Citizens was not broken down. The instrument also measures complex personality traits with very few questions, thus possibly biasing the results. The validity of the instrument is further called into question due to the lack of information regarding the sample group as well as the inclusion of too many homogenous groups, i.e. an over representation of males.

We feel that this test could be used to evaluate certain developmental objectives for international program participation. While it does not address any objectives dealing with language, it does address issues of cultural assumption and how comfortable a student feels in a cross-cultural situation. The test is relatively short and very easy to administer. We feel it is an efficient, relatively valid tool that can adequately assess our program objectives and outcomes.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download