Assessment Project – 583:



Test Proposal – APLNG 583: by Jennifer Miller (4/30/08)

1. Type, purposes and uses of the test

a. My test is for a course titled: “American Culture Through Music.”

Description of course:

An ESL through Music course. Content contains topics about American culture taught in a variety of ways, with music as a tool to help the students understand the culture/events better. The teaching approach for this class will likely be different forms of lecture, discussion, student interaction and partnering, student reflection, and hands-on activities. Music will be an integral part of each lesson, however, the way it is used in each lesson may differ. There will be readings, worksheets (to assess various skills), journal writing and reflection, free writing, and opportunities to listen to and speak with each other. The overall desired outcome of this course is that the students will improve their 1) knowledge of American culture, 2) its music, and 3) the English language.

There are three main units in this course:

1) Music Genres: Jazz, Folk, Country, Bluegrass, Gospel, Rock ‘n Roll

Assessment tool I and II

2) Historical Events: Slavery, the Civil War, Martin Luther King Jr. and

the Civil Rights Movement - Assessment tool III

3) Holidays and Special Events: Patriotic Holidays, Minor Holidays,

Halloween, Thanksgiving, Christmas, Graduation, Wedding, Birthday,

and New Year’s parties - Assessment tool IV

Learning goal for Unit #1 -

The students will be able to complete a "lower order thinking skills" assessment based on the types of music mentioned above. They will learn the differences, some history, and the development of each genre. They will be able to decipher the various types by listening to examples of each.

Learning goal for Unit #2 -

The students will be able to process and synthesize their knowledge and thoughts about the history of slavery, the Civil War, and the Civil Rights Movement, by completing a "higher order thinking skills" form of assessment (Oral Summary).

Learning goal for Unit #3 -

The students will be able to create an alternative type of assessment based on their knowledge and experience of holidays and special events in their own country (including music and/or video clips), as compared to America (Power Point project).

Objectives for goal #1 -

By listening to examples, SWBAT differentiate between each genre of music.

SWBAT distinguish factual differences between the music genres.

SWBAT correctly recognize a historical fact about the development of each genre.

Objectives for goal #2 -

SWBAT develop, synthesize, and articulate their thoughts and knowledge about the historical events contained in this unit.

Objectives for goal #3 -

SWBAT produce a power point project containing the following:

- A factual introduction to their home country

- Some pictures of their home country with captions

- A list of the holidays and special events celebrated in their home

country.

- Information about their experience of one or more holidays/special

events in their home country, and how that compares to American

holidays/special events (with music and/or video clip examples).

In class, I will be looking for student progress in these things:

- Ability to follow along in conversation and discussion throughout class (Listening and Speaking).

- Ability to follow written and verbal directions. (Reading and Listening)

- Answer basic questions about content of the lesson. (Reading and Listening comprehension and Speaking)

- Ability to understand and synthesize the lesson material. (Listening, Reading, and Writing)

- Reflect on lessons through journal entries. (Writing)

- Respond to the music samples and historical/cultural facts through written or verbal activities. (Writing and Speaking)

- Ability to interact with classmates and teachers comfortably (Speaking, Listening, and Social skills).

b. Targeted examinee group: upper intermediate and/or advanced ESL/EFL students in the U.S. or overseas. Class of approximately 5-25 students (it will vary).

c. Appropriate use of test: as a final exam for this course. Used to determine amount of material retained by students throughout the course. The focus of the test is on content knowledge.

d. Type of Test: CRT. It will incorporate both lower order and higher order thinking skills. Each component will focus on at least one of the four English skills used during class (Listening, Speaking, Reading, and Writing). The intent of the test is to measure student mastery vs. non-mastery of the content from the course.

2. Structure and components of the test

a. The test has three components (total weight: 100%):

1. Lower-order thinking skills section (weight: 30%)

a. Fill in the blank (music listening test)

1. This section has 10 questions – 1 point each

b. Short-answer

1. This section has 4 questions – 3 points each

c. True/False

1. This section has 8 questions - 1 point each

2. Reading Comprehension and Dictation (weight: 30%)

a. Short reading on a music genre

b. After they read the short article, they will be asked to answer 10 written multiple choice questions and 5 verbal (dictated) multiple choice questions.

c. Each written question is worth 1 point each. Each dictated question is worth 4 points each. (See Scoring section of this proposal for explanation)

d. The dictated questions are to test their listening skills, in addition to their reading comprehension.

3. Oral Summaries of two topics (weight: 40%)

a. Two topics will be chosen from three choices.

b. Each summary must contain at least five facts from the topic handouts given for this unit.

1. There will be eight sections on the rubric that the teacher will use to grade these summaries.

2. Each section is worth five points.

3. Grading is based on content, not grammar.

c. The summaries will be spoken to the teacher one at a time. The students will be given 5 minutes each to speak their response.

d. The genre of Oral Summaries will be taught in class. The students will know ahead of time what the teacher is looking for.

e. Item Specifications for this section are not included in this proposal.

b. The structure of this test allows me to measure all four English skill sets in the following ways:

1. Section One: Music listening skills, music genre content knowledge, and composing brief factual sentences.

2. Section Two: Ability to comprehend a short reading, while also being able to answer written and dictated questions (listening skills).

3. Section Three: This section measures speaking and composing skills, as well as historical content knowledge.

4. Section Four (not included as part of this proposal): This section is a power point project which incorporates writing and synthesizing, as well as composing and research skills. It is part of the Final grade for the course, but not part of the final exam.

3. Item specifications

a. Section One – Fill in the Blank

1) General Description:

This section is a music listening test. When presented with 10 of the songs studied in this course, the students will be able to remember the title of the song and the genre the song belongs to. They will pick the title of the piece of music from a list and write it in the blank, along with the appropriate music genre.

2) Sample Item:

Directions – Identify the title of the song, and the music genre it belongs to. Write your answer in each blank provided. Each song can be found in the “Music Titles” list. The genres are also listed below.

Genres:

Jazz

Folk

Gospel

Rock ‘n Roll

Country

Bluegrass

Music Titles:

Ain’t Nothin’ But a Hound Dog

Shout to the Lord

Blowin’ in the Wind

Take the “A” Train

Fifth Symphony

It’s Good To Know Jesus

Swing, Swing, Swing

Boot Scootin Boogie

In the Mood

Victory in Jesus

Feelin’ Groovy

Rocky Mountain High

My Girl

Man of Constant Sorrow

Achey Breaky Heart

Ave Maria

Responses (title and genre)

1. _____________________________________________

2. _____________________________________________

3. _____________________________________________

4. _____________________________________________

5. _____________________________________________

6. _____________________________________________

7. _____________________________________________

8. _____________________________________________

9. _____________________________________________

10. _____________________________________________

3) Prompt Attributes:

An excerpt of each piece of music will be played at least two times. A blank will be provided for each music title and genre. The student will choose the appropriate song title and genre from the lists and put it in the blank.

4) Response Attributes:

The student will write the title of the piece of music and its genre in the blank provided. A list of possible music titles and genres is provided for the student as a reference.

b. Section One – Short Answer

1) General Description:

Students will choose four out of the six major music genres studied in class and write three main points for each of the four chosen genres. This is mainly testing their retention of facts about each music genre. The students should use complete sentences.

2) Sample Item:

Directions: Think about the six major music genres we studied in class. Choose four of them and write three main facts for each of the four genres you chose. These facts should be specific to each genre, and should be based on what we learned in class. Please write in complete sentences. (You can refer to the music genre list given in the previous section. However, you may not use any of the facts given in the True and False section of this test.)

Genre #1: _______________________

____________________________________________________________

____________________________________________________________

____________________________________________________________

Genre #2: _______________________

____________________________________________________________

____________________________________________________________

____________________________________________________________

Genre #3: _______________________

____________________________________________________________

____________________________________________________________

____________________________________________________________

Genre #4: _______________________

____________________________________________________________

____________________________________________________________

____________________________________________________________

3) Prompt Attributes:

The student will choose four of the six major music genres we studied in class. Once they have chosen, they must write the name of each genre they will be writing facts about (blanks are provided). After providing the name of the genre, they are to write three sentences that explain at least three facts specific to that genre (space is provided).

4) Response Attributes:

The student will write one major music genre for each of the four blanks provided (each one must be different). The student will then write three sentences that describe the main facts about that specific music genre (for each of the four listed). A list of acceptable responses, which were taught in class, will be provided for the test scorer. Each sentence should contain at least one major fact.

c. Section One – True/False

1) General Description: This section contains eight True/False questions about the history and facts of the six major music genres presented in class. The purpose is to test the student’s retention of some of the facts presented in class.

2) Sample Item: Directions - Please answer each question below with the response: “True,” or “False.” “True” indicates that the statement is correct. “False” indicates that the statement is incorrect. Try to remember the facts about each music genre that you were taught in class. Please write the word “True” or “False,” not T or F.

True-False Question Test

1. _______ - Jazz music began in New Orleans, Louisiana (U.S.A.).

2. _______ - Folk music is created by and for “common” people.

3. _______ - The “roots” of Bluegrass music are from Germany, Austria, and

Poland.

4. _______ - Traditional Jazz music blended Caribbean music styles with European

music forms.

5. _______ - Country music developed in the Western United States and the

Appalachian Mountains during the 1950s.

6. _______ - Rock ‘n Roll developed by mixing together various popular styles of

music during the 1940s and early 1950s.

7. _______ - Gospel music originated in the Catholic tradition of Christianity.

8. _______ - Elvis Presley is one of the music artists from the “Rockabilly” period.

3) Prompt Attributes:

The eight questions each contain a correct or incorrect factual statement about one of the six music genres. Space is given for the student to write their response (“True” or “False”).

4) Response Attributes:

The student will fill in the blank provided with either the word “True” or the word “False.” “T” and “F” are not acceptable.

d. Section Two – Multiple Choice Questions

1) General Description:

This section of the test contains a reading sample that the students will read and then answer questions about. The sample is titled: “Spirituals to Hip Hop: The Importance of Black Music in America.” The 15 questions are multiple choice, each containing four responses to choose from. 10 of the questions are in written form, and 5 of the questions will be dictated. The students will write down the entire question and answer options provided through dictation. The students will circle the letter of their chosen response for each question (both written and dictated). The intent of this section is to measure reading comprehension and listening skills.

2) Sample Item: (See Reading Excerpt)

Directions – Read the excerpt provided and answer the multiple choice questions. 10 of the questions are provided in written form, however 5 of the questions will be dictated. When listening to the dictation, please write down the entire question and all four possible answers. For both the written part and the dictated part, please circle the letter containing the correct answer.

Sample Multiple Choice Question:

1. The meaning of cutting edge is:

a. African Americans contribute sharp styles to musical innovation.

b. African Americans have been creative innovators of new types of music.

c. African Americans have not contributed to musical innovation very much.

d. African Americans have destroyed existing styles of music and created their own.

3) Prompt Attributes:

Each multiple choice question will contain a statement and four possible responses. 10 will be in written form, and 5 will be dictated by the teacher to the students. The teacher will state each dictated question up to 3 times for the students. The dictation will be given slowly.

4) Response Attributes:

The student will be provided with 10 written and 5 dictated multiple choice questions. For the 10 written questions, they only have to circle the letter of the correct answer. For the 5 dictated questions, they have to write the question and all four possible answers, and then circle the letter of the correct answer.

e. Degree of Specificity and Content Validity

1) Degree of Specificity

a. I received feedback from four TESL colleagues about this proposal.

b. Each person had a number of good points regarding the test structure. Some of the points were:

1. The original idea of the portfolio project should be a power point with the option of including music and/or video clips from their home country.

2. I’m really testing content knowledge, not so much language skills.

3. The topics are rich and could be expanded in many ways.

4. My course objectives are too broad.

5. I need to make sure I’m testing what I’m teaching.

6. The Oral Summary is a genre that needs to be explicitly taught in class.

7. Clarification is needed in regard to how many times I will repeat the dictation and whether or not the students can have notes for the oral summaries, or if they must be memorized.

c. I was able to incorporate most of these points into my final test proposal. The advice from my colleagues was very beneficial and helped me to refine my test.

2) Content Validity

a. Theoretical Foundation: Communicative Language Teaching

b. Based on David Nunan’s (1991) five features of CLT:

(From )

1. An emphasis on learning to communicate through interaction in the target language.

2. The introduction of authentic texts into the learning situation.

3. The provision of opportunities for learners to focus, not only on language but also on the Learning Management process.

4. An enhancement of the learner’s own personal experiences as important contributing elements to classroom learning.

5. An attempt to link classroom language learning with language activities outside the classroom.

c. According to these five features of CLT, I believe my test supports my theoretical foundation, as I intend to incorporate these features into my teaching (as mentioned in the beginning of this proposal).

d. In addition, I believe my test measures what it is supposed to measure – based on the course objectives, the content, and the skills taught in class. As stated earlier, the intent of this test is to measure content knowledge, more than language skills. Therefore each section focuses mostly on the content learned in class. The skills needed to complete each section of the test will also be taught/reinforced in class.

4. Instructions for administration, scoring, and score interpretation

- Administration of the Test

Test administration will depend upon the time allotted for each class. If the class is around 1 hour, the teacher should administer section 1 and 2 during one class, and section 3 during a different class period. Section 4 is intended to be given to the students at least a month before the end of the semester – so they have ample time to put their project together.

If desired, the teacher can separate all of the first 3 sections into different class periods…depending upon student needs and time constraints.

The teacher should prepare a review sheet and some test taking guidelines for the students, to help them prepare before they are given the test. It is imperative that the teacher instructs the students with the appropriate content and skills needed for the students to do well on the test.

The test should not be rushed. It need not be timed – except for the Oral Summaries (each student is given 5 minutes each). The teacher should prepare the necessary papers and materials ahead of time – to ensure proper test administration in a timely manner. This includes making sure the CD player is ready to play each music listening piece. Ideally, the students should have a quiet room in which to take the test.

- Scoring of the Test

Section 1:

Fill-in-the-blank – Only music titles and genres on the list will be accepted.

There is only one correct answer for the title, and the genre. If one part of

the answer is wrong, but the other part is correct, the student will receive 0.5

points.

Short Answer – The test scorer will be given a list of the facts learned in

class about each of the six music genres. The students will have been made

aware of what is acceptable and what is not acceptable in this section

(during the test review). Only facts about each genre that were studied in

class are considered acceptable for this section. If the student does not use

complete sentences but gets the facts correct, 0.5 points will be deducted per

instance.

True/False – This section is very straightforward. An answer key will be

given to the test scorer.

Note – Teaching assistants may score all of section 1, except for the Short

Answer (which should be graded by the teacher). Section 2 may also be

graded by teaching assistants, but not section 3 or 4.

Section 2:

Multiple Choice Questions – Each written multiple choice question is worth

1 point each. An answer key will be provided. For the dictated questions 2

points will be given for the correct answer (in the answer key), and 2 points

will be given for writing the question and answers in an understandable way

(meaning: the scorer knows the student understood the dictation, but may

not have used the correct spelling for each word – spelling does not count as

part of the student’s score. Whoever scores this section of the test should be

able to understand the students’ dictation without having to read the version

the teacher dictated.) For the second half of the 4 points, 1 point will

be based on writing the question understandably, and 1 point will be based

on writing the four possible answers understandably.

Section 3:

Oral Summaries - Ideally grammar should be part of the score for this

section, but because the students are from various skill levels – and the basic

purpose of this test is retention of the course content – scoring is based on

content. Summaries must be memorized. The teacher should score the

summaries based on the rubric provided.

- Score Interpretation: The cut-off grade for passing this class is 70%. This

grade includes participation (10%), homework/activities (25%),

journals(15%), the final exam (25%), and the power point project (25%). A

final grade of 70% or better can be interpreted as “mastery” of the course

content (the higher the grade, the greater the mastery).

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download