University Of Southern California - Rossier School Of ...



University Of Southern California - Rossier School Of Education

EDUC 714 – Summer 2012 – Section 050-26520D

Measurement & Evaluation for Decision Making

Instructor: Robert G. Keim, DDS, EdD

Class Time and Place: Wednesdays 4-10:15pm, WPH 201

Office Hours: Wednesdays 2:30-3:30pm by appointment only

Office Location: Waite Phillips Hall (WPH) 600B

Contact Info: rkeim@usc.edu; voicemail 213-740-6864

Required Texts

Salkind, N. (2013). Tests and measurement for people who think they hate tests and measurement (2nd. Ed.). Thousand Oaks: Sage.

American Psychological Association (most recent version). Publication manual of the American Psychological Association. Washington, DC: Author.

Catalog Description: Survey of theories, concepts, and procedures for assessing intellective and non-intellective characteristics and for evaluating individual and program performances.

Course Objectives

1. Understand and apply measurement theory to research (in preparation for dissertation). Each student will develop analysis and evaluation skills in reading research on measurement and evaluation, and how to creatively use that research to develop or select a measurement variable in their dissertations or another study.

2. Understand and apply measurement theory to practice in educational setting. Each student will get practice providing the empirical basis for a specific theory, or practice by applying research through analysis of a measurement problem in an educational or training environment.

| |

Course Requirements

A. Class attendance and participation (10 points):

Attendance and participation are essential to the collaborative learning process. Students are expected to come to class having completed the assigned readings for the week and be prepared to discuss the material and participate in class activities. Due to the intensive nature of summer courses, absences of any kind may jeopardize a student’s ability to pass the course. Students are to notify the instructor via email BEFORE the absence and take responsibility for initiating and making arrangements for make-up work. Regardless of reason, more than 1 absence will result in a non-passing grade for the course.

B. Group Presentation (20 points)

Students will work together in small groups of 3 to present an overview and facilitate discussion of a current topic related to measurement and evaluation. Students will generate their own topics.

C. Project 1 - Instrument Development (20 points)

Each student is to develop an evaluation plan of a learning objective OR develop a specific assessment instrument, demonstrating knowledge of basic principles of assessment and measurement. Length: 6-8 pages. DUE 6/13 before midnight.

Detailed guidelines to follow.

D. Project 2 - Review of Literature on a Chosen Construct (50 points)

Each student is to write a 9-11 page review of literature on a construct of choice. The review should cover the theory behind the construct, how it has been operationalized, and a critique of the most commonly used measures of the construct. The intent of this exercise is to help students build toward their dissertations. DUE 6/26 before midnight.

Detailed guidelines to follow.

Grading Range

A 93-100 pts, A- 88-92 pts, B+ 83-87 pts, B 78-82 pts, B- 74-77 pts, C+ 70-73 pts.

Course Notes:

A. To prevent unnecessary distractions in class, ALL CELL PHONES ARE TO BE TURNED OFF (and not just in vibrate mode) and put away out of sight. Laptops are allowed only for the purpose of taking notes or other course-related activity. However, if it appears to the instructor that the computer is being used for other purposes, then the student will receive 1 email warning. In case of a second occurrence, the student will not be allowed to use the computer for the remainder of the course. These infringements will also affect the course participation grade.

B. This course will use Blackboard, but ALL DUE ASSIGNMENTS MUST BE SUBMITTED VIA EMAIL TO rkeim@usc.edu

C. All work must be completed according to APA style.

- On page 1, include the title and date of the assignment as listed in the syllabus, your institutional affiliation (i.e., University of Southern California, Rossier School of Education), the title of the course, name of the professor, your name, email, and the date.

- In the header, include your name, title and date of assignment, and page number.

- On the last page include the references.

D. Late papers will result in 1/3 letter grade reduction for each day late (e.g., A to A– to B+ to B to B– to C+).

E. Note on Incompletes

An incomplete can be assigned only if there is work not completed because of a documented illness or some other emergency. Arrangements for the incomplete and incomplete removal must be instituted by the student and agreed to by the instructor in advance and reported on the official “Incomplete (IN) completion form.” (USC Catalogue, Academic Standards)

F. Students with Disabilities

Any student requesting accommodations based due to a disability is required to register with the Disability Services and Programs office (DSP) each semester. A letter of verification for approved recommendations can be obtained through DSP. Please be sure the letter is delivered to the instructor as early in the semester as possible. DSP is located in STU 301 and is open 8:30 AM - 5:00 PM, M-F. The telephone number for DSP is (213) 740-0776. Email: ability@usc.edu; Website:

G. Academic Dishonesty ()

University policy on academic dishonesty states that plagiarism is the use of others’ ideas, writings, research, drawings, etc. in a paper without appropriate acknowledgment. Paraphrasing (putting the material in your own words, etc.) is usually acceptable as long as the original reference is cited, and you do not imply that the ideas or thoughts are your own. You must always use quotation marks and a citation, including page number, when using someone else’s words verbatim. Cheating is defined as the act of obtaining or attempting to obtain academic credit for work by the use of any dishonest, deceptive, or fraudulent means. For a comprehensive overview of the actions that constitute plagiarism and academic dishonesty, see the website above or SCampus. All cases of academic dishonesty will be pursued and actions will be taken in accordance with University guidelines.

Guidelines for Group Presentation

The purpose of this exercise is to help students gain personal in-depth knowledge in at least one content topic of interest. (20 points total)

1) Form groups of 3 and generate a list of possible topics related to class.

2) Email instructor before 2nd class with list of group members and 3 possible topics in order of preference. Topic must be approved by the instructor. (1 pt)

3) Research the topic utilizing assigned readings and some outside readings. (8 pts)

- outside readings limited to 5 NON WEB-BASED sources

- identify key issues or controversies related to topic

4) Prepare and make a PowerPoint presentation summarizing above (8 pts)

- distribute hardcopies of PowerPoint

- presentations will be distributed through the class starting the 3rd week

- date of presentation will be given when topic is assigned

- should be about 15-20 minutes total with at least 5 minutes for discussion.

5) Facilitate discussion about topic (3)

A peer assessment will be conducted to ensure equal contribution among group members. If a student is identified as not having contributed equally, then that person’s grade will be reduced by a letter grade for this exercise.

Guidelines for Project 1 – Developing an Assessment Measure

Each student is to write a 6-8 page description of the following: (20 pts)

Option A – Classroom/Student assessment

1. Define the context & target group for assessment. (2 pts)

2. Define the problem/construct/learning objective being evaluated. (2 pts)

3. Provide an evaluation plan that includes a list of assessments to be used. (5 pts)

4. Take ONE item from the list above and either develop or select an assessment instrument.

Describe how some of key principles of assessment are illustrated in the selected/developed instrument. (7 pts)

5. Summarize the reliability validity evidence available for this instrument. (4 pts)

Option B – Instrument Development

1. Define a construct to be measured, including intended population and setting. (2)

2. Briefly describe relevant theory and background related to construct (3)

2. Construct a 5-10 item instrument to measure that construct (4)

3. Describe and apply principles of instrument development in writing the 5-10 items. (4)

4. Describe the process of establishing reliability and validity. (7)

Penalty points deducted for

- Not following APA style (3)

- Poor editing (3)

- Length too long or too short (2)

Guidelines for Project 2 – Review of Literature on a Construct of Choice

Each student is to write a 9-11 page review of literature on a construct of interest for possible inclusion in their dissertation. The project is graded on 50 point system.

1) Students are to email the instructor with a list of 3 proposed constructs. This is to ensure that the construct is appropriate in scope for the exercise. (2 pts)

2) Provide a clear description of the construct and its significance. (5 pts)

3) Describe how the construct has been conceptualized and defined. Incorporate key theories and models related to construct. (10 pts)

4) Discuss key measurement issues related to the construct. (8 pts)

5) Provide a summary of the most commonly used measures of the construct with particular focus on psychometric properties (reliability and validity) along with appropriateness for use with specific demographic groups (adolescents, women, minorities, etc.). (20 pts)

6) Conclude with a recommendation for use a particular instrument(s) for your proposed target population. (5 pts)

7) Length should be 9-11 pages, including title page and references.

8) Utilize 10-15 articles, 80% should be primary sources and 20% can be secondary sources

Penalty points deducted for

- Not following APA style (5)

- Poor editing (5)

- References lacking or insufficient in quantity and quality (5)

- Length too long or too short (3)

Course Schedule

Week Date Topic Readings & Assignments

Section I – Measurement in research

W 1 5/16 Overview of course Salkind Chapters 1-2

History of testing Presentation topics due 5/20

Context of science and research

Levels of measurement

W2 5/23 Reliability and validity Salkind Chapters 3- 4

Statistical overview Review construct due 5/30

Article review

W3 5/30 Selecting and evaluating instruments Salkind Chapter 5

Four Levels of Evaluation

Concept review

Section II – Measurement and evaluation in education: The Tao and How of Testing

W4 6/6 Types of tests I Salkind Chapters 6 - 8

W5 6/13 Types of Tests 2 Salkind Chapters 9 - 12

Project 1 is due.

Group presentations

W6 6/20 Issues in Tests and Measurement Salkind Chapters 18 & 19

Group presentations

Project 2 is due June 26 before midnight.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download