Planning tests and assessments - UNCW Faculty and Staff ...
Planning Tests and Assessments
Four Steps in Planning an Assessment
1. Deciding its purpose
2. Developing test specifications
3. Selecting best item types
4. Preparing items
Step 1: Decide the Purpose
What location in instruction? Or, the role of time in assessment!
1. pre-testing
o readiness
i. limited in scope
ii. low difficulty level
iii. serve as basis of remedial work, adapting instruction
o pretest (placement)
i. items similar to outcome measure
ii. but not the same (like an alternative form)
2. during instruction
o formative
i. monitor learning progress
ii. detect learning errors
iii. feedback for teacher and students
iv. limited sample of learning outcomes
v. must assure that mix and difficulty of items sufficient
vi. try to use to make correction prescriptions (e.g., review for whole group, practice exercises for a few)
o diagnostic
i. enough items needed in each specific area
ii. items in one area should have slight variations
3. end of instruction
o mostly summative –broad coverage of objectives
o can be formative too
Step 2: Develop Test Specifications
• Why? Need good sample!
• How? Table of specifications (2-way chart, "blueprint")
1. Prepare list of learning objectives
2. outline instructional content
3. prepare 2-way chart
4. or, use alternative to 2-way chart when more appropriate
5. doublecheck sampling
Sample of a Content Domain (such as this course)
1. trends/controversies in assessment
2. interdependence of teaching, learning, and assessment
3. purposes and forms of classroom assessment
4. planning a classroom assessment (item types, table of specs)
5. item types (advantages and limitations)
6. strategies for writing good items
7. compiling and administering classroom assessments
8. evaluating and improving classroom assessments
9. grading and reporting systems
10. uses of standardized tests
11. interpreting standardized test scores
Sample Table of Specifications (For content from this course)
|Sample SLOs (you would typically have |Bloom Levels |
|more) | |
| |Remember |Understand |Apply |Analyze |Evaluate |Create |
|Identifies definition of key terms (e.g.,|X | | | | | |
|validity) | | | | | | |
|Identifies examples of threats to test | |X | | | | |
|reliability and validity | | | | | | |
|Selects best item type for given | | |X | | | |
|objectives | | | | | | |
|Compares the pros and cons of different | | | |X | | |
|kinds of tests for given purposes | | | | | | |
|Evaluates particular educational reforms | | | | |X | |
|(e.g., whether they will hurt or help | | | | | | |
|instruction) | | | | | | |
|Create a unit test | | | | | |X |
|Total number of items | | | | | | |
Spot the Poor Specific Learning Outcomes (use previous table of specifications)
Which entries are better or worse than others? Why? Improve the poor ones.
1. Knowledge
a. Knows correct definitions
b. Able to list major limitations of different types of items
2. Comprehension
a. Selects correct item type for learning outcome
b. Understands limitations of true-false items
c. Distinguishes poor true-false items from good ones
3. Application
a. Applies construction guidelines to a new content area
b. Creates a table of specifications
4. Analysis
a. Identifies flaws in poor items
b. Lists general and specific learning outcomes
5. Synthesis
a. Lists general and specific content areas
b. Provides weights for areas in table of specifications
6. Evaluation
a. Judges quality of procedure/product
b. Justifies product
c. Improves a product
Why are These Better Specific Learning Outcomes?
1. Knowledge
a. Selects correct definitions
b. Lists major limitations of different item types
2. Comprehension
a. Selects proper procedures for assessment purpose
b. Distinguishes poor procedures from good ones
c. Distinguishes poor decisions/products from good ones
3. Application
a. Applies construction guidelines to a new content area
4. Analysis
a. Identifies flaws in procedure/product
b. Lists major and specific content areas
c. Lists general and specific learning outcomes
5. Synthesis
a. Creates a component of the test
b. Provides weights for cells in table of specifications
6. Evaluation
a. Judges quality of procedure/product
b. Justifies product
c. Improves a product
Step 3: Select the Best Types of Items/Tasks
What types to choose from? Many!
1. objective--supply-type
a. short answer
b. completion
2. objective--selection-type
a. true-false
b. matching
c. multiple choice
3. essays
a. extended response
b. restricted response
4. performance-based
a. extended response
b. restricted response
Which type to use? The one that fits best!
1. most directly measures learning outcome
2. where not clear, use selection-type (more objective)
a. multiple choice best (less guessing, fewer clues)
b. matching only if items homogeneous
c. true-false only if only two possibilities
Strengths and Limitations of Objective vs. Essay/Performance
Objective Items
• Strengths
o Can have many items
o Highly structured
o Scoring quick, easy, accurate
• Limitations
o Cannot assess higher level skills (problem formulation, organization, creativity)
Essay/Performance Tasks
• Strengths
o Can assess higher level skills
o More realistic
• Limitations
o Inefficient for measuring knowledge
o Few items (poorer sampling)
o Time consuming
o Scoring difficult, unreliable
Step 4: Prepare Items/Tasks
Strategies to Measure the Domain Well—Reliably and Validly
1. specifying more precise learning outcomes leads to better-fitting items
2. use 2-way table to assure good sampling of complex skills
3. use enough items for reliable measurement of each objective
o number depends on purpose, task type, age
o if performance-based tasks, use fewer but test more often
4. keep in mind how good assessment can improve (not just measure) learning
o signals learning priorities to students
o clarifies teaching goals for teacher
o if perceived as fair and useful
Strategies to Avoid Contamination
1. eliminate barriers that lead good students to get the item wrong
2. don’t provide clues that help poor students get the item correct
General Suggestions for Item Writing
1. use table of specifications as guide
2. write more items than needed
3. write well in advance of testing date
4. task to be performed is clear, unambiguous, unbiased, and calls forth the intended outcome
5. use appropriate reading level (don’t be testing for ancillary skills)
6. write so that items provide no clues (minimize value of "test-taking skills")
a. a/an
b. avoid specific determiners (always, never, etc.)
c. don’t use more detailed, longer, or textbook language for correct answers
d. don’t have answers in an identifiable pattern
7. write so that item provides no clues to other items
8. seeming clues should lead away from the correct answer
9. experts would agree on the answer
10. if item revised, recheck its relevance
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
Related searches
- free grammar tests and answers
- printable personality tests and quizzes
- funny tests and quizzes
- fun online tests and quizzes
- free online tests and quizzes
- biology practice tests and answers
- general chemistry tests and answers
- percentage tests and answers
- free psychology tests and quizzes
- cardiology tests and procedures
- nephrology tests and procedures
- neurological diagnostic tests and procedures