Evaluating Special Education Preservice Programs: Resource ...

Center to Improve Project Performance

Evaluating Special Education Programs

Resource Toolkit

Authors Jill Lammert, Westat Sarah Heinemeier, Compass Evaluation & Research Jennifer M. Schaaf, Westat

Thomas A. Fiore, Westat Bethany Howell, Compass Evaluation & Research

November 2016

See Evaluating Special Education Preservice Programs Resource Toolkit (Lammert, Heinemeier, Schaaf & Fiore, 2016) for guidance specific to Personnel Development Project Evaluations.

For easy reference, the Quick Start Guide on page iv lists a number of questions an evaluator may ask regarding the design or conduct of a project evaluation, with links to the specific section in the Toolkit where related information may be found. A Table of Contents with hyperlinks to the different sections is on page v.

Prepared for: Office of Special Education Programs U.S. Department of Education Washington, D.C.

Prepared by: Westat 1600 Research Boulevard Rockville, Maryland 20850-3129 (301) 251-1500

About this Toolkit

This Toolkit was developed as part of the Center to Improve Project Performance (CIPP) operated by Westat for the U.S. Department of Education, Office of Special Education Programs (OSEP). The authors thank the OSEP, Westat, and CEEDAR (Collaboration for Effective Educator Development, Accountability, and Reform) staff who provided input.

Suggested Citation:

Lammert, J. D., Heinemeier, S., Schaaf, J. M., Fiore, T.A., & Howell, B. (2016). Evaluating special education programs: Resource Toolkit. Rockville, MD: Westat.

The Center to Improve Project Performance has been funded with Federal funds from the U.S. Department of Education, Office of Special Education Programs, under contract number ED-OSE-13C-0049. The project officer is Dr. Patricia Gonzalez. The content of this publication does not necessarily reflect the views or policies of the U.S. Department of Education nor does mention of

trade names, commercial products, or organizations imply endorsement by the U.S. government.

Evaluation Resource Toolkit

i

Overview of the Center to Improve Project Performance

First formed in 2008, CIPP's overall mission is to advance the rigor and objectivity of evaluations conducted by or for OSEP-funded projects so that the results of these evaluations can be used by projects to improve their performance and used by OSEP for future funding decisions, strategic planning, and program performance measurement.

CIPP provides evaluation support, oversight, and technical assistance (TA) to grantees of OSEP projects. CIPP staff work with project and OSEP staff to refine project logic models and develop evaluations. Based on the evaluation design and plan, CIPP staff have overseen evaluation activities and provided TA, as needed, to grantees including selecting samples; developing draft instruments; monitoring data collection and performing reliability checks; analyzing study data; providing accurate descriptions of the methods and valid interpretations of findings; and organizing, reviewing, and editing project evaluation reports. In addition to providing TA to OSEP-funded projects on request, CIPP staff prepare a variety of TA products and briefs focused on evaluation issues, and deliver presentations on evaluation through webinars and conferences.

Contact Information: 1-888-843-4101

Thomas Fiore, CIPP Project Director Westat ThomasFiore@

Jill Lammert, CIPP Deputy Project Director Westat JillLammert@

Patricia Gonzalez, Project Officer Office of Special Education Programs U.S. Department of Education Patricia.Gonzalez@

Evaluation Resource Toolkit

ii

Evaluation Resource Toolkit

iii

Quick Start Guide

Instructions: Click on a specific Toolkit Section to go to that section.

First Things to Consider for Evaluations

What are the benefits of evaluations? What components do evaluations typically contain? How do I budget for my evaluation? What should I think about if I am considering working with a third-party evaluator?

Creating a Theory of Change and Logic Model

How do I identify the needs my project will address? What outcomes should I focus on? How do I create a theory of change? How do I develop a high-quality logic model? Is there a sample logic model for reference? Is there a logic model template that I can use? How do I develop evaluation questions?

Creating an Evaluation Plan

What elements should my evaluation plan contain? Is there an evaluation plan template that I can use? Is there a sample evaluation plan?

Selecting a Sample & Choosing an Evaluation Design

How do I select my study sample? What evaluation design should I use? How can I use goal attainment scaling to measure my project outcomes How can I use single case designs in my project?

Planning for Data Collection

What should I consider when planning for data collection? Do I need to get approval from an Institutional Review Board (IRB)? What do I need to consider when planning data collection in schools? Are there sample district, school and individual notification letters that I can use?

How do I get consent from participants? Are there sample consent forms that I can use?

How do I obtain access to student data?

Selecting Data Collection Methods

What should I consider when creating a new survey or adapting an existing one? How do I get a good response rate to my surveys? What should I consider when I want to include observations in my data collection?

Toolkit Section

1.1 1.2 1.3; Appendix A.1 1.4

2.1 2.1.3 2.2 2.3 Figure 2 Appendix A.3 2.4

2.5 Appendix A.4 Appendix A.5

2.5.3; 4.2 2.5.1; 4.1 4.3.5 4.1.2

2.5.4 3.1.1 3.1.2 Appendix C.1, C.2, C.3, & C.4 3.2.2 Appendix C.5, C.6, C.7, & C.8 3.1.3

4.3.1 4.3.1.4.1 4.3.2

Evaluation Resource Toolkit

iv

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download