SEPPS Excel Scoring Program - IDEA Data



Instructions for Use of the Schools’ Efforts to Partner with Parents Scale (SEPPS) Excel Scoring ProgramBatya Elbaum, Ph.D. and Seniz Celimli, Ph.D., University of MiamiWilliam P. Fisher, Jr., Ph.D., University of California BerkeleyINTRODUCTIONThe purpose of this document is to provide instructions for use of an Excel-based scoring program to derive individual measures (popularly referred to as scores) from parents’ responses to the School’s Efforts to Partner with Parents Scale (SEPPS; Elbaum, Fisher, & Coulter, 2011). The SEPPS was developed under the auspices of the National Center for Special Education Accountability Monitoring (NCSEAM) for states’ use in addressing Indicator 8 of the Part B State Performance Plan (SPP).Once all responses from a state or local sample are entered into the first worksheet of the SEPPS Excel Scoring Program, the program will calculate a measure for each respondent. It will also calculate the percent on Indicator 8 for the state or local program, using the NCSEAM-recommended standard (Elbaum et al., 2011) and the 95% confidence interval around that percent.Because the SEPPS was built on the Rasch measurement framework, the developers of the instrument recommended that responses to state or locally administered surveys incorporating the SEPPS be analyzed using the same measurement framework. The estimation of individual measures is usually accomplished through a specialized software program such as WINSTEPS (Linacre, 2012). Use of such software also allows analysts to confirm that item difficulties have remained the same over time and that the pattern of responses from each new sample conforms reasonably well to expectations.The observed stability of the SEPPS across states and over time suggests that use of a scoring algorithm based on raw scores should yield measures that are very close, if not identical, to those that would be generated through existing software programs. To examine the similarity of measures calculated through WINSTEPS and through the SEPPS Excel Scoring Program, a sample data set consisting of 11,179 cases was analyzed. The sample data were from 72 different local educational agencies (LEAs) having between 3 and 1903 respondents. For all cases combined, mean measures produced through WINSTEPS and the SEPPS Excel Scoring Program (using a scale of 0-1000) were virtually identical, M = 571 vs. M = 569, respectively, with standard errors of these means equal to 1.59 and 1.59, respectively. Within-LEA differences between mean measures derived from a WINSTEPS analysis vs. the SEPPS Excel Scoring Program ranged from 0.05 to 20.03, with a mean difference of 3.44 and a standard deviation of 3.89; the standard errors of the mean differences ranged from 4.99 to 205.45 (for the sample with n = 3). Within-LEA differences in the percent on Indicator 8 ranged from 0.0% to 4.2%, with a mean difference of 0.5% and a standard deviation of 0.1%. Differences in both the measures and percentages obtained through use of the two different programs were well within the 95% confidence intervals for the individual LEAs; these confidence intervals ranged from +/-2.5% to +/-36.6%.The comparative results reported above suggest that the SEPPS Excel Scoring Program represents a reasonable alternative method of estimating SEPPS measures and percentages on Part B SPP Indicator 8. It should be borne in mind, however, that the extent to which responses from any given sample of parents conform to the Rasch measurement model cannot be ascertained through use of the SEPPS Excel Scoring Program. Some data may not conform to the measurement model, owing to the particular experiences of the sample of parents and/or to the existence of particular response sets in this sample; only an in-depth measurement analysis can ascertain fit to the model. Accordingly, neither the Office of Special Education Programs, NCSEAM, or the developers of the SEPPS can be held accountable either for the results of analyses that utilize the SEPPS Excel Scoring Program or for inferences that might be drawn from these results. Consistent with policy related to the SPP, it is the sole responsibility of states to conduct their own data quality checks and to ensure the reliability and validity of the results that are reported in the Annual Performance Report.The following points should also be noted by potential users of the SEPPS Excel Scoring Program:The program can accommodate missing responses. It is not necessary for a parent to have responded to every survey item in order for the program to calculate a measure for that parent.Many states’ parent surveys use the same 25 items recommended by NCSEAM in 2006. These 25 items are a subset of the 78 SEPPS items that were included in the initial scale validation (Elbaum et al., 2011). The SEPPS Excel Scoring Program was therefore designed to analyze responses to the 25 most commonly used SEPPS items. Responses to other items that may have been administered are not utilized by the program, but as long as there are responses to most of the 25 included items, the measure calculated by the program will approximate the measure that would be obtained if the omitted items had been included. Confidence in the approximation obtained will decline (i.e., the confidence intervals will increase) as the number of the original 25 items included decreases.The program does not analyze data from other Part B NCSEAM scales.ReferencesElbaum, B., Fisher, W. P. Jr., & Coulter, W. A. (2011). Measuring schools’ efforts to partner with parents of children served under IDEA: Scaling and standard setting for accountability reporting. Journal of Applied Measurement, 12 (3), 261-278.Linacre, J. M. (2012). Winsteps? (Version 3.74.0) [Computer Software]. Beaverton, Oregon: . Available at to using the SEPPS Excel Scoring Program, please note:To use the SEPPS Excel Scoring Program, you must either (a) enter item-level data directly into the Excel scoring spreadsheet or (b) copy the data from another source file. What follows are instructions for importing item-level SEPPS data that are in an existing Excel spreadsheet. We distinguish between the two Excel files by referring to the existing SEPPS data file as your data file and the Excel file that runs the scoring program as the Excel Scoring Program file. The SEPPS Excel Scoring Program has been designed for the response format of (1) Very Strongly Disagree; (2) Strongly Disagree; (3) Disagree; (4) Agree; (5) Strongly Agree; and (6) Very Strongly Agree. If the response options in your survey are presented in the reverse order, make sure to recode them before you start using the SEPPS Excel Scoring Program. Always make a copy of your data file before you start working and continue working with a copy of your original data file.If your data file has more than 1000 cases, follow the instructions in Appendix A first.STEP 1. Familiarize yourself with the Excel Scoring Program fileThe Excel Scoring Program file consists of the following worksheets (all except the first have built-in formulas):Responses – this is the sheet into which you will enter or import your item-level SEPPS data. Scores – this sheet converts the item-level data in the Responses sheet into the scores that will be used in further analyses.Missing Score Imputation – this sheet imputes scores for cells with missing values.Summary Scores – this sheet calculates (a) the total summed (converted) score, (b) the measure, and (c) the error (in scale points) for each record. It also calculates the percent of measures in the sample that are at or above 600, the standard that stakeholders convened by NCSEAM recommended be applied to the measures to determine a state’s percent on the indicator.Score Lookup Table – this is a background table needed for calculations in the Scores worksheet.Summary Score Lookup Table – this is a background table needed for calculations in the Summary Scores worksheet.Confidence Interval – this sheet includes background formulas to calculate a 95% confidence interval for the Indicator 8 percentage.STEP 2. Create an ID field and sort the records in your data fileThe records in your data file may already have a unique ID number. If not, you should assign each record a unique number, for example, by creating a column in which you assign consecutive numbers to each row. If you don’t have a unique ID number column: To create an ID field, go to the Responses worksheet and find the first column called ‘ID number.’ Number the first three cells starting from the first cell (i.e., put 1, 2 and 3). Then select all three cells at once, and then you will see the small black square (called filled handle) in the lower-right corner of the selection. When you point your mouse to the fill handle, the pointer will change to a black cross. To fill in the rest of the column, press your mouse in the black cross and drag it down. This action will add consecutive numbers to the column.If you do have a unique ID number column: To sort by the column that contains either your existing unique ID or your assigned record number, bring your mouse on top of the label cell of the column. Your curser will turn into a downward-pointing arrow. When you click on it, the entire column will be selected. Go to the Sort & Filter option of the editing tab located in the Excel Home menu bar. Select ‘sort smallest to largest.’ When prompted, select ‘expand the selection’. If the Editing tab is not on your home screen, select the ‘Data’ tab at the top of the screen, and choose ‘Sort’.After sorting the ID number column that you have in your data file, you should copy and paste this column into the Excel Scoring Program file. To do this, go to your data base, select the first cell right under the ID number label then select the rest of the numbers by pressing ‘Shift’ and ‘Page down’ keys on your keyboard at the same time. Once you have selected the entire column (only the ID numbers excluding the column label cell), copy it by pressing the ‘Ctrl’ and ‘C’ keys at the same time. Then go to the first cell under the ‘ID number’ label in the Excel Scoring Program file and paste what you copied by clicking in the first cell in the ‘ID number’ column and pressing ‘Ctrl’ and ‘V’ at the same time.STEP 3. Tell the Excel Scoring Program file where each SEPPS item is located in your fileThe survey item columns in the Excel Scoring Program file are in a particular order. This is NOT necessarily the same order in which the survey items appear on states’ survey forms. The purpose of this step, then, is to make sure that when you copy your data into the Excel Scoring Program file, the item-level data will be in the correct columns. To complete this step, you first need to code your survey item locations and then re-order your items.Code your survey item locationsInsert one empty row in your data file by selecting the entire first row and then right clicking on it. Select that row by clicking on the label cell of the first row at the left of the screen; then right click on it. A window with several options will open. Click on ‘insert’; an empty row will be created as the first row.Start matching survey items in your data file by locating them one by one in the ‘Responses’ worksheet of the Excel Scoring Program file. Once you locate one, note the survey item’s number in the Excel Scoring Program file and enter it into the corresponding cell in the first row that you created in your data file.For example:Let’s say the first survey item in your data file is: “1. I am considered an equal partner with teachers and other professionals in planning my child's program.”Go to the Excel Scoring Program file and locate the same survey item: It is survey item 20 in the Excel Scoring Program file (Q20. I am considered an equal partner with teachers and other professionals in planning my child's program.)Go back to your data file and put the number ‘20’ in the cell of the new first row, right above the survey item that you plete steps (1) through (3) for all of the survey items in your data file.What if your data file has non-matching survey items and/or fields?If you have additional survey items and/or fields in your data file that do not exist in the Excel Scoring Program file, fill in the first cell in their column with a zero.If you have missing survey items in your data file (i.e., survey items that are in the Excel Scoring Program file but are not in your data file), nothing further needs to be done.Re-order your survey itemsSelect your entire data file by clicking on the little triangular shaped point at the top-left corner of your dataset.Find the ‘Sort & Filter’ option on the Home tab under the ‘editing’ field. Click on it and select the ‘custom sort’ option. ‘Sort’ window will open. Click on the ‘options’ button and select the orientation ‘sort left to right’. Click OK. The sort window should still be open. Click on the dropdown menu of the ‘sort by’ section, select ‘Row 1’, make sure that the option is selected for ‘smallest to largest’, then click OK.Now all of the survey items in your data file should be ordered identically to the survey items in the Excel Scoring Program file.STEP 4. Ensure that your data include numerical values only (i.e., 1 through 6)Since the Excel Scoring Program file is based on calculations and includes formulas, your data file should include survey item response data only in number format, coded from 1 to 6. Any survey item cells with text in them (e.g., N/A) or with numbers other than 1 through 6 (e.g., 0, 9, or 99) need to be replaced with empty cells in your data file. Follow these steps in your data file:Identify the cell values that are not in the required format. Let’s say you have N/A (standing for ‘not applicable’) as an item response for some or all survey items.Invoke the Find command by pressing the ‘Ctrl’ and ‘F’ keys at the same time. The ‘Find and Replace’ window will open. This window has two tabs: ‘Find’ and ‘Replace’.Click on the ‘Replace’ tab. Type your text value (such as N/A) in the ‘Find what’ field. Leave the ‘Replace with’ field empty. Then select the button ‘Replace All’ on the bottom line of that window.A new window will pop up saying “Excel has completed its search and has made XX replacements.” The higher the number of replacements, the more time it will take to run. Be patient to see this confirmation window!Then click OK in this new pop-up window. You should still have the ‘Find and Replace’ window open in front of you.Apply the same procedures (steps a through e) for any text or cell value other than 1 through 6 entered in cells as a survey item response.When you are done, close the ‘Find and Replace’ window.NOTE: At the end of this step you should have only numbers (i.e., 1 through 6) and empty cells in the survey item columns. There should be no remaining text values.STEP 5. Copy your data into the Excel Scoring Program fileNow that all of the survey items in your data file have been ordered, it is time to transfer the responses in your data file into the Excel Scoring Program file. Follow these steps:In your data file:Go to the first cell under your first survey item and click on it. Then press the ‘Shift’ key and highlight the rest of the survey items by first pressing the ‘End’ key followed by the ‘right arrow’ key on your keyboard. After highlighting the cells under your survey item label row, keep holding the ‘Shift’ key while you press ‘Ctrl’ and ‘End’ at the same time. Click ‘Ctrl’ and ‘C’ at the same time to copy your selected data.In the Excel Scoring Program file:Go to the Responses worksheet and select the first cell under the first survey item. Invoke the Paste command by pressing ‘Ctrl’ and ‘V’ at the same time. This action will paste your responses into the Excel Scoring Program file.STEP 6. View the results in the Summary Scores worksheet. Once you have transferred your responses into the Excel Scoring Program file, go to the Summary Scores worksheet. This sheet will give you the total summed (converted) score, the measure, and the error (in scale points) for each record. It also gives the percent of measures in the sample that are at or above 600, the standard that stakeholders convened by NCSEAM recommended be applied to the Rasch measures to determine a state’s percent on the indicator.STEP 7. Copy the summary score calculations into your data fileIf you have fields other than survey item fields in your data file (e.g., student’s race/ethnicity, school, LEA, etc.) you can merge the individual Rasch measures that you now have in the Excel Scoring Program file back into your data file to conduct analyses of the data by these variables. To complete this final step:First, you need to delete the row you created in your data file. Select that row in the worksheet by clicking on label cell of the first row (at the far left of your screen). To delete, right click and choose ‘delete.’ Go to the Summary Scores worksheet in the Excel Scoring Program file. Select all columns in the worksheet (except the ID number column) by bringing your mouse on top of the label cell of the first column next to the ID number column. Your cursor will turn into a downward-pointing arrow. When you click on it, the entire column will be selected. Then select the rest of the columns by pressing the right arrow key on your keyboard while pressing the ‘Shift’ key. Once you select all the columns (except the ID number column), release ‘Shift’ + ‘right arrow’ keys, then copy selected columns by pressing ‘Ctrl’ and ‘C’ key at the same.Go to your data file. Bring your mouse over the first empty cell in the top row next to the last survey item, click on it and right click while you are on it. A window will open with several options. Select the one called ‘paste special.’ The ‘Paste special’ window will open. Select the ‘values’ option under the paste section, then click OK. You now have all the measures in your data file.Appendix AHow to Expand Formulas in the Excel Scoring Program file to Accommodate Data for More than 1000 RespondentsThe Excel Scoring Program file has built-in formulas for 1000 rows of data. If you have more than 1000 cases, you need to go through the following steps:Increasing the consecutive numbers in the first column of the Scores worksheet: Go to the Scores worksheet. Find the first column called ‘row number.’ Scroll down and find the last numbered cell in that column. Select the last two cells in that column, then you will see the small black square (called ‘fill handle’) in the lower right-hand corner of the selection. When you point your mouse to the fill handle, the pointer will change to a black cross. To fill in the rest of the column, left click your mouse in the black cross and drag it down. This action will add as many cells as you need to the column.Expanding formulas in the Scores worksheet:In the Scores worksheet, go to cell B-1001, and then press the ‘Shift’ key and highlight the consecutive cells on the same row by first pressing the ‘End’ key followed by the ‘right arrow’ key on your keyboard. Point your mouse to the fill handle; the pointer will change to a black cross. To fill in the rest of the rows with formulas, double left click on the black cross. This action will fill in the rows you need with the formulas.Expanding formulas in the Missing Score Imputation worksheet:Go to the Missing Score Imputation worksheet. Find the last numbered cell in the first column called ‘ID number’ and select it. Then press the ‘Shift’ key and highlight the consecutive cells on the same row by first pressing the ‘End’ key followed by the ‘right arrow’ key on your keyboard. Point your mouse to the fill handle; the pointer will change to a black cross. To fill in the rest of the rows with formulas, double left click on the black cross. This action will fill in the rows you need with the formulas.Expanding formulas in the Summary Scores worksheet:Go to the Summary Score worksheet. Find the last numbered cell in the first column called ‘ID number’ and select it. Then press the ‘Shift’ key and highlight the consecutive cells on the same row by first pressing the ‘End’ key followed by the ‘right arrow’ key on your keyboard. Point your mouse to the fill handle; the pointer will change to a black cross. To fill in the rest of the rows with formulas, double left click on the black cross. This action will fill in the rows you need with the formulas. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download