General Guidelines - Milwaukee Recreation: Milwaukee ...



Outcome Data Collection Guide: Pre-Post Assessment VersionUpdated: September 2019Instrument Contents TOC \o "1-3" \h \z \u General Guidelines PAGEREF _Toc524694864 \h 1Choosing a method PAGEREF _Toc524694865 \h 1Timing and who to assess PAGEREF _Toc524694866 \h 1Guidelines for administering surveys or short answer questions PAGEREF _Toc524694867 \h 2Guidelines for Observations PAGEREF _Toc524694868 \h 4Passive Parental Consent PAGEREF _Toc524694869 \h 5Data Tracking Spreadsheets PAGEREF _Toc524694870 \h 6Increase in self-discipline and self-direction PAGEREF _Toc524694871 \h 7Increased understanding of one’s responsibility to and impact on the wider community PAGEREF _Toc524694872 \h 12Increased ability to communicate one’s self through a specific medium (such as visual art, writing, performance, etc.) PAGEREF _Toc524694873 \h 14Increased understanding of differing cultures and how they connect to one’s own PAGEREF _Toc524694874 \h 16Increase in self-confidence to perform a task. PAGEREF _Toc524694875 \h 18Increased ability to reflect upon one’s own abilities and performance PAGEREF _Toc524694876 \h 21Increased cooperative and collaborative behaviors to successfully complete a shared task PAGEREF _Toc524694877 \h 24Frequently Asked Questions PAGEREF _Toc524694878 \h 26Wording of surveys PAGEREF _Toc524694879 \h 26Scale items PAGEREF _Toc524694880 \h 26Youth having a “bad day” PAGEREF _Toc524694881 \h 26We have existing outcome measures PAGEREF _Toc524694882 \h 26I saw change in my students last year, but surveys didn’t pick it up PAGEREF _Toc524694883 \h 26Multiple sessions and/or programs PAGEREF _Toc524694884 \h 27Matching students on surveys PAGEREF _Toc524694885 \h 27Outcome achievement PAGEREF _Toc524694886 \h 27Ceiling effect PAGEREF _Toc524694887 \h 27Appendix A: Passive Parent/Guardian Permission Form (English) PAGEREF _Toc524694888 \h 28Appendix B: Passive Parent/Guardian Permission Form (Spanish) PAGEREF _Toc524694889 \h 29General GuidelinesThe Center for Urban Initiatives and Research at UWM developed indicators and measurement tools to help assure consistent program outcome measurement across Partnership for the Arts & Humanities programs. The following guide provides suggestions for facilitating appropriate, simple, and reliable data collection.Choosing a methodAll instruments are intended to assess each student individually.It is strongly recommended that organizations choose only one method to measure outcome indicators. Three methods for collecting data are provided for each outcome:Close-ended survey questionsAppropriate for intermediate and advanced readers (approximately 7th grade and above).To be administered via paper and pencil.Open-ended short answersCan be administered via paper and pencil or through oral interviews, where the interviewer takes careful notes.Appropriate for approximately grade 3 and above (interviews) or 7th grade and above (paper and pencil).ObservationsAppropriate primarily for younger students.Some items may require brief inquiry, but most do not.Timing and who to assessThe pre-test should be completed near the beginning of your program, but not during the first few days while everyone is getting settled and acclimated. You should have distributed Passive Parent/Guardian Permission Forms to all program participant’s parents or guardians in advance of administering the pre-test. See Page 9 for more information. The post-test should be completed as near the end of the program as possible.Only analyze results from those students who were in your program long enough to reasonably expect they would achieve the outcome (e.g., 50% or more attendance).You might consider only doing post-tests with this group and not assessing the other students.Guidelines for administering surveys or short answer questionsCreate a code in collaboration with program participants that will allow you to match each individual pre-test to the post-test for surveys or short answer questions. This code will need be entered in the “Student ID” line of the survey or short-answer question tool. You could try a combination of the following:First initialLast initialMonth bornDay born“B” for boy or “G” for girlYou may modify the survey or short answer questions:Specifics may need to be added about the art medium or to be more relevant (e.g., change generic “task” to the relevant activity).Some questions may not be applicable.Some vocabulary might not be appropriate for your students.You may add other questions to measure other outcomes your organization is interested in.You may modify survey response categories to be more age-appropriate. Some examples include:Strongly Disagree, Disagree, Agree, Strongly AgreeNO!, no, yes, YES!Not like me, A little like me, Usually like me, A lot like me If you have questions regarding modifications to the tools provided in this guide, please contact the Supervisor of the Partnership for the Arts & Humanities. For surveys or short-answer instruments, the following procedure is recommended:Schedule time for completing the instrument during programming.Gather students in one area. Provide an alternative activity for students whose parents have requested their students not be involved and/or for when students are finished.Tell students that you are collecting information that will be used to help explain the impact of arts programming on students in the organization’s rm students that their participation is voluntary and in no way affects their participation in the program. Anyone who does not want to answer a particular question may skip that question, and those students who do not want to complete the instrument may turn in a blank document.Explain that they should not write their names anywhere on the instrument. Explain how to create a unique identifier using the formula (see above). Go over each part of the identifier together to help assure accuracy and completion.Then explain that there are no “right” answers, only answers that are close to how they feel and think. Ask them to choose the answers that are most like them.Stay in an area where you cannot see responses so students don’t feel any pressure to answer a certain way.Providing help with questions:Depending on reading ability level or English comprehension, some may need help with understanding words. You may assist a youth by explaining what a word means, providing examples or using a synonym. Sometimes youth will not know what to include or not include in their response, which can make it harder to give help. You may have to put it back on the youth to decide if they want to respond or not.If a student is requiring extensive help, it may not be appropriate to continue. If the amount of help you need to give is enough that the questions are being altered, their responses will not be a relevant comparison. Have students place completed (or blank) instruments in an envelope at the front of the room.Remember to thank them when data collection is complete.Guidelines for ObservationsObservations are labor-intensive for staff, as you will need time to observe each student individually. Keep this in mind when scheduling observations.Do not refer to pre-test observations when completing the post-test. We suggest you not even have the pre-test scores with you.Observation instruments use frequency. Guidelines for frequency:Never or seldom = less than 10% of the time.Occasionally = between 10% and 25% of the time.Often = between 26% and 75% of the time.Almost always = more than 75% of the time.Remember, the Supervisor of the Partnership for the Arts & Humanities and the MPS research team are available to provide guidance as you facilitate data collection!Passive Parental Consent Organizations are required to send a Passive Parent/Guardian Permission Form home with each program participant before any survey instruments are administered. The form instructs parents to sign and return the form only if they do not want their child to participate in the evaluation process. The form, which is available in English and Spanish, can be found at the end of this guide (Appendix A and Appendix B, respectively) or here: rec/Programs/Arts-Humanities.htm.Data Tracking Spreadsheets Along with the tools presented in this guide, the team at the Center for Urban Initiatives and Research created a set of spreadsheets (one for each outcome) that can be used to track collected data. The spreadsheets have a section for pre- and post-test scores and will automatically analyze the submitted data. These spreadsheets will be useful as organizations get ready to complete Final Reports, which asks about the percent of program participants that achieved the selected outcome. The data tracking spreadsheets are available here: rec/Programs/Arts-Humanities.htm.Increase in self-discipline and self-directionNote: Self-discipline and self-direction are closely related, but not necessarily co-emergent. Arts and humanities program providers will need to determine if one or both are appropriate, given your program activities and emphasis.Survey ItemsSelf-disciplineStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meWhen problems get in the way, I still stick to my goals.○○○○When I make plans, I follow through on them. ○○○○I have problems focusing on completing tasks. ○○○○I put off doing things I WANT to do in order to finish what I NEED to get done. ○○○○Scoring—Circle the number and add (student should not see this)When problems get in the way, I still stick to my goals Not like me at all1 A little like me2 Mostly like me3 A lot like me4When I make plans, I follow through on them. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I have problems focusing on completing tasks. Not like me at all4 A little like me3 Mostly like me2 A lot like me1I put off doing things I WANT to do in order to finish what I NEED to get done. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Total: _________Self-directionStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meI have goals and plans for the future.○○○○I am good at planning ahead.○○○○I can solve most problems on my own.○○○○I am good at organizing my time.○○○○Scoring—Circle the number and add (student should not see this)I have goals and plans for the future. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I am good at planning ahead. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I can solve most problems on my own. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I am good at organizing my time. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Total: _________Short-Answer Questions (written or oral)Self-disciplineStudent ID: ______________Program/Session Name: _________________Date: ______________Think about a project or activity you recently completed. Name the steps you had to take to complete it, any challenges you faced, and how you worked through the challenges.Scoring—Circle the rating that best fits the response.(student should not see this)Cannot name any steps or challenges.1Can name at least one step or challenge.2Can name at least one step and one challenge and how it was overcome.3Can name more than one step and at least one challenge and how it was overcome.4Self-directionStudent ID: ______________Program/Session Name: _________________Date: ______________Think of a goal you have for your future—it could be a goal for school, in sports or arts, or another goal you have for yourself. Name the goal you have in mind and the steps you need to take to reach your goal.Scoring—Circle the rating that best fits the response.(student should not see this)Cannot name a realistic goal.1Can name a goal, but no steps.2Can name a goal and one step.3Can name a goal and more than one step to achieve it.4Observations Self-disciplineCircle One Pre-test Post-testStudent NameProgram/Session NameDateCheck one box per row for each behavior observed on the date above. Never/ Seldom Occasionally Often Almost always1. Stays on task.□□□□2. Completes tasks. □□□□3. Has disruptive behavior.□□□□Scoring—Circle the number and add1. Stays on task. Never/ Seldom1 Occasionally2 Often3 Almost always42. Completes tasks. Never/ Seldom1 Occasionally2 Often3 Almost always43. Has disruptive behavior. Never/ Seldom4 Occasionally3 Often2 Almost always1Total: _________Self-directionCircle One Pre-test Post-testStudent NameProgram/Session NameDateCheck one box for each behavior observed on the date above. Never/ Seldom Occasionally Often Almost always1. Starts projects without needing direction or very little direction. □□□□2. Tries to solve problems without asking for help. □□□□Scoring—Circle the number and add1. Starts projects without needing direction or very little direction. Never/ Seldom1 Occasionally2 Often3 Almost always42. Tries to solve problems without asking for help. Never/ Seldom1 Occasionally2 Often3 Almost always4Total: _________Increased understanding of one’s responsibility to and impact on the wider communitySurvey ItemsStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meI speak up for other people when they need help.○○○○I believe that I can make a difference in my neighborhood. ○○○○I understand how I can contribute to my neighborhood. ○○○○I feel connected to my neighborhood. ○○○○I am important to my neighborhood. ○○○○Being actively involved in neighborhood issues is everyone’s responsibility, including mine. ○○○○Scoring—Circle the number and add (student should not see this)I speak up for other people when they need help. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I believe that I can make a difference in my neighborhood. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I understand how I can contribute to my neighborhood. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I feel connected to my neighborhood. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I am important to my neighborhood. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Being actively involved in neighborhood issues is everyone’s responsibility, including mine. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Total: _________Short-Answer Questions (written or oral)Student ID: ______________Program/Session Name: _________________Date: ______________Name an issue you feel strongly about that impacts others. What could you do to make the situation better?Scoring—Circle the rating that best fits the response.(student should not see this)Cannot name an issue.1Can name an issue but no solution OR can name an issue but not how it impacts others.2Can name an issue and either how it impacts others OR at least one solution.3Can name an issue, how it impacts others, and at least one solution.4Student ID: ______________Program/Session Name: _________________Date: ______________Do you think the decisions you make and actions you take impact others? Why or why not?Scoring—Circle the rating that best fits the response.(student should not see this)Does not think decisions impact others.1Thinks decisions impact others but cannot say why.2Thinks decisions impact others and can give one reason or example.3Thinks decisions impact others and can give more than one reason or example.4Observations Not available. Increased ability to communicate one’s self through a specific medium (such as visual art, writing, performance, etc.)Note that the specific art medium needs to be specified where [brackets] indicate.Survey ItemsStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meI can express what I’m thinking through [art medium].○○○○When I see someone’s [art project], it tells me a lot about them. ○○○○When I’m doing [art project], I try to make it look like something else I’ve seen before.○○○○Scoring—Circle the number and add (student should not see this)I can express what I’m thinking through [art medium]. Not like me at all1 A little like me2 Mostly like me3 A lot like me4When I see someone’s [art project], it tells me a lot about them. Not like me at all1 A little like me2 Mostly like me3 A lot like me4When I’m doing [art project], I try to make it look like something else I’ve seen before. Not like me at all4 A little like me3 Mostly like me2 A lot like me1Total: _________Short-Answer Questions (written or oral)Student ID: ______________Program/Session Name: _________________Date: ______________If you were going to do a [art medium] project, what would you do and why?Scoring—Circle the rating that best fits the response.(student should not see this)Cannot identify a project.1Can identify a project, but no reasons are given.2Can identify a project, and one reason.3Can identify a project, and at least one reason involves self-expression.4Observations This observation will require inquiry.Ask the student to explain the choices he/she made when completing [art project].Cannot identify any choices.1Can identify at least one choice, but cannot provide one reason.2Can identify at least one choice and provide one reason.3Can identify a more than one choice and reasons.4Increased understanding of differing cultures and how they connect to one’s ownSurvey ItemsStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meI feel comfortable being around people who sound or look different than me.○○○○Art or dance from other cultures really interests me. ○○○○I think other cultures are just as important as mine. ○○○○I think of myself as being part of a culture. ○○○○Scoring—Circle the number and add (student should not see this)I feel comfortable being around people who sound or look different than me. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Art or dance from other cultures really interests me. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I think other cultures are just as important as mine. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I think of myself as being part of a culture. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Total: _________Short-Answer Questions (written or oral)Student ID: ______________Program/Session Name: _________________Date: ______________Describe a difference between your culture and another culture. Is one way better than the other? Why or why not?Scoring—Circle the rating that best fits the response.(student should not see this)Cannot name a difference.1Can name a difference, but cannot provide a reason for their opinion.2Can name a difference, can provide one or more reasons for their opinion, but shows bias.3Can name a difference and provides one or more reasons that show respect.4Observations Circle One Pre-test Post-testStudent NameProgram/Session NameDateCheck one box per row for each behavior observed on the date above. Never/ Seldom Occasionally Often Almost always1. Comfortable working with students who look or sound different.□□□□2. Bullies students who look or sound different. □□□□Scoring—Circle the number and add1. Comfortable working with students who look or sound different. Never/ Seldom1 Occasionally2 Often3 Almost always42. Bullies students who look or sound different. Never/ Seldom4 Occasionally3 Often2 Almost always1Total: _________Increase in self-confidence to perform a task.Note that the specific art medium needs to be specified where [brackets] indicate.Survey ItemsStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meI know how to prepare for an [art medium] demonstration or performance.○○○○I would be comfortable helping younger or less experienced people with their [art medium] project. ○○○○I am comfortable [“performing” or “showing my artwork”] in front of others. ○○○○I know how to [insert task] from beginning to end. ○○○○I know I can finish anything I start.○○○○Scoring—Circle the number and add (student should not see this)I know how to prepare for an [art medium] demonstration or performance. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I would be comfortable helping younger or less experienced people with their [art medium] project. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I am comfortable [“performing” or “showing my artwork”] in front of others. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I know how to [insert task] from beginning to end. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I know I can finish anything I start. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Total: _________Short-Answer Questions (written or oral)Student ID: ______________Program/Session Name: _________________Date: ______________If you were going to [perform a task or project], how would you start? What would you do next?Scoring—Circle the rating that best fits the response.(student should not see this)Is not clear on how to begin.1Can provide a step in the project or task, but not how it begins.2Can describe an appropriate beginning.3Can describe how to start and at least one more step in the process.4Student ID: ______________Program/Session Name: _________________Date: ______________Are you comfortable [performing task]? Why or why not?Scoring—Circle the rating that best fits the response.(student should not see this)Not comfortable.1At least some comfort, but no reasons provided.2At least some comfort, with one reason provided.3At least some comfort, with more than one reason provided.4Observations Circle One Pre-test Post-testStudent NameProgram/Session NameDateObserve the student performing the task and choose the category that best fits.Student does not seem able to perform task on his/her own OR performs the task, but in a consistently incorrect manner.1For the most part, student seems unsure of how to perform the task.2Student seems confident in performing the task with minimal direction.3Student is very confident performing the task and even acts as a leader for others.4Circle One Pre-test Post-testStudent NameProgram/Session NameDateObserve the student performing the task or the product of a project and choose the category that best fits.Student is willing to take risks in work. Never/ Seldom1 Occasionally2 Often3 Almost always4Increased ability to reflect upon one’s own abilities and performanceNote that the specific art medium needs to be specified where [brackets] indicate.Survey ItemsStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meI know where I could improve my skills in [arts or task].○○○○When I complete a task or assignment, I think about how I could do it differently next time. ○○○○I am comfortable thinking about how well I can [insert skill or task].○○○○If my [insert performance or project] is not going well, I can do something about it.○○○○I know which parts of [insert performance or task] are easy for me.○○○○Scoring—Circle the number and add (student should not see this)I know where I could improve my skills in [arts or task]. Not like me at all1 A little like me2 Mostly like me3 A lot like me4When I complete a task or assignment, I think about how I could do it differently next time. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I am comfortable thinking about how well I can [insert skill or task]. Not like me at all1 A little like me2 Mostly like me3 A lot like me4If my [insert performance or project] is not going well, I can do something about it. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I know which parts of [insert performance or task] are easy for me. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Total: _________Short-Answer Questions (written or oral)Student ID: ______________Program/Session Name: _________________Date: ______________Think about a [task or performance] you did. Name one thing you did well and one thing you can improve on next time.Scoring—Circle the rating that best fits the response.(student should not see this)Cannot name a positive or negative aspect of performance.1Can name either a positive or negative aspect, but not both.2Can name both a positive and negative aspect of performance, but not much detail is provided.3Can name both a positive and negative aspect of performance and provides detail.4Student ID: ______________Program/Session Name: _________________Date: ______________How would you know if your [performance, art piece, project] is successful or not?Scoring—Circle the rating that best fits the response.(student should not see this)Cannot provide criteria.1Criteria are provided but are subjective in nature (e.g., “I like it.”)2At least one objective criterion is provided.3More than one objective criteria are provided.4Observations Use with caution, as self-reflection is an internal process.Circle One Pre-test Post-testStudent NameProgram/Session NameDateObserve the student performing the task and choose the category that best fits.Student does not demonstrate awareness of his/her own skills or performance.1Student is minimally aware of his/her own skills and performance.2Student is somewhat aware of his/her own skills and performance.3Student is very aware of his/her own skills and performance.4Increased cooperative and collaborative behaviors to successfully complete a shared taskSurvey ItemsStudent IDProgram/Session NameDatePlease choose the response that is most like you. Not like me at all A little like me Mostly like me A lot like meI like to do my part when working in a group.○○○○Others in my group count on me. ○○○○I can learn more if I work in a group than if I work on my own. ○○○○When you are working with others, you can’t always expect to get your way.○○○○What I do affects others in the group. ○○○○Scoring—Circle the number and add (student should not see this)I like to do my part when working in a group. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Others in my group count on me. Not like me at all1 A little like me2 Mostly like me3 A lot like me4I can learn more if I work in a group than if I work on my own. Not like me at all1 A little like me2 Mostly like me3 A lot like me4When you are working with others, you can’t always expect to get your way. Not like me at all1 A little like me2 Mostly like me3 A lot like me4What I do affects others in the group. Not like me at all1 A little like me2 Mostly like me3 A lot like me4Total: _________Short-Answer Questions (written or oral)Student ID: ______________Program/Session Name: _________________Date: ______________What are the advantages of doing a project with others rather than on your own?Scoring—Circle the rating that best fits the response.(student should not see this)Cannot name a difference between individual and group work.1Can name a difference, but only reflecting negative attitudes toward group work.2Can name one advantage of group work compared to individual work.3Can name more than one advantage of group work compared to individual work.4Observations Circle One Pre-test Post-testStudent NameProgram/Session NameDateCheck one box per row for each behavior observed on the date above. Never/ Seldom Occasionally Often Almost always1. Compromises.□□□□2. Participates with effort in group activities. □□□□3. Demonstrates affirming behavior by openly showing support for others’ contributions and abilities. □□□□Scoring—Circle the number and add1. Compromises. Never/ Seldom1 Occasionally2 Often3 Almost always42. Participates with effort in group activities. Never/ Seldom1 Occasionally2 Often3 Almost always43. Demonstrates affirming behavior by openly showing support for others’ contributions and abilities. Never/ Seldom1 Occasionally2 Often3 Almost always4Total: _________Frequently Asked QuestionsWording of surveysQ: The surveys are not age-appropriate or the surveys are not easily understood by students. What should we do?A: In the short-term, you can modify the survey to make it more relevant to your program (e.g., replace task with an actual activity) and substitute words that have the same meaning (e.g., “neighborhood” for “community”). Please note: wording should not change the meaning of the item. When in doubt, please contact the Supervisor of the Partnership for the Arts & Humanities. Scale itemsQ: The provided scale items of “Not like me” to “A lot like me” do not work for our program. What should we do?A: See Page 6 of this guide for alternatives.Youth having a “bad day”Q: Sometimes our youth have really bad days and it affects their assessment. Help!A: If a youth is clearly not acting in typical ways due to a “bad day,” consider assessing them another time. But do not wait for the “perfect” mood before assessing! Just use this option if someone is obviously behaving in ways that are out of character; otherwise, go ahead and assess.We have existing outcome measuresQ: Is it acceptable to use our existing outcome measurement tools?A: If those tools relate directly to the outcome you selected, it is generally acceptable to use another tool. However, keep in mind that these should be summative (outcome-focused), as opposed to formative (process-related), tools. When in doubt, please contact the Supervisor of the Partnership for the Arts & Humanities.I saw change in my students last year, but surveys didn’t pick it upQ: Can I just report my observations of whether students achieved the outcome?A: You may use observations rather than surveys, but use the provided instrument. We need to be precise about what kind of change happened and how much. Also, it’s human nature to rate a student high when we feel positive about what they’ve accomplished, regardless of the criteria tied to a particular outcome. The observation instrument has clear scoring guidelines to help data be reliable and less subject to (understandable) bias. Multiple sessions and/or programs Q: Our program happens over multiple sessions and/or we have multiple programs. How can this process work for us?A: This issue is very much case by case, but in general, if your session/programs are attended by the same students, it is advised that you assess once in the beginning (pre) and once at the very end (post). If attendees are different, for instance you have a spring program and a summer program, it is advised that you complete the pre/post cycle twice—once for each separate group of students.Matching students on surveysQ: We have trouble matching pre/post surveys. Is there something we can do?A: Because attendance varies in out-of-school time programming, there will naturally be unmatched surveys. However, to help match the maximum number of surveys, we suggest going through each matching variable together as a group each time the survey is administered. For instance, you might say, “Everyone write their first initial; now you should write a ‘B’ for boy or ‘G’ for girl…” This will help the matching information be accurate and complete.Outcome achievementQ: What counts as outcome achievement?A: Remember that all outcomes are stated in terms of an increase or improvement. If a student raised their score on the assessment tool on the post-test compared with the matched pre-test, this student achieved the outcome. If the student maintained their score, this does not count as an improvement or increase.Ceiling effectQ: We think our students are exhibiting a “ceiling effect,” where they enter our program at a high level of achievement. What can we do?A: First, in order for us to determine whether there is a true ceiling effect, we need more data. At this point, it’s unclear how widespread this is, whether it’s limited to certain tools, and/or whether this is related to age. However, if your experience is that students consistently rate at the top for your chosen outcome at the beginning of programming, consider that a different outcome or level of outcome (intermediate vs. short-term) might be more appropriate.Appendix A: Passive Parent/Guardian Permission Form (English)PASSIVE PARENT/GUARDIAN PERMISSION FORM[Insert PROGRAM INFORMATION, title, or other information]We want to make our program better. To do this, we want to receive feedback from your student on the program. We may do this through surveys, observations and/or interviews/focus groups of all students. Please know that your child’s privacy is important to us. Students will not put their names on any documents used for feedback. Each student will use a unique identifier that will not include their name. Your child’s responses will be added to the information of others in the program and reported only as a group. Risks to your child being part of this process include possible feelings of stress when having to answer questions.?Benefits to your child include a chance to reflect on their experience with the program and helping us to improve our program.While we want your student to provide feedback on the program, this is voluntary. If you choose not to allow data collection for your child, this will NOT affect your child’s participation in the program. If you would like to see the survey, observation instrument, or interview questions, a copy is available by contacting us. Please read the section below. If you do not want your child to take part in the evaluation, sign the form and return it at the next program session. If you will allow us to collect information, there is nothing you need to do – you can ignore this form. Thank you!For questions about this process, please contact April Heding, Supervisor of the Partnership for the Arts & Humanities at Milwaukee Public Schools, by email hedingal@milwaukee.k12.wi.us or phone (414) 475-8508.___________________________________________________________________________Child’s name:___________________________________________ Grade: ______________By signing below, I indicate that my child’s information may not be used in evaluation.Parent/Guardian’s signature:_____________________________________ Date:_________________Phone number: __________________________________________Please sign and return this form only if you DO NOT want your child to take part in the evaluation. If you agree to have your child take part in the evaluation, you do not need to do anything with this form. Appendix B: Passive Parent/Guardian Permission Form (Spanish)As of September 14, 2018, the Partnership for the Arts & Humanities program staff are still waiting on a Spanish version of the Passive Parent/Guardian Permission Form to be completed. If organizations need the Spanish version of the document, they can reach out to the Supervisor of the Partnership for the Arts & Humanities. The Spanish version will be shared with organizations as soon as it’s available. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download