Q: Do I need to have a local evaluator?



PA 21st Century EvaluationFrequently Asked Questions (FAQs)Q:? Do I need to have a local evaluator?A:? Yes, grantees are required to have an external?local evaluator.??For information on planning for evaluation,?view the webinars at? refer to your grant's Request for Applications (RFA).Q:? Where can I find a local evaluator?A:? Grantees should have identified their local evaluator in their application for funds, which is now part of your grant contract.? Review your contract to determine who that evaluator is, or that one was not identified.? If you do not currently have an evaluator or the evaluator you have used in the past?is not meeting your needs, you have several options for identifying a local evaluator:Talk to other 21st Century grantees in your region about who they are using and their satisfaction with the evaluator.Contact other grant-funded programs in your organization to identify evaluators that have worked or are working with your organization.Contact intermediate units or colleges/universities in your area?to see if they offer evaluation services.Consult the?American Evaluation Association's Find an Evaluator?tool.Your evaluator may be an individual or a team, depending on your program's needs and the evaluator's capacity.Be sure to share with your current or new evaluator your grant contract, the RFA (grant guidelines), and evaluation requirements and report timelines.?Changing evaluators may require a program revision.? Contact your PDE Program Officer for clarification.Q:? What happens if I don't submit my reports by the deadline?A:? Timely completion of your reports is a contractual obligation of your grant.? Non-submission of reports may result in PDE ceasing to process expenditure reports and reimbursements until the report(s) is completed. Q:? I have programs operating through two (or more) cohorts of funding.? How do I report results?A:? Each cohort must be reported separately in each report system/form.? Q:? I have students who attended 21st Century programming at two different centers.? How do I report their results?A:? You will need to assign each student to their primary 21st CCLC center.? This is most easily done by determining which center the student(s) attended most frequently and assigning the student(s) to that center for the purposes of reporting.? Once a primary center is assigned to a student, that student's information should be attributed ONLY to that center.? That student should not be counted in any other center they may have attended.Q: I have several students who are near the 30+ days attended?and may reach 30+ days by the program end, but?may not reach 30 days?by the time I need to administer the Teacher Survey.? What should I do?A:? It may be appropriate to survey the teachers of those students who are?near?the 30 days mark.? Programs need to use their judgment as to when it is appropriate to survey near-30 day attendees.? It is always up to the grantee how it operationalizes fulfilling reporting requirements within the provided parameters.??The classroom teacher should have a handle on how the student has changed over the course of the year by that point.? Also, because the survey is not an objective measure like an assessment, any slight variation will probably not be picked up by the teacher and would also not likely affect the results of the whole group.Also, a Teacher Survey is a requirement by the federal funding for each student who attends 30+ days.? This requirement does not take into account that the program may proceed past the point at which the survey could be administered for logistical reasons (you probably wouldn’t have access to – or much success with - surveying teachers over the summer after the school year program ends).Example:??Let’s say you have 100 students who, over the course of the school year program, attend 30+ days of your program’s 90 total days of operation, but at the time of the survey, about half have attended 30 or more days and the remaining students around 25-29 days.? You choose to distribute the survey for the half of the kids already at 30 days.? During the administration time (which for many grantees might be several weeks between distribution and collection), more students reach the 30 mark, but you may or may not follow up with those additional students’ teachers.? What happens is that you then report in the federal data system that you had 100 regular attendees over the course of your program term, but only 50 of them have surveys.? The federal funder and legislators are then making decisions about the program based on a) only half the population’s results and b) that the grantee wasn’t able to submit the required data.While it is?not always possible?to get 100% of data for 100% of students, you need to be as close to 100% as possible.Furthermore, results from teacher surveys for non-regular attendees may be useful for program planning or comparison purposes.Q:??Can center/program academic staff complete the Teacher Survey instead of the student's classroom teacher???A:??Grantees are to survey students' classroom teacher?at the student's home school.? This may mean that the grantee needs to distribute surveys to and collect surveys from multiple buildings.? However, if the program staff?IS?the student's regular classroom teacher, this would be allowable.? Grantees have a number of options to make this task easier.?Ask teachers to complete the survey online using AIU's web-based Teacher Survey or?another system?Have center staff enter paper surveys into AIU's web-based?21st CCLC Teacher Survey in order to get an organized, consistent data poolBe sure to collect contact information for each student's teacher(s) at enrollmentEstablish a relationship with the building/district administrator to ensure ease of access to teachersQ:? What resources are available to help me collect and manage data needed for evaluation?A:? The AIU Evaluation Team developed an Evaluation Handbook and various tools for grantees to download for?free.? There are a number of templates, forms, directions, and guidance to help grantees and their local evaluator efficiently conduct evaluation.? You can download these resources from the?21st CCLC evaluation webpage.Q:? Do I have to complete my report all at once?A:? Yes and No.? Some reports allow the grantee to log in/out in order to complete reports in more than one “sitting.” Other reports may be downloaded and completed offline to later be submitted electronically. Q: How is the information I report for 21st Century used?A:? Information and results?reported are aggregated at the state and national level to examine program implementation and outcomes for the program nationwide.? Each 21st CCLC grantee's results contribute to the overall state and national results.? This information may be used at the USDE or legislative levels to make decisions about the 21st Century Community Learning Centers program.The state evaluation team for 21st Century at the Allegheny Intermediate Unit collects and analyzes?center, grantee, cohort, and state information reported in?the various reports?to provide an evaluation report for PDE outlining 21st CCLC implementation and results.? These results are organized at the grantee level for highlighting successful grantees and providing targeted technical assistance to struggling grantees.? Information contained in the evaluation report is used to determine program trends and plan for the future.Q:? I have students who don't have PSSA data.? How do I report?? What about Keystone Exams?A:? It is expected that some students will not have PSSA/PASA data to report in the state assessment fields in the Annual Performance Report, as not all grades take the PSSA/PASA.? The following grades will usually have both years (prior and current) of PSSA/PASA data: Grades 4, 5, 6, 7, 8.?Grade 3?will usually have current year data only, as PSSA/PASA is not administered in Grade 2.??The exception would be students who were retained in?(repeated)?Grade 3.? Students in the PreK-Grade 2 grade bands?should not have any state assessment data reported.? Keystone Exam data may only be entered for Grade 11 students. ?If a student does not have?data to report in the state assessment fields, enter "not available."??DO NOT report local assessment data?in the federal APR in place of PSSA/PASA/Keystone Exams, even if the assessment is aligned with state assessments.?Q:? How/When do I report Keystone Exam data?A:?For federal reporting:? Grantees will report PSSA/PASA data for grades 3-8 as in previous?years. ?Keystone?Exams will only be used for Grade 11 students.? 'Not available' would continue to be marked for grades 9, 10, or 12.??As a grantee, you may want to examine Keystone Exams; however, you can only compare the most recent Keystone Exam results with prior?Keystone?Exam results.? You CANNOT compare Keystone Exam data to PSSA data.???Q:? I'm overwhelmed.? How do I get started preparing to evaluate and report on my 21st Century program?A:? Pennsylvania's 21st Century program is multi-layered and complex.? The reporting requirements reflect this complexity.??Here are some steps to help you get started.Collect and review the following items:? Your?organization's approved grant contract and the Request for Applications (RFA) that applies to your funding cycle (cohort).? Read these documents to determine what you are required to be doing in your program.Identify your local evaluator.?? All 21st Century grantees are required to have one. ?If this individual or organization is not named in your grant, ask others in your organization who may have served in this capacity in the past.? Collect and review the evaluator's contract or service agreement with your organization to determine what the evaluator has agreed to do for pare the evaluation requirements and reports (and timelines) with the evaluation agreement with the local evaluator.? Locate and review?your evaluation plan.? If you do not have one, determine what the organization is responsible for and what the evaluator will be doing for you based on the evaluator's agreement.Contact the evaluator and discuss your needs, your program, and the evaluator's status with your evaluation.? Determine what has already been done and what?still needs to be done.Create or revise the evaluation plan.? AIU provides an?Evaluation Framework?to help you if you need a template.Review?AIU's evaluation webpage for 21st Century.? ? Read the?21st Century Evaluation Handbook.? Review the various reporting systems and what may have been done in the past.Identify what data are being collected and what needs to be collected.? If it is not already being followed, implement the evaluation activities in your evaluation plan or take steps to collect the information identified in the grant contract and reporting requirements.If needed, review the presentations from the 21st Century state evaluation team (AIU) on the 21st CCLC evaluation webpage.?Implement, adjust, refine, and operationalize the data and evaluation processes, taking the requirements and reports into consideration.? Discuss changing needs with your local evaluator.Locate your report system login information.Identify who will be responsible for entering program and student data into the reports/reporting systems.? If needed, share the login information with them.Contact the?21st CCLC state evaluation team at AIU?with questions.Submit your reports by the deadlines. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download