Foodbank of Santa Barbara County Evaluation Framework and ...



030670500Foodbank of Santa Barbara County Evaluation Framework and Definitions for Strategic Plan Years 2013-2015Framework for EvaluationRE-AIM: Evaluation of social programs encompasses more than collecting data that lead to judgments about a program or policy. Social program evaluation should paint a picture of how well an organization is delivering its services, if the public and recipients find value in those services and what are the outcomes of those services on the target population (Rossi and Freeman 1993). Of note evaluation differs from research is several important ways. One evaluation is controlled by those involved, the stakeholders, instead of being rigorously designed by an investigation. Two, evaluation is ongoing rather than being limited to a specific timeframe. Three, the scope of evaluation is broad, in an attempt to be integrative, rather than narrowly focused. Thus the Foodbank of Santa Barbara County is employing an evaluation approach based on the RE-AIM framework championed by Russell Glasgow. RE-AIM is an acronym for Reach, Effectiveness, Adoption, Implementation, and Maintenance. RE-AIM moves beyond the current reductionist approach to assess interventions with its tendency to oversimplify issues and their outcomes in order to isolate effective progirams or activities. In particular, the emphasis on eliminating the potential for confounding factors typically results in samples of very homogeneous, highly motivated, healthy individuals which equates to samples of non-representative people. Moreover the weight on showing immediate results focuses attention on high-intensity interventions that produce outcomes within a short period of time but might only produce short term results or be implemented by few sites. RE-AIM hypothesizes that the overall social-change impact of an intervention is a function of all five RE-AIM dimensions not simply client-based outcomes. That is all five dimensions are important and equally in need of evaluation. The implication is that to have a substantial impact at the population level, an intervention must do reasonably well on all or most RE-AIM dimensions. Importantly, Foodbank RE-AIM evaluation allows summary indices (Success Score) to be developed for the use in determining overall impact of individual programs as well as initiative areas. Story Corp: The types of quantitative data used to develop the summary indices for the Foodbank RE-AIM evaluation does not tell the complete story. Qualitative data collection is a necessary and fun addition to the process of collecting and interpreting quantitative data. Thus the Foodbank developed an idea called Foodbank Story Corp. Any Foodbank staff or volunteer can collect the information as interview templates are available to help guide the questioning process. Picture and video collection are another important aspect of Story Corp. Overall Story Corp gathers data in the form of stories and feedback from stakeholders. This qualitative data provides important formative information about the delivery of services as well as how those services are perceived by the community. DefinitionsFoodbank RE-AIM: Evaluation framework used to assess the many dimensions of social change/preventative health. Foodbank RE-AIM is based on the RE-AIM framework and equally weights key outcomes to create a summary score, a Success Score, that allows for holistic evaluation of Foodbank programming and activities. Foodbank Story Corp: a qualitative data collection and presentation approach. Collected every month Story Corp includes pictures, stories and feedback from stakeholders in Foodbank programs and activities. Story Corp provides important formative information about the delivery of services as well as how those services are perceived by the stakeholders. Indicator: a quantitative value that represents a measurable outcome for Foodbank activities. The outcomes can be either process or impact related. Indicators are essentially numeric approximations that estimate chance of social or diet-related health change based on client, site, staff, or Foodbank effectiveness and efficiency. Just as cholesterol is a measure for risk of heart disease the % of the population reporting very low food security is a measure for risk of anemia in children. Logically following that if you improve blood cholesterol levels you will decrease rates of heart disease with the final impact being longer, healthier lives and lower medical costs. The same follows for food security that if you improve food security in children you will improve iron levels leading to a final impact of better focus in the classroom and better learning outcomes. Process outcome: is a numeric measure of Foodbank efficiency and effectiveness of conducting work.Impact outcome: is a measurement of the outcomes at the subject level, including clients and staff or volunteers that are administering the intervention (e.g. program or activity) and organizational changes typically manifesting themselves as policy or guideline changes.Measurement tool: what is used to collect the data that is then inputted to generate the indicator value. The tools are specific for each indicator although the specificity, validity and reliability of some of the measurement tools have yet to be assessed. The tools are based on what has been determined by the Foodbank (and hopefully the community) as the most relevant information to collect keeping in mind Foodbank and community resources (e.g. time, language barriers, data collection technology, feasibility of collecting specific types of data), invasiveness of questions and pre-existing data the Foodbank has access to. Score card: visual summary of all indicator values applied within the Foodbank RE-AIM evaluation framework. Shows Success Score trends over time.Success Score: based on a total of 50 points it is the summation of 10 indicators. Each indicator has a maximum point value of 5 with 5 being the best and 1 being needs significant improvement. All 10 indicators are given equal weight when determining the Success Score. The Foodbank has arbitrarily defined a Success Score of 33 or greater as excellent, a score of 17 to 32 as room for growth and a score of less than 17 requires significant improvement. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download