Monitoring and Evaluation Manual - FSN) Network

Monitoring and Evaluation Manual

Prepared for ADRA International Food Security Department

Prepared by TANGO International, Inc.

March 2007

Table of Contents

Acronyms........................................................................................................................... iv Part I. Overview of Monitoring and Evaluation ................................................................. 1 1. Introduction..................................................................................................................... 1

1.1 Purpose of the Monitoring and Evaluation Manual ............................................................ 1 1.2 Using the Manual ................................................................................................................ 2 2. Guiding Principles and Elements of Monitoring and Evaluation ................................... 3 2.1 What is Monitoring and Evaluation? .................................................................................. 3 2.2 Why Does a Program Need a Monitoring and Evaluation System? ................................... 5 2.3 General Overview of Basic Elements of a M&E System ................................................... 6 2.4 Monitoring and Evaluation Framework .............................................................................. 7 2.5 General Overview of Indicators and Indicator Development ............................................. 8 2.6 Managing for Impact......................................................................................................... 10 3. Conceptual Frameworks for Programming.................................................................. 12 3.1 Conceptual Frameworks for Project Design ..................................................................... 12 3.2 The USAID/FFP Expanded Conceptual Framework for Food Security........................... 15 Part II. Key Components and Stages of M&E Systems .................................................. 17 4. Vulnerability or Holistic Assessments.......................................................................... 17 4.1 Assessment Preparation .................................................................................................... 17 4.2 Defining Assessment Objectives ...................................................................................... 20 4.3 Target Area Selection ....................................................................................................... 20 5. Problem Analysis .......................................................................................................... 21 5.1 Using Cause and Effect Logic in Project Design.............................................................. 22 5.2 Hierarchical Problem Analysis ......................................................................................... 23 5.3 From Problem Analysis to Project Strategy...................................................................... 24 6. Program Design and Logical Frameworks ................................................................... 27 6.1 Establishing SMART Goals and Objectives ..................................................................... 28 6.2 Design Principles for Monitoring and Evaluation ............................................................ 29 6.3 Logical Frameworks ......................................................................................................... 29 6.4 General Logical Frameworks: linking indicators to program design................................ 30 6.5 The Logical Framework Matrix (LogFrame).................................................................... 31 7. Steps to Setup a Monitoring Systems ........................................................................... 37 7.1 Six steps to Setup an M&E System .................................................................................. 37 7.2 Monitoring for Performance and Participation ................................................................. 40 7.3 Participatory Monitoring................................................................................................... 41 7.3 Monitoring the Risk and Vulnerability Context ............................................................... 43 8. Establishing Indicators and Performance Targets ........................................................ 45 8.1 What are Indicators and Targets ....................................................................................... 45 8.2 Types of Indicators ........................................................................................................... 46 8.3 Identifying what and how to measure ............................................................................... 47 8.4 Characteristics of Ideal Indicators .................................................................................... 48 8.5 Criteria for selection of sound indicators .......................................................................... 49 8.6 Outcome and impact indicators......................................................................................... 49 8.7 Performance targets and benchmarks ............................................................................... 50 8.9 Approaches to establishing/setting targets ........................................................................ 51 8.10 Limitations of targets ...................................................................................................... 51 9. Sampling: Key Concepts .............................................................................................. 52 9.1 Introduction to Sampling .................................................................................................. 52 9.2 Types of Sampling ............................................................................................................ 52 9.3 Key Definitions................................................................................................................. 53 9.4 Sampling Methods/Design................................................................................................ 54

ADRA ? Monitoring and Evaluation Manual

i

9.5 Non-Probability Sampling ................................................................................................ 56 10. Baseline Surveys and Data Management.................................................................... 58

10.1 Initial considerations in conducting a baseline survey.................................................... 58 10.2 Methods of data collection.............................................................................................. 59 10.3 Developing a baseline survey ......................................................................................... 60 10.4 Data collection process ................................................................................................... 63 10.5 Data management............................................................................................................ 65 11. Data Analysis .............................................................................................................. 67 11.1 Data Cleaning.................................................................................................................. 67 11.2 Quantitative Data Analysis ............................................................................................. 68 11.3 Qualitative Data Analysis ............................................................................................... 70 11.4 Data Interpretation and Presentation............................................................................... 70 12. Information Sharing .................................................................................................... 72 Part III. M&E Considerations for Sector-Specific Programming..................................... 76 13. M&E Considerations for Food Security Programs..................................................... 76 13.1 Seasonality and Cross-program Comparability .............................................................. 76 13.2 Food Availability ............................................................................................................ 76 13.3 Food Access .................................................................................................................... 77 13.4 Food Utilization .............................................................................................................. 79 13.5 Market Information ......................................................................................................... 80 14. M&E Considerations for Health and Nutrition........................................................... 80 14.1 Selection of Indicators .................................................................................................... 80 14.2 Integrated Disease Surveillance and Nutrition Surveillance Systems ............................ 82 14.3 Nutrition.......................................................................................................................... 83 15. M&E Considerations for Micro Enterprise Programs ................................................ 84 16. M&E Considerations for Reproductive Health Programs .......................................... 85 17. Integrating Cross-Cutting Issues into Data Collection ............................................... 87 17.1 HIV/AIDS ....................................................................................................................... 87 17.2 Gender............................................................................................................................. 88 17.3 Conflict ........................................................................................................................... 88 References......................................................................................................................... 89 Annexes ............................................................................................................................ 91 Annex 1: Components of a Humanitarian Information System ....................................... 92 Annex 2: Sampling ........................................................................................................... 94 Annex 4: Food Security Weathervane Indicators ........................................................... 106 Annex 5: Potential Indicators for Use in Health and HIV/AIDS ................................... 113 Annex 6: Potential Indicators for Use in Micro & Small Enterprise Development ....... 114 Annex 9: Example of Topical Outline for Use in Qualitative Assessments................... 122 Annex 10: Qualitative Assessment Tools ....................................................................... 127 Annex 11: Example of Community Questionnaire......................................................... 129 Annex 12: Example of Qualitative Data Matrix ............................................................. 140 Annex 13: Preparatory Tasks for Vulnerability Assessments ........................................ 142 Annex 14: How to Develop M&E Plan .......................................................................... 148

ADRA ? Monitoring and Evaluation Manual

ii

List of Figures

Figure 1: Project Hierarchy................................................................................................. 6 Figure 2: Monitoring and Evaluation Framework .............................................................. 8 Figure 3: Indicators and Targets Along With the Project Hierarchy ................................ 13 Figure 4: Livelihoods Framework .................................................................................... 13 Figure 5: USAID/FPP Expanded Conceptual Framework ............................................... 13 Figure 6: Hierarchical Problem Tree .................................................................................... 13 Figure 7: The Project Cycle ................................................................................................ 13 Figure 8: Linking the Results Framework to a Logical Framework ........................................ 30 Figure 9: A Logical Framework with indicators.................................................................... 31 Figure 10: Indicators for LogFrame ..................................................................................... 13 Figure 11: Partial Results framework for National TB Program............................................. 13 Figure 12: Definition of Results Based Management ............................................................ 13 Figure 13: Factors Influence People's Participation in M&E ................................................. 13 Figure 14: Key Steps in Creating and Selecting Indicators .................................................... 13 Figure 15: Transforming Ideas about Quality into Measures for Which Targets Can Be Set .... 50 Figure 16: Various Methods of Random Sampling ............................................................... 13 Figure 17: Essential Components of a Baseline Survey Plan .......................................... 61 Figure 18: TIPS for Questionnaire Design ........................................................................... 61 Figure 19: Sample Frequency Table ................................................................................. 69 Figure 20: Minimum Information Requirements for Food Availability........................... 77 Figure 21: Minimum Information Requirements for Food Access.......................................... 77 Figure 22: Information Requirements for Food Utilization .................................................... 77

List of Tables

Table 1: Types of Indicators and Purpose........................................................................... 9 Table 2: Common Sources of Secondary Data ..................................................................18 Table 3: Descriptive Information Obtained through Secondary Data Analysis ................19 Table 4:Generic LogFrame Matrix (WFP) ........................................................................32 Table 5: Objective Hierarchy Links to Monitoring & Evaluation .....................................38 Table 6: Difference between traditional evaluation and participatory evaluation .............42 Table 7: Sharing of Information needs to be based on Information Needs................................72

ADRA ? Monitoring and Evaluation Manual

iii

ADRA AIDS BCC CDC CDD CHW CORE DD DHS DME DR FANTA FFP GAVI HIF HFA HFS HIS HIV HLS HMIS ICHS IEC IFAD IMCI KAP MCH MEASURE MDG M&E MFI MICS MIS MSE NGO PMTCT PHC RH TB UNDP UNICEF USAID WHO

Acronyms

Adventist Development and Relief Agency International Auto Immune Deficiency Syndrome Behavior Change Communication Centers for Disease Control Control of Diarrheal Disease Community Health Worker Child Survival Collaborations and Resources Group Diarrheal Disease Demographic and Health Survey Design Monitoring and Evaluation Development Relief Food and Nutrition Technical Assistance Food for Peace Global Alliance for Vaccines and Immunization Hygiene Improvement Framework Health Facility Assessment Health Facility Survey Health Information Survey Human Immunodeficiency Virus Household Livelihood Security Health Management Information Systems Integrated Child Health Survey Information, Education and Communication International Fund for Agricultural Development Integrated Management of Childhood Illnesses Knowledge, Attitude, Practice Maternal and Child Health Monitoring and Evaluation to Assess and Use Results Millennium Development Goal Monitoring and Evaluation Micro-finance Institutions Multiple Indicator Cluster Survey Management Information System Micro and Small Enterprise Nongovernmental Organization Promotion of Maternal to Child Transmission Primary Health Care Reproductive Health Tuberculosis United Nations Development Programme United Nations Children's Fund United States Agency for International Development World Health Organization

ADRA Monitoring and Evaluation Manual

iv

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download