OUTCOME MAPPING

[Pages:155]OUTCOME MAPPING

BUILDING LEARNING AND REFLECTION

INTO DEVELOPMENT PROGRAMS

Outcome Mapping provides not only a guide to essential evaluation map-making, but also a guide to learning and increased effectiveness, and affirmation that being attentive along the journey is as important as, and critical to, arriving at a destination.

-- Michael Quinn Patton

More and more, development organizations are under pressure to demonstrate that their programs result in significant and lasting changes in the well-being of their intended beneficiaries. However, such "impacts" are often the product of a confluence of events for which no single agency or group of agencies can realistically claim full credit. As a result, assessing development impacts is problematic, yet many organizations continue to struggle to measure results far beyond the reach of their programs.

Outcome Mapping recognizes that development is essentially about people relating to each other and their environment. The originality of this approach lies in its shift away from assessing the products of a program to focus on changes in behaviour, relationships, actions, and activities in the people, groups, and organizations it works with directly. In doing so, Outcome Mapping debunks many of the myths about measuring impact. It will help a program be specific about the actors it targets, the changes it expects to see, and the strategies it employs and, as a result, be more effective in terms of the results it achieves. This publication explains the various steps in the outcome mapping approach and provides detailed information on workshop design and facilitation. It includes numerous worksheets and examples.

Sarah Earl holds a master's degree in Russian politics and development from Carleton University and an MA in Russian history from the University of Toronto. She joined IDRC's Evaluation Unit in 1998. Fred Carden has taught and carried out research at York University, the Cooperative College of Tanzania, the Bandung Institute of Technology (Indonesia), and the University of Indonesia. Dr Carden is coauthor of Enhancing Organizational Performance (IDRC 1999) and senior program specialist in IDRC's Evaluation Unit. Terry Smutylo has been the Director of IDRC's Evaluation Unit since its creation in 1992. Mr Smutylo has worked extensively throughout Asia, Africa, and Latin America, as well as in Canada, the United States, and Europe, conducting evaluations, providing evaluation training, and facilitating workshops.

OUTCOME MAPPING Earl, Carden, Smutylo

OUTCOME MAPPING

BUILDING LEARNING AND REFLECTION INTO DEVELOPMENT PROGRAMS Sarah Earl, Fred Carden & Terry Smutylo

Foreword by Michael Quinn Patton

Om outcome mapping

Om

Outcome Mapping

Building Learning and Reflection into Development Programs

Sarah Earl, Fred Carden, and Terry Smutylo

Foreword by Michael Quinn Patton

INTERNATIONAL DEVELOPMENT RESEARCH CENTRE Ottawa s Cairo s Dakar s Montevideo s Nairobi s New Delhi s Singapore

? International Development Research Centre 2001

Published by the International Development Research Centre PO Box 8500, Ottawa, ON, Canada K1G 3H9

National Library of Canada cataloguing in publication data

Earl, Sarah, 1971?

Outcome mapping : building learning and reflection into development programs

Includes bibliographical references. ISBN 0-88936-959-3

1. Economic development projects -- Evaluation. 2. Technical assistance -- Developing countries -- Evaluation. 3. International cooperation. I. Carden, Fred. II. Smutylo, Terry. III. International Development Research Centre (Canada) IV. Title.

HD75.9E72 2001 338.91

C2001-980277-3

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, or otherwise, without the prior permission of the International Development Research Centre. Mention of a proprietary name does not constitute endorsement of the product and is given only for information.

IDRC Books endeavours to produce environmentally friendly publications. All paper used is recycled as well as recyclable. All inks and coatings are vegetable-based products. The full catalogue of IDRC Books is available at .

CONTENTS

Foreword --Michael Quinn Patton

vii

Preface

xi

1. Outcome Mapping: The Theory

1

What Is Outcome Mapping?

1

Three Stages of Outcome Mapping

3

Why Not Impact?

5

How Can Outcome Mapping Be Used?

10

How Outcome Mapping Differs from Other Logic Models

12

When Is Outcome Mapping Best Used?

14

Is Outcome Mapping Appropriate for You?

15

2. Outcome Mapping: The Workshop Approach

17

Overview of the Steps

17

Workshop Outputs

19

Who Should Participate?

19

Who Should Facilitate?

21

Workshop Materials

22

Presentation Methods

22

Preparing for an Outcome Mapping Workshop

23

Conducting a Historical Scan

24

Developing a Common Understanding of Evaluation

26

Guiding Principles for Evaluation

28

3. Stage1: Intentional Design

31

Introduction to Stage 1

32

Step 1. Describe the Vision

33

Step 2. Identify the Mission

37

Step 3. Identify the Boundary Partners

41

Step 4. Identify the Outcome Challenge

47

Design Worksheet 1: Program Framework

51

Step 5. Develop Graduated Progress Markers

53

Design Worksheet 2: Progress Markers

59

Step 6. Complete a Strategy Map for Each Outcome Challenge

61

Design Worksheet 3: Strategy Map

67

Step 7. Articulate Organizational Practices

69

Design Worksheet 4: Organizational Practices

74

Contents

v

4. Stage 2: Outcome & Performance Monitoring

75

Introduction to Stage 2

76

Monitoring Three Parallel Processes

77

Will Outcome Mapping Provide the Appropriate Monitoring System?

78

Step 8. Setting Monitoring Priorities

83

Monitoring Worksheet 1: Monitoring Plan

86

Step 9. Develop an Outcome Journal

87

Monitoring Worksheet 2: Outcome Journal

94

Step 10. Customize a Strategy Journal

97

Monitoring Worksheet 3: Strategy Journal

101

Step 11. Customize a Performance Journal

103

Monitoring Worksheet 4: Performance Journal

108

Monitoring Worksheet 5: Program Response

110

Monitoring Worksheet 6: Reviewing the Logic of the Program

111

5. Stage 3: Evaluation Planning

113

Introduction to Stage 3

114

Step 12. Evaluation Plan

115

Evaluation Worksheet 1: Evaluation Plan

124

Appendix A: Sample Intentional Design Framework

125

Appendix B: Overview of Evaluation Methods

129

Appendix C: Glossary

131

Appendix D: Terms in French, English, and Spanish

135

References

137

About the Authors

138

The Publisher

139

vi

Contents

FOREWORD

Imagine a map ... drawn from your memory instead of from the atlas. It is made of strong places stitched together by the vivid threads of transforming journeys. It contains all the things you learned from the land and shows where you learned them .... Think of this map as a living thing, not a chart but a tissue of stories that grows half-consciously with each experience. It tells where and who you are with respect to the earth, and in times of stress or disorientation it gives you the bearings you need in order to move on. We all carry such maps within us as sentient and reflective beings, and we depend upon them unthinkingly, as we do upon language or thought .... And it is part of wisdom, to consider this ecological aspect of our identity.

? John Tallmadge, Meeting the Tree of Life (1997: IX)

M aps are cognitive guides. They locate us, helping us to figure out where we are now in relation to where we've been, and to plan where we're going. It is altogether appropriate, then, that IDRC's Evaluation Unit has chosen the metaphor of mapping to guide those interested in development on the sometimes confusing, even frightening journey through the hazardous territory of outcomes.

The language can be daunting: outcomes, impacts, goals, objectives, purposes, mission, and outputs -- and these terms just scratch the surface. The questions can overwhelm. What's the difference between evaluation and monitoring? How do short-term changes relate to intermediate changes and long-term results? What kinds of results count as outcomes? How can the need for accountability be balanced against the need for learning? Then there's the attribution problem. To what extent and in what ways can one establish a causal linkage between activities, outputs, outcomes, and impacts? Who gets credit for results? What kinds of evidence are credible? What's the unit of analysis? What role does stakeholder involvement play in all this?

The territory of developmental change and evaluation is vast, complex, and ever-changing. Trying to manoeuver through that territory one is likely to encounter deep ravines of uncertainty, mountains of data, and side-tracks that lead nowhere. It sure would help to have a map to the

Foreword

vii

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download