Basic Concepts in - PSC

[Pages:80]Basic Concepts in Monitoring and Evaluation

February 2008

Published in the Republic of South Africa by: THE PUBLIC SERVICE COMMISSION (PSC) Commission House Cnr. Hamilton & Ziervogel Streets Arcadia, 0083

Private Bag x121 Pretoria, 0001

Tel: (012) 352-1000 Fax: (012) 325-8382 Website: .za

National Anti-Corruption Hotline Number: 0800 701 701 (Toll-Free)

Compiled by: Branch Monitoring and Evaluation Distributed by Directorate: Communication and Infomation Services

ISBN: 978-0-621-37612-8 RP: 14/2008

FOREWORD

iv

CHAPTER 1 INTRODUCTION

1

1.1

The purpose of this document

2

1.2

Intended audience

2

1.3

Definition of Monitoring and Evaluation

3

1.4

Importance of Monitoring and Evaluation

4

1.5

Purposes (and uses) of Monitoring and Evaluation

4

1.6

Outline of this document

6

CHAPTER 2: THE CONTEXT OF MONITORING

AND EVALUATION

7

2.1

The Context of the Developmental State

8

2.2

Location of M&E in the policy process

8

2.3

Location of M&E in the planning process

10

2.4

The Government-wide M&E System

11

2.5

Important institutions with a role in M&E

13

2.5.1 Departments at the centre of government on the national level

14

The Presidency

14

National Treasury

14

Department of Public Service and Administration (DPSA)

14

Department of Provincial and Local Government (DPLG)

15

Statistics South Africa (Stats SA)

15

South African Management Development Institute (SAMDI)

15

2.5.2 Departments at the centre of government on the provincial level

15

2.5.3 Line departments

15

ii

2.5.4 Constitutional institutions

15

Public Service Commission (PSC)

16

Auditor-General

16

Human Rights Commission

16

CHAPTER 3: EVALUATION PERSPECTIVES

17

3.1

The idea of evaluation perspective

18

3.2

The Balanced Scorecard of Kaplan and Norton

19

3.3

Programme performance perspective

20

3.4

Financial perspective

20

3.5

Governance perspective

21

3.6

Human Resource Management (HRM) perspective

21

3.7

Ethics perspective

22

3.8

The perspectives adopted by National Treasury Guidelines

23

CHAPTER 4: VALUES

25

4.1

The value basis of monitoring and evaluation

26

4.2

Deriving standards of performance from values

27

4.3

The values and principles governing public administration

28

4.4

The Eight Principles of Batho Pele

34

4.5

Other concepts/ principles expressing some dimension of public service

performance

35

CHAPTER 5: EVALUATING PROGRAMME PERFORMANCE 38

5.1

Programme evaluation

39

5.2

Logic models

42

5.3

Results-Based Management

44

5.4

Theory-based evaluation

44

iii

CHAPTER 6: APPLYING THE CONCEPTS

47

6.1

Focusing monitoring and evaluation

48

6.2

Designing monitoring frameworks

48

6.3

Framing evaluation questions

51

6.4

Examples of monitoring and evaluation of different dimensions

54

CHAPTER 7: SUMMARY

60

LIST OF SOURCES CONSULTED

64

INDEX

68

Figures

1

The policy life cycle

9

2

Planning and review cycle

10

3

Policy frameworks of the Government-wide Monitoring and Evaluation System 12

4

Components of the logic model

42

5

Example of an outcome chain

45

Tables

1

Deriving indicators and standards from values

27

2

Examples of objectives and indicators according to the logic model

50

3

Examples of evaluation questions: Housing programme

52

4

Types of monitoring and evaluation in relation to evaluation

perspectives and values

55

Boxes

1

Example of a theory underlying causal relationship between outputs and

outcomes

29

iv

2

Examples of evaluative findings with regard to the efficiency of programmes 30

3

Examples of evaluative findings with regard to the effectiveness of programmes 36

4

Examples of evaluative findings with regard to the sustainability of programmes 37

5

Examples of evaluative findings with regard to the scale of engagement of

programmes

37

6

Examples of evaluative findings with regard to the targeting of programmes 40

7

Illustrations of types of evaluations: Housing programme

44

8

Examples of a theory underlying the causal relationship between outputs

and outcomes

45

9

Illustrations of types of evaluations: Housing Programme

58

v

Foreword

The growth of Monitoring and Evaluation (M&E) units in government, together with an increased supply of M&E expertise from the private sector, calls for a common language on M&E. M&E is a relatively new practice, which tends to be informed by varied ideologies and concepts. A danger for government departments is that these diverse ideological and conceptual approaches can exacerbate confusion and misalignment. The standardisation of concepts and approaches in government is particularly crucial for the enhancement of service delivery.

The PSC's mandate requires of it to monitor and evaluate the organization and administration, and the personnel practices, of the Public Service. Taking this mandate and the need for a common understanding of concepts and approaches into account, the PSC decided to produce this text on basic M&E concepts.

A very basic question asked when a monitoring system must be developed or when an evaluation is planned is: What to monitor or evaluate, that is, what should the focus of the monitoring or the evaluation be? This document tries to answer this basic question by introducing concepts and frameworks.

Evaluation involves a value judgement. Many of the concepts discussed in the document have the status of values.The PSC has the specific constitutional responsibility to promote the application of these values in the Public Service.

This document is by no means definitive, but the PSC hopes that it will contribute to better understanding and enriched debate about the utility of M&E as a tool for improving the performance of the Public Service. We hope that it will fill the gap that currently exists for an accessible document that caters for managers in the Public Service, whilst also providing a common point of reference to the more advanced practitioner. It is hoped that readers will feel compelled to delve more deeply into the discipline.

I trust that this document helps you to deepen your interest and understanding of monitoring and evaluation.

Yours sincerely

PROFESSOR STAN S SANGWENI CHAIRPERSON: PUBLIC SERVICE COMMISSION vi

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download