Handbout 03v8 KPI - Oman Academic Accreditation Authority

[Pages:11]Ministry of Higher Education & Oman Accreditation Council

Training Module 03 v8

KPI

Designing Key Performance Indicators

Workshop Handout

? Dr Josephine Palermo and Martin Carroll Last updated: 9 September 2006

? Ministry of Higher Education and Oman Accreditation Council

Training Module 03 Handout

Training Module Version Control Table

Version 1

2-5 6

Author G Palermo &

M Carroll

M Carroll

7

M Carroll

8

M Carroll

Date 12-06-2006 20-06-2006

5-09-2006 9-09-2006

Summary of Main Changes

? New Training Module Handout.

? Omitted to bring numbering in line with its PPT partner file version.

? SMARTER altered to be more consistent with local usage. ? Workshops modified to better reflect use of SMARTER as a

diagnostic tool. ? Session Outline reorganized to make different concepts clearer. ? Added disclaimer to section 3. ? Added version control table.

? Slide 6 enlarged. ? Minor text amendments to slides 10 and 14. ? Workshops numbered.

? OAC Logo changed. ? Reordered "Conceptually Understanding KPI" and "A Model for

Designing Good KPI" sections. ? Added planning hierarchy slides 5, 6, 14 and 15. ? Minor text amendments to slides 7 and 10. ? References to `Goals' changed to `Objectives' (for simplicity's sake)

TABLE OF CONTENTS

1 Module Aims ........................................................................................................3 1.1 Intended Participants .....................................................................................3 1.2 Learning Outcomes........................................................................................3

2 Module Slides.......................................................................................................4 2.1 Summary of the Presentation.........................................................................4 2.2 Workshop Activity...........................................................................................6

3 Additional Materials...........................................................................................10 3.1 Business Resources ....................................................................................10 3.2 Government Resources ...............................................................................10 3.3 Institutional Resources .................................................................................10 3.4 Internet Search Tips.....................................................................................11 3.5 Discussion Board Details .............................................................................11

This Training Module is part of the Training Program open to staff of public and private higher education institutions, the Oman Accreditation Council, the Ministry of Higher Education and the Ministry of Manpower. The Training Program is a joint initiative of the Directorate General, Private Universities and Colleges and the Oman Accreditation Council. It aims to raise the capacity and capability of the higher education sector in issues and practices related to assuring and improving quality.

Last updated 5 September 2006

Page 2 of 11

? Ministry of Higher Education and Oman Accreditation Council

Training Module 03 Handout

1 MODULE AIMS

1.1 Intended Participants This Training Module is for anyone in higher education involved in monitoring and reviewing performance across one or many areas of work. Specifically, this is likely to include: ? Management of higher education providers; ? Staff of higher education providers; ? Members of external audit and accreditation panels; and ? Government higher education policy makers.

1.2 Learning Outcomes Key Performance Indicators (KPI) are related to goals or objectives and provide a means for tracking performance against that goal or objective. KPI are used to assess current performance and guide action toward improvement and enhancement. The specific learning outcomes of this training module are: ? Understand the role of KPI in a quality assurance system. ? Understand the attributes of well designed KPI. ? Learn to construct SMARTER KPIs. ? Recognize different types of KPI and the consequences of using different types. ? Be able to constructively critique a KPI against its intended purpose.

Last updated 5 September 2006

Page 3 of 11

? Ministry of Higher Education and Oman Accreditation Council

Training Module 03 Handout

2 MODULE SLIDES

2.1 Summary of the Presentation Note that a version designed for use in presentations is available online and free of charge (.om/enhancement/training). It contains slides that are not included in this printed version (such as model answers to the workshop scenarios).

KPI

Designing Key Performance Indicators

? 2006 Dr Josephine Palermo & Martin Carroll Training Module 03 v7

A Ministry of Higher Education & Oman Accreditation Council joint initiative

Module Objectives

By the end of this module, participants should: 1. Understand the role of KPI in a quality

assurance system. 2. Understand the attributes of well designed KPI. 3. Recognise different types of KPI and the

consequences of using different types. 4. Develop KPI for given goals. 5. Be able to constructively critique a KPI against

its intended purpose.

Slide 1

Session Outline

1. A model for designing good KPI

? KPI in the planning hierarchy ? Objectives, indicators, measures, targets and results

2. Conceptually understanding KPI:

? Why use KPI? ? ADRI and KPI ? Lag and Lead Indicators

3. Tools for critically analysing the design of KPI:

? SMARTER KPI ? Tips for good KPI

4. Workshops

Slide 2

1. A model for designing good KPI

Slide 3

KPI in the Planning Hierarchy

Mission

Goal Objective Strategy

A one-to-many relationship (e.g. one goal may have many strategies)

Indicator Measure

A one-to-one relationship (e.g. one measure has one target)

Target

Result

Slide 5

Slide 4

KPI in the Planning Hierarchy

Mission Goal

Objective Strategy Indicator Measure Target Result

KPI may be thought of as essential elements of the overall planning and monitoring system.

They are derived from the

} College's Mission. A KPI is not a single component, but an integrated collection of components.

Slide 6

Last updated 5 September 2006

Page 4 of 11

? Ministry of Higher Education and Oman Accreditation Council

Training Module 03 Handout

Designing Good KPIs

? Objectives: what are we trying to achieve?

? May be more than one indicator for each objective ? Each objective will have strategies on how to achieve them

? Indicators: what are you going to measure?

? used to assess the present state of progress and to suggest an appropriate course of action.

? Measures: how are you going to measure it?

? can be qualitative or quantitative data related to inputs, processes or outputs.

? Targets: what is the result that you want?

? can be minima targets, stretch targets or a combination.

? Results: what have you actually achieved?

Constructing Good KPIs

? Objective

? High Quality Teaching

? Indicator (one of several)

? Student satisfaction with the teaching they experience

? Measure

? mean student response, per class, to the question, "overall, how satisfied are you with this teacher?"

? on a 1 to 5 Likert-type scale

? Targets

? At least 3.5 on a 1 to 5 scale (where 5 is the best): OR ? Best in class compared with benchmark partners

Slide 7

2. Conceptually understanding KPI

Slide 8

Why use KPIs?

? Key Performance Indicators help us to know whether we are succeeding in our mission.

? They are an important management tool for tracking progress against strategic goals.

? Working with KPIs encourages system thinking.

? in most systems 85% of problems can usually be attributed to the system and 15% to the individual (Juran)

? KPIs direct and prioritise behavior towards achievement of the vision, goals and objectives.

Slide 9 RESULTS

IMPROVEMENT

APPROACH

Slide 10

ADRI and KPIs

? Each phase of the ADRI process can involve KPIs.

? Examples of `Approach' KPIs include:

? Staff training program participation; Student recruitment by target

? Examples of `Deployment' KPIs include:

? Internet uptime; Teaching space usage; Student class attendance

? Examples of `Results' KPIs include:

? Student pass rates; Program accreditation

? Examples of `Improvement' KPIs include:

? Number of review recommendations resolved on time; Policy updates

DEPLOYMENT

Slide 11

Slide 12

Balanced Scorecard KPIs

Lag, Lead & the Planning Hierarchy

? Kaplan & Norton designed a `Balanced Scorecard', with `lag' and `lead' KPIs. Useful for managing large numbers of KPIs in a complex organisation.

? `Lag' indicators tell how well a system performed. Mostly used in ADRI `Results' and `Deployment' phases. Examples include:

? Student employment rates; research publications; number of prizes

? `Lead' indicators help predict system performance. Mostly used in ADRI `Approach' and `Improvement' phases. Examples include:

? Staff training; Investment in advertising; Student-computer ratios

Mission Goal

Objective Strategy Indicator Measure Target Result

Put another way, `lag' KPI focus on the extent to which outcomesbased objectives have been achieved.

`Lead' KPI focus on the extent to which strategies for achieving the objectives are being successfully implemented.

Therefore we can structurally separate lag and lead KPI in the planning hierarchy.

Slide 13

Slide 14

Last updated 5 September 2006

Page 5 of 11

? Ministry of Higher Education and Oman Accreditation Council

Training Module 03 Handout

Lag, Lead & the Planning Hierarchy

Mission

Goal Objective

Indicator Measure

Target

Strategy Indicator Measure Target

There may be separate KPI for objectives and their associated strategies.

Provided that objectives are outcomes-based, and strategies support objectives, then:

Lag KPI set Lead KPI set

3. Tools for critically

analysing KPI

Result

Result

Slide 15

Slide 16

S pecific M easurable A ligned R ealistic T imely E thical R ecorded

Slide 17

Other Tips for Good KPIs

1. Have them! 2. Align them to standards. 3. Test them for validity and reliability. 4. Discuss them and review them. Are they really key?

Don't overestimate their importance! 5. Differentiate between lag and lead indicators. 6. Benchmark them. 7. Do something with them. KPI should lead to change. 8. Keep them simple ? but not too simple! 9. Balance your focus on positive and negative news

(i.e. opportunities for improvement).

Slide 18

Humanise the Process

? Address staff concerns and ideas about performance management.

? Address concerns about the uncertainty of the data.

? Regardless of the topic, keep performance review processes fair and transparent.

? Place performance review within a context of learning.

? Enable a two way dialogue about performance.

? Remember to highlight good news!

Slide 19

Slide 20

2.2 Workshop Activity

The following slides outline the workshop activity. For further information on the workshop, see the online PowerPoint presentation for this Training Module ().

Last updated 5 September 2006

Page 6 of 11

? Ministry of Higher Education and Oman Accreditation Council

Workshop Example

Indicators for Learning and Teaching

Training Module 03 Handout

Objectives

1. Students will learn to solve problems.

2. The program will be viable.

3. The program will be relevant to industry and employers.

Example

? For each of these three objectives, strategies are provided.

? KPIs are then constructed using the model:

indicator measure target

? KPIs are then tested using SMARTER

Workshop Example #1

Indicators for Learning and Teaching

Objective

Strategy

KPIs

Students learn to solve problems

Design and deliver Indicator: courses include learning

problem based

outcomes that relate to problem

curriculum

solving

Measure: number of courses each semester with problem solving learning outcomes included in institutional course guides

Target: 100% of new courses; 100% of all courses by 2010

Design and implement problem based assessment of student learning

Indicator: courses include PBL assessment exercises

Measure: student pass rate/semester

Target: 80% by 2007

Workshop Example #2a

Indicators for Learning and Teaching

Objective

Strategy

KPIs

The program will be viable.

Attract the best students by direct marketing of programs to school leavers.

Indicator: Student satisfaction.

Measure: (a) Student pass rates

(b) Excellent program

THESE

ARE NOT SMARTER

KPI

Target: (a) 100% (b) All staff qualified

Last updated 5 September 2006

Page 7 of 11

? Ministry of Higher Education and Oman Accreditation Council

Training Module 03 Handout

Workshop Example #2b

Indicators for Learning and Teaching

Objective

Strategy

KPIs

The program Attract the best

Indicator: Student program preferences

will be viable. students by direct

by academic performance.

marketing of

Measure:

programs to school leavers.

(a) Percentage of enrolled students who listed the program as their 1st

preference (via Admission Centre)

THESE

(b) Proportion of 1st preference students in national top 10% by academic rank

ARE BETTER!

Target:

(a) More than 50%

(b) At least 20% and up to 40% (assuming four competitor programs in Oman)

Workshop Example #3

Indicators for Learning and Teaching

Objective

Strategy

KPIs

The program will be relevant to industry and employers.

Establish Industry Advisory Boards (IABs) for each program. Invite key employers to be members.

Indicator: Graduate employability rate

Measure: % graduates in full time employment in the industry

Target: above national average

Indicator: Acceptance of program by industry

Measure: (a) Professional accreditation of

program by 2009 (b) Positive feedback from employers

and industry

Target: (a) Professional accreditation passed (b) Positive IAB Program Review

Workshop Instructions

Select your topic

1. Teaching and Learning (done) 2. Student Outcomes 3. Educational Support Resources 4. Research and Research

Training 5. Community Service /

Engagement 6. Organisational Leadership 7. Staffing 8. Institutional Infrastructure 9. Information and Communication

Activity

? Select an area (from the list) that you are currently working on.

? What are two or three main objectives for that area?

? What strategies could you have in place to achieve each of those objectives?

? Construct appropriate KPIs, using:

Indicators Measures Targets

? Check them. Are your KPI SMARTER?

Last updated 5 September 2006

Page 8 of 11

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download