Effectiveness of the Service - Performance Measures ...



.

Introduction

The New Zealand Fire Service Commission publishes a five-year strategic plan that identifies its objectives and how it intends to achieve them. This plan places very strong emphasis on fire safety and fire prevention, and one of the key strategies is a sound promotional plan to educate vulnerable communities. Understanding fire risk in the community and identifying how it may be reduced is the role of the Fire Service. However, observing that there is risk does not necessarily reveal the cause, and even knowing the cause does not necessarily reveal the treatment. To improve the ways in which we tackle risk requires research into causes and treatments, analysis of the data that we have, combined with the data that other people have, and the use of technology to bring imaginative and fresh solutions to long-standing problems.

Baldrige Business Excellence framework

The organisation measures and monitors its performance within the context of the Baldrige Business Excellence framework. This is a formal way, originating in the USA and internationally recognised, for different kinds of organisations to look at all aspects of performance, to identify opportunities for improvement and to strive for continuous change for the better. The framework provides mechanisms for organisations to be scored against an ideal achievement of excellence as a way of monitoring performance improvement.

The business excellence approach challenges organisations to look at their achievements under a series of categories. These categories not only focus on conventional business and financial performance, but also on how planning is undertaken, how organisation is led, workforce satisfaction and how the organisation collects interprets manages and uses its data. It is this last that is the focus of the present paper.

Business excellence does not tell organisations how to do things, or to define what should be important to them. Rather, on the basis of what other highly successful organisations do, it challenges us to show that we are doing what we do in the best way that we can.

Overview of the Strategic Plan

The organisations vision and mission arise out of our legislation and communities’ expectation of what the Fire Service is there to do.

Our Vision: Working with communities to protect what they value

Our Mission: To reduce the incidence and consequence of fire and to provide a professional response to other emergencies

Table 1 Objectives and specific five year goals

|Objective |Specific Goals to 2009 |

|Reduce the Incidence of Fire |Reduce the total number of fires by 20% by June 2006 (from the 2000/01 base year). |

| |Maintain the new base through to June 2009. |

|Reduce the Consequence of Fire for People |Achieve and maintain an avoidable residential structure fire fatality rate per 100,000 |

| |population of less than 0.5. |

| |Reduce the number of injuries as a result of fire by 50% by June 2006 (from the 2001/02 |

| |base year). Maintain the new base level through to June 2009. |

|Reduce the Consequence of Fire for Property |Save at least 80% of the structure in 70% (increase of 5%) of fires in structures by |

| |June 2006. Maintain this level through to June 2009. |

| |Ninety-five percent of all wildfires will be contained within two hours of being |

| |reported. |

| |Annual area burnt by wildfires will be five percent less than the previous ten-year |

| |average upper 25 percentile. |

|Reduce the Consequences of Fire on |Analyse the results of research on communities that are at the highest risk if a fire |

|Communities |occurred in a major employer, historical site or other significant community feature. |

| |Develop performance measures covering this area of work. |

| |Develop partnership plans for the communities most at-risk. |

|Reduce the Consequences of Fire on the |Research how the Commission’s emergency response activities impact on the environments |

|Environment |in the long-term. |

| |Improve processes and develop key performance indicators as a result of the research. |

| |Identify the Commission’s key material and energy resource flows and impacts. |

|Reduce the consequence from non-fire |Goals to be developed in conjunction with the organisations accountable for the outcomes|

|emergencies. |from non-fire emergencies. |

Strategic Priorities

To achieve the goals in Table 1 the following strategic priorities and strategies have been established:

• Strategic Priority 1 - Community fire outcomes: fire prevention and fire safety

• Strategic Priority 2 - Integration of urban and rural service delivery.

• Strategic Priority 3 - Enhanced community security.

• Strategic Priority 4 - Supporting regional, national and international security.

• Strategic Priority 5 - People and partnerships.

• Strategic Priority 6 - Future performance.

•

It is clear these priorities reflect more than just Fire Service business as usual which is covered in priorities 1 and 2. A good deal of interagency operation is covered both locally and nationally in priorities 3 and 4; recruiting a diverse well trained and motivated workforce is covered in strategic priority 5; the organisation’s financial, environmental and IT performance are covered under priority 6.

Each of these priorities gives rise to a set of critical actions that have to be undertaken and monitored each year.

Performance measurement

The challenge addressed in this paper is how the organisation identifies the measures that would show that progress towards the high level goals identified in Table 1 is being achieved. In setting out to identify measures that will be meaningful, we use the process outlined in Figure 1. This process recognises that there is no point in establishing measures where the data required cannot be collected or would be unreliable. We have to be extremely practical. What the process also recognises is that in some instances, the data is so sparse that it may not be practical to monitor it directly.

For example, New Zealand does not have a high fire fatality rate. Monitoring of fire fatalities in itself, in any one year, will not show whether or not the desired downward trend is being achieved. In this instance we do two things. The first is to report our performance not in a single year but over a long number of years where a slow but steady downward trend is observed (Figure 2). We cannot of course show that this downward trend is due to Fire Service promotional campaigns directly. So what we have to do as well is to identify a proxy measure, which provides that link. The measures we use are contained in an annual fire knowledge survey, conducted by specialists by telephone to randomly selected households countrywide.

We assume that if people have greater knowledge of fire danger then they will act in ways that will protect themselves and the result will be reduced fire incidence and reduced fire fatalities. In stating this assumption, it is recognised that on many cases people have the knowledge but do not act, so the fire knowledge survey covers actions that people might have taken, as well as knowledge that they possess. The results of the fire knowledge surveys over 5 years are shown in Table 2.

[pic]

Table 2 Fire knowledge survey results

Awareness

|Knowledge about fire |2004 |2003 |2002 |2001 |2000 |1999 |Good |

|People who estimate a fire can become unsurvivable in five |92% |88% |85% |80% |78% |70% |⎡ |

|minutes or less | | | | | | | |

|Recall of a fire safety message |73% |78% |77% |75% |74% |56% |⎡ |

Changing peoples views

|Changing views |2004 |2003 |2002 |2001 |2000 |1999 |Good |

|How important do you think fire safety is for your household |89% |82% |80% | | | |⎡ |

|% very important | | | | | | | |

|As long as children have been taught to play with lighters | | | | | | | |

|and matches safely it is OK to leave them within their reach |8% |9% |8% |9% |8% |7% |⎢ |

Behaviour change

|Actions taken as a result fire safety message |2004 |2003 |2002 |2001 |2000 |1999 |Good |

|Installed smoke alarms |30% |43% |29% |25% |- |- |⎡ |

|Kept matches away from children |22% |38% |21% |10% |- |- |⎡ |

|Do night safety checks |22% |25% |14% |7% |- |- |⎡ |

Maintenance of change

|Sustained changes to behaviour |2004 |2003 |2002 |2001 |2000 |1999 |Good |

|Households with at least one smoke alarm installed |91% |87% |84% |81% |81% |79% |⎡ |

|Households with four or more smoke alarms installed |23% |24% |25% |24% |19% |- |⎡ |

|Households where children do not have access to lighters or | | | | | | | |

|matches |80% |81% |70% |70% |67% |65% |⎡ |

Fire knowledge survey results for the national promotion plan at-risk groups

| |Natioanl |Low |Renters % |Older |Rural |Minor |

|Extract from our 2004 independent survey results |Sample % |incomes % | |People % |Households|Ethnicitie|

| | | | | |% |s % |

|Taken action as a result of messages |60 |62 |56 |57 |61 |66 |

|Children having access to or using lighters/matches |91 |83 |89 |92 |88 |89 |

|Have at least one smoke alarm installed |91 |90 |88 |91 |90 |86 |

We further have to assume that the publicity campaigns that we embark on will have the effect of making people aware and changing their behaviour. To that end we endeavour to have all our publicity material (programmes, leaflets, TV advertisements) designed by professionals are targeted to the people we need to reach. We have a range of measures that look at the number of these programmes delivered to the target groups. These measures go right down to station and watch level, as discussed in the previous paper.

Similarly, property loss in fire for the community may be avoided with increased fire knowledge. But we also assume that speedy intervention and suppression of fires on the part of the Fire Service using appropriate equipment and well trained, fit staff will help to reduce the property loss consequences of fire. So a number of measures derive from the goals surrounding reduced property loss that relate to dispatch, attendance times, training of operational crews and physical competency assessments.

Less directly, there is an implicit assumption that an organisation with good leaders and satisfied staff will out-perform one that does not have these. So we put in place performance agreements for all managers that require that they deliver on the outcomes that relate to their part of the business in the strategic plan. We also conduct 3600 survey amongst senior mangers, and have started an annual staff survey to see how happy they are. We also aim to have a workforce that reflects the communities it serves and put measures in place to assess workforce diversity.

The business excellence framework in its formalised methods for identifying opportunities for improvement has helped us by suggesting performance measures to us that we might not have considered otherwise.

Benchmarking

Figure 1 includes mention of a benchmarking process. This is there, as a matter of good practice, to ensure that we attempt to compare our performance with other organisations that perform well. In practice, it is not easy to identify appropriate benchmarking partners. Fire Service’s worldwide are constituted within a range of different legislative frameworks with differing responsibilities, priorities and resources. We have retained the notion of benchmarking within our performance measure process, however, with a view that long term we may find organisations with whom we can compare.

Performance management

We manage performance using a feedback process summarised in Figure 3. It will be noted that in order to manage the organisation’s performance, a wide range of data has to be collected form a number of systems, some of which (like the financial system) will be electronic and others (like the fire knowledge survey) will not. Some measures will be collected right down at district level by the Station Management System, and some will be national. This diverse set of data is collated quarterly for reporting to ourselves and to government, but as the quantity of information to be gathered and interpreted has increased we have had to make the move to more sophisticated information management.

[pic]

Performance Monitoring

New Zealand Fire Service has recently purchased a performance management and reporting tool[1] that allows managers to see all of the significant organisational data in one place. We have had this customized to suit our own data and have named it SMART-metrics.

The system can access any data from any database within the organisation. This includes financial data, HR data, incident data, attendance times – if it is stored in a database, it can be accessed. The data collected can be viewed geographically or rolled into a national picture. The data for any individual performance measure is compared to the appropriate target that has been set and is presented to the user in a traffic light format. Thus a manager can have 20 or so individual performance measure on the screen in front of her, and can see at a glance which are performing well (green) and can be ignored, those that that require urgent investigation (red) and those that need to be watched (amber). Figure 4 gives an indication of how this looks for financial performance across the fire regions of New Zealand. All managers have access to the system and anyone can view anyone else’s results.

[pic]

The system has a drill down capability. For example, the Chief Executive notices that one region’s financial performance is giving rise for concern, and can click that region’s financial icon to see why. The problem may be associated with one district or it may be tracked to problems at a particular station. Figure 5 illustrates drill down capability for response times. Further clicking might reveal exactly where the problem is located.

[pic]

Conclusion

The process of establishing performance measures that genuinely reflect improvements in delivery of service top the public is one that provides many challenges. Identifying data that is meaningful and can be collected reliably is only the first of these. As organisations become more complex, the range of data that managers have to monitor becomes wider. Management of data and information becomes crucial, and the use of advanced reporting tools that highlight exceptions in terms of poor performance can help to focus valuable effort.

-----------------------

[1] Cognos Corporate Performance management Software

-----------------------

New Zealand Fire Service

Effectiveness of the Service

Performance Measures & Monitoring

Paula Beever

National Director Fire Risk Management

Abstract

Public bodies are increasingly required to demonstrate that they have performance measures in place and to publicly account for their performance against them. The [

\

ƒ

8Feí./; ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download