Introduction to QOF Exception Reporting



QOF exception reporting for Scotland 2014/15

Published at qof on 13th October 2015

“Questions & Answers” explanatory document

List of questions and topics

Click on a topic or question in the list below to go straight to that section. This document can be read sequentially, or in any order using the hyperlinks provided.

1 Introduction 2

1.1 What is this text about? 2

1.2 What is Exception Reporting? 2

1.3 Which parts of the QOF does exception reporting apply to? 3

2 Some Data Definitions 4

2.1 Registers 4

2.2 Denominators 5

2.3 Numerators 5

2.4 Exclusions 5

2.5 Exceptions 6

3 Presentation of data within this publication 7

3.1 Which QOF indicators are included in this publication? 7

3.2 How are patients counted as exclusions or exceptions? 7

3.3 Why can a breakdown of exceptions by exception reason not be shown? 8

3.4 What numbers do the tables contain? 8

3.5 Why are small numbers important? 10

3.6 Why are some practices not included in these data? 10

4 Commentary on the observed exception reporting rates 12

5 Annex A: Full list of criteria for exception reporting 13

6 Annex B: QOF Business Rules and exception reporting 14

Users of Scotland’s exception reporting data are strongly urged to read the text provided here before accessing the exception reporting tables

Introduction

1 What is this text about?

This text has been prepared to accompany QOF exception reporting data for Scotland for 2014/15. The nature and contents of the QOF are explained at qof and elsewhere, and are not reintroduced here. Although many of the concepts discussed in this text are applicable across the UK, data availability and detailed interpretation may vary from country to country. Those who want to use exception reporting data from elsewhere in the UK should consult the relevant web pages for that country. Links to QOF reporting for other UK countries are available through qof/About-QOF.

QMAS (the IT system that originally supported the QOF payment process in Scotland between 2004/05 and 2009/10) was not originally designed to deliver information about exception reporting. However, the functionality was revised in time for the 2005/06 QOF year to allow summary information on the levels of exception reporting to be generated. From 2010/11 QMAS has been replaced by the QOF Calculator system which has been designed to deliver information on exception reporting from implementation. Therefore, exception reporting data are available from 2005/06 where relevant to the indicator.

Users of Scotland’s exception reporting data are strongly urged to read the text provided here before accessing the exception reporting tables

2 What is Exception Reporting?

The concept of exception reporting was included in the Quality and Outcomes Framework in order that practices, whilst pursuing the quality improvement agenda, would not be penalised for patient characteristics that were beyond their reasonable control.

When patients are exception reported from an indicator, they are not included in the calculation of a practice’s achievement against that indicator. Reasons why a patient might be exception reported include: - the treatment not being clinically appropriate for the patient, the patient not attending for treatment, the patient refusing to have the treatment, or the patient only having been diagnosed/registered with the practice very recently. A fuller list of the criteria, as agreed within the new General Medical Services contract, with links to QOF exception reporting guidance, are shown in Annex A: Full list of criteria for exception reporting

3 Which parts of the QOF does exception reporting apply to?

To see which parts of the QOF exception reporting applies to, please refer to Which QOF indicators are included in this publication?

Some Data Definitions

The QOF exception reporting data for the QOF need to be understood within the context of other concepts applicable to these data. The following is a summary of key terms and definitions as relevant to exception reporting during 2014/15.

1 Registers

Registers relate to each of the indicator groups within the clinical domain of the QOF. Before 2014/15, there were separate registers for the QOF depression indicator referring to a “new diagnosis of depression” and for “conditions assessed for depression but this is no longer the case. No new registers were introduced in 2014/15.

There are currently 22 registers relating to either clinical or public health indicators for 2014/15 QOF data:

Clinical:

• Asthma

• Atrial Fibrillation

• Cancer

• CHD (Coronary Heart Disease)

• CKD (Chronic Kidney Disease)

• COPD (Chronic Obstructive Pulmonary Disease)

• Dementia

• Depression: new diagnosis of depression

• Diabetes

• Heart Failure

• Hypertension

• Mental Health

• Osteoporosis

• Palliative Care

• Peripheral Arterial Disease (PAD)

• Rheumatoid Arthritis

• Stroke & TIA (Transient Ischaemic Attack)

Public Health:

• “Blood Pressure” (conditions assessed for high blood pressure)

• CVD-PP (Cardiovascular Disease – Primary Prevention)

• Cervical Screening

• Contraception

• "Smoking" (conditions assessed for smoking)

Information systems underpinning the QOF (currently QOF Calculator in the case of Scotland) hold, for each practice participating in the QOF, the numbers of patients on each of these registers. For example, there is a register count for the total number of people on each practice list who have CHD. Register totals, as a proportion or percentage of the total practice list size, are used as a measure of raw disease prevalence. Scotland’s practice-level prevalence rates for 2014/15 are published as part of the achievement publication at qof.

2 Denominators

A denominator is the number of patients who can appropriately be included in the measurement, monitoring, treatment, outcome etc. as identified for a specific QOF indicator. Conceptually, denominators for clinical indicators are subsets of the relevant registers, the subsets being arrived at as follows:

• Some patients are EXCLUDED from the denominator due to the indicator definition (see the explanation of Exclusions, below).

• Some patients are reported as EXCEPTIONS from the denominator on the basis of one of the criteria agreed as part of the new GMS contract (see What is Exception Reporting? or Annex A: Full list of criteria for exception reporting for more details).

For the Cervical Screening indicator CS002(S) and the Contraception indicator CON003(S), the practice “target population” for that service applies, rather than a “register”. The target population for CS002(S) is the number of females who are aged 20 to 60 and are thus normally eligible to be called/recalled for cervical smear tests (NB the age bracket is different elsewhere in the UK). The target population for CON003(S) is the number of females aged under 55 who have been prescribed any method of contraception at least once in the last year, or other appropriate interval.

For Blood Pressure indicator BP001(S) the total number of persons registered with the practice is the baseline from which the denominator is drawn.

3 Numerators

An indicator numerator is the number of those in the denominator who meet the specific indicator success criteria. Using the BP001(S) indicator as an example, if a practice has a total of 20 people recorded with hypertension and 16 of those people meet the specific success criteria, the numerator is 16 and the denominator is 20. Therefore, the practice will have 80% achievement against the BP001(S) indicator.

4 Exclusions

Exclusions are those patients who are included on a particular register, but whom for definitional reasons, are not included in a specific QOF indicator denominator. For example, an indicator (and therefore the denominator) may refer only to patients of a specific age group, patients with a specific status (e.g. those who smoke), or patients with a specific length of diagnosis. To illustrate, the indicator AST004(S) refers only to those patients with asthma who are aged between 14 and 19. Therefore, asthma patients who do not fit into this age bracket are excluded from the denominator for this indicator on the basis of the indicator definition.

5 Exceptions

Exceptions are patients who are on the relevant QOF register, and who would ordinarily be included in the indicator denominator (after exclusions for definitional reasons, see above). However, they are excepted from the indicator denominator because they meet at least one of the exception criteria outlined in the What is Exception Reporting? above. Although they may meet more than one of the criteria, they will only be counted against the first one found within their clinical record. The total number of exceptions is the total number of patients excepted, NOT the total number of individual reasons for patients being excepted.

It can be seen from this definitional summary that it is not always appropriate to infer levels of exceptions from the differences between QOF register totals and indicator denominators.

Presentation of data within this publication

1 Which QOF indicators are included in this publication?

Exception reporting applies to indicators with numerators and denominators, which have points awarded on a ‘sliding scale’ according to the level of achievement reached by the practice. The main group of indicators for which exception reporting is possible are the clinical and public health indicators. Register indicators, which are not awarded on a ‘sliding scale’, do not need to have numerators, denominators, exceptions or exclusions to measure achievement. Exception reporting data for all relevant clinical and public health indicators are the focus of this publication.

For the current QOF clinical indicator of Palliative Care, there is no exception reporting. For this group, neither a numerator or denominator exist, therefore exception reporting again does not apply.

By contrast, the Cervical Screening indicator CS002(S) is also subject to exception reporting. It should be noted that exception reporting data for Scotland are not consistently available for this indicator. This is because the data for this indicator are sourced from Scotland’s National Cervical Screening System, the numerators, denominators and exceptions required for QOF being entered manually into QOF Calculator.

The Contraception indicators CON003(S) is subject to exception and exclusion reporting.  However, most of the patients are recorded as exclusions and omitted from the denominator. This indicator has relatively low numbers of exceptions reported against it and therefore they may be of fairly limited interest from a clinical perspective.

Exception reporting also applies to the Blood Pressure indicator BP001(S). However exception reporting rates are influenced entirely by recent registrations to the practice, which means that they are of relatively less interest from a clinical or quality improvement viewpoint.

2 How are patients counted as exclusions or exceptions?

The concepts of Exclusions or Exceptions from indicator denominators have been introduced above. However, the distinction between the two is not explicit within QOF Calculator. Therefore, a pragmatic separation of the two concepts has been defined to present a clearer picture of exception reporting under the criteria agreed in the GMS contract (see What is Exception Reporting? or Annex A: Full list of criteria for exception reporting). A separate exceptions mapping table (in excel format) shows the detailed exception/exclusion reasons and how they have been grouped into exclusions or exceptions, for the purpose of this publication and comparison with QOF publications from other parts of the UK.

3 Why can a breakdown of exceptions by exception reason not be shown?

The potential for publishing exception reporting data is limited due to the way in which the data are recorded. It is possible to use data to obtain the total number of patients excepted from a specific indicator. However, the way in which the data are supplied means that it is not possible to provide a breakdown by each of the individual criteria outlined in Annex A: Full list of criteria for exception reporting nor to more detailed reasons for the exceptions such as individual Read Codes (clinical codes).

There are two reasons for this:

1) Any individual patient can be associated with more than one of the exception criteria, but only one such reason needs to be identified during the process of submitting data from GP clinical systems to QOF Calculator in order to except this patient from inclusion in the indicator denominator.

2) The testing of GP clinical system compliance with QOF Business Rules (such that they can supply data appropriately to QOF Calculator and other national QOF systems) is primarily focused on ensuring that data values used for achievement calculations are accurate for payment purposes. Any testing of the order of sequencing (i.e. the order whereby systems check for different exception codes or criteria) is secondary to this purpose. Different GP clinical information systems may follow different sequencing without this impacting on payment accuracy.

The QOF Calculator system counts are the numbers of patients excepted, according to the first reason found when the patient records are queried. Therefore the system does not count the total number of individual reasons for a patient being excepted from an indicator denominator. Thus, an analysis of the breakdown of exception reporting according to each individual reason is impossible.

4 What numbers do the tables contain?

All analyses presented in this publication are at individual indicator level.

Tables for individual practices and for Community Health Partnerships (CHPs/CHCPs) contain the following items of information for each indicator:

• Indicator numerator (here labelled as item “a” for purposes below)

• Indicator denominator (item “b”)

• number of patients excluded from the denominator (item “c”)

• number of patients excepted from the denominator (item “d”)

• percentage of patients excluded, calculated as c /( b + c + d) x 100 (the “exclusions rate”)

• percentage of patients excepted, calculated as d /( b + d) x 100 (“exceptions rate”)

Tables at Scotland and NHS Board level contain equivalent information, plus the following additional items to illustrate the distribution of exception reporting rates:

• exceptions rate - 10th percentile

• exceptions rate - 50th percentile (median)

• exceptions rate - 90th percentile

All three percentiles are shown at Scotland level. At NHS Board level, percentiles are shown subject to sufficient numbers of practices for the percentile calculations to provide reliable estimates. For Boards with fewer than 50 practices included in the tables, median rates are shown but the 10th and 90th percentiles are not shown. No percentiles are shown for the Island Boards as there are few practices in these areas and the calculation of these percentiles would not be reliable.

Percentiles are calculated by ordering a series of values in ascending order and identifying the value that occupies the specified place in the list. For example, if the exception reporting rates for 1000 practices in Scotland are ordered from lowest to highest, the value for the 100th practice is the 10th percentile, and the value for the 900th practice is the 90th percentile. Meanwhile, the value in the middle – the 500th practice in this case – is the 50th percentile, commonly known as the median.

The median is a robust measure of the “average” exception reporting rate, unlike the mean, as it is not influenced disproportionately by small numbers of very low or very high rates. The 10th and 90th percentile values are included to illustrate the spread of exception reporting rates amongst the majority of practices. In conjunction with medians they also demonstrate the variation between different individual indicators in terms of relative levels of exception reporting. Minimum and maximum rates are not shown as they may be influenced by very small numbers and are therefore unreliable as measures of exception reporting, as exampled below.

5 Why are small numbers important?

It is crucial to note that observed exception reporting rates, particularly at practice level, are influenced by small numbers. Some apparently ‘very high’, or ‘very low’, rates may be so because only a a few patients are involved. This is particularly likely to affect small practices and/or instances where indicators relate to a relatively small number of patients in the practice. An illustrative example follows:

Practice A has recorded QOF achievement data for indicator X

They have 3 patients on the register for this condition

• 2 patients are exception-reported from the denominator

• 1 patient remains in the indicator denominator

• The effective exception rate thus appears to be 2 / (2+1) x 100 = 66.7%

• However, had only 1 patient been exception-reported (and thus 2 patients included in the denominator), the rate would instead be 1 / (1+2) x 100 = 33.3%.

This example shows that there can be a wide variation in apparent rates on the basis of very small numbers of patients. Therefore, it is important that rates are viewed in conjunction with the absolute numbers of patients that are included in the calculations.

6 Why are some practices not included in these data?

The exception reporting tables presented in this publication include data from a large majority of the practices participating in the QOF during 2014/15. However, data for some practices, or specific indicators within practices, have been omitted from the tables. The broad reasons for excluding certain records from the calculations are listed below.

1) There are no exclusions/exceptions data available for the practice. For some practices, no exclusions/exceptions data are available within QOF Calculator for 2014/15. As it is therefore not possible to calculate exception reporting rates for any of their indicators, these practices are excluded from all tables.

A complete absence of exclusions/exception reporting data can occur where QOF data have been submitted manually to QOF Calculator rather than by electronic transfer of data; exclusions/exception reporting data are not required as part of manual submission of QOF achievement data. Alternatively, this circumstance can arise when there have been technical issues with the electronic submission of achievement data from practice clinical systems to QOF Calculator.

In the 2014/15 QOF report, no practices were excluded for this reason; all practices submitted their data and no electronic or manual errors occurred.

2) The practice has no exclusions/exceptions or achievement data available for the individual indicator. For some individual indicators within practices, there are no denominators, exclusions or exceptions available, so it is not possible to calculate an exception rate. This can occur if the practice did not attempt to achieve against the indicator (as participation in each element of the standard QOF is voluntary) or if it was not able to record any achievement data against that particular indicator e.g. small practice size or small subgroup of patients.

3) The Exclusions/exceptions data are not consistent with other elements of achievement data. For some individual indicators within practices, it may not be possible to calculate valid exception reporting rates if specific elements of their end of year achievement data have been manually adjusted following the end of the QOF year (adjustments can occur when practices and NHS Boards undertake accuracy checks of the data during the process of sign-off and payment verification). Denominators, numerators and registers can be adjusted manually, but it is not possible within QOF Calculator to similarly adjust the exclusions/exceptions data. Where data have been adjusted, denominators may no longer be consistent with the exception figures.

Where exclusions/exceptions do not tie in with other elements of achievement data, practices are excluded from the exception reporting tables for the affected indicator(s). However, these practices are included in tables for non-affected indicators.

4) The extract of achievement data for some practices may not be definitive. Achievement data for 2014/15, published in October 2015 at qof, are based on a data extract taken from QOF Calculator in June 2015. As noted on the achievement web pages and the achievement tables, data are included only for those practices whose QOF achievement had been signed off, or otherwise indicated as final, in time for the publication. Data to support the exception reporting section of the publication are also based on this extract. As detailed in point 3 above, data are not included if the exceptions/exclusions were not consistent with the achievement data. Because of this it would be inappropriate to report on the exception data when achievement has yet to be finalised.

Commentary on the observed exception reporting rates

Levels of exception reporting for Scotland as a whole varied considerably between the clinical indicators, ranging from 1.5% to 34.4%. For all indicators, exception reporting varied from 0.5% to 98.0%.

In general, indicators with the lowest levels of exception reporting include those that involve recording information in the patient records (e.g. blood pressure measurement, recording smoking status), whereas the highest levels involve clinical treatments or clinically measurable outcomes.

The lowest exception rate for a Public Health indicator for Blood Pressure BP001(S) (0.5%), the lowest indicator rate out of any clinical or Public Health domains, followed by the Public Health Smoking indicators of SMOK002(S) (0.9%) and SMOK004(S) (1.0%). The Public Health indicator of CVD-PP001(S) had by far the highest level of exception reporting (98.0%). This indicator measures those patients with a new diagnosis of Hypertension aged 30 or over and who have not attained the age of 75, recorded between the preceding 1 April to 31 March (excluding those with pre-existing CHD, diabetes, stroke and/or TIA), who have a recorded CVD risk assessment score (using an assessment tool agreed with the NHS CB) of ≥20% in the preceding 15 months: the percentage who are currently treated with statins.

The lowest level of exception reporting for a clinical indicator (1.4%) was seen in the COPD005(S) which is the percentage of patients with COPD dyspnoea grade greater than 3 at any time in the preceding 15 months with a record of oxygen saturation value within the preceding 3 months. The clinical indicator with the highest level of exception reporting overall in 2014/15 was DM014 (34.4%). This indicator measures the percentage of patients newly diagnosed with Diabetes, on the register, in the preceding 1 April to 31 March who have a record of being referred to a structured education programme within 9 months after entry on to the Diabetes register. The second highest level of exception reporting is for CHD006(S) (27.3%). This indicator records the percentage of patients with a history of Myocardial Infarction (on or after 1 April 2011) currently treated with an ACE (or ARB if ACE I intolerant), aspirin or an alternative anti- platelet therapy, beta-blocker and statin. The level of exception reporting for this indicator was similarly high in the 2013/14 QOF (30.8%).

There was substantial variation between practices in the level of exception reporting within many of the individual indicators. Whilst the highest variability tended to be seen in the indicators that relate to clinical outcomes, it was also more pronounced for indicators where there were few patients involved at practice level (e.g. the Mental Health indicators). An example of the influence that small numbers can have on rates, and therefore why the figures should be treated with caution, is shown above in Why are small numbers important?

Annex A: Full list of criteria for exception reporting

The concept of Exception reporting has been introduced earlier in What is Exception Reporting?. This annex provides further information.

Within the General Medical Services contract, nine criteria have been agreed for exception reporting, i.e. nine types of reason why a patient might not be included in calculating a practice’s achievement against an individual indicator. They are as follows:

a. Patients who have been recorded as refusing to attend review, who have been invited on at least three occasions during the preceding twelve months.

b. Patients for whom it is not appropriate to review the chronic disease parameters due to particular circumstances, e.g. terminal illness or extreme frailty.

c. Patients newly diagnosed within the practice, or who have recently registered with the practice, who should have measurements made within three months and delivery of clinical standards within nine months, e.g. blood pressure or cholesterol measurements within target levels.

d. Patients who are on maximum tolerated doses of medication whose levels remain sub-optimal.

e. Patients for whom prescribing a medication is not clinically appropriate, e.g. those who have an allergy, another contraindication or have experienced an adverse reaction.

f. Where a patient has not tolerated medication.

g. Where a patient does not agree to investigation or treatment (informed dissent), and this has been recorded in their medical records.

h. Where the patient has a supervening condition, which makes treatment of their condition inappropriate, e.g. cholesterol reduction where the patient has liver disease.

i. Where an investigative or secondary care service is unavailable.

The list above is taken from Section 8 of the Scottish Quality and Outcomes Framework 2014/15 Guidance for NHS Boards and GP practices.

Annex B: QOF Business Rules and exception reporting

QMAS (the IT system that originally supported the QOF payment process in Scotland) was not originally designed to deliver information about exception reporting. However, the functionality was revised in time for the 2005/06 QOF year to allow summary information on the levels of exception reporting to be generated. The current system, QOF Calculator, was designed to deliver exception reporting information from implementation in 2010/11. Therefore 2014/15 is the 10th year for which publication of exception reporting data has been possible.

The selection criteria (e.g. dates or clinical codes such as Read codes) by which exclusions/exceptions data are identified for a given indicator group (e.g. CHD) is specified in detail within the QOF Business Rules. The rules also specify how registers, denominators and numerators are identified. The version of the Business Rules applicable at the end of the 2014/15 QOF year was version 30.0; copies of these rules are linked to below, and from the exception reporting page on ISD’s QOF website, qof. It is crucial to note that the v30.0 rules DO NOT apply to years previous to or subsequent to 2014/15. As the QOF develops, revised sets of business rules are published at the Primary Care Contracting website.

Some reasons for exclusions/exceptions apply across all the indicators for a given condition, whereas others apply to specific indicators. The first two criteria outlined in Annex A: Full list of criteria for exception reporting (patients refusing to attend review; or patients for whom treatment is not appropriate) would apply to all the indicators in that practice area. For example, where there were 100 patients on a practice’s CHD register at the QOF year end, and where four patients failed to respond to invitations to review, and where one patient was terminally ill, the denominator for reporting would be 95 patients. This would apply to all indicators in the CHD indicator set. Meanwhile, the other criteria would be applied to specific indicators, depending on the relevance of the exception criteria to those indicators.

Note that if clinical care as outlined in the indicators is performed, then regardless of any valid exception criteria present in the patient records, the patient will be included in the indicator denominator and numerator.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download