Introduction to QOF Exception Reporting



QOF exception reporting for Scotland 2013/14

Published at qof on 30th September 2014

“Questions & Answers” explanatory document

List of questions and topics

Click on a topic or question in the list below to go straight to that section. This document can be read sequentially, or in any order using the hyperlinks provided.

1 Introduction 2

1.1 What is this text about? 2

1.2 What is Exception Reporting? 2

1.3 Which parts of the QOF does exception reporting apply to? 3

2 Some Data Definitions 4

2.1 Registers 4

2.2 Denominators 5

2.3 Numerators 5

2.4 Exclusions 5

2.5 Exceptions 6

3 Presentation of data within this publication 7

3.1 Which QOF indicators are included in this publication? 7

3.2 How are patients counted as exclusions or exceptions? 8

3.3 Why can a breakdown of exceptions by exception reason not be shown? 8

3.4 What numbers do the tables contain? 8

3.5 Why are small numbers important? 10

3.6 Why are some practices not included in these data? 10

4 Commentary on the observed exception reporting rates 12

5 Annex A: Full list of criteria for exception reporting 14

6 Annex B: QOF Business Rules and exception reporting 15

Users of Scotland’s exception reporting data are strongly urged to read the text provided here before accessing the exception reporting tables

Introduction

1 What is this text about?

This text has been prepared to accompany QOF exception reporting data for Scotland for 2013/14. The nature and contents of the QOF are explained at qof and elsewhere, and are not reintroduced here. Although many of the concepts discussed in this text are applicable across the UK, data availability and detailed interpretation may vary from country to country. Those wanting to use exception reporting data from elsewhere in the UK should consult the relevant web pages for that country. Links to QOF reporting for other UK countries are available through qof.

QMAS (the IT system that originally supported the QOF payment process in Scotland between 2004/05 and 2009/10) was not originally designed to deliver information about exception reporting. However, the functionality was revised in time for the 2005/06 QOF year to allow summary information on the levels of exception reporting to be generated. From 2010/11 QMAS has been replaced by the QOF Calculator system which has been designed to deliver information on exception reporting from implementation. Therefore, exception reporting data are available from 2005/06 where relevant to the indicator.

Users of Scotland’s exception reporting data are strongly urged to read the text provided here before accessing the exception reporting tables

2 What is Exception Reporting?

The concept of exception reporting was included in the Quality and Outcomes Framework in order that practices, whilst pursuing the quality improvement agenda, would not be penalised for patient characteristics that were beyond their reasonable control.

When patients are exception reported from an indicator, they are not included in the calculation of a practice’s achievement against that indicator. Reasons why a patient might be exception reported include: - the treatment not being clinically appropriate for the patient, the patient not attending for treatment, the patient refusing to have the treatment, or the patient only having been diagnosed/registered with the practice very recently. A fuller list of the criteria, as agreed within the new General Medical Services contract, with links to QOF exception reporting guidance, are shown in Annex A: Full list of criteria for exception reporting

3 Which parts of the QOF does exception reporting apply to?

To see which parts of the QOF exception reporting applies to, please refer to Which QOF indicators are included in this publication?

Some Data Definitions

The QOF exception reporting data for the QOF need to be understood within the context of other concepts applicable to these data. The following is a summary of key terms and definitions as relevant to exception reporting during 2013/14.

1 Registers

Registers relate to each of the indicator groups within the clinical domain of the QOF, there is only one indicator group which has more than one register in 2013/14. The Heart Failure indicator group has two registers, Heart Failure and a second register refers specifically to Left Ventricular Dysfunction (LVD). In years prior to 2013/14 Depression, had separate registers for indicators referring to a new diagnosis of depression and for conditions assessed for depression but this is no longer the case and only new diagnosis of dementia features in the 2013/14 QOF registers.

In all there are 24 registers relating to clinical indicators for 2013/14 QOF data:

• Asthma

• Atrial Fibrillation

• Cancer

• CHD (Coronary Heart Disease)

• CKD (Chronic Kidney Disease)

• COPD (Chronic Obstructive Pulmonary Disease)

• CVD (Primary Prevention of Cardiovascular Disease)

• Dementia

• Depression: new diagnosis of depression

• Diabetes

• Epilepsy

• Heart Failure

• Hypertension

• Hypothyroidism

• Learning Disabilities

• LVD (Left Ventricular Dysfunction). This is part of the Heart Failure indicator group.

• Mental Health

• Obesity

• Osteoporosis

• Palliative Care

• Peripheral Arterial Disease (PAD)

• Rheumatoid Arthritis

• "Smoking" (conditions assessed for smoking)

• Stroke & TIA (Transient Ischaemic Attack)

Information systems underpinning the QOF (currently QOF Calculator in the case of Scotland) hold, for each practice participating in the QOF, the numbers of patients on each of these registers. For example, there is a register count for the total number of people on each practice list who have CHD. Register totals, as a proportion or percentage of the total practice list size, are used as a measure of raw disease prevalence. Scotland’s practice-level prevalence rates for 2013/14 are published as part of the achievement publication at qof.

2 Denominators

An indicator denominator is the number of patients who can appropriately be included in the measurement, monitoring, treatment, outcome etc. as identified for a specific QOF indicator. Conceptually, denominators for clinical indicators are subsets of the relevant registers, the subsets being arrived at as follows:

• Some patients are EXCLUDED from the denominator due to the indicator definition (see the explanation of Exclusions, below).

• Some patients are EXCEPTION REPORTED from the denominator on the basis of one of the criteria agreed as part of the new GMS contract (see What is Exception Reporting? or Annex A: Full list of criteria for exception reporting for more details)

For the cervical screening indicator CS002 and the Sexual Health indicators, CON002 and CON003, the practice “target population” for that service applies, rather than a “register”. The target population for CS002 is the number of females who are aged 20 to 60 and are thus normally eligible to be called/recalled for cervical smear tests (NB the age bracket is different elsewhere in the UK). The target population for Sexual Health is the number of females aged under 55 who have been prescribed any method of contraception at least once in the last year, or other appropriate interval. Note however that CS002, CON002 and CON003 are not included in this publication – refer to Which QOF indicators are included in this publication? for more information.

For indicators BP001 the total number of persons registered with the practice is the baseline from which the denominator is drawn. Note however that the BP001indicator is not included in this publication – refer to Which QOF indicators are included in this publication?

3 Numerators

An indicator numerator is the number of those in the denominator who meet the specific indicator success criteria – so if the numerator is 16 and the denominator is 20, the practice will have 80% achievement against that indicator.

4 Exclusions

As noted within the explanation of Denominators, above, exclusions are those patients who are included on a particular register, but who for definitional reasons, are not included in a specific QOF indicator denominator. For example, an indicator (and therefore the denominator) may refer only to patients of a specific age group, patients with a specific status (e.g. those who smoke), or patients with a specific length of diagnosis.

To illustrate, the indicator ASTHMA004 refers only to those patients with asthma who are aged between 14 and 19. Therefore, asthma patients who do not fit into this age bracket are excluded from the denominator for this indicator on the basis of the indicator definition.

5 Exceptions

Exceptions are patients who are on the relevant QOF register, and who would ordinarily be included in the indicator denominator (after exclusions for definitional reasons, see above). However, they are excepted from the indicator denominator because they meet at least one of the exception criteria outlined in the What is Exception Reporting? above. Although they may meet more than one of the criteria, they will only be counted against the first one found within their clinical record. The total number of exceptions is the total number of patients excepted, NOT the total number of individual reasons for patients being excepted.

It can be seen from this definitional summary that it is not always appropriate to infer levels of exceptions from the differences between QOF register totals and indicator denominators.

Presentation of data within this publication

1 Which QOF indicators are included in this publication?

Exception reporting applies to those indicators having numerators and denominators, and for which points are awarded on a sliding scale according to the level of achievement reached by the practice. The main group of indicators for which exception reporting is possible are the clinical indicators, apart from the register indicators (e.g. exception reporting does not apply in ASTHMA001 but it does apply to all the other individual Asthma indicators). This is because achievement for register indicators is not awarded on a sliding scale therefore there is no need to have numerators, denominators, exceptions or exclusions for these indicators. Exception reporting data for all relevant clinical indicators are the focus of this publication.

There are currently two clinical indicator groups where there is no exception reporting. These are Obesity and Palliative Care. For Obesity there is only the register indicator within the group; with no numerator or denominator in this indicator, exception reporting is not applicable. With Palliative Care, neither indicator is such that a numerator or denominator exists, therefore exception reporting again does not apply.

The cervical screening indicator CS002 (within the additional services domain) is also subject to exception reporting. However, it is not included in this publication, as CS002 exception reporting data for Scotland are not consistently available. This is because the data for this indicator are sourced from Scotland’s National Cervical Screening System, the numerators, denominators and exceptions required for QOF being entered manually into QOF Calculator.

Also within the additional services domain, the Sexual Health indicators CON002 and CON003 are subject to exception and exclusion reporting against them in QOF Calculator.  However, neither is included in exception reporting tables for this publication.  This is because for both of these indicators (CON003 in particular) exclusions dominate those who were omitted from the denominator. The main reason for exclusion was that the initial prescription (oral or patch contraceptive for CON002 and emergency contraception for CON003) was outwith the relevant timescales for the indicators.  Both indicators have relatively low numbers of exceptions reported against them and therefore they may be of fairly limited interest from a clinical perspective; however these data are available on request.

Exception reporting also applies to the Blood Pressure indicator (BP001) however this indicator is not included in this publication. This is because the exception reporting rates are influenced entirely by recent registrations to the practice, which means that they are of relatively less interest from a clinical or quality improvement viewpoint.

2 How are patients counted as exclusions or exceptions?

The concepts of Exclusions or Exceptions from indicator denominators have been introduced above. However, the distinction between the two is not explicit within QOF Calculator. Therefore, a pragmatic separation of the two has been defined in order to present a clearer picture of exception reporting under the criteria agreed in the GMS contract (see What is Exception Reporting? or Annex A: Full list of criteria for exception reporting). A separate exceptions mapping table (in excel format) shows the detailed exception/exclusion reasons and how they have been grouped into exclusions or exceptions, for the purposes of this and the corresponding exception reporting publications from elsewhere in the UK.

3 Why can a breakdown of exceptions by exception reason not be shown?

The potential for publishing exception reporting data is limited due to the way in which the data are recorded. It is possible to use data to obtain the total number of patients excepted from a specific indicator. However, the way in which the data are supplied to means that it is not possible to break exceptions down by each of the individual criteria outlined in Annex A: Full list of criteria for exception reporting nor to more detailed reasons such as individual Read Codes (clinical codes) entered for exceptions.

There are two reasons for this:

1) Any individual patient can be associated with more than one of the exception criteria, but only one such reason needs to be identified during the process of submitting data from GP clinical systems to QOF Calculator in order to except this patient from inclusion in the indicator denominator.

2) The testing of GP clinical system compliance with QOF Business Rules (such that they can supply data appropriately to QOF Calculator and other national QOF systems) is primarily focused on ensuring that data values used for achievement calculations are accurate for payment purposes. Any testing of the order of sequencing (i.e. the order whereby systems check for different exception codes or criteria) is secondary to this purpose. Different GP clinical information systems may follow different sequencing without this impacting on payment accuracy.

Clearly therefore, the QOF Calculator system cannot guarantee to count the total number of individual reasons for a patient being excepted from an indicator denominator; what it counts are the numbers of patients excepted, according to the first reason found when the patient records are queried. Thus, an analysis of the breakdown of exception reporting according to each individual reason is impossible.

4 What numbers do the tables contain?

All analyses presented in this publication are at individual indicator level.

Tables for individual practices, and for Community Health Partnerships (CHPs/CHCPs) contain the following items of information for each indicator:

• Indicator numerator (here labelled as item “a” for purposes below)

• Indicator denominator (item “b”)

• number of patients excluded from the denominator (item “c”)

• number of patients excepted from the denominator (item “d”)

• percentage of patients excluded, calculated as c /( b + c + d) x 100 (referred to for short as the “exclusions rate”)

• percentage of patients excepted, calculated as d /( b + d) x 100 (the “exceptions rate”)

Tables at Scotland and NHS Board level contain equivalent information, plus the following additional items to illustrate the distribution of exception-reporting rates:

• exceptions rate - 10th percentile

• exceptions rate - 50th percentile (median)

• exceptions rate - 90th percentile

All three of these percentiles are shown at Scotland level. At NHS Board level, percentiles are shown subject to there being sufficient numbers of practices for the percentile calculations not to be overly unstable. For Boards having fewer than 50 practices included in the tables, median rates are shown but the 10th and 90th percentiles are not. No percentiles are shown for the Island Boards as there are very few practices in these areas and the calculation of these percentiles would not be robust.

Percentiles are worked out by ordering a series of values in ascending order and identifying the value that occupies the specified place in the list. For example, if the exception reporting rates for 1000 practices in Scotland are ordered from lowest to highest, the value for the 100th practice is the 10th percentile, and the value for the 900th practice is the 90th percentile. Meanwhile, the value in the middle – the 500th practice in this case – is the 50th percentile, commonly known as the median.

The median is a robust measure of the “average” exception reporting rate as (unlike the mean) it is not influenced disproportionately by small numbers of very low or very high rates. The 10th and 90th percentile values are included to illustrate the spread of exception reporting rates amongst the majority of practices. In conjunction with medians they also demonstrate something of the variation between different individual indicators in terms of relative levels of exception reporting. Minimum and maximum rates are not shown as they are highly influenced by very small numbers, as exampled below, and are therefore unreliable as measures of exception reporting.

5 Why are small numbers important?

It is crucial to note that observed exception reporting rates, particularly at practice level, are highly influenced by small numbers. Some apparently very high, or very low, rates may be so because only a very few patients are involved. This is particularly likely to affect small practices and/or instances where indicators relate to a relatively small number of patients in the practice. An illustrative example follows:

Practice A has recorded QOF achievement data for indicator X

They have 3 patients on the register for this condition

• 2 patients are exception-reported from the denominator

• 1 patient remains in the indicator denominator

• The effective exception rate thus appears to be 2 / (2+1) x 100 = 66.7%

• However, had only 1 patient been exception-reported (and thus 2 patients included in the denominator), the rate would instead be 1 / (1+2) x 100 = 33.3%.

This example shows that there can be a very large swing in apparent rates on the basis of very small numbers of patients. Therefore, it is important that rates are viewed in conjunction with the absolute numbers of patients that are included in the tabulations.

6 Why are some practices not included in these data?

The exception reporting tables presented in this publication include data from a large majority of the practices participating in the QOF during 2013/14. However, data for some practices, or specific indicators within practices, have been omitted from the tables. The broad reasons for excluding certain records from the calculations are listed below.

1) There are no exclusions/exceptions data available for the practice. For some practices, no exclusions/exceptions data are available within QOF Calculator for 2013/14. As it is therefore not possible to calculate exception reporting rates for any of their indicators, these practices are excluded from all tables.

A complete absence of exclusions/exception reporting data can occur where QOF data have been submitted manually to QOF Calculator rather than by electronic transfer of data; exclusions/exception reporting data are not required as part of manual submission of QOF achievement data. Alternatively, this circumstance can arise when there have been technical issues with the electronic submission of achievement data from practice clinical systems to QOF Calculator.

2) The practice has no exclusions/exceptions or achievement data available for the individual indicator. For some individual indicators within practices, there are no denominators, exclusions or exceptions available, so it is not possible to calculate an exception rate. This can occur if the practice did not attempt to achieve against the indicator (as participation in each element of the standard QOF is voluntary) or if it was not able to record any achievement data against that particular indicator e.g. small practice size or small subgroup of patients.

3) The Exclusions/exceptions data are not consistent with other elements of achievement data. For some individual indicators within practices, it may not be possible to calculate valid exception reporting rates if specific elements of their end of year achievement data have been manually adjusted following the end of the QOF year (adjustments can occur when practices and NHS Boards undertake accuracy checks of the data during the process of sign-off and payment verification). Denominators, numerators and registers can be adjusted manually, but it is not possible within QOF Calculator to similarly adjust the exclusions/exceptions data. Where data have been adjusted, denominators may no longer be consistent with the exception figures.

Where exclusions/exceptions do not tie in with other elements of achievement data, practices are excluded from the exception reporting tables for the affected indicator(s). However, these practices are included in tables for non-affected indicators.

4) The extract of achievement data for some practices may not be definitive. Achievement data for 2013/14, published in September 2014 at qof, are based on a data extract taken from QOF Calculator in June 2014. As noted on the achievement web pages and the achievement tables, data are included only for those practices whose QOF achievement had been signed off, or otherwise indicated as final, in time for the publication. Data to support the exception reporting section of the publication are also based on this extract. As detailed in point 3 above, data are not included if the exceptions/exclusions were not consistent with the achievement data. Because of this it would be inappropriate to report on the exception data when achievement has yet to be finalised.

Commentary on the observed exception reporting rates

Levels of exception reporting for Scotland as a whole varied considerably between the clinical indicators, ranging from 0.4% to 36.2%.

In general, indicators with the lowest levels of exception reporting include those that involve recording information in the patient records (e.g. blood pressure measurement, recording smoking status, offering smoking cessation advice, patients with hypothyroidism who have received thyroid function tests), whereas the highest are seen in those that involve clinical treatments, influenza immunisation, or clinically measurable outcomes.

The lowest level of exception reporting overall (0.4%) was seen in the indicator SMOKE001 which is the recording of the percentage of patients aged 15 or over with a status of smoking preceding 24 months. This indicator was previously within the organisational domain.

The next two lowest exception rates were for SMOKE002 (0.9%), SMOKE004 (1.0%), which are complementary to SMOKE001.

THY002 (1.5%) was the indicator with the next lowest level of overall exception reporting. This indicator records the percentage of patients with hypothyroidism, on the register, with thyroid function tests recorded in the preceding 12 months.

The indicator with the highest level of exception reporting overall in 2013/14 was EP003 (36.2%). This indicator was introduced in 2011/12 and also had the highest level of exception reporting in previous two years (40.6% & 40.1%). This indicator measures the percentage of women aged 18 and over who have not attained the age of 55 years who are taking antiepileptic drugs who have a record of information and counselling about contraception, conception and pregnancy in the preceding 12 months.

The second highest level of exception reporting is for CHD006(S) (30.8%). This indicator records the percentage of patients with a history of myocardial infarction (on or after 1 April 2011) currently treated with an ACE (or ARB if ACE-I intolerant), aspirin or an alternative anti- platelet therapy, beta-blocker and statin.

CVD-PP001, had the third highest level of exception reporting (30.7%). This indicator measures those patients with a new diagnosis of hypertension aged 30 or over and who have not attained the age of 75, recorded between the preceding 1 April to 31 March (excluding those with preexisting CHD, diabetes, stroke and/or TIA), who have a recorded CVD risk assessment score (using an assessment tool agreed with the NHS CB) of ≥20% in the preceding 125 months: the percentage who are currently treated with statins.

There was substantial variation between practices in the level of exception reporting within many of the individual indicators. Whilst the highest variability tended to be seen in the indicators that relate to clinical outcomes, it was also more pronounced for indicators where there were often small numbers of patients involved at practice level (e.g. the Mental Health indicators). An example of the influence that small numbers can have on rates, and therefore why the figures should be treated with caution, is shown above in Why are small numbers important?

Annex A: Full list of criteria for exception reporting

The concept of Exception reporting has been introduced earlier in What is Exception Reporting?. This annex provides further information.

Within the General Medical Services, contract, nine criteria have been agreed for exception reporting, i.e. nine types of reason why a patient might not be included in calculating a practice’s achievement against an individual indicator. They are as follows:

a. Patients who have been recorded as refusing to attend review, who have been invited on at least three occasions during the preceding twelve months.

b. Patients for whom it is not appropriate to review the chronic disease parameters due to particular circumstances, e.g. terminal illness or extreme frailty.

c. Patients newly diagnosed within the practice, or who have recently registered with the practice, who should have measurements made within three months and delivery of clinical standards within nine months, e.g. blood pressure or cholesterol measurements within target levels.

d. Patients who are on maximum tolerated doses of medication whose levels remain sub-optimal.

e. Patients for whom prescribing a medication is not clinically appropriate, e.g. those who have an allergy, another contraindication or have experienced an adverse reaction.

f. Where a patient has not tolerated medication.

g. Where a patient does not agree to investigation or treatment (informed dissent), and this has been recorded in their medical records.

h. Where the patient has a supervening condition, which makes treatment of their condition inappropriate, e.g. cholesterol reduction where the patient has liver disease.

i. Where an investigative or secondary care service is unavailable.

The list above is taken from Section 8 of the Scottish Quality and Outcomes Framework 2013/14 Guidance for NHS Boards and GP practices.

Annex B: QOF Business Rules and exception reporting

QMAS (the IT system that originally supported the QOF payment process in Scotland) was not originally designed to deliver information about exception reporting. However, the functionality was revised in time for the 2005/06 QOF year to allow summary information on the levels of exception reporting to be generated. The current system, QOF Calculator, was designed to deliver exception reporting information from implementation in 2010/11. Therefore 2013/14 is the 9th year for which publication of exception reporting data has been possible.

The selection criteria (e.g. dates or clinical codes such as Read codes) by which exclusions/exceptions data are identified for a given indicator group (e.g. CHD) is specified in detail within the QOF Business Rules. The rules also specify how registers, denominators and numerators are identified. The version of the Business Rules applicable at the end of the 2013/14 QOF year was version 27.0; copies of these rules are linked to below, and from the exception reporting page on ISD’s QOF website, qof. It is crucial to note that the v27.0 rules DO NOT apply to years previous to or subsequent to 2013/14. As the QOF develops, revised sets of business rules are published at the Primary Care Contracting website

Some reasons for exclusions/exceptions apply across all the indicators for a given condition, whereas others apply to specific indicators. The first two criteria outlined in Annex A: Full list of criteria for exception reporting (patients refusing to attend review; or patients for whom treatment is not appropriate) would apply to all the indicators in that area. For example, where there were 100 patients on a practice’s CHD register at the QOF year end, and where four patients failed to respond to invitations to review, and where one patient was terminally ill, the denominator for reporting would be 95 patients. This would apply to all indicators in the CHD indicator set. Meanwhile, the other criteria would be applied to specific indicators, depending on the relevance of the exception criteria to those indicators.

Note that if clinical care as outlined in the indicators is performed, then regardless of any valid exception criteria present in the patient records, the patient will be included in the indicator denominator and numerator.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download