Application Form - NICE | The National Institute for ...



NHS DigitalIndicator Supporting DocumentationIAP00137 People who have a follow-up assessment between 4 and 8 months after initial admission for strokeFIELDCONTENTSIAP CodeIAP00137TitlePeople who have a follow-up assessment between 4 to 8 months after initial admission for strokePublished byDepartment of Health and Social CareReporting periodAnnualGeographical CoverageEnglandReporting level(s)NationalBased on data fromRoyal College of Physicians’ Sentinel Stroke National Audit Programme (RCP SSNAP), Office for National Statistics (ONS) mortality dataContact Author NameAlison Roe, Senior Service Delivery Manager, HSCICContact Author Emailccgois@.ukRatingUse with caution Assurance date14/12/2015Review date14/12/2016Indicator setCCG Outcomes Indicator Set (OIS) 3.8Brief Description [This appears as a blurb in search results]This indicator is the percentage of patients in the Sentinel Stroke National Audit Programme who had a follow-up assessment between 4 and 8 months after initial admission for stroke.PurposeThis indicator is an important measure of the effectiveness of rehabilitation within the wider local healthcare system outside of the inpatient setting. It is expected that this indicator will be used to track progress over time and highlight areas for improvement.DefinitionOf the number of eligible stroke patients entered into the Sentinel Stroke National Audit Programme (SSNAP), the percentage who had a follow-up assessment between 4 and 8 months after initial admission for stroke.Stroke is defined within this indicator as intracerebral haemorrhage, cerebral infarction and stroke (not specified as haemorrhage or infarction).Data SourceRCP SSNAP, ONS mortality dataNumeratorThe number of patients who had a follow-up assessment between 4 and 8 months after initial admission for strokeDenominatorThe number of patients entered into SSNAPCalculationThe indicator is calculated as a percentageInterpretation GuidelinesThis indicator requires careful interpretation and should not be viewed in isolation. It should be considered alongside information from other indicators and alternative sources, such as the other Clinical Commissioning Group Outcome Indicator Set (CCG OIS) stroke measures and the CCG level Stroke Sentinel National Audit Programme (SSNAP) stroke unit key indicators. When evaluated together, these will help to provide a comprehensive view of CCG outcomes and provide a more complete overview of the impact of the CCGs’ processes on outcomes.A high percentage of stroke patients who have a follow-up assessment between 4 and 8 months after initial admission for stroke is desirable.There is currently very wide variation across CCGs. Recent analysis of the completeness of the data for this indicator shows that almost a third of eligible patient records do not hold the relevant assessment information and are, therefore, excluded from the indicator.CaveatsCare may vary between organisations in terms of hospital inpatient admission practices and policies.There may be variation in the prevalence of stroke due to differing levels of deprivation, for other regional or demographic reasons or between patients of different ethnic heritages.Application FormIndicator and Methodology Assurance ServiceTitle: Set or domain: CCG OIS 3.8IAS Reference Code: IAP00137Version HistoryVersionDateChanged ByChangeV0.123/06/2017Andy BeschInitial uplift to most recent application formApplication FormIndicator and Methodology Assurance ServiceTitle: Set or domain: CCG OIS 3.8IAS Reference Code: IAP00137Version HistoryVersionDateChanged ByChangeV0.123/06/2017Andy BeschInitial uplift to most recent application formApplication FormSection 1 Introduction / Overview1.1 Title1.2 Set or domainCCG Outcomes Indicator Set (OIS) 3.81.3 Topic areaCardiovascular1.4 DefinitionThe percentage of people who have a follow-up assessment between 4 and 8 months after initial admission for stroke.Technical description: Of the number of eligible stroke patients entered into the SSNAP, the percentage who had a follow-up assessment between 4 and 8 months after initial admission for stroke.Eligible patients exclude;Patients who died within 8 months of initial admission for stroke and who did not have a follow-up assessment Patients who died whilst on the stroke care pathway (reported by either an inpatient team or a community team)Patients who decline an appointment offeredPatients for whom an attempt is made to offer an appointment but are untraceable as they are not registered with a GPStroke is defined within this indicator as intracerebral haemorrhage (ICD-10 code: I61), cerebral infarction (I63) and stroke, not specified as haemorrhage or infarction (I64).It was published for the first time in December 2014 (2013/14 data) and has been published annually in December since then.1.5 Indicator owner & contact details1.6 Publication statusCurrently in publicationSection 2 Rationale2.1 PurposeThe National Stroke Strategy recognises that people who have had a stroke should be offered a follow-up assessment of their health and social care status and secondary prevention needs. This indicator is to ensure follow-up assessments take place, making it possible for the patient to access further specialist advice, information and rehabilitation where needed. For practical reasons, given the nature of care provided, a follow-up assessment period of 4-8 months after discharge is considered appropriate to satisfy the conditions of the indicator.Reviews should be a multifaceted assessment of need and should encompass:Medicines/general health needsOngoing therapy and rehabilitation needsMood, memory cognitive and psychological statusSocial care needs, carer wellbeing, finances and benefits, driving, travel and transport.2.2 Sponsor2.3 EndorsementThe indicator was constructed following consultation with the following clinical and stroke data experts: Professor Anthony Rudd, Chair of the Intercollegiate Stroke Working Party, Associate Director for Stroke, Consultant Stroke PhysicianJames Campbell, Sentinel Stroke National Audit Programme (SSNAP) Intelligence Programme Manager, Royal College of Physicians (RCP) Lizz Paley, Acting Stroke Programme Intelligence Manager – Data, RCP2.4 Evidence and Policy baseIncluding related national incentives, critical business question, NICE quality standard and set or domain rationale, if appropriateThis indicator supports the Quality Standard for stroke (QS2) which covers care provided to adult stroke patients by healthcare staff during diagnosis and initial management, acute-phase care, rehabilitation and long-term management.Guidance suggests that regular reviews after transfer home provide a key opportunity to ensure people get the support they need. Recovery from stroke can continue over a long time and rehabilitation should continue until it is clear that maximum recovery has been achieved. Some patients will need ongoing support, possibly for many years. These people and their carers should have access to a stroke care co-ordinator who can provide advice, arrange reassessment when needs or circumstances change, co-ordinate long-term support or arrange for specialist care.The National Stroke Strategy advised that people who have had strokes (and their carers), either living at home or in care homes, are offered a review from primary care services of their health and social care status and secondary prevention needs, typically within six weeks of discharge home or to care home and again before six months after leaving hospital. The offer of regular review is one way to ensure people continue to feel supported. Of key importance is the enabling and supporting of people in navigating through the system. Self-referral systems that do not take this into account are likely to increase inequalities of access and outcome.Section 3 Data3.1 Data source RCP SSNAP and Office for National Statistics (ONS) mortality data, via record linkage.The SSNAP is guided by the Intercollegiate Stroke Working Party (ICSWP) and delivered by the Stroke Programme within the Clinical Effectiveness and Evaluation Unit in the RCP. It is centrally funded by the Healthcare Quality Improvement Partnership (HQIP) on behalf of NHS England as part of the National Clinical Audit and Patient Outcomes Programme (NCAPOP). Justification of source and others considered The SSNAP is the single source of data on stroke services, processes of care and outcomes. It provides the data for other statutory reporting mechanisms in England, including the NICE Quality Standard and the five other CCG OIS stroke measures; it is also due to be used in the NHS Outcomes Framework. SSNAP metrics are aligned with those in the Cardiovascular Disease Outcomes Strategy. Hospital Episode Statistics (HES) was considered as a data source for this indicator; however it does not contain the necessary detail required to measure follow-up assessment.3.2 Data availabilityCCG OIS indicators are published annually. SSNAP data for the full financial year is available to produce the indicator approximately 8 months after the financial year end; therefore the indicator is published each year in G OIS indicators are official statistics and the publication date is pre-announced. There is no gap between the planned and actual publication date.The RCP make this indicator, along with a number of others, accessible to the public via RCP reporting, including an Easy Access Version aimed at stroke survivors and carers. It is available via Excel spreadsheets and other formats including graphical representation.3.4 Data quality i) What data quality checks are relevant to this indicator?Coverage ? Completeness ? Validity ? Default ? Integrity ? Timeliness ? Other ?If you included ‘Other’ as a data quality check, please describe the check, how it will be measured, and its reason for use below: FORMTEXT ????? ii) What are the current values for the data quality checks selected? The period of data the current values are calculated from should be stated. Current values should be recorded as a percentage and calculated as described below. Period of data: Coverage: Calculation: Completeness: Calculation: Validity: Calculation: Default: Calculation: Integrity: Calculation: Timeliness: Calculation: Other: Calculation:iii) What are the thresholds for the data quality checks selected? Coverage: Completeness: Validity: Default: Integrity: Timeliness: Other: iv) What is the rationale for the selection of the data quality checks and thresholds selected above? v) Describe how you would plan to improve data quality should it not meet, or subsequently fall below, the thresholds required for this indicator. vi) Who will own the data quality risks and issues for this indicator? Name: Job Title: Role: Email: Telephone: vii) Describe how the data quality risks and issues will be managed for this indicator, including the escalation process. viii) Describe any assumptions you have made about data quality for this indicator. ix) Describe any data quality constraints you are aware of for this indicator. x) Additional data quality information: 3.5 Quality assuranceAs SSNAP data is subject to strong built-in validation via the secure web tool, it means that it is not possible for providers to enter illogical timings; however, this is double checked during analysis and therefore the accuracy of the indicator is very high. No assumptions are made regarding the arrival and discharge times, apart from when a patient died in hospital.When submitting SSNAP data, security and confidentiality are maintained through the use of passwords and a person specific registration process. A dedicated helpdesk is in place to answer queries from SSNAP participants, helping to ensure questions are interpreted consistently (which informs updates to FAQs and data set help notes). Users can register for their team on the SSNAP web tool and input data for their team. Once records are complete and correct they can be ‘locked’ at different levels. Records can be ‘locked’ to 72 hours once this information is completed, they can then be locked to discharge once this is applicable. Locking confirms that all data have been clinically signed off and are ready for central analysis. The ‘Lead clinical contact’ role is responsible for ensuring that the overall system of data collection and entry onto the web tool is accurate, robust and functioning. The SSNAP encourage the lead to routinely check data. Only complete and locked to 72 hours records go into data analysis for the 72 hour section and complete and locked to discharge records go into data analysis for the post-72h section.Eligibility criteria are applied to determine which records can be included in the audit. The criteria are: ICD-10 codes I61, I63, I64, but hospitals have means of checking for eligible patients other than their coding system and participants are encouraged to enter cases prospectively meaning the stroke team have more control over selecting records to be included and can also refer to their stroke register, should they have one.If this question is not answered, it is interpreted in the RCP SSNAP output as an assessment did not take place; this stance is providing an incentive for stroke teams to record this information.3.6 Data linkageSSNAP records are linked with mortality information from ONS. The SSNAP data are securely sent for linkage following each quarterly deadline, and the information on any death notifications is provided back monthly. This enables SSNAP to track mortality other than as reported on SSNAP (i.e. after patients have left care). As well as providing casemix adjusted mortality rates, this is also used for other purposes, such as to determine eligibility for receiving a six month assessment.3.7 Quality of data linkageThe match rate between SSNAP and ONS data stood at over 98% for 2013/14 data; therefore the indicator is considered an accurate representation of mortality for stroke patients. The RCP are still awaiting 2014/15 mortality data but there are not expected to be concerns with the linkage.3.8 Data fields The data fields supplied by the RCP are as follows: 1. Number of patients considered applicable to be assessed at 4-8 months2. Number of patients alive when due a 4-8 month assessment3. Percentage of patients alive who are considered applicable to be assessed at 4-8 months4. Number of applicable patients assessed5. Number of patients applicable to be assessed6. Percentage of applicable patients who are assessed at 4-8 months3.9 Data filtersSSNAP-derived records meeting all of the following requirements are valid for the denominator in this indicator:Audit Question 7.1 – The response to ‘The patient…’ is not recorded as ‘Died’. Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘Yes’ or ‘No’ or is not completed.Please note that in order to improve the recording of the six month follow-up assessment, the ‘or is not completed’ element has been introduced into the filter for future data releases. This part of the filter was not present in 2013/14 data.SSNAP-derived records meeting all of the following requirements are valid for the numerator in this indicator:Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘Yes’A patient is included in both the denominator and numerator if they had a six month follow-up assessment, regardless of if they died after receiving it. A patient is excluded from the denominator if they did not have an assessment and died within 8 months of admission; identified using ONS data (Date of Death within 8 months of Admission Date).3.10 Justifications of inclusions and exclusions and how these adhere to standard definitionsAudit Question 7.1 is not recorded as ‘Died’ – Identifies patients who did not die whilst on the stroke care pathway (reported by either an inpatient team or a community team). This is implicit within the indicator; if a patient is recorded as having died at this stage, it is not possible for the stroke team to complete Audit Question 8.1.Audit Question 8.1 is equal to ‘Yes’ or ‘No’ – Identifies patients who had or could have had a follow-up assessment at 6 months (plus or minus 2 months). Audit Question 8.1 is equal to ‘Yes’ – Identifies patients who had a follow-up assessment at 6 months (plus or minus 2 months).The rationale for including a patient in both the numerator and denominator if they had a six month follow-up assessment, regardless of if they died after receiving it, is that credit should be given where six month assessments are completed. Patients should be excluded if they could have received a six month assessment within the recommended time frame (4-8 months) but died before being offered or having that assessment.The SSNAP uses the following ICD-10 diagnosis codes to identify stroke patients:I61 - Intracerebral haemorrhageI63 - Cerebral infarctionI64 - Stroke, not specified as haemorrhage or infarctionThe coding advice from the Clinical Classifications Service also includes I60 (Subarachnoid haemorrhage) and I62 (Other nontraumatic intracranial haemorrhage), however this advice would not be endorsed by the RCP as subarachnoid haemorrhage and other non-traumatic intracranial haemorrhage have a different care pathway and outcome.Subarachnoid haemorrhages and other non-traumatic intracranial haemorrhages are routinely and nearly always managed entirely outside of the stroke unit by neurosurgeons or by interventional neuroradiologists, which is what is recommended in national guidelines for these cases. The indicators need to reflect the care given on appropriate clinical pathways, not arbitrary groupings.3.11 Data processingThe calculated CCG level indicator is provided by the RCP and includes the percentage, numerator, denominator and contextual information. It is provided with any necessary data suppression.A 95% confidence interval is calculated by Clinical Indicators for each CCG prior to publicationSection 4 Construction4.1 NumeratorOf the denominator, the number of patients who had a follow-up assessment between 4 and 8 months after initial admission for stroke.4.2 DenominatorThe number of stroke patients entered into the SSNAP excluding;Patients who died within 8 months of initial admission for stroke and who did not have a follow-up assessment Patients who died whilst on the stroke care pathway (reported by either an inpatient team or a community team)Patients who decline an appointment offeredPatients for whom an attempt is made to offer an appointment but are untraceable as they are not registered with a GPA patient is included in both the denominator and numerator if they had a six month follow-up assessment, regardless of if they died after receiving it. A patient is excluded from the denominator if they did not have an assessment and died within 8 months of admission (identified using ONS data - Date of Death within 8 months of admission date).Patients who decline an appointment offered are patients who are offered a follow-up assessment and decline or do not attend the assessment. These are identified in the SSNAP where Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘No but’ and therefore excluded from the denominator.4.3 ComputationThe percentage p is given by:p=On×100where:O is the numerator and n is the denominator.4.4 Risk adjustment or standardisation type and methodologyNoneVariables and methodology:4.5 Justification of risk adjustment type and variablesor why risk adjustment is not usedAll eligible patients should receive a follow-up assessment between 4 and 8 months after initial admission for stroke.4.6 Confidence interval / control limit use and methodologyConfidence IntervalsMethodology:Confidence intervals are calculated using the Wilson Score method, as specified in “Commonly used public health statistics and their confidence intervals” (Public Health England (PHE), March 2008 ).The formulae for the 100(1 – α)% confidence interval limits for the proportion p are:Plower=2O+z2-zz2+4oq2n+z2Pupper=2O+z2+zz2+4oq2n+z2where:O is the observed number of individuals in the sample/population having the specified characteristic (i.e., the numerator);n is the total number of individuals in the sample/population (i.e., the denominator);q = (1 – p) is the proportion without the specified characteristic;z is the 100(1 – α/2)th percentile value from the Standard Normal distribution. For example for a 95% confidence interval, α = 0.05, and z = 1.96 (i.e. the 97.5th percentile value from the Standard Normal distribution).4.7 Justification of confidence intervals / control limits usedThe preferred PHE confidence interval method for proportions is the Wilson Score method which has been evaluated and recommended by Newcombe and Altman;. It can be used with any data values and, unlike some methods, it does not fail to give an interval when the numerator count, and therefore the proportion, is zero.Section 5 Presentation and InterpretationPresentation5.1 Presentation of indicatorThe indicator is presented on the NHS Digital Indicator Portal in a consistent format to other CCG OIS indicators. It is accompanied by indicator specification and quality statement documents, which provide details of indicator construction, data quality, statistical methods and interpretation considerations data is presented with a detailed header including information on the statistic presented, the reporting period, level of coverage, publication date, data source, and any further notes to be aware of. The customer is also able to make use of drop-down filtering.Column name Output Reporting period Financial year Breakdown England, CCG Level CCG Code Level description CCG Name PercentageThe indicator percentage calculationCI lower Lower 95% confidence interval CI upper Upper 95% confidence interval DenominatorThe number of patients entered into SSNAPNumerator The number of patients who had a follow-up assessment between 4 and 8 months after initial admission for strokeNumber of records in SSNAP (care delivered within the first 72hrs)The number of cases in SSNAP Estimated expected number of patients (from HES)The number of cases in HESCase ascertainment bandCase ascertainment between SSNAP and HES5.2 Contextual information provided alongside indicatorwith justificationAlongside the numerator, denominator and percentage, the number of records in SSNAP (care delivered within first 72hrs) is provided for each CCG as contextual information.The indicator is published in the context of case ascertainment between SSNAP and HES. The ‘Estimated expected number of patients from HES’ figure is the number of patients who have been coded as a primary diagnosis of stroke during their admission in a year’s worth of HES, split by the patient’s CCG recorded in the HES record. Case ascertainment is reported alongside the indicator for all CCGs to highlight audit coverage against HES. MRG requested this further analysis in the original assurance process.Case ascertainment is reported within the context of the ‘care delivered within the first 72hrs’ cohort of stroke patients for this indicator. The ‘Case ascertainment band’ column in the published output uses the following bandings:90%+80-89%70-79%50-69%Less than 50%The indicator is not reported for any CCGs with lower than 50% case ascertainment or for those with fewer than 20 patients.5.3 Calculation and data source of contextual informationThe contextual information is sourced from the SSNAP and provided by the RCP.5.4 Use of bandings, benchmarks or targetswith justificationNone. If a CCG believes their figure to be disproportionately low, the factors contributing to this can be investigated and appropriate action can be taken, however data completeness is likely to be an issue for many CCGs.5.5 Banding, benchmark or target methodologyif appropriateN/AInterpretation5.6 Interpretation guidelinesA high percentage of stroke patients who have a follow-up assessment between 4 and 8 months after initial admission for stroke is desirable.As stated in section 3.4, there is currently very wide variation across CCGs. Recent analysis of the completeness of the data for this indicator shows that almost a third of eligible patient records do not hold the relevant assessment information and are therefore excluded from the indicator. As stated previously, it is proposed that in future data releases, if this question is not answered; it will be interpreted as an assessment did not take place (see section 3.10 for the amendment to the data filter).This indicator requires careful interpretation and should not be viewed in isolation but instead be considered alongside information from other indicators and alternative sources, such as the other CCG OIS stroke measures and the CCG level SSNAP stroke unit key indicators. When evaluated together, these will help to provide a holistic view of CCG outcomes and provide a more complete overview of the impact of the CCGs’ processes on outcomes.5.7 Limitations and potential biasThe patterns of providing care may vary between organisations in terms of hospital inpatient admission practices and policies.There may be variation in the prevalence of stroke due to differing levels of deprivation, for other geo-demographic reasons or between patients of different ethnic heritages.5.8 Improvement actionsIt is expected that CCGs will use this indicator to identify improvements in care and how they can be delivered.Improvements could be made by enhancing aspects of the services CCGs commission for patients. This could come in the form of improving follow-up processes to make it easier to assess discharged patients in a timely manner.While the vast majority of patients alive at this time after stroke are applicable to receive a six month follow-up, this is recorded to have happened in less than 20% of cases. Clinical teams and commissioners need to work closely together to see recording improve to get the most value from the audit for service improvement.5.9 Evidence of variabilityThe data within this section is taken from the December 2014 CCG OIS publication. During the financial year 2013/14 there were 24,873 patients eligible to receive a follow-up assessment between 4 and 8 months. Of these, 4,059 had a follow-up assessment..The data below shows the ten CCGs with the lowest and the ten CCGs with the highest percentages in 2013/14. Five CCGs have been suppressed due to insufficient case ascertainment between SSNAP and HES and are not included within the data G%LCIUCIDenNumRecords in SSNAPRecords in HESCase ascertainmentCCG10.00.06.853015817880-89%CCG20.00.02.5147040947080-89%CCG30.00.04.975017523870-79%CCG40.00.04.287019327070-79%CCG50.00.02.2168032338180-89%CCG60.00.03.3113029138370-79%CCG70.00.02.8133028927690%+CCG80.00.04.483022628180-89%CCG90.00.04.975028527890%+CCG100.00.03.5106026834070-79%CCG%LCIUCIDenNumRecords in SSNAPRecords in HESCase ascertainmentCCG19768.860.775.91419736837790%+CCG19869.563.874.726918788198380-89%CCG19970.863.677.117112139051470-79%CCG20077.671.982.523718448955680-89%CCG20178.571.384.414911734333590%+CCG20284.077.289.114412142437890%+CCG20385.579.490.017214759169280-89%CCG20491.284.695.211410426024790%+CCG20592.981.097.5423916117890%+CCG20694.474.299.0181718831550-69%There is very wide variation in the 2013/14 data with 64 CCGs (30.3%) recording that none of their eligible stroke patients received a follow-up assessment between 4 and 8 months after initial admission. The RCP acknowledged the variation in the SSNAP 2013/14 annual report () stating that, ‘At present, an increasing number of (stroke) teams are starting to enter information routinely to SSNAP, but based on the patients who could have had a six month (plus or minus two months) follow-up assessment within the financial year, the rate is less than one in five.’ As stated in section 3.4, the subsequent RCP SSNAP Clinical audit October - December 2014 public report () provides an indication of how widely this section of the audit is being answered. Between July and December 2014, 34,476 patient records should have had an answer for the 6 month assessment. Of these, 10,709 (31.1%) did have an answerSection 6 Risks6. 1Similar existing indicatorsThis indicator is published in different formats at CCG, trust and stroke team level on the SSNAP results portal Accelerating Stroke Improvement National Plan was a national initiative designed to ensure that maximum implementation of the Quality Markers in the National Stroke Strategy were achieved before the end of the 2010/11 financial year. One of the measures (measure 8) looked at the proportion of stroke patients that were reviewed six months after leaving hospital, with the aim of 95% by April 2011 Coherence and comparabilityThe methodology and results for this indicator are consistent with the same indicator published on the SSNAP results portal.6.3 Undesired behaviours and/or gamingGaming would involve stroke teams purposefully not completing the non-mandatory 6 month follow-up assessment field when the assessment did not take place. However, this issue will be addressed in future releases of data by being interpreted in the SSNAP output as an assessment did not take place where the question is not answered.6.4 Approach to indicator reviewThe Indicator Governance Board (IGB) set a review period of one year when the indicator was originally assured, due to the relative immaturity of the SSNAP data set at that time. The time period for the next review will again be set by IGB.User feedback and comments on this indicator are welcomed via NHS Digital Enquires enquiries@.uk or the CCG OIS mailbox ccgois@.uk6.5 Disclosure controlCase ascertainment used is the proportion of patients per CCG with primary ICD-10 codes I61, I63 and I64 in HES data who are included in SSNAP for the same time period. Case ascertainment is reported alongside the indicator for all CCGs. The indicator is not reported for any CCGs with lower than 50% case ascertainment or for those with fewer than 20 patients, instead replacing the SMR with ‘*’. Ratios are rounded to two decimal places before publication. 6.6 CopyrightThere are no restrictions on the use of these data. Any subsequent use or publishing of these data should reference the RCP SSNAP.Indicator Assurance Report TITLE \* MERGEFORMAT IAP00137Final Assurance Rating from the Indicator Governance BoardClarityFit for useRationale Fit for use with caveatsDataUse with cautionConstruction Fit for usePresentation and InterpretationFit for use with caveatsRisks and Usefulness Fit for useOverall ratingUse with cautionThis indicator has been approved for inclusion in the National Library of Quality Assured IndicatorsKey findings from AssuranceIGB members accepted the conclusions reached by MRG, identifying the indicator should be reviewed in 1 year to assess any improvement in data quality.Approval date14/12/2015Review date14/12/2016Details of Methodology Appraisal - 10/09/2015Methodology appraisal bodyHSCIC's Indicator & Methodology Assurance ServiceReason for assessmentScheduled review (review date reached)Iteration1st MRG meetingSuggested Assurance Rating by Methodology Appraisal BodyClarityFit for useRationaleFit for use with caveatsDataUse with CautionConstruction Fit for usePresentation and Interpretation Fit for use with caveatsRisks and Usefulness Fit for useOverall ratingUse with cautionSummary Recommendation to Applicant:MRG noted that the indicator has been previously assured (with comments) as suitable for inclusion in the Library of Quality Assured Indicators, however this was under an earlier iteration of the assurance process. Members thanked the applicant for the “uplift” in documentation which has allowed the indicator to be assessed against the standard criteria assessment and “levels of assurance”. Upon review the indicator has been given an overall rating of “Use with caution”.This rating has been assigned as the data quality of the indicator is low, however due to the aim of increasing data quality through including null responses in the denominator, and the quality of the indicator otherwise, MRG are endorsing it for inclusion in the Library.Summary Recommendation to IGB:MRG endorse the indicator for inclusion in the Library, however suggest that data quality is reassessed upon review. There are small improvements which could be made to the metadata, specifically around justifying the data source, how and why HES is used to measure case ascertainment, and the interpretation guidelines. In addition, there is currently no named sponsor for the indicator. Please find a detailed description of recommendations and actions in the appraisal log at the end of the document.What do the Assurance Ratings mean?RatingDescriptionFit for useThis indicator can be used with confidence that it is constructed in a sound manner that is fit for purpose.Fit for use with caveatsThe indicator is fit for use, however users should be aware of caveats and/or recommendations for improvement that have been identified during the assurance process.Use with cautionThe indicator is based on a sound methodology for which the assurance process endorse the use, however issues have been identified with the national data source which have implications for its use as an indicator.Not fit for useIssues have been identified with the indicator which have resulted in the assurance process currently not endorsing its use as a quality indicator.Not enough information providedThere has not been enough information supplied to the assurance process to be able to accurately give the indicator a level of assurance.Appraisal Log ClarityRec. noIssue or recommendationRaised by / DateResponse or Action taken by applicantResponse dateResolvedSign off by / Date?RationaleRec. noIssue or recommendationRaised by / DateResponse or Action taken by applicantResponse dateResolvedSign off by / Date2aA sponsor for the indicator needs to be identified.MRG10/09/15The sponsor of the CCG OIS is Richard Owen, Outcomes Strategy Lead, NHS Medical Directorate, NHS England.2bThe definition should be clear as to the types of stroke included in the indicator.MRG10/09/15A sentence is included in the definition section of the IAS application form and Indicator Quality Statement, stating: Stroke is defined within this indicator as intracerebral haemorrhage (ICD-10 code: I61), cerebral infarction (I63) and stroke, not specified as haemorrhage or infarction (I64).2cThe paperwork should be clearer regarding what constitutes a follow-up.MRG10/09/15The National Stroke Strategy recognises that people who have had a stroke, either living at home or in care homes, should be offered a review of their health and social care status and secondary prevention needs. Reviews should be a multifaceted assessment of need and should encompass:Medicines/general health needsOngoing therapy and rehabilitation needsMood, memory cognitive and psychological statusSocial care needs, carer wellbeing, finances and benefits, driving, travel and transport.DataRec. noIssue or recommendationRaised by / DateResponse or Action taken by applicantResponse dateResolvedSign off by / Date3aMRG recommended that investigations into whether case ascertainment is the same for different age and sex breakdowns as selection bias could affect the calculation of the indicators. MRG6/9/12The age and sex breakdowns were investigated by the RCP in 2012 and found to be comparable with published literature and therefore not felt to represent a selection bias. The SSNAP annual report provides some overall demographic details of patients included in the SSNAP (April-2013-March-2014).pdf Along with a host of other detailed audit information, the quarterly SSNAP public report provides specific details on the casemix breakdowns, including patient numbers, gender, age, co-morbidities, stroke type, Modified Rankin Scales scores, NIHSS and the onset of symptoms (Section 2: Casemix, p48) MRG10/09/153bThe rationale for selecting the ICD-10 codes used to identify stroke patients should be clearly stated in the documentation for each indicator.Update:There is a discrepancy between what SSNAP and the clinical classifications service consider a stroke, therefore further justification for the codes used is required and the definition should be updated (as stated in recommendation 2b).MRG6/9/12MRG10/09/15The SSNAP uses the following ICD-10 diagnosis codes to identify stroke patients:I61 - Intracerebral haemorrhageI63 - Cerebral infarctionI64 - Stroke, not specified as haemorrhage or infarctionThe coding advice from the Clinical Classifications Service also includes I60 (Subarachnoid haemorrhage) and I62 (Other nontraumatic intracranial haemorrhage), however this advice would not be endorsed by the RCP as subarachnoid haemorrhage and other non-traumatic intracranial haemorrhage have a different care pathway and outcome.Update:Subarachnoid haemorrhages and other non-traumatic intracranial haemorrhages are routinely and nearly always managed entirely outside of the stroke unit by neurosurgeons or by interventional neuroradiologists, which is what is recommended in national guidelines for these cases. The indicators need to reflect the care given on appropriate clinical pathways, not arbitrary groupings.During initial assurance3cPeer review commented on whether there was uncertainty around data completeness in the audit collection as it is new.Peer review19/11/12See section 6a13/08/15MRG10/09/153dPeer reviewer asked for clarification on how follow-ups, usually done in the community, would be captured as the peer reviewer didn’t have full understanding of how data will be completed in the audit.Peer review19/11/12There is variation across the country in what processes are used for follow-up assessment with teams employing local recording practices. Some areas may use a shared or integrated care record. The data are submitted by providers via a secure web tool.13/08/15MRG10/09/153dThe narrative around why SSNAP is being used as opposed to HES should be strengthened. The application states that over-coding occurs in HES, however the results in section 5.9 show that case “ascertainment” against HES is over 100%.MRG10/09/15The application for this indicator did not state that over-coding occurs in HES. The application stated that HES does not contain the necessary detail required to measure this indicator.3eThe applicant should consider how useful it is to provide case ascertainment against HES data, since it is recognised that over-coding occurs in HES, making the figure hard to interpret. If the figure is to be presented, MRG recommend changing the name from “case ascertainment” to “case comparison” and to present bands above 90+%.MRG10/09/15This contextual case ascertainment information aligns to the information and bandings presented in the RCP SSNAP publication. The RCP view is that it is not case comparison as it is not comparing the same year’s HES with SSNAP. Since the purpose of including case ascertainment is to highlight CCGs with low case ascertainment indicating that hospitals within the CCG have not been entering in all their patients onto SSNAP (and the results may therefore not reflect the care that all the CCGs patients received), having bands above 100% would not be useful. HES is not the ‘gold standard’, but it is a useful indication of case selection. The HES case ascertainment figure (‘Estimated expected number of patients from HES’) is the number of patients who have been coded as a primary diagnosis of stroke during their admission in a year’s worth of HES, split by the patient’s CCG recorded in the HES record. The indicator is not reported for CCGs with less than 50% case ascertainment.3fThe data quality is currently low, however MRG endorse the approach taken by the applicant to include null responses in the denominator of the indicator to halp increase the quality. The quality of the data should be reassessed when the indicator is reviewed.MRG10/09/15If Audit Question 8.1 - Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)? - is not completed, it is interpreted as an assessment did not take place and therefore included within the indicator denominator but not the numerator.As SSNAP started data collection in January 2013, 6 month assessments did not start being undertaken until approximately June 2013 and even then the first quarter’s data collection was during a pilot phase and therefore low. This meant that not all patients were entered onto SSNAP to begin with, so they could not have their 6 month assessment recorded. Now that SSNAP records upwards of 95% of stroke admissions, this is no longer an issue for 2014/15 onwards; the denominator spans the whole year.ConstructionRec. noIssue or recommendationRaised by / DateResponse or Action taken by applicantResponse dateResolvedSign off by / Date4aMRG requested clarification on which patients were included and excluded from the indicator, as it is important to ensure that there isn’t a mismatch between the numerator and denominator. It was suggested that a useful way to do this would be to complete a matrix of possible scenarios (e.g. assessment occurring at 4, 5, 6, 7, 8 months on one axis and patient alive at 4, 5, 6, 7, 8 months on the other axis).MRG6/9/12The original concern around a possible mismatch between the denominator and numerator was resolved. The RCP use a combination of SSNAP and ONS mortality data to determine applicability for the indicator. In the first instance, patients that died whilst on the stroke care pathway are excluded from the indicator automatically, as it is not possible for the stroke team to complete Audit Question 8.1 (detailed below) on the web tool. The denominator includes records where Audit Question 8.1 (‘Did this patient have a follow up assessment at 6 months (plus or minus 2 months)’) is equal to ‘Yes’ or ‘No’. The question provides a ‘No, patient died within 6 months of admission’ option. The numerator then includes patients where Audit Question 8.1 is equal to ‘Yes’. A patient is included in both the denominator and numerator if they had a six month follow-up assessment (+/- 2 months), regardless of if they died after receiving it. A patient is excluded from the denominator if they did not have an assessment and died within 8 months of admission; identified using ONS data (Date of Death within 8 months of Admission Date).Patients are only included once in the indicator and are only reported on after 8 months has elapsed. The median time from admission (or onset, if in hospital) is 6.4 months, based on 12,631 assessments completed in 2014/15. 13/08/15MRG10/09/154bThe description of the denominator should state ‘following admission’ rather than ‘following discharge’, so that it is consistent with both the title of the indicator and the description of the numerator.Following MRG’s suggestions, the indicator denominator is described in the indicator specification as:The number of stroke patients entered into SSNAP excluding;Patients who died within 6 months of initial admission for strokePatients who decline an appointment offeredPatients for whom an attempt is made to offer an appointment but are untraceable as they are not registered with a GPThe indicator numerator is described in the indicator specification as:Of the denominator, the number of patients who had a follow-up assessment between 4 and 8 months after initial admission for stroke.During initial assuranceMRG10/09/154cThe application provided should give additional information regarding what is meant by “Patients who decline an appointment offered”, as this group are currently excluded from the indicator.MRG10/09/15Patients who decline an appointment offered are patients who are offered a follow-up assessment and decline or do not attend the assessment. These are identified in the SSNAP where Audit Question 8.1 – ‘Did this patient have a follow-up assessment at 6 months (plus or minus 2 months)?’ is equal to ‘No but’ and therefore excluded from the denominator.Presentation and InterpretationRec. noIssue or recommendationRaised by / DateResponse or Action taken by applicantResponse dateResolvedSign off by / Date5aIt is recommended that a break-down of response is given as contextual information, (i.e. “Yes, “No” and “Not completed”) so users can distinguish between cases when patients have notreceived the follow up and where the field hasn’t been filled in.MRG10/09/15It is not proposed to provide a breakdown of responses for this indicator. Including patient records in the denominator where the follow-up question has not been completed should encourage an improvement in data quality for this indicator.Risks and UsefulnessRec. noIssue or recommendationRaised by / DateResponse or Action taken by applicantResponse dateResolvedSign off by / Date6aMRG recommended that work on cross-validating the audit data set with HES data should continue, as the credibility of the indicators could be impacted by conflicting sources. MRG recommended including a contextual indicator on the relationship between the audit and HES data as this would also encourage improvements in data quality. MRG6/9/12The indicator is published in the context of case ascertainment between SSNAP and HES. This is the percentage of patients with primary ICD-10 codes I61, I63 and I64 in HES who are included in SSNAP for the same time period.The SSNAP is a mandatory collection and overall case ascertainment increased from 72% in Quarter 1 to 95% in Quarter 4, 2013/14 (Quarter 2: 83%, Quarter 3: 90%). It has further improved to 97% by Quarter 4, 2014/15. Case ascertainment is reported alongside the indicator for all CCGs in the published CCG OIS data files. Five CCGs (2.4%) had their percentages suppressed in the published 2013/14 data due to less than 50% case ascertainment with HES.Patient records are only included in audit analyses if they include the minimum requirements of completion of mandatory fields. However, the follow-up assessment at six months (plus or minus two months) field is not mandatory, as it is not in the acute part of the data set. Case ascertainment is high, as the records have been submitted, but many do not include information on follow-up assessment. There is very wide variation in 2013/14 data for this indicator. The subsequent RCP SSNAP Clinical audit October - December 2014 public report () provides an indication of how widely this section of the audit is being answered, rather than indicating the numbers of patients who had a six month assessment completed. In future quarters, if this question is not answered, it will be interpreted in the RCP SSNAP output as an assessment did not take place.Between July and December 2014, 34,476 patient records should have had an answer. Of these, 10,709 (31.1%) did have an answer. The RCP feel it is extremely important that data regarding a patient’s six month follow-up is recorded in the SSNAP. This is regardless of whether or not the assessment was provided. While the vast majority of patients alive at this time after stroke are applicable to receive a six month review, this is only recorded to have happened in less than 20% of cases. It is proposed that in future data releases, if this question is not answered; it will be interpreted as an assessment did not take place. The relevant data filter has been amended to include records which are not completed in the denominator (see section 3.10 of the application form for the amendment to the data filter).13/08/15MRG10/09/15Any complaints or appeals against the decisions made during the assurance process should be made to the Indicator & Methodology Assurance Service (IMAS) Team at HSCIC. Likewise, if you are unclear regarding any of the recommendations in this report, or have any queries about the assurance process in general, please contact the IMAS team.Indicator and Methodology Assurance ServiceHealth and Social Care Information Centre1 Trevelyan Square, Boar Lane,LEEDS LS1 6AE.Email: indicator.assurance@.ukWebsite: complaints or appeals against the decisions made during the assurance process should be made to the Indicator & Methodology Assurance Service (IMAS) Team at HSCIC. Likewise, if you are unclear regarding any of the recommendations in this report, or have any queries about the assurance process in general, please contact the IMAS team.Indicator and Methodology Assurance ServiceHealth and Social Care Information Centre1 Trevelyan Square, Boar Lane,LEEDS LS1 6AE.Email: indicator.assurance@.ukWebsite: ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download