Factors predicting death from Covid-19 in the UK



Risk stratification of patients admitted to hospital with covid-19 in the UK: Development and validation of a multivariable prediction model for mortalityISARIC-4C ConsortiumStephen Knight* (UoE) Antonia Ho* (UoG)Riinu Pius (UoE)*Alphabetical*Iain Buchan (UoL)Gail Carson (ISARIC)Thomas Drake (UoE)Jake Dunning (PHE)Cameron Fairfield (UoE)Caroll Gamble (UoL)Christopher A Green (UoB)Hayley E Hardwick (UoL)Karl A Holden (UoL)Peter W Horby (ISARIC)Kenneth A Mclean (UoE)Laura Merson (ISARIC)Lisa Norman (UoE)Jonathan S Nguyen-Van-Tam (UoN)Piero L OlliaroMark Pritchard (ISARIC)Clark D Russell (UoE)Catherine Shaw (UoE)Aziz Sheikh (UoE)Cathie Sudlow (UoE)Olivia Swann (UoE)Lance Turtle (UoL)Peter JM Openshaw (Imperial)J Kenneth Baillie (UoE)Malcolm G Semple (UoL)Annemarie Docherty (UoE)Ewen Harrison (UoE) + OthersAbstractObjectives To develop and validate a pragmatic risk score to predict inpatient mortality for patients with covid-19 who were enrolled in the International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) World Health Organization (WHO) Clinical Characterisation Protocol UK (CCP-UK) study. Design Prospective observational cohort study. Model training was performed on a dataset up to 20th May 2020. A second cohort of patients was recruited after this and used for validation.Setting 260 hospitals across England, Scotland and Wales.Participants Adult patients (≥18 years) admitted to hospital with covid-19 with at least four weeks follow-up were included.Main outcome measures In-hospital mortality.Results 34,692 patients were included in the derivation dataset (mortality rate 31.7%). The validation dataset contained 22,454 patients (mortality 31.5%). The final model (4C) included eight variables readily available at initial hospital assessment: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, level of consciousness, urea, and C-reactive protein. The 4C risk stratification score demonstrated high discrimination for mortality (Internal: AUROC 0.79; 95% CI 0.78 - 0.79; Validation cohort 0.78, 0.77-0.79) with excellent calibration (slope = 1.0). Patients with a score ≥15 (n = 2310, 17.4%) had a 67% mortality (positive predictive value). Mortality for those with a low score (≤3; n = 918, 7%) was 1.0% (negative predictive value 99%). Discriminatory performance was higher than 15 pre-existing risk stratification scores (AUROC range 0.60-0.76), with scores developed in other covid-19 cohorts often performing poorly (range 0.63-0.73).Conclusions An easy-to-use risk stratification score based on commonly available parameters at hospital presentation outperformed existing scores and can be used to stratify inpatients with covid-19 into different management groups. The prediction model may help clinicians identify patients with covid-19 at high risk of dying during current and subsequent waves. Study registration ISRCTN66726260What is already known on this topicHospitalised patients with covid-19 are at high risk of mortality and uncertainty exists about to how to stratify themThere is considerable interest in risk stratification scores to support frontline clinical decision makingAvailable risk stratification tools however suffer from a high risk of bias, small sample size resulting in uncertainty, poor reporting and lack of formal validationWhat this study addsOur 4C (Coronavirus Clinical Characterisation Consortium) score is an easy-to-use and valid prediction tool for inpatient mortality, accurately categorising patients as being at low, intermediate, high, or very risk of death.This pragmatic and clinically applicable score outperformed other risk stratification tools and had similar performance to more complex models.The majority of pre-existing novel covid-19 risk stratification tools performed poorly in our cohorts – caution should be applied when using novel tools based on small patient populations to in-hospital cohorts with covid-19.IntroductionThe current global pandemic, due to infection with the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has resulted primarily in an acute respiratory illness with deaths predominantly due to respiratory failure. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"tVuxbbZ6","properties":{"formattedCitation":"\\super 1\\nosupersub{}","plainCitation":"1","noteIndex":0},"citationItems":[{"id":5212,"uris":[""],"uri":[""],"itemData":{"id":5212,"type":"article-journal","abstract":"BACKGROUND: An ongoing outbreak of pneumonia associated with the severe acute respiratory coronavirus 2 (SARS-CoV-2) started in December, 2019, in Wuhan, China. Information about critically ill patients with SARS-CoV-2 infection is scarce. We aimed to describe the clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia.\nMETHODS: In this single-centered, retrospective, observational study, we enrolled 52 critically ill adult patients with SARS-CoV-2 pneumonia who were admitted to the intensive care unit (ICU) of Wuhan Jin Yin-tan hospital (Wuhan, China) between late December, 2019, and Jan 26, 2020. Demographic data, symptoms, laboratory values, comorbidities, treatments, and clinical outcomes were all collected. Data were compared between survivors and non-survivors. The primary outcome was 28-day mortality, as of Feb 9, 2020. Secondary outcomes included incidence of SARS-CoV-2-related acute respiratory distress syndrome (ARDS) and the proportion of patients requiring mechanical ventilation.\nFINDINGS: Of 710 patients with SARS-CoV-2 pneumonia, 52 critically ill adult patients were included. The mean age of the 52 patients was 59·7 (SD 13·3) years, 35 (67%) were men, 21 (40%) had chronic illness, 51 (98%) had fever. 32 (61·5%) patients had died at 28 days, and the median duration from admission to the intensive care unit (ICU) to death was 7 (IQR 3-11) days for non-survivors. Compared with survivors, non-survivors were older (64·6 years [11·2] vs 51·9 years [12·9]), more likely to develop ARDS (26 [81%] patients vs 9 [45%] patients), and more likely to receive mechanical ventilation (30 [94%] patients vs 7 [35%] patients), either invasively or non-invasively. Most patients had organ function damage, including 35 (67%) with ARDS, 15 (29%) with acute kidney injury, 12 (23%) with cardiac injury, 15 (29%) with liver dysfunction, and one (2%) with pneumothorax. 37 (71%) patients required mechanical ventilation. Hospital-acquired infection occurred in seven (13·5%) patients.\nINTERPRETATION: The mortality of critically ill patients with SARS-CoV-2 pneumonia is considerable. The survival time of the non-survivors is likely to be within 1-2 weeks after ICU admission. Older patients (>65 years) with comorbidities and ARDS are at increased risk of death. The severity of SARS-CoV-2 pneumonia poses great strain on critical care resources in hospitals, especially if they are not adequately staffed or resourced.\nFUNDING: None.","container-title":"The Lancet. Respiratory Medicine","DOI":"10.1016/S2213-2600(20)30079-5","ISSN":"2213-2619","issue":"5","journalAbbreviation":"Lancet Respir Med","language":"eng","note":"PMID: 32105632\nPMCID: PMC7102538","page":"475-481","source":"PubMed","title":"Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study","title-short":"Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China","volume":"8","author":[{"family":"Yang","given":"Xiaobo"},{"family":"Yu","given":"Yuan"},{"family":"Xu","given":"Jiqian"},{"family":"Shu","given":"Huaqing"},{"family":"Xia","given":"Jia'an"},{"family":"Liu","given":"Hong"},{"family":"Wu","given":"Yongran"},{"family":"Zhang","given":"Lu"},{"family":"Yu","given":"Zhui"},{"family":"Fang","given":"Minghao"},{"family":"Yu","given":"Ting"},{"family":"Wang","given":"Yaxin"},{"family":"Pan","given":"Shangwen"},{"family":"Zou","given":"Xiaojing"},{"family":"Yuan","given":"Shiying"},{"family":"Shang","given":"You"}],"issued":{"date-parts":[["2020"]]}}}],"schema":""} 1 As of 24th July 2020, there are over 14.5 million confirmed cases worldwide and at least 630 000 deaths. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"HWvU3w1M","properties":{"formattedCitation":"\\super 2,3\\nosupersub{}","plainCitation":"2,3","noteIndex":0},"citationItems":[{"id":5188,"uris":[""],"uri":[""],"itemData":{"id":5188,"type":"webpage","abstract":"Coronavirus COVID-19 Global Cases by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU)","container-title":"Johns Hopkins Coronavirus Resource Center","language":"en","note":"source: coronavirus.jhu.edu","title":"COVID-19 Map","URL":"","accessed":{"date-parts":[["2020",5,31]]}}},{"id":5247,"uris":[""],"uri":[""],"itemData":{"id":5247,"type":"webpage","abstract":"Situation reports provide the latest updates on the COVID-19 outbreak. These include updated numbers of infected people and location, and actions that WHO and countries are taking to respond to the outbreak.","language":"en","note":"source: who.int","title":"COVID-19 situation reports","URL":"","accessed":{"date-parts":[["2020",6,1]]}}}],"schema":""} 2,3 As hospitals around the world are faced with an influx of patients with covid-19, there is an urgent need for a pragmatic risk stratification tool that will allow the early identification of patients infected with SARS-CoV-2 who are at the highest risk of death, in order to manage them optimally in a hospital setting.Prognostic scores attempt to transform complex clinical pictures into tangible numerical values. The task of prognostication is more difficult when dealing with a novel illness such as covid-19. Early information has suggested that the clinical course of a patient with covid-19 is different from that of pneumonia, seasonal influenza or sepsis. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"A2n06B2D","properties":{"formattedCitation":"\\super 4\\nosupersub{}","plainCitation":"4","noteIndex":0},"citationItems":[{"id":5249,"uris":[""],"uri":[""],"itemData":{"id":5249,"type":"article-journal","abstract":"Information on severity of coronavirus disease (COVID-19) (transmissibility, disease seriousness, impact) is crucial for preparation of healthcare sectors. We present a simple approach to assess disease seriousness, creating a reference cohort of pneumonia patients from sentinel hospitals. First comparisons exposed a higher rate of COVID-19 patients requiring ventilation. There were more case fatalities among COVID-19 patients without comorbidities than in the reference cohort. Hospitals should prepare for high utilisation of ventilation and intensive care resources.","container-title":"Eurosurveillance","DOI":"10.2807/1560-7917.ES.2020.25.11.2000258","ISSN":"1560-7917","issue":"11","language":"en","note":"publisher: European Centre for Disease Prevention and Control","page":"2000258","source":"","title":"Influenza-associated pneumonia as reference to assess seriousness of coronavirus disease (COVID-19)","volume":"25","author":[{"family":"Tolksdorf","given":"Kristin"},{"family":"Buda","given":"Silke"},{"family":"Schuler","given":"Ekkehard"},{"family":"Wieler","given":"Lothar H."},{"family":"Haas","given":"Walter"}],"issued":{"date-parts":[["2020",3,19]]}}}],"schema":""} 4 The majority of patients with severe covid-19 have developed a clinical picture characterised by pneumonitis, profound hypoxia, and systemic inflammation affecting multiple organs. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"6I6v7o4y","properties":{"formattedCitation":"\\super 1\\nosupersub{}","plainCitation":"1","noteIndex":0},"citationItems":[{"id":5212,"uris":[""],"uri":[""],"itemData":{"id":5212,"type":"article-journal","abstract":"BACKGROUND: An ongoing outbreak of pneumonia associated with the severe acute respiratory coronavirus 2 (SARS-CoV-2) started in December, 2019, in Wuhan, China. Information about critically ill patients with SARS-CoV-2 infection is scarce. We aimed to describe the clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia.\nMETHODS: In this single-centered, retrospective, observational study, we enrolled 52 critically ill adult patients with SARS-CoV-2 pneumonia who were admitted to the intensive care unit (ICU) of Wuhan Jin Yin-tan hospital (Wuhan, China) between late December, 2019, and Jan 26, 2020. Demographic data, symptoms, laboratory values, comorbidities, treatments, and clinical outcomes were all collected. Data were compared between survivors and non-survivors. The primary outcome was 28-day mortality, as of Feb 9, 2020. Secondary outcomes included incidence of SARS-CoV-2-related acute respiratory distress syndrome (ARDS) and the proportion of patients requiring mechanical ventilation.\nFINDINGS: Of 710 patients with SARS-CoV-2 pneumonia, 52 critically ill adult patients were included. The mean age of the 52 patients was 59·7 (SD 13·3) years, 35 (67%) were men, 21 (40%) had chronic illness, 51 (98%) had fever. 32 (61·5%) patients had died at 28 days, and the median duration from admission to the intensive care unit (ICU) to death was 7 (IQR 3-11) days for non-survivors. Compared with survivors, non-survivors were older (64·6 years [11·2] vs 51·9 years [12·9]), more likely to develop ARDS (26 [81%] patients vs 9 [45%] patients), and more likely to receive mechanical ventilation (30 [94%] patients vs 7 [35%] patients), either invasively or non-invasively. Most patients had organ function damage, including 35 (67%) with ARDS, 15 (29%) with acute kidney injury, 12 (23%) with cardiac injury, 15 (29%) with liver dysfunction, and one (2%) with pneumothorax. 37 (71%) patients required mechanical ventilation. Hospital-acquired infection occurred in seven (13·5%) patients.\nINTERPRETATION: The mortality of critically ill patients with SARS-CoV-2 pneumonia is considerable. The survival time of the non-survivors is likely to be within 1-2 weeks after ICU admission. Older patients (>65 years) with comorbidities and ARDS are at increased risk of death. The severity of SARS-CoV-2 pneumonia poses great strain on critical care resources in hospitals, especially if they are not adequately staffed or resourced.\nFUNDING: None.","container-title":"The Lancet. Respiratory Medicine","DOI":"10.1016/S2213-2600(20)30079-5","ISSN":"2213-2619","issue":"5","journalAbbreviation":"Lancet Respir Med","language":"eng","note":"PMID: 32105632\nPMCID: PMC7102538","page":"475-481","source":"PubMed","title":"Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study","title-short":"Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China","volume":"8","author":[{"family":"Yang","given":"Xiaobo"},{"family":"Yu","given":"Yuan"},{"family":"Xu","given":"Jiqian"},{"family":"Shu","given":"Huaqing"},{"family":"Xia","given":"Jia'an"},{"family":"Liu","given":"Hong"},{"family":"Wu","given":"Yongran"},{"family":"Zhang","given":"Lu"},{"family":"Yu","given":"Zhui"},{"family":"Fang","given":"Minghao"},{"family":"Yu","given":"Ting"},{"family":"Wang","given":"Yaxin"},{"family":"Pan","given":"Shangwen"},{"family":"Zou","given":"Xiaojing"},{"family":"Yuan","given":"Shiying"},{"family":"Shang","given":"You"}],"issued":{"date-parts":[["2020"]]}}}],"schema":""} 1A recent review identified many prognostic scores used for covid-19, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"gW4Tzetq","properties":{"formattedCitation":"\\super 5\\nosupersub{}","plainCitation":"5","noteIndex":0},"citationItems":[{"id":5193,"uris":[""],"uri":[""],"itemData":{"id":5193,"type":"article-journal","abstract":"Objective To review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.\nDesign Rapid systematic review and critical appraisal.\nData sources PubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.\nStudy selection Studies that developed or validated a multivariable covid-19 related prediction model.\nData extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).\nResults 2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.\nConclusion Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.\nSystematic review registration Protocol , registration .","container-title":"BMJ","DOI":"10.1136/bmj.m1328","ISSN":"1756-1833","journalAbbreviation":"BMJ","language":"en","note":"publisher: British Medical Journal Publishing Group\nsection: Research\nPMID: 32265220","source":"","title":"Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal","title-short":"Prediction models for diagnosis and prognosis of covid-19 infection","URL":"","volume":"369","author":[{"family":"Wynants","given":"Laure"},{"family":"Calster","given":"Ben Van"},{"family":"Bonten","given":"Marc M. J."},{"family":"Collins","given":"Gary S."},{"family":"Debray","given":"Thomas P. A."},{"family":"Vos","given":"Maarten De"},{"family":"Haller","given":"Maria C."},{"family":"Heinze","given":"Georg"},{"family":"Moons","given":"Karel G. M."},{"family":"Riley","given":"Richard D."},{"family":"Schuit","given":"Ewoud"},{"family":"Smits","given":"Luc J. M."},{"family":"Snell","given":"Kym I. E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Wallisch","given":"Christine"},{"family":"Smeden","given":"Maarten","dropping-particle":"van"}],"accessed":{"date-parts":[["2020",5,31]]},"issued":{"date-parts":[["2020",4,7]]}}}],"schema":""} 5 which varied in their setting, predicted outcome measure, and the clinical parameters included. The large number of risk stratification tools reflects difficulties in their application, with most scores demonstrating moderate performance at best and poorer performance when generalised across different populations. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"gfnhqeoW","properties":{"formattedCitation":"\\super 6\\nosupersub{}","plainCitation":"6","noteIndex":0},"citationItems":[{"id":5207,"uris":[""],"uri":[""],"itemData":{"id":5207,"type":"article-journal","abstract":"BACKGROUND: Age-related alterations in the clinical characteristics and performance of severity scoring systems for community-acquired pneumonia (CAP) are unknown.\nMETHODS: Consecutive patients with CAP presenting to the emergency department were prospectively studied. Patients were classified as younger adults (age 18-64 years), elderly (age 65-84 years) and very old subjects (age ≥85 years). Clinical characteristics, complications, outcomes and validity of the pneumonia severity index (PSI) and CURB-65 categories were compared across these three age categories.\nRESULTS: Analysis involved 348 (35.3%) younger adult patients, 438 (44.3%) elderly patients and 201 (20.0%) very old patients. Compared with younger adults, elderly and very old patients had a higher burden of comorbidities and a higher incidence of CAP-related complications. The 30-day mortality rate was 5.2% in younger adults, 7.1% in elderly patients and 9.5% in very old patients. The area under the ROC curve (AUCs) for PSI were 0.87 (95% CI 0.77 to 0.97), 0.85 (95% CI 0.803 to 0.897) and 0.69 (95% CI 0.597 to 0.787) and the AUCs for CURB-65 were 0.80 (95% CI 0.67 to 0.93), 0.73 (95% CI 0.65 to 0.82) and 0.60 (95% CI 0.47 to 0.73) in the younger adult, elderly and very old patients, respectively. A modified PSI or CURB-65 excluding the age variable increased the AUC in most age categories. There was no significant effect of age on 30-day mortality after adjusting for other PSI or CURB-65 variables.\nCONCLUSION: Elderly patients with CAP have more atypical clinical manifestations and worse outcomes. The underperformance of the PSI in elderly patients may be due to the inappropriate weight given to the age variable. A modification of the cut-off point for PSI or CURB-65 to define severe pneumonia may improve the score performance in elderly patients.","container-title":"Thorax","DOI":"10.1136/thx.2009.129627","ISSN":"1468-3296","issue":"11","journalAbbreviation":"Thorax","language":"eng","note":"PMID: 20965934","page":"971-977","source":"PubMed","title":"Comparison of clinical characteristics and performance of pneumonia severity score and CURB-65 among younger adults, elderly and very old subjects","volume":"65","author":[{"family":"Chen","given":"Jung-Hsiang"},{"family":"Chang","given":"Shy-Shin"},{"family":"Liu","given":"Jason J."},{"family":"Chan","given":"Rai-Chi"},{"family":"Wu","given":"Jiunn-Yih"},{"family":"Wang","given":"Wei-Chuan"},{"family":"Lee","given":"Si-Huei"},{"family":"Lee","given":"Chien-Chang"}],"issued":{"date-parts":[["2010",11]]}}}],"schema":""} 6 It has been suggested that a number of novel covid-19 prognostic scores have a high risk of bias, which may reflect development in small cohorts, and many have been published without clear details of model derivation and testing. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"YhSLS6pR","properties":{"formattedCitation":"\\super 5\\nosupersub{}","plainCitation":"5","noteIndex":0},"citationItems":[{"id":5193,"uris":[""],"uri":[""],"itemData":{"id":5193,"type":"article-journal","abstract":"Objective To review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.\nDesign Rapid systematic review and critical appraisal.\nData sources PubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.\nStudy selection Studies that developed or validated a multivariable covid-19 related prediction model.\nData extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).\nResults 2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.\nConclusion Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.\nSystematic review registration Protocol , registration .","container-title":"BMJ","DOI":"10.1136/bmj.m1328","ISSN":"1756-1833","journalAbbreviation":"BMJ","language":"en","note":"publisher: British Medical Journal Publishing Group\nsection: Research\nPMID: 32265220","source":"","title":"Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal","title-short":"Prediction models for diagnosis and prognosis of covid-19 infection","URL":"","volume":"369","author":[{"family":"Wynants","given":"Laure"},{"family":"Calster","given":"Ben Van"},{"family":"Bonten","given":"Marc M. J."},{"family":"Collins","given":"Gary S."},{"family":"Debray","given":"Thomas P. A."},{"family":"Vos","given":"Maarten De"},{"family":"Haller","given":"Maria C."},{"family":"Heinze","given":"Georg"},{"family":"Moons","given":"Karel G. M."},{"family":"Riley","given":"Richard D."},{"family":"Schuit","given":"Ewoud"},{"family":"Smits","given":"Luc J. M."},{"family":"Snell","given":"Kym I. E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Wallisch","given":"Christine"},{"family":"Smeden","given":"Maarten","dropping-particle":"van"}],"accessed":{"date-parts":[["2020",5,31]]},"issued":{"date-parts":[["2020",4,7]]}}}],"schema":""} 5 To the authors knowledge, a risk stratification tool is yet to be developed and validated within a large national cohort of hospitalised patients with covid-19.Our aim was to develop and validate a pragmatic, clinically relevant risk stratification score using routinely available clinical information at hospital presentation to predict in-hospital mortality in hospitalised covid-19 patients recruited to the ISARIC CCP-UK study and then compare this with existing prognostic models.Methods Study design and settingThe International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) WHO Clinical Characterisation Protocol UK (CCP-UK) study is an ongoing prospective cohort study in 260 acute care hospitals in England, Scotland, and Wales (National Institute for Health Research Clinical Research Network Central Portfolio Management System ID: 14152) performed by the ISARIC Covid-19 Clinical Characterisation Consortium (ISARIC-4C). The protocol and further study details are available online. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"4mJqGS2b","properties":{"formattedCitation":"\\super 7\\nosupersub{}","plainCitation":"7","noteIndex":0},"citationItems":[{"id":5186,"uris":[""],"uri":[""],"itemData":{"id":5186,"type":"webpage","title":"Features of 20?133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study | The BMJ","URL":"","accessed":{"date-parts":[["2020",5,30]]}},"locator":"16"}],"schema":""} 7 Model development and reporting followed the Transparent Reporting of a multivariate prediction mode for Individual Prediction or Diagnosis (TRIPOD) guidelines. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"vs4T1YWQ","properties":{"formattedCitation":"\\super 8\\nosupersub{}","plainCitation":"8","noteIndex":0},"citationItems":[{"id":5190,"uris":[""],"uri":[""],"itemData":{"id":5190,"type":"article-journal","abstract":"Prediction models are developed to aid health care providers in estimating the probability or risk that a specific disease or condition is present (diagnostic models) or that a specific event will occur in the future (prognostic models), to inform their decision making. However, the overwhelming evidence shows that the quality of reporting of prediction model studies is poor. Only with full and clear reporting of information on all aspects of a prediction model can risk of bias and potential usefulness of prediction models be adequately assessed. The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Initiative developed a set of recommendations for the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. This article describes how the TRIPOD Statement was developed. An extensive list of items based on a review of the literature was created, which was reduced after a Web based survey and revised during a three day meeting in June 2011 with methodologists, health care professionals, and journal editors. The list was refined during several meetings of the steering group and in e-mail discussions with the wider group of TRIPOD contributors. The resulting TRIPOD Statement is a checklist of 22 items, deemed essential for transparent reporting of a prediction model study. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. The TRIPOD Statement is best used in conjunction with the TRIPOD explanation and elaboration document. To aid the editorial process and readers of prediction model studies, it is recommended that authors include a completed checklist in their submission (also available at tripod-).To encourage dissemination of the TRIPOD Statement, this article is freely accessible on the Annals of Internal Medicine Web site () and will be also published in BJOG, British Journal of Cancer, British Journal of Surgery, BMC Medicine, The BMJ, Circulation, Diabetic Medicine, European Journal of Clinical Investigation, European Urology, and Journal of Clinical Epidemiology. The authors jointly hold the copyright of this article. An accompanying explanation and elaboration article is freely available only on ; Annals of Internal Medicine holds copyright for that article.","container-title":"BMJ (Clinical research ed.)","DOI":"10.1136/bmj.g7594","ISSN":"1756-1833","journalAbbreviation":"BMJ","language":"eng","note":"PMID: 25569120","page":"g7594","source":"PubMed","title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement","title-short":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD)","volume":"350","author":[{"family":"Collins","given":"Gary S."},{"family":"Reitsma","given":"Johannes B."},{"family":"Altman","given":"Douglas G."},{"family":"Moons","given":"Karel G. M."}],"issued":{"date-parts":[["2015",1,7]]}}}],"schema":""} 8 The study was performed according to a pre-defined protocol (Appendix 1).ParticipantsPatients aged ≥18 years old with a completed index admission to one of 260 hospitals in England, Scotland, and Wales were included. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"FXV2YPHv","properties":{"formattedCitation":"\\super 7\\nosupersub{}","plainCitation":"7","noteIndex":0},"citationItems":[{"id":5186,"uris":[""],"uri":[""],"itemData":{"id":5186,"type":"webpage","title":"Features of 20?133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study | The BMJ","URL":"","accessed":{"date-parts":[["2020",5,30]]}},"locator":"133"}],"schema":""} 7 As specified in the protocol, only patients with proven or high likelihood of infection with a pathogen of public health interest, defined as SARS-CoV-2 for this event by Public Health England, were eligible. Reverse transcriptase polymerase chain reaction was the only diagnostic testing available during the study period, with the decision to test at the discretion of the clinician attending the patient, and not defined in the protocol. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"YfFt24wq","properties":{"formattedCitation":"\\super 7\\nosupersub{}","plainCitation":"7","noteIndex":0},"citationItems":[{"id":5186,"uris":[""],"uri":[""],"itemData":{"id":5186,"type":"webpage","title":"Features of 20?133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study | The BMJ","URL":"","accessed":{"date-parts":[["2020",5,30]]}},"locator":"16"}],"schema":""} 7 The enrolment criterion “high likelihood of infection” reflects that a preparedness protocol cannot assume that a diagnostic test will be available for an emergent pathogen. Site training emphasises that only patients who tested positive for SARS-CoV-2 were eligible for enrolment.Data collectionDemographic, clinical and outcomes data were collected using a pre-specified case report form. Comorbidities were defined according to a modified Charlson Comorbidity Index ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"ZLewhkoK","properties":{"formattedCitation":"\\super 9\\nosupersub{}","plainCitation":"9","noteIndex":0},"citationItems":[{"id":5309,"uris":[""],"uri":[""],"itemData":{"id":5309,"type":"article-journal","abstract":"The objective of this study was to develop a prospectively applicable method for classifying comorbid conditions which might alter the risk of mortality for use in longitudinal studies. A weighted index that takes into account the number and the seriousness of comorbid disease was developed in a cohort of 559 medical patients. The 1-yr mortality rates for the different scores were: \"0\", 12% (181); \"1-2\", 26% (225); \"3-4\", 52% (71); and \"greater than or equal to 5\", 85% (82). The index was tested for its ability to predict risk of death from comorbid disease in the second cohort of 685 patients during a 10-yr follow-up. The percent of patients who died of comorbid disease for the different scores were: \"0\", 8% (588); \"1\", 25% (54); \"2\", 48% (25); \"greater than or equal to 3\", 59% (18). With each increased level of the comorbidity index, there were stepwise increases in the cumulative mortality attributable to comorbid disease (log rank chi 2 = 165; p less than 0.0001). In this longer follow-up, age was also a predictor of mortality (p less than 0.001). The new index performed similarly to a previous system devised by Kaplan and Feinstein. The method of classifying comorbidity provides a simple, readily applicable and valid method of estimating risk of death from comorbid disease for use in longitudinal studies. Further work in larger populations is still required to refine the approach because the number of patients with any given condition in this study was relatively small.","container-title":"Journal of Chronic Diseases","DOI":"10.1016/0021-9681(87)90171-8","ISSN":"0021-9681","issue":"5","journalAbbreviation":"J Chronic Dis","language":"eng","note":"PMID: 3558716","page":"373-383","source":"PubMed","title":"A new method of classifying prognostic comorbidity in longitudinal studies: development and validation","title-short":"A new method of classifying prognostic comorbidity in longitudinal studies","volume":"40","author":[{"family":"Charlson","given":"M. E."},{"family":"Pompei","given":"P."},{"family":"Ales","given":"K. L."},{"family":"MacKenzie","given":"C. R."}],"issued":{"date-parts":[["1987"]]}}}],"schema":""} 9 with chronic cardiac disease; chronic respiratory disease (excluding asthma); chronic renal disease (estimated glomerular filtration rate ≤30); mild to severe liver disease; dementia; chronic neurological conditions; connective tissue disease; diabetes mellitus (diet, tablet or insulin-controlled); HIV/AIDS, and malignancy collected. These were selected a priori by a global consortium to provide rapid, coordinated clinical investigation of patients presenting with any severe or potentially severe acute infection of public interest and enabled standardisation. Clinician-defined obesity was also included as a comorbidity due to possible association with adverse outcomes in patients with covid-19. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"awIco09u","properties":{"formattedCitation":"\\super 10,11\\nosupersub{}","plainCitation":"10,11","noteIndex":0},"citationItems":[{"id":5305,"uris":[""],"uri":[""],"itemData":{"id":5305,"type":"article-journal","container-title":"Circulation","DOI":"10.1161/CIRCULATIONAHA.120.047659","ISSN":"1524-4539","issue":"1","journalAbbreviation":"Circulation","language":"eng","note":"PMID: 32320270","page":"4-6","source":"PubMed","title":"Obesity Is a Risk Factor for Severe COVID-19 Infection: Multiple Potential Mechanisms","title-short":"Obesity Is a Risk Factor for Severe COVID-19 Infection","volume":"142","author":[{"family":"Sattar","given":"Naveed"},{"family":"McInnes","given":"Iain B."},{"family":"McMurray","given":"John J. V."}],"issued":{"date-parts":[["2020"]],"season":"07"}}},{"id":5304,"uris":[""],"uri":[""],"itemData":{"id":5304,"type":"article-journal","abstract":"OBJECTIVE: The COVID-19 pandemic is rapidly spreading worldwide, notably in Europe and North America where obesity is highly prevalent. The relation between obesity and severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) has not been fully documented.\nMETHODS: This retrospective cohort study analyzed the relationship between clinical characteristics, including BMI,?and the requirement for invasive mechanical ventilation (IMV) in 124 consecutive patients admitted in intensive care for SARS-CoV-2 in a single French center.\nRESULTS: Obesity (BMI?>?30) and severe obesity (BMI?>?35) were present in 47.6% and 28.2% of cases, respectively. Overall, 85 patients (68.6%) required IMV. The proportion of patients who required IMV increased with BMI categories (P?<?0.01, χ2 test for trend), and it was greatest in patients with BMI?>?35 (85.7%). In multivariate logistic regression, the need for IMV was significantly associated with male sex (P?<?0.05) and BMI (P?<?0.05), independent of age, diabetes, and hypertension. The odds ratio for IMV in patients with BMI?>?35 versus patients with BMI?<?25 was?7.36 (1.63-33.14; P?=?0.02).\nCONCLUSIONS: The present study showed a high frequency of obesity among patients admitted in intensive care for SARS-CoV-2. Disease severity increased with BMI. Obesity is a risk factor for SARS-CoV-2 severity, requiring increased attention to preventive measures in susceptible individuals.","container-title":"Obesity (Silver Spring, Md.)","DOI":"10.1002/oby.22831","ISSN":"1930-739X","issue":"7","journalAbbreviation":"Obesity (Silver Spring)","language":"eng","note":"PMID: 32271993\nPMCID: PMC7262326","page":"1195-1199","source":"PubMed","title":"High Prevalence of Obesity in Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) Requiring Invasive Mechanical Ventilation","volume":"28","author":[{"family":"Simonnet","given":"Arthur"},{"family":"Chetboun","given":"Mikael"},{"family":"Poissy","given":"Julien"},{"family":"Raverdy","given":"Violeta"},{"family":"Noulette","given":"Jerome"},{"family":"Duhamel","given":"Alain"},{"family":"Labreuche","given":"Julien"},{"family":"Mathieu","given":"Daniel"},{"family":"Pattou","given":"Francois"},{"family":"Jourdain","given":"Merce"},{"literal":"LICORN and the?Lille COVID-19?and?Obesity study group"}],"issued":{"date-parts":[["2020"]]}}}],"schema":""} 10,11 In patients where data on all comorbidities were missing, the absence of comorbidity was assumed. The clinical information used to calculate prognostic scores was taken from the day of admission to hospital. Ethical approval for the study was given by the South Central – Oxford C Research Ethics Committee in England (Ref 13/SC/0149), the Scotland A Research Ethics Committee (Ref 20/SS/0028), and the WHO Ethics Review Committee (RPC571 and RPC572, 25 April 2013).OutcomesThe primary outcome was in-hospital mortality. This outcome was selected due to the importance of the early identification of patients who were likely to develop severe illness from SARS-CoV-2 infection (a ‘rule in’ test). We chose a priori to restrict analysis of outcomes to patients who were admitted more than four weeks before final data extraction (29th June 2020) to enable most patients to complete their hospital admission.Independent predictor variablesA reduced set of potential predictor variables was selected a priori including patient demographic information, common clinical investigations, and parameters consistently identified as clinically important in covid-19 cohorts following methodology described by Wynants et al. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"IKzj7oPc","properties":{"formattedCitation":"\\super 5\\nosupersub{}","plainCitation":"5","noteIndex":0},"citationItems":[{"id":5193,"uris":[""],"uri":[""],"itemData":{"id":5193,"type":"article-journal","abstract":"Objective To review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.\nDesign Rapid systematic review and critical appraisal.\nData sources PubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.\nStudy selection Studies that developed or validated a multivariable covid-19 related prediction model.\nData extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).\nResults 2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.\nConclusion Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.\nSystematic review registration Protocol , registration .","container-title":"BMJ","DOI":"10.1136/bmj.m1328","ISSN":"1756-1833","journalAbbreviation":"BMJ","language":"en","note":"publisher: British Medical Journal Publishing Group\nsection: Research\nPMID: 32265220","source":"","title":"Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal","title-short":"Prediction models for diagnosis and prognosis of covid-19 infection","URL":"","volume":"369","author":[{"family":"Wynants","given":"Laure"},{"family":"Calster","given":"Ben Van"},{"family":"Bonten","given":"Marc M. J."},{"family":"Collins","given":"Gary S."},{"family":"Debray","given":"Thomas P. A."},{"family":"Vos","given":"Maarten De"},{"family":"Haller","given":"Maria C."},{"family":"Heinze","given":"Georg"},{"family":"Moons","given":"Karel G. M."},{"family":"Riley","given":"Richard D."},{"family":"Schuit","given":"Ewoud"},{"family":"Smits","given":"Luc J. M."},{"family":"Snell","given":"Kym I. E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Wallisch","given":"Christine"},{"family":"Smeden","given":"Maarten","dropping-particle":"van"}],"accessed":{"date-parts":[["2020",5,31]]},"issued":{"date-parts":[["2020",4,7]]}}}],"schema":""} 5 Candidate predictor variables were selected based on three common criteria ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"6kmSJOH4","properties":{"formattedCitation":"\\super 12\\nosupersub{}","plainCitation":"12","noteIndex":0},"citationItems":[{"id":5269,"uris":[""],"uri":[""],"itemData":{"id":5269,"type":"article-journal","abstract":"The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from tripod-.","container-title":"Annals of Internal Medicine","DOI":"10.7326/M14-0698","ISSN":"1539-3704","issue":"1","journalAbbreviation":"Ann. Intern. Med.","language":"eng","note":"PMID: 25560730","page":"W1-73","source":"PubMed","title":"Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration","title-short":"Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD)","volume":"162","author":[{"family":"Moons","given":"Karel G. M."},{"family":"Altman","given":"Douglas G."},{"family":"Reitsma","given":"Johannes B."},{"family":"Ioannidis","given":"John P. A."},{"family":"Macaskill","given":"Petra"},{"family":"Steyerberg","given":"Ewout W."},{"family":"Vickers","given":"Andrew J."},{"family":"Ransohoff","given":"David F."},{"family":"Collins","given":"Gary S."}],"issued":{"date-parts":[["2015",1,6]]}}}],"schema":""} 12: (1) patient and clinical variables known to influence outcome in pneumonia and flu-like illness; (2) clinical biomarkers previously identified within the literature as potential predictors in covid-19 patients; and (3) values were available for at least two-thirds of patients within the derivation cohort.With the overall aim to develop an easy-to-use risk stratification score, an a priori decision was made to include an overall comorbidity count for each patient within model development, rather than individual comorbidities. Recent evidence suggests an additive effect of comorbidity in covid-19 patients, with increasing number of comorbidities associated with poorer outcomes. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"UD16oazv","properties":{"formattedCitation":"\\super 13\\nosupersub{}","plainCitation":"13","noteIndex":0},"citationItems":[{"id":5312,"uris":[""],"uri":[""],"itemData":{"id":5312,"type":"article-journal","abstract":"Background The coronavirus disease 2019 (COVID-19) outbreak is evolving rapidly worldwide.\nObjective To evaluate the risk of serious adverse outcomes in patients with COVID-19 by stratifying the comorbidity status.\nMethods We analysed data from 1590 laboratory confirmed hospitalised patients from 575 hospitals in 31 provinces/autonomous regions/provincial municipalities across mainland China between 11 December 2019 and 31 January 2020. We analysed the composite end-points, which consisted of admission to an intensive care unit, invasive ventilation or death. The risk of reaching the composite end-points was compared according to the presence and number of comorbidities.\nResults The mean age was 48.9 years and 686 (42.7%) patients were female. Severe cases accounted for 16.0% of the study population. 131 (8.2%) patients reached the composite end-points. 399 (25.1%) reported having at least one comorbidity. The most prevalent comorbidity was hypertension (16.9%), followed by diabetes (8.2%). 130 (8.2%) patients reported having two or more comorbidities. After adjusting for age and smoking status, COPD (HR (95% CI) 2.681 (1.424–5.048)), diabetes (1.59 (1.03–2.45)), hypertension (1.58 (1.07–2.32)) and malignancy (3.50 (1.60–7.64)) were risk factors of reaching the composite end-points. The hazard ratio (95% CI) was 1.79 (1.16–2.77) among patients with at least one comorbidity and 2.59 (1.61–4.17) among patients with two or more comorbidities.\nConclusion Among laboratory confirmed cases of COVID-19, patients with any comorbidity yielded poorer clinical outcomes than those without. A greater number of comorbidities also correlated with poorer clinical outcomes.\nTweetable abstract @ERSpublications\nclick to tweetThe presence and number of comorbidities predict clinical outcomes of COVID-19 ","container-title":"European Respiratory Journal","DOI":"10.1183/13993003.00547-2020","ISSN":"0903-1936, 1399-3003","issue":"5","language":"en","note":"publisher: European Respiratory Society\nsection: Original Articles\nPMID: 32217650","source":"erj.","title":"Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis","title-short":"Comorbidity and its impact on 1590 patients with COVID-19 in China","URL":"","volume":"55","author":[{"family":"Guan","given":"Wei-jie"},{"family":"Liang","given":"Wen-hua"},{"family":"Zhao","given":"Yi"},{"family":"Liang","given":"Heng-rui"},{"family":"Chen","given":"Zi-sheng"},{"family":"Li","given":"Yi-min"},{"family":"Liu","given":"Xiao-qing"},{"family":"Chen","given":"Ru-chong"},{"family":"Tang","given":"Chun-li"},{"family":"Wang","given":"Tao"},{"family":"Ou","given":"Chun-quan"},{"family":"Li","given":"Li"},{"family":"Chen","given":"Ping-yan"},{"family":"Sang","given":"Ling"},{"family":"Wang","given":"Wei"},{"family":"Li","given":"Jian-fu"},{"family":"Li","given":"Cai-chen"},{"family":"Ou","given":"Li-min"},{"family":"Cheng","given":"Bo"},{"family":"Xiong","given":"Shan"},{"family":"Ni","given":"Zheng-yi"},{"family":"Xiang","given":"Jie"},{"family":"Hu","given":"Yu"},{"family":"Liu","given":"Lei"},{"family":"Shan","given":"Hong"},{"family":"Lei","given":"Chun-liang"},{"family":"Peng","given":"Yi-xiang"},{"family":"Wei","given":"Li"},{"family":"Liu","given":"Yong"},{"family":"Hu","given":"Ya-hua"},{"family":"Peng","given":"Peng"},{"family":"Wang","given":"Jian-ming"},{"family":"Liu","given":"Ji-yang"},{"family":"Chen","given":"Zhong"},{"family":"Li","given":"Gang"},{"family":"Zheng","given":"Zhi-jian"},{"family":"Qiu","given":"Shao-qin"},{"family":"Luo","given":"Jie"},{"family":"Ye","given":"Chang-jiang"},{"family":"Zhu","given":"Shao-yong"},{"family":"Cheng","given":"Lin-ling"},{"family":"Ye","given":"Feng"},{"family":"Li","given":"Shi-yue"},{"family":"Zheng","given":"Jin-ping"},{"family":"Zhang","given":"Nuo-fu"},{"family":"Zhong","given":"Nan-shan"},{"family":"He","given":"Jian-xing"}],"accessed":{"date-parts":[["2020",7,18]]},"issued":{"date-parts":[["2020",5,1]]}}}],"schema":""} 13Model developmentModels were trained using all available data (up to 20th May 2020). The primary intention was to create a pragmatic model for bedside use not requiring complex equations, online calculators, or applications. An a priori decision was therefore made to categorise continuous variables in the final prognostic score. A three-stage model building process was used (Figure 1). Firstly, generalised additive models (GAMs) were built incorporating continuous smoothed predictors (thin-plate splines) in combination with categorical predictors as linear components. A criterion-based approach to variable selection was taken based on the deviance explained, the unbiased risk estimator, and the area under the receiver operating characteristic curve (AUROC). Secondly, plots of component smoothed continuous predictors were inspected for linearity and optimal cut-points were selected. Lastly, final models using categorised variables were specified using least absolute shrinkage and selection operator (LASSO) logistic regression. L1-penalised coefficients were derived using ten-fold cross-validation to select the value of lambda (minimised cross-validated sum of squared residuals). Shrunk coefficients were converted to a prognostic index with appropriate scaling to create the pragmatic “4C” score (4C stands for Coronavirus Clinical Characterisation Consortium).Machine learning approaches were used in parallel for comparison of predictive performance. Given issues with interpretability, this was intended to provide a “best-in-class” comparison of predictive performance when accounting for any complex underlying interactions. Extreme gradient boosting trees were used (XGBoost). All candidate predictor variables identified were included within the model, with the exception of those with high missing values (>33%). Individual major comorbidity variables, defined as chronic cardiac disease; chronic respiratory disease (excluding asthma); chronic renal disease (estimated glomerular filtration rate ≤30); moderate to severe liver failure (presence of portal hypertension); diabetes mellitus (diet, tablet or insulin-controlled) and solid malignancy, together with obesity, were contained within the model to determine whether their inclusion enhanced predictive performance. An 80%/20% random split of the derivation dataset was used to define train/test sets. The validation datasets were held back and not used in the training process. A mortality label and design matrix of centred/standardised continuous and categorical variables including all candidate variables was used to train gradient boosted trees minimising the binary classification error rate (defined as number wrong cases / number all cases). Hyperparameters were tuned including the learning rate (eta) and maximum tree depth to maximise the AUROC in the test set. Discrimination was assessed for all above models (4C and XGBoost model) using the AUROC in the derivation cohort, with 95% confidence intervals calculated using bootstrapping resampling (2000 samples). An AUROC value of 0.5 indicates no predictive ability, 0.8 is considered good, and 1.0 is perfect. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"vOX5LGgS","properties":{"formattedCitation":"\\super 14\\nosupersub{}","plainCitation":"14","noteIndex":0},"citationItems":[{"id":5280,"uris":[""],"uri":[""],"itemData":{"id":5280,"type":"article-journal","abstract":"The performance of a diagnostic test in the case of a binary predictor can be evaluated using the measures of sensitivity and speci?city. However, in many instances, we encounter predictors that are measured on a continuous or ordinal scale. In such cases, it is desirable to assess performance of a diagnostic test over the range of possible cutpoints for the predictor variable. This is achieved by a receiver operating characteristic (ROC) curve that includes all the possible decision thresholds from a diagnostic test result. In this brief report, we discuss the salient features of the ROC curve, as well as discuss and interpret the area under the ROC curve, and its utility in comparing two different tests or predictor variables of interest.","container-title":"Journal of Thoracic Oncology","DOI":"10.1097/JTO.0b013e3181ec173d","ISSN":"15560864","issue":"9","language":"en","page":"1315-1316","source":"Crossref","title":"Receiver Operating Characteristic Curve in Diagnostic Test Assessment","volume":"5","author":[{"family":"Mandrekar","given":"Jayawant N."}],"issued":{"date-parts":[["2010",9]]}}}],"schema":""} 14 Overall goodness-of-fit was assessed with the Brier score, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"VWNsQpLw","properties":{"formattedCitation":"\\super 15\\nosupersub{}","plainCitation":"15","noteIndex":0},"citationItems":[{"id":5285,"uris":[""],"uri":[""],"itemData":{"id":5285,"type":"article-journal","abstract":"The performance of prediction models can be assessed using a variety of different methods and metrics. Traditional measures for binary and survival outcomes include the Brier score to indicate overall model performance, the concordance (or c) statistic for discriminative ability (or area under the receiver operating characteristic (ROC) curve), and goodness-of-fit statistics for calibration., Several new measures have recently been proposed that can be seen as refinements of discrimination measures, including variants of the c statistic for survival, reclassification tables, net reclassification improvement (NRI), and integrated discrimination improvement (IDI). Moreover, decision–analytic measures have been proposed, including decision curves to plot the net benefit achieved by making decisions based on model predictions., We aimed to define the role of these relatively novel approaches in the evaluation of the performance of prediction models. For illustration we present a case study of predicting the presence of residual tumor versus benign tissue in patients with testicular cancer (n=544 for model development, n=273 for external validation)., We suggest that reporting discrimination and calibration will always be important for a prediction model. Decision-analytic measures should be reported if the predictive model is to be used for making clinical decisions. Other measures of performance may be warranted in specific applications, such as reclassification metrics to gain insight into the value of adding a novel predictor to an established model.","container-title":"Epidemiology (Cambridge, Mass.)","DOI":"10.1097/EDE.0b013e3181c30fb2","ISSN":"1044-3983","issue":"1","journalAbbreviation":"Epidemiology","note":"PMID: 20010215\nPMCID: PMC3575184","page":"128-138","source":"PubMed Central","title":"Assessing the performance of prediction models: a framework for some traditional and novel measures","title-short":"Assessing the performance of prediction models","volume":"21","author":[{"family":"Steyerberg","given":"Ewout W."},{"family":"Vickers","given":"Andrew J."},{"family":"Cook","given":"Nancy R."},{"family":"Gerds","given":"Thomas"},{"family":"Gonen","given":"Mithat"},{"family":"Obuchowski","given":"Nancy"},{"family":"Pencina","given":"Michael J."},{"family":"Kattan","given":"Michael W."}],"issued":{"date-parts":[["2010",1]]}}}],"schema":""} 15 a measure to quantify how close predictions are to the truth ranging between 0 and 1, where smaller values indicate superior model performance. We plotted model calibration curves to examine agreement between predicted and observed risk across deciles of mortality risk to ascertain the presence of over- or under-prediction. Risk cut-off values were defined by the total point score for an individual which represented a low (<2% mortality rate), intermediate (2-14.9%) or high-risk (≥15%) groups, similar to commonly used pneumonia risk stratification scores. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"edRiR2KH","properties":{"formattedCitation":"\\super 16,17\\nosupersub{}","plainCitation":"16,17","noteIndex":0},"citationItems":[{"id":5300,"uris":[""],"uri":[""],"itemData":{"id":5300,"type":"article-journal","abstract":"Background: In the assessment of severity in community acquired pneumonia (CAP), the modified British Thoracic Society (mBTS) rule identifies patients with severe pneumonia but not patients who might be suitable for home management. A multicentre study was conducted to derive and validate a practical severity assessment model for stratifying adults hospitalised with CAP into different management groups.\nMethods: Data from three prospective studies of CAP conducted in the UK, New Zealand, and the Netherlands were combined. A derivation cohort comprising 80% of the data was used to develop the model. Prognostic variables were identified using multiple logistic regression with 30 day mortality as the outcome measure. The final model was tested against the validation cohort.\nResults: 1068 patients were studied (mean age 64 years, 51.5% male, 30 day mortality 9%). Age ?65 years (OR 3.5, 95% CI 1.6 to 8.0) and albumin <30 g/dl (OR 4.7, 95% CI 2.5 to 8.7) were independently associated with mortality over and above the mBTS rule (OR 5.2, 95% CI 2.7 to 10). A six point score, one point for each of Confusion, Urea >7 mmol/l, Respiratory rate ?30/min, low systolic(<90 mm Hg) or diastolic (?60 mm Hg) Blood pressure), age ?65 years (CURB-65 score) based on information available at initial hospital assessment, enabled patients to be stratified according to increasing risk of mortality: score 0, 0.7%; score 1, 3.2%; score 2, 3%; score 3, 17%; score 4, 41.5% and score 5, 57%. The validation cohort confirmed a similar pattern.\nConclusions: A simple six point score based on confusion, urea, respiratory rate, blood pressure, and age can be used to stratify patients with CAP into different management groups.","container-title":"Thorax","DOI":"10.1136/thorax.58.5.377","ISSN":"0040-6376, 1468-3296","issue":"5","language":"en","note":"publisher: BMJ Publishing Group Ltd\nsection: Respiratory infection\nPMID: 12728155","page":"377-382","source":"thorax.","title":"Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study","title-short":"Defining community acquired pneumonia severity on presentation to hospital","volume":"58","author":[{"family":"Lim","given":"W. S."},{"family":"Eerden","given":"M. M.","dropping-particle":"van der"},{"family":"Laing","given":"R."},{"family":"Boersma","given":"W. G."},{"family":"Karalus","given":"N."},{"family":"Town","given":"G. I."},{"family":"Lewis","given":"S. A."},{"family":"Macfarlane","given":"J. T."}],"issued":{"date-parts":[["2003",5,1]]}}},{"id":5229,"uris":[""],"uri":[""],"itemData":{"id":5229,"type":"article-journal","abstract":"BACKGROUND: There is considerable variability in rates of hospitalization of patients with community-acquired pneumonia, in part because of physicians' uncertainty in assessing the severity of illness at presentation.\nMETHODS: From our analysis of data on 14,199 adult inpatients with community-acquired pneumonia, we derived a prediction rule that stratifies patients into five classes with respect to the risk of death within 30 days. The rule was validated with 1991 data on 38,039 inpatients and with data on 2287 inpatients and outpatients in the Pneumonia Patient Outcomes Research Team (PORT) cohort study. The prediction rule assigns points based on age and the presence of coexisting disease, abnormal physical findings (such as a respiratory rate of > or = 30 or a temperature of > or = 40 degrees C), and abnormal laboratory findings (such as a pH <7.35, a blood urea nitrogen concentration > or = 30 mg per deciliter [11 mmol per liter] or a sodium concentration <130 mmol per liter) at presentation.\nRESULTS: There were no significant differences in mortality in each of the five risk classes among the three cohorts. Mortality ranged from 0.1 to 0.4 percent for class I patients (P=0.22), from 0.6 to 0.7 percent for class II (P=0.67), and from 0.9 to 2.8 percent for class III (P=0.12). Among the 1575 patients in the three lowest risk classes in the Pneumonia PORT cohort, there were only seven deaths, of which only four were pneumonia-related. The risk class was significantly associated with the risk of subsequent hospitalization among those treated as outpatients and with the use of intensive care and the number of days in the hospital among inpatients.\nCONCLUSIONS: The prediction rule we describe accurately identifies the patients with community-acquired pneumonia who are at low risk for death and other adverse outcomes. This prediction rule may help physicians make more rational decisions about hospitalization for patients with pneumonia.","container-title":"The New England Journal of Medicine","DOI":"10.1056/NEJM199701233360402","ISSN":"0028-4793","issue":"4","journalAbbreviation":"N. Engl. J. Med.","language":"eng","note":"PMID: 8995086","page":"243-250","source":"PubMed","title":"A prediction rule to identify low-risk patients with community-acquired pneumonia","volume":"336","author":[{"family":"Fine","given":"M. J."},{"family":"Auble","given":"T. E."},{"family":"Yealy","given":"D. M."},{"family":"Hanusa","given":"B. H."},{"family":"Weissfeld","given":"L. A."},{"family":"Singer","given":"D. E."},{"family":"Coley","given":"C. M."},{"family":"Marrie","given":"T. J."},{"family":"Kapoor","given":"W. N."}],"issued":{"date-parts":[["1997",1,23]]}}}],"schema":""} 16,17 Sensitivity analyses of missing values in potential candidate variables were performed using multiple imputation by chained equations, under the missing at random assumption. Ten sets, each with 10 iterations, were imputed using available explanatory variables for both cohorts (Derivation and Validation). The outcome variable was included as a predictor in the derivation but not validation dataset. Model derivation was explored in imputed datasets. All models developed in the complete case derivation dataset were tested in the imputed validation dataset, with Rubin’s rules ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"IabXYvWK","properties":{"formattedCitation":"\\super 18\\nosupersub{}","plainCitation":"18","noteIndex":0},"citationItems":[{"id":5311,"uris":[""],"uri":[""],"itemData":{"id":5311,"type":"book","collection-title":"Wiley Series in Probability and Statistics","event-place":"Hoboken, NJ, USA","ISBN":"978-0-470-31669-6","language":"en","note":"DOI: 10.1002/9780470316696","publisher":"John Wiley & Sons, Inc.","publisher-place":"Hoboken, NJ, USA","source":" (Crossref)","title":"Multiple Imputation for Nonresponse in Surveys","URL":"","editor":[{"family":"Rubin","given":"Donald B."}],"accessed":{"date-parts":[["2020",7,18]]},"issued":{"date-parts":[["1987",6,9]]}}}],"schema":""} 18 used to combine model parameter estimates. Model validationPatients entered subsequently into the ISARIC CCP-UK study after 20th May 2020 were included within a separate validation cohort (Figure 1). We determined discrimination, calibration, and performance across a range of clinically relevant metrics to avoid bias in the assessment of outcomes, patients were required to have at least four weeks follow-up on 29th June 2020, including those patients whose admission resulted in death, to avoid bias in assessment of parison with existing risk stratification scoresAll derived models in the derivation dataset were compared within the validation cohort with existing scores. Model performance was compared to existing risk stratification models using the AUROC statistic, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Existing risk stratification scores were identified through a systematic literature search of EMBASE, WHO Medicus, and Google Scholar databases. We used the search terms “pneumonia”, “sepsis”, “influenza”, “COVID-19”, “SARS-CoV-2”, “coronavirus” combined with “score”’ and “prognosis”. We applied no language or date restrictions. The last search was performed on 1st July 2020. Risk stratification tools were included whose variables were available within the database and had accessible methodology for calculation.Performance characteristics were calculated according to original publications, and score cut-offs for adverse outcomes were selected based on the most commonly used criteria identified during the literature search. Cut-offs were the score value for which the patient was considered at low- or high-risk of adverse outcome, as defined by study authors. Patients with one or more missing input variables were omitted for that particular score. A sensitivity analysis was also performed, with stratification of the validation cohort by geography. This geographical categorisation was selected based on well-described economic and health inequalities between the two regions. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"huIhaUEU","properties":{"formattedCitation":"\\super 19,20\\nosupersub{}","plainCitation":"19,20","noteIndex":0},"citationItems":[{"id":5294,"uris":[""],"uri":[""],"itemData":{"id":5294,"type":"article-journal","abstract":"Objective To compare all cause mortality between the north and south of England over four decades., Design Population wide comparative observational study of mortality., Setting Five northernmost and four southernmost English government office regions., Population All residents in each year from 1965 to 2008., Main outcome measures Death rate ratios of north over south England by age band and sex, and northern excess mortality (percentage of excess deaths in north compared with south, adjusted for age and sex and examined for annual trends, using Poisson regression)., Results During 1965 to 2008 the northern excess mortality remained substantial, at an average of 13.8% (95% confidence interval 13.7% to 13.9%). This geographical inequality was significantly larger for males than for females (14.9%, 14.7% to 15.0% v 12.7%, 12.6% to 12.9%, P<0.001). The inequality decreased significantly but temporarily for both sexes from the early 80s to the late 90s, followed by a steep significant increase from 2000 to 2008. Inequality varied with age, being higher for ages 0-9 years and 40-74 years and lower for ages 10-39 years and over 75 years. Time trends also varied with age. The strongest trend over time by age group was the increase among the 20-34 age group, from no significant northern excess mortality in 1965-95 to 22.2% (18.7% to 26.0%) in 1996-2008. Overall, the north experienced a fifth more premature (<75 years) deaths than the south, which was significant: a pattern that changed only by a slight increase between 1965 and 2008., Conclusion Inequalities in all cause mortality in the north-south divide were severe and persistent over the four decades from 1965 to 2008. Males were affected more than females, and the variation across age groups was substantial. The increase in this inequality from 2000 to 2008 was notable and occurred despite the public policy emphasis in England over this period on reducing inequalities in health.","container-title":"The BMJ","DOI":"10.1136/bmj.d508","ISSN":"0959-8138","journalAbbreviation":"BMJ","note":"PMID: 21325004\nPMCID: PMC3039695","source":"PubMed Central","title":"Trends in mortality from 1965 to 2008 across the English north-south divide: comparative observational study","title-short":"Trends in mortality from 1965 to 2008 across the English north-south divide","URL":"","volume":"342","author":[{"family":"Hacking","given":"John M"},{"family":"Muller","given":"Sara"},{"family":"Buchan","given":"Iain E"}],"accessed":{"date-parts":[["2020",7,8]]},"issued":{"date-parts":[["2011",2,15]]}}},{"id":5288,"uris":[""],"uri":[""],"itemData":{"id":5288,"type":"article-journal","abstract":"Background Social, economic and health disparities between northern and southern England have persisted despite Government policies to reduce them. We examine long-term trends in premature mortality in northern and southern England across age groups, and whether mortality patterns changed after the 2008–2009 Great Recession.\nMethods Population-wide longitudinal (1965–2015) study of mortality in England's five northernmost versus four southernmost Government Office Regions – halves of overall population. Main outcome measure: directly age-sex adjusted mortality rates; northern excess mortality (percentage excess northern vs southern deaths, age-sex adjusted).\nResults From 1965 to 2010, premature mortality (deaths per 10 000 aged <75 years) declined from 64 to 28 in southern versus 72 to 35 in northern England. From 2010 to 2015 the rate of decline in premature mortality plateaued in northern and southern England. For most age groups, northern excess mortality remained consistent from 1965 to 2015. For 25–34 and 35–44 age groups, however, northern excess mortality increased sharply between 1995 and 2015: from 2.2% (95% CI –3.2% to 7.6%) to 29.3% (95% CI 21.0% to 37.6%); and 3.3% (95% CI –1.0% to 7.6%) to 49.4% (95% CI 42.8% to 55.9%), respectively. This was due to northern mortality increasing (ages 25–34) or plateauing (ages 35–44) from the mid-1990s while southern mortality mainly declined.\nConclusions England's northern excess mortality has been consistent among those aged <25 and 45+ for the past five decades but risen alarmingly among those aged 25–44 since the mid-90s, long before the Great Recession. This profound and worsening structural inequality requires more equitable economic, social and health policies, including potential reactions to the England-wide loss of improvement in premature mortality.","container-title":"J Epidemiol Community Health","DOI":"10.1136/jech-2017-209195","ISSN":"0143-005X, 1470-2738","issue":"9","journalAbbreviation":"J Epidemiol Community Health","language":"en","note":"publisher: BMJ Publishing Group Ltd\nsection: Research report\nPMID: 28784630","page":"928-936","source":"jech.","title":"North-South disparities in English mortality1965–2015: longitudinal population study","title-short":"North-South disparities in English mortality1965–2015","volume":"71","author":[{"family":"Buchan","given":"Iain E."},{"family":"Kontopantelis","given":"Evangelos"},{"family":"Sperrin","given":"Matthew"},{"family":"Chandola","given":"Tarani"},{"family":"Doran","given":"Tim"}],"issued":{"date-parts":[["2017",9,1]]}}}],"schema":""} 19,20 Recent analysis has demonstrated the impact between deprivation and mortality risk with covid-19. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"GASw7UF7","properties":{"formattedCitation":"\\super 21\\nosupersub{}","plainCitation":"21","noteIndex":0},"citationItems":[{"id":5316,"uris":[""],"uri":[""],"itemData":{"id":5316,"type":"webpage","title":"Deaths involving COVID-19 by local area and socioeconomic deprivation - Office for National Statistics","URL":"","accessed":{"date-parts":[["2020",7,23]]}}}],"schema":""} 21 As a result, population differences between regions may change the discriminatory performance of risk stratification scores. Two geographical cohorts were created, based on north-south geographical locations across the United Kingdom as defined by Hacking et al. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"DkUITwwd","properties":{"formattedCitation":"\\super 19\\nosupersub{}","plainCitation":"19","noteIndex":0},"citationItems":[{"id":5294,"uris":[""],"uri":[""],"itemData":{"id":5294,"type":"article-journal","abstract":"Objective To compare all cause mortality between the north and south of England over four decades., Design Population wide comparative observational study of mortality., Setting Five northernmost and four southernmost English government office regions., Population All residents in each year from 1965 to 2008., Main outcome measures Death rate ratios of north over south England by age band and sex, and northern excess mortality (percentage of excess deaths in north compared with south, adjusted for age and sex and examined for annual trends, using Poisson regression)., Results During 1965 to 2008 the northern excess mortality remained substantial, at an average of 13.8% (95% confidence interval 13.7% to 13.9%). This geographical inequality was significantly larger for males than for females (14.9%, 14.7% to 15.0% v 12.7%, 12.6% to 12.9%, P<0.001). The inequality decreased significantly but temporarily for both sexes from the early 80s to the late 90s, followed by a steep significant increase from 2000 to 2008. Inequality varied with age, being higher for ages 0-9 years and 40-74 years and lower for ages 10-39 years and over 75 years. Time trends also varied with age. The strongest trend over time by age group was the increase among the 20-34 age group, from no significant northern excess mortality in 1965-95 to 22.2% (18.7% to 26.0%) in 1996-2008. Overall, the north experienced a fifth more premature (<75 years) deaths than the south, which was significant: a pattern that changed only by a slight increase between 1965 and 2008., Conclusion Inequalities in all cause mortality in the north-south divide were severe and persistent over the four decades from 1965 to 2008. Males were affected more than females, and the variation across age groups was substantial. The increase in this inequality from 2000 to 2008 was notable and occurred despite the public policy emphasis in England over this period on reducing inequalities in health.","container-title":"The BMJ","DOI":"10.1136/bmj.d508","ISSN":"0959-8138","journalAbbreviation":"BMJ","note":"PMID: 21325004\nPMCID: PMC3039695","source":"PubMed Central","title":"Trends in mortality from 1965 to 2008 across the English north-south divide: comparative observational study","title-short":"Trends in mortality from 1965 to 2008 across the English north-south divide","URL":"","volume":"342","author":[{"family":"Hacking","given":"John M"},{"family":"Muller","given":"Sara"},{"family":"Buchan","given":"Iain E"}],"accessed":{"date-parts":[["2020",7,8]]},"issued":{"date-parts":[["2011",2,15]]}}}],"schema":""} 19 All tests were two-tailed and p values <0.05 were considered statistically significant. We used R (version 3.6.3) with the finalfit, glmnet, pROC, recipes, xgboost, and tidyverse packages for all statistical analysis.Patient and public involvementThis was an urgent public health research study in response to a Public Health Emergency of International Concern. Patients or the public were not involved in the design, conduct, or reporting of this rapid response research.Role of the funding sourceThis work is supported by grants from: the National Institute for Health Research (NIHR; award CO-CIN-01), the Medical Research Council (MRC; grant MC_PC_19059), the NIHR Health Protection Research Unit in Emerging and Zoonotic Infections at University of Liverpool in partnership with Public Health England (PHE), in collaboration with Liverpool School of Tropical Medicine and the University of Oxford (NIHR award 200907), Wellcome Trust and Department for International Development (DID; 215091/Z/18/Z), and the Bill and Melinda Gates Foundation (OPP1209135), and Liverpool Experimental Cancer Medicine Centre for providing infrastructure support for this research (grant reference C18616/A25153). JSN-V-T is seconded to the Department of Health and Social Care, England (DHSC). The views expressed are those of the authors and not necessarily those of the DHSC, DID, NIHR, MRC, Wellcome Trust, or PHE. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Figure 1. Model derivation and validation workflow.ResultsIn the derivation cohort, we collected data from 34 692 patients between 6th February 2020 and 20th May 2020. The overall mortality was 32% (10 998 patients). The median age of patients in the cohort was 74 (interquartile range (IQR) 59-83) years. 58% (20184) were male and 75% (261350) patients had at least one comorbidity. Demographic and clinical characteristics for the derivation and validation datasets are shown in Table 1. Model developmentIn total, 40 candidate predictor variables measured at hospital admission were identified for model creation (Figure 1; Appendix 2). Following the creation of a composite variable containing all seven individual comorbidities and the exclusion of 13 variables due to high levels of missing values (Appendix 3), 20 variables remained.Generalised additive modelling (GAM) identified eight important prognostic factors as likely predictors of mortality; age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, Glasgow Coma Scale (GCS), urea, and C-reactive protein (CRP) (for variable selection process, see Appendix 4 and 5). Given the a priori need for a pragmatic score for use at the bedside, continuous variables were converted to factors with cut-points chosen using component smoothed functions (on linear predictor scale) from GAM model. The resulting variables were entered into a penalised logistic regression model (LASSO), which retained all variables within the final model (Appendix 6). Penalised regression coefficients were converted into a prognostic index using appropriate scaling (4C score range 0-21 points; Table 2).To provide a “best in class” comparison to this pragmatic score, a machine learning method was developed in parallel (Figure 1). The extreme gradient boosting decision tree approach (XGBoost) was used to train a classifier using the 20 candidate variables. The derivation cohort was randomly split 80/20% to provide a training and testing datasets. Models were trained by minimising the binary classification error and hyperparameters were tuned to maximise the AUROC. The 4C score demonstrated good discrimination for mortality within the derivation cohort (Table 3) with performance that approached that of the XGBoost model. The 4C score showed good calibration (Calibration slope = 1; Figure 2) across the range of risk and no adjustment to the model was required.Table 1. Demographic and clinical characteristics for derivation and validation cohorts for patients hospitalized with covid-19.Derivation cohort(n = 34692)Validation cohort(n = 22454)Mortality (%)10998 (31.7)6428 (28.6)Age on admission (years)<504397 (12.7)2825 (12.6)50-594603 (13.3)2630 (11.7)60-695563 (16.0)3155 (14.1)70-797986 (23.0)4971 (22.1)≥8012143 (35.0)8873 (39.5)Sex at BirthMale20184 (58.3)12202 (54.4)Female14411 (41.7)10211 (45.6)EthnicityWhite25680 (83.2)16837 (85.0)South Asian1423 (4.6)786 (4.0)East Asian269 (0.9)138 (0.7)Black1108 (3.6)765 (3.9)Other Ethnic Minority2387 (7.7)1275 (6.4)Chronic cardiac disease10192 (32.0)6853 (33.9)Chronic kidney disease5451 (17.3)3682 (18.4)Malignant neoplasm3201 (10.2)2144 (10.8)Moderate or severe liver disease581 (1.9)427 (2.2)Clinician-reported obesity3250 (11.3)2127 (11.9)Chronic pulmonary disease (not asthma)5684 (17.9)3671 (18.3)Diabetes8245 (26.3)4210 (22.0)Number of comorbidities08557 (24.7)5524 (24.6)19633 (27.8)6045 (26.9)≥216502 (47.6)10885 (48.5)Respiratory Rate22.0 (8.0)20.0 (8.0)Peripheral oxygen saturation (%)94.0 (6.0)94.0 (5.0)Systolic blood pressure (mmHg)125.0 (33.0)129.0 (33.0)Diastolic blood pressure (mmHg)70.0 (19.0)73.0 (20.0)Temperature (°C)37.3 (1.6)37.1 (1.5)Heart Rate (bpm)90.0 (27.0)90.0 (28.0)Glasgow Coma Score15.0 (0.0)15.0 (0.0)pH7.4 (0.1)7.4 (0.1)Bicarbonate (mmol/L)24.5 (4.5)24.4 (5.0)Infiltrates on chest radiograph13984 (62.9)8244 (61.1)Haemoglobin (g/L)130.0 (29.0)127.0 (31.0)White cell count (109/L)7.4 (5.0)7.6 (5.3)Neutrophil count (109/L)5.6 (4.6)5.8 (4.9)Lymphocyte count (109/L)0.9 (0.6)0.9 (0.7)Haematocrit (%)35.0 (40.6)25.2 (38.6)Platelet Count (109/L)215.0 (120.0)223.0 (126.0)Prothrombin (seconds)13.2 (3.0)13.2 (3.2)Activated partial thromboplastin time (APTT) (seconds)29.6 (8.6)29.2 (8.8)Sodium (mmol/L)137.0 (6.0)137.0 (6.0)Potassium (mmol/L)4.1 (0.8)4.1 (0.7)Total bilirubin (mg/dL)10.0 (7.0)10.0 (7.0)Alanine aminotransferase (ALT) (units/L)26.0 (27.0)25.0 (26.0)Aspartate aminotransferase (AST) (units/L)42.0 (41.0)48.0 (53.0)Lactate dehydrogenase (Units/L)432.0 (330.2)416.5 (311.8)Glucose (mmol/L)6.8 (3.1)6.8 (3.2)Urea (mmol/L)7.1 (6.4)7.3 (6.8)Creatinine (?mol/L)86.0 (53.0)86.0 (56.0)Lactate (mmol/L)1.5 (1.0)1.5 (1.1)C-reactive protein (CRP) (mg/dL)85.0 (121.0)78.0 (120.0)Values stated as median with IQR in parentheses for continuous variables, patient number with percentage in parentheses for categorical variables. Comorbidities were defined using the Charlson Comorbidity Index, with the addition of clinician-defined obesity. Information on missing data contained within Appendix 3.Table 2. Final scoring system for 4C score to predict in-hospital mortality following categorisation of continuous variables in LASSO regression model. Variable4C scoreAge (years)<5050-59+260-69+470-79+6≥80+7Sex at birthFemaleMale+1Number of comorbidities*01+1≥2+2Respiratory rate (breaths/minute)<2020-29+1≥30+2Peripheral oxygen saturation on room air (%)≥92<92+2Glasgow Coma Scale15<15+2Urea (mmol/L)≤77-14+1>14+3CRP (mg/dL)<5050-99+1≥100+2*Comorbidities were defined using the Charlson Comorbidity Index, with the addition of clinician-defined obesityTable 3. Model discrimination in derivation cohort.ModelAUROC95% CIBrier score4C score0.790.78 - 0.790.174Machine learning comparison (extreme gradient boosting [XGBoost])0.810.79 - 0.820.179AUROC, area under receiver operator curve; CI, confidence interval. Figure 2. Distribution (A), observed inpatient mortality across index score range (B), and calibration for 4C score within derivation cohort.-45720015938500Model validationThe validation cohort included data from 22 454 patients between 21st May 2020 and 29th June 2020 who had at least four weeks follow-up. The overall mortality was 28.6% (10 998 patients). The median age of patients in the cohort was 76 (interquartile range (IQR) 60-85) years. 12202 (54.4%) were male and 16930 patients (75.4%) had at least one comorbidity (Table 1). Missing data for predictor variables within the validation cohort are summarised in Appendix 7.Discrimination of the 4C score in the validation cohort was similar to that of the XGBoost model. Calibration was also found to be excellent in the validation cohort (Calibration slope = 1; Appendix 8), with a similar Brier score to the derivation cohort (0.174). The 4C score demonstrated good performance in clinically relevant metrics, across a range of cut-offs (Table 4).Four risk groups were defined with corresponding mortality rates determined: low risk (0-3 score; mortality rate 1%), intermediate risk (4-8 score; 10%), high risk (9-14 score; 35%), and very high risk (≥15 score; 67%)(Table 5). Performance metrics demonstrated a high sensitivity (99.8%) and negative predictive value (NPV; 99.0%) for the low-risk group, covering 6.9% of the cohort and a corresponding mortality rate of 1.0%. Patients in the intermediate risk group (score 4-8; n = 3007, 22.7%) had a mortality rate of 9.8% (NPV 90.2%). High-risk patients (score 9-14; n = 7009, 52.9%) had a 35.2% mortality (NPV 64.8), while patients scoring ≥15 (n = 2310, 17.4%) had a 66.8% mortality (positive predictive value 66.8).Comparison with existing toolsA total of 26 risk stratification scores were identified through a systematic literature search, of which 15 were suitable for validation the two cohorts (Appendix 9). The 4C model compared well against existing risk stratification scores in predicting inpatient mortality (Table 6). Risk stratification scores originally validated in patients with community-acquired pneumonia (n = 9) generally had higher discrimination for inpatient mortality in the validation cohort (e.g., A-DROP [AUROC 0.74; 0.73-0.75], E-CURB65 [0.76; 0.74-0.79]) than those developed within covid-19 cohorts (n = 4: Surgisphere [0.63; 0.62-0.64], DL score [0.67; 0.66-0.68], COVID-GRAM [0.71; 0.68-0.74] and Xie score [0.73; 0.71-0.76]). Performance metrics for the 4C score compared well against existing risk stratification scores at specified cut-offs (Appendix 10).The number of patients in whom risk stratification scores could be applied differed due to certain variables not being available, either due of missingness or because they were never tested for in clinical practice. Seven scores could be applied to fewer than 2000 patients (<10%) in the validation cohort, due to the requirement for biomarkers or physiological parameters that were not routinely captured in the cohort (e.g. lactate dehydrogenase [LDH]).Table 4. Performance metrics of derived 4C risk stratification score to rule-out mortality (A) and rule-in mortality (B) at different cut-offs in validation cohort.ANumber of patients at cut-off (%)TPTNFPFNSensitivitySpecificityPPVNPVMortality(%)≤2512 (3.9)431051084222100.05.733.999.60.4≤3 918 (6.9)43039098023999.810.234.999.01.0≤4 1353 (10.2)4282132376093099.314.836.097.82.2≤62417 (18.2)41962301663111697.325.838.895.24.8≤8 3925 (29.6)40073620531230592.940.543.092.27.8BNumber of patients at cut-off (%)TPTNFPFNSensitivitySpecificityPPVNPVMortality (%)≥99319 (70.4)40073620531230592.940.543.092.243.0≥117109 (53.7)34895312362082380.959.549.186.649.1≥134495 (33.9)257170081924174159.678.557.280.157.2≥152310 (17.4)15428164768277035.891.466.874.766.8TP, true positive; TN, true negative; FP, false positive; FP, false positive; PPV, positive predictive value; NPV, negative predictive value. Table 5. Comparison of mortality rates for 4C score risk groups across derivation and validation cohortsDerivation cohort Validation cohortRisk groupNumber of patients (%)Mortality (%)Number of patients (%)Mortality (%)Low (0-3)1275 (6.5)1.5918 (6.9)1.0Intermediate (4-8)4642 (23.8)9.93007 (22.7)9.8High (9-14)10430 (53.4)38.17009 (52.9)35.2Very high (≥15)3175 (16.3)69.82310 (17.4)66.8Overall1952234.21324432.6Table 6. Discriminatory performance of risk stratification scores within validation cohort to predict inpatient mortality in patients hospitalised with covid-19. See appendix 10 for other metrics. Validation cohort (N = 22 454)Number of patients with required parametersAUROC (95% CI)SOFA1900.60 (0.50 - 0.69)qSOFA177160.62 (0.62 - 0.63)SMARTCOP4760.63 (0.58 - 0.68)Surgisphere*173590.63 (0.62 - 0.64)NEWS2174550.66 (0.65 - 0.67)SCAP3580.66 (0.60 - 0.71)DL score*151420.67 (0.66 - 0.68)CRB65177160.69 (0.68 - 0.69)COVID-GRAM*11520.71 (0.68 - 0.74)DS-CRB65171270.72 (0.71 - 0.73)CURB65143180.72 (0.72 - 0.73)PSI3580.73 (0.67 - 0.78)Xie score*16270.73 (0.71 - 0.76)A-DROP143380.74 (0.73 - 0.75)E-CURB6514380.76 (0.74 - 0.79)4C score*132440.78 (0.77 - 0.79)Machine learning comparison (XGBoost)-0.79 (0.78 - 0.80) *novel covid-19 risk stratification scoreSensitivity analysisMultiple imputation was performed in both the derivation and validation cohorts (predictor variables n = 40). Prognostic models derived using from the multiply imputed derivation cohort had poorer performance than models derived using complete case data. This was true whether performance was assessed in the complete case or multiply imputed validation datasets. As a sensitivity analysis, the final model was assessed in the multiply imputed datasets. Discriminatory performance demonstrated a small reduction (≤0.02 change in AUROC) (Appendix 11) across both derivation and validation cohorts. A further sensitivity analysis was performed stratifying the validation cohort into two geographical cohorts (Validation North and South; Appendix 12). Discrimination remained similar for the 4C score in both the North (0.78, 0.77-0.79) and South (0.77, 0.76-0.79) subsets (Appendix 13).DiscussionWe developed and validated an eight-variable clinical prognostic (4C) score to predict in-hospital mortality in a UK prospective cohort of 57 146 patients hospitalised with covid-19. The 4C score uses patient demographics, clinical observations, and blood parameters that are commonly available at the time of hospital admission and can accurately predict patients at a high risk of in-hospital death. It compared favourably compared to other models, including ‘best-in-class’ machine learning techniques and demonstrated consistent performance across the validation cohorts.The 4C score contains parameters reflecting patient demographics, comorbidity, and physiology/inflammation on hospital admission. It shares many characteristics with existing prognostic scores for sepsis and community-acquired pneumonia, as well as for scores developed in covid-19 patients. Altered consciousness and high respiratory rate are included in most risk stratification scores for sepsis and community-acquired pneumonia (Appendix 9). Elevated urea is a component of pneumonia prognostic scores (e.g. CURB-65, A-DROP and PSI). Increasing age is a strong predictor of inpatient mortality within our hospitalised covid-19 cohort, reflected in the number of points attributed to it in the 4C score. Age is also included in other existing covid-19 scores, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"tBIhMJHR","properties":{"formattedCitation":"\\super 22\\uc0\\u8211{}24\\nosupersub{}","plainCitation":"22–24","noteIndex":0},"citationItems":[{"id":5231,"uris":[""],"uri":[""],"itemData":{"id":5231,"type":"article-journal","abstract":"<p>Since late December 2019 a new epidemic outbreak has emerged from Whuhan, China. Rapidly the new coronavirus has spread worldwide. China CDC has reported results of a descriptive exploratory analysis of all cases diagnosed until the 11th February 2020, presenting the epidemiologic curves and geo-temporal spread of COVID-19 along with case fatality rate according to some baseline characteristics, such as age, gender and several well-established high prevalence comorbidities. Despite this, we intend to increase even further the predictive value of that manuscript by presenting the odds ratio for mortality due to COVID-19 adjusted for the presence of those comorbidities and baseline characteristics such as age and gender. Besides, we present a way to determine the risk of each particular patient, given his characteristics. We found that age is the variable that presents higher risk of COVID-19 mortality, where 60 or older patients have an OR = 18.8161 (CI95%[7.1997; 41.5517]). Regarding comorbidities, cardiovascular disease appears to be the riskiest (OR=12.8328 CI95%[10.2736; 15.8643], along with chronic respiratory disease (OR=7.7925 CI95%[5.5446; 10.4319]). Males are more likely to die from COVID-19 (OR=1.8518 (CI95%[1.5996; 2.1270]). Some limitations such as the lack of information about the correct prevalence of gender per age or about comorbidities per age and gender or the assumption of independence between risk factors are expected to have a small impact on results. A final point of paramount importance is that the equation presented here can be used to determine the probability of dying from COVID-19 for a particular patient, given its age interval, gender and comorbidities associated.</p>","container-title":"medRxiv","DOI":"10.1101/2020.02.24.20027268","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.02.24.20027268","source":"","title":"Estimation of risk factors for COVID-19 mortality - preliminary results","author":[{"family":"Caramelo","given":"Francisco"},{"family":"Ferreira","given":"Nuno"},{"family":"Oliveiros","given":"Barbara"}],"issued":{"date-parts":[["2020",2,25]]}}},{"id":5234,"uris":[""],"uri":[""],"itemData":{"id":5234,"type":"article-journal","abstract":"<p>Abstract Importance COVID-19 is causing high mortality worldwide. Developing models to quantify the risk of poor outcomes in infected patients could help develop strategies to shield the most vulnerable during de-confinement. Objective To develop and externally validate COVID-19 Estimated Risk (COVER) scores that quantify a patient9s risk of hospital admission (COVER-H), requiring intensive services (COVER-I), or fatality (COVER-F) in the 30-days following COVID-19 diagnosis. Design Multinational, distributed network cohorts. Setting We analyzed a federated network of electronic medical records and administrative claims data from 13 data sources and 6 countries, mapped to a common data model. Participants Model development used a patient population consisting of &gt;2 million patients with a general practice (GP), emergency room (ER), or outpatient (OP) visit with diagnosed influenza or flu-like symptoms any time prior to 2020. The model was validated on patients with a GP, ER, or OP visit in 2020 with a confirmed or suspected COVID-19 diagnosis across four databases from South Korea, Spain and the United States. Outcomes Age, sex, historical conditions, and drug use prior to index date were considered as candidate predictors. Outcomes included i) hospitalization with pneumonia, ii) hospitalization with pneumonia requiring intensive services or death, and iii) death in the 30 days after index date. Results Overall, 43,061 COVID-19 patients were included for model validation, after initial model development and validation using 6,869,127 patients with influenza or flu-like symptoms. We identified 7 predictors (history of cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, hyperlipidemia, and kidney disease) which combined with age and sex could discriminate which patients would experience any of our three outcomes. The models achieved high performance in influenza. When transported to COVID-19 cohorts, the AUC ranges were, COVER-H: 0.73-0.81, COVER-I: 0.73-0.91, and COVER-F: 0.82-0.90. Calibration was overall acceptable, with overestimated risk in the most elderly and highest risk strata. Conclusions and relevance A 9-predictor model performs well for COVID-19 patients for predicting hospitalization, intensive services and death. The models could aid in providing reassurance for low risk patients and shield high risk patients from COVID-19 during de-confinement to reduce the virus9 impact on morbidity and mortality.</p>","container-title":"medRxiv","DOI":"10.1101/2020.05.26.20112649","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.05.26.20112649","source":"","title":"Seek COVER: Development and validation of a personalized risk calculator for COVID-19 outcomes in an international network","title-short":"Seek COVER","author":[{"family":"Williams","given":"Ross D."},{"family":"Markus","given":"Aniek F."},{"family":"Yang","given":"Cynthia"},{"family":"Salles","given":"Talita Duarte"},{"family":"Falconer","given":"Thomas"},{"family":"Jonnagaddala","given":"Jitendra"},{"family":"Kim","given":"Chungsoo"},{"family":"Rho","given":"Yeunsook"},{"family":"Williams","given":"Andrew"},{"family":"An","given":"Min Ho"},{"family":"Aragón","given":"María"},{"family":"Areia","given":"Carlos"},{"family":"Burn","given":"Edward"},{"family":"Choi","given":"Young"},{"family":"Drakos","given":"Iannis"},{"family":"Abrah?o","given":"Maria Fernandes"},{"family":"Fernández-Bertolín","given":"Sergio"},{"family":"Hripcsak","given":"George"},{"family":"Kaas-Hansen","given":"Benjamin"},{"family":"Kandukuri","given":"Prasanna"},{"family":"Kors","given":"Jan A."},{"family":"Kostka","given":"Kristin"},{"family":"Liaw","given":"Siaw-Teng"},{"family":"Machnicki","given":"Gerardo"},{"family":"Morales","given":"Daniel"},{"family":"Nyberg","given":"Fredrik"},{"family":"Park","given":"Rae Woong"},{"family":"Prats-Uribe","given":"Albert"},{"family":"Pratt","given":"Nicole"},{"family":"Rao","given":"Gowtham"},{"family":"Reich","given":"Christian G."},{"family":"Rivera","given":"Marcela"},{"family":"Seinen","given":"Tom"},{"family":"Shoaibi","given":"Azza"},{"family":"Spotnitz","given":"Matthew E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Suchard","given":"Marc A."},{"family":"You","given":"Seng Chan"},{"family":"Zhang","given":"Lin"},{"family":"Zhou","given":"Lili"},{"family":"Ryan","given":"Patrick B."},{"family":"Prieto-Alhambra","given":"Daniel"},{"family":"Reps","given":"Jenna M."},{"family":"Rijnbeek","given":"Peter R."}],"issued":{"date-parts":[["2020",5,29]]}}},{"id":5237,"uris":[""],"uri":[""],"itemData":{"id":5237,"type":"article-journal","abstract":"<h3>Importance</h3><p>Early identification of patients with novel corona virus disease 2019 (COVID-19) who may develop critical illness is of great importance and may aid in delivering proper treatment and optimizing use of resources.</p><h3>Objective</h3><p>To develop and validate a clinical score at hospital admission for predicting which patients with COVID-19 will develop critical illness based on a nationwide cohort in China.</p><h3>Design, Setting, and Participants</h3><p>Collaborating with the National Health Commission of China, we established a retrospective cohort of patients with COVID-19 from 575 hospitals in 31 provincial administrative regions as of January 31, 2020. Epidemiological, clinical, laboratory, and imaging variables ascertained at hospital admission were screened using Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression to construct a predictive risk score (COVID-GRAM). The score provides an estimate of the risk that a hospitalized patient with COVID-19 will develop critical illness. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC). Data from 4 additional cohorts in China hospitalized with COVID-19 were used to validate the score. Data were analyzed between February 20, 2020 and March 17, 2020.</p><h3>Main Outcomes and Measures</h3><p>Among patients with COVID-19 admitted to the hospital, critical illness was defined as the composite measure of admission to the intensive care unit, invasive ventilation, or death.</p><h3>Results</h3><p>The development cohort included 1590 patients. the mean (SD) age of patients in the cohort was 48.9 (15.7) years; 904 (57.3%) were men. The validation cohort included 710 patients with a mean (SD) age of 48.2 (15.2) years, and 382 (53.8%) were men and 172 (24.2%). From 72 potential predictors, 10 variables were independent predictive factors and were included in the risk score: chest radiographic abnormality (OR, 3.39; 95% CI, 2.14-5.38), age (OR, 1.03; 95% CI, 1.01-1.05), hemoptysis (OR, 4.53; 95% CI, 1.36-15.15), dyspnea (OR, 1.88; 95% CI, 1.18-3.01), unconsciousness (OR, 4.71; 95% CI, 1.39-15.98), number of comorbidities (OR, 1.60; 95% CI, 1.27-2.00), cancer history (OR, 4.07; 95% CI, 1.23-13.43), neutrophil-to-lymphocyte ratio (OR, 1.06; 95% CI, 1.02-1.10), lactate dehydrogenase (OR, 1.002; 95% CI, 1.001-1.004) and direct bilirubin (OR, 1.15; 95% CI, 1.06-1.24). The mean AUC in the development cohort was 0.88 (95% CI, 0.85-0.91) and the AUC in the validation cohort was 0.88 (95% CI, 0.84-0.93). The score has been translated into an online risk calculator that is freely available to the public ()</p><h3>Conclusions and Relevance</h3><p>In this study, a risk score based on characteristics of COVID-19 patients at the time of admission to the hospital was developed that may help predict a patient’s risk of developing critical illness.</p>","container-title":"JAMA Internal Medicine","DOI":"10.1001/jamainternmed.2020.2033","journalAbbreviation":"JAMA Intern Med","language":"en","source":"","title":"Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19","URL":"","author":[{"family":"Liang","given":"Wenhua"},{"family":"Liang","given":"Hengrui"},{"family":"Ou","given":"Limin"},{"family":"Chen","given":"Binfeng"},{"family":"Chen","given":"Ailan"},{"family":"Li","given":"Caichen"},{"family":"Li","given":"Yimin"},{"family":"Guan","given":"Weijie"},{"family":"Sang","given":"Ling"},{"family":"Lu","given":"Jiatao"},{"family":"Xu","given":"Yuanda"},{"family":"Chen","given":"Guoqiang"},{"family":"Guo","given":"Haiyan"},{"family":"Guo","given":"Jun"},{"family":"Chen","given":"Zisheng"},{"family":"Zhao","given":"Yi"},{"family":"Li","given":"Shiyue"},{"family":"Zhang","given":"Nuofu"},{"family":"Zhong","given":"Nanshan"},{"family":"He","given":"Jianxing"}],"accessed":{"date-parts":[["2020",6,1]]},"issued":{"date-parts":[["2020",5,12]]}}}],"schema":""} 22–24 together with comorbidity count ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"4U3FN30V","properties":{"formattedCitation":"\\super 22\\uc0\\u8211{}24\\nosupersub{}","plainCitation":"22–24","noteIndex":0},"citationItems":[{"id":5231,"uris":[""],"uri":[""],"itemData":{"id":5231,"type":"article-journal","abstract":"<p>Since late December 2019 a new epidemic outbreak has emerged from Whuhan, China. Rapidly the new coronavirus has spread worldwide. China CDC has reported results of a descriptive exploratory analysis of all cases diagnosed until the 11th February 2020, presenting the epidemiologic curves and geo-temporal spread of COVID-19 along with case fatality rate according to some baseline characteristics, such as age, gender and several well-established high prevalence comorbidities. Despite this, we intend to increase even further the predictive value of that manuscript by presenting the odds ratio for mortality due to COVID-19 adjusted for the presence of those comorbidities and baseline characteristics such as age and gender. Besides, we present a way to determine the risk of each particular patient, given his characteristics. We found that age is the variable that presents higher risk of COVID-19 mortality, where 60 or older patients have an OR = 18.8161 (CI95%[7.1997; 41.5517]). Regarding comorbidities, cardiovascular disease appears to be the riskiest (OR=12.8328 CI95%[10.2736; 15.8643], along with chronic respiratory disease (OR=7.7925 CI95%[5.5446; 10.4319]). Males are more likely to die from COVID-19 (OR=1.8518 (CI95%[1.5996; 2.1270]). Some limitations such as the lack of information about the correct prevalence of gender per age or about comorbidities per age and gender or the assumption of independence between risk factors are expected to have a small impact on results. A final point of paramount importance is that the equation presented here can be used to determine the probability of dying from COVID-19 for a particular patient, given its age interval, gender and comorbidities associated.</p>","container-title":"medRxiv","DOI":"10.1101/2020.02.24.20027268","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.02.24.20027268","source":"","title":"Estimation of risk factors for COVID-19 mortality - preliminary results","author":[{"family":"Caramelo","given":"Francisco"},{"family":"Ferreira","given":"Nuno"},{"family":"Oliveiros","given":"Barbara"}],"issued":{"date-parts":[["2020",2,25]]}}},{"id":5234,"uris":[""],"uri":[""],"itemData":{"id":5234,"type":"article-journal","abstract":"<p>Abstract Importance COVID-19 is causing high mortality worldwide. Developing models to quantify the risk of poor outcomes in infected patients could help develop strategies to shield the most vulnerable during de-confinement. Objective To develop and externally validate COVID-19 Estimated Risk (COVER) scores that quantify a patient9s risk of hospital admission (COVER-H), requiring intensive services (COVER-I), or fatality (COVER-F) in the 30-days following COVID-19 diagnosis. Design Multinational, distributed network cohorts. Setting We analyzed a federated network of electronic medical records and administrative claims data from 13 data sources and 6 countries, mapped to a common data model. Participants Model development used a patient population consisting of &gt;2 million patients with a general practice (GP), emergency room (ER), or outpatient (OP) visit with diagnosed influenza or flu-like symptoms any time prior to 2020. The model was validated on patients with a GP, ER, or OP visit in 2020 with a confirmed or suspected COVID-19 diagnosis across four databases from South Korea, Spain and the United States. Outcomes Age, sex, historical conditions, and drug use prior to index date were considered as candidate predictors. Outcomes included i) hospitalization with pneumonia, ii) hospitalization with pneumonia requiring intensive services or death, and iii) death in the 30 days after index date. Results Overall, 43,061 COVID-19 patients were included for model validation, after initial model development and validation using 6,869,127 patients with influenza or flu-like symptoms. We identified 7 predictors (history of cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, hyperlipidemia, and kidney disease) which combined with age and sex could discriminate which patients would experience any of our three outcomes. The models achieved high performance in influenza. When transported to COVID-19 cohorts, the AUC ranges were, COVER-H: 0.73-0.81, COVER-I: 0.73-0.91, and COVER-F: 0.82-0.90. Calibration was overall acceptable, with overestimated risk in the most elderly and highest risk strata. Conclusions and relevance A 9-predictor model performs well for COVID-19 patients for predicting hospitalization, intensive services and death. The models could aid in providing reassurance for low risk patients and shield high risk patients from COVID-19 during de-confinement to reduce the virus9 impact on morbidity and mortality.</p>","container-title":"medRxiv","DOI":"10.1101/2020.05.26.20112649","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.05.26.20112649","source":"","title":"Seek COVER: Development and validation of a personalized risk calculator for COVID-19 outcomes in an international network","title-short":"Seek COVER","author":[{"family":"Williams","given":"Ross D."},{"family":"Markus","given":"Aniek F."},{"family":"Yang","given":"Cynthia"},{"family":"Salles","given":"Talita Duarte"},{"family":"Falconer","given":"Thomas"},{"family":"Jonnagaddala","given":"Jitendra"},{"family":"Kim","given":"Chungsoo"},{"family":"Rho","given":"Yeunsook"},{"family":"Williams","given":"Andrew"},{"family":"An","given":"Min Ho"},{"family":"Aragón","given":"María"},{"family":"Areia","given":"Carlos"},{"family":"Burn","given":"Edward"},{"family":"Choi","given":"Young"},{"family":"Drakos","given":"Iannis"},{"family":"Abrah?o","given":"Maria Fernandes"},{"family":"Fernández-Bertolín","given":"Sergio"},{"family":"Hripcsak","given":"George"},{"family":"Kaas-Hansen","given":"Benjamin"},{"family":"Kandukuri","given":"Prasanna"},{"family":"Kors","given":"Jan A."},{"family":"Kostka","given":"Kristin"},{"family":"Liaw","given":"Siaw-Teng"},{"family":"Machnicki","given":"Gerardo"},{"family":"Morales","given":"Daniel"},{"family":"Nyberg","given":"Fredrik"},{"family":"Park","given":"Rae Woong"},{"family":"Prats-Uribe","given":"Albert"},{"family":"Pratt","given":"Nicole"},{"family":"Rao","given":"Gowtham"},{"family":"Reich","given":"Christian G."},{"family":"Rivera","given":"Marcela"},{"family":"Seinen","given":"Tom"},{"family":"Shoaibi","given":"Azza"},{"family":"Spotnitz","given":"Matthew E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Suchard","given":"Marc A."},{"family":"You","given":"Seng Chan"},{"family":"Zhang","given":"Lin"},{"family":"Zhou","given":"Lili"},{"family":"Ryan","given":"Patrick B."},{"family":"Prieto-Alhambra","given":"Daniel"},{"family":"Reps","given":"Jenna M."},{"family":"Rijnbeek","given":"Peter R."}],"issued":{"date-parts":[["2020",5,29]]}}},{"id":5237,"uris":[""],"uri":[""],"itemData":{"id":5237,"type":"article-journal","abstract":"<h3>Importance</h3><p>Early identification of patients with novel corona virus disease 2019 (COVID-19) who may develop critical illness is of great importance and may aid in delivering proper treatment and optimizing use of resources.</p><h3>Objective</h3><p>To develop and validate a clinical score at hospital admission for predicting which patients with COVID-19 will develop critical illness based on a nationwide cohort in China.</p><h3>Design, Setting, and Participants</h3><p>Collaborating with the National Health Commission of China, we established a retrospective cohort of patients with COVID-19 from 575 hospitals in 31 provincial administrative regions as of January 31, 2020. Epidemiological, clinical, laboratory, and imaging variables ascertained at hospital admission were screened using Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression to construct a predictive risk score (COVID-GRAM). The score provides an estimate of the risk that a hospitalized patient with COVID-19 will develop critical illness. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC). Data from 4 additional cohorts in China hospitalized with COVID-19 were used to validate the score. Data were analyzed between February 20, 2020 and March 17, 2020.</p><h3>Main Outcomes and Measures</h3><p>Among patients with COVID-19 admitted to the hospital, critical illness was defined as the composite measure of admission to the intensive care unit, invasive ventilation, or death.</p><h3>Results</h3><p>The development cohort included 1590 patients. the mean (SD) age of patients in the cohort was 48.9 (15.7) years; 904 (57.3%) were men. The validation cohort included 710 patients with a mean (SD) age of 48.2 (15.2) years, and 382 (53.8%) were men and 172 (24.2%). From 72 potential predictors, 10 variables were independent predictive factors and were included in the risk score: chest radiographic abnormality (OR, 3.39; 95% CI, 2.14-5.38), age (OR, 1.03; 95% CI, 1.01-1.05), hemoptysis (OR, 4.53; 95% CI, 1.36-15.15), dyspnea (OR, 1.88; 95% CI, 1.18-3.01), unconsciousness (OR, 4.71; 95% CI, 1.39-15.98), number of comorbidities (OR, 1.60; 95% CI, 1.27-2.00), cancer history (OR, 4.07; 95% CI, 1.23-13.43), neutrophil-to-lymphocyte ratio (OR, 1.06; 95% CI, 1.02-1.10), lactate dehydrogenase (OR, 1.002; 95% CI, 1.001-1.004) and direct bilirubin (OR, 1.15; 95% CI, 1.06-1.24). The mean AUC in the development cohort was 0.88 (95% CI, 0.85-0.91) and the AUC in the validation cohort was 0.88 (95% CI, 0.84-0.93). The score has been translated into an online risk calculator that is freely available to the public ()</p><h3>Conclusions and Relevance</h3><p>In this study, a risk score based on characteristics of COVID-19 patients at the time of admission to the hospital was developed that may help predict a patient’s risk of developing critical illness.</p>","container-title":"JAMA Internal Medicine","DOI":"10.1001/jamainternmed.2020.2033","journalAbbreviation":"JAMA Intern Med","language":"en","source":"","title":"Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19","URL":"","author":[{"family":"Liang","given":"Wenhua"},{"family":"Liang","given":"Hengrui"},{"family":"Ou","given":"Limin"},{"family":"Chen","given":"Binfeng"},{"family":"Chen","given":"Ailan"},{"family":"Li","given":"Caichen"},{"family":"Li","given":"Yimin"},{"family":"Guan","given":"Weijie"},{"family":"Sang","given":"Ling"},{"family":"Lu","given":"Jiatao"},{"family":"Xu","given":"Yuanda"},{"family":"Chen","given":"Guoqiang"},{"family":"Guo","given":"Haiyan"},{"family":"Guo","given":"Jun"},{"family":"Chen","given":"Zisheng"},{"family":"Zhao","given":"Yi"},{"family":"Li","given":"Shiyue"},{"family":"Zhang","given":"Nuofu"},{"family":"Zhong","given":"Nanshan"},{"family":"He","given":"Jianxing"}],"accessed":{"date-parts":[["2020",6,1]]},"issued":{"date-parts":[["2020",5,12]]}}}],"schema":""} 22–24 and elevated CRP. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"fwfRi00S","properties":{"formattedCitation":"\\super 25,26\\nosupersub{}","plainCitation":"25,26","noteIndex":0},"citationItems":[{"id":5240,"uris":[""],"uri":[""],"itemData":{"id":5240,"type":"article-journal","abstract":"Tweetable abstract @ERSpublications\nclick to tweetAge significantly determined the clinical features and prognosis of the disease. The prognosis was worse in patients older than 60 years, calling for clinicians to pay more attention to patients on this special age.","container-title":"European Respiratory Journal","DOI":"10.1183/13993003.01112-2020","ISSN":"0903-1936, 1399-3003","language":"en","note":"publisher: European Respiratory Society\nsection: Research letter\nPMID: 32312864","source":"erj.","title":"Association Between Ages and Clinical Characteristics and Outcomes of Coronavirus Disease 2019","URL":"","author":[{"family":"Liu","given":"Yang"},{"family":"Mao","given":"Bei"},{"family":"Liang","given":"Shuo"},{"family":"Yang","given":"Jia-wei"},{"family":"Lu","given":"Hai-wen"},{"family":"Chai","given":"Yan-hua"},{"family":"Wang","given":"Lan"},{"family":"Zhang","given":"Li"},{"family":"Li","given":"Qiu-hong"},{"family":"Zhao","given":"Lan"},{"family":"He","given":"Yan"},{"family":"Gu","given":"Xiao-long"},{"family":"Ji","given":"Xiao-bin"},{"family":"Li","given":"Li"},{"family":"Jie","given":"Zhi-jun"},{"family":"Li","given":"Qiang"},{"family":"Li","given":"Xiang-yang"},{"family":"Lu","given":"Hong-zhou"},{"family":"Zhang","given":"Wen-hong"},{"family":"Song","given":"Yuan-lin"},{"family":"Qu","given":"Jie-ming"},{"family":"Xu","given":"Jin-fu"}],"accessed":{"date-parts":[["2020",6,1]]},"issued":{"date-parts":[["2020",1,1]]}}},{"id":5244,"uris":[""],"uri":[""],"itemData":{"id":5244,"type":"article-journal","abstract":"AbstractBackground. Elevated serum C-reactive protein (CRP) level was observed in most patients with COVID-19.Methods. Data of COVID-19 patients with clinical","container-title":"Clinical Infectious Diseases","DOI":"10.1093/cid/ciaa641","journalAbbreviation":"Clin Infect Dis","language":"en","source":"academic.","title":"Prognostic value of C-reactive protein in patients with COVID-19","URL":"","author":[{"family":"Luo","given":"Xiaomin"},{"family":"Zhou","given":"Wei"},{"family":"Yan","given":"Xiaojie"},{"family":"Guo","given":"Tangxi"},{"family":"Wang","given":"Benchao"},{"family":"Xia","given":"Hongxia"},{"family":"Ye","given":"Lu"},{"family":"Xiong","given":"Jun"},{"family":"Jiang","given":"Zongping"},{"family":"Liu","given":"Yu"},{"family":"Zhang","given":"Bicheng"},{"family":"Yang","given":"Weize"}],"accessed":{"date-parts":[["2020",6,1]]}}}],"schema":""} 25,26 Model performance compared well against other generated models, with minimal loss in discrimination despite its pragmatic nature. Machine learning approaches demonstrated improved performance, however the loss of interpretability and the requirement for complex equations for prediction limits use at the bedside. The 4C score demonstrated good applicability within the validation cohort (around 60% population) and consistency across all performance measures.We also evaluated the predictive performance of 11 existing risk stratification scores, primarily developed to predict inpatient death in sepsis and community-acquired pneumonia ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"JIMSbDX8","properties":{"formattedCitation":"\\super 17,27\\uc0\\u8211{}36\\nosupersub{}","plainCitation":"17,27–36","noteIndex":0},"citationItems":[{"id":5229,"uris":[""],"uri":[""],"itemData":{"id":5229,"type":"article-journal","abstract":"BACKGROUND: There is considerable variability in rates of hospitalization of patients with community-acquired pneumonia, in part because of physicians' uncertainty in assessing the severity of illness at presentation.\nMETHODS: From our analysis of data on 14,199 adult inpatients with community-acquired pneumonia, we derived a prediction rule that stratifies patients into five classes with respect to the risk of death within 30 days. The rule was validated with 1991 data on 38,039 inpatients and with data on 2287 inpatients and outpatients in the Pneumonia Patient Outcomes Research Team (PORT) cohort study. The prediction rule assigns points based on age and the presence of coexisting disease, abnormal physical findings (such as a respiratory rate of > or = 30 or a temperature of > or = 40 degrees C), and abnormal laboratory findings (such as a pH <7.35, a blood urea nitrogen concentration > or = 30 mg per deciliter [11 mmol per liter] or a sodium concentration <130 mmol per liter) at presentation.\nRESULTS: There were no significant differences in mortality in each of the five risk classes among the three cohorts. Mortality ranged from 0.1 to 0.4 percent for class I patients (P=0.22), from 0.6 to 0.7 percent for class II (P=0.67), and from 0.9 to 2.8 percent for class III (P=0.12). Among the 1575 patients in the three lowest risk classes in the Pneumonia PORT cohort, there were only seven deaths, of which only four were pneumonia-related. The risk class was significantly associated with the risk of subsequent hospitalization among those treated as outpatients and with the use of intensive care and the number of days in the hospital among inpatients.\nCONCLUSIONS: The prediction rule we describe accurately identifies the patients with community-acquired pneumonia who are at low risk for death and other adverse outcomes. This prediction rule may help physicians make more rational decisions about hospitalization for patients with pneumonia.","container-title":"The New England Journal of Medicine","DOI":"10.1056/NEJM199701233360402","ISSN":"0028-4793","issue":"4","journalAbbreviation":"N. Engl. J. Med.","language":"eng","note":"PMID: 8995086","page":"243-250","source":"PubMed","title":"A prediction rule to identify low-risk patients with community-acquired pneumonia","volume":"336","author":[{"family":"Fine","given":"M. J."},{"family":"Auble","given":"T. E."},{"family":"Yealy","given":"D. M."},{"family":"Hanusa","given":"B. H."},{"family":"Weissfeld","given":"L. A."},{"family":"Singer","given":"D. E."},{"family":"Coley","given":"C. M."},{"family":"Marrie","given":"T. J."},{"family":"Kapoor","given":"W. N."}],"issued":{"date-parts":[["1997",1,23]]}}},{"id":5320,"uris":[""],"uri":[""],"itemData":{"id":5320,"type":"article-journal","abstract":"Community-acquired pneumonia (CAP) continues to be a major medical problem. Since CAP is a potentially fatal disease, early appropriate antibiotic treatment is vital. Epidemiologic studies have shown that in the combined cause-of-death category, pneumonia ranks fourth as the leading cause of death in Japan. Therefore, the Japanese Respiratory Society (JRS) provided guidelines for the management of CAP in adults in 2000. Because of evolving resistance to antimicrobials and advances in diagnosis, treatment and prevention of CAP, it is felt that an update should be provided every three years so that important developments can be highlighted and pressing questions can be answered. Thus, the guidelines committee updated its guidelines in 2005. The basic policy and main purposes of the JRS guidelines include; 1) prevention of bacterial resistance and 2) effective and long-term use of medical resources. The JRS guidelines have recommended the exclusion of potential and broad spectrum antibiotics, fluoroquinolones and carbapenems, from the list of first-choice drugs for empirical treatment. In addition, the JRS guidelines have recommended short-term usage of antibiotics of an appropriate dose and pathogen-specific treatment using rapid diagnostic methods if possible.","container-title":"Internal Medicine","DOI":"10.2169/internalmedicine.45.1691","issue":"7","page":"419-428","source":"J-Stage","title":"The JRS Guidelines for the Management of Community-acquired Pneumonia in Adults:An Update and New Recommendations","title-short":"The JRS Guidelines for the Management of Community-acquired Pneumonia in Adults","volume":"45","author":[{"family":"Miyashita","given":"Naoyuki"},{"family":"Matsushima","given":"Toshiharu"},{"family":"Oka","given":"Mikio"}],"issued":{"date-parts":[["2006"]]}}},{"id":5323,"uris":[""],"uri":[""],"itemData":{"id":5323,"type":"article-journal","abstract":"OBJECTIVE: The study was performed to validate the CURB, CRB and CRB-65 scores for the prediction of death from community-acquired pneumonia (CAP) in both the hospital and out-patient setting.\nDESIGN: Data were derived from a large multi-centre prospective study initiated by the German competence network for community-acquired pneumonia (CAPNETZ) which started in March 2003 and were censored for this analysis in October 2004.\nSETTING: Out- and in-hospital patients in 670 private practices and 10 clinical centres.\nSUBJECTS: Analysis was done for n = 1343 patients (n = 208 out-patients and n = 1135 hospitalized) with all data sets completed for the calculation of CURB and repeated for n = 1967 patients (n = 482 out-patients and n = 1485 hospitalized) with complete data sets for CRB and CRB-65.\nINTERVENTION: None. 30-day mortality from CAP was determined by personal contacts or a structured interview.\nRESULTS: Overall 30-day mortality was 4.3% (0.6% in out-patients and 5.5% in hospitalized patients, P < 0.0001). Overall, the CURB, CRB and CRB-65 scores provided comparable predictions for death from CAP as determined by receiver-operator-characteristics (ROC) curves. However, in hospitalized patients, CRB misclassified 26% of deaths as low risk patients. Availability of the CRB-65 score (90%) was far superior to that of CURB (65%), due to missing blood urea nitrogen values (P < 0.001).\nCONCLUSIONS: Both the CURB and CRB-65 scores can be used in the hospital and out-patients setting to assess pneumonia severity and the risk of death. Given that the CRB-65 is easier to handle, we favour the use of CRB-65 where blood urea nitrogen is unavailable.","container-title":"Journal of Internal Medicine","DOI":"10.1111/j.1365-2796.2006.01657.x","ISSN":"0954-6820","issue":"1","journalAbbreviation":"J. Intern. Med.","language":"eng","note":"PMID: 16789984","page":"93-101","source":"PubMed","title":"CRB-65 predicts death from community-acquired pneumonia","volume":"260","author":[{"family":"Bauer","given":"T. T."},{"family":"Ewig","given":"S."},{"family":"Marre","given":"R."},{"family":"Suttorp","given":"N."},{"family":"Welte","given":"T."},{"literal":"CAPNETZ Study Group"}],"issued":{"date-parts":[["2006",7]]}},"locator":"65"},{"id":5203,"uris":[""],"uri":[""],"itemData":{"id":5203,"type":"webpage","title":"Comparison of Clinical Characteristics and Performance of Pneumonia Severity Score and CURB-65 Among Younger Adults, Elderly and Very Old Subjects - PubMed","URL":"","accessed":{"date-parts":[["2020",5,31]]}},"locator":"65"},{"id":5329,"uris":[""],"uri":[""],"itemData":{"id":5329,"type":"article-journal","abstract":"BACKGROUND: Patients with community-acquired pneumonia (CAP) often require hospitalisation. CRB-65 is a simple and useful scoring system to predict mortality. However, prognostic factors such as underlying disease and blood oxygenation are not included despite their potential to increase the performance of CRB-65.\nMETHODS: The study included 1172 consecutive patients (830 inpatients, 342 outpatients) with CAP. Mortality, sensitivity, specificity, positive predictive value and negative predictive value, and the area under the receiver operating characteristic (ROC) curve with 95% CI were calculated. Prognostic accuracy was evaluated after adding coexisting illnesses according to the Pneumonia Severity Index (malignancy, heart failure, hepatic, renal and cerebrovascular disease) and pulse oximetry (SpO2).\nRESULTS: Mean age was 65?years, 30-day mortality 7% (inpatients 9%, outpatients 1%). Addition of one point for the presence of ≥1 coexisting condition and one point for SpO2 <90% increased the area under the ROC curve of CRB-65 from 0.82 (95% CI 0.77 to 0.85) to 0.87 (95% CI 0.84 to 0.90; p<0.0001).\nCONCLUSIONS: Modification of CRB-65 by including hypoxaemia and presence of specified underlying diseases increased the scoring system's prognostic accuracy while retaining its independence of laboratory tests. DS CRB-65 may have the potential to further facilitate site of care decision for patients with CAP.","container-title":"BMJ open respiratory research","DOI":"10.1136/bmjresp-2014-000038","ISSN":"2052-4439","issue":"1","journalAbbreviation":"BMJ Open Respir Res","language":"eng","note":"PMID: 25478185\nPMCID: PMC4212804","page":"e000038","source":"PubMed","title":"Improvement of CRB-65 as a prognostic tool in adult patients with community-acquired pneumonia","volume":"1","author":[{"family":"Dwyer","given":"Richard"},{"family":"Hedlund","given":"Jonas"},{"family":"Henriques-Normark","given":"Birgitta"},{"family":"Kalin","given":"Mats"}],"issued":{"date-parts":[["2014"]]}}},{"id":5332,"uris":[""],"uri":[""],"itemData":{"id":5332,"type":"article-journal","abstract":"Aim of this study was to develop a new simpler and more effective severity score for community-acquired pneumonia (CAP) patients. A total of 1640 consecutive hospitalized CAP patients in Second Affiliated Hospital of Zhejiang University were included. The effectiveness of different pneumonia severity scores to predict mortality was compared, and the performance of the new score was validated on an external cohort of 1164 patients with pneumonia admitted to a teaching hospital in Italy. Using age?≥?65 years, LDH?>?230?u/L, albumin?<?3.5?g/dL, platelet count?<?100?×?109/L, confusion, urea?>?7?mmol/L, respiratory rate?≥?30/min, low blood pressure, we assembled a new severity score named as expanded-CURB-65. The 30-day mortality and length of stay were increased along with increased risk score. The AUCs in the prediction of 30-day mortality in the main cohort were 0.826 (95%?CI, 0.807–0.844), 0.801 (95%?CI, 0.781–0.820), 0.756 (95%?CI, 0.735–0.777), 0.793 (95%?CI, 0.773–0.813) and 0.759 (95%?CI, 0.737–0.779) for the expanded-CURB-65, PSI, CURB-65, SMART-COP and A-DROP, respectively. The performance of this bedside score was confirmed in CAP patients of the validation cohort although calibration was not successful in patients with health care-associated pneumonia (HCAP). The expanded CURB-65 is objective, simpler and more accurate scoring system for evaluation of CAP severity, and the predictive efficiency was better than other score systems.","container-title":"Scientific Reports","DOI":"10.1038/srep22911","ISSN":"2045-2322","journalAbbreviation":"Sci Rep","note":"PMID: 26987602\nPMCID: PMC4796818","page":"22911","source":"PubMed Central","title":"Expanded CURB-65: a new score system predicts severity of community-acquired pneumonia with superior efficiency","title-short":"Expanded CURB-65","volume":"6","author":[{"family":"Liu","given":"Jin-liang"},{"family":"Xu","given":"Feng"},{"family":"Hui Zhou","given":""},{"family":"Wu","given":"Xue-jie"},{"family":"Shi","given":"Ling-xian"},{"family":"Lu","given":"Rui-qing"},{"family":"Farcomeni","given":"Alessio"},{"family":"Venditti","given":"Mario"},{"family":"Zhao","given":"Ying-li"},{"family":"Luo","given":"Shu-ya"},{"family":"Dong","given":"Xiao-jun"},{"family":"Falcone","given":"Marco"}],"issued":{"date-parts":[["2016",3,18]]}}},{"id":5351,"uris":[""],"uri":[""],"itemData":{"id":5351,"type":"webpage","abstract":"NEWS2 is the latest version of the National Early Warning Score (NEWS), first produced in 2012 and updated in December 2017, which advocates a system to standardise the assessment and response to acute illness.","container-title":"RCP London","note":"source: rcplondon.ac.uk","title":"National Early Warning Score (NEWS) 2","URL":"","accessed":{"date-parts":[["2020",7,23]]},"issued":{"date-parts":[["2017",12,19]]}},"locator":"2"},{"id":5338,"uris":[""],"uri":[""],"itemData":{"id":5338,"type":"article-journal","abstract":"<h3>Importance</h3><p>Definitions of sepsis and septic shock were last revised in 2001. Considerable advances have since been made into the pathobiology (changes in organ function, morphology, cell biology, biochemistry, immunology, and circulation), management, and epidemiology of sepsis, suggesting the need for reexamination.</p><h3>Objective</h3><p>To evaluate and, as needed, update definitions for sepsis and septic shock.</p><h3>Process</h3><p>A task force (n = 19) with expertise in sepsis pathobiology, clinical trials, and epidemiology was convened by the Society of Critical Care Medicine and the European Society of Intensive Care Medicine. Definitions and clinical criteria were generated through meetings, Delphi processes, analysis of electronic health record databases, and voting, followed by circulation to international professional societies, requesting peer review and endorsement (by 31 societies listed in the Acknowledgment).</p><h3>Key Findings From Evidence Synthesis</h3><p>Limitations of previous definitions included an excessive focus on inflammation, the misleading model that sepsis follows a continuum through severe sepsis to shock, and inadequate specificity and sensitivity of the systemic inflammatory response syndrome (SIRS) criteria. Multiple definitions and terminologies are currently in use for sepsis, septic shock, and organ dysfunction, leading to discrepancies in reported incidence and observed mortality. The task force concluded the term<i>severe sepsis</i>was redundant.</p><h3>Recommendations</h3><p>Sepsis should be defined as life-threatening organ dysfunction caused by a dysregulated host response to infection. For clinical operationalization, organ dysfunction can be represented by an increase in the Sequential [Sepsis-related] Organ Failure Assessment (SOFA) score of 2 points or more, which is associated with an in-hospital mortality greater than 10%. Septic shock should be defined as a subset of sepsis in which particularly profound circulatory, cellular, and metabolic abnormalities are associated with a greater risk of mortality than with sepsis alone. Patients with septic shock can be clinically identified by a vasopressor requirement to maintain a mean arterial pressure of 65 mm Hg or greater and serum lactate level greater than 2 mmol/L (&gt;18 mg/dL) in the absence of hypovolemia. This combination is associated with hospital mortality rates greater than 40%. In out-of-hospital, emergency department, or general hospital ward settings, adult patients with suspected infection can be rapidly identified as being more likely to have poor outcomes typical of sepsis if they have at least 2 of the following clinical criteria that together constitute a new bedside clinical score termed quickSOFA (qSOFA): respiratory rate of 22/min or greater, altered mentation, or systolic blood pressure of 100 mm Hg or less.</p><h3>Conclusions and Relevance</h3><p>These updated definitions and clinical criteria should replace previous definitions, offer greater consistency for epidemiologic studies and clinical trials, and facilitate earlier recognition and more timely management of patients with sepsis or at risk of developing sepsis.</p>","container-title":"JAMA","DOI":"10.1001/jama.2016.0287","ISSN":"0098-7484","issue":"8","journalAbbreviation":"JAMA","language":"en","note":"publisher: American Medical Association","page":"801-810","source":"","title":"The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)","volume":"315","author":[{"family":"Singer","given":"Mervyn"},{"family":"Deutschman","given":"Clifford S."},{"family":"Seymour","given":"Christopher Warren"},{"family":"Shankar-Hari","given":"Manu"},{"family":"Annane","given":"Djillali"},{"family":"Bauer","given":"Michael"},{"family":"Bellomo","given":"Rinaldo"},{"family":"Bernard","given":"Gordon R."},{"family":"Chiche","given":"Jean-Daniel"},{"family":"Coopersmith","given":"Craig M."},{"family":"Hotchkiss","given":"Richard S."},{"family":"Levy","given":"Mitchell M."},{"family":"Marshall","given":"John C."},{"family":"Martin","given":"Greg S."},{"family":"Opal","given":"Steven M."},{"family":"Rubenfeld","given":"Gordon D."},{"family":"Poll","given":"Tom","dropping-particle":"van der"},{"family":"Vincent","given":"Jean-Louis"},{"family":"Angus","given":"Derek C."}],"issued":{"date-parts":[["2016",2,23]]}}},{"id":5346,"uris":[""],"uri":[""],"itemData":{"id":5346,"type":"article-journal","abstract":"BACKGROUND: Existing severity assessment tools, such as the pneumonia severity index (PSI) and CURB-65 (tool based on confusion, urea level, respiratory rate, blood pressure, and age >or=65 years), predict 30-day mortality in community-acquired pneumonia (CAP) and have limited ability to predict which patients will require intensive respiratory or vasopressor support (IRVS).\nMETHODS: The Australian CAP Study (ACAPS) was a prospective study of 882 episodes in which each patient had a detailed assessment of severity features, etiology, and treatment outcomes. Multivariate logistic regression was performed to identify features at initial assessment that were associated with receipt of IRVS. These results were converted into a simple points-based severity tool that was validated in 5 external databases, totaling 7464 patients.\nRESULTS: In ACAPS, 10.3% of patients received IRVS, and the 30-day mortality rate was 5.7%. The features statistically significantly associated with receipt of IRVS were low systolic blood pressure (2 points), multilobar chest radiography involvement (1 point), low albumin level (1 point), high respiratory rate (1 point), tachycardia (1 point), confusion (1 point), poor oxygenation (2 points), and low arterial pH (2 points): SMART-COP. A SMART-COP score of >or=3 points identified 92% of patients who received IRVS, including 84% of patients who did not need immediate admission to the intensive care unit. Accuracy was also high in the 5 validation databases. Sensitivities of PSI and CURB-65 for identifying the need for IRVS were 74% and 39%, respectively.\nCONCLUSIONS: SMART-COP is a simple, practical clinical tool for accurately predicting the need for IRVS that is likely to assist clinicians in determining CAP severity.","container-title":"Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America","DOI":"10.1086/589754","ISSN":"1537-6591","issue":"3","journalAbbreviation":"Clin. Infect. Dis.","language":"eng","note":"PMID: 18558884","page":"375-384","source":"PubMed","title":"SMART-COP: a tool for predicting the need for intensive respiratory or vasopressor support in community-acquired pneumonia","title-short":"SMART-COP","volume":"47","author":[{"family":"Charles","given":"Patrick G. P."},{"family":"Wolfe","given":"Rory"},{"family":"Whitby","given":"Michael"},{"family":"Fine","given":"Michael J."},{"family":"Fuller","given":"Andrew J."},{"family":"Stirling","given":"Robert"},{"family":"Wright","given":"Alistair A."},{"family":"Ramirez","given":"Julio A."},{"family":"Christiansen","given":"Keryn J."},{"family":"Waterer","given":"Grant W."},{"family":"Pierce","given":"Robert J."},{"family":"Armstrong","given":"John G."},{"family":"Korman","given":"Tony M."},{"family":"Holmes","given":"Peter"},{"family":"Obrosky","given":"D. Scott"},{"family":"Peyrani","given":"Paula"},{"family":"Johnson","given":"Barbara"},{"family":"Hooy","given":"Michelle"},{"literal":"Australian Community-Acquired Pneumonia Study Collaboration"},{"family":"Grayson","given":"M. Lindsay"}],"issued":{"date-parts":[["2008",8,1]]}}},{"id":5349,"uris":[""],"uri":[""],"itemData":{"id":5349,"type":"article-journal","container-title":"Intensive Care Medicine","DOI":"10.1007/BF01709751","ISSN":"0342-4642","issue":"7","journalAbbreviation":"Intensive Care Med","language":"eng","note":"PMID: 8844239","page":"707-710","source":"PubMed","title":"The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine","volume":"22","author":[{"family":"Vincent","given":"J. L."},{"family":"Moreno","given":"R."},{"family":"Takala","given":"J."},{"family":"Willatts","given":"S."},{"family":"De Mendon?a","given":"A."},{"family":"Bruining","given":"H."},{"family":"Reinhart","given":"C. K."},{"family":"Suter","given":"P. M."},{"family":"Thijs","given":"L. G."}],"issued":{"date-parts":[["1996",7]]}}},{"id":5344,"uris":[""],"uri":[""],"itemData":{"id":5344,"type":"article-journal","abstract":"BACKGROUND: The comparative accuracy and discriminatory power of three validated rules for predicting clinically relevant outcomes other than mortality in patients hospitalized with community-acquired pneumonia (CAP) are unknown.\nMETHODS: We prospectively compared the newly developed severe community-acquired pneumonia (SCAP) score, pneumonia severity index (PSI), and the British Thoracic Society confusion, urea > 7 mmol/L, respiratory rate > or = 30 breaths/min, BP < 90 mm Hg systolic or < 60 mm Hg diastolic, age > or = 65 years (CURB-65) rule in an internal validation cohort of 1,189 consecutive adult inpatients with CAP from one hospital and an external validation cohort of 671 consecutive adult inpatients from three other hospitals. Major adverse outcomes were admission to ICU, need for mechanical ventilation, progression to severe sepsis, or treatment failure. Mean hospital length of stay (LOS) was also evaluated. The rules were compared based on sensitivity, specificity, and area under the curve (AUC) of the receiver operating characteristic.\nRESULTS: The rate of all adverse outcomes and hospital LOS increased directly with increasing SCAP, PSI, or CURB-65 scores (p < 0.001) in both cohorts. Patients classified as high risk by the SCAP score showed higher rates of adverse outcomes (ICU admission, 35.8%; mechanical ventilation, 16.4%; severe sepsis, 98.5%; treatment failure, 22.4%) than PSI and CURB-65 high-risk classes. The discriminatory power of SCAP, as measured by AUC, was 0.75 for ICU admission, 0.76 for mechanical ventilation, 0.79 for severe sepsis, and 0.61 for treatment failure in the external validation cohort. AUC differences with PSI or CURB-65 were found.\nCONCLUSIONS: The SCAP score is as accurate or better than other current scoring systems in predicting adverse outcomes in patients hospitalized with CAP while helping classify patients into different categories of increasing risk for potentially closer monitoring.","container-title":"Chest","DOI":"10.1378/chest.08-2179","ISSN":"1931-3543","issue":"6","journalAbbreviation":"Chest","language":"eng","note":"PMID: 19141524","page":"1572-1579","source":"PubMed","title":"Prospective comparison of severity scores for predicting clinically relevant outcomes for patients hospitalized with community-acquired pneumonia","volume":"135","author":[{"family":"Yandiola","given":"Pedro Pablo Espa?a"},{"family":"Capelastegui","given":"Alberto"},{"family":"Quintana","given":"José"},{"family":"Diez","given":"Rosa"},{"family":"Gorordo","given":"Inmaculada"},{"family":"Bilbao","given":"Amaia"},{"family":"Zalacain","given":"Rafael"},{"family":"Menendez","given":"Rosario"},{"family":"Torres","given":"Antonio"}],"issued":{"date-parts":[["2009",6]]}}}],"schema":""} 17,27–36, and four novel scores developed in covid-19 patients ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"stKlamSg","properties":{"formattedCitation":"\\super 24,37\\uc0\\u8211{}39\\nosupersub{}","plainCitation":"24,37–39","noteIndex":0},"citationItems":[{"id":5237,"uris":[""],"uri":[""],"itemData":{"id":5237,"type":"article-journal","abstract":"<h3>Importance</h3><p>Early identification of patients with novel corona virus disease 2019 (COVID-19) who may develop critical illness is of great importance and may aid in delivering proper treatment and optimizing use of resources.</p><h3>Objective</h3><p>To develop and validate a clinical score at hospital admission for predicting which patients with COVID-19 will develop critical illness based on a nationwide cohort in China.</p><h3>Design, Setting, and Participants</h3><p>Collaborating with the National Health Commission of China, we established a retrospective cohort of patients with COVID-19 from 575 hospitals in 31 provincial administrative regions as of January 31, 2020. Epidemiological, clinical, laboratory, and imaging variables ascertained at hospital admission were screened using Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression to construct a predictive risk score (COVID-GRAM). The score provides an estimate of the risk that a hospitalized patient with COVID-19 will develop critical illness. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC). Data from 4 additional cohorts in China hospitalized with COVID-19 were used to validate the score. Data were analyzed between February 20, 2020 and March 17, 2020.</p><h3>Main Outcomes and Measures</h3><p>Among patients with COVID-19 admitted to the hospital, critical illness was defined as the composite measure of admission to the intensive care unit, invasive ventilation, or death.</p><h3>Results</h3><p>The development cohort included 1590 patients. the mean (SD) age of patients in the cohort was 48.9 (15.7) years; 904 (57.3%) were men. The validation cohort included 710 patients with a mean (SD) age of 48.2 (15.2) years, and 382 (53.8%) were men and 172 (24.2%). From 72 potential predictors, 10 variables were independent predictive factors and were included in the risk score: chest radiographic abnormality (OR, 3.39; 95% CI, 2.14-5.38), age (OR, 1.03; 95% CI, 1.01-1.05), hemoptysis (OR, 4.53; 95% CI, 1.36-15.15), dyspnea (OR, 1.88; 95% CI, 1.18-3.01), unconsciousness (OR, 4.71; 95% CI, 1.39-15.98), number of comorbidities (OR, 1.60; 95% CI, 1.27-2.00), cancer history (OR, 4.07; 95% CI, 1.23-13.43), neutrophil-to-lymphocyte ratio (OR, 1.06; 95% CI, 1.02-1.10), lactate dehydrogenase (OR, 1.002; 95% CI, 1.001-1.004) and direct bilirubin (OR, 1.15; 95% CI, 1.06-1.24). The mean AUC in the development cohort was 0.88 (95% CI, 0.85-0.91) and the AUC in the validation cohort was 0.88 (95% CI, 0.84-0.93). The score has been translated into an online risk calculator that is freely available to the public ()</p><h3>Conclusions and Relevance</h3><p>In this study, a risk score based on characteristics of COVID-19 patients at the time of admission to the hospital was developed that may help predict a patient’s risk of developing critical illness.</p>","container-title":"JAMA Internal Medicine","DOI":"10.1001/jamainternmed.2020.2033","journalAbbreviation":"JAMA Intern Med","language":"en","source":"","title":"Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19","URL":"","author":[{"family":"Liang","given":"Wenhua"},{"family":"Liang","given":"Hengrui"},{"family":"Ou","given":"Limin"},{"family":"Chen","given":"Binfeng"},{"family":"Chen","given":"Ailan"},{"family":"Li","given":"Caichen"},{"family":"Li","given":"Yimin"},{"family":"Guan","given":"Weijie"},{"family":"Sang","given":"Ling"},{"family":"Lu","given":"Jiatao"},{"family":"Xu","given":"Yuanda"},{"family":"Chen","given":"Guoqiang"},{"family":"Guo","given":"Haiyan"},{"family":"Guo","given":"Jun"},{"family":"Chen","given":"Zisheng"},{"family":"Zhao","given":"Yi"},{"family":"Li","given":"Shiyue"},{"family":"Zhang","given":"Nuofu"},{"family":"Zhong","given":"Nanshan"},{"family":"He","given":"Jianxing"}],"accessed":{"date-parts":[["2020",6,1]]},"issued":{"date-parts":[["2020",5,12]]}}},{"id":5256,"uris":[""],"uri":[""],"itemData":{"id":5256,"type":"article-journal","abstract":"<p>Background: COVID-19 pandemic has developed rapidly and the ability to stratify the most vulnerable patients is vital. However, routinely used severity scoring systems are often low on diagnosis, even in non-survivors. Therefore, clinical prediction models for mortality are urgently required. Methods: We developed and internally validated a multivariable logistic regression model to predict inpatient mortality in COVID-19 positive patients using data collected retrospectively from Tongji Hospital, Wuhan (299 patients). External validation was conducted using a retrospective cohort from Jinyintan Hospital, Wuhan (145 patients). Nine variables commonly measured in these acute settings were considered for model development, including age, biomarkers and comorbidities. Backwards stepwise selection and bootstrap resampling were used for model development and internal validation. We assessed discrimination via the C statistic, and calibration using calibration-in-the-large, calibration slopes and plots. Findings: The final model included age, lymphocyte count, lactate dehydrogenase and SpO2 as independent predictors of mortality. Discrimination of the model was excellent in both internal (c=0.89) and external (c=0.98) validation. Internal calibration was excellent (calibration slope=1). External validation showed some over-prediction of risk in low-risk individuals and under-prediction of risk in high-risk individuals prior to recalibration. Recalibration of the intercept and slope led to excellent performance of the model in independent data. Interpretation: COVID-19 is a new disease and behaves differently from common critical illnesses. This study provides a new prediction model to identify patients with lethal COVID-19. Its practical reliance on commonly available parameters should improve usage of limited healthcare resources and patient survival rate.</p>","container-title":"medRxiv","DOI":"10.1101/2020.03.28.20045997","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.03.28.20045997","source":"","title":"Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19","author":[{"family":"Xie","given":"Jianfeng"},{"family":"Hungerford","given":"Daniel"},{"family":"Chen","given":"Hui"},{"family":"Abrams","given":"Simon T."},{"family":"Li","given":"Shusheng"},{"family":"Wang","given":"Guozheng"},{"family":"Wang","given":"Yishan"},{"family":"Kang","given":"Hanyujie"},{"family":"Bonnett","given":"Laura"},{"family":"Zheng","given":"Ruiqiang"},{"family":"Li","given":"Xuyan"},{"family":"Tong","given":"Zhaohui"},{"family":"Du","given":"Bin"},{"family":"Qiu","given":"Haibo"},{"family":"Toh","given":"Cheng-Hock"}],"issued":{"date-parts":[["2020",4,7]]}}},{"id":5254,"uris":[""],"uri":[""],"itemData":{"id":5254,"type":"webpage","title":"Surgisphere - Mortality Risk Calculator","URL":"","accessed":{"date-parts":[["2020",6,1]]}}},{"id":5353,"uris":[""],"uri":[""],"itemData":{"id":5353,"type":"webpage","title":"Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK | medRxiv","URL":"","author":[{"family":"Zhang","given":"Huayu"},{"family":"Shi","given":"Ting"},{"family":"Wu","given":"Xiaodong"},{"family":"Zhang","given":"Zin"},{"family":"Wang","given":"Kun"},{"family":"Bean","given":"Daniel"},{"family":"Dobson","given":"Richard"}],"accessed":{"date-parts":[["2020",7,23]]}}}],"schema":""} 24,37–39. Among existing risk stratification scores, E-CURB65 and the Xie score had the highest discriminatory power. However, we were only able to calculate E-CURB65 and Xie scores for 6.4% and 7.2% of the validation cohort, respectively, due to the requirement for biomarkers or physiological parameters that were not routinely captured.A-DROP, a modified version of CURB-65, had moderate discriminatory power and good applicability (calculated for 63.9% of the validation cohorts overall) among existing prognostic tools. The relatively high predictive performance despite comprising a small number of parameters is likely, in part, due to age stratification within the score (age: >70 years in men, >75 years in women). This highlights the trade-off between accuracy versus simplicity and applicability when developing a prognostic model.The 4C score has several methodological advantages over current covid-19 prognostic scores. The use of penalised regression methods, an event-to-variable ratio greater than 100 reducing the risk of model over-fitting, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"6Pm9dC7p","properties":{"formattedCitation":"\\super 40,41\\nosupersub{}","plainCitation":"40,41","noteIndex":0},"citationItems":[{"id":5181,"uris":[""],"uri":[""],"itemData":{"id":5181,"type":"article-journal","abstract":"When the number of events is low relative to the number of predictors, standard regression could produce over?tted risk models that make inaccurate predictions. Use of penalised regression may improve the accuracy of risk prediction","container-title":"The BMJ","DOI":"10.1136/bmj.h3868","ISSN":"0959-8138","journalAbbreviation":"BMJ","note":"PMID: 26264962\nPMCID: PMC4531311","source":"PubMed Central","title":"How to develop a more accurate risk prediction model when there are few events","URL":"","volume":"351","author":[{"family":"Pavlou","given":"Menelaos"},{"family":"Ambler","given":"Gareth"},{"family":"Seaman","given":"Shaun R"},{"family":"Guttmann","given":"Oliver"},{"family":"Elliott","given":"Perry"},{"family":"King","given":"Michael"},{"family":"Omar","given":"Rumana Z"}],"accessed":{"date-parts":[["2020",5,23]]},"issued":{"date-parts":[["2015",8,11]]}}},{"id":5184,"uris":[""],"uri":[""],"itemData":{"id":5184,"type":"article-journal","abstract":"The analytical effect of the number of events per variable (EPV) in a proportional hazards regression analysis was evaluated using Monte Carlo simulation techniques for data from a randomized trial containing 673 patients and 252 deaths, in which seven predictor variables had an original significance level of p < 0.10. The 252 deaths and 7 variables correspond to 36 events per variable analyzed in the full data set. Five hundred simulated analyses were conducted for these seven variables at EPVs of 2, 5, 10, 15, 20, and 25. For each simulation, a random exponential survival time was generated for each of the 673 patients, and the simulated results were compared with their original counterparts. As EPV decreased, the regression coefficients became more biased relative to the true value; the 90% confidence limits about the simulated values did not have a coverage of 90% for the original value; large sample properties did not hold for variance estimates from the proportional hazards model, and the Z statistics used to test the significance of the regression coefficients lost validity under the null hypothesis. Although a single boundary level for avoiding problems is not easy to choose, the value of EPV = 10 seems most prudent. Below this value for EPV, the results of proportional hazards regression analyses should be interpreted with caution because the statistical model may not be valid.","container-title":"Journal of Clinical Epidemiology","DOI":"10.1016/0895-4356(95)00048-8","ISSN":"0895-4356","issue":"12","journalAbbreviation":"J Clin Epidemiol","language":"eng","note":"PMID: 8543964","page":"1503-1510","source":"PubMed","title":"Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates","volume":"48","author":[{"family":"Peduzzi","given":"P."},{"family":"Concato","given":"J."},{"family":"Feinstein","given":"A. R."},{"family":"Holford","given":"T. R."}],"issued":{"date-parts":[["1995",12]]}}}],"schema":""} 40,41 and the use of clinical parameters at first assessment increases the clinical applicability of the score and limits use of highly selective predictors prevalent in other risk stratification scores.4, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"MTcfDrJX","properties":{"formattedCitation":"\\super 42\\nosupersub{}","plainCitation":"42","noteIndex":0},"citationItems":[{"id":2513,"uris":[""],"uri":[""],"itemData":{"id":2513,"type":"article-journal","abstract":"Purpose Reducing cigarette smoking has been proposed as a method of harm reduction. The effect of smoking reduction on cancer risk has not been studied in Asian populations. Patients and Methods A total of 479,156 Korean men, age 30 to 58 years, were stratified into nine groups based on smoking status in 1990 and 1992. From 1992 to 2003, patients were observed and tested for the occurrence of cancer. Results There was no association between smoking reduction and risk of all cancers. However, the risk of smoking-related cancers tended to decrease, though not significantly, when heavy smokers (>= 20 cigarettes/d) became moderate smokers (10 to 19 cigarettes/d), with a hazard ratio (HR) of 0.91 (95% CI, 0.82 to 1.02). For lung cancer, patients who reduced from heavy to moderate smoking and from heavy to light smoking (< 10 cigarettes/d) had significantly decreased risks based on multivariable-adjusted HRs (HR = 0.72, 95% CI, 0.49 to 0.89; HR = 0.63, 95% CI, 0.46 to 0.84, respectively). Study participants who never smoked, sustained ex-smokers, and quitters had lower risks for all cancers, smoking-related cancers, and lung cancer in a dose-response manner as compared with heavy smokers. Conclusion Smoking reduction was associated with a significant decrease in the risk of lung cancer, but the size of risk reduction was disproportionately smaller than that expected from the reduced amount of cigarette consumption. Although smoking cessation should be the cornerstone of preventing smoking-related cancers, smoking reduction could be considered as a strategy to supplement smoking cessation for those who are unable to quit smoking immediately. J Clin Oncol 26: 5101-5106. (C) 2008 by American Society of Clinical Oncology","container-title":"Journal of Clinical Oncology","DOI":"10.1200/JCO.2008.17.0498","ISSN":"0732-183X","issue":"31","note":"WOS:000260537600018","page":"5101-5106","title":"Reduction and Cessation of Cigarette Smoking and Risk of Cancer: A Cohort Study of Korean Men","volume":"26","author":[{"family":"Song","given":"Yun-Mi"},{"family":"Sung","given":"Joohon"},{"family":"Cho","given":"Hong-Jun"}],"issued":{"date-parts":[["2008",11,1]]}}}],"schema":""} 42 In addition, the sensitivity analyses demonstrated score performance was robust.Existing prognostic scores developed in covid-19 patients have a high risk of bias, with their intended use commonly unreported and the majority failing to provide a usable equation, format or reference for use. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"jy3U1K9l","properties":{"formattedCitation":"\\super 5\\nosupersub{}","plainCitation":"5","noteIndex":0},"citationItems":[{"id":5193,"uris":[""],"uri":[""],"itemData":{"id":5193,"type":"article-journal","abstract":"Objective To review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.\nDesign Rapid systematic review and critical appraisal.\nData sources PubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.\nStudy selection Studies that developed or validated a multivariable covid-19 related prediction model.\nData extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).\nResults 2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.\nConclusion Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.\nSystematic review registration Protocol , registration .","container-title":"BMJ","DOI":"10.1136/bmj.m1328","ISSN":"1756-1833","journalAbbreviation":"BMJ","language":"en","note":"publisher: British Medical Journal Publishing Group\nsection: Research\nPMID: 32265220","source":"","title":"Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal","title-short":"Prediction models for diagnosis and prognosis of covid-19 infection","URL":"","volume":"369","author":[{"family":"Wynants","given":"Laure"},{"family":"Calster","given":"Ben Van"},{"family":"Bonten","given":"Marc M. J."},{"family":"Collins","given":"Gary S."},{"family":"Debray","given":"Thomas P. A."},{"family":"Vos","given":"Maarten De"},{"family":"Haller","given":"Maria C."},{"family":"Heinze","given":"Georg"},{"family":"Moons","given":"Karel G. M."},{"family":"Riley","given":"Richard D."},{"family":"Schuit","given":"Ewoud"},{"family":"Smits","given":"Luc J. M."},{"family":"Snell","given":"Kym I. E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Wallisch","given":"Christine"},{"family":"Smeden","given":"Maarten","dropping-particle":"van"}],"accessed":{"date-parts":[["2020",5,31]]},"issued":{"date-parts":[["2020",4,7]]}}}],"schema":""} 5 High discriminatory performance above 0.90 and use of small cohorts for model development suggest over-fitting, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"25U5LlZn","properties":{"formattedCitation":"\\super 37,43,44\\nosupersub{}","plainCitation":"37,43,44","noteIndex":0},"citationItems":[{"id":5256,"uris":[""],"uri":[""],"itemData":{"id":5256,"type":"article-journal","abstract":"<p>Background: COVID-19 pandemic has developed rapidly and the ability to stratify the most vulnerable patients is vital. However, routinely used severity scoring systems are often low on diagnosis, even in non-survivors. Therefore, clinical prediction models for mortality are urgently required. Methods: We developed and internally validated a multivariable logistic regression model to predict inpatient mortality in COVID-19 positive patients using data collected retrospectively from Tongji Hospital, Wuhan (299 patients). External validation was conducted using a retrospective cohort from Jinyintan Hospital, Wuhan (145 patients). Nine variables commonly measured in these acute settings were considered for model development, including age, biomarkers and comorbidities. Backwards stepwise selection and bootstrap resampling were used for model development and internal validation. We assessed discrimination via the C statistic, and calibration using calibration-in-the-large, calibration slopes and plots. Findings: The final model included age, lymphocyte count, lactate dehydrogenase and SpO2 as independent predictors of mortality. Discrimination of the model was excellent in both internal (c=0.89) and external (c=0.98) validation. Internal calibration was excellent (calibration slope=1). External validation showed some over-prediction of risk in low-risk individuals and under-prediction of risk in high-risk individuals prior to recalibration. Recalibration of the intercept and slope led to excellent performance of the model in independent data. Interpretation: COVID-19 is a new disease and behaves differently from common critical illnesses. This study provides a new prediction model to identify patients with lethal COVID-19. Its practical reliance on commonly available parameters should improve usage of limited healthcare resources and patient survival rate.</p>","container-title":"medRxiv","DOI":"10.1101/2020.03.28.20045997","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.03.28.20045997","source":"","title":"Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19","author":[{"family":"Xie","given":"Jianfeng"},{"family":"Hungerford","given":"Daniel"},{"family":"Chen","given":"Hui"},{"family":"Abrams","given":"Simon T."},{"family":"Li","given":"Shusheng"},{"family":"Wang","given":"Guozheng"},{"family":"Wang","given":"Yishan"},{"family":"Kang","given":"Hanyujie"},{"family":"Bonnett","given":"Laura"},{"family":"Zheng","given":"Ruiqiang"},{"family":"Li","given":"Xuyan"},{"family":"Tong","given":"Zhaohui"},{"family":"Du","given":"Bin"},{"family":"Qiu","given":"Haibo"},{"family":"Toh","given":"Cheng-Hock"}],"issued":{"date-parts":[["2020",4,7]]}}},{"id":5259,"uris":[""],"uri":[""],"itemData":{"id":5259,"type":"article-journal","abstract":"<p>Abstract Objectives To develop and test machine learning-based CT radiomics models for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection. Design Cross-sectional Setting Multicenter Participants A total of 52 patients with laboratory-confirmed SARS-CoV-2 infection and their initial CT images were enrolled from 5 designated hospitals in Ankang, Lishui, Zhenjiang, Lanzhou, and Linxia between January 23, 2020 and February 8, 2020. As of February 20, patients remained in hospital or with non-findings in CT were excluded. Therefore, 31 patients with 72 lesion segments were included in the final analysis. Intervention CT radiomics models based on logistic regression (LR) and random forest (RF) were developed on features extracted from pneumonia lesions in training and inter-validation datasets. The predictive performance was further evaluated in test dataset on lung lobe- and patients-level. Main outcomes Short-term hospital stay (≤10 days) and long-term hospital stay (&gt;10 days). Results The CT radiomics models based on 6 second-order features were effective in discriminating short- and long-term hospital stay in patients with pneumonia associated with SARS-CoV-2 infection, with areas under the curves of 0.97 (95%CI 0.83-1.0) and 0.92 (95%CI 0.67-1.0) by LR and RF, respectively, in the test dataset. The LR model showed a sensitivity and specificity of 1.0 and 0.89, and the RF model showed similar performance with sensitivity and specificity of 0.75 and 1.0 in test dataset. Conclusions The machine learning-based CT radiomics models showed feasibility and accuracy for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection.</p>","container-title":"medRxiv","DOI":"10.1101/2020.02.29.20029603","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.02.29.20029603","source":"","title":"Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: A multicenter study","title-short":"Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection","author":[{"family":"Qi","given":"Xiaolong"},{"family":"Jiang","given":"Zicheng"},{"family":"Yu","given":"Qian"},{"family":"Shao","given":"Chuxiao"},{"family":"Zhang","given":"Hongguang"},{"family":"Yue","given":"Hongmei"},{"family":"Ma","given":"Baoyi"},{"family":"Wang","given":"Yuancheng"},{"family":"Liu","given":"Chuan"},{"family":"Meng","given":"Xiangpan"},{"family":"Huang","given":"Shan"},{"family":"Wang","given":"Jitao"},{"family":"Xu","given":"Dan"},{"family":"Lei","given":"Junqiang"},{"family":"Xie","given":"Guanghang"},{"family":"Huang","given":"Huihong"},{"family":"Yang","given":"Jie"},{"family":"Ji","given":"Jiansong"},{"family":"Pan","given":"Hongqiu"},{"family":"Zou","given":"Shengqiang"},{"family":"Ju","given":"Shenghong"}],"issued":{"date-parts":[["2020",3,3]]}}},{"id":5262,"uris":[""],"uri":[""],"itemData":{"id":5262,"type":"article-journal","abstract":"Radiologic characteristics of 2019 novel coronavirus (2019-nCoV) infected pneumonia (NCIP) which had not been fully understood are especially important for diagnosing and predicting prognosis. We retrospective studied 27 consecutive patients who were confirmed NCIP, the clinical characteristics and CT image findings were collected, and the association of radiologic findings with mortality of patients was evaluated. 27 patients included 12 men and 15 women, with median age of 60 years (IQR 47-69). 17 patients discharged in recovered condition and 10 patients died in hospital. The median age of mortality group was higher compared to survival group (68 (IQR 63-73) vs 55 (IQR 35-60), P = 0.003). The comorbidity rate in mortality group was significantly higher than in survival group (80% vs 29%, P = 0.018). The predominant CT characteristics consisted of ground glass opacity (67%), bilateral sides involved (86%), both peripheral and central distribution (74%), and lower zone involvement (96%). The median CT score of mortality group was higher compared to survival group (30 (IQR 7-13) vs 12 (IQR 11-43), P = 0.021), with more frequency of consolidation (40% vs 6%, P = 0.047) and air bronchogram (60% vs 12%, P = 0.025). An optimal cutoff value of a CT score of 24.5 had a sensitivity of 85.6% and a specificity of 84.5% for the prediction of mortality. 2019-nCoV was more likely to infect elderly people with chronic comorbidities. CT findings of NCIP were featured by predominant ground glass opacities mixed with consolidations, mainly peripheral or combined peripheral and central distributions, bilateral and lower lung zones being mostly involved. A simple CT scoring method was capable to predict mortality.","container-title":"PloS One","DOI":"10.1371/journal.pone.0230548","ISSN":"1932-6203","issue":"3","journalAbbreviation":"PLoS ONE","language":"eng","note":"PMID: 32191764\nPMCID: PMC7082074","page":"e0230548","source":"PubMed","title":"Association of radiologic findings with mortality of patients infected with 2019 novel coronavirus in Wuhan, China","volume":"15","author":[{"family":"Yuan","given":"Mingli"},{"family":"Yin","given":"Wen"},{"family":"Tao","given":"Zhaowu"},{"family":"Tan","given":"Weijun"},{"family":"Hu","given":"Yi"}],"issued":{"date-parts":[["2020"]]}}}],"schema":""} 37,43,44 while unclear methodology ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"mRvjs9gC","properties":{"formattedCitation":"\\super 38\\nosupersub{}","plainCitation":"38","noteIndex":0},"citationItems":[{"id":5254,"uris":[""],"uri":[""],"itemData":{"id":5254,"type":"webpage","title":"Surgisphere - Mortality Risk Calculator","URL":"","accessed":{"date-parts":[["2020",6,1]]}}}],"schema":""} 38 and limited information around calibration limit generalisability. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"KdHOoi8e","properties":{"formattedCitation":"\\super 5,24\\nosupersub{}","plainCitation":"5,24","noteIndex":0},"citationItems":[{"id":5193,"uris":[""],"uri":[""],"itemData":{"id":5193,"type":"article-journal","abstract":"Objective To review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.\nDesign Rapid systematic review and critical appraisal.\nData sources PubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.\nStudy selection Studies that developed or validated a multivariable covid-19 related prediction model.\nData extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).\nResults 2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.\nConclusion Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.\nSystematic review registration Protocol , registration .","container-title":"BMJ","DOI":"10.1136/bmj.m1328","ISSN":"1756-1833","journalAbbreviation":"BMJ","language":"en","note":"publisher: British Medical Journal Publishing Group\nsection: Research\nPMID: 32265220","source":"","title":"Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal","title-short":"Prediction models for diagnosis and prognosis of covid-19 infection","URL":"","volume":"369","author":[{"family":"Wynants","given":"Laure"},{"family":"Calster","given":"Ben Van"},{"family":"Bonten","given":"Marc M. J."},{"family":"Collins","given":"Gary S."},{"family":"Debray","given":"Thomas P. A."},{"family":"Vos","given":"Maarten De"},{"family":"Haller","given":"Maria C."},{"family":"Heinze","given":"Georg"},{"family":"Moons","given":"Karel G. M."},{"family":"Riley","given":"Richard D."},{"family":"Schuit","given":"Ewoud"},{"family":"Smits","given":"Luc J. M."},{"family":"Snell","given":"Kym I. E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Wallisch","given":"Christine"},{"family":"Smeden","given":"Maarten","dropping-particle":"van"}],"accessed":{"date-parts":[["2020",5,31]]},"issued":{"date-parts":[["2020",4,7]]}}},{"id":5237,"uris":[""],"uri":[""],"itemData":{"id":5237,"type":"article-journal","abstract":"<h3>Importance</h3><p>Early identification of patients with novel corona virus disease 2019 (COVID-19) who may develop critical illness is of great importance and may aid in delivering proper treatment and optimizing use of resources.</p><h3>Objective</h3><p>To develop and validate a clinical score at hospital admission for predicting which patients with COVID-19 will develop critical illness based on a nationwide cohort in China.</p><h3>Design, Setting, and Participants</h3><p>Collaborating with the National Health Commission of China, we established a retrospective cohort of patients with COVID-19 from 575 hospitals in 31 provincial administrative regions as of January 31, 2020. Epidemiological, clinical, laboratory, and imaging variables ascertained at hospital admission were screened using Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression to construct a predictive risk score (COVID-GRAM). The score provides an estimate of the risk that a hospitalized patient with COVID-19 will develop critical illness. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC). Data from 4 additional cohorts in China hospitalized with COVID-19 were used to validate the score. Data were analyzed between February 20, 2020 and March 17, 2020.</p><h3>Main Outcomes and Measures</h3><p>Among patients with COVID-19 admitted to the hospital, critical illness was defined as the composite measure of admission to the intensive care unit, invasive ventilation, or death.</p><h3>Results</h3><p>The development cohort included 1590 patients. the mean (SD) age of patients in the cohort was 48.9 (15.7) years; 904 (57.3%) were men. The validation cohort included 710 patients with a mean (SD) age of 48.2 (15.2) years, and 382 (53.8%) were men and 172 (24.2%). From 72 potential predictors, 10 variables were independent predictive factors and were included in the risk score: chest radiographic abnormality (OR, 3.39; 95% CI, 2.14-5.38), age (OR, 1.03; 95% CI, 1.01-1.05), hemoptysis (OR, 4.53; 95% CI, 1.36-15.15), dyspnea (OR, 1.88; 95% CI, 1.18-3.01), unconsciousness (OR, 4.71; 95% CI, 1.39-15.98), number of comorbidities (OR, 1.60; 95% CI, 1.27-2.00), cancer history (OR, 4.07; 95% CI, 1.23-13.43), neutrophil-to-lymphocyte ratio (OR, 1.06; 95% CI, 1.02-1.10), lactate dehydrogenase (OR, 1.002; 95% CI, 1.001-1.004) and direct bilirubin (OR, 1.15; 95% CI, 1.06-1.24). The mean AUC in the development cohort was 0.88 (95% CI, 0.85-0.91) and the AUC in the validation cohort was 0.88 (95% CI, 0.84-0.93). The score has been translated into an online risk calculator that is freely available to the public ()</p><h3>Conclusions and Relevance</h3><p>In this study, a risk score based on characteristics of COVID-19 patients at the time of admission to the hospital was developed that may help predict a patient’s risk of developing critical illness.</p>","container-title":"JAMA Internal Medicine","DOI":"10.1001/jamainternmed.2020.2033","journalAbbreviation":"JAMA Intern Med","language":"en","source":"","title":"Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19","URL":"","author":[{"family":"Liang","given":"Wenhua"},{"family":"Liang","given":"Hengrui"},{"family":"Ou","given":"Limin"},{"family":"Chen","given":"Binfeng"},{"family":"Chen","given":"Ailan"},{"family":"Li","given":"Caichen"},{"family":"Li","given":"Yimin"},{"family":"Guan","given":"Weijie"},{"family":"Sang","given":"Ling"},{"family":"Lu","given":"Jiatao"},{"family":"Xu","given":"Yuanda"},{"family":"Chen","given":"Guoqiang"},{"family":"Guo","given":"Haiyan"},{"family":"Guo","given":"Jun"},{"family":"Chen","given":"Zisheng"},{"family":"Zhao","given":"Yi"},{"family":"Li","given":"Shiyue"},{"family":"Zhang","given":"Nuofu"},{"family":"Zhong","given":"Nanshan"},{"family":"He","given":"Jianxing"}],"accessed":{"date-parts":[["2020",6,1]]},"issued":{"date-parts":[["2020",5,12]]}}}],"schema":""} 5,24 Furthermore, most existing covid-19 scores are complex in their calculation, requiring online calculators to run logistic regression models, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"X2YrH8gk","properties":{"formattedCitation":"\\super 24,37\\nosupersub{}","plainCitation":"24,37","noteIndex":0},"citationItems":[{"id":5237,"uris":[""],"uri":[""],"itemData":{"id":5237,"type":"article-journal","abstract":"<h3>Importance</h3><p>Early identification of patients with novel corona virus disease 2019 (COVID-19) who may develop critical illness is of great importance and may aid in delivering proper treatment and optimizing use of resources.</p><h3>Objective</h3><p>To develop and validate a clinical score at hospital admission for predicting which patients with COVID-19 will develop critical illness based on a nationwide cohort in China.</p><h3>Design, Setting, and Participants</h3><p>Collaborating with the National Health Commission of China, we established a retrospective cohort of patients with COVID-19 from 575 hospitals in 31 provincial administrative regions as of January 31, 2020. Epidemiological, clinical, laboratory, and imaging variables ascertained at hospital admission were screened using Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression to construct a predictive risk score (COVID-GRAM). The score provides an estimate of the risk that a hospitalized patient with COVID-19 will develop critical illness. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC). Data from 4 additional cohorts in China hospitalized with COVID-19 were used to validate the score. Data were analyzed between February 20, 2020 and March 17, 2020.</p><h3>Main Outcomes and Measures</h3><p>Among patients with COVID-19 admitted to the hospital, critical illness was defined as the composite measure of admission to the intensive care unit, invasive ventilation, or death.</p><h3>Results</h3><p>The development cohort included 1590 patients. the mean (SD) age of patients in the cohort was 48.9 (15.7) years; 904 (57.3%) were men. The validation cohort included 710 patients with a mean (SD) age of 48.2 (15.2) years, and 382 (53.8%) were men and 172 (24.2%). From 72 potential predictors, 10 variables were independent predictive factors and were included in the risk score: chest radiographic abnormality (OR, 3.39; 95% CI, 2.14-5.38), age (OR, 1.03; 95% CI, 1.01-1.05), hemoptysis (OR, 4.53; 95% CI, 1.36-15.15), dyspnea (OR, 1.88; 95% CI, 1.18-3.01), unconsciousness (OR, 4.71; 95% CI, 1.39-15.98), number of comorbidities (OR, 1.60; 95% CI, 1.27-2.00), cancer history (OR, 4.07; 95% CI, 1.23-13.43), neutrophil-to-lymphocyte ratio (OR, 1.06; 95% CI, 1.02-1.10), lactate dehydrogenase (OR, 1.002; 95% CI, 1.001-1.004) and direct bilirubin (OR, 1.15; 95% CI, 1.06-1.24). The mean AUC in the development cohort was 0.88 (95% CI, 0.85-0.91) and the AUC in the validation cohort was 0.88 (95% CI, 0.84-0.93). The score has been translated into an online risk calculator that is freely available to the public ()</p><h3>Conclusions and Relevance</h3><p>In this study, a risk score based on characteristics of COVID-19 patients at the time of admission to the hospital was developed that may help predict a patient’s risk of developing critical illness.</p>","container-title":"JAMA Internal Medicine","DOI":"10.1001/jamainternmed.2020.2033","journalAbbreviation":"JAMA Intern Med","language":"en","source":"","title":"Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19","URL":"","author":[{"family":"Liang","given":"Wenhua"},{"family":"Liang","given":"Hengrui"},{"family":"Ou","given":"Limin"},{"family":"Chen","given":"Binfeng"},{"family":"Chen","given":"Ailan"},{"family":"Li","given":"Caichen"},{"family":"Li","given":"Yimin"},{"family":"Guan","given":"Weijie"},{"family":"Sang","given":"Ling"},{"family":"Lu","given":"Jiatao"},{"family":"Xu","given":"Yuanda"},{"family":"Chen","given":"Guoqiang"},{"family":"Guo","given":"Haiyan"},{"family":"Guo","given":"Jun"},{"family":"Chen","given":"Zisheng"},{"family":"Zhao","given":"Yi"},{"family":"Li","given":"Shiyue"},{"family":"Zhang","given":"Nuofu"},{"family":"Zhong","given":"Nanshan"},{"family":"He","given":"Jianxing"}],"accessed":{"date-parts":[["2020",6,1]]},"issued":{"date-parts":[["2020",5,12]]}}},{"id":5256,"uris":[""],"uri":[""],"itemData":{"id":5256,"type":"article-journal","abstract":"<p>Background: COVID-19 pandemic has developed rapidly and the ability to stratify the most vulnerable patients is vital. However, routinely used severity scoring systems are often low on diagnosis, even in non-survivors. Therefore, clinical prediction models for mortality are urgently required. Methods: We developed and internally validated a multivariable logistic regression model to predict inpatient mortality in COVID-19 positive patients using data collected retrospectively from Tongji Hospital, Wuhan (299 patients). External validation was conducted using a retrospective cohort from Jinyintan Hospital, Wuhan (145 patients). Nine variables commonly measured in these acute settings were considered for model development, including age, biomarkers and comorbidities. Backwards stepwise selection and bootstrap resampling were used for model development and internal validation. We assessed discrimination via the C statistic, and calibration using calibration-in-the-large, calibration slopes and plots. Findings: The final model included age, lymphocyte count, lactate dehydrogenase and SpO2 as independent predictors of mortality. Discrimination of the model was excellent in both internal (c=0.89) and external (c=0.98) validation. Internal calibration was excellent (calibration slope=1). External validation showed some over-prediction of risk in low-risk individuals and under-prediction of risk in high-risk individuals prior to recalibration. Recalibration of the intercept and slope led to excellent performance of the model in independent data. Interpretation: COVID-19 is a new disease and behaves differently from common critical illnesses. This study provides a new prediction model to identify patients with lethal COVID-19. Its practical reliance on commonly available parameters should improve usage of limited healthcare resources and patient survival rate.</p>","container-title":"medRxiv","DOI":"10.1101/2020.03.28.20045997","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.03.28.20045997","source":"","title":"Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19","author":[{"family":"Xie","given":"Jianfeng"},{"family":"Hungerford","given":"Daniel"},{"family":"Chen","given":"Hui"},{"family":"Abrams","given":"Simon T."},{"family":"Li","given":"Shusheng"},{"family":"Wang","given":"Guozheng"},{"family":"Wang","given":"Yishan"},{"family":"Kang","given":"Hanyujie"},{"family":"Bonnett","given":"Laura"},{"family":"Zheng","given":"Ruiqiang"},{"family":"Li","given":"Xuyan"},{"family":"Tong","given":"Zhaohui"},{"family":"Du","given":"Bin"},{"family":"Qiu","given":"Haibo"},{"family":"Toh","given":"Cheng-Hock"}],"issued":{"date-parts":[["2020",4,7]]}}}],"schema":""} 24,37 or have high numbers of variable cut-offs. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"EcZ2aAXI","properties":{"formattedCitation":"\\super 23\\nosupersub{}","plainCitation":"23","noteIndex":0},"citationItems":[{"id":5234,"uris":[""],"uri":[""],"itemData":{"id":5234,"type":"article-journal","abstract":"<p>Abstract Importance COVID-19 is causing high mortality worldwide. Developing models to quantify the risk of poor outcomes in infected patients could help develop strategies to shield the most vulnerable during de-confinement. Objective To develop and externally validate COVID-19 Estimated Risk (COVER) scores that quantify a patient9s risk of hospital admission (COVER-H), requiring intensive services (COVER-I), or fatality (COVER-F) in the 30-days following COVID-19 diagnosis. Design Multinational, distributed network cohorts. Setting We analyzed a federated network of electronic medical records and administrative claims data from 13 data sources and 6 countries, mapped to a common data model. Participants Model development used a patient population consisting of &gt;2 million patients with a general practice (GP), emergency room (ER), or outpatient (OP) visit with diagnosed influenza or flu-like symptoms any time prior to 2020. The model was validated on patients with a GP, ER, or OP visit in 2020 with a confirmed or suspected COVID-19 diagnosis across four databases from South Korea, Spain and the United States. Outcomes Age, sex, historical conditions, and drug use prior to index date were considered as candidate predictors. Outcomes included i) hospitalization with pneumonia, ii) hospitalization with pneumonia requiring intensive services or death, and iii) death in the 30 days after index date. Results Overall, 43,061 COVID-19 patients were included for model validation, after initial model development and validation using 6,869,127 patients with influenza or flu-like symptoms. We identified 7 predictors (history of cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, hyperlipidemia, and kidney disease) which combined with age and sex could discriminate which patients would experience any of our three outcomes. The models achieved high performance in influenza. When transported to COVID-19 cohorts, the AUC ranges were, COVER-H: 0.73-0.81, COVER-I: 0.73-0.91, and COVER-F: 0.82-0.90. Calibration was overall acceptable, with overestimated risk in the most elderly and highest risk strata. Conclusions and relevance A 9-predictor model performs well for COVID-19 patients for predicting hospitalization, intensive services and death. The models could aid in providing reassurance for low risk patients and shield high risk patients from COVID-19 during de-confinement to reduce the virus9 impact on morbidity and mortality.</p>","container-title":"medRxiv","DOI":"10.1101/2020.05.26.20112649","language":"en","note":"publisher: Cold Spring Harbor Laboratory Press","page":"2020.05.26.20112649","source":"","title":"Seek COVER: Development and validation of a personalized risk calculator for COVID-19 outcomes in an international network","title-short":"Seek COVER","author":[{"family":"Williams","given":"Ross D."},{"family":"Markus","given":"Aniek F."},{"family":"Yang","given":"Cynthia"},{"family":"Salles","given":"Talita Duarte"},{"family":"Falconer","given":"Thomas"},{"family":"Jonnagaddala","given":"Jitendra"},{"family":"Kim","given":"Chungsoo"},{"family":"Rho","given":"Yeunsook"},{"family":"Williams","given":"Andrew"},{"family":"An","given":"Min Ho"},{"family":"Aragón","given":"María"},{"family":"Areia","given":"Carlos"},{"family":"Burn","given":"Edward"},{"family":"Choi","given":"Young"},{"family":"Drakos","given":"Iannis"},{"family":"Abrah?o","given":"Maria Fernandes"},{"family":"Fernández-Bertolín","given":"Sergio"},{"family":"Hripcsak","given":"George"},{"family":"Kaas-Hansen","given":"Benjamin"},{"family":"Kandukuri","given":"Prasanna"},{"family":"Kors","given":"Jan A."},{"family":"Kostka","given":"Kristin"},{"family":"Liaw","given":"Siaw-Teng"},{"family":"Machnicki","given":"Gerardo"},{"family":"Morales","given":"Daniel"},{"family":"Nyberg","given":"Fredrik"},{"family":"Park","given":"Rae Woong"},{"family":"Prats-Uribe","given":"Albert"},{"family":"Pratt","given":"Nicole"},{"family":"Rao","given":"Gowtham"},{"family":"Reich","given":"Christian G."},{"family":"Rivera","given":"Marcela"},{"family":"Seinen","given":"Tom"},{"family":"Shoaibi","given":"Azza"},{"family":"Spotnitz","given":"Matthew E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Suchard","given":"Marc A."},{"family":"You","given":"Seng Chan"},{"family":"Zhang","given":"Lin"},{"family":"Zhou","given":"Lili"},{"family":"Ryan","given":"Patrick B."},{"family":"Prieto-Alhambra","given":"Daniel"},{"family":"Reps","given":"Jenna M."},{"family":"Rijnbeek","given":"Peter R."}],"issued":{"date-parts":[["2020",5,29]]}}}],"schema":""} 23This ISARIC CCP-UK study represents the largest prospectively collected covid-19 hospitalised patient cohort in the world and will represent clinical data available in most economically developed healthcare settings. We developed a clinically applicable score with clear methodology and tested it against existing risk stratification scores in a large hospitalised patient cohort. With good to excellent discrimination, calibration and performance characteristics, it compared favourably to other prognostic tools. The key aim of clinical risk stratification scores is to support clinical management decisions. Three risk classes were identified and demonstrated similar adverse outcome rates across the validation cohort. Patients with a 4C score falling within the low-risk groups (mortality rate <2%) might be suitable for management as hospital outpatients, while those within the intermediate-risk group were at lower risk of mortality (~10%; 30.3% of the cohort) and may be suitable for ward-level monitoring. Similar mortality rates have been identified as an appropriate cut-off in pneumonia risk stratification scores (CURB-65 and PSI). ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"rB1MDNcK","properties":{"formattedCitation":"\\super 16,17\\nosupersub{}","plainCitation":"16,17","noteIndex":0},"citationItems":[{"id":5300,"uris":[""],"uri":[""],"itemData":{"id":5300,"type":"article-journal","abstract":"Background: In the assessment of severity in community acquired pneumonia (CAP), the modified British Thoracic Society (mBTS) rule identifies patients with severe pneumonia but not patients who might be suitable for home management. A multicentre study was conducted to derive and validate a practical severity assessment model for stratifying adults hospitalised with CAP into different management groups.\nMethods: Data from three prospective studies of CAP conducted in the UK, New Zealand, and the Netherlands were combined. A derivation cohort comprising 80% of the data was used to develop the model. Prognostic variables were identified using multiple logistic regression with 30 day mortality as the outcome measure. The final model was tested against the validation cohort.\nResults: 1068 patients were studied (mean age 64 years, 51.5% male, 30 day mortality 9%). Age ?65 years (OR 3.5, 95% CI 1.6 to 8.0) and albumin <30 g/dl (OR 4.7, 95% CI 2.5 to 8.7) were independently associated with mortality over and above the mBTS rule (OR 5.2, 95% CI 2.7 to 10). A six point score, one point for each of Confusion, Urea >7 mmol/l, Respiratory rate ?30/min, low systolic(<90 mm Hg) or diastolic (?60 mm Hg) Blood pressure), age ?65 years (CURB-65 score) based on information available at initial hospital assessment, enabled patients to be stratified according to increasing risk of mortality: score 0, 0.7%; score 1, 3.2%; score 2, 3%; score 3, 17%; score 4, 41.5% and score 5, 57%. The validation cohort confirmed a similar pattern.\nConclusions: A simple six point score based on confusion, urea, respiratory rate, blood pressure, and age can be used to stratify patients with CAP into different management groups.","container-title":"Thorax","DOI":"10.1136/thorax.58.5.377","ISSN":"0040-6376, 1468-3296","issue":"5","language":"en","note":"publisher: BMJ Publishing Group Ltd\nsection: Respiratory infection\nPMID: 12728155","page":"377-382","source":"thorax.","title":"Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study","title-short":"Defining community acquired pneumonia severity on presentation to hospital","volume":"58","author":[{"family":"Lim","given":"W. S."},{"family":"Eerden","given":"M. M.","dropping-particle":"van der"},{"family":"Laing","given":"R."},{"family":"Boersma","given":"W. G."},{"family":"Karalus","given":"N."},{"family":"Town","given":"G. I."},{"family":"Lewis","given":"S. A."},{"family":"Macfarlane","given":"J. T."}],"issued":{"date-parts":[["2003",5,1]]}}},{"id":5229,"uris":[""],"uri":[""],"itemData":{"id":5229,"type":"article-journal","abstract":"BACKGROUND: There is considerable variability in rates of hospitalization of patients with community-acquired pneumonia, in part because of physicians' uncertainty in assessing the severity of illness at presentation.\nMETHODS: From our analysis of data on 14,199 adult inpatients with community-acquired pneumonia, we derived a prediction rule that stratifies patients into five classes with respect to the risk of death within 30 days. The rule was validated with 1991 data on 38,039 inpatients and with data on 2287 inpatients and outpatients in the Pneumonia Patient Outcomes Research Team (PORT) cohort study. The prediction rule assigns points based on age and the presence of coexisting disease, abnormal physical findings (such as a respiratory rate of > or = 30 or a temperature of > or = 40 degrees C), and abnormal laboratory findings (such as a pH <7.35, a blood urea nitrogen concentration > or = 30 mg per deciliter [11 mmol per liter] or a sodium concentration <130 mmol per liter) at presentation.\nRESULTS: There were no significant differences in mortality in each of the five risk classes among the three cohorts. Mortality ranged from 0.1 to 0.4 percent for class I patients (P=0.22), from 0.6 to 0.7 percent for class II (P=0.67), and from 0.9 to 2.8 percent for class III (P=0.12). Among the 1575 patients in the three lowest risk classes in the Pneumonia PORT cohort, there were only seven deaths, of which only four were pneumonia-related. The risk class was significantly associated with the risk of subsequent hospitalization among those treated as outpatients and with the use of intensive care and the number of days in the hospital among inpatients.\nCONCLUSIONS: The prediction rule we describe accurately identifies the patients with community-acquired pneumonia who are at low risk for death and other adverse outcomes. This prediction rule may help physicians make more rational decisions about hospitalization for patients with pneumonia.","container-title":"The New England Journal of Medicine","DOI":"10.1056/NEJM199701233360402","ISSN":"0028-4793","issue":"4","journalAbbreviation":"N. Engl. J. Med.","language":"eng","note":"PMID: 8995086","page":"243-250","source":"PubMed","title":"A prediction rule to identify low-risk patients with community-acquired pneumonia","volume":"336","author":[{"family":"Fine","given":"M. J."},{"family":"Auble","given":"T. E."},{"family":"Yealy","given":"D. M."},{"family":"Hanusa","given":"B. H."},{"family":"Weissfeld","given":"L. A."},{"family":"Singer","given":"D. E."},{"family":"Coley","given":"C. M."},{"family":"Marrie","given":"T. J."},{"family":"Kapoor","given":"W. N."}],"issued":{"date-parts":[["1997",1,23]]}}}],"schema":""} 16,17 Meanwhile patients with a score ≥9 were at high risk of death (45.5%), which may prompt aggressive treatment, including the commencement of steroids, ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"GmL9X966","properties":{"formattedCitation":"\\super 45\\nosupersub{}","plainCitation":"45","noteIndex":0},"citationItems":[{"id":5282,"uris":[""],"uri":[""],"itemData":{"id":5282,"type":"webpage","title":"Welcome — RECOVERY Trial","URL":"","accessed":{"date-parts":[["2020",6,23]]}}}],"schema":""} 45 and early escalation to critical care if appropriate.Our study has some limitations. First, we were unable to evaluate the predictive performance of a number of existing scores that comprise a large number of parameters (for example APACHE II ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"HSPmRlSO","properties":{"formattedCitation":"\\super 46\\nosupersub{}","plainCitation":"46","noteIndex":0},"citationItems":[{"id":5272,"uris":[""],"uri":[""],"itemData":{"id":5272,"type":"article-journal","abstract":"This paper presents the form and validation results of APACHE II, a severity of disease classification system. APACHE II uses a point score based upon initial values of 12 routine physiologic measurements, age, and previous health status to provide a general measure of severity of disease. An increasing score (range 0 to 71) was closely correlated with the subsequent risk of hospital death for 5815 intensive care admissions from 13 hospitals. This relationship was also found for many common diseases. When APACHE II scores are combined with an accurate description of disease, they can prognostically stratify acutely ill patients and assist investigators comparing the success of new or differing forms of therapy. This scoring index can be used to evaluate the use of hospital resources and compare the efficacy of intensive care in different hospitals or over time.","container-title":"Critical Care Medicine","ISSN":"0090-3493","issue":"10","journalAbbreviation":"Crit. Care Med.","language":"eng","note":"PMID: 3928249","page":"818-829","source":"PubMed","title":"APACHE II: a severity of disease classification system","title-short":"APACHE II","volume":"13","author":[{"family":"Knaus","given":"W. A."},{"family":"Draper","given":"E. A."},{"family":"Wagner","given":"D. P."},{"family":"Zimmerman","given":"J. E."}],"issued":{"date-parts":[["1985",10]]}}}],"schema":""} 46), as well as several other covid-19 prognostic scores that include computed tomography findings or uncommonly measured biomarkers. ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"O2FXSKzL","properties":{"formattedCitation":"\\super 5\\nosupersub{}","plainCitation":"5","noteIndex":0},"citationItems":[{"id":5193,"uris":[""],"uri":[""],"itemData":{"id":5193,"type":"article-journal","abstract":"Objective To review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.\nDesign Rapid systematic review and critical appraisal.\nData sources PubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.\nStudy selection Studies that developed or validated a multivariable covid-19 related prediction model.\nData extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).\nResults 2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.\nConclusion Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.\nSystematic review registration Protocol , registration .","container-title":"BMJ","DOI":"10.1136/bmj.m1328","ISSN":"1756-1833","journalAbbreviation":"BMJ","language":"en","note":"publisher: British Medical Journal Publishing Group\nsection: Research\nPMID: 32265220","source":"","title":"Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal","title-short":"Prediction models for diagnosis and prognosis of covid-19 infection","URL":"","volume":"369","author":[{"family":"Wynants","given":"Laure"},{"family":"Calster","given":"Ben Van"},{"family":"Bonten","given":"Marc M. J."},{"family":"Collins","given":"Gary S."},{"family":"Debray","given":"Thomas P. A."},{"family":"Vos","given":"Maarten De"},{"family":"Haller","given":"Maria C."},{"family":"Heinze","given":"Georg"},{"family":"Moons","given":"Karel G. M."},{"family":"Riley","given":"Richard D."},{"family":"Schuit","given":"Ewoud"},{"family":"Smits","given":"Luc J. M."},{"family":"Snell","given":"Kym I. E."},{"family":"Steyerberg","given":"Ewout W."},{"family":"Wallisch","given":"Christine"},{"family":"Smeden","given":"Maarten","dropping-particle":"van"}],"accessed":{"date-parts":[["2020",5,31]]},"issued":{"date-parts":[["2020",4,7]]}}}],"schema":""} 5 Second, a substantial proportion of recruited patients had incomplete episodes and were thus excluded from the analysis. Selection bias is possible if patients with incomplete episodes, such as those with prolonged hospital admission, had a differential mortality risk to those with completed episodes. Nevertheless, the size of our patient cohort compares favourably to other datasets for model creation. Furthermore, the patient cohort on which the 4C score was derived comprised hospitalised patients who were seriously ill (mortality rate of 30.5%) and were of advanced age (median age 74 years). Further external validation is required to determine whether the 4C score is generalisable in community or younger patient cohorts. We have derived and validated an easy-to-use eight-variable risk stratification score that enables accurate stratification of hospitalised covid-19 patients by mortality risk at hospital presentation. Application within the validation cohorts demonstrated this tool may guide clinician decisions, including treatment escalation. Further evaluation in different clinical and geographical settings is required to determine the performance of the 4C score in lower-risk patient cohorts.Data sharing We welcome applications for data and material access through our Independent Data and Material Access Committee ().Funding Statement for all ISARIC4C outputs This work is supported by grants from: the National Institute for Health Research [award CO-CIN-01], the Medical Research Council [grant MC_PC_19059] and by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emerging and Zoonotic Infections at University of Liverpool in partnership with Public Health England (PHE), in collaboration with Liverpool School of Tropical Medicine and the University of Oxford [NIHR award 200907], Wellcome Trust and Department for International Development [215091/Z/18/Z], and the Bill and Melinda Gates Foundation [OPP1209135], and Liverpool Experimental Cancer Medicine Centre for providing infrastructure support for this research (Grant Reference: C18616/A25153). The views expressed are those of the authors and not necessarily those of the DHSC, DID, NIHR, MRC, Wellcome Trust or PHE.Ethical considerationsEthical approval was given by the South Central - Oxford C Research Ethics Committee in England (Ref 13/SC/0149), the Scotland A Research Ethics Committee (Ref 20/SS/0028), and the WHO Ethics Review Committee (RPC571 and RPC572, 25 April 2013Data availabilityThis work uses data provided by patients and collected by the NHS as part of their care and support #DataSavesLives. The CO-CIN data was collated by ISARIC4C Investigators. ISARIC4C welcomes applications for data and material access through our Independent Data and Material Access Committee ().ISARIC 4C InvestigatorsConsortium Lead Investigator: J Kenneth Baillie, Chief Investigator Malcolm G SempleCo-Lead Investigator Peter JM Openshaw. ISARIC Clinical Coordinator Gail Carson. Co-Investigators: Beatrice Alex, Benjamin Bach, Wendy S Barclay, Debby Bogaert, Meera Chand, Graham S Cooke, Annemarie B Docherty, Jake Dunning, Ana da Silva Filipe, Tom Fletcher, Christopher A Green, Ewen M Harrison, Julian A Hiscox, Antonia Ying Wai Ho, Peter W Horby, Samreen Ijaz, Saye Khoo, Paul Klenerman, Andrew Law, Wei Shen Lim, Alexander, J Mentzer, Laura Merson, Alison M Meynert, Mahdad Noursadeghi, Shona C Moore, Massimo Palmarini, William A Paxton, Georgios Pollakis, Nicholas Price, Andrew Rambaut, David L Robertson, Clark D Russell, Vanessa Sancho-Shimizu, Janet T Scott, Louise Sigfrid, Tom Solomon, Shiranee Sriskandan, David Stuart, Charlotte Summers, Richard S Tedder, Emma C Thomson, Ryan S Thwaites, Lance CW Turtle, Maria Zambon. Project Managers Hayley Hardwick, Chloe Donohue, Jane Ewins, Wilna Oosthuyzen, Fiona Griffiths. Data Analysts: Lisa Norman, Riinu Pius, Tom M Drake, Cameron J Fairfield, Stephen Knight, Kenneth A Mclean, Derek Murphy, Catherine A Shaw. Data and Information System Manager: Jo Dalton, Michelle Girvan, Egle Saviciute, Stephanie Roberts Janet Harrison, Laura Marsh, Marie Connor. Data integration and presentation: Gary Leeming, Andrew Law, Ross Hendry. Material Management: William Greenhalf, Victoria Shaw, Sarah McDonald. Outbreak Laboratory Volunteers: Katie A. Ahmed, Jane A Armstrong, Milton Ashworth, Innocent G Asiimwe, Siddharth Bakshi, Samantha L Barlow, Laura Booth, Benjamin Brennan, Katie Bullock, Benjamin WA Catterall, Jordan J Clark, Emily A Clarke, Sarah Cole, Louise Cooper, Helen Cox, Christopher Davis, Oslem Dincarslan, Chris Dunn, Philip Dyer, Angela Elliott, Anthony Evans, Lewis WS Fisher, Terry Foster, Isabel Garcia-Dorival, Willliam Greenhalf, Philip Gunning, Catherine Hartley, Antonia Ho, Rebecca L Jensen, Christopher B Jones, Trevor R Jones, Shadia Khandaker, Katharine King, Robyn T. Kiy, Chrysa Koukorava, Annette Lake, Suzannah Lant, Diane Latawiec, L Lavelle-Langham, Daniella Lefteri, Lauren Lett, Lucia A Livoti, Maria Mancini, Sarah McDonald, Laurence McEvoy, John McLauchlan, Soeren Metelmann, Nahida S Miah, Joanna Middleton, Joyce Mitchell, Shona C Moore, Ellen G Murphy, Rebekah Penrice-Randal, Jack Pilgrim, Tessa Prince, Will Reynolds, P. Matthew Ridley, Debby Sales, Victoria E Shaw, Rebecca K Shears, Benjamin Small, Krishanthi S Subramaniam, Agnieska Szemiel, Aislynn Taggart, Jolanta Tanianis, Jordan Thomas, Erwan Trochu, Libby van Tonder, Eve Wilcock, J. Eunice Zhang. Local Principal Investigators: Kayode Adeniji, Daniel Agranoff, Ken Agwuh, Dhiraj Ail, Ana Alegria, Brian Angus, Abdul Ashish, Dougal Atkinson, Shahedal Bari, Gavin Barlow, Stella Barnass, Nicholas Barrett, Christopher Bassford, David Baxter, Michael Beadsworth, Jolanta Bernatoniene, John Berridge , Nicola Best , Pieter Bothma, David Brealey, Robin Brittain-Long, Naomi Bulteel, Tom Burden , Andrew Burtenshaw, Vikki Caruth, David Chadwick, Duncan Chambler, Nigel Chee, Jenny Child, Srikanth Chukkambotla, Tom Clark, Paul Collini, Catherine Cosgrove, Jason Cupitt, Maria-Teresa Cutino-Moguel, Paul Dark, Chris Dawson, Samir Dervisevic, Phil Donnison, Sam Douthwaite, Ingrid DuRand, Ahilanadan Dushianthan, Tristan Dyer, Cariad Evans , Chi Eziefula, Chrisopher Fegan, Adam Finn, Duncan Fullerton, Sanjeev Garg, Sanjeev Garg, Atul Garg, Jo Godden, Arthur Goldsmith, Clive Graham, Elaine Hardy, Stuart Hartshorn, Daniel Harvey, Peter Havalda, Daniel B Hawcutt, Maria Hobrok, Luke Hodgson, Anita Holme, Anil Hormis, Michael Jacobs, Susan Jain, Paul Jennings, Agilan Kaliappan, Vidya Kasipandian, Stephen Kegg, Michael Kelsey, Jason Kendall, Caroline Kerrison, Ian Kerslake, Oliver Koch, Gouri Koduri, George Koshy , Shondipon Laha, Susan Larkin, Tamas Leiner, Patrick Lillie, James Limb, Vanessa Linnett, Jeff Little, Michael MacMahon, Emily MacNaughton, Ravish Mankregod, Huw Masson , Elijah Matovu, Katherine McCullough, Ruth McEwen , Manjula Meda, Gary Mills , Jane Minton, Mariyam Mirfenderesky, Kavya Mohandas, Quen Mok, James Moon, Elinoor Moore, Patrick Morgan, Craig Morris, Katherine Mortimore, Samuel Moses, Mbiye Mpenge, Rohinton Mulla, Michael Murphy, Megan Nagel, Thapas Nagarajan, Mark Nelson, Igor Otahal, Mark Pais, Selva Panchatsharam, Hassan Paraiso, Brij Patel, Justin Pepperell, Mark Peters, Mandeep Phull , Stefania Pintus, Jagtur Singh Pooni, Frank Post, David Price, Rachel Prout, Nikolas Rae, Henrik Reschreiter, Tim Reynolds, Neil Richardson, Mark Roberts, Devender Roberts, Alistair Rose, Guy Rousseau, Brendan Ryan, Taranprit Saluja, Aarti Shah, Prad Shanmuga, Anil Sharma, Anna Shawcross, Jeremy Sizer, Richard Smith, Catherine Snelson, Nick Spittle, Nikki Staines , Tom Stambach, Richard Stewart, Pradeep Subudhi, Tamas Szakmany, Kate Tatham, Jo Thomas, Chris Thompson, Robert Thompson, Ascanio Tridente, Darell Tupper - Carey, Mary Twagira, Andrew Ustianowski, Nick Vallotton, Lisa Vincent-Smith, Shico Visuvanathan , Alan Vuylsteke, Sam Waddy, Rachel Wake, Andrew Walden, Ingeborg Welters, Tony Whitehouse, Paul Whittaker, Ashley Whittington, Meme Wijesinghe, Martin Williams, Lawrence Wilson, Sarah Wilson, Stephen Winchester, Martin Wiselka, Adam Wolverson, Daniel G Wooton, Andrew Workman, Bryan Yates, Peter Young.Acknowledgements This work uses data provided by patients and collected by the NHS as part of their care and support #DataSavesLives. We are extremely grateful to the 2,648 frontline NHS clinical and research staff and volunteer medical students, who collected this data in challenging circumstances; and the generosity of the participants and their families for their individual contributions in these difficult times. We also acknowledge the support of Jeremy J Farrar, Nahoko Shindo, Devika Dixit, Nipunie Rajapakse, Piero Olliaro, Lyndsey Castle, Martha Buckley, Debbie Malden, Katherine Newell, Kwame O’Neill, Emmanuelle Denis, Claire Petersen, Scott Mullaney, Sue MacFarlane, Chris Jones, Nicole Maziere, Katie Bullock, Emily Cass, William Reynolds, Milton Ashworth, Ben Catterall, Louise Cooper, Terry Foster, Paul Matthew Ridley, Anthony Evans, Catherine Hartley, Chris Dunn, Debby Sales, Diane Latawiec, Erwan Trochu, Eve Wilcock, Innocent Gerald Asiimwe, Isabel Garcia-Dorival, J. Eunice Zhang, Jack Pilgrim, Jane A Armstrong, Jordan J. Clark, Jordan Thomas, Katharine King, Katie Alexandra Ahmed, Krishanthi S Subramaniam , Lauren Lett, Laurence McEvoy, Libby van Tonder, Lucia Alicia Livoti, Nahida S Miah, Rebecca K. Shears, Rebecca Louise Jensen, Rebekah Penrice-Randal, Robyn Kiy, Samantha Leanne Barlow, Shadia Khandaker, Soeren Metelmann, Tessa Prince, Trevor R Jones, Benjamin Brennan, Agnieska Szemiel, Siddharth Bakshi, Daniella Lefteri, Maria Mancini, Julien Martinez, Angela Elliott, Joyce Mitchell, John McLauchlan, Aislynn Taggart, Oslem Dincarslan, Annette Lake, Claire Petersen, and Scott Mullaney.References ADDIN ZOTERO_BIBL {"uncited":[],"omitted":[],"custom":[]} CSL_BIBLIOGRAPHY 1. Yang X, Yu Y, Xu J, Shu H, Xia J, Liu H, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med. 2020;8(5):475–81. 2. COVID-19 Map [Internet]. Johns Hopkins Coronavirus Resource Center. [cited 2020 July 24]. Available from: . COVID-19 situation reports [Internet]. [cited 2020 Jun 1]. Available from: . Tolksdorf K, Buda S, Schuler E, Wieler LH, Haas W. Influenza-associated pneumonia as reference to assess seriousness of coronavirus disease (COVID-19). Eurosurveillance. 2020 Mar;25(11):2000258. 5. Wynants L, Calster BV, Bonten MMJ, Collins GS, Debray TPA, Vos MD, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ 2020; 369:m13286. Chen J-H, Chang S-S, Liu JJ, Chan R-C, Wu J-Y, Wang W-C, et al. Comparison of clinical characteristics and performance of pneumonia severity score and CURB-65 among younger adults, elderly and very old subjects. Thorax. 2010;65(11):971–7. 7. Docherty AB, Harrison EM, Green CA, Hardwick HE, Pius R, Norman L, et al. Features of 20?133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ 2020; 369:m19858. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594. 9. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. 10. Sattar N, McInnes IB, McMurray JJV. Obesity Is a Risk Factor for Severe COVID-19 Infection: Multiple Potential Mechanisms. Circulation. 2020;142(1):4–6. 11. Simonnet A, Chetboun M, Poissy J, Raverdy V, Noulette J, Duhamel A, et al. High Prevalence of Obesity in Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) Requiring Invasive Mechanical Ventilation. Obesity. 2020;28(7):1195–9. 12. Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162(1):W1-73. 13. Guan W, Liang W, Zhao Y, Liang H, Chen Z, Li Y, et al. Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis. European Respiratory Journal. 2020;55(5): 2000547.14. Mandrekar JN. Receiver Operating Characteristic Curve in Diagnostic Test Assessment. Journal of Thoracic Oncology. 2010;5(9):1315–6. 15. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology. 2010;21(1):128–38. 16. Lim WS, Eerden MM van der, Laing R, Boersma WG, Karalus N, Town GI, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377–82. 17. Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336(4):243–50. 18. Rubin DB, editor. Multiple Imputation for Nonresponse in Surveys [Internet]. Hoboken, NJ, USA: John Wiley & Sons, Inc.; 1987. doi: . Hacking JM, Muller S, Buchan IE. Trends in mortality from 1965 to 2008 across the English north-south divide: comparative observational study. BMJ 2011: 342: d508.20. Buchan IE, Kontopantelis E, Sperrin M, Chandola T, Doran T. North-South disparities in English mortality1965–2015: longitudinal population study. J Epidemiol Community Health. 2017;71(9):928–36. 21. Deaths involving COVID-19 by local area and socioeconomic deprivation - Office for National Statistics [Internet]. [cited 2020 Jul 1]. Available from: . Caramelo F, Ferreira N, Oliveiros B. Estimation of risk factors for COVID-19 mortality - preliminary results. medRxiv. 2020 Feb 25;2020. doi:? 23. Williams RD, Markus AF, Yang C, Salles TD, Falconer T, Jonnagaddala J, et al. Seek COVER: Development and validation of a personalized risk calculator for COVID-19 outcomes in an international network. medRxiv. 2020. doi: 24. Liang W, Liang H, Ou L, Chen B, Chen A, Li C, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Intern Med 2020; May 12: e20203325. Liu Y, Mao B, Liang S, Yang J, Lu H, Chai Y, et al. Association Between Ages and Clinical Characteristics and Outcomes of Coronavirus Disease 2019. European Respiratory Journal 2020; 55(5): 200111226. Luo X, Zhou W, Yan X, Guo T, Wang B, Xia H, et al. Prognostic value of C-reactive protein in patients with COVID-19. Clin Infect Dis 2020; doi: https//10.1093/cid/ciaa641 027. Miyashita N, Matsushima T, Oka M. The JRS Guidelines for the Management of Community-acquired Pneumonia in Adults:An Update and New Recommendations. Internal Medicine. 2006;45(7):419–28. 28. Bauer TT, Ewig S, Marre R, Suttorp N, Welte T, CAPNETZ Study Group. CRB-65 predicts death from community-acquired pneumonia. J Intern Med. 2006;260(1):93–101. 29. Chen J-H, Chang S-S, Liu J, Chan R-C, Wu J-Y, Wang W-C, et al. Comparison of Clinical Characteristics and Performance of Pneumonia Severity Score and CURB-65 Among Younger Adults, Elderly and Very Old Subjects. Thorax 2010; 65(11); 971-7.30. Dwyer R, Hedlund J, Henriques-Normark B, Kalin M. Improvement of CRB-65 as a prognostic tool in adult patients with community-acquired pneumonia. BMJ Open Respir Res. 2014;1(1):e000038. 31. Liu J, Xu F, Hui Zhou, Wu X, Shi L, Lu R, et al. Expanded CURB-65: a new score system predicts severity of community-acquired pneumonia with superior efficiency. Sci Rep. 2016;6:22911. 32. National Early Warning Score (NEWS) 2 [Internet]. RCP London. 2017 [cited 2020 Jul 1]. Available from: . Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016: 3;315(8):801–10. 34. Charles PGP, Wolfe R, Whitby M, Fine MJ, Fuller AJ, Stirling R, et al. SMART-COP: a tool for predicting the need for intensive respiratory or vasopressor support in community-acquired pneumonia. Clin Infect Dis. 2008;47(3):375–84. 35. Vincent JL, Moreno R, Takala J, Willatts S, De Mendon?a A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–10. 36. Yandiola PPE, Capelastegui A, Quintana J, Diez R, Gorordo I, Bilbao A, et al. Prospective comparison of severity scores for predicting clinically relevant outcomes for patients hospitalized with community-acquired pneumonia. Chest. 2009;135(6):1572–9. 37. Xie J, Hungerford D, Chen H, Abrams ST, Li S, Wang G, et al. Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19. medRxiv. 2020. doi: . Surgisphere - Mortality Risk Calculator [Internet]. [cited 2020 Jun 1]. Available from: . Zhang H, Shi T, Wu X, Zhang Z, Wang K, Bean D, et al. Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK | medRxiv [Internet]. [cited 2020 Jul 23]. doi: . Pavlou M, Ambler G, Seaman SR, Guttmann O, Elliott P, King M, et al. How to develop a more accurate risk prediction model when there are few events. BMJ 2015; 351: h3868.41. Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995;48(12):1503–10. 42. Song Y-M, Sung J, Cho H-J. Reduction and Cessation of Cigarette Smoking and Risk of Cancer: A Cohort Study of Korean Men. Journal of Clinical Oncology. 2008;26(31):5101–6. 43. Qi X, Jiang Z, Yu Q, Shao C, Zhang H, Yue H, et al. Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: A multicenter study. medRxiv. 2020. . Yuan M, Yin W, Tao Z, Tan W, Hu Y. Association of radiologic findings with mortality of patients infected with 2019 novel coronavirus in Wuhan, China. PLoS ONE. 2020;15(3):e0230548. 45. Welcome — RECOVERY Trial [Internet]. [cited 2020 Jun 23]. Available from: . Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29. Appendix 1. ISARIC CCP-UK risk stratification score derivation and validation protocolBackgroundPatients hospitalised with covid-19 are at high risk of mortality. Stratification of patients on admission may aid clinicians in determining immediate management decisions (home discharge, ward-level care, escalation to ICU) and medical treatment. High risk of bias exists in novel covid-19 risk stratification tools, with small cohorts in limited geographical areas and potential for over-fitting. Many of these scores are also complex to calculate and limit clinical utility.AimsDevelop risk stratification score using the largest known hospitalised cohort of covid-19 patientsThe risk stratification score must have high clinical utility (defined here as having the ability to be calculated without a complex equation/algorithm)Following derivation, determine discriminatory performance in validation cohorts and compare to existing risk stratification tools (for pneumonia, influenza and covid-19) Primary outcomeIn-hospital mortalityPatient inclusionAll adult patients (≥18 years old on admission)Index admission (readmission episode excluded)Completed index admissionPotential candidate variablesIdentificationThe systematic literature search (see below) will identify potential predictor variables for mortality, disease severity and/or critical care requirement in pneumonia, influenza or covid-19 patients.Inclusion criteriaReadily available patient or clinical characteristic to attending clinicians upon presentation to hospital (Accident & Emergency department, Acute Medical Receiving Unit)Blood markers should be commonly measured and results available for review within the first 24 hours of admissionMeasured within the ISARIC CCP-UK database (pre-specified case report form)Exclusion criteriaHigh number of missing values (>1/3rd of patients) within the derivation cohortScore developmentAll patients within the database on 20th May 2020 will be included within the derivation cohort.With the overall aim to derive a risk stratification score with high clinical utility, a decision a priori has been made to categorise final included predictor variables for ease of calculation in a clinical environment. However, to avoid loss of information through categorisation, generalised additive models (GAMs) will be used first to identify final predictor variables prior to categorisation.All remaining candidate predictor variables following application of exclusion criteria (availability in database, missingness) will be entered into a GAMs. These variables will then be removed individually and GAMs run again, determining the explained deviance and unbiased risk estimator (UBRE; essentially a scaled AIC) following exclusion of each individual variable. Final variables to include within the risk stratification score will then be selected by explained deviance, R2 and UBRE.GAMs curves for each continuous variable will then be created for each final included variable and cut-offs determined based on outcome risk. Once categorised, the final variables will be placed in a least absolute shrinkage and selection operator (LASSO) to ensure all final variables should be selected within the risk score and to reduce the risk of over-fitting. Shrunk coefficients will be converted to produce an index score.In parallel, a machine learning (ML) model will be derived using extreme gradient boosted trees methodology (XGBoost), representing a ‘best-in-class’ model. This will include all candidate predictor variables included within the GAM model.Statistical analysisModel performance will be determined using the AUROC, with calibration and Brier score calculated for each final model (derived risk score and ML model). To determine the impact of missingness, multivariate imputation by chained equations (MICE) will be performed for all candidate predictor variables (except those with high levels of missingness). It will be assumed that variables are missing at random and the primary outcome will be imputed. Ten sets with ten iterations will be performed. Model performance will again be determined as detailed above, with Rubin’s Rules used to combine model parameter estimates.All statistical analysis will use the R (v3.6.3).ValidationAll patients entered into the database after the specified derivation cohort cut-off will be included, with the same patient inclusion/criteria applied.Exiting risk stratification tool identificationRisk stratification scores created and or validated for pneumonia, influenza and covid-19 will be included. These will be identified through the systematic literature search (see below) and do not have to have been peer-reviewed for inclusion.Only risk stratification scores with all predictor variables will be considered for inclusion. Decisions for inclusion of risk stratification score where one variable is missing will be made on a case-by-case basis by consensus within the study group. If the missing variable is deemed a key contributor to risk prediction within the tool it will be excluded. Statistical analysisDiscriminatory performance (AUROC) and other performance metrics (sensitivity, specificity, PPV and NPV) will be calculated for all included risk stratification tools and compared with the derived risk and ML model in each of the validation cohorts. Calibration and Brier score will also be determined for the derived risk score in each validation cohort.Systematic literature searchDatabasesEMBASE, WHO Medicus and Google Scholar (particularly for pre-print publications)Search termsPneumonia; sepsis; influenza; covid-19; SARS-CoV-2; coronavirus; Combined with: score and prognosis No language or date restrictionsAppendix 2. Candidate predictor variables evaluated for potential inclusion in modeling processEvidence and/or modelsInclusion / exclusionPatient demographicsAge on admission (years)CURB-65a, COVID-GRAMbIncludedSex at BirthA-DROPc, PSIdIncludedHypertensionComorbidity predicts clinical outcomes in covid-19eExcluded – not initially recorded within the ISARIC CCP-UK databaseChronic cardiac diseaseComorbidity predicts clinical outcomes in covid-19eCombined with other comorbidities for model developmentChronic kidney diseaseComorbidity predicts clinical outcomes in covid-19eCombined with other comorbidities for model developmentMalignant neoplasm Comorbidity predicts clinical outcomes in covid-19eCombined with other comorbidities for model developmentModerate or severe liver diseaseComorbidity predicts clinical outcomes in covid-19eCombined with other comorbidities for model developmentClinician-defined obesityComorbidity predicts clinical outcomes in covid-19eCombined with other comorbidities for model developmentChronic pulmonary disease (not asthma)Comorbidity predicts clinical outcomes in covid-19eCombined with other comorbidities for model developmentDiabetesComorbidity predicts clinical outcomes in covid-19eCombined with other comorbidities for model developmentNumber of comorbiditiesNumber of comorbidities predicts clinical outcomes in covid-19eIncluded – composite count of all included comorbidities defined by Charlson Comorbidity Index plus obesityClinical signs/ observationsRespiratory RateCURB65a, NEWS2fIncludedPeripheral oxygen saturations (%)Xie scoreg, ADROPcIncludedSystolic blood pressure (mmHg)CURB-65a, NEWS2fIncludedDiastolic blood pressure (mmHg)CURB-65aIncludedTemperature (°C)PSId, NEWS2fIncludedHeart Rate (bpm)NEWS2fIncludedGlasgow Coma ScoreCOVID-GRAMb, CURB-65aIncludedBedside investigationsFiO2NEWS2f, SOFAhExcluded – too many values missing from derivation datasetPaO2 (kPa)PSId, SCAPiExcluded – too many values missing from derivation datasetpHPSId, SCAPiExcluded – too many values missing from derivation datasetGlucose (mmol/L)PSIdExcluded – too many values missing from derivation datasetInfiltrates on chest radiographCOVID-GRAMb , PSId, SMART-COPjExcluded – too many values missing from derivation datasetLaboratory measuresHaemoglobin (g/L)Severe covid-19 known to lower haemoglobin concentrationkIncludedWhite cell count (109/L)COVID-GRAMbIncludedNeutrophil count (109/L)COVID-GRAMb, DL scorelIncludedLymphocyte count (109/L)COVID-GRAMbIncludedHaematocrit (%)PSIdExcluded – too many values missing from derivation datasetPlatelet Count (109/L)DL scorel, E-CURB65mIncludedProthrombin (seconds)Coagulopathy associated with mortality in covid-19 patientsnExcluded – too many values missing from derivation datasetActivated partial thromboplastin time (APTT) (seconds)Coagulopathy associated with mortality in covid-19 patientsnExcluded – too many values missing from derivation datasetSodium (mmol/L)PSIdIncludedTotal Bilirubin (mg/dL)COVID-GRAMb, SOFAhIncludedAlanine aminotransferase (ALT) (units/L)Abnormal liver tests associated with severe covid-19oExcluded – too many values missing from derivation datasetAspartate aminotransferase (AST) (units/L)Abnormal liver tests associated with severe covid-19oExcluded – too many values missing from derivation datasetAlbumin (g/L)Association between low albumin and severe covid-19pExcluded - not recorded within ISARIC CCP-UK databaseLactate dehydrogenase (Units/L)COVID-GRAMb, E-CURB65mExcluded – too many values missing from derivation datasetUrea (mmol/L)CURB-65a, A-DROPc, PSIdIncludedCreatinine (?mol/L)SOFAhIncludedC-reactive protein (CRP; mg/dL)Associated with poorer outcomes in patients with covid-19q, rIncludedaCURB65 (mneumonic representing included model variables)bCOVID-GRAM (Liang JAMA Int Med 2020)cA-DROP (mneumonic representing included model variables)dPneumonia Severity Index (PSI)eGuan 2020. Eur Respir J. May; 55(5): 2000547fNational Early Warning Score (NEWS2)gXie score (Xie MedRxiv 2020)hSequential Organ Failure Assessment (SOFA) scoreiSevere Community Acquired Pneumonia (SCAP) scorejSMART-COP (mneumonic representing included model variables)kLippi G et al. Hematol Transfus Cell Ther. 2020lDL score (Zhang MedRxiv 2020)mExpanded CURB65 score (E-CURB65)nZhou 2020, Lancet 395; P1054-1062oCai 2020, DOI: 2020, Critical Care. 24:255qLiu 2020, European Respiratory Journal; 55(5): 2001112rLuo 2020, Clin Infect Dis; doi: https//10.1093/cid/ciaa641Appendix 3. Availability of candidate prediction variables within the ISARIC CCP-UK database (derivation cohort)AvailableMissingMissing (%)Age on admission (years)3469200.0Sex at Birth34595970.3Ethnicity30867382511.0Chronic cardiac disease3184628468.2Chronic kidney disease3158531079.0Malignant neoplasm 3131333799.7Moderate or severe liver disease3128934039.8Clinician-defined obesity28636605617.5Chronic pulmonary disease (not asthma)3174629468.5Diabetes3138733059.5Number of comorbidities3469200.0Respiratory Rate3210725857.5Peripheral oxygen saturations (%)3246522276.4Systolic blood pressure (mmHg)3254221506.2Diastolic blood pressure (mmHg)3247722156.4Temperature (°C)3223124617.1Heart Rate (bpm)3217025227.3Glasgow Coma Score29297539515.6pH109012379168.6Infiltrates on chest radiograph222271246535.9Haemoglobin (g/L)28842585016.9White cell count (109/L)28662603017.4Neutrophil count (109/L)28467622517.9Lymphocyte count (109/L)28426626618.1Haematocrit (%)173701732249.9Platelet Count (109/L)28501619117.8Prothrombin (seconds)175451714749.4Activated partial thromboplastin time (APTT) (seconds)144682022458.3Sodium (mmol/L)28375631718.2Total Bilirubin (mg/dL)235511114132.1Alanine aminotransferase (ALT) (units/L)217231296937.4Aspartate aminotransferase (AST) (units/L)24063228693.1Lactate dehydrogenase (Units/L)23883230493.1Glucose (mmol/L)138472084560.1Urea (mmol/L)25148954427.5Creatinine (?mol/L)28371632118.2C-reactive protein (CRP; mg/dL)26864782822.6Appendix 4. Criterion-based approach using generalised additive models for remaining 20 candidate variables following exclusion for missing valuesVariableDeviance explained (%) Difference with 20 variable model R2Unbiased Risk Estimator (UBRE)Inclusion / exclusion in final model*All candidate variables24.1-0.273-0.019-Age19.64.50.2290.037IncludedSex at Birth23.70.40.270-0.016IncludedNumber of comorbidities23.60.50.271-0.015IncludedRespiratory Rate23.50.60.266-0.012IncludedPeripheral oxygen saturations22.81.30.258-0.003IncludedSystolic blood pressure24.00.10.272-0.019ExcludedDiastolic blood pressure24.10.00.273-0.019ExcludedTemperature 24.00.10.272-0.019ExcludedHeart Rate24.00.10.272-0.019ExcludedGlasgow Coma Score23.50.60.266-0.012IncludedHaemoglobin23.90.20.272-0.019ExcludedWhite cell count 24.10.00.273-0.019ExcludedNeutrophil count24.00.10.272-0.019ExcludedLymphocyte count 24.10.00.273-0.019ExcludedPlatelet Count 24.00.10.272-0.019ExcludedSodium 23.90.20.271-0.018ExcludedTotal Bilirubin 24.00.10.272-0.019ExcludedUrea23.60.50.271-0.015IncludedCreatinine24.00.10.273-0.019ExcludedC-reactive protein23.11.20.260-0.006IncludedFinal eight variable model23.01.10.2600.001-*Inclusion criteria specified as >1% change in deviance (>0.2) or > 10% (>0.002) in UBRE compared to GAM model containing all candidate variablesAppendix 5. Continuous smoothed predictors (thin-plate splines) for numerical variables using generated primary generalised additive modelAppendix 6. Final lasso regression coefficients (log-odds scale) (A) and shrinkage of coefficients (y-axis) at different values of lambda (x-axis) (B) for 4C score.ALevelβ-coefficient (standard error)Intercept--4.239 (0.009)Age (years)50-590.712 (0.003)60-691.304 (0.006)70-791.861 (0.006)≥802.286 (0.006) Sex at birthMale0.184 (0.003) Number of comorbidities*10.380 (0.005)≥20.590 (0.004)Respiratory rate (breaths/minute)20-290.219 (0.003)≥300.644 (0.005) Oxygen saturation on room air (%)<920.550 (0.003) Glasgow Coma Scale<150.570 (0.004)Urea (mmol/L)7-140.513 (0.004)>141.013 (0.003)CRP (mg/dL)50-990.382 (0.004)≥1000.782 (0.004)*Comorbidities were defined using the Charlson Comorbidity Index, with the addition of clinician-defined obesityBAppendix 7. Completeness of predictor variables within validation cohortAvailableMissingMissing (%)Age on admission (years)2245400.0Sex at Birth22413410.2Chronic cardiac disease20203225110.0Chronic kidney disease20022243210.8Malignant neoplasm 19877257711.5Moderate or severe liver disease19801265311.8Obesity17918453620.2Chronic pulmonary disease (not asthma)20088236610.5Diabetes19120333414.8Number of comorbidities2245400.0Respiratory Rate2063118238.1Peripheral oxygen saturations (%)2093015246.8Glasgow Coma Score19589286512.8Urea (mmol/L)16631582325.9C-reactive protein (CRP; mg/dL)16883557124.8Appendix 8. Calibration plot of 4C score in validation cohortAppendix 9. Components of included risk stratification scores (*indicates novel covid-19 risk score).ABScore ConditionOutcomeA-DROP (Miyashita Int Med 2006) Community-acquired pneumonia30-day mortalityCOVID-GRAM (Liang JAMA Int Med 2020)covid-19Mortality and/or ICU admissionCRB65 (Bauer J Int Med 2006)Community-acquired pneumonia30-day mortalityCURB65 (Lim Thorax 2003)Community-acquired pneumonia30-day mortalityDL score (Zhang MedRxiv 2020)covid-19Mortality / ICU admissionDS-CRB65 (Dwyer BMJ ORR 2014)Community-acquired pneumonia30-day mortalityE-CURB65 (Liu Sci Rep 2016)Community-acquired pneumonia30-day mortalityNEWS2 (Royal College of Physicians, UK 2012)SepsisIn-hospital mortalityPSI (Fine NEJM 1997)Community-acquired pneumoniaLow risk of 30-day mortalityqSOFA (Singer JAMA 2016)SepsisIn-hospital mortalitySCAP (Yandiola Chest 2009)Community-acquired pneumoniaAdverse outcome*SMART-COP (Charles Clin Infect Dis 2008)Community-acquired pneumoniaNeed for ventilator or vasopressor supportSOFA (Vincent Int Care Med 1996)SepsisICU mortalitySurgisphere (no definitive publication)covid-19In-hospital mortality / critical illness**Xie score (Xie MedRxiv 2020)covid-19Mortality*ICU admission, need for mechanical ventilation, severe sepsis, or treatment failure)**Measured outcome unclear in online materialAppendix 10. Operative characteristics of included risk stratification scores to predict mortality at usual cut-offs within validation cohortsTestTPTNFPFNSensitivitySpecificityPPVNPVSurgisphere (>4)406751836688142274.143.737.878.5Surgisphere (>7)240589282943308443.875.245.074.3qSOFA (>1)1267110991040431122.791.454.972.0qSOFA (>2)188120667353903.499.472.069.1NEWS (>4)292183413627256753.269.744.676.5NEWS (>6)1649104191549383930.087.151.673.1SMART-COP (>2)1261092162583.433.536.881.3SMART-COP (>4)472755010431.184.648.572.6SMART-COP (>6)10313121416.696.345.568.9SCAP (≥10)157341571094.017.850.077.3SCAP (≥20)91134577654.570.261.563.8SCAP (≥30)381791212922.893.776.058.1DL score (Low)383046565560109677.845.640.880.9DL score (High)328360514165164366.659.244.178.6CRB65 (=0)52553415872432294.228.137.691.4CRB65 (>2)81211686453476514.696.364.271.0DS-CRB65 (>1)45015625609390883.248.042.586.1DS-CRB65 (>2)265493292389275549.179.652.677.2DS-CRB65 (>3)107911153565433019.995.265.672.0CURB65 (>1)37785167451585881.553.445.685.8CURB65 (>2)197680491633266042.683.154.875.2CURB65 (>3)6149399283402213.297.168.570.0A-DROP (≥3)175786001101288037.988.761.574.9A-DROP (≥4)426954315842119.298.472.969.4E-CURB65 (>2)3496173809279.161.947.987.0E-CURB65 (>4)669673037515.097.068.872.1PSI (>70)12243188596.118.639.489.6PSI (>90)111891421687.438.543.984.8PSI (>130)73171605457.574.054.976.0Some included scores did not provide cut-off values. TP – True positive; TN - True negative; FP – False positive; FP – False positive; PPV – Positive predictive value; NPV – Negative predictive value. *Derived in covid-19 cohort Appendix 11. Discriminatory performance in imputed sets for models across derivation and validation cohortsDerivation cohortValidation cohortOriginalImputed setOriginalImputed set4C0.790.770.780.76Machine learning comparison (XGBoost)0.810.790.790.78Appendix 12. Demographic and clinical characteristics for validation cohort after stratification by geography Validation North dataset (n = 13836)Validation South dataset(n = 8618)Number of hospitals included11786Mortality (%)4140 (29.9)2291 (26.6)Age on admission (years)<501607 (11.6)1218 (14.1)50-591574 (11.4)1056 (12.3)60-691885 (13.6)1271 (14.7)70-793209 (23.2)1762 (20.4)≥805561 (40.2)3311 (38.4)Sex at BirthMale7398 (53.6)4805 (55.8)Female6408 (46.4)3802 (44.2)EthnicityWhite11125 (89.1)5713 (78.1)South Asian426 (3.4)361 (4.9)East Asian55 (0.4)83 (1.1)Black281 (2.2)484 (6.6)Other Ethnic Minority604 (4.8)671 (9.2)Chronic cardiac disease4431 (34.9)2423 (32.3)Chronic kidney disease2394 (19.0)1288 (17.4)Malignant neoplasm 1358 (10.9)786 (10.7)Moderate or severe liver disease288 (2.3)139 (1.9)Obesity (BMI ≥30)1365 (12.2)763 (11.4)Chronic pulmonary disease (not asthma)2448 (19.4)1224 (16.4)Diabetes2618 (21.7)1592 (22.5)Number of comorbidities03078 (22.2)2447 (28.4)13704 (26.8)2340 (27.2)≥27054 (51.0)3832 (44.5)Respiratory Rate20.0 (7.0)21.0 (8.0)Peripheral oxygen saturation (%)94.0 (5.0)94.0 (6.0)Systolic blood pressure (mmHg)128.0 (33.0)129.0 (33.0)Diastolic blood pressure (mmHg)73.0 (20.0)73.0 (20.0)Temperature (°C)37.1 (1.5)37.1 (1.5)Heart Rate (bpm)90.0 (28.0)90.0 (27.0)Glasgow Coma Score15.0 (0.0)15.0 (0.0)pH7.4 (0.1)7.4 (0.1)Bicarbonate (mmol/L)24.1 (4.7)24.6 (5.0)Infiltrates on chest radiograph5125 (60.7)3122 (61.8)Haemoglobin (g/L)127.0 (31.0)127.0 (31.0)White cell count (109/L)7.5 (5.3)7.7 (5.4)Neutrophil count (109/L)5.7 (4.9)6.0 (5.0)Lymphocyte count (109/L)0.9 (0.7)0.9 (0.7)Haematocrit (%)28.0 (38.6)14.5 (38.6)Platelet Count (109/L)222.0 (125.0)224.0 (128.0)Prothrombin (seconds)13.2 (3.2)13.4 (3.4)Activated partial thromboplastin time (APTT) (seconds)29.0 (8.4)29.7 (9.2)Sodium (mmol/L)137.0 (6.0)137.0 (6.0)Potassium (mmol/L)4.1 (0.8)4.1 (0.7)Total Bilirubin (mg/dL)9.0 (8.0)10.0 (7.0)Alanine aminotransferase (ALT) (units/L)24.0 (25.0)26.0 (28.0)Aspartate aminotransferase (AST) (units/L)39.0 (43.0)69.0 (57.0)Lactate dehydrogenase (Units/L)390.0 (285.0)443.0 (334.5)Glucose (mmol/L)6.7 (3.0)6.8 (3.4)Urea (mmol/L)7.4 (6.8)7.3 (6.8)Creatinine (?mol/L)86.0 (57.0)87.0 (55.0)Lactate (mmol/L)1.6 (1.1)1.4 (1.1)C-reactive protein (CRP) (mg/dL)76.0 (115.0)82.0 (127.7)Appendix 13. Sensitivity analysis of discriminatory performance for risk stratification scores after stratification of validation cohort by geography to predict inpatient mortality in patients hospitalised with covid-19 Validation North (N = 13836) Validation South (N = 8618)NAUC (95% CI)NAUC (95% CI)SOFA390.59 (0.40 - 0.78)1510.60 (0.50 - 0.70)SMARTCOP2040.61 (0.53 - 0.68)2720.69 (0.62 - 0.76)qSOFA113400.62 (0.61 - 0.63)63750.63 (0.61 - 0.64)SCAP1850.63 (0.55 - 0.71)1730.70 (0.61 - 0.78)Surgisphere*111060.64 (0.62 - 0.65)62530.63 (0.62 - 0.64)NEWS2111780.66 (0.65 - 0.67)62770.65 (0.64 - 0.67)DL score*93780.68 (0.67 - 0.69)57630.67 (0.65 - 0.68)CRB65113400.69 (0.68 - 0.70)63750.69 (0.67 - 0.70)COVID-GRAM*5050.70 (0.65 - 0.75)6470.72 (0.67 - 0.76)CURB6591440.72 (0.71 - 0.73)51740.72 (0.71 - 0.74)DS-CRB65109590.72 (0.72 - 0.73)61680.71 (0.70 - 0.73)PSI2150.73 (0.66 - 0.80)1430.73 (0.64 - 0.82)A-DROP91480.74 (0.73 - 0.75)51900.73 (0.72 - 0.75)Xie score*6650.74 (0.70 - 0.78)9620.73 (0.70 - 0.77)E-CURB656030.78 (0.74 - 0.81)8350.76 (0.72 - 0.79)4C83220.78 (0.77 - 0.79)49220.77 (0.76 - 0.79)Machine learning comparison (XGBoost)-0.80 (0.79 - 0.81)-0.79 (0.78 - 0.80) ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download