TITLE



TITLEThe intensive care AI clinician learns optimal treatment strategies for sepsis.AUTHORSMatthieu Komorowski 1,2,3, Leo Anthony Celi 3,4, Omar Badawi 3,5,6 Anthony C. Gordon 1*, A. Aldo Faisal 2,7,8*AFFILIATIONS1- Department of Surgery and Cancer, Imperial College London, SW7 2AZ London, UK.2- Department of Bioengineering, Imperial College London, SW7 2AZ London, UK.3- Laboratory for Computational Physiology, Harvard-MIT Division of Health Sciences & Technology, Cambridge, Massachusetts, USA.4- Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA.5- Department of eICU Research and Development, Philips Healthcare, Baltimore, Maryland, USA.6- Department of Pharmacy Practice and Science, University of Maryland, School of Pharmacy, Baltimore, Maryland, USA.7- Department of Computer Science, Imperial College London, SW7 2AZ London, UK.8- MRC Clinical Sciences Centre, Faculty of Medicine, Imperial College London, Hammersmith Hospital Campus, Du Cane Road, W12 0NN London, UK.INTRODUCTIONSepsis is the third leading cause of death worldwide and the main cause of mortality in hospitals ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"JQtyVooF","properties":{"formattedCitation":"\\super 1\\uc0\\u8211{}3\\nosupersub{}","plainCitation":"1–3","noteIndex":0},"citationItems":[{"id":2578,"uris":[""],"uri":[""],"itemData":{"id":2578,"type":"article-journal","title":"Sepsis: pathophysiology and clinical management","container-title":"BMJ","page":"i1585","volume":"353","source":"","abstract":"Sepsis, severe sepsis, and septic shock represent increasingly severe systemic inflammatory responses to infection. Sepsis is common in the aging population, and it disproportionately affects patients with cancer and underlying immunosuppression. In its most severe form, sepsis causes multiple organ dysfunction that can produce a state of chronic critical illness characterized by severe immune dysfunction and catabolism. Much has been learnt about the pathogenesis of sepsis at the molecular, cell, and intact organ level. Despite uncertainties in hemodynamic management and several treatments that have failed in clinical trials, investigational therapies increasingly target sepsis induced organ and immune dysfunction. Outcomes in sepsis have greatly improved overall, probably because of an enhanced focus on early diagnosis and fluid resuscitation, the rapid delivery of effective antibiotics, and other improvements in supportive care for critically ill patients. These improvements include lung protective ventilation, more judicious use of blood products, and strategies to reduce nosocomial infections.","DOI":"10.1136/bmj.i1585","ISSN":"1756-1833","note":"PMID: 27217054","shortTitle":"Sepsis","journalAbbreviation":"BMJ","language":"en","author":[{"family":"Gotts","given":"Jeffrey E."},{"family":"Matthay","given":"Michael A."}],"issued":{"date-parts":[["2016",5,23]]}}},{"id":2069,"uris":[""],"uri":[""],"itemData":{"id":2069,"type":"chapter","title":"National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2011: Statistical Brief #160","container-title":"Healthcare Cost and Utilization Project (HCUP) Statistical Briefs","publisher":"Agency for Health Care Policy and Research (US)","publisher-place":"Rockville (MD)","source":"PubMed","event-place":"Rockville (MD)","abstract":"This Statistical Brief presents data from the Healthcare Cost and Utilization Project (HCUP) on costs of inpatient hospital stays in the United States in 2011. This report describes the distribution of costs by expected primary payer and illustrates the conditions accounting for the largest percentage of each payer’s hospital costs. The primary payers examined are Medicare, Medicaid, private insurance, and the uninsured. The hospital costs represent the hospital’s cost to produce the services—not the amount paid for services by payers—and they do not include the physician fees associated with the hospitalization. All differences between estimates noted in the text are statistically significant at the .05 level or better.","URL":"","call-number":"NBK169005","note":"PMID: 24199255","shortTitle":"National Inpatient Hospital Costs","language":"eng","author":[{"family":"Torio","given":"Celeste M."},{"family":"Andrews","given":"Roxanne M."}],"issued":{"date-parts":[["2013",8]]},"accessed":{"date-parts":[["2015",11,30]]}}},{"id":3030,"uris":[""],"uri":[""],"itemData":{"id":3030,"type":"article-journal","title":"Hospital Deaths in Patients With Sepsis From 2 Independent Cohorts","container-title":"JAMA","page":"90","volume":"312","issue":"1","source":"CrossRef","DOI":"10.1001/jama.2014.5804","ISSN":"0098-7484","language":"en","author":[{"family":"Liu","given":"Vincent"},{"family":"Escobar","given":"Gabriel J."},{"family":"Greene","given":"John D."},{"family":"Soule","given":"Jay"},{"family":"Whippy","given":"Alan"},{"family":"Angus","given":"Derek C."},{"family":"Iwashyna","given":"Theodore J."}],"issued":{"date-parts":[["2014",7,2]]}}}],"schema":""} 1–3, but the best treatment strategy remains uncertain. In particular, evidence suggests that current practices in the administration of intravenous fluids and vasopressors are suboptimal and likely induce harm in a proportion of patients ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"QMRjzPdt","properties":{"formattedCitation":"\\super 1,4\\uc0\\u8211{}6\\nosupersub{}","plainCitation":"1,4–6","noteIndex":0},"citationItems":[{"id":2596,"uris":[""],"uri":[""],"itemData":{"id":2596,"type":"article-journal","title":"Fluid resuscitation in human sepsis: Time to rewrite history?","container-title":"Annals of Intensive Care","page":"4","volume":"7","issue":"1","source":"annalsofintensivecare.","abstract":"Fluid resuscitation continues to be recommended as the first-line resuscitative therapy for all patients with severe sepsis and septic shock. The current acceptance of the therapy is based in part on long history and familiarity with its use in the resuscitation of other forms of shock, as well as on an incomplete and incorrect understanding of the pathophysiology of sepsis. Recently, the safety of intravenous fluids in patients with sepsis has been called into question with both prospective and observational data suggesting improved outcomes with less fluid or no fluid. The current evidence for the continued use of fluid resuscitation for sepsis remains contentious with no prospective evidence demonstrating benefit to fluid resuscitation as a therapy in isolation. This article reviews the historical and physiological rationale for the introduction of fluid resuscitation as treatment for sepsis and highlights a number of significant concerns based on current experimental and clinical evidence. The research agenda should focus on the development of hyperdynamic animal sepsis models which more closely mimic human sepsis and on experimental and clinical studies designed to evaluate minimal or no fluid strategies in the resuscitation phase of sepsis.","DOI":"10.1186/s13613-016-0231-8","ISSN":"2110-5820","shortTitle":"Fluid resuscitation in human sepsis","language":"En","author":[{"family":"Byrne","given":"Liam"},{"family":"Haren","given":"Frank"}],"issued":{"date-parts":[["2017",1,3]]}}},{"id":2332,"uris":[""],"uri":[""],"itemData":{"id":2332,"type":"article-journal","title":"The demise of early goal-directed therapy for severe sepsis and septic shock","container-title":"Acta Anaesthesiologica Scandinavica","page":"561-567","volume":"59","issue":"5","source":"Wiley Online Library","abstract":"A protocol for the quantitative resuscitation of severe sepsis and septic shock known as early goal-directed therapy (EGDT) was published in 2001. Despite serious limitations, this study became widely adopted around the world and formed the basis of the Surviving Sepsis Campaign 6?h resuscitation bundle. Subsequently, a large number of observational before-and-after studies were published which demonstrated that EGDT reduced mortality. However, during this time period, there has been a substantial reduction in the mortality from sepsis in many Western nations that appears unrelated to EGDT. Recently, the Protocolized Care for Early Septic Shock (ProCESS) and The Australasian Resuscitation in Sepsis Evaluation (ARISE) trials failed to demonstrate any outcome benefit from EGDT. These two large, multicenter, randomized controlled studies raise serious questions regarding the validity of the original EGDT study and the scientific rigor of the uncontrolled, largely retrospective before–after clinical studies. Furthermore, accruing data suggest an association between the amount of fluid administered in the first 72?h and the mortality of patients with severe sepsis. Patients in all arms of the ProCESS and ARISE trials received substantial and nearly equivalent amounts of fluid. It is proposed that a more conservative fluid strategy and the earlier use of norepinephrine in patients with septic shock may be associated with further improvements in the outcome of patients with sepsis.","DOI":"10.1111/aas.12479","ISSN":"1399-6576","journalAbbreviation":"Acta Anaesthesiol Scand","language":"en","author":[{"family":"Marik","given":"P. E."}],"issued":{"date-parts":[["2015",5,1]]}}},{"id":2665,"uris":[""],"uri":[""],"itemData":{"id":2665,"type":"article-journal","title":"A rational approach to fluid therapy in sepsis","container-title":"British Journal of Anaesthesia","page":"339-349","volume":"116","issue":"3","source":"bja.","abstract":"Aggressive fluid resuscitation to achieve a central venous pressure (CVP) greater than 8 mm Hg has been promoted as the standard of care, in the management of patients with severe sepsis and septic shock. However recent clinical trials have demonstrated that this approach does not improve the outcome of patients with severe sepsis and septic shock. Pathophysiologically, sepsis is characterized by vasoplegia with loss of arterial tone, venodilation with sequestration of blood in the unstressed blood compartment and changes in ventricular function with reduced compliance and reduced preload responsiveness. These data suggest that sepsis is primarily not a volume-depleted state and recent evidence demonstrates that most septic patients are poorly responsive to fluids. Furthermore, almost all of the administered fluid is sequestered in the tissues, resulting in severe oedema in vital organs and, thereby, increasing the risk of organ dysfunction. These data suggest that a physiologic, haemodynamically guided conservative approach to fluid therapy in patients with sepsis would be prudent and would likely reduce the morbidity and improve the outcome of this disease.","DOI":"10.1093/bja/aev349","ISSN":"0007-0912, 1471-6771","note":"PMID: 26507493","journalAbbreviation":"Br. J. Anaesth.","language":"en","author":[{"family":"Marik","given":"P."},{"family":"Bellomo","given":"R."}],"issued":{"date-parts":[["2016",3,1]]}}},{"id":2578,"uris":[""],"uri":[""],"itemData":{"id":2578,"type":"article-journal","title":"Sepsis: pathophysiology and clinical management","container-title":"BMJ","page":"i1585","volume":"353","source":"","abstract":"Sepsis, severe sepsis, and septic shock represent increasingly severe systemic inflammatory responses to infection. Sepsis is common in the aging population, and it disproportionately affects patients with cancer and underlying immunosuppression. In its most severe form, sepsis causes multiple organ dysfunction that can produce a state of chronic critical illness characterized by severe immune dysfunction and catabolism. Much has been learnt about the pathogenesis of sepsis at the molecular, cell, and intact organ level. Despite uncertainties in hemodynamic management and several treatments that have failed in clinical trials, investigational therapies increasingly target sepsis induced organ and immune dysfunction. Outcomes in sepsis have greatly improved overall, probably because of an enhanced focus on early diagnosis and fluid resuscitation, the rapid delivery of effective antibiotics, and other improvements in supportive care for critically ill patients. These improvements include lung protective ventilation, more judicious use of blood products, and strategies to reduce nosocomial infections.","DOI":"10.1136/bmj.i1585","ISSN":"1756-1833","note":"PMID: 27217054","shortTitle":"Sepsis","journalAbbreviation":"BMJ","language":"en","author":[{"family":"Gotts","given":"Jeffrey E."},{"family":"Matthay","given":"Michael A."}],"issued":{"date-parts":[["2016",5,23]]}}}],"schema":""} 1,4–6. To tackle this sequential decision-making problem, we developed a reinforcement learning agent, the artificial intelligence (AI) Clinician, which learns from data to predict patient dynamics given specific treatment decisions. Our agent extracted implicit knowledge from an amount of patient data that exceeds many-fold the life-time experience of human clinicians and learned optimal treatment by having analysed myriads of (mostly sub-optimal) treatment decisions. We demonstrate that the value of the AI Clinician’s selected treatment is on average reliably higher than the human clinicians. In a large validation cohort independent from the training data, mortality was lowest in patients where clinicians’ actual doses matched the AI policy. Our model provides individualized and clinically interpretable treatment decisions for sepsis that could improve patient outcomes.MAIN TEXTSepsis is defined as severe infection leading to life-threatening acute organ dysfunction ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"3m8en1lll","properties":{"formattedCitation":"\\super 7\\nosupersub{}","plainCitation":"7","noteIndex":0},"citationItems":[{"id":2330,"uris":[""],"uri":[""],"itemData":{"id":2330,"type":"article-journal","title":"The third international consensus definitions for sepsis and septic shock (sepsis-3)","container-title":"JAMA","page":"801-810","volume":"315","issue":"8","source":"Silverchair","abstract":"Importance?\nDefinitions of sepsis and septic shock were last revised in 2001. Considerable advances have since been made into the pathobiology (changes in organ function, morphology, cell biology, biochemistry, immunology, and circulation), management, and epidemiology of sepsis, suggesting the need for reexamination.Objective\nTo evaluate and, as needed, update definitions for sepsis and septic shock.Process\nA task force (n?=?19) with expertise in sepsis pathobiology, clinical trials, and epidemiology was convened by the Society of Critical Care Medicine and the European Society of Intensive Care Medicine. Definitions and clinical criteria were generated through meetings, Delphi processes, analysis of electronic health record databases, and voting, followed by circulation to international professional societies, requesting peer review and endorsement (by 31 societies listed in the Acknowledgment).Key Findings From Evidence Synthesis\nLimitations of previous definitions included an excessive focus on inflammation, the misleading model that sepsis follows a continuum through severe sepsis to shock, and inadequate specificity and sensitivity of the systemic inflammatory response syndrome (SIRS) criteria. Multiple definitions and terminologies are currently in use for sepsis, septic shock, and organ dysfunction, leading to discrepancies in reported incidence and observed mortality. The task force concluded the term severe sepsis was redundant.Recommendations\nSepsis should be defined as life-threatening organ dysfunction caused by a dysregulated host response to infection. For clinical operationalization, organ dysfunction can be represented by an increase in the Sequential [Sepsis-related] Organ Failure Assessment (SOFA) score of 2 points or more, which is associated with an in-hospital mortality greater than 10%. Septic shock should be defined as a subset of sepsis in which particularly profound circulatory, cellular, and metabolic abnormalities are associated with a greater risk of mortality than with sepsis alone. Patients with septic shock can be clinically identified by a vasopressor requirement to maintain a mean arterial pressure of 65 mm Hg or greater and serum lactate level greater than 2 mmol/L (>18 mg/dL) in the absence of hypovolemia. This combination is associated with hospital mortality rates greater than 40%. In out-of-hospital, emergency department, or general hospital ward settings, adult patients with suspected infection can be rapidly identified as being more likely to have poor outcomes typical of sepsis if they have at least 2 of the following clinical criteria that together constitute a new bedside clinical score termed quickSOFA (qSOFA): respiratory rate of 22/min or greater, altered mentation, or systolic blood pressure of 100 mm Hg or less.Conclusions and Relevance\nThese updated definitions and clinical criteria should replace previous definitions, offer greater consistency for epidemiologic studies and clinical trials, and facilitate earlier recognition and more timely management of patients with sepsis or at risk of developing sepsis.","DOI":"10.1001/jama.2016.0287","ISSN":"0098-7484","journalAbbreviation":"JAMA","author":[{"literal":"Singer M"},{"literal":"Deutschman CS"},{"literal":"Seymour C"},{"literal":"et al"}],"issued":{"date-parts":[["2016",2,23]]}}}],"schema":""} 7. The management of intravenous fluids and vasopressors in sepsis is a key clinical challenge and a top research priority ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"67M28WrZ","properties":{"formattedCitation":"\\super 1,4\\nosupersub{}","plainCitation":"1,4","noteIndex":0},"citationItems":[{"id":2596,"uris":[""],"uri":[""],"itemData":{"id":2596,"type":"article-journal","title":"Fluid resuscitation in human sepsis: Time to rewrite history?","container-title":"Annals of Intensive Care","page":"4","volume":"7","issue":"1","source":"annalsofintensivecare.","abstract":"Fluid resuscitation continues to be recommended as the first-line resuscitative therapy for all patients with severe sepsis and septic shock. The current acceptance of the therapy is based in part on long history and familiarity with its use in the resuscitation of other forms of shock, as well as on an incomplete and incorrect understanding of the pathophysiology of sepsis. Recently, the safety of intravenous fluids in patients with sepsis has been called into question with both prospective and observational data suggesting improved outcomes with less fluid or no fluid. The current evidence for the continued use of fluid resuscitation for sepsis remains contentious with no prospective evidence demonstrating benefit to fluid resuscitation as a therapy in isolation. This article reviews the historical and physiological rationale for the introduction of fluid resuscitation as treatment for sepsis and highlights a number of significant concerns based on current experimental and clinical evidence. The research agenda should focus on the development of hyperdynamic animal sepsis models which more closely mimic human sepsis and on experimental and clinical studies designed to evaluate minimal or no fluid strategies in the resuscitation phase of sepsis.","DOI":"10.1186/s13613-016-0231-8","ISSN":"2110-5820","shortTitle":"Fluid resuscitation in human sepsis","language":"En","author":[{"family":"Byrne","given":"Liam"},{"family":"Haren","given":"Frank"}],"issued":{"date-parts":[["2017",1,3]]}}},{"id":2578,"uris":[""],"uri":[""],"itemData":{"id":2578,"type":"article-journal","title":"Sepsis: pathophysiology and clinical management","container-title":"BMJ","page":"i1585","volume":"353","source":"","abstract":"Sepsis, severe sepsis, and septic shock represent increasingly severe systemic inflammatory responses to infection. Sepsis is common in the aging population, and it disproportionately affects patients with cancer and underlying immunosuppression. In its most severe form, sepsis causes multiple organ dysfunction that can produce a state of chronic critical illness characterized by severe immune dysfunction and catabolism. Much has been learnt about the pathogenesis of sepsis at the molecular, cell, and intact organ level. Despite uncertainties in hemodynamic management and several treatments that have failed in clinical trials, investigational therapies increasingly target sepsis induced organ and immune dysfunction. Outcomes in sepsis have greatly improved overall, probably because of an enhanced focus on early diagnosis and fluid resuscitation, the rapid delivery of effective antibiotics, and other improvements in supportive care for critically ill patients. These improvements include lung protective ventilation, more judicious use of blood products, and strategies to reduce nosocomial infections.","DOI":"10.1136/bmj.i1585","ISSN":"1756-1833","note":"PMID: 27217054","shortTitle":"Sepsis","journalAbbreviation":"BMJ","language":"en","author":[{"family":"Gotts","given":"Jeffrey E."},{"family":"Matthay","given":"Michael A."}],"issued":{"date-parts":[["2016",5,23]]}}}],"schema":""} 1,4. Besides general guidelines such as the Surviving Sepsis Campaign, no tool currently exists to personalise treatment of sepsis and assist clinicians making decisions in real-time, at the patient level ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"VbH6KHfg","properties":{"formattedCitation":"\\super 4\\uc0\\u8211{}6\\nosupersub{}","plainCitation":"4–6","noteIndex":0},"citationItems":[{"id":2596,"uris":[""],"uri":[""],"itemData":{"id":2596,"type":"article-journal","title":"Fluid resuscitation in human sepsis: Time to rewrite history?","container-title":"Annals of Intensive Care","page":"4","volume":"7","issue":"1","source":"annalsofintensivecare.","abstract":"Fluid resuscitation continues to be recommended as the first-line resuscitative therapy for all patients with severe sepsis and septic shock. The current acceptance of the therapy is based in part on long history and familiarity with its use in the resuscitation of other forms of shock, as well as on an incomplete and incorrect understanding of the pathophysiology of sepsis. Recently, the safety of intravenous fluids in patients with sepsis has been called into question with both prospective and observational data suggesting improved outcomes with less fluid or no fluid. The current evidence for the continued use of fluid resuscitation for sepsis remains contentious with no prospective evidence demonstrating benefit to fluid resuscitation as a therapy in isolation. This article reviews the historical and physiological rationale for the introduction of fluid resuscitation as treatment for sepsis and highlights a number of significant concerns based on current experimental and clinical evidence. The research agenda should focus on the development of hyperdynamic animal sepsis models which more closely mimic human sepsis and on experimental and clinical studies designed to evaluate minimal or no fluid strategies in the resuscitation phase of sepsis.","DOI":"10.1186/s13613-016-0231-8","ISSN":"2110-5820","shortTitle":"Fluid resuscitation in human sepsis","language":"En","author":[{"family":"Byrne","given":"Liam"},{"family":"Haren","given":"Frank"}],"issued":{"date-parts":[["2017",1,3]]}}},{"id":2332,"uris":[""],"uri":[""],"itemData":{"id":2332,"type":"article-journal","title":"The demise of early goal-directed therapy for severe sepsis and septic shock","container-title":"Acta Anaesthesiologica Scandinavica","page":"561-567","volume":"59","issue":"5","source":"Wiley Online Library","abstract":"A protocol for the quantitative resuscitation of severe sepsis and septic shock known as early goal-directed therapy (EGDT) was published in 2001. Despite serious limitations, this study became widely adopted around the world and formed the basis of the Surviving Sepsis Campaign 6?h resuscitation bundle. Subsequently, a large number of observational before-and-after studies were published which demonstrated that EGDT reduced mortality. However, during this time period, there has been a substantial reduction in the mortality from sepsis in many Western nations that appears unrelated to EGDT. Recently, the Protocolized Care for Early Septic Shock (ProCESS) and The Australasian Resuscitation in Sepsis Evaluation (ARISE) trials failed to demonstrate any outcome benefit from EGDT. These two large, multicenter, randomized controlled studies raise serious questions regarding the validity of the original EGDT study and the scientific rigor of the uncontrolled, largely retrospective before–after clinical studies. Furthermore, accruing data suggest an association between the amount of fluid administered in the first 72?h and the mortality of patients with severe sepsis. Patients in all arms of the ProCESS and ARISE trials received substantial and nearly equivalent amounts of fluid. It is proposed that a more conservative fluid strategy and the earlier use of norepinephrine in patients with septic shock may be associated with further improvements in the outcome of patients with sepsis.","DOI":"10.1111/aas.12479","ISSN":"1399-6576","journalAbbreviation":"Acta Anaesthesiol Scand","language":"en","author":[{"family":"Marik","given":"P. E."}],"issued":{"date-parts":[["2015",5,1]]}}},{"id":2665,"uris":[""],"uri":[""],"itemData":{"id":2665,"type":"article-journal","title":"A rational approach to fluid therapy in sepsis","container-title":"British Journal of Anaesthesia","page":"339-349","volume":"116","issue":"3","source":"bja.","abstract":"Aggressive fluid resuscitation to achieve a central venous pressure (CVP) greater than 8 mm Hg has been promoted as the standard of care, in the management of patients with severe sepsis and septic shock. However recent clinical trials have demonstrated that this approach does not improve the outcome of patients with severe sepsis and septic shock. Pathophysiologically, sepsis is characterized by vasoplegia with loss of arterial tone, venodilation with sequestration of blood in the unstressed blood compartment and changes in ventricular function with reduced compliance and reduced preload responsiveness. These data suggest that sepsis is primarily not a volume-depleted state and recent evidence demonstrates that most septic patients are poorly responsive to fluids. Furthermore, almost all of the administered fluid is sequestered in the tissues, resulting in severe oedema in vital organs and, thereby, increasing the risk of organ dysfunction. These data suggest that a physiologic, haemodynamically guided conservative approach to fluid therapy in patients with sepsis would be prudent and would likely reduce the morbidity and improve the outcome of this disease.","DOI":"10.1093/bja/aev349","ISSN":"0007-0912, 1471-6771","note":"PMID: 26507493","journalAbbreviation":"Br. J. Anaesth.","language":"en","author":[{"family":"Marik","given":"P."},{"family":"Bellomo","given":"R."}],"issued":{"date-parts":[["2016",3,1]]}}}],"schema":""} 4–6. As a consequence, clinical variability in sepsis treatment is extreme, with consistent evidence that suboptimal decisions lead to poorer outcomes ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"1LzNIoBE","properties":{"formattedCitation":"\\super 8\\uc0\\u8211{}10\\nosupersub{}","plainCitation":"8–10","noteIndex":0},"citationItems":[{"id":2557,"uris":[""],"uri":[""],"itemData":{"id":2557,"type":"article-journal","title":"Interaction between fluids and vasoactive agents on mortality in septic shock: a multicenter, observational study","container-title":"Critical Care Medicine","page":"2158-2168","volume":"42","issue":"10","source":"PubMed","abstract":"OBJECTIVE: Fluids and vasoactive agents are both used to treat septic shock, but little is known about how they interact or the optimal way to administer them. We sought to determine how hospital mortality was influenced by combined use of these two treatments.\nDESIGN: Retrospective evaluation using multivariable logistic regression to evaluate the association between hospital mortality and categorical variables representing initiation of vasoactive agents and volumes of IV fluids given 0-1, 1-6, and 6-24 hours after onset, including interactions and adjusting for potential confounders.\nSETTING: ICUs of 24 hospitals in 3 countries.\nPATIENTS: Two thousand eight hundred forty-nine patients who survived more than 24 hours after after onset of septic shock, admitted between 1989 and 2007.\nINTERVENTIONS: None.\nMEASUREMENTS AND MAIN RESULTS: Fluids and vasoactive agents had strong, interacting associations with mortality (p < 0.0001). Mortality was lowest when vasoactive agents were begun 1-6 hours after onset, with more than 1 L of fluids in the initial hour after shock onset, more than 2.4 L from hours 1-6, and 1.6-3.5 L from 6 to 24 hours. The lowest mortality rates were associated with starting vasoactive agents 1-6 hours after onset.\nCONCLUSIONS: The focus during the first hour of resuscitation for septic shock should be aggressive fluid administration, only thereafter starting vasoactive agents, while continuing aggressive fluid administration. Starting vasoactive agents in the initial hour may be detrimental, and not all of that association is due to less fluids being given with such early initiation of vasoactive agents.","DOI":"10.1097/CCM.0000000000000520","ISSN":"1530-0293","note":"PMID: 25072761","shortTitle":"Interaction between fluids and vasoactive agents on mortality in septic shock","journalAbbreviation":"Crit. Care Med.","language":"eng","author":[{"family":"Waechter","given":"Jason"},{"family":"Kumar","given":"Anand"},{"family":"Lapinsky","given":"Stephen E."},{"family":"Marshall","given":"John"},{"family":"Dodek","given":"Peter"},{"family":"Arabi","given":"Yaseen"},{"family":"Parrillo","given":"Joseph E."},{"family":"Dellinger","given":"R. Phillip"},{"family":"Garland","given":"Allan"},{"literal":"Cooperative Antimicrobial Therapy of Septic Shock Database Research Group"}],"issued":{"date-parts":[["2014",10]]}}},{"id":2537,"uris":[""],"uri":[""],"itemData":{"id":2537,"type":"article-journal","title":"Early versus delayed administration of norepinephrine in patients with septic shock","container-title":"Critical Care (London, England)","page":"532","volume":"18","issue":"5","source":"PubMed","abstract":"INTRODUCTION: This study investigated the incidence of delayed norepinephrine administration following the onset of septic shock and its effect on hospital mortality.\nMETHODS: We conducted a retrospective cohort study using data from 213 adult septic shock patients treated at two general surgical intensive care units of a tertiary care hospital over a two year period. The primary outcome was 28-day mortality.\nRESULTS: The 28-day mortality was 37.6% overall. Among the 213 patients, a strong relationship between delayed initial norepinephrine administration and 28-day mortality was noted. The average time to initial norepinephrine administration was 3.1?±?2.5 hours. Every 1-hour delay in norepinephrine initiation during the first 6 hours after septic shock onset was associated with a 5.3% increase in mortality. Twenty-eight day mortality rates were significantly higher when norepinephrine administration was started more than or equal to 2 hours after septic shock onset (Late-NE) compared to less than 2 hours (Early-NE). Mean arterial pressures at 1, 2, 4, and 6 hours after septic shock onset were significantly higher and serum lactate levels at 2, 4, 6, and 8 hours were significantly lower in the Early-NE than the Late-NE group. The duration of hypotension and norepinephrine administration was significantly shorter and the quantity of norepinephrine administered in a 24-hour period was significantly less for the Early-NE group compared to the Late-NE group. The time to initial antimicrobial treatment was not significantly different between the Early-NE and Late-NE groups.\nCONCLUSION: Our results show that early administration of norepinephrine in septic shock patients is associated with an increased survival rate.","DOI":"10.1186/s13054-014-0532-y","ISSN":"1466-609X","note":"PMID: 25277635\nPMCID: PMC4194405","journalAbbreviation":"Crit Care","language":"eng","author":[{"family":"Bai","given":"Xiaowu"},{"family":"Yu","given":"Wenkui"},{"family":"Ji","given":"Wu"},{"family":"Lin","given":"Zhiliang"},{"family":"Tan","given":"Shanjun"},{"family":"Duan","given":"Kaipeng"},{"family":"Dong","given":"Yi"},{"family":"Xu","given":"Lin"},{"family":"Li","given":"Ning"}],"issued":{"date-parts":[["2014",10,3]]}}},{"id":2931,"uris":[""],"uri":[""],"itemData":{"id":2931,"type":"article-journal","title":"Fluid administration in severe sepsis and septic shock, patterns and outcomes: an analysis of a large national database","container-title":"Intensive Care Medicine","page":"625-632","volume":"43","issue":"5","source":"PubMed","abstract":"PURPOSE: The optimal strategy of fluid resuscitation in the early hours of severe sepsis and septic shock is controversial, with both an aggressive and conservative approach being recommended.\nMETHODS: We used the 2013 Premier Hospital Discharge database to analyse the administration of fluids on the first ICU day,?in 23,513 patients with severe sepsis and septic shock,?who were admitted to an ICU from the emergency department. Day?1 fluid was grouped into categories 1?L wide,?starting with 1-1.99?L up to ≥9?L,?to examine the effect of day?1 fluids on patient mortality. We built binary response models for hospital mortality and the propensity for receiving more than 5?L of fluids on day?1, using patient age and acute conditions present on admission. Patients were grouped by the requirement for mechanical ventilation and the presence or absence of shock. We assessed trends in the difference between actual and expected mortality,?in the low fluid range (1-5?L day?1 fluids) and the high fluid range (5 to ≥9?L day?1 fluids) categories,?using weighted linear regression controlling for the effects of sample size and variation within the day?1 fluid category.\nRESULTS: Day?1 fluid administration averaged 4.4?L being lowest in the group with no mechanical ventilation and no shock (3.6?L) and highest (5.4?L) in the group receiving mechanical ventilation and in shock. The administration of day?1 fluids was remarkably consistent on the basis of hospital size, teaching status, rural/urban location, and region of the country. The hospital mortality in the entire cohort was 25.8%, with a mean ICU and hospital length of stay of 5.1 and 9.1?days, respectively. In the entire cohort, low volume resuscitation (1-4.99?L) was associated with a small but significant reduction in mortality,?of -0.7% per litre (95% CI -1.0%, -0.4%; p?=?0.02). However, in patients receiving high volume resuscitation (5 to ≥9?L),?the mortality increased by 2.3% (95% CI 2.0, 2.5%; p?=?0.0003) for each additional litre above 5?L. Total hospital cost increased by $999 for each litre of fluid above 5?L (adjusted R (2)?=?92.7%, p?=?0.005).\nCONCLUSION: The mean amount of fluid administered to patients with severe sepsis and septic shock in the USA during the first ICU day is less than that recommended by the Surviving Sepsis Campaign guidelines. The administration of more than 5?L of fluid during the first ICU day is associated with a significantly increased risk of death and significantly higher?hospital costs.","DOI":"10.1007/s00134-016-4675-y","ISSN":"1432-1238","note":"PMID: 28130687","shortTitle":"Fluid administration in severe sepsis and septic shock, patterns and outcomes","journalAbbreviation":"Intensive Care Med","language":"eng","author":[{"family":"Marik","given":"Paul E."},{"family":"Linde-Zwirble","given":"Walter T."},{"family":"Bittner","given":"Edward A."},{"family":"Sahatjian","given":"Jennifer"},{"family":"Hansell","given":"Douglas"}],"issued":{"date-parts":[["2017",5]]}}}],"schema":""} 8–10.We developed the AI Clinician, a computational model using reinforcement learning, able to dynamically suggest optimal treatments for adult patients with sepsis in the intensive care unit (ICU). Reinforcement learning is a category of artificial intelligence tools where a virtual agent learns from trial-and-error an optimized set of rules – a policy – that maximizes an expected return ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"WjQFyxO3","properties":{"formattedCitation":"\\super 11,12\\nosupersub{}","plainCitation":"11,12","noteIndex":0},"citationItems":[{"id":348,"uris":[""],"uri":[""],"itemData":{"id":348,"type":"book","title":"Reinforcement Learning: An Introduction","publisher":"A Bradford Book","publisher-place":"Cambridge, Mass","number-of-pages":"322","source":"Amazon","event-place":"Cambridge, Mass","abstract":"Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.","ISBN":"978-0-262-19398-6","shortTitle":"Reinforcement Learning","language":"English","author":[{"family":"Sutton","given":"Richard S."},{"family":"Barto","given":"Andrew G."}],"issued":{"date-parts":[["1998",3,1]]}}},{"id":2431,"uris":[""],"uri":[""],"itemData":{"id":2431,"type":"article-journal","title":"Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach","container-title":"Artificial Intelligence in Medicine","page":"9-19","volume":"57","issue":"1","source":"ScienceDirect","abstract":"Objective\nIn the modern healthcare system, rapidly expanding costs/complexity, the growing myriad of treatment options, and exploding information streams that often do not effectively reach the front lines hinder the ability to choose optimal treatment decisions over time. The goal in this paper is to develop a general purpose (non-disease-specific) computational/artificial intelligence (AI) framework to address these challenges. This framework serves two potential functions: (1) a simulation environment for exploring various healthcare policies, payment methodologies, etc., and (2) the basis for clinical artificial intelligence – an AI that can “think like a doctor”.\nMethods\nThis approach combines Markov decision processes and dynamic decision networks to learn from clinical data and develop complex plans via simulation of alternative sequential decision paths while capturing the sometimes conflicting, sometimes synergistic interactions of various components in the healthcare system. It can operate in partially observable environments (in the case of missing observations or data) by maintaining belief states about patient health status and functions as an online agent that plans and re-plans as actions are performed and new observations are obtained. This framework was evaluated using real patient data from an electronic health record.\nResults\nThe results demonstrate the feasibility of this approach; such an AI framework easily outperforms the current treatment-as-usual (TAU) case-rate/fee-for-service models of healthcare. The cost per unit of outcome change (CPUC) was $189 vs. $497 for AI vs. TAU (where lower is considered optimal) – while at the same time the AI approach could obtain a 30–35% increase in patient outcomes. Tweaking certain AI model parameters could further enhance this advantage, obtaining approximately 50% more improvement (outcome change) for roughly half the costs.\nConclusion\nGiven careful design and problem formulation, an AI simulation framework can approximate optimal decisions even in complex and uncertain environments. Future work is described that outlines potential lines of research and integration of machine learning algorithms for personalized medicine.","DOI":"10.1016/j.artmed.2012.12.003","ISSN":"0933-3657","shortTitle":"Artificial intelligence framework for simulating clinical decision-making","journalAbbreviation":"Artificial Intelligence in Medicine","author":[{"family":"Bennett","given":"Casey C."},{"family":"Hauser","given":"Kris"}],"issued":{"date-parts":[["2013",1]]}}}],"schema":""} 11,12. Similarly, a clinician’s goal is to make therapeutic decisions in order to maximize a patient’s probability of a good outcome ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"rXoEVqAF","properties":{"formattedCitation":"\\super 12,13\\nosupersub{}","plainCitation":"12,13","noteIndex":0},"citationItems":[{"id":2431,"uris":[""],"uri":[""],"itemData":{"id":2431,"type":"article-journal","title":"Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach","container-title":"Artificial Intelligence in Medicine","page":"9-19","volume":"57","issue":"1","source":"ScienceDirect","abstract":"Objective\nIn the modern healthcare system, rapidly expanding costs/complexity, the growing myriad of treatment options, and exploding information streams that often do not effectively reach the front lines hinder the ability to choose optimal treatment decisions over time. The goal in this paper is to develop a general purpose (non-disease-specific) computational/artificial intelligence (AI) framework to address these challenges. This framework serves two potential functions: (1) a simulation environment for exploring various healthcare policies, payment methodologies, etc., and (2) the basis for clinical artificial intelligence – an AI that can “think like a doctor”.\nMethods\nThis approach combines Markov decision processes and dynamic decision networks to learn from clinical data and develop complex plans via simulation of alternative sequential decision paths while capturing the sometimes conflicting, sometimes synergistic interactions of various components in the healthcare system. It can operate in partially observable environments (in the case of missing observations or data) by maintaining belief states about patient health status and functions as an online agent that plans and re-plans as actions are performed and new observations are obtained. This framework was evaluated using real patient data from an electronic health record.\nResults\nThe results demonstrate the feasibility of this approach; such an AI framework easily outperforms the current treatment-as-usual (TAU) case-rate/fee-for-service models of healthcare. The cost per unit of outcome change (CPUC) was $189 vs. $497 for AI vs. TAU (where lower is considered optimal) – while at the same time the AI approach could obtain a 30–35% increase in patient outcomes. Tweaking certain AI model parameters could further enhance this advantage, obtaining approximately 50% more improvement (outcome change) for roughly half the costs.\nConclusion\nGiven careful design and problem formulation, an AI simulation framework can approximate optimal decisions even in complex and uncertain environments. Future work is described that outlines potential lines of research and integration of machine learning algorithms for personalized medicine.","DOI":"10.1016/j.artmed.2012.12.003","ISSN":"0933-3657","shortTitle":"Artificial intelligence framework for simulating clinical decision-making","journalAbbreviation":"Artificial Intelligence in Medicine","author":[{"family":"Bennett","given":"Casey C."},{"family":"Hauser","given":"Kris"}],"issued":{"date-parts":[["2013",1]]}}},{"id":187,"uris":[""],"uri":[""],"itemData":{"id":187,"type":"chapter","title":"Modeling Medical Treatment Using Markov Decision Processes","container-title":"Operations Research and Health Care","collection-title":"International Series in Operations Research & Management Science","collection-number":"70","publisher":"Springer US","page":"593-612","source":"link.","URL":"","ISBN":"978-1-4020-7629-9","language":"en","author":[{"family":"Schaefer","given":"Andrew J."},{"family":"Bailey","given":"Matthew D."},{"family":"Shechter","given":"Steven M."},{"family":"Roberts","given":"Mark S."}],"editor":[{"family":"Brandeau","given":"Margaret L."},{"family":"Sainfort","given":"Fran?ois"},{"family":"Pierskalla","given":"William P."}],"issued":{"date-parts":[["2005"]]},"accessed":{"date-parts":[["2015",8,31]]}}}],"schema":""} 12,13. Reinforcement learning has many desirable properties that may help medical decision-making. Their intrinsic design can handle sparse reward signals, which makes them well suited to overcome the complexity related to the heterogeneity of patient responses to medical interventions and the delayed indications of the efficacy of treatments ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"a210713ke6f","properties":{"formattedCitation":"\\super 11\\nosupersub{}","plainCitation":"11","noteIndex":0},"citationItems":[{"id":348,"uris":[""],"uri":[""],"itemData":{"id":348,"type":"book","title":"Reinforcement Learning: An Introduction","publisher":"A Bradford Book","publisher-place":"Cambridge, Mass","number-of-pages":"322","source":"Amazon","event-place":"Cambridge, Mass","abstract":"Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.","ISBN":"978-0-262-19398-6","shortTitle":"Reinforcement Learning","language":"English","author":[{"family":"Sutton","given":"Richard S."},{"family":"Barto","given":"Andrew G."}],"issued":{"date-parts":[["1998",3,1]]}}}],"schema":""} 11. Importantly, these algorithms are able to infer optimal decisions from suboptimal training examples. Reinforcement learning has been successfully applied in the past to medical problems, such as diabetic retinopathy and mechanical ventilation in the ICU ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"2903a9el51","properties":{"formattedCitation":"\\super 14,15\\nosupersub{}","plainCitation":"14,15","noteIndex":0},"citationItems":[{"id":2662,"uris":[""],"uri":[""],"itemData":{"id":2662,"type":"article-journal","title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs","container-title":"JAMA","page":"2402-2410","volume":"316","issue":"22","source":"","abstract":"This study assesses the sensitivity and specificity of an algorithm based on deep machine learning for automated detection of diabetic retinopathy and diabetic macular edema in retinal fundus photographs.","DOI":"10.1001/jama.2016.17216","ISSN":"0098-7484","journalAbbreviation":"JAMA","author":[{"family":"Gulshan","given":"Varun"},{"family":"Peng","given":"Lily"},{"family":"Coram","given":"Marc"},{"family":"Stumpe","given":"Martin C."},{"family":"Wu","given":"Derek"},{"family":"Narayanaswamy","given":"Arunachalam"},{"family":"Venugopalan","given":"Subhashini"},{"family":"Widner","given":"Kasumi"},{"family":"Madams","given":"Tom"},{"family":"Cuadros","given":"Jorge"},{"family":"Kim","given":"Ramasamy"},{"family":"Raman","given":"Rajiv"},{"family":"Nelson","given":"Philip C."},{"family":"Mega","given":"Jessica L."},{"family":"Webster","given":"Dale R."}],"issued":{"date-parts":[["2016",12,13]]}}},{"id":2904,"uris":[""],"uri":[""],"itemData":{"id":2904,"type":"article-journal","title":"A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units","container-title":"arXiv:1704.06300 [cs]","source":"","abstract":"The management of invasive mechanical ventilation, and the regulation of sedation and analgesia during ventilation, constitutes a major part of the care of patients admitted to intensive care units. Both prolonged dependence on mechanical ventilation and premature extubation are associated with increased risk of complications and higher hospital costs, but clinical opinion on the best protocol for weaning patients off of a ventilator varies. This work aims to develop a decision support tool that uses available patient information to predict time-to-extubation readiness and to recommend a personalized regime of sedation dosage and ventilator support. To this end, we use off-policy reinforcement learning algorithms to determine the best action at a given patient state from sub-optimal historical ICU data. We compare treatment policies from fitted Q-iteration with extremely randomized trees and with feedforward neural networks, and demonstrate that the policies learnt show promise in recommending weaning protocols with improved outcomes, in terms of minimizing rates of reintubation and regulating physiological stability.","URL":"","note":"arXiv: 1704.06300","author":[{"family":"Prasad","given":"Niranjani"},{"family":"Cheng","given":"Li-Fang"},{"family":"Chivers","given":"Corey"},{"family":"Draugelis","given":"Michael"},{"family":"Engelhardt","given":"Barbara E."}],"issued":{"date-parts":[["2017",4,20]]}}}],"schema":""} 14,15.Our AI Clinician was built and validated on two large non-overlapping ICU databases, containing data routinely collected from adult patients in the U.S.A. The Medical Information Mart for Intensive Care version III (MIMIC-III) ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"a1c7m7llicq","properties":{"formattedCitation":"\\super 16\\nosupersub{}","plainCitation":"16","noteIndex":0},"citationItems":[{"id":2458,"uris":[""],"uri":[""],"itemData":{"id":2458,"type":"article-journal","title":"MIMIC-III, a freely accessible critical care database","container-title":"Scientific Data","page":"160035","volume":"3","source":"PubMed","abstract":"MIMIC-III ('Medical Information Mart for Intensive Care') is a large, single-center database comprising information relating to patients admitted to critical care units at a large tertiary care hospital. Data includes vital signs, medications, laboratory measurements, observations and notes charted by care providers, fluid balance, procedure codes, diagnostic codes, imaging reports, hospital length of stay, survival data, and more. The database supports applications including academic and industrial research, quality improvement initiatives, and higher education coursework.","DOI":"10.1038/sdata.2016.35","ISSN":"2052-4463","note":"PMID: 27219127","journalAbbreviation":"Sci Data","language":"ENG","author":[{"family":"Johnson","given":"Alistair E. W."},{"family":"Pollard","given":"Tom J."},{"family":"Shen","given":"Lu"},{"family":"Lehman","given":"Li-Wei H."},{"family":"Feng","given":"Mengling"},{"family":"Ghassemi","given":"Mohammad"},{"family":"Moody","given":"Benjamin"},{"family":"Szolovits","given":"Peter"},{"family":"Anthony Celi","given":"Leo"},{"family":"Mark","given":"Roger G."}],"issued":{"date-parts":[["2016"]]}}}],"schema":""} 16 was used for model development and the eICU Research Institute Database (eRI) for model testing. In both datasets, we included adult patients fulfilling the international consensus sepsis-3 criteria ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"OhuYKFvm","properties":{"formattedCitation":"\\super 7\\nosupersub{}","plainCitation":"7","noteIndex":0},"citationItems":[{"id":2330,"uris":[""],"uri":[""],"itemData":{"id":2330,"type":"article-journal","title":"The third international consensus definitions for sepsis and septic shock (sepsis-3)","container-title":"JAMA","page":"801-810","volume":"315","issue":"8","source":"Silverchair","abstract":"Importance?\nDefinitions of sepsis and septic shock were last revised in 2001. Considerable advances have since been made into the pathobiology (changes in organ function, morphology, cell biology, biochemistry, immunology, and circulation), management, and epidemiology of sepsis, suggesting the need for reexamination.Objective\nTo evaluate and, as needed, update definitions for sepsis and septic shock.Process\nA task force (n?=?19) with expertise in sepsis pathobiology, clinical trials, and epidemiology was convened by the Society of Critical Care Medicine and the European Society of Intensive Care Medicine. Definitions and clinical criteria were generated through meetings, Delphi processes, analysis of electronic health record databases, and voting, followed by circulation to international professional societies, requesting peer review and endorsement (by 31 societies listed in the Acknowledgment).Key Findings From Evidence Synthesis\nLimitations of previous definitions included an excessive focus on inflammation, the misleading model that sepsis follows a continuum through severe sepsis to shock, and inadequate specificity and sensitivity of the systemic inflammatory response syndrome (SIRS) criteria. Multiple definitions and terminologies are currently in use for sepsis, septic shock, and organ dysfunction, leading to discrepancies in reported incidence and observed mortality. The task force concluded the term severe sepsis was redundant.Recommendations\nSepsis should be defined as life-threatening organ dysfunction caused by a dysregulated host response to infection. For clinical operationalization, organ dysfunction can be represented by an increase in the Sequential [Sepsis-related] Organ Failure Assessment (SOFA) score of 2 points or more, which is associated with an in-hospital mortality greater than 10%. Septic shock should be defined as a subset of sepsis in which particularly profound circulatory, cellular, and metabolic abnormalities are associated with a greater risk of mortality than with sepsis alone. Patients with septic shock can be clinically identified by a vasopressor requirement to maintain a mean arterial pressure of 65 mm Hg or greater and serum lactate level greater than 2 mmol/L (>18 mg/dL) in the absence of hypovolemia. This combination is associated with hospital mortality rates greater than 40%. In out-of-hospital, emergency department, or general hospital ward settings, adult patients with suspected infection can be rapidly identified as being more likely to have poor outcomes typical of sepsis if they have at least 2 of the following clinical criteria that together constitute a new bedside clinical score termed quickSOFA (qSOFA): respiratory rate of 22/min or greater, altered mentation, or systolic blood pressure of 100 mm Hg or less.Conclusions and Relevance\nThese updated definitions and clinical criteria should replace previous definitions, offer greater consistency for epidemiologic studies and clinical trials, and facilitate earlier recognition and more timely management of patients with sepsis or at risk of developing sepsis.","DOI":"10.1001/jama.2016.0287","ISSN":"0098-7484","journalAbbreviation":"JAMA","author":[{"literal":"Singer M"},{"literal":"Deutschman CS"},{"literal":"Seymour C"},{"literal":"et al"}],"issued":{"date-parts":[["2016",2,23]]}}}],"schema":""} 7. After exclusion of non-eligible cases, we included 17,083 admissions (88.4% of eligible sepsis patients) from MIMIC-III from 5 separate ICUs in one tertiary teaching hospital and 79,073 admissions (73.6% of eligible sepsis patients) from 128 different hospitals from eRI (Supplementary Figure 1). Patient demographics and clinical characteristics are shown in Table 1 and Supplementary Table 1.In both datasets, we extracted a set of 48 variables, including demographics, Elixhauser premorbid status ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"a1ou9p3bec3","properties":{"formattedCitation":"\\super 17\\nosupersub{}","plainCitation":"17","noteIndex":0},"citationItems":[{"id":2892,"uris":[""],"uri":[""],"itemData":{"id":2892,"type":"article-journal","title":"Comorbidity measures for use with administrative data","container-title":"Medical Care","page":"8-27","volume":"36","issue":"1","source":"PubMed","abstract":"OBJECTIVES: This study attempts to develop a comprehensive set of comorbidity measures for use with large administrative inpatient datasets.\nMETHODS: The study involved clinical and empirical review of comorbidity measures, development of a framework that attempts to segregate comorbidities from other aspects of the patient's condition, development of a comorbidity algorithm, and testing on heterogeneous and homogeneous patient groups. Data were drawn from all adult, nonmaternal inpatients from 438 acute care hospitals in California in 1992 (n = 1,779,167). Outcome measures were those commonly available in administrative data: length of stay, hospital charges, and in-hospital death.\nRESULTS: A comprehensive set of 30 comorbidity measures was developed. The comorbidities were associated with substantial increases in length of stay, hospital charges, and mortality both for heterogeneous and homogeneous disease groups. Several comorbidities are described that are important predictors of outcomes, yet commonly are not measured. These include mental disorders, drug and alcohol abuse, obesity, coagulopathy, weight loss, and fluid and electrolyte disorders.\nCONCLUSIONS: The comorbidities had independent effects on outcomes and probably should not be simplified as an index because they affect outcomes differently among different patient groups. The present method addresses some of the limitations of previous measures. It is based on a comprehensive approach to identifying comorbidities and separates them from the primary reason for hospitalization, resulting in an expanded set of comorbidities that easily is applied without further refinement to administrative data for a wide range of diseases.","ISSN":"0025-7079","note":"PMID: 9431328","journalAbbreviation":"Med Care","language":"eng","author":[{"family":"Elixhauser","given":"A."},{"family":"Steiner","given":"C."},{"family":"Harris","given":"D. R."},{"family":"Coffey","given":"R. M."}],"issued":{"date-parts":[["1998",1]]}}}],"schema":""} 17, vital signs, laboratory values, fluids and vasopressors received (Supplementary Table 2). Patients’ data were coded as multidimensional discrete time series with 4-hour time steps, and we included up to 72 hours of measurements per patient, around the estimated time of onset of sepsis. The total volume of intravenous fluids and maximum dose of vasopressors administered over each 4-hour period defined the medical treatments of interest. The model aims at optimizing patient mortality, so a reward was associated to survival and a penalty to death.A Markov decision process (MDP) was used to model the patient environment and trajectories ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"so4uf1194","properties":{"formattedCitation":"\\super 11,18\\nosupersub{}","plainCitation":"11,18","noteIndex":0},"citationItems":[{"id":269,"uris":[""],"uri":[""],"itemData":{"id":269,"type":"book","title":"Markov Decision Processes: Discrete Stochastic Dynamic Programming","publisher":"Wiley","number-of-pages":"672","source":"Google Books","abstract":"An up-to-date, unified and rigorous treatment of theoretical, computational and applied research on Markov decision process models. Concentrates on infinite-horizon discrete-time models. Discusses arbitrary state spaces, finite-horizon and continuous-time discrete-state models. Also covers modified policy iteration, multichain models with average reward criterion and sensitive optimality. Features a wealth of figures which illustrate examples and an extensive bibliography.","ISBN":"978-0-471-61977-2","shortTitle":"Markov Decision Processes","language":"en","author":[{"family":"Puterman","given":"Martin L."}],"issued":{"date-parts":[["1994",4,29]]}}},{"id":348,"uris":[""],"uri":[""],"itemData":{"id":348,"type":"book","title":"Reinforcement Learning: An Introduction","publisher":"A Bradford Book","publisher-place":"Cambridge, Mass","number-of-pages":"322","source":"Amazon","event-place":"Cambridge, Mass","abstract":"Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.","ISBN":"978-0-262-19398-6","shortTitle":"Reinforcement Learning","language":"English","author":[{"family":"Sutton","given":"Richard S."},{"family":"Barto","given":"Andrew G."}],"issued":{"date-parts":[["1998",3,1]]}}}],"schema":""} 11,18. The various elements of the model were defined using patient data time series from the training set (a random sample of 80% of MIMIC-III, see Figure 1). We deployed the AI Clinician to solve the MDP and predict outcomes of treatment strategies. First, we evaluated the actual treatments of clinicians, by analysing all the prescriptions and computing the average return of each treatment option, which can take values from -100 to +100 in our model. Then, the MDP was solved using policy iteration, which identified the treatments that maximised return, that is, the expected 90-day survival of patients in the MIMIC-III cohort ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"lc5hm4p5g","properties":{"formattedCitation":"\\super 11\\nosupersub{}","plainCitation":"11","noteIndex":0},"citationItems":[{"id":348,"uris":[""],"uri":[""],"itemData":{"id":348,"type":"book","title":"Reinforcement Learning: An Introduction","publisher":"A Bradford Book","publisher-place":"Cambridge, Mass","number-of-pages":"322","source":"Amazon","event-place":"Cambridge, Mass","abstract":"Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.","ISBN":"978-0-262-19398-6","shortTitle":"Reinforcement Learning","language":"English","author":[{"family":"Sutton","given":"Richard S."},{"family":"Barto","given":"Andrew G."}],"issued":{"date-parts":[["1998",3,1]]}}}],"schema":""} 11. The resultant policy is referred to thereafter as the “AI policy”. Evaluating the performance of this new AI policy using the trajectories of patients generated by another policy (the clinicians’ policy) is termed off-policy evaluation ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"Qct7NPTf","properties":{"formattedCitation":"\\super 19\\uc0\\u8211{}21\\nosupersub{}","plainCitation":"19–21","noteIndex":0},"citationItems":[{"id":3039,"uris":[""],"uri":[""],"itemData":{"id":3039,"type":"paper-conference","title":"High-Confidence Off-Policy Evaluation","container-title":"Twenty-Ninth AAAI Conference on Artificial Intelligence","source":"","event":"Twenty-Ninth AAAI Conference on Artificial Intelligence","abstract":"Many reinforcement learning algorithms use trajectories collected from the execution of one or more policies to propose a new policy. Because execution of a bad policy can be costly or dangerous, techniques for evaluating the performance of the new policy without requiring its execution have been of recent interest in industry. Such off-policy evaluation methods, which estimate the performance of a policy using trajectories collected from the execution of other policies, heretofore have not provided confidences regarding the accuracy of their estimates. In this paper we propose an off-policy method for computing a lower confidence bound on the expected return of a policy.","URL":"","language":"en","author":[{"family":"Thomas","given":"Philip S."},{"family":"Theocharous","given":"Georgios"},{"family":"Ghavamzadeh","given":"Mohammad"}],"issued":{"date-parts":[["2015",2,21]]},"accessed":{"date-parts":[["2018",2,14]]}}},{"id":3032,"uris":[""],"uri":[""],"itemData":{"id":3032,"type":"article-journal","title":"Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation","container-title":"arXiv:1606.06126 [cs, stat]","source":"","abstract":"For an autonomous agent, executing a poor policy may be costly or even dangerous. For such agents, it is desirable to determine confidence interval lower bounds on the performance of any given policy without executing said policy. Current methods for exact high confidence off-policy evaluation that use importance sampling require a substantial amount of data to achieve a tight lower bound. Existing model-based methods only address the problem in discrete state spaces. Since exact bounds are intractable for many domains we trade off strict guarantees of safety for more data-efficient approximate bounds. In this context, we propose two bootstrapping off-policy evaluation methods which use learned MDP transition models in order to estimate lower confidence bounds on policy performance with limited data in both continuous and discrete state spaces. Since direct use of a model may introduce bias, we derive a theoretical upper bound on model bias for when the model transition function is estimated with i.i.d. trajectories. This bound broadens our understanding of the conditions under which model-based methods have high bias. Finally, we empirically evaluate our proposed methods and analyze the settings in which different bootstrapping off-policy confidence interval methods succeed and fail.","URL":"","note":"arXiv: 1606.06126","shortTitle":"Bootstrapping with Models","author":[{"family":"Hanna","given":"Josiah P."},{"family":"Stone","given":"Peter"},{"family":"Niekum","given":"Scott"}],"issued":{"date-parts":[["2016",6,20]]},"accessed":{"date-parts":[["2018",2,14]]}}},{"id":3036,"uris":[""],"uri":[""],"itemData":{"id":3036,"type":"paper-conference","title":"High Confidence Policy Improvement","container-title":"PMLR","page":"2380-2388","source":"proceedings.mlr.press","event":"International Conference on Machine Learning","abstract":"We present a batch reinforcement learning (RL) algorithm that provides probabilistic guarantees about the quality of each policy that it proposes, and which has no hyper-parameter that requires exp...","URL":"","language":"en","author":[{"family":"Thomas","given":"Philip"},{"family":"Theocharous","given":"Georgios"},{"family":"Ghavamzadeh","given":"Mohammad"}],"issued":{"date-parts":[["2015",6,1]]},"accessed":{"date-parts":[["2018",2,14]]}}}],"schema":""} 19–21. It was crucial to obtain reliable estimates of the performance of this new policy without deploying it, since executing a bad policy would be dangerous for patients ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"atj2trcc2p","properties":{"formattedCitation":"\\super 19,20\\nosupersub{}","plainCitation":"19,20","noteIndex":0},"citationItems":[{"id":3032,"uris":[""],"uri":[""],"itemData":{"id":3032,"type":"article-journal","title":"Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation","container-title":"arXiv:1606.06126 [cs, stat]","source":"","abstract":"For an autonomous agent, executing a poor policy may be costly or even dangerous. For such agents, it is desirable to determine confidence interval lower bounds on the performance of any given policy without executing said policy. Current methods for exact high confidence off-policy evaluation that use importance sampling require a substantial amount of data to achieve a tight lower bound. Existing model-based methods only address the problem in discrete state spaces. Since exact bounds are intractable for many domains we trade off strict guarantees of safety for more data-efficient approximate bounds. In this context, we propose two bootstrapping off-policy evaluation methods which use learned MDP transition models in order to estimate lower confidence bounds on policy performance with limited data in both continuous and discrete state spaces. Since direct use of a model may introduce bias, we derive a theoretical upper bound on model bias for when the model transition function is estimated with i.i.d. trajectories. This bound broadens our understanding of the conditions under which model-based methods have high bias. Finally, we empirically evaluate our proposed methods and analyze the settings in which different bootstrapping off-policy confidence interval methods succeed and fail.","URL":"","note":"arXiv: 1606.06126","shortTitle":"Bootstrapping with Models","author":[{"family":"Hanna","given":"Josiah P."},{"family":"Stone","given":"Peter"},{"family":"Niekum","given":"Scott"}],"issued":{"date-parts":[["2016",6,20]]},"accessed":{"date-parts":[["2018",2,14]]}}},{"id":3039,"uris":[""],"uri":[""],"itemData":{"id":3039,"type":"paper-conference","title":"High-Confidence Off-Policy Evaluation","container-title":"Twenty-Ninth AAAI Conference on Artificial Intelligence","source":"","event":"Twenty-Ninth AAAI Conference on Artificial Intelligence","abstract":"Many reinforcement learning algorithms use trajectories collected from the execution of one or more policies to propose a new policy. Because execution of a bad policy can be costly or dangerous, techniques for evaluating the performance of the new policy without requiring its execution have been of recent interest in industry. Such off-policy evaluation methods, which estimate the performance of a policy using trajectories collected from the execution of other policies, heretofore have not provided confidences regarding the accuracy of their estimates. In this paper we propose an off-policy method for computing a lower confidence bound on the expected return of a policy.","URL":"","language":"en","author":[{"family":"Thomas","given":"Philip S."},{"family":"Theocharous","given":"Georgios"},{"family":"Ghavamzadeh","given":"Mohammad"}],"issued":{"date-parts":[["2015",2,21]]},"accessed":{"date-parts":[["2018",2,14]]}}}],"schema":""} 19,20. Therefore, we implemented a type of high confidence off-policy evaluation (HCOPE) method (weighted importance sampling, WIS), and used bootstrapping to estimate the true distribution of the policy value in the MIMIC-III 20% validation set (Figure 2b, Supplementary Figure 1) ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"av04er6442","properties":{"formattedCitation":"\\super 20,21\\nosupersub{}","plainCitation":"20,21","noteIndex":0},"citationItems":[{"id":3032,"uris":[""],"uri":[""],"itemData":{"id":3032,"type":"article-journal","title":"Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation","container-title":"arXiv:1606.06126 [cs, stat]","source":"","abstract":"For an autonomous agent, executing a poor policy may be costly or even dangerous. For such agents, it is desirable to determine confidence interval lower bounds on the performance of any given policy without executing said policy. Current methods for exact high confidence off-policy evaluation that use importance sampling require a substantial amount of data to achieve a tight lower bound. Existing model-based methods only address the problem in discrete state spaces. Since exact bounds are intractable for many domains we trade off strict guarantees of safety for more data-efficient approximate bounds. In this context, we propose two bootstrapping off-policy evaluation methods which use learned MDP transition models in order to estimate lower confidence bounds on policy performance with limited data in both continuous and discrete state spaces. Since direct use of a model may introduce bias, we derive a theoretical upper bound on model bias for when the model transition function is estimated with i.i.d. trajectories. This bound broadens our understanding of the conditions under which model-based methods have high bias. Finally, we empirically evaluate our proposed methods and analyze the settings in which different bootstrapping off-policy confidence interval methods succeed and fail.","URL":"","note":"arXiv: 1606.06126","shortTitle":"Bootstrapping with Models","author":[{"family":"Hanna","given":"Josiah P."},{"family":"Stone","given":"Peter"},{"family":"Niekum","given":"Scott"}],"issued":{"date-parts":[["2016",6,20]]},"accessed":{"date-parts":[["2018",2,14]]}}},{"id":3036,"uris":[""],"uri":[""],"itemData":{"id":3036,"type":"paper-conference","title":"High Confidence Policy Improvement","container-title":"PMLR","page":"2380-2388","source":"proceedings.mlr.press","event":"International Conference on Machine Learning","abstract":"We present a batch reinforcement learning (RL) algorithm that provides probabilistic guarantees about the quality of each policy that it proposes, and which has no hyper-parameter that requires exp...","URL":"","language":"en","author":[{"family":"Thomas","given":"Philip"},{"family":"Theocharous","given":"Georgios"},{"family":"Ghavamzadeh","given":"Mohammad"}],"issued":{"date-parts":[["2015",6,1]]},"accessed":{"date-parts":[["2018",2,14]]}}}],"schema":""} 20,21. We built 500 different models using 500 different clustering solutions of the training data, and the selected final model maximised the 95% confidence lower bound of the AI policy ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"a2e6dglhvcf","properties":{"formattedCitation":"\\super 20\\nosupersub{}","plainCitation":"20","noteIndex":0},"citationItems":[{"id":3032,"uris":[""],"uri":[""],"itemData":{"id":3032,"type":"article-journal","title":"Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation","container-title":"arXiv:1606.06126 [cs, stat]","source":"","abstract":"For an autonomous agent, executing a poor policy may be costly or even dangerous. For such agents, it is desirable to determine confidence interval lower bounds on the performance of any given policy without executing said policy. Current methods for exact high confidence off-policy evaluation that use importance sampling require a substantial amount of data to achieve a tight lower bound. Existing model-based methods only address the problem in discrete state spaces. Since exact bounds are intractable for many domains we trade off strict guarantees of safety for more data-efficient approximate bounds. In this context, we propose two bootstrapping off-policy evaluation methods which use learned MDP transition models in order to estimate lower confidence bounds on policy performance with limited data in both continuous and discrete state spaces. Since direct use of a model may introduce bias, we derive a theoretical upper bound on model bias for when the model transition function is estimated with i.i.d. trajectories. This bound broadens our understanding of the conditions under which model-based methods have high bias. Finally, we empirically evaluate our proposed methods and analyze the settings in which different bootstrapping off-policy confidence interval methods succeed and fail.","URL":"","note":"arXiv: 1606.06126","shortTitle":"Bootstrapping with Models","author":[{"family":"Hanna","given":"Josiah P."},{"family":"Stone","given":"Peter"},{"family":"Niekum","given":"Scott"}],"issued":{"date-parts":[["2016",6,20]]},"accessed":{"date-parts":[["2018",2,14]]}}}],"schema":""} 20. Figure 2a shows that this bound consistently exceeded the 95% confidence upper bound of the clinicians’ policy, provided that enough models were built. This model selection method maximizes the theoretical statistical safety of the new AI policy. The chosen AI policy was then tested on the independent eRI dataset.Good model calibration was confirmed by plotting the relationship between the return of the clinicians’ policy and patients’ 90-day mortality (Figure 2c). In Figure 2d, we show the average return measured in survivors and non-survivors. Figure 3a shows the distribution of the estimated value of the clinicians’ policy and the AI policy in the selected final model tested on the eRI cohort. Using bootstrapping with 2,000 resamplings, the median value of clinicians’ policy and the AI policy were estimated at 56.9 (interquartile range 54.7-58.8) and 84.5 (interquartile range 84.3-87.7), respectively. Figures 3b and 3c show the distribution of treatment doses according to clinicians’ and AI policies. On average, the AI Clinician recommended lower doses of intravenous fluids and higher doses of vasopressors, compared to clinicians’ actual treatments. While the proportion of time the eRI patients received vasopressors was only 17%, that would have been 30% if following the AI Clinician.We further validated the model by analysing patient mortality when the dose actually administered corresponded to or differed from the dose suggested by the AI Clinician. Fifty eight percent of the time, the patients received a dose of vasopressor very close to the suggested dose, within 0.02 mcg/kg/min or 10% (whichever was smaller). For fluids, patients received the suggested dose approximately 36% of the time, within 10 mL/hour or 10%. These patients, who received doses similar to the doses recommended by the AI Clinician, had the lowest mortality. When the actual dose given was different from the suggested dose, clinicians gave more or less fluids in similar proportions, and less vasopressors 75% of the time. Giving more or less than the AI policy of either treatment was associated with increasing mortality rates, in a dose-dependent fashion. Figures 3d and 3e demonstrate this association, when the dose gap was averaged at the patient level. The median dose deficit in patients who received too little vasopressors was 0.13 mcg/kg/min (interquartile range 0.04-0.27 mcg/kg/min). Using a random forest classification model, we gained some insight into the model representations and interpretability by estimating the relative importance of the model parameters for predicting the administration of both medications (Supplementary Figure 4). This confirmed that the decisions suggested by the AI Clinician were clinically interpretable and relied primarily on sensible clinical and biological parameters.Here we demonstrate how reinforcement learning could be applied to solve a complex medical problem, and suggest individualized and clinically interpretable treatment strategies for sepsis. In an independent cohort, the patients that had received the treatments suggested by the AI Clinician had the lowest mortality rate. When clinicians’ actual treatments varied from the AI Clinician’s suggested policy, this was most commonly to administer too little vasopressor. Early use of low-dose vasopressor has been suggested in sepsis ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"F9UlfytL","properties":{"formattedCitation":"\\super 4,5,8,9\\nosupersub{}","plainCitation":"4,5,8,9","noteIndex":0},"citationItems":[{"id":2332,"uris":[""],"uri":[""],"itemData":{"id":2332,"type":"article-journal","title":"The demise of early goal-directed therapy for severe sepsis and septic shock","container-title":"Acta Anaesthesiologica Scandinavica","page":"561-567","volume":"59","issue":"5","source":"Wiley Online Library","abstract":"A protocol for the quantitative resuscitation of severe sepsis and septic shock known as early goal-directed therapy (EGDT) was published in 2001. Despite serious limitations, this study became widely adopted around the world and formed the basis of the Surviving Sepsis Campaign 6?h resuscitation bundle. Subsequently, a large number of observational before-and-after studies were published which demonstrated that EGDT reduced mortality. However, during this time period, there has been a substantial reduction in the mortality from sepsis in many Western nations that appears unrelated to EGDT. Recently, the Protocolized Care for Early Septic Shock (ProCESS) and The Australasian Resuscitation in Sepsis Evaluation (ARISE) trials failed to demonstrate any outcome benefit from EGDT. These two large, multicenter, randomized controlled studies raise serious questions regarding the validity of the original EGDT study and the scientific rigor of the uncontrolled, largely retrospective before–after clinical studies. Furthermore, accruing data suggest an association between the amount of fluid administered in the first 72?h and the mortality of patients with severe sepsis. Patients in all arms of the ProCESS and ARISE trials received substantial and nearly equivalent amounts of fluid. It is proposed that a more conservative fluid strategy and the earlier use of norepinephrine in patients with septic shock may be associated with further improvements in the outcome of patients with sepsis.","DOI":"10.1111/aas.12479","ISSN":"1399-6576","journalAbbreviation":"Acta Anaesthesiol Scand","language":"en","author":[{"family":"Marik","given":"P. E."}],"issued":{"date-parts":[["2015",5,1]]}}},{"id":2557,"uris":[""],"uri":[""],"itemData":{"id":2557,"type":"article-journal","title":"Interaction between fluids and vasoactive agents on mortality in septic shock: a multicenter, observational study","container-title":"Critical Care Medicine","page":"2158-2168","volume":"42","issue":"10","source":"PubMed","abstract":"OBJECTIVE: Fluids and vasoactive agents are both used to treat septic shock, but little is known about how they interact or the optimal way to administer them. We sought to determine how hospital mortality was influenced by combined use of these two treatments.\nDESIGN: Retrospective evaluation using multivariable logistic regression to evaluate the association between hospital mortality and categorical variables representing initiation of vasoactive agents and volumes of IV fluids given 0-1, 1-6, and 6-24 hours after onset, including interactions and adjusting for potential confounders.\nSETTING: ICUs of 24 hospitals in 3 countries.\nPATIENTS: Two thousand eight hundred forty-nine patients who survived more than 24 hours after after onset of septic shock, admitted between 1989 and 2007.\nINTERVENTIONS: None.\nMEASUREMENTS AND MAIN RESULTS: Fluids and vasoactive agents had strong, interacting associations with mortality (p < 0.0001). Mortality was lowest when vasoactive agents were begun 1-6 hours after onset, with more than 1 L of fluids in the initial hour after shock onset, more than 2.4 L from hours 1-6, and 1.6-3.5 L from 6 to 24 hours. The lowest mortality rates were associated with starting vasoactive agents 1-6 hours after onset.\nCONCLUSIONS: The focus during the first hour of resuscitation for septic shock should be aggressive fluid administration, only thereafter starting vasoactive agents, while continuing aggressive fluid administration. Starting vasoactive agents in the initial hour may be detrimental, and not all of that association is due to less fluids being given with such early initiation of vasoactive agents.","DOI":"10.1097/CCM.0000000000000520","ISSN":"1530-0293","note":"PMID: 25072761","shortTitle":"Interaction between fluids and vasoactive agents on mortality in septic shock","journalAbbreviation":"Crit. Care Med.","language":"eng","author":[{"family":"Waechter","given":"Jason"},{"family":"Kumar","given":"Anand"},{"family":"Lapinsky","given":"Stephen E."},{"family":"Marshall","given":"John"},{"family":"Dodek","given":"Peter"},{"family":"Arabi","given":"Yaseen"},{"family":"Parrillo","given":"Joseph E."},{"family":"Dellinger","given":"R. Phillip"},{"family":"Garland","given":"Allan"},{"literal":"Cooperative Antimicrobial Therapy of Septic Shock Database Research Group"}],"issued":{"date-parts":[["2014",10]]}}},{"id":2596,"uris":[""],"uri":[""],"itemData":{"id":2596,"type":"article-journal","title":"Fluid resuscitation in human sepsis: Time to rewrite history?","container-title":"Annals of Intensive Care","page":"4","volume":"7","issue":"1","source":"annalsofintensivecare.","abstract":"Fluid resuscitation continues to be recommended as the first-line resuscitative therapy for all patients with severe sepsis and septic shock. The current acceptance of the therapy is based in part on long history and familiarity with its use in the resuscitation of other forms of shock, as well as on an incomplete and incorrect understanding of the pathophysiology of sepsis. Recently, the safety of intravenous fluids in patients with sepsis has been called into question with both prospective and observational data suggesting improved outcomes with less fluid or no fluid. The current evidence for the continued use of fluid resuscitation for sepsis remains contentious with no prospective evidence demonstrating benefit to fluid resuscitation as a therapy in isolation. This article reviews the historical and physiological rationale for the introduction of fluid resuscitation as treatment for sepsis and highlights a number of significant concerns based on current experimental and clinical evidence. The research agenda should focus on the development of hyperdynamic animal sepsis models which more closely mimic human sepsis and on experimental and clinical studies designed to evaluate minimal or no fluid strategies in the resuscitation phase of sepsis.","DOI":"10.1186/s13613-016-0231-8","ISSN":"2110-5820","shortTitle":"Fluid resuscitation in human sepsis","language":"En","author":[{"family":"Byrne","given":"Liam"},{"family":"Haren","given":"Frank"}],"issued":{"date-parts":[["2017",1,3]]}}},{"id":2537,"uris":[""],"uri":[""],"itemData":{"id":2537,"type":"article-journal","title":"Early versus delayed administration of norepinephrine in patients with septic shock","container-title":"Critical Care (London, England)","page":"532","volume":"18","issue":"5","source":"PubMed","abstract":"INTRODUCTION: This study investigated the incidence of delayed norepinephrine administration following the onset of septic shock and its effect on hospital mortality.\nMETHODS: We conducted a retrospective cohort study using data from 213 adult septic shock patients treated at two general surgical intensive care units of a tertiary care hospital over a two year period. The primary outcome was 28-day mortality.\nRESULTS: The 28-day mortality was 37.6% overall. Among the 213 patients, a strong relationship between delayed initial norepinephrine administration and 28-day mortality was noted. The average time to initial norepinephrine administration was 3.1?±?2.5 hours. Every 1-hour delay in norepinephrine initiation during the first 6 hours after septic shock onset was associated with a 5.3% increase in mortality. Twenty-eight day mortality rates were significantly higher when norepinephrine administration was started more than or equal to 2 hours after septic shock onset (Late-NE) compared to less than 2 hours (Early-NE). Mean arterial pressures at 1, 2, 4, and 6 hours after septic shock onset were significantly higher and serum lactate levels at 2, 4, 6, and 8 hours were significantly lower in the Early-NE than the Late-NE group. The duration of hypotension and norepinephrine administration was significantly shorter and the quantity of norepinephrine administered in a 24-hour period was significantly less for the Early-NE group compared to the Late-NE group. The time to initial antimicrobial treatment was not significantly different between the Early-NE and Late-NE groups.\nCONCLUSION: Our results show that early administration of norepinephrine in septic shock patients is associated with an increased survival rate.","DOI":"10.1186/s13054-014-0532-y","ISSN":"1466-609X","note":"PMID: 25277635\nPMCID: PMC4194405","journalAbbreviation":"Crit Care","language":"eng","author":[{"family":"Bai","given":"Xiaowu"},{"family":"Yu","given":"Wenkui"},{"family":"Ji","given":"Wu"},{"family":"Lin","given":"Zhiliang"},{"family":"Tan","given":"Shanjun"},{"family":"Duan","given":"Kaipeng"},{"family":"Dong","given":"Yi"},{"family":"Xu","given":"Lin"},{"family":"Li","given":"Ning"}],"issued":{"date-parts":[["2014",10,3]]}}}],"schema":""} 4,5,8,9. This may avoid administering excessive amounts of fluids, which has been linked with poorer outcome ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"FZcPYnQY","properties":{"formattedCitation":"\\super 1,4,5,22\\nosupersub{}","plainCitation":"1,4,5,22","noteIndex":0},"citationItems":[{"id":558,"uris":[""],"uri":[""],"itemData":{"id":558,"type":"article-journal","title":"A positive fluid balance is an independent prognostic factor in patients with sepsis","container-title":"Critical Care","page":"251","volume":"19","issue":"1","source":"","abstract":"PMID: 26073560","DOI":"10.1186/s13054-015-0970-1","ISSN":"1364-8535","note":"PMID: 26073560","language":"en","author":[{"family":"Acheampong","given":"Angela"},{"family":"Vincent","given":"Jean-Louis"}],"issued":{"date-parts":[["2015",6,15]]}}},{"id":2578,"uris":[""],"uri":[""],"itemData":{"id":2578,"type":"article-journal","title":"Sepsis: pathophysiology and clinical management","container-title":"BMJ","page":"i1585","volume":"353","source":"","abstract":"Sepsis, severe sepsis, and septic shock represent increasingly severe systemic inflammatory responses to infection. Sepsis is common in the aging population, and it disproportionately affects patients with cancer and underlying immunosuppression. In its most severe form, sepsis causes multiple organ dysfunction that can produce a state of chronic critical illness characterized by severe immune dysfunction and catabolism. Much has been learnt about the pathogenesis of sepsis at the molecular, cell, and intact organ level. Despite uncertainties in hemodynamic management and several treatments that have failed in clinical trials, investigational therapies increasingly target sepsis induced organ and immune dysfunction. Outcomes in sepsis have greatly improved overall, probably because of an enhanced focus on early diagnosis and fluid resuscitation, the rapid delivery of effective antibiotics, and other improvements in supportive care for critically ill patients. These improvements include lung protective ventilation, more judicious use of blood products, and strategies to reduce nosocomial infections.","DOI":"10.1136/bmj.i1585","ISSN":"1756-1833","note":"PMID: 27217054","shortTitle":"Sepsis","journalAbbreviation":"BMJ","language":"en","author":[{"family":"Gotts","given":"Jeffrey E."},{"family":"Matthay","given":"Michael A."}],"issued":{"date-parts":[["2016",5,23]]}}},{"id":2332,"uris":[""],"uri":[""],"itemData":{"id":2332,"type":"article-journal","title":"The demise of early goal-directed therapy for severe sepsis and septic shock","container-title":"Acta Anaesthesiologica Scandinavica","page":"561-567","volume":"59","issue":"5","source":"Wiley Online Library","abstract":"A protocol for the quantitative resuscitation of severe sepsis and septic shock known as early goal-directed therapy (EGDT) was published in 2001. Despite serious limitations, this study became widely adopted around the world and formed the basis of the Surviving Sepsis Campaign 6?h resuscitation bundle. Subsequently, a large number of observational before-and-after studies were published which demonstrated that EGDT reduced mortality. However, during this time period, there has been a substantial reduction in the mortality from sepsis in many Western nations that appears unrelated to EGDT. Recently, the Protocolized Care for Early Septic Shock (ProCESS) and The Australasian Resuscitation in Sepsis Evaluation (ARISE) trials failed to demonstrate any outcome benefit from EGDT. These two large, multicenter, randomized controlled studies raise serious questions regarding the validity of the original EGDT study and the scientific rigor of the uncontrolled, largely retrospective before–after clinical studies. Furthermore, accruing data suggest an association between the amount of fluid administered in the first 72?h and the mortality of patients with severe sepsis. Patients in all arms of the ProCESS and ARISE trials received substantial and nearly equivalent amounts of fluid. It is proposed that a more conservative fluid strategy and the earlier use of norepinephrine in patients with septic shock may be associated with further improvements in the outcome of patients with sepsis.","DOI":"10.1111/aas.12479","ISSN":"1399-6576","journalAbbreviation":"Acta Anaesthesiol Scand","language":"en","author":[{"family":"Marik","given":"P. E."}],"issued":{"date-parts":[["2015",5,1]]}}},{"id":2596,"uris":[""],"uri":[""],"itemData":{"id":2596,"type":"article-journal","title":"Fluid resuscitation in human sepsis: Time to rewrite history?","container-title":"Annals of Intensive Care","page":"4","volume":"7","issue":"1","source":"annalsofintensivecare.","abstract":"Fluid resuscitation continues to be recommended as the first-line resuscitative therapy for all patients with severe sepsis and septic shock. The current acceptance of the therapy is based in part on long history and familiarity with its use in the resuscitation of other forms of shock, as well as on an incomplete and incorrect understanding of the pathophysiology of sepsis. Recently, the safety of intravenous fluids in patients with sepsis has been called into question with both prospective and observational data suggesting improved outcomes with less fluid or no fluid. The current evidence for the continued use of fluid resuscitation for sepsis remains contentious with no prospective evidence demonstrating benefit to fluid resuscitation as a therapy in isolation. This article reviews the historical and physiological rationale for the introduction of fluid resuscitation as treatment for sepsis and highlights a number of significant concerns based on current experimental and clinical evidence. The research agenda should focus on the development of hyperdynamic animal sepsis models which more closely mimic human sepsis and on experimental and clinical studies designed to evaluate minimal or no fluid strategies in the resuscitation phase of sepsis.","DOI":"10.1186/s13613-016-0231-8","ISSN":"2110-5820","shortTitle":"Fluid resuscitation in human sepsis","language":"En","author":[{"family":"Byrne","given":"Liam"},{"family":"Haren","given":"Frank"}],"issued":{"date-parts":[["2017",1,3]]}}}],"schema":""} 1,4,5,22. Our results support this strategy but importantly allow the treatment to be individualized for each patient. The vision is that this system would be used in real-time, with patient data coming from different streams being fed into electronic health record software fitted with our algorithm, which would suggest a course of action. Physicians will always need to make subjective clinical judgments about treatment strategies, but computational models can provide additional insight about optimal decisions, avoiding targeting short-term resuscitation goals and instead following trajectories towards longer-term survival ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"u3AbfU1R","properties":{"formattedCitation":"\\super 23\\uc0\\u8211{}25\\nosupersub{}","plainCitation":"23–25","noteIndex":0},"citationItems":[{"id":2533,"uris":[""],"uri":[""],"itemData":{"id":2533,"type":"article-journal","title":"Machine Learning and Decision Support in Critical Care","container-title":"Proceedings of the IEEE","page":"444-466","volume":"104","issue":"2","source":"IEEE Xplore","abstract":"Clinical data management systems typically provide caregiver teams with useful information, derived from large, sometimes highly heterogeneous, data sources that are often changing dynamically. Over the last decade there has been a significant surge in interest in using these data sources, from simply reusing the standard clinical databases for event prediction or decision support, to including dynamic and patient-specific information into clinical monitoring and prediction problems. However, in most cases, commercial clinical databases have been designed to document clinical activity for reporting, liability, and billing reasons, rather than for developing new algorithms. With increasing excitement surrounding “secondary use of medical records” and “Big Data” analytics, it is important to understand the limitations of current databases and what needs to change in order to enter an era of “precision medicine.” This review article covers many of the issues involved in the collection and preprocessing of critical care data. The three challenges in critical care are considered: compartmentalization, corruption, and complexity. A range of applications addressing these issues are covered, including the modernization of static acuity scoring; online patient tracking; personalized prediction and risk assessment; artifact detection; state estimation; and incorporation of multimodal data sources such as genomic and free text data.","DOI":"10.1109/JPROC.2015.2501978","ISSN":"0018-9219","author":[{"family":"Johnson","given":"A. E. W."},{"family":"Ghassemi","given":"M. M."},{"family":"Nemati","given":"S."},{"family":"Niehaus","given":"K. E."},{"family":"Clifton","given":"D. A."},{"family":"Clifford","given":"G. D."}],"issued":{"date-parts":[["2016",2]]}}},{"id":2084,"uris":[""],"uri":[""],"itemData":{"id":2084,"type":"article-journal","title":"The Future of Critical Care Medicine: Integration and Personalization","container-title":"Critical Care Medicine","page":"386-389","volume":"44","issue":"2","source":"CrossRef","DOI":"10.1097/CCM.0000000000001530","ISSN":"0090-3493","shortTitle":"The Future of Critical Care Medicine","language":"en","author":[{"family":"Vincent","given":"Jean-Louis"}],"issued":{"date-parts":[["2016",2]]}}},{"id":2888,"uris":[""],"uri":[""],"itemData":{"id":2888,"type":"article-journal","title":"Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations","container-title":"New England Journal of Medicine","page":"2507-2509","volume":"376","issue":"26","source":"Taylor and Francis+NEJM","abstract":"Big data, we have all heard, promise to transform health care with the widespread capture of electronic health records and high-volume data streams from sources ranging from insurance claims and registries to personal genomics and biosensors.1 Artificial-intelligence and machine-learning predictive algorithms, which can already automatically drive cars, recognize spoken language, and detect credit card fraud, are the keys to unlocking the data that can precisely inform real-time decisions. But in the “hype cycle” of emerging technologies, machine learning now rides atop the “peak of inflated expectations.”2 Prediction is not new to medicine. From risk scores to guide anticoagulation (CHADS2) and . . .","DOI":"10.1056/NEJMp1702071","ISSN":"0028-4793","note":"PMID: 28657867","author":[{"family":"Chen","given":"Jonathan H."},{"family":"Asch","given":"Steven M."}],"issued":{"date-parts":[["2017",6,29]]}}}],"schema":""} 23–25. The reinforcement learning approach that we developed is agnostic to the data used and could in principle be applied to any data-rich clinical environment and many medical interventions. In the future, it is likely that as “-omic” technologies develop, this additional information will be added to the AI clinician to improve state definition and guide more therapies in selected patient groups.There are limitations to our study. Although the datasets are large and used routinely collected clinical data, some sites and patients had to be excluded due to poor quality data recording?or missing data. Due to differences between the two datasets, slightly different implementations of the sepsis-3 criteria were used, and hospital mortality was used in eRI instead of 90-day mortality. Finally, some laboratory values would not have been immediately available to clinicians to inform decision making but were available to the AI Clinician. This work will clearly require prospective evaluation using real-time data and decision making in clinical trials and also testing in different healthcare settings, but if only a few percent reduction in mortality from sepsis could be achieved, this would represent several tens of thousands of lives saved annually, worldwide ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"lDLKBer0","properties":{"formattedCitation":"\\super 26\\nosupersub{}","plainCitation":"26","noteIndex":0},"citationItems":[{"id":2642,"uris":[""],"uri":[""],"itemData":{"id":2642,"type":"article-journal","title":"Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations","container-title":"American Journal of Respiratory and Critical Care Medicine","page":"259-272","volume":"193","issue":"3","source":" (Atypon)","abstract":"Rationale: Reducing the global burden of sepsis, a recognized global health challenge, requires comprehensive data on the incidence and mortality on a global scale.Objectives: To estimate the worldwide incidence and mortality of sepsis and identify knowledge gaps based on available evidence from observational studies.Methods: We systematically searched 15 international citation databases for population-level estimates of sepsis incidence rates and fatality in adult populations using consensus criteria and published in the last 36 years.Measurements and Main Results: The search yielded 1,553 reports from 1979 to 2015, of which 45 met our criteria. A total of 27 studies from seven high-income countries provided data for metaanalysis. For these countries, the population incidence rate was 288 (95% confidence interval [CI], 215–386; τ?=?0.55) for hospital-treated sepsis cases and 148 (95% CI, 98–226; τ?=?0.99) for hospital-treated severe sepsis cases per 100,000 person-years. Restricted to the last decade, the incidence rate was 437 (95% CI, 334–571; τ?=?0.38) for sepsis and 270 (95% CI, 176–412; τ?=?0.60) for severe sepsis cases per 100,000 person-years. Hospital mortality was 17% for sepsis and 26% for severe sepsis during this period. There were no population-level sepsis incidence estimates from lower-income countries, which limits the prediction of global cases and deaths. However, a tentative extrapolation from high-income country data suggests global estimates of 31.5 million sepsis and 19.4 million severe sepsis cases, with potentially 5.3 million deaths annually.Conclusions: Population-level epidemiologic data for sepsis are scarce and nonexistent for low- and middle-income countries. Our analyses underline the urgent need to implement global strategies to measure sepsis morbidity and mortality, particularly in low- and middle-income countries.","DOI":"10.1164/rccm.201504-0781OC","ISSN":"1073-449X","journalAbbreviation":"Am J Respir Crit Care Med","author":[{"family":"Fleischmann","given":"Carolin"},{"family":"Scherag","given":"André"},{"family":"Adhikari","given":"Neill K. J."},{"family":"Hartog","given":"Christiane S."},{"family":"Tsaganos","given":"Thomas"},{"family":"Schlattmann","given":"Peter"},{"family":"Angus","given":"Derek C."},{"family":"Reinhart","given":"Konrad"}],"issued":{"date-parts":[["2015",9,28]]}}}],"schema":""} 26. In the last 10 to 15 years, attempts to develop novel treatments to reduce sepsis mortality have uniformly been unsuccessful ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"a1u08map2m6","properties":{"formattedCitation":"\\super 27,28\\nosupersub{}","plainCitation":"27,28","noteIndex":0},"citationItems":[{"id":2900,"uris":[""],"uri":[""],"itemData":{"id":2900,"type":"article-journal","title":"Levosimendan for the Prevention of Acute Organ Dysfunction in Sepsis","container-title":"New England Journal of Medicine","page":"1638-1648","volume":"375","issue":"17","source":"Taylor and Francis+NEJM","abstract":"Sepsis is defined as life-threatening organ dysfunction caused by a dysregulated host response to infection1 and is a leading cause of death worldwide. Septic shock is the most severe form of the condition and results in circulatory and metabolic abnormalities.2 Persisting hypotension despite adequate fluid resuscitation is due to a combination of profound vasodilatation, vascular hyporeactivity to catecholamines, and myocardial depression.3 Although catecholamines are the recommended first-line therapy for septic shock,4 high doses of administered catecholamines and high levels of circulating catecholamines are associated with poor outcomes and severe side effects, including myocardial injury and peripheral ischemia.5–7 Levosimendan is . . .","DOI":"10.1056/NEJMoa1609409","ISSN":"0028-4793","note":"PMID: 27705084","author":[{"family":"Gordon","given":"Anthony C."},{"family":"Perkins","given":"Gavin D."},{"family":"Singer","given":"Mervyn"},{"family":"McAuley","given":"Daniel F."},{"family":"Orme","given":"Robert M.L."},{"family":"Santhakumaran","given":"Shalini"},{"family":"Mason","given":"Alexina J."},{"family":"Cross","given":"Mary"},{"family":"Al-Beidh","given":"Farah"},{"family":"Best-Lane","given":"Janis"},{"family":"Brealey","given":"David"},{"family":"Nutt","given":"Christopher L."},{"family":"McNamee","given":"James J."},{"family":"Reschreiter","given":"Henrik"},{"family":"Breen","given":"Andrew"},{"family":"Liu","given":"Kathleen D."},{"family":"Ashby","given":"Deborah"}],"issued":{"date-parts":[["2016",10,27]]}}},{"id":2897,"uris":[""],"uri":[""],"itemData":{"id":2897,"type":"article-journal","title":"Drotrecogin Alfa (Activated) in Adults with Septic Shock","container-title":"New England Journal of Medicine","page":"2055-2064","volume":"366","issue":"22","source":"Taylor and Francis+NEJM","abstract":"Recombinant human activated protein C, or drotrecogin alfa (activated) (DrotAA), was approved for the treatment of severe sepsis in 2001 on the basis of the Prospective Recombinant Human Activated Protein C Worldwide Evaluation in Severe Sepsis (PROWESS) study,1 a phase 3 international, randomized, controlled trial that was stopped early for efficacy after the enrollment of 1690 patients with severe sepsis. Absolute mortality in the intention-to-treat population was reduced by 6.1 percentage points, a relative risk reduction of 19.4%. Subsequent subgroup analysis suggested that the mortality benefit was limited to patients with increased illness severity (i.e., those with more than one . . .","DOI":"10.1056/NEJMoa1202290","ISSN":"0028-4793","note":"PMID: 22616830","author":[{"family":"Ranieri","given":"V. Marco"},{"family":"Thompson","given":"B. Taylor"},{"family":"Barie","given":"Philip S."},{"family":"Dhainaut","given":"Jean-Fran?ois"},{"family":"Douglas","given":"Ivor S."},{"family":"Finfer","given":"Simon"},{"family":"G?rdlund","given":"Bengt"},{"family":"Marshall","given":"John C."},{"family":"Rhodes","given":"Andrew"},{"family":"Artigas","given":"Antonio"},{"family":"Payen","given":"Didier"},{"family":"Tenhunen","given":"Jyrki"},{"family":"Al-Khalidi","given":"Hussein R."},{"family":"Thompson","given":"Vivian"},{"family":"Janes","given":"Jonathan"},{"family":"Macias","given":"William L."},{"family":"Vangerow","given":"Burkhard"},{"family":"Williams","given":"Mark D."}],"issued":{"date-parts":[["2012",5,31]]}}}],"schema":""} 27,28. The use of computer decision support systems to better guide treatments and improve outcomes is therefore a much needed approach. ACKNOWLEDGEMENTSWe are grateful for support from the National Institute of Health Research (NIHR) Comprehensive Biomedical Research Centre based at Imperial College Healthcare NHS Trust and Imperial College London. We are thankful to the Laboratory of Computational Physiology at the Massachusetts Institute of Technology and the eICU Research Institute for providing the data used in this research. MK and this project are funded by the Engineering and Physical Sciences Research Council and an Imperial College President’s PhD Scholarship. ACG is funded by an NIHR Research Professorship award (RP-2015-06-018). We are grateful to Finale Doshi-Velez and Omer Gottesman for their assistance with the methodology.AUTHOR CONTRIBUTIONSMK, ACG and AF conceived the overall study. MK and AF designed and conducted the experiments, and analysed the data. LAC and OB contributed to the experimental design and analyses. OB provided key input in extracting and processing data from the eRI. All authors contributed to the interpretation of the results and MK drafted the manuscript, which was reviewed, revised and approved by all PETING INTERESTSThe authors declare competing financial interests: ACG reports that outside of this work he has received speaker fees from Orion Corporation Orion Pharma and Amomed Pharma. He has consulted for Ferring Pharmaceuticals, Tenax Therapeutics, Baxter Healthcare, Bristol-Myers Squibb and GSK, and received grant support from Orion Corporation Orion Pharma, Tenax Therapeutics and HCA International with funds paid to his institution. LAC receives funding from Philips Healthcare. OB is an employee of Philips Healthcare. MK and AF do not have competing financial interests.MATERIAL AND CORRESPONDENCECorrespondence and requests for materials should be addressed to Anthony Gordon for clinical questions and A. Aldo Faisal for technical questions.Anthony Gordonanthony.gordon@imperial.ac.ukIntensive Care Unit, Charing Cross Hospital, Fulham Palace Rd, London W6 8RF, UK+44 (0)20 3313 0657A. Aldo Faisala.faisal@imperial.ac.ukDepartment of Bioengineering, Imperial College London, SW7 2AZ London, UK.+44 (0)20 7594 6373REFERENCES ADDIN ZOTERO_BIBL {"uncited":[],"omitted":[],"custom":[]} CSL_BIBLIOGRAPHY 1.Gotts, J. E. & Matthay, M. A. Sepsis: pathophysiology and clinical management. BMJ 353, i1585 (2016).2.Torio, C. M. & Andrews, R. M. National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2011: Statistical Brief #160. in Healthcare Cost and Utilization Project (HCUP) Statistical Briefs (Agency for Health Care Policy and Research (US), 2013).3.Liu, V. et al. Hospital Deaths in Patients With Sepsis From 2 Independent Cohorts. JAMA 312, 90 (2014).4.Byrne, L. & Haren, F. Fluid resuscitation in human sepsis: Time to rewrite history? Annals of Intensive Care 7, 4 (2017).5.Marik, P. E. The demise of early goal-directed therapy for severe sepsis and septic shock. Acta Anaesthesiol Scand 59, 561–567 (2015).6.Marik, P. & Bellomo, R. A rational approach to fluid therapy in sepsis. Br. J. Anaesth. 116, 339–349 (2016).7.Singer M, Deutschman CS, Seymour C & et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 315, 801–810 (2016).8.Waechter, J. et al. Interaction between fluids and vasoactive agents on mortality in septic shock: a multicenter, observational study. Crit. Care Med. 42, 2158–2168 (2014).9.Bai, X. et al. Early versus delayed administration of norepinephrine in patients with septic shock. Crit Care 18, 532 (2014).10.Marik, P. E., Linde-Zwirble, W. T., Bittner, E. A., Sahatjian, J. & Hansell, D. Fluid administration in severe sepsis and septic shock, patterns and outcomes: an analysis of a large national database. Intensive Care Med 43, 625–632 (2017).11.Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction. (A Bradford Book, 1998).12.Bennett, C. C. & Hauser, K. Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach. Artificial Intelligence in Medicine 57, 9–19 (2013).13.Schaefer, A. J., Bailey, M. D., Shechter, S. M. & Roberts, M. S. Modeling Medical Treatment Using Markov Decision Processes. in Operations Research and Health Care (eds. Brandeau, M. L., Sainfort, F. & Pierskalla, W. P.) 593–612 (Springer US, 2005).14.Gulshan, V. et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 316, 2402–2410 (2016).15.Prasad, N., Cheng, L.-F., Chivers, C., Draugelis, M. & Engelhardt, B. E. A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units. arXiv:1704.06300 [cs] (2017).16.Johnson, A. E. W. et al. MIMIC-III, a freely accessible critical care database. Sci Data 3, 160035 (2016).17.Elixhauser, A., Steiner, C., Harris, D. R. & Coffey, R. M. Comorbidity measures for use with administrative data. Med Care 36, 8–27 (1998).18.Puterman, M. L. Markov Decision Processes: Discrete Stochastic Dynamic Programming. (Wiley, 1994).19.Thomas, P. S., Theocharous, G. & Ghavamzadeh, M. High-Confidence Off-Policy Evaluation. in Twenty-Ninth AAAI Conference on Artificial Intelligence (2015).20.Hanna, J. P., Stone, P. & Niekum, S. Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation. arXiv:1606.06126 [cs, stat] (2016).21.Thomas, P., Theocharous, G. & Ghavamzadeh, M. High Confidence Policy Improvement. in PMLR 2380–2388 (2015).22.Acheampong, A. & Vincent, J.-L. A positive fluid balance is an independent prognostic factor in patients with sepsis. Critical Care 19, 251 (2015).23.Johnson, A. E. W. et al. Machine Learning and Decision Support in Critical Care. Proceedings of the IEEE 104, 444–466 (2016).24.Vincent, J.-L. The Future of Critical Care Medicine: Integration and Personalization. Critical Care Medicine 44, 386–389 (2016).25.Chen, J. H. & Asch, S. M. Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations. New England Journal of Medicine 376, 2507–2509 (2017).26.Fleischmann, C. et al. Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am J Respir Crit Care Med 193, 259–272 (2015).27.Gordon, A. C. et al. Levosimendan for the Prevention of Acute Organ Dysfunction in Sepsis. New England Journal of Medicine 375, 1638–1648 (2016).28.Ranieri, V. M. et al. Drotrecogin Alfa (Activated) in Adults with Septic Shock. New England Journal of Medicine 366, 2055–2064 (2012).FIGURES Figure 1: Data flow of the AI Clinician. Eighty percent of the MIMIC-III dataset was used to define the elements of the Markov decision process. Time series of patient data were clustered into finite patient states using k-means method. The dose of intravenous fluids and vasopressors were discretized into 25 possible actions. Patients’ survival at 90 days after ICU admission defined rewards. Reinforcement learning was used to evaluate the value of clinicians’ policy and estimate optimal treatment strategies, the AI policy. The remaining 20% MIMIC-III data was used to identify the best model among 500 candidates, which was then tested on an independent dataset from the eRI database.Figure 2. Selection of the best AI policy (a,b) and model calibration (c, d). a, evolution of the 95% lower bound (LB) of the best AI policy and 95% upper bound (UB) of highest valued clinicians’ policy during building of 500 models. After only a few models, a higher value for the AI policy than the clinicians’ treatment, within the accepted risk, is guaranteed. b, distribution of the estimated value of the clinicians’ actual treatments, the AI policy, a random policy and a zero-drug policy across the 500 models, in the MIMIC-III test set. The chosen AI policy maximises the 95% confidence lower bound. c, relationship between the return of clinicians’ treatments and patient 90-day mortality, in the MIMIC-III training set. Return of actions were sorted into 100 bins and the mean observed mortality was computed in each of these bins. Treatments with a low return were associated with a high risk of mortality, while treatments with a high return led to better survival rates. d, average return in survivors and non-survivors in the MIMIC-III training set. Figs. c and d were generated by bootstrapping in the training data with 2,000 resamplings.Figure 3. Comparison of clinicians and AI policies in eRI (a, b, c); Average dose excess received per patient of both drugs in eRI and corresponding mortality (d, e). a. distribution of the estimated value of the clinicians’ and the AI policy in the selected model, built by bootstrapping with 2,000 resamplings. b and c, Visualization of the clinicians’ and AI policies. All actions were aggregated over all time steps for the 5 dose bins of both medications. On average, patients were administered more intravenous fluid (b) and less vasopressor (c) medications than recommended by the AI policy. Vasopressor dose is in mcg/kg/min of norepinephrine equivalent and intravenous fluids dose is in mL / 4 hours. d and e, The dose excess refers to the difference between the given and suggested dose averaged over all time points per patient. The figure was generated by bootstrapping with 2,000 resamplings. In both plots, the smallest dose difference was associated with the best survival rates (vertical dotted line). The further away the dose received was from the suggested dose, the worse the outcome.TABLEMIMIC-IIIeRIUnique ICUs (N)5128Unique ICU admissions (N)17,08379,073Characteristics of hospitals, per number of ICU admissions.Teaching tertiary hospital.Non-teaching: 37,146 (47.0%)Teaching: 29,388 (37.2%)Unknown: 12,539 (15.9%)Age, years (Mean, SD)64.4 (16.9)65.0 (16.7)Male gender (N, %)9,604 (56.2%)40,949 (51.8%)Premorbid status (N, %)Hypertension DiabetesCHFCancerCOPD/RLDCKD9,384 (54.9%)4,902 (28.7%)5,206 (30.5%)1,803 (10.5%)4,248 (28.7%)3,087(18.1%)43,365 (54.8%)25,290 (32.0%)15,023 (19.0%)11,807 (14.9%)18,406 (23.3%)14,553 (18.4%)Primary ICD-9 diagnosis (N, %)Sepsis, including pneumoniaCardiovascularRespiratory NeurologicalRenalOthers5,824 (34.1%)5,270 (30.8%)1,798 (10.5%)1,590 (9.3%) 429 (2.5%)2,172 (12.7%)41,396 (52.3%)11,221 (14.2%)9,127 (11.5%)7,127 (9.0%)1,454 (1.8%)8,747 (11.1%)Initial OASIS (Mean, SD)33.5 (8.8)34.8 (12.4)Initial SOFA (Mean, SD)7.2 (3.2)6.4 (3.5)Procedures during the 72h of data collection:Mechanical ventilation (N, %)Vasopressors (N, %)Renal replacement therapy (N, %)9,362 (54.8%)6,023 (35.3%)1,488 (8.7%)39,115 (49.5%)23,877 (30.2%)6,071 (7.7%)Length of stay, days (Median, IQR)3.1 (1.8 – 7)2.9 (1.7 – 5.6)ICU mortality7.4%9.8%Hospital mortality11.3%16.4%90-day mortality18.9%Not availableTable 1: Description of the datasets. CHF: Congestive Heart Failure; CKD: Chronic Kidney Disease; COPD: Chronic Obstructive Pulmonary Disease; ICD-9: International Classification of Diseases version 9; IQR: Interquartile Range; OASIS: Oxford Acute Severity of Illness Score; RLD: Restrictive Lung Disease; SD: Standard Deviation; SOFA: Sequential Organ Failure Assessment. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download