Microsimulation of organ dysfunction



Dynamic microsimulation to model multiple outcomes in cohorts of critically ill patients

Gilles Clermont*, MD, MSc

Vladimir Kaplan*, MD

Rui Moreno†, MD

Jean-Louis Vincent‡, MD, PhD

Walter T. Linde-Zwirble§

Ben Van Hout¢, PhD

Derek C. Angus*¶, MD, MPH

* Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA

† Intensive Care Medicine, Hospital de St. Antonio dos Capuchos, Lisbon, Portugal

‡ Department of Intensive Care, Erasmus University Hospital, Brussels, Belgium

§ Health Process Management, Inc, Doylestown, PA

¢ Department of Health Care Policy and Management, Erasmus University, Rotterdam, Netherlands

¶ Center for Research on Health Care and Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA

Running head

Microsimulation of organ dysfunction

Word count

3,239

Financial support

Partially supported by Eli Lilly & Company (Gilles Clermont and Derek C. Angus) and by the Stiefel-Zangger Foundation, University of Zurich, Switzerland (Vladimir Kaplan)

Address for correspondence

Gilles Clermont, MD, MSc

Room 606B, Scaife Hall

Critical Care Medicine

University of Pittsburgh

3550 Terrace Street

Pittsburgh, PA 15261

Tel: (412) 647 7980

Fax: (412) 647 3791

E-mail: clermontg@ccm.upmc.edu

Abstract

Background: Existing intensive care unit (ICU) prediction tools forecast single outcomes, (e.g., risk of death) and do not provide information on timing.

Objective: To build a model that predicts the temporal patterns of multiple outcomes, such as survival, organ dysfunction, and ICU length of stay, from the profile of organ dysfunction observed on admission.

Design: Dynamic microsimulation of a cohort of ICU patients.

Setting: 49 ICUs in 11 countries.

Patients: 1,449 patients admitted to the ICU in May 1995.

Interventions: None.

Model Construction: We developed the model on all patients (n=989) from 37 randomly-selected ICUs using daily Sequential Organ Function Assessment (SOFA) scores. We validated the model on all patients (n=460) from the remaining 12 ICUs, comparing predicted-to-actual ICU mortality, SOFA scores, and ICU length of stay (LOS).

Main Results: In the validation cohort, the predicted and actual mortality were 20.1% (95%CI: 16.2%-24.0%) and 19.9% at 30 days. The predicted and actual mean ICU LOS were 7.7 (7.0-8.3) and 8.1 (7.4-8.8) days, leading to a 5.5% underestimation of total ICU bed-days. The predicted and actual cumulative SOFA scores per patient were 45.2 (39.8-50.6) and 48.2 (41.6-54.8). Predicted and actual mean daily SOFA scores were close (5.1 vs 5.5, P=0.32). Several organ-organ interactions were significant. Cardiovascular dysfunction was most, and neurological dysfunction was least, linked to scores in other organ systems.

Conclusions: Dynamic microsimulation can predict the time course of multiple short-term outcomes in cohorts of critical illness from the profile of organ dysfunction observed on admission. Such a technique may prove practical as a prediction tool that evaluates ICU performance on additional dimensions besides the risk of death.

Descriptor

Severity-of-disease scoring systems

Keywords

Intensive care, multiple organ failure, mortality, resource use, computer simulation, microsimulation

Progress in acute care medicine and new resuscitation techniques have led to significant improvement in the immediate survival of victims of severe trauma, burns, and infections. However, early resuscitation is often followed by progressive dysfunction of multiple organ systems (MODS) [1, 2] that may lead to prolonged morbidity and death [3, 4]. Indeed, MODS accounts for most late-onset deaths in critical illness [5] and consumes large amounts of healthcare resources [6].

Several investigators have developed schemes to quantify organ dysfunction and have consistently demonstrated a close relationship between the presence and intensity of MODS and hospital mortality [7-9]. Specifically, the cumulative burden of organ failure in terms of both, the number of organs failing [8, 10] and the degree of organ dysfunction within each organ system [11] was the strongest predictor of death [12]. However, these prediction tools typically predict single outcomes (e.g., risk of death) at fixed time points. Other statistical techniques provide prediction the timing of events [13, 14], but only simulations provide simultaneous predictions for the incidence and timing of multiple outcomes.

Our first objective was to build a single model that predicts, in a cohort of critically ill patients, the temporal patterns of multiple outcomes, such as survival, organ dysfunction, and intensive care unit (ICU) length of stay, from demographic variables and the profile of organ dysfunction assessed on admission by the sequential organ dysfunction score (SOFA) [15]. Our second objective was to use the model to explore organ-organ interactions. Because such predictions cannot typically be constructed using standard analytic methods[16], we developed a microsimulation model, a technique suited to predict multiple events over time in systems where characteristics change in a time dependent fashion. We validated its predictive performance in a separate group of patients using basic demographic data and the first ICU day SOFA score only.

Such simulations have a range of potential applications in the ICU such as predicting the rate and timing of various events and outcomes, and the potential impact of interventions aimed at modifying the predictors of these outcomes.

Materials and methods

Patient population

We used an international database of 1449 critically ill patients from 40 institutions (49 ICUs) in 11 countries. The database included all adult patients admitted to the ICUs in May 1995, except for those who stayed in the ICU for less than 48 hours after uncomplicated surgery. The data were prospectively collected by the European Society of Intensive Care Medicine (ESCIM) to evaluate and validate the usefulness of the SOFA score [8, 17]. SOFA scores were collected daily until ICU discharge or a maximum of 33 days. Details regarding data collection were described previously [8].

Missing values

Scores missing on days prior to the first recorded value were attributed the first available score. Scores missing between the last recorded score and ICU discharge were attributed the last recorded score. Other missing scores were assigned according to the following rules: linear interpolation was used for organ systems with a slowly evolving physiology (hematologic, renal, and hepatic) and last available scores were carried forward for the other organ systems (cardiovascular, pulmonary, and neurologic). We chose not to implement a priori stochastic rules[18] of imputing missing data a because of the expected non-randomness and high predictability of missing values [19]. To assess the sensitivity of predictions to different imputation rules for missing data, we provide predictions using last value carried forward and next value carried backwards as alternatives imputation rules for missing intercurrent values.

Dynamic microsimulation

Dynamic microsimulation is a method particularly suitable for modeling multiple events over time that occur in systems where the interactions within the system are complex, the characteristics of the systems change in a time-dependent fashion, and the analysis of the system is intractable by conventional analytic methods [16]. Such models allow probabilistic projections forward in time on cohorts with known baseline characteristics. In our model, the patient represents the complex system, defined by his/her profile of organ dysfunction, which evolves over time and dictates the occurrence of multiple outcomes (i.e., death, organ failure, or ICU discharge). Microsimulation is well-suited to describe cohort behavior, and not the time course of individual patients.

Model development

We used a subset of 989 patients from 37 randomly selected ICUs to develop the microsimulation model. We built the model in three steps. First, using the SOFA scores of the current day, we constructed trinomial logistic regression equations to generate daily probabilities of “discharge from the ICU on the current day”, “ICU death on the current day”, or “remain in the ICU until the next day” (the ternary outcome sub-model). Because we anticipated that the predictive probabilities of a given pattern of SOFA scores would change over time, we also included an explicit time factor as independent predictor (periods A, B and C corresponding to ICU day 1, ICU days 2 to 9, and ICU days 10 to 30). We ignored data beyond 30 days because of a paucity of data points. We also included sex, type of patient (emergent/scheduled surgery, trauma, medical/cardiac/others), age (65) as predictors. Second, to update SOFA scores in individual patients remaining in the ICU, we constructed multinomial logistic regression equations to generate SOFA scores for the next day based on SOFA scores of the current day as well as the same demographic predictors described above and ICU day (the SOFA sub-models). We developed 6 SOFA sub-models (one for each organ systems). Third, we integrated all sub-models in a global model, a dynamic microsimulation, and propelled each patient in daily steps until ICU discharge or a maximum of 30 days.

To verify the ability of the microsimulation model to reproduce the outcome and the level of organ dysfunction, we simulated the time course of the ICU stay in the development cohorts (Figure 1). A specific example on how the microsimulation decides of a patient’s outcome given a set of independent predictors on day 1 is provided as supplementary material (Tables E1-E4). If the patient remained in the ICU to the next day, the simulation generated SOFA scores for day 2 using the appropriate SOFA sub-models (Tables E5-E22). The probability of being discharged alive from the ICU on day 2, remaining in the ICU to the next day, and dying in the ICU on day 2 was recalculated and the fate of the patient determined (Figure E1). This process was iterated until the patient was ICU discharge or ICU day 30.

To assess model accuracy we calculated mean ICU mortality, mean ICU length of stay, average daily organ-specific and global (sum of organ-specific scores on any given day) SOFA scores, and cumulative (over the entire ICU stay) organ-specific and global SOFA scores for the entire simulated cohort. The probability distribution of the predictions was derived from running the microsimulation 500 times. We generated standardized ratios (SR) and 95% confidence intervals (CI) for all evaluated outcomes and organ dysfunction scores using prediction from the model as the numerator and actual observations in the development cohort as the denominator.

Model validation

We validated the model in 460 patients from the remaining 12 ICUs using the day 1 predictors to initiate the microsimulation. We used a random sample of ICUs to increase the external validity of the model. Indeed, prediction models are typically applied in situations where both patients and therapeutic approaches to those patients vary from the development environment. Again, we simulated the ICU time course of 460 patients selected with replacement from the validation cohort and predicted the mean values for ICU mortality, ICU length of stay, and daily and cumulative organ-specific and global SOFA scores. We compared the mean values for each outcome predicted by the model to those observed in the validation cohort and calculated SR and 95% CI.

Organ-organ interaction

To investigate organ-organ interaction we constructed standard linear regression equations for SOFA scores for the entire cohort irrespective of time period and examined the magnitude and significance of the regression coefficients for predicting single organ SOFA scores of the next day based on SOFA scores of the current day. The strength of interaction is conveyed by the magnitude of the regression coefficients.

Statistical procedures

We compared proportions using Chi-square statistics. We compared lengths of stay and organ failure scores using the Mann-Whitney U test. We assumed a significance level of p ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download