Lippincott Williams & Wilkins



NON-ELECTIVE REHOSPITALIZATIONS AND POST-DISCHARGE MORTALITY: PREDICTIVE MODELS SUITABLE FOR USE IN REAL TIMEGabriel J. Escobar, MD; Arona Ragins, MA; Peter Scheirer, MA; Vincent Liu, MD, MS; Jay Robles, BA; Patricia Kipnis, PhDWEB APPENDIX FOR INTERESTED READERSAll SAS code used for this project’s data processing or analysis is available to interested readers. SAS and SQL code for the LAPS2 and LAPS2dc severity scores and the COPS2 longitudinal comorbidity score are also available to interested readers.Number DescriptionPages1 Predictors included in descriptive analyses and modeling 2 - 52 Additional information on cohort6 3 Predictive modeling methodology employed 74 Cohort description (patient as unit of analysis) 8 - 95 Bivariate comparisons (patient as unit of analysis) 10 - 116 Comparison of 7 day EMR models to LACE 12 7 Calibration curves in validation dataset 13 - 198 Relative contribution of predictors 209 Supplemental analyses – 7 and 30 day model performance characteristics 21 - 2310 Supplemental analyses – Impact of incorporating diagnosis in EMR models 2411 Supplemental analyses – Model performance across subgroups 25 - 3012 Beta Coefficients for models from derivation data31 - 3613 Kaplan-Meier Curves for Models 37 - 4214 Platform presentation describing current KPNC early warning system pilot 43 – 71 (American Thoracic Society meeting, San Diego, California, 5/19/14)15 Relationship between model predictors and disaggregated outcomes (odds ratios using univariate logistic regression) 72 – 74APPENDIX 1: PREDICTORS INCLUDED IN DESCRIPTIVE ANALYSES AND MODELING The figure on page 5 shows the predictors available to us. The figure does not include some predictors that are commonly available (e.g., age and sex). As noted in the text, because we cannot capture diagnosis reliably in real time, we did not include diagnoses as predictors. Going from top to bottom and left to right, predictors were as follows:LAPS2 (Laboratory Acute Physiology Score, version 2)The LAPS2, described in citation 19, is based on 15 laboratory tests, vital signs (temperature, heart rate, respiratory rate, blood pressure), pulse oximetry, neurological status as documented in nursing flow sheets, and interaction terms (e.g., the shock index). The “look back” period for LAPS2 is 72 hours from the time of rooming in at the patients’ first hospital unit. Additional information on this predictor can be obtained from the principal investigator. The figure below shows the distribution of scores in the original paper describing LAPS2, which was based on 248,383 patients.COPS2 (COmorbidity Point Score, version 2)Every month, the KPNC Decision Support Department scans data from outpatient and inpatient encounters from the entire KPNC membership. Using the ICD codes from these encounters, the MIA department assigns these codes to 70 possible Hierarchical Condition Categories (HCCs) using their inpatient and outpatient utilization during the preceding 12 month period. A given patient may have multiple HCC assignments. The ICD code categories used for these HCCs are those used by the Centers for Medicare and Medicaid Services. The COPS2 employs 45 of these HCCs to assign a point score.The figure below shows the distribution of scores in the original paper describing COPS2, which was based on 248,383 patients (citation 19).HOSPITALIZATION AND/OR ED VISITWe scanned each patient’s records in KPNC hospitalization and emergency department databases (these are now part of the Epic inpatient record). Out of plan use was also captured for these predictors.LENGTH OF STAYFor hospitalizations included in the predictive model, the T0 was time of rooming in, and the TEND was the date/time stamp from the last linked hospital stay (as noted in the text, we concatenated stays of patients who experienced inter-hospital transport). Patients who experienced a non-KPNC hospital stay as part of a hospitalization episode are included (length of stay was bridged across stays).OR (operating room stay)KPNC bed history databases capture when patients entered or left the operating room. For the purposes of our analyses, we categorized patients as having had 0, 1, or 2+ operating room stays.ICU (admission to Intensive Care Unit)KPNC bed history databases capture when patients entered or left the ICU; it is also possible to determine whether a patient experienced assisted ventilation (V) or received continuous infusions of pressor agents (P). All KPNC patients admitted to the ICU are also assigned retrospective eSAPS3 scores (see citation 16). The eSAPS3 has a 12 hour “look back” time frame and a +1 hour “look forward” time frame for data capture, with the T0 being time of physical entry into the ICU.LAPS2dc (LAPS2 at discharge)The LAPS2dc is calculated in the same way as the LAPS2 except that (a) the T0 was set to 0800 hours on the day of discharge, (b) the “look back” time frame was set to 24 hours instead of 72, and (c) all patients were assumed to be “low risk” for the purposes of imputation of missing data (see citation 19 for additional details on the LAPS2 imputation algorithm).CARE DIRECTIVEWe examined both admission and discharge care directives. As we have reported in citation 19, admission care directives are mandatory (“hard stop”) in the EMR, and a physician’s other orders cannot be signed without a care directive being specified. For the discharge care directive, we employed the last care directive ordered by the treating physicians. We grouped these as “full code” vs. “not full code” (which included “do not resuscitate,” “partial code,” and “comfort care only”).APPENDIX 2: ADDITIONAL INFORMATION ON COHORT TABLE A: SITE OF DEATHMortalityInpatientOther location Total7 day 803 7,198 8,00130 day 4,646 18,04722,693TABLE B: INCLUSION CRITERIA AMONG THE REHOSPITALIZATIONS1ACS2Via ED3LAPS2 ≥ 604 Number of rehospitalizationsDeaths (%)NONONO17,818241 (1.35%)NONOYES2,247189 (8.41%)NOYESNO15,675409 (2.61%)NOYESYES41,4323,605 (8.70%)YESNONO1,11317 (1.53%)YESNOYES32622 (6.75%)YESYESNO1,61221 (1.30%)YESYESYES9,452431 (4.56%)1. Top row shows that some deaths occurred in elective rehospitalizations.2. Ambulatory Care Sensitive diagnosis; see main main text for details.3. Emergency Department4. Laboratory Acute Physiology Score, version 2; see main text for details.1. APPENDIX 3: PREDICTIVE MODELING METHODOLOGY EMPLOYEDThe body of this paper discussed four models. For one of them (30-day combined outcome at discharge) we were fortunate in that the superior model ended up using the simple-to-implement logistic regression method. Other methods considered (using SAS version 9.3, JMP version 7, or R version 3 as needed) were ANCOVA, saturated ANOVA with smoothing by logistic regression when needed, random forest, conditional inference recursive partition, neural network, recursive-partition-then-logistic regression, and a nearest-neighbor Mahananobis-distance-weighted approach that does not limit itself to a fixed number of neighbors.Some of these methods built models on a random 1/6 of patients and then tested on another random 1/6. Other methods built models on a random 1/4 of patients and then tested on another random 1/4. For all methods the selected model was based on its performance in their respective test data, and a separate random half of all patients (who had not been used previously by any method) was set aside for validation of that selected model. The best model was selected based on a high c-statistic with a subjective penalty for the number of covariates used and the complexity of the model itself. Covariates considered were an indicator if admission was in the ED or not, the number of prior hospitalizations up to 7 days and up to 30 days prior to admission, the length of stay, age, gender, COPS2 at admission, the care order level at discharge, LAPS2 at admission and at 8am of day discharge, and the discharge day-of-week.For when employing the logistic regression method, considered also were all possible two-way interactions and (for the continuously-valued covariates) four-knot restricted cubic splines (via Harrell’s %DASPLINE macro in SAS). Also the two parts of the combined outcome (death only or undesired rehospitalization only) were modelled separately and then blended to form an estimated combined outcome. Derived variables were considered as well, such as truncating length of stay and combining age, gender, COPS2, and LAPS2 via a quadratic response surface (which was in turn calibrated separately to the combined outcome, death only, and undesired rehospitalization only).The other three models presented in this paper were developed by shorter routes. With the model for the 30-day combined outcome at discharge selected, we applied the same model structure onto the 7-day combined outcome at discharge and found the performance adequate. The two models to be run at admission benefitted from our past experience that a greedy recursive partition algorithm (JMP version 7) to identify six terminal nodes of a particular spread (including one node with a small sample size and a very high outcome rate and another node with a large sample size and a very low outcome rate) followed by a separate logistic regression (SAS version 9.3) of a same model structure for each terminal node generally yields satisfactory results. For this paper the covariates allowable for the recursive partition algorithm and the model structure of each of the six logistic regressions were age, gender, COPS2, and LAPS2. These four covariates were selected as they represent the classic two demographics, a comorbidity metric, and an acuity metric that all can be ascertained at admission in an emergency department. These four covariates entered the logistic regressions linearly and without interactions. Built in this fashion, the resulting two admission models were found to have sufficient performance, as demonstrated in the body of the paper.APPENDIX 4: COHORT DESCRIPTION (PATIENT AS UNIT OF ANALYSIS)COHORT CHARACTERISTICS1 Derivation datasetValidation datasetEntire cohortN (patients)179,978180,058360,036N (index hospitalizations)179,978180,058360,036Age (mean ± SD2)61.7 ± 18.161.7 ± 18.161.7 ± 18.1Sex (% male)45.8%45.8%45.8%Race/ethnicity White47.9%47.8%47.9%African American6.4%6.5%6.5%Asian17.7%17.7%17.7%Hispanic23.8%23.8%23.8%Other or missing4.1%4.1%4.1%COPS23 (mean ± SD)21.8 ± 23.421.8 ± 23.521.8 ± 23.4Charlson score4 (median, IQR)1 , 21 , 21 , 2LAPS25 (mean ± SD)49.7 ± 37.549.5 ± 37.449.6 ± 37.4LAPS2dc (mean ± SD)39.7 ± 23.339.6 ± 23.239.7 ± 23.3% with these primary conditions6 Community-acquired pneumonia1.6%1.6%1.6%Sepsis7.2%7.3%7.3%Gastrointestinal bleeding1.4%1.4%1.4%Hip fracture1.5%1.6%1.6%Any malignancy7.6%7.5%7.5%Rehospitalization77 day (any)4.1%4.2%4.2%7 day (undesirable)3.5%3.6%3.6%30 day (any)9.6%9.8%9.7%30 day (undesirable)7.9%8.0%7.9%Mortality77 day0.8%0.8%0.8%30 day2.2%2.2%2.2%Composite outcome77 day4.2%4.4%4.3%30 day9.4%9.5%9.5%Footnotes to Appendix 4 This table describes study cohort using individual patients as the unit of analysis. It is a sister to Table 1 in the main manuscript (which reports data using an index hospitalization as the unit of analysis). If a patient had multiple index hospitalizations then only the first one is retained for this table.SD: standard deviation; IQR: interquartile rangeSee text and citation 19 for description of the COmorbidity Point Score, version 2. The univariate relationship of COPS2 with 30 day mortality is as follows: 0 – 39, 1.7%; 40 – 64, 5.2%, 65+, 9.0%.See citations 19 and 30 for description of methodology used to assign this score.See text, citation 19, and appendix for description of the Laboratory Acute Physiology Score, version 2 as well as the discharge score (LAPS2dc). The univariate relationship of an admission LAPS2 with 30 day mortality is as follows: 0 – 59, 1.0%; 60 – 109, 5.0%, 110+, 13.7%; for LAPS2dc, the relationship with 30 day mortality is as follows: 0 – 59, 2.2%; 60 – 109, 8.1%, 110+, 20.5%.See citation 24 and the appendix for description of how Agency for Healthcare Research and Quality software was employed to group diagnoses into primary conditions. See text for study outcomes definitions. Mortality includes deaths that occurred in and outside the hospital. APPENDIX 5: BIVARIATE COMPARISONS (PATIENT AS UNIT OF ANALYSIS)SELECTED CHARACTERISTICS OF INDEX HOSPITALIZATIONS WITH AND WITHOUT THE 30-DAY COMPOSITE OUTCOME1 BIVARIATE COMPARISONS1Index hospitalizations not followed by the composite outcomeIndex hospitalizations followed by the composite outcomeN32592034116Age (mean ± SD2)60.8 ± 18.070.2 ± 16.7Sex (% male)45.5%48.4%COPS23 (mean ± SD)20.5 ± 21.934.8 ± 32.2Charlson score4 (median, IQR)1 , 22 , 2LAPS25 (mean ± SD)47.2 ± 36.172.5 ± 42.3LAPS2dc (mean ± SD)38.4 ± 22.251.5 ± 29.0% with these primary conditions6 Community-acquired pneumonia1.6%2.5%Sepsis6.8%11.7%Gastrointestinal bleeding1.4%1.5%Hip fracture1.5%2.1%Any malignancy7.8%5.5%“Full code” at admission (%)793.2%79.5%“Full code” at time of hospital discharge (%)92.2%73.5%Admitted through emergency department (%)57.4%75.9%Length of stay (mean ± SD)4.5 ± 5.96.5 ± 10.2Ever admitted to ICU8 (%)13.1%19.6%Experienced unplanned transfer to ICU (%)2.1%4.7%Experienced 1+ surgical procedure after already experiencing one such procedure9 (%)4.7%9.9%Footnotes to This table provides bivariate comparisons using individual patients as the unit of analysis. It is a sister to Table 2 in the main manuscript (which reports data using an index hospitalization as the unit of analysis). If a patient had multiple index hospitalizations then only the first one is retained for this table.All variables are significantly different from the mean or median with a p value of <0.0001 except for gastrointestinal bleeding. The p value for gastrointestinal bleeding is 0.0742. SD: standard deviation; IQR: interquartile rangeSee text, citation 19, and Table 1 for details of the COmorbidity Point Score, version 2. See citations 19 and 30 for description of methodology used to assign this score.See text, citation 19, and Table 1 for details of the the Laboratory Acute Physiology Score, version 2 as well as its discharge version. See citation 24 for description of how Agency for Healthcare Research and Quality software was employed to group diagnoses into primary conditions.See citation 19 for details on how patient care directives are captured in the electronic record.ICU: intensive care unit. See citations 13 and 19 and the Appendix for a description of how we employed bed history data to distinguish between ever admits to the ICU and unplanned transfer to the ICU. This is a proxy for a major surgical complication (i.e., one requiring return to the operating room). See citations 13 and 19 for details on how we employed bed history data. For this item, the denominator consists of 158,299 hospitalizations in which the patient had surgery; of these, 9,055 experienced the combined outcome. APPENDIX 6: COMPARISON OF 7 DAY MODELS TO LACEMODEL2[95% Confidence Interval]Difference vs LACE3Point Estimate [95% Confidence Interval]METRIC1LACEED 7Discharge Day 7 EMR – admitEMR – dischargec statistic0.690(0.686, 0.694)0.698(0.694, 0.701)0.715(0.711, 0.719)0.008(0.004, 0.011)0.025(0.022, 0.028)R20.0709(0.0679, 0.0740)0.0778(0.0744, 0.0809)0.0900(0.0862, 0.0934)0.0069(0.0040, 0.0097)0.0191(0.0163, 0.0216)Calibration break40%30%20%NRI (vs. LACE)0.00004(-0.00003, 0.00016)0.0080(0.0066, 0.0095)IDI (vs. LACE)0.0034(0.0027, 0.0042)0.0135(0.0126, 0.0144)Footnotes Metrics (top: value of metric; bottom: 95% confidence interval) are as follows: c statistic is the area under the receiver operator characteristic curve|cit|; R2 is the Nagelkerke pseudo-R2|cit|; calibration break refers to the percentile range at which the predictive model deteriorates (see text and appendix 7 for details and graphical displays); NRI (net reclassification improvement) and IDI (integrated discrimination improvement) are calculated according to the methodology of Pencina et al. (see citation 28); for NRI, we used a threshold of ≥ 40% risk for the composite study outcome.ED = electronic medical record models using data available at the time of admission Discharge Day=electronic medical record models using data available at the time of discharge; LACE = length of stay, acuity, Charlson, and emergency visits score of van Walraven et al. (see citation 8).The point estimate is the result from a simple subtraction of one column from another. The confidence intervals are the 2.5% and 97.5% quantiles after 1000 bootstrap replications. APPENDIX 7: CALIBRATION CURVES IN VALIDATION DATASETThe figures in the pages that follow provide a graphical illustration of the calibration of the various models we tested for a combined outcome (death or undesirable rehospitalization within 7 or 30 days).These figures present information as LEFT: The X axis shows 10 predicted ranges (< 10%, 10 to < 20%, etc.) for the combined outcome, while the Y axis shows the actual observed rate (with its associated 95% confidence interval) in the validation dataset for all observations with that predicted risk. The dotted line shows what would be found were calibration to be RIGHT: This figure shows the distribution of observations with a given probability of the outcome where the patient did not have the combined outcome (0, top) and those where the patient did (1, bottom). As can be seen by examining sequential figures of this type, as a model performs better, the “spread” between the two subsets will increase.BOTTOM LEFT: This figure splits all observations in the validation dataset into 10 deciles on predicted probability of the outcome shows the number of index hospitalizations where the patient was expected to have the outcome (black bars) as well as the number of index hospitalizations where the patient actually had the outcome (grey bars).BOTTOM RIGHT: The X axis shows 10 predicted outcome ranges (< 10%, 10 to < 20%, etc.), while the Y axis shows the total number of index hospitalizations that fell within each of these ranges.Model: LACE Outcome time frame: 30 daysModel: EDOutcome time frame: 30 daysModel: Discharge DayOutcome time frame: 30 daysModel: LACEOutcome time frame: 7 daysModel: EDOutcome time frame: 7 daysModel: Discharge DayOutcome time frame: 7 daysAPPENDIX 8: RELATIVE CONTRIBUTION OF PREDICTORSRelative contribution of a specific covariate was calculated in the following manner. First we determined the log likelihood from the full model and also separately from dropping each covariate in turn. The relative contribution of a specific covariate was a ratio where the denominator was the sum of the changes in the log likelihood and the numerator was the change from dropping that specific covariate. The prior hospitalization category is a four-level factor that captures any hospitalizations before admission: none up to 30 days previous; 1+ up to 7 days previous and none 8-30 days previous; none up to 7 days previous and 1+ 8-30 days previous; and 1+ up to 7 days previous and another 1+ 8-30 days previous.Model?Outcome time frameRelative contribution of predictors???AGESEXCOPS2LAPS2ED 7 days5.40%0.60%47.40%46.50%ED30 days5.70%0.10%61.90%32.30%ModelOutcome time frameRelative contribution of predictors??COPS2LAPS2Length of StayDischarge care directive Prior hospitalization categoryDischarge Day 7 days22.10%31.80%13.30%17.40%15.50%Discharge Day30 days36.90%25.50%8.90%13.00%15.70%APPENDIX 9: SUPPLEMENTAL ANALYSES – 7 AND 30 MODEL PERFORMANCE CHARACTERISTICS TABLE 9.1: OPERATIONAL PERFORMANCE CHARACTERISTICS OF 30 DAY MODEL IN VALIDATION DATASETRisk threshold1Work-up to detection ratio2≥ 10%≥ 20%≥ 30%≥ 50%LACE4.483.372.622.16ED 304.483.292.671.90Discharge Day 304.052.932.451.97Each cell is formatted as “30 day admit model / 30 day discharge model”Risk threshold1≥ 10%≥ 20%≥ 30%≥ 50%Work-up to detection ratio24.52 / 4.103.30 / 2.932.67 / 2.451.90 / 1.98% capture3Death or undesirable hospitalization84.2 / 80.355.6 / 51.130.2 / 31.12.5 / 10.2Death96.3 / 95.674.6 / 76.144.1 / 51.74.1 / 19.3Undesirable hospitalization81.8 / 77.051.7 / 45.527.4 / 26.52.2 / 8.2Undesirable hospitalization days83.7 / 80.954.7 / 50.631.1 / 31.23.2 / 10.9Any hospitalization76.9 / 72.446.7 / 41.324.2 / 23.61.9 / 7.2Any hospitalization days80.3 / 77.950.7 / 47.428.1 / 28.72.8 / 9.8Footnotes to table Results for the admit model are in plain font; for the discharge model, in bold italics font; 7 day models’ results are in the appendix.Refers to the predicted risk assigned by either the admit or discharge modelRefers to the ratio of number of patients flagged as having at least a risk of 10, 20, 30 or 50% to the number who actually had the composite outcome within the 30 day time frame. Thus, at a ≥ 10% risk level, the admit model flags 172,667 patients in the validation dataset, of whom 38,168 had the study outcome, giving a W:D of 4.52; in contrast, this ratio falls to 2.67 at a predicted risk threshold of ≥ 30%.Refers to the proportion of indicated outcomes that are identified at the specified threshold. For example, in the entire validation dataset there are 45,304 patients with the study outcome. At a ≥ 10% risk level, the admit model flags 172,667 patients in the validation dataset, of whom 38,168 had the study outcome. Thus the percent of all outcomes caputured by this threshold is 84.2%.APPENDIX TABLE 4: OPERATIONAL PERFORMANCE CHARACTERISTICS OF 7 DAY MODEL IN VALIDATION DATASETRisk threshold1Work-up to detection ratio2≥ 10%≥ 20%≥ 30%≥ 50%LACE7.125.203.75NAED 77.144.424.27NADischarge Day 76.354.493.612.77Each cell is formatted as “7 day admit model / 7 day discharge model”Risk threshold1≥ 10%≥ 20%≥ 30%≥ 50%Work-up to detection ratio27.11 / 6.304.42 / 4.494.27 / 3.58NA / 2.77% capture3Death or undesirable hospitalization37.6 / 37.44.2 / 10.20.2 / 3.1NA / 0.2Death60.4 / 69.59.1 / 23.40.4 / 7.0NA / 0.5Undesirable hospitalization32.5 / 29.93.0 / 7.00.2 / 2.2NA / 0.2Undesirable hospitalization days35.0 / 34.44.2 / 9.50.3 / 3.5NA / 0.3Any hospitalization29.5 / 27.42.7 / 6.40.2 / 2.0NA / 0.1Any hospitalization days32.9 / 32.73.8 / 9.00.3 / 3.2NA / 0.3Footnotes to table Results for the admit model are in plain font; for the discharge model, in bold italics font.Refers to the predicted risk assigned by either the admit or discharge modelRefers to the ratio of number of patients flagged as having at least a risk of 10, 20, 30 or 50% to the number who actually had the composite outcome within the 7 day time frame. Refers to the proportion of indicated outcomes that are identified at the specified threshold. APPENDIX 10: Supplemental analyses: Impact of incorporating diagnosis in EMR models We assessed the value of incorporating diagnosis information following the general guidance of Pepe et al. (citation 29). Each of the four models was rebuilt and we then recalculated the various fit assessment metrics (such as the c-statistic). In each of the four, this new model used an intercept, the estimated probability from the original model, the diagnosis (as a factor), and an interaction between the latter two. Using the derivation data the optimum weight for each covariate was allowed to deviate from the value of zero (for the intercept) or one (for the other covariates). Thus diagnosis adjusts the intercept and weight for the predicted value separately for each diagnosis. The results below are from the validation dataset. Details on how individual diagnoses were grouped are provided in citation 19 and also in Appendix 11.C-statisticNagelkerke pseudo-R2-2*log(likelihood)NRI @ 40%IDIWithout DxWith DxWithout DxWith DxWithout DxWith DxWithout vs With Without vs With ED 70.6980.7020.07810.07911344471343190.00560.0031Discharge Day 70.7150.7050.08920.07971331341342530.0087-0.0027ED 30 0.7390.7430.15770.15962274502270820.02420.0051Discharge Day 30 0.7550.7490.17220.1646224675226133-0.0053-0.0038APPENDIX 11: Supplemental analyses – Model performance across subgroups in validation datasetGROUPING OF ICD CODES INTO PRIMARY CONDITIONSAs described in citation 19, we combined Health Care Utilization Project (data/hcup) single-level diagnosis clinical classification software (CCS) categories to cluster all possible ICD admission diagnosis codes into 30 groups, which we refer to as Primary Conditions. The HCUP single-level diagnosis CCS categories were grouped based on biologic plausibility (i.e., relative similarity from a disease standpoint) and on the observed mortality rate because, for modeling purposes, it was desirable to have ~30 patient groupings with at least 30-40 deaths in the derivation dataset. The table below shows our 30 groupings with their corresponding HCUP category numbers.Primary Condition NameHCUP single-level diagnosis clinical classification software (CCS) category number(s) Sepsis2Fluid and electrolyte disorders55Coma; stupor; and brain damage85AMI100Cardiac arrest107CHF108Acute CVD109CAP122GI bleed153UTI159Hip fracture226Residual codes259Renal failure (all)156, 157, 158Less severe cancer11-16, 18, 20-26, 28-32, 34, 36, 37, 44-47, 207Endocrine & related conditions48-51, 53, 54, 56, 57, 200, 202, 210, 211Miscellaneous GI conditions137-140, 155, 214Primary Condition NameHCUP single-level diagnosis clinical classification software (CCS) category number(s) Other cardiac conditions96-99, 103-105, 114, 116, 117, 213, 217HCUP Hyper Group 1101, 102, 106HCUP Hyper Group 20, 10, 141, 144-146, 147, 154, 160-166, 168-196, 201, 215, 218-224, 241-243, 255, 256, 258, 650-652, 654-663, 670, 999, 2601-2621HCUP Hyper Group 3115, 129, 131, 249HCUP Hyper Group 4127, 128, 130, 132, 133Hematologic conditions59-64Ill-defined signs and symptoms250-253Liver and pancreatic disorders151, 152Highly malignant cancer17, 19, 27, 33, 35, 38-43Miscellaneous neurological conditions79-84, 93-95, 110-113, 216, 245, 653Problems with nutrition52, 58Other infectious conditions1, 3-9, 76-78, 90, 92, 123-126, 134, 135, 148, 197-199, 201, 246-248Miscellaneous surgical conditions86-89, 91, 118-121, 136, 142, 143, 167, 203, 204, 206, 208, 209, 212, 237, 238, 254, 257Trauma205, 225, 227-236, 239, 240, 244c statistic (area under receiver operator characteristic curve)RTCO7ARTCO7DRTCO30ARTCO30DGenderMale0.69100.71100.73500.7510Female0.70300.71800.74300.7590AgeAge < 650.69700.72000.74300.7620Age ≥ 650.66500.68500.70100.7220Bed history-basedPatients ever in ICU0.65000.68100.69400.7180Patients who experienced unplanned transfer to the ICU (EDIP outcome)0.63700.67600.67100.6990Patients who ever had an operating room stay0.68600.70700.72500.7460Principal HCUP SGSepsis 0.66800.69500.69500.7220Fluid and Electrolyte Disorders0.64700.65100.66100.6690Coma, Stupor, and Brain Damage0.69400.72300.68600.7460AMI 0.68400.67900.72000.7220Atherosclerosis0.66200.66900.69500.6980Chest Pain 0.67500.68400.72700.7370Dysrhythmia0.63900.66300.68700.7010Cardiac Arrest0.61000.61900.71300.6890CHF 0.61500.63900.62700.6530Acute CVD0.65000.68900.68600.7100CAP 0.63000.64900.66700.6860ASP Pneumonia0.55300.60200.58500.6210GI Bleed0.65200.67800.70300.7220UTI 0.62900.66200.67400.6890Hip Fracture0.65700.69600.69400.7080Residual Codes0.72400.73300.73200.7570Renal Failure (All)0.62500.64500.65800.6880Less Severe Cancer0.70800.73700.73200.7650Catastrophic Conditions0.62000.65700.68000.7050COPD Asthma and Misc Lung Problems0.62500.65500.67400.6950Endocrine and Related Conditions0.68600.71400.71100.7410Miscellaneous GI Conditions0.68700.70000.70100.7190Other Cardiac Conditions0.65800.68800.70100.7190Hematologic Conditions0.62600.66000.66700.6910Ill Defined Signs and Symptoms0.63200.63600.69200.7000Liver and Pancreatic Disorders0.67900.69600.71100.7380Miscellaneous Conditions0.68700.66600.72300.7260Highly Malignant Cancer0.70400.73900.73100.7650Miscellaneous Neurological Conditions0.66700.69400.71600.7400Problems with Nutrition0.77400.78000.81100.8220Obstruction and Diverticula0.63800.65400.66400.6920Other GI0.61300.62900.68700.7010Other Genitourinary Conditions0.63900.69500.69200.7310Other Infectious Conditions0.68800.68800.73200.7410Other Respiratory Conditions0.67300.68500.71000.7140Psychiatric Conditions0.68700.71900.67900.6950Miscellaneous Surgical Conditions0.72800.74100.76800.7780Trauma0.74000.73800.77500.7710Nagelkerke pseudo-R2RTCO7ARTCO7DRTCO30ARTCO30DGenderMale0.07430.08830.15490.1711Female0.08030.08870.15900.1720AgeAge < 650.07240.08580.15030.1630Age ≥ 650.05930.06950.12370.1404EventsPatients ever in ICU0.04080.06950.10810.1374Patients who experienced unplanned transfer to the ICU (EDIP outcome)0.00900.06500.05650.1079Patients who ever had an operating room stay0.06010.06700.11580.1243Principal HCUP SGSepsis 0.06380.08500.12000.1476Fluid and Electrolyte Disorders0.04960.05750.07200.0771Coma, Stupor, and Brain Damage0.06950.10020.09500.1743AMI 0.05660.04030.13490.1303Atherosclerosis0.04960.05830.08970.0970Chest Pain 0.01720.03890.06440.1061Dysrhythmia0.04250.05760.09430.1109Cardiac Arrest0.0143-0.01510.11220.0803CHF 0.02960.04300.05100.0649Acute CVD0.02350.05260.08750.1095CAP 0.03560.04700.07940.0963ASP Pneumonia-0.0758-0.0274-0.0435-0.0167GI Bleed0.04620.06850.11140.1446UTI 0.04190.05330.09800.1031Hip Fracture0.04230.07430.10120.1155Residual Codes0.09380.10280.14670.1651Renal Failure (All)0.02950.03920.07850.1079Less Severe Cancer0.07100.08100.12580.1534Catastrophic Conditions0.01810.03980.09740.1244COPD Asthma and Misc Lung Problems0.01960.04460.08780.1179Endocrine and Related Conditions0.06510.08200.12360.1497Miscellaneous GI Conditions0.06420.07780.10960.1262Other Cardiac Conditions0.05220.07130.11720.1379Hematologic Conditions0.02840.05320.07250.1017Ill Defined Signs and Symptoms0.02150.02540.09630.0995Liver and Pancreatic Disorders0.05900.07230.11950.1594Miscellaneous Conditions0.05730.04150.13180.1265Highly Malignant Cancer0.03180.06700.04900.1027Miscellaneous Neurological Conditions0.05330.07060.11810.1432Problems with Nutrition0.14860.14260.27130.2652Obstruction and Diverticula0.02980.04210.06320.1008Other GI0.00940.02840.09050.1105Other Genitourinary Conditions0.03360.03770.08390.0900Other Infectious Conditions0.06970.06790.15010.1506Other Respiratory Conditions0.06210.06030.13070.1200Psychiatric Conditions0.05670.09060.08060.1020Miscellaneous Surgical Conditions0.08650.08990.16410.1609Trauma0.09100.08180.15550.1333APPENDIX 12: BETA COEFFICIENTS FOR MODELS FROM DERIVATION DATAThis paper developed four models: one for ED admission for a 7-day horizon (real time combined outcome, or “RTCO7A”), one for the same with a 30-day horizon (“RTCO30A”), and two more for discharge day (“RTCO7D” and “RTCO30D”). Below are the specifications of those four models. To briefly summarize, the two at admission each are a six-node tree followed by a logistic regression for each node, and the two models at discharge are simply a single logistic regression.Below are the tree portion of RTCO7A and then of RTCO30A. In each call the left-most terminal node as “cohort 1”, continue with the node to the right of that as “cohort 2”, etc until the right-most terminal node as “cohort 6”. These trees were grown with six nodes in this specific structure as such trees have been useful for our group in the past: cohort 1 is to have a relatively small sample size but the highest outcome rate, cohort 6 is to be with a relatively large sample size and the lowest outcome rate, and all the other cohorts are to be with different sample sizes that may end up with similar outcome rates with each other.RTCO7ARTCO30ABelow are the results of the logistic regression fits for all four models. In terms of the discrete covariates, MALE is an indicator if the patient was male or not, and its reference level was “male”. DCO_4 is an indicator for the care level order in effect at discharge, and its reference level was “full code”. PCAT categorizes the pattern of any prior hospitalizations with these four levels, with the reference level being “4”.:LevelMeaning1None up to 30 days ago21+ up to 7 days ago, and none 8-30 days ago3None up to 7 days ago, and 1+ 8-30 days ago41+ up to 7 days ago, and another 1+ 8-30 days agoModel and CohortCovariateCompared LevelPoint EstimateStandard Error95% Confidence IntervalLowerUpperRTCO7A1INTERCEPT-3.8320.228-4.279-3.385AGE0.0070.0020.0030.011MALEFemale-0.0350.023-0.0800.011COPS20.0030.0000.0030.004LAPS20.0100.0010.0080.0122INTERCEPT-3.1880.178-3.537-2.839AGE-0.0010.001-0.0040.001MALEFemale-0.0070.017-0.0410.026COPS20.0050.0000.0040.006LAPS20.0080.0020.0050.0113INTERCEPT-3.3410.077-3.491-3.190AGE0.0000.001-0.0020.002MALEFemale-0.0110.014-0.0390.017COPS20.0060.0000.0060.007LAPS20.0070.0010.0060.0084INTERCEPT-4.4740.086-4.644-4.305AGE0.0130.0010.0110.015MALEFemale-0.0590.017-0.092-0.026COPS20.0080.0020.0040.012LAPS20.0090.0010.0080.0105INTERCEPT-4.6440.187-5.011-4.277AGE0.0080.0010.0050.011MALEFemale-0.0510.026-0.1030.000COPS20.0160.0040.0080.023LAPS20.0140.0040.0060.0216INTERCEPT-5.2020.112-5.422-4.983AGE0.0130.0020.0100.016MALEFemale-0.0870.024-0.133-0.040COPS20.0260.0030.0200.032LAPS20.0180.0030.0120.024RTCO30A1INTERCEPT-1.5260.098-1.718-1.333AGE-0.0060.001-0.008-0.004MALEFemale0.0140.013-0.0120.039COPS20.0060.0000.0050.006LAPS20.0070.0000.0060.0082INTERCEPT-2.7430.080-2.899-2.587AGE0.0030.0010.0010.004MALEFemale-0.0200.011-0.0410.002COPS20.0100.0010.0080.011LAPS20.0090.0000.0080.0103INTERCEPT-2.4450.064-2.570-2.319AGE-0.0020.001-0.004-0.001MALEFemale0.0060.012-0.0180.030COPS20.0090.0000.0090.010LAPS20.0100.0010.0080.0114INTERCEPT-3.9170.062-4.038-3.795AGE0.0150.0010.0140.016MALEFemale-0.0290.012-0.052-0.006COPS20.0150.0010.0120.018LAPS20.0100.0000.0090.0105INTERCEPT-6.1370.385-6.892-5.383AGE0.0400.0050.0310.050MALEFemale-0.0710.025-0.120-0.022COPS20.0140.0030.0080.019LAPS20.0120.0010.0090.0156INTERCEPT-4.2330.064-4.359-4.108AGE0.0060.0010.0040.008MALEFemale-0.0240.014-0.0520.004COPS20.0330.0020.0290.037LAPS20.0180.0010.0170.020RTCO7DINTERCEPT-3.2030.025-3.253-3.154COPS20.0050.0000.0050.006LAPS20.0080.0000.0080.008LOS_300.0380.0010.0360.041DCO_4Not Full Code0.2760.0090.2590.294PCAT1-0.4690.016-0.501-0.437PCAT20.0620.0240.0150.110PCAT3-0.0610.020-0.101-0.021RTCO30DINTERCEPT-2.3470.019-2.384-2.311COPS20.0090.0000.0090.009LAPS20.0090.0000.0090.009LOS_300.0410.0010.0390.043DCO_4Not Full Code0.3050.0070.2920.318PCAT1-0.5810.013-0.606-0.557PCAT20.0050.019-0.0320.042PCAT30.0580.0160.0270.088APPENDIX 13: KAPLAN-MEIER CURVES FOR MODELSKaplan-Meier curves for patients in the validation dataset, which was divided into terciles based on predicted risk for the combined outcome (undesirable rehospitalization and/or death within 30 days). The model employed was the 30 day discharge model. The dotted line (???) shows the lowest risk tercile (predicted risk up to 6.2%); the dashed line (- - -) hospitalizations with predicted risk of 6.2 to 9.6%; and the solid line (—) hospitalizations with predicted risk of 9.6% or more. -76390586995The Pop-Up will display until “Accepted”. The Link will Display the Report0The Pop-Up will display until “Accepted”. The Link will Display the Report96774092075-602615422275The Report Contains the Scores and Last Time it was Calculated, along with Additional Information0The Report Contains the Scores and Last Time it was Calculated, along with Additional Information64325512065042545-728980New columns in the Patient List activity show the latest Advance Alert (EDIP) scores & the admission LAPS2 & COPS2New columns in the Patient List activity show the latest Advance Alert (EDIP) scores & the admission LAPS2 & COPS21213485522605Appendix 15: Relationship between model predictors and disaggregated outcomes (odds ratios using univariate logistic regression)The tables below show the following general trends in the relationship between model predictors and our composite outcome (death or non-elective rehospitalization within 30 days) as compared to each outcome alone (non-elective rehospitalization within 30 days only, death within 30 days only):The directionality of predictor-outcomes relationships is similar across outcomesCare directives are very strong predictors for mortality, but, nonetheless, they are still significant predictors for non-elective rehospitalization (even when the patient survived)As is the case with care directives, physiologic derangement (LAPS2) and longitudinal comorbidity burden (COPS2) are very strong predictors for mortality (particularly when very elevated – for example, a LAPS2 of ≥ 110 has an odds ratio of 18.6 for mortality). Nonetheless, they are still strong predictors for rehospitalization even when the patient survived.PredictorComposite outcome (death or non-elective rehospitalization within 30 days)Non-elective rehospitalization within 30 daysDeath within 30 daysWith survivalWith in-hospital deathAny Out of hospitalAnyAge in years<40 (reference)40-641.42 (1.38,1.47)1.26 (1.22,1.3)3.57 (2.94,4.33)1.32 (1.27,1.36)5.62 (4.71,6.71)4.54 (3.96,5.19)65-792.25 (2.19,2.33)1.74 (1.69,1.8)7.83 (6.47,9.47)1.91 (1.85,1.97)14.62 (12.28,17.41)11.08 (9.69,12.65)80+3.39 (3.29,3.50)1.95 (1.89,2.02)14.38 (11.90,17.38)2.31 (2.23,2.38)39.37 (33.09,46.85)27.36 (23.96,31.24)SexMale (reference)......Female0.89 (0.88,0.91)0.91 (0.90,0.93)0.82 (0.79,0.86)0.9 (0.88,0.91)0.88 (0.85,0.91)0.87 (0.85,0.89)Full CodeYes (reference)......No3.42 (3.36,3.47)1.39 (1.36,1.42)5.13 (4.90,5.37)1.70 (1.67,1.73)21.44 (20.70,22.20)15.46 (15.01,15.91)Hospital length of stay (calendar days)< 2......20.96 (0.92,1.01)0.90 (0.86,0.95)1.67 (1.31,2.12)0.93 (0.89,0.98)1.26 (1.10,1.45)1.36 (1.20,1.55)31.39 (1.33,1.46)1.22 (1.16,1.28)2.92 (2.31,3.69)1.29 (1.23,1.35)2.25 (1.97,2.57)2.44 (2.15,2.76)4-51.91 (1.82,1.20)1.56 (1.49,1.64)4.41 (3.50,5.56)1.69 (1.61,1.77)3.55 (3.11,4.06)3.83 (3.39,4.33)6-303.08 (2.94,3.22)2.27 (2.16,2.38)7.66 (6.08,9.65)2.55 (2.43,2.67)6.30 (5.52,7.18)6.93 (6.13,7.83)PredictorComposite outcomeNon-elective rehospitalizationDeathWith survivalWith in-hospital deathAny Out of hospitalAnyLAPS2 (points)<50 (reference)......50-1092.77 (2.73,2.82)2.29 (2.25,2.33)5.25 (4.87,5.66)2.46 (2.42,2.51)5.43 (5.17,5.71)5.36 (5.13,5.60)110+5.76 (5.64,5.88)3.13 (3.06,3.21)14.21 (13.14,15.36)3.82 (3.73,3.91)19.21 (18.25,20.22)18.63 (17.80,19.49)COPS2 (points)<40 (reference)......40-642.48 (2.43,2.53)2.14 (2.09,2.19)3.26 (3.04,3.50)2.25 (2.20,2.30)3.23 (3.09,3.37)3.25 (3.12,3.38)65+4.51 (4.44,4.59)3.54 (3.48,3.60)6.47 (6.13,6.83)3.92 (3.86,3.99)5.50 (5.31,5.69)5.82 (5.64,6.00)Hospitalizations prior to current admissionNone within previous 30 days (reference)......≥ 1 up to 7 days ago, and none 8-30 days ago2.63 (2.55,2.70)2.36 (2.29,2.44)2.88 (2.65,3.12)2.49 (2.42,2.57)2.46 (2.33,2.60)2.59 (2.47,2.72)None up to 7 days ago, and ≥ 1 8-30 days ago3.19 (3.12,3.25)2.86 (2.79,2.93)3.36 (3.16,3.56)3.04 (2.97,3.10)2.71 (2.60,2.82)2.89 (2.79,3.00)≥ 1 up to 7 days ago, and ≥ 1 8-30 days ago5.49 (5.23,5.76)4.68 (4.45,4.93)5.07 (4.51,5.71)5.15 (4.90,5.41)3.52 (3.22,3.85)3.99 (3.70,4.32) ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download