Author + information
- Received December 17, 2010
- Accepted January 5, 2011
- Published online April 1, 2011.
- Stephen G. Ellis, MD⁎ (, )
- Mehdi H. Shishehbor, DO, MPH,
- Samir R. Kapadia, MD,
- A. Michael Lincoff, MD,
- Ravi Nair, MD,
- Patrick L. Whitlow, MD,
- Christopher T. Bajzer, MD,
- Leslie L. Cho, MD,
- E. Murat Tuzcu, MD,
- Russell Raymond, DO,
- Patrick Vargo, BS,
- Rebecca Cunningham, RN and
- Sandra J. Dushman-Ellis, MPH
- ↵⁎Reprint requests and correspondence:
Dr. Stephen G. Ellis, Cleveland Clinic, 9500 Euclid Avenue, J2-3, Cleveland, Ohio 44195
Objectives This study sought to improve methodology for predicting post–percutaneous coronary intervention (PCI) mortality.
Background Recently, an increased proportion of post-PCI deaths caused by noncardiac causes has been suggested, often in rapidly triaged patients resuscitated from sudden cardiac death or presenting with cardiogenic shock. Older risk adjustment algorithms may not adequately reflect these issues.
Methods Consecutive patients undergoing PCI from 2000 to 2009 were randomly divided into training (n = 8,966) and validation (n = 8,891) cohorts. The 2010 ACC-NCDR (American College of Cardiology–National Cardiovascular Data Registry) mortality algorithm was applied to the training cohort and its highest risk decile, separately. Variables describing general and neurological status at admission were then tested for their additional predictive capability and new algorithms developed. These were tested in the validation cohort, using receiver-operator characteristic curve, Hosmer-Lemeshow, and reclassification measures as principal outcome measures.
Results In-hospital mortality was 1.0%, of which 52.2% had noncardiac causes or major contributions. Baseline model C-statistics for the total and upper decile training cohorts were 0.904 and 0.830. The Aldrete score (addressing consciousness, respiration, skin color, muscle function, and circulation) and neurology scores added incremental information, resulting in improved validation cohort C-statistics (entire group: 0.883 to 0.914, p < 0.001; high-risk decile: 0.829 to 0.874, p < 0.001). Reclassification of the ACC-NCDR <90th and ≥90th risk percentiles by the new score yielded improved mortality prediction (p < 0.001 and p = 0.033, respectively).
Conclusions Half of in-hospital deaths in this series were of noncardiac causation. Prediction of in-hospital mortality after PCI can be considerably improved over conventional models by the inclusion of variables describing general and neurological status.
Publicly available comparison of hospital outcomes has been advocated to improve both hospital quality and patient outcomes (1). Appropriate comparison requires risk adjustment modeling. Models to allow for comparison of mortality after percutaneous coronary intervention (PCI) have been refined over the past 2 decades (2). Perhaps the most widely used model is that by the ACC/NCDR (American College of Cardiology–National Cardiovascular Registry) (2,3). Although an excellent model by current standards when initiallytested, it derives from a database originally intended to categorize variables and outcomes most directly related to the procedure itself and its cardiac outcomes. As noncardiac mortality has assumed an increasingly relative role in overall in-hospital mortality, one might postulate the addition of variables that might relate to noncardiac mortality in this setting might provide incremental prognostic and adjustive value.
Study population and clinical end point
All data were retrieved from our Institutional Review Board–approved coronary interventional database. Consecutive patients treated for the first time at our institution from January 2000 through December 2009 were studied. Patients were divided into a developmental cohort (n = 8,966) and a validation cohort (n = 8,891) using a random number generator. Data were coded by trained cardiologists and nurses according to American College of Cardiology definitions, and cross-validated by a trained cadre of research nurses. For patients who expired, specific data were recorded on dedicated report forms, and a hospitalization summary was prepared by 1 of the trained research nurses. Final adjudication was performed by 1 of the authors (S.G.E.).
We initially applied the ACC/NCDR basic pre-catheterization model for in-hospital death following PCI (3) to the developmental cohort (the full model was not used as certain variables [e.g., subacute thrombosis] had not been captured for the entirety of the data collection period).
The following variables were then tested for additional prognostic value using multivariate logistic regression analyses with backward elimination and a retention p value <0.05: Aldrete score (originally formulated to evaluate recovery from anesthesia) (4) (Fig. 1), neurological score (both scores being routinely coded by the catheterization laboratory nurses before the procedure), and cardiac arrest within the preceding 24 h before PCI. A final model was constructed using the variable coefficients as multipliers for each variable in the additive model.
The developed models were then applied to the validation cohort with significance testing evaluated between ACC-NCDR and the new models' discrimination.
To assess model discrimination, we calculated the C-statistic (5). Hosmer-Lemeshow testing was performed across deciles of risk to assess calibration (6). Reclassification testing was pre-specified using the validation cohort's ACC-NCDR <90th and ≥90th percentiles, testing improvement in correlation with mortality with the new models. Net reclassification index (7) was also calculated.
A similar strategy was applied separately to the highest risk decile determined by ACC-NCDR risk modeling, except Hosmer-Lemeshow statistics were assessed across quintiles (because there were fewer patients evaluated).
All tests were 2-sided. A value of p < 0.05 was considered significant for all tests. Statistical calculations were performed using a statistical program (SYSTAT, Version 11, Systat Systems, Richmond, California).
Baseline demographics, treatments, and summary outcomes for the training and validation cohorts are shown in Table 1. The incidence of cardiac arrest before the time of starting PCI increased from 0.6% during 2000 to 2003, to 0.85% during 2004 to 2006, and to 1.88% during 2007 to 2009 (p < 0.001). In-hospital mortality for those 3 periods was 1.0%, 0.8%, and 1.3%, respectively. Overall mortality in the entire cohort was 1.0% and, since we began coding causes of death in 2007, 52.2% were of noncardiac cause (31.2% neurological or with major neurological contribution, 8.3% pulmonary, 6.3% bleeding).
In the training sample, Aldrete score, individual components of the Aldrete score, cardiac arrest, and the neurological score were each individually highly correlated with mortality (Fig. 2, Table 2).
C- and Hosmer-Lemeshow statistics for the training and validation, global, and high-risk cohorts are shown in Table 5. Validation cohort C-statistics were improved with the new model in both the entire and highest risk decile cohorts (both p < 0.001) (Table 5). Reclassification by the new score in the validation cohort led to improved discrimination for mortality (ACC-NCDR <90th percentile reclassified to ≥90th percentile, no vs. yes: 0.17% vs. 2.87% mortality, p < 0.001; and ACC-NCDR ≥ 90th percentile reclassified to ≤90th percentile, yes vs. no: 0.86% vs. 4.50% mortality, p = 0.033). The net reclassification index = 0.141.
The new score performed comparably in the ST-segment elevation myocardial infarction and non–ST-segment elevation myocardial infarction populations (C-statistics: 0.861 and 0.898, respectively; interaction: p = 0.31), but did not perform as well in the salvage as in the nonsalvage population (C-statistics: 0.754 and 0.902, respectively; p < 0.001).
The principal findings of this study include: 1) In recent years, more than half of PCI-related in-hospital mortality appears to be due to, or has a large component related to, noncardiac causes. 2) Somewhat similar to other recent reports (8), the current ACC-NCDR in-hospital mortality model has very good, but not excellent, predictive capability as measured by receiver-operator characteristic testing (especially at the upper end of risk). 3) The addition of the easy-to-collect Aldrete score characterizing general patient status, and a simple neurological score, to the ACC-NCDR model appreciably improves prediction of in-hospital mortality.
The potential benefit of public reporting of risk-adjusted clinical outcomes is clear (1). Interventional cardiologists, taking the lead from our surgical colleagues, began sophisticated efforts to risk-adjusted outcomes in the 1990s (9–11). Typically, with the exception of renal failure, these models included only variables related to cardiac status (9–12). Nonetheless, at least using the metric of the C-statistic (13), these models achieved very good to excellent prediction of risk among general populations. Whereas these models have been refined over the years (3), they have continued to focus principally on cardiac-specific variables and, in their relation with mortality, have reported C-statistics between 0.82 and 0.92 (2,3,9–11). Despite this, there remain concerns in both the clinical community and among clinician researchers that models do not adequately address risk adjustment for patients at the highest risk (14,15). Perhaps as a consequence, there have been several reports of physician avoidance of high-risk patients once outcomes became reported publicly (e.g., dramatic reduction in patients with cardiogenic shock treated with PCI in Massachusetts between 2003 and 2005) (16,17).
Resnic and Welt (16) were perhaps the first to draw attention to the fact that much of the in-hospital mortality after PCI is no longer related to the procedure. In their study of the Brigham and Women's Hospital's (2003 to 2005) experience, 45% of patients who died had at least 1 severe acute medical condition not accounted for by data collection in the ACC-NCDR registry, and 42% of patients died of a noncardiac cause.
The results of several key clinical trials: establishing the superiority of PCI over fibrinolysis (18), suggesting lesser need for revascularization in modestly symptomatic patients (19,20), and the survival benefit for cooling of cardiac arrest patients (21,22) (leading to more aggressive treatment of these very ill patients) has changed the nature of patients undergoing PCI. In fact, PCI within 24 h of acute myocardial infarction has increased from 18.5% in 1999 to 31.8% in 2009 of all cases reported to the ACC-NCDR (23). Concomitantly, the improving technical and pharmacological armamentarium available to the interventionalist has reduced the ischemic complications of PCI (24). It would not be surprising, therefore, if the clinical correlates of PCI-related mortality have changed somewhat over the last decade.
Several investigators have noted the difficulty in increasing a model's C-statistic with the addition of highly significant parameters with odds ratios of 3 or even higher (7,13,25). As an example, the difference in Framingham risk score with or without the variable current smoking (risk ratio [RR]: 2.9) in the Women's Health Study data (n = 26,901) was only 0.76 versus 0.78 (26). The present model improves on the ACC-NCDR model with discrimination (C-statistic) increasing by 0.031 and 0.045 in the entire validation and high-risk decile validation cohorts, respectively, without loss of calibration (Hosmer-Lemeshow statistic). Reclassification tests show significant improvement in prediction mortality also.
Importantly, the model is most appreciably improved for patients at the highest decile of risk. That said, accurately assessing risk at the very high end of the risk spectrum (e.g., the 0.5% of all patients undergoing “salvage” procedures with 44% mortality) is a particular challenge. Perhaps such patients should be excluded from scorecarding altogether.
This study has several limitations. First, it derives from the experience of a single institution and hence will require external validation. Second, accurate prediction of risk for the highest risk patient remains a challenge. Despite the fact that factors such as poor neurological status (neurological score 0 to 1) increase the risk of mortality over 30-fold, the highest risk percentile of patients only have a 35% predicted mortality, and it is not until risk scores get into the upper fraction of this percentile that expected mortality exceeds 50%. The vast majority of patients undergoing PCI are low risk (75% have predicted mortality less than 0.5%) (Fig. 3B). Third, follow-up through hospital discharge allows analysis only of a short window in time. Better models evaluating long-term outcomes are needed. Lastly, we could not assess the influence of Aldrete score on the full ACC-NCDR model for reasons noted in the Methods section. However, there is little reason to think that the addition of general and neurological assessments would not add additional prognostic information in that the full ACC-NCDR model does not contain any variables touching on clinical parameters evaluated by the Aldrete score (Fig. 4).
Accurate prediction of risk for patients undergoing PCI should improve patient selection for this procedure and facilitate appropriate comparison of outcomes among operators and among hospital systems. Our results suggest that routine, accurate, and monitored collection of data relating to the patient's general and neurological status before PCI should become standard as we attempt to improve overall patient care. Appreciation of this fact should be considered as only 1 step in the iterative process of healthcare improvement in the field of interventional cardiology.
Dr. Shishehbor has received honoraria from Abbott Vascular, Medtronic, and Medrad. Dr. Lincoff has received research grants from Roche, Bristol-Myers Squibb, AstraZeneca, Kai Pharmaceuticals, Pfizer, and Schering-Plough. Dr. Nair is an advisor for Boston Scientific and a speaker for Sanofi-Aventis and Daiichi-Sankyo. All other authors have reported that they have no relationships to disclose.
- Abbreviation and Acronym
- percutaneous coronary intervention
- Received December 17, 2010.
- Accepted January 5, 2011.
- American College of Cardiology Foundation
- Hollenbeak C.S.,
- Gorton C.P.,
- Tabak Y.P.,
- Jones J.L.,
- Milstein A.,
- Johannes R.S.
- Peterson E.D.,
- Dai D.,
- DeLong E.R.,
- et al.,
- NCDR Registry Participants
- Ellis S.G.,
- Omoigui N.,
- Bittle J.A.,
- et al.
- O'Connor G.T.,
- Malenka D.J.,
- Quinton H.,
- et al.,
- Northern New England Cardiovascular Disease Study Group
- Shaw R.E.,
- Anderson H.V.,
- Brindis R.G.,
- et al.
- Block P.C.,
- Peterson E.C.,
- Krone R.,
- et al.
- Cook N.R.
- Resnic F.S.,
- Welt F.G.P.
- Omoigui N.A.,
- Miller D.P.,
- Brown K.J.,
- et al.
- Grines C.,
- Patel A.,
- Zijlstra F.,
- et al.,
- PCAT Collaborators
- National Cardiovascular Data Registry
- Seshadri N.,
- Whitlow P.L.,
- Acharya N.,
- Houghtaling P.,
- Blackstone E.H.,
- Ellis S.G.
- Pepe M.S.,
- Janes H.,
- Longton G.,
- Leisenring W.,
- Newcomb P.