Author + information
- Received June 1, 2012
- Accepted July 25, 2012
- Published online November 1, 2012.
- Tullio Palmerini, MD⁎,†,
- Philippe Genereux, MD†,
- Adriano Caixeta, MD†,
- Ecaterina Cristea, MD†,
- Alexandra Lansky, MD‡,
- Roxana Mehran, MD§,
- Diego Della Riva, MD⁎,
- Martin Fahy, MSc†,
- Ke Xu, PhD† and
- Gregg W. Stone, MD†,⁎ ()
- ↵⁎Reprint requests and correspondence:
Dr. Gregg W. Stone, Columbia University Medical Center, New York–Presbyterian Hospital, The Cardiovascular Research Foundation, 111 East 59th Street, 11th Floor, New York, New York 10022
Objectives This study sought to develop a new score specific for patients with non–ST-segment elevation acute coronary syndromes (NSTEACS) undergoing percutaneous coronary intervention (PCI) (the ACUITY-PCI [Acute Catheterization and Urgent Intervention Triage Strategy–Percutaneous Coronary Intervention] risk score).
Background The TIMI (Thrombolysis In Myocardial Infarction) and GRACE (Global Registry for Acute Coronary Events) risk scores are recommended for risk stratification of patients with NSTEACS. However, these scores were not optimized for patients undergoing an early invasive strategy with PCI.
Methods The ACUITY-PCI risk score was created from data for 1,692 patients enrolled in the formal angiographic substudy of the ACUITY (Acute Catheterization and Urgent Intervention Triage Strategy) trial by integrating clinical, angiographic, laboratory, and electrocardiographic variables selected by multivariable analysis. The score was subsequently validated in a different population of 846 patients and compared with the GRACE and TIMI risk scores, and the SYNTAX (Synergy Between PCI with Taxus and Cardiac Surgery) and Clinical SYNTAX scores.
Results Six variables (2 clinical, 1 laboratory/electrocardiographic, and 3 angiographic) were included in the ACUITY-PCI score: insulin-treated diabetes; renal insufficiency; baseline cardiac biomarker elevation or ST-segment deviation; bifurcation lesion; small vessel/diffuse coronary artery disease; and the extent of coronary artery disease. Event rates increased significantly across tertiles of ACUITY-PCI score. Compared with the other scores, the ACUITY-PCI score had the best discrimination (C-statistic), calibration (Hosmer-Lemeshow statistic), and index of separation. Moreover, the net reclassification improvement varied from 9% to 38% and the integrated discrimination index from 1.9% to 2.7%.
Conclusions The ACUITY-PCI risk score is a new tool integrating clinical, angiographic, and laboratory/electrocardiographic variables specifically developed for patients with NSTEACS undergoing PCI. This score displayed better prognostic accuracy in terms of discrimination and calibration than other currently available scores for risk stratification of patients with NSTEACS. (Comparison of Angiomax Versus Heparin in Acute Coronary Syndromes [ACS]; NCT00093158)
Patients with non–ST-segment elevation acute coronary syndromes (NSTEACS) have a wide range of risk for morbidity and mortality according to their baseline risk factors, clinical syndrome acuity, and management strategy. Prospective risk stratification is essential to estimate patient prognosis and aid in clinical decision making. Current guidelines for the management of patients with NSTEACS recommend the use of the TIMI (Thrombolysis In Myocardial Infarction) risk score or the GRACE (Global Registry for Acute Coronary Events) score for risk stratification of patients with NSTEACS (1,2). These scores integrate several clinical, electrocardiographic, and cardiac biomarker variables, but they do not include angiographic variables and have not been optimized for patients undergoing percutaneous coronary intervention (PCI).
Recent studies have reported that angiographic variables contribute incremental prognostic information for risk stratification of patients with NSTEACS (3). The purely angiographic SYNTAX (Synergy Between PCI With Taxus and Cardiac Surgery) score has also shown to be an independent predictor of mortality, myocardial infarction (MI), and target vessel revascularization in patients with NSTEACS undergoing PCI (4). However, the SYNTAX score was not developed from a derivation cohort in NSTEACS with subsequent validation in a test cohort, but it was based on an arbitrary ranking of lesion complexity (5) and does not include clinical or other baseline variables. For these reasons, we sought to develop a new risk score integrating clinical, angiographic, laboratory, and electrocardiographic variables for risk assessment of 1-year mortality and MI in patients with NSTEACS undergoing PCI in the large multicenter, prospective randomized ACUITY (Acute Catheterization and Urgent Intervention Triage Strategy) trial. Moreover, the prognostic accuracy of this new score (the ACUITY-PCI risk score) was compared with that of currently available score systems for risk stratification of patients with NSTEACS.
The ACUITY trial design has been previously reported in detail (6). Briefly, ACUITY was a large, international, multicenter, prospective randomized trial of patients with moderate- and high-risk NSTEACS undergoing an early invasive strategy. Patients were randomly assigned before coronary angiography to heparin (unfractionated or enoxaparin) plus a glycoprotein IIb/IIIa inhibitor, bivalirudin plus a glycoprotein IIb/IIIa inhibitor, or bivalirudin monotherapy. Coronary angiography was performed in all patients within 72 h of randomization, after which patients were treated with PCI, coronary artery bypass graft (CABG) or medical therapy at the physician's discretion. All major adverse events observed during the study were adjudicated by an independent clinical events committee blinded to treatment assignment. The study protocol was approved by the institutional review board or ethics committee at each center, and written informed consent was obtained in all patients.
Objectives, study population, and definitions
The objective of this study was to develop and validate a new score integrating clinical, angiographic, laboratory, and electrocardiographic variables for risk prediction of 1-year death or MI in patients with NSTEACS and native coronary artery disease undergoing PCI. The prognostic accuracy of this new score was then compared with the SYNTAX score (5), Clinical SYNTAX score (7), TIMI score (8), and GRACE score (2) in terms of discrimination (9), calibration (10), index of separation (IoS) (1), net reclassification improvement (NRI), and integrated discrimination index (IDI) (1,11). The score was created and then validated in different cohorts of patients included in the formal angiographic substudy of the ACUITY trial. Quantitative coronary angiography (QCA) was performed as previously described (12) by experienced core angiographic laboratory technicians (Cardiovascular Research Foundation, New York, New York) blinded to treatment assignment and clinical outcomes.
The SYNTAX score and the Clinical SYNTAX score were calculated as previously described (4). The extent of coronary artery disease was defined as the sum of the lengths of all lesions in the coronary tree with >30% diameter stenosis in vessels with a reference diameter ≥1.5 mm determined by QCA. To determine this variable, QCA was performed of the entire coronary tree, and all lesions with >30% diameter stenosis were identified. As defined for the SYNTAX score, small vessel/diffuse coronary artery disease was considered present when at least 75% of the length of any segment proximal to the lesion, at the site of the lesion or distal to the lesion had a vessel diameter of <2 mm (5). The jeopardy score was defined as previously described (13). Renal insufficiency was defined as a calculated creatinine clearance of <60 ml/min determined by the Cockcroft-Gault equation.
Statistical analysis and score determination
Continuous data are presented as mean ± SD and were compared using the Student t test. Categorical variables were summarized as counts and percentages and were compared by chi-square test or Fisher exact test as appropriate. Correlations between angiographic and clinical variables included in the score were assessed by the Pearson test. The ACUITY-PCI risk score was created by fitting clinical, angiographic, laboratory, and electrocardiographic variables into a Cox multivariable analysis for risk prediction of 1-year death or MI. To not exclude variables potentially correlated with the outcome, univariable selection was performed setting the entry criteria at p ≤ 0.1. The multivariable model was then built by stepwise variable selection with same entry and exit criteria as in the univariable analysis. The following variables were considered: all variables included in the SYNTAX score (total occlusion, trifurcation, bifurcation, aorto-ostial lesion, severe tortuosity, length >20 mm, heavy calcification, thrombus, small vessel/diffuse coronary artery disease); pre- and post-procedural TIMI flow; jeopardy score; the extent of coronary artery disease (per 10-mm increment of lesions >30%); age; sex; diabetes; renal insufficiency; prior MI; prior PCI; baseline cardiac biomarker elevation; or ST-segment deviation ≥1 mm. The score was then formulated by attributing integer numbers to the variables retained in the multivariable model. The variable with the smallest estimated coefficient was attributed 1 point and was considered as the baseline variable. The score of the other variables were determined by dividing their estimated coefficients by the coefficient of the baseline variable. Multicollinearity between variables was assessed using the variance inflation factor. The simple random sampling method Proc Surveyselect (SAS, SAS Institute, Cary, North Carolina) was used to randomly assign patients to the derivation or validation cohort with 2:1 sample ratio. One-year outcomes were determined using Kaplan-Meier methodology and compared using the log-rank test.
Discrimination and calibration were determined by the C-statistic and the Hosmer-Lemeshow goodness-of-fit test, respectively (9,10). Differences in discrimination power between the ACUITY-PCI score and the 4 other scores (TIMI and GRACE scores, SYNTAX and Clinical SYNTAX scores) were evaluated using the chi-square test. The intrinsic prognostic information of each score was assessed by the IoS, which is the difference between the predicted probability of an event for a patient in the group with the worst prognosis and the predicted probability of an event for a patient in the group with the best prognosis (1). Prognostic accuracy of each score was further assessed by IDI, which is a measure of how well the model improves the integrated sensitivity without sacrificing integrated specificity, and the NRI, which is a measure of how well a model correctly reclassifies predicted probabilities (1,11). Statistical analyses were performed using SAS (version 9.2, SAS Institute, Cary, North Carolina). We considered p values <0.05 statistically significant.
The flow diagram of the study is shown in Figure 1. QCA was performed in 6,921 patients enrolled in the ACUITY trial angiographic substudy, including 3,826 patients who underwent PCI. After excluding patients with prior CABG (n = 862) and those for whom all angiographic (n = 337) or clinical (n = 89) variables were not available, 2,538 patients remained for the present analysis. The score was derived from data for 1,692 randomly selected patients and subsequently validated in the remaining 846 patients. As shown in Table 1, no significant differences in the baseline clinical characteristics were apparent between patients in the derivation and validation cohorts, except for insulin-treated diabetes, which was more frequent in the validation cohort. Similarly, no significant differences in 30-day or 1-year clinical outcomes were apparent between the 2 groups (Table 2).
The ACUITY-PCI risk score
As shown in Table 3, after univariable and multivariable selection, 2 clinical, 3 angiographic, and 1 laboratory/electrocardiographic variables remained significantly associated with the risk of 1-year death or MI. The scores attributed to each variable according to their estimated coefficients from the derivation dataset are shown in Table 3. Minimal correlation was apparent between these variables (Online Table 1), demonstrating that each retained variable provides unique prognostic utility. Moreover, the variance inflation factor showed absence of multicollinearity among variables in the model (Table 3). The C-statistic for this model was 0.67 (95% confidence interval [CI]: 0.62 to 0.72); the chi-square statistic for calibration was 7.13 (p = 0.52); and the IoS was 0.44. The range of each tertile and the frequency distribution of each variable across tertiles of ACUITY-PCI score are displayed in Online Table 2. As shown in Figure 2A, event rates in the derivation cohort increased significantly across tertiles of ACUITY-PCI score: 5.3% in the lower tertile; 9.1% in the middle tertile; and 19.0% in the upper tertile (p < 0.001). The hazard ratio (HR) of the 1-year composite rate of death or MI was 2.02 (95% CI: 1.21 to 3.36) for tertile III versus tertile II, 3.88 (95% CI: 2.13 to 7.06) for tertile III versus tertile I, and 1.88 (95% CI: 0.96 to 3.67) for tertile II versus tertile I.
For the 846 patients included in the validation cohort, the ACUITY-PCI score displayed good prognostic accuracy with a C-statistic of 0.70 (95% CI: 0.61 to 0.73), a chi-square statistic for calibration of 6.2 (p = 0.62), and an IoS of 0.42. As shown in Figure 2B, event rates in the validation cohort increased significantly across tertiles of ACUITY-PCI score: 5.1% in the lower tertile; 9.3% in the middle tertile; and 18.6% in the upper tertile (p < 0.001). In addition, the ACUITY-PCI score displayed good prognostic accuracy (C-statistic = 0.72, p < 0.0001) and was an independent predictor (HR: 1.09, 95% CI: 1.02 to 1.16, p = 0.008) of the 1-year risk of definite/probable stent thrombosis.
Statistical performance of 5 varying risk scores
Score performances in the validation cohort are shown in Table 4. Among the 5 scores, the ACUITY-PCI score displayed the best discrimination (p < 0.0001), the best calibration, and the best IoS. Moreover, the NRI varied from 9% to 36% and the IDI from 1.9% to 2.7%. As shown in Figure 3, the ACUITY-PCI score was the only one to display both good discrimination and good calibration. Clinical scores (TIMI and GRACE) were reasonably well calibrated, but they displayed poor discrimination, whereas the SYNTAX score and the Clinical SYNTAX score displayed fair discrimination, but less than optimal calibration.
Antithrombotic therapy and the ACUITY-PCI score
Clinical outcomes of patients treated with heparin plus a glycoprotein IIb/IIIa inhibitor or bivalirudin plus a glycoprotein IIb/IIIa inhibitor versus bivalirudin monotherapy, stratified by tertiles of the ACUITY-PCI sore are shown in Figure 4. In the entire patient cohort (n = 2,538), no significant difference in the 1-year risk of death or MI was apparent between the 2 treatment groups across tertiles of the ACUITY-PCI score.
The main findings from the present study are as follows. 1) A novel prognostic scoring system (the ACUITY-PCI risk score) consisting of 6 readily available variables has been developed that is able to accurately predict the 1-year rates of death or MI in patients with NSTEACS in whom coronary intervention is performed. 2) The 6 variables independently predictive of adverse outcomes included 2 clinical variables (insulin-treated diabetes and renal insufficiency), 1 laboratory/electrocardiographic variable (baseline cardiac biomarker elevation or ST-segment deviation), and 3 angiographic variables (bifurcation lesion, small vessel/diffuse coronary artery disease, and the extent of coronary artery disease). 3) In the validation cohort, the ACUITY-PCI score displayed good discrimination and calibration, with 1-year event rates increasing more than 3-fold across tertiles of this risk score. 4) When compared with the TIMI and GRACE scores, and the SYNTAX and Clinical SYNTAX scores, the ACUITY-PCI score displayed the best accuracy in terms of discrimination, calibration, and IoS, with the NRI varying from 9% to 36% and IDI from 1.9% to 2.7%.
The ACUITY-PCI risk score was created specifically for patients with NSTEACS undergoing PCI. In this regard, although the TIMI and the GRACE scores have been shown to be valuable prognostic tools at the time of hospital admission for selecting pharmacological strategies and identifying those patients most likely to benefit from an invasive strategy (1,2,14), they have not been optimized for patients undergoing PCI and, thus, have relatively poor prognostic power to further risk stratify acute coronary syndrome patients undergoing PCI. The ACUITY-PCI score is therefore intended to supplement the TIMI and GRACE scores when an invasive strategy has been undertaken and PCI is being considered. The present study demonstrates that angiographic variables (which only become available after initial cardiac catheterization and, thus, are not available in the TIMI and GRACE scores) add important independent information to risk stratify NSTEACS patients undergoing PCI. In support of this concept, we have previously demonstrated that the SYNTAX score, a purely angiographic measure highly different from the ACUITY-PCI score (which combines clinical, laboratory, and angiographic variables), is by itself an independent predictor of the occurrence of death, MI, and target vessel revascularization at 1 year in this same high-risk group of patients (4). In the present study, we have shown that combining select angiographic measures with clinical, electrocardiographic, and laboratory variables (in the ACUITY-PCI score) results in greater predictive accuracy and discrimination than either the purely angiographic SYNTAX score or pre-angiographic TIMI and GRACE scores.
The observation that the total burden of coronary atherosclerosis, as reflected by the extent of coronary artery disease and the presence of small vessel/diffuse coronary disease, was prognostically more powerful than descriptors of focal lesion pathology, such as the presence of a total occlusion, heavy calcification, thrombus, or aorto-ostial lesion bears comment. The prognostic relevance of the global burden of atherosclerotic disease has been recently underscored by the PROSPECT (Predictors of Response to CRT) study, a prospective evaluation of the natural history of coronary artery atherosclerosis in patients with acute coronary syndromes (15). In that study, major adverse cardiovascular events during 3-year follow-up were equally distributed between recurrence at the site of treated culprit lesions (those responsible for the initial clinical syndrome) and to untreated nonculprit lesions, most of which were angiographically mild at baseline (mean diameter stenosis: 32.3 ± 20.6%).
The clinical variables incorporated in the ACUITY-PCI score were insulin-treated diabetes and renal insufficiency. Patients with diabetes and renal insufficiency have more extensive and complex coronary artery disease, which is often deemed responsible for the adverse prognosis of these patients (16–18). It is noteworthy that in our analysis, insulin-treated diabetes and renal insufficiency were not significantly correlated with the angiographic factors that were retained in the prognostic ACUITY-PCI score. Moreover, insulin-treated diabetes and renal insufficiency remained independent predictors of adverse events after PCI, even after correcting for these angiographic variables, suggesting that other factors, such as elevated platelet reactivity, poor response to thienopyridines, and/or bleeding propensity may contribute to the poor prognosis in patients with diabetes and chronic kidney disease (19–21).
Not only was the ACUITY-PCI risk score more prognostically accurate than the other scores examined, but with only 6 variables, the ACUITY-PCI score is simple to calculate. Compared with the SYNTAX score, only 3 angiographic variables are required. It is important to underscore that the presence of any bifurcation lesion (as classified by the Medina score) is included when determining the ACUITY-PCI score, whether treated or not. However, the ACUITY-PCI score does not require any further detailing of the bifurcation lesion, as opposed to the SYNTAX score in which the exact Medina classification of the bifurcation is an integral component (22). This simplification may translate into better reproducibility of the ACUITY-PCI score than the SYNTAX score, for which substantial inter- and intraobserver variability has been reported (23–25).
There may be several important clinical applications of the ACUITY-PCI risk score. Whereas bivalirudin alone and heparin plus a glycoprotein IIb/IIIa inhibitor have equivalent results across the risk spectrum identified by the ACUITY-PCI risk score, this score does identify a group of patients with an ongoing high rate of adverse ischemic events with either of these 2 therapies. Identification of such high-risk acute coronary syndrome patients with a poor projected prognosis after PCI might result in preferential referral to CABG, or the selective use of more potent antiplatelet agents, such as prasugrel or ticagrelor, which significantly reduce adverse ischemic events (26). Future studies are required to confirm these approaches. Nonetheless, implementation of the most accurate risk stratification tools is important for quality control and for providing appropriate information to patients about their risk of future events.
The ACUITY-PCI score was developed in patients enrolled in the ACUITY trial. Thus, although the discrimination of the ACUITY-PCI score was confirmed in a distinct cohort of patients from ACUITY, its predictive accuracy in NSTEACS patients undergoing early PCI should be further validated from a different study dataset and from real-world registry studies. Of note, however, neither the TIMI nor GRACE scores, which are now universally accepted, were externally validated upon their introduction (2,8). Subsequent reports in different cohorts of patients validated the prognostic accuracy of these scores before they were implemented in clinical practice (27,28). The extent of coronary artery disease was determined by QCA, and the 30% diameter stenosis cutoff was arbitrarily selected, which roughly correlates with a visually estimated diameter stenosis of approximately 40% to 50%. Although QCA may be time-consuming and/or not immediately available, it provides a more objective determination of the extent and severity of coronary artery disease than visual assessment does (29); therefore, scores that use QCA may be more reproducible than those based only on visual estimation of angiographic variables (24). Recently, a new score specific for patients undergoing PCI, the NCDR (National Cardiovascular Data Registry) Cath-PCI score, has been described (30). However, as some variables used in that score were not systematically collected in the ACUITY trial, we could not determine its relative prognostic accuracy. The 3 cohorts of patients in whom the comparative efficacy of the randomized antithrombotic regimens was evaluated in relation to the ACUITY-PCI score may not have been large enough to detect modest differences in clinical outcomes. Patients with prior CABG were excluded from the present study. As the natural history and progression of coronary artery disease is affected by interactions between bypass graft conduits and grafted coronary arteries, we believe that they should be considered as a separate group. Different variables were associated with the primary outcome measure in the present study than in previous reports from the ACUITY trial (3,31). These differences may be explained by inclusion in this study of new variables with greater predictive power, as well as evaluation of different endpoints, patient cohorts, and time points for the primary outcome measure.
The ACUITY-PCI risk score is a novel and simple-to-calculate prognostic tool specifically developed for patients with NSTEACS undergoing PCI, which integrates clinical, angiographic, laboratory, and electrocardiographic variables for risk prediction of 1-year death or MI. Compared with other available scores, the ACUITY-PCI risk score displays the best predictive accuracy in terms of both discrimination and calibration.
For supplemental tables, please see the online version of this article.
The ACUITY trial was funded by The Medicines Company and Nycomed. Dr. Mehran is a consultant for Abbott, Regado, Ortho McNeal, Janssen, Merck, Maya Medical, and AstraZeneca; and has received a research grant from Sanofi/Bristol-Myers Squibb, Eli Lilly, and Daiichi Sankyo. Dr. Stone has served as a consultant to Abbott Vascular, Boston Scientific, Medtronic, and The Medicines Company. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- coronary artery bypass graft
- confidence interval
- hazard ratio
- integrated discrimination index
- index of separation
- myocardial infarction
- net reclassification improvement
- non–ST-segment elevation acute coronary syndromes
- percutaneous coronary intervention
- quantitative coronary angiography
- Thrombolysis In Myocardial Infarction
- Received June 1, 2012.
- Accepted July 25, 2012.
- American College of Cardiology Foundation
- Lansky A.J.,
- Goto K.,
- Cristea E.,
- et al.
- Palmerini T.,
- Genereux P.,
- Caixeta A.,
- et al.
- Garg S.,
- Sarno G.,
- Garcia-Garcia H.M.,
- et al.,
- ARTS-II Investigators
- Cook N.R.
- Aoki J.,
- Lansky A.J.,
- Mehran R.,
- et al.
- Califf R.M.,
- Phillips H.R. 3rd.,
- Hindman M.C.,
- et al.
- Cannon C.P.,
- Weintraub W.S.,
- Demopoulos L.A.,
- et al.,
- TACTICS-TIMI 18 Investigators
- Angiolillo D.J.,
- Bernardo E.,
- Capodanno D.,
- et al.
- Angiolillo D.J.,
- Bernardo E.,
- Sabaté M.,
- et al.
- Ferreiro J.L.,
- Angiolillo D.J.
- Généreux P.,
- Palmerini T.,
- Caixeta A.,
- et al.
- Beauman G.J.,
- Vogel R.A.
- Peterson E.D.,
- Dai D.,
- DeLong E.R.,
- et al.,
- NCDR Registry Participants
- Goto K.,
- Lansky A.J.,
- Nikolsky E.,
- et al.