Author + information
- Received August 31, 2014
- Revision received January 14, 2015
- Accepted January 28, 2015
- Published online May 1, 2015.
- Beau M. Hawkins, MD∗,
- Kevin F. Kennedy, MS†,
- Herbert D. Aronow, MD, MPH‡,
- Louis L. Nguyen, MD, MBA, MPH§,
- Christopher J. White, MD‖,
- Kenneth Rosenfield, MD¶,
- Sharon-Lise T. Normand, PhD#,
- John A. Spertus, MD, MPH† and
- Robert W. Yeh, MD, MSc¶∗ ()
- ∗Cardiovascular Section, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma
- †Saint Luke’s Mid-America Heart Institute/University of Missouri-Kansas City, Kansas City, Missouri
- ‡St. Joseph Mercy Hospital–Michigan Heart Division, Ann Arbor, Michigan
- §Department of Surgery, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
- ‖John Ochsner Heart and Vascular Institute, Ochsner Medical Center, New Orleans, Louisiana
- ¶Division of Cardiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
- #Department of Health Care Policy (Biostatistics), Harvard Medical School, Boston, Massachusetts
- ↵∗Reprint requests and correspondence:
Dr. Robert W. Yeh, Cardiology Division, GRB 8-843, Massachusetts General Hospital, Boston, Massachusetts 02114.
Objectives The aim of this study was to examine variation in outcomes for patients receiving carotid artery stenting (CAS) across a sample of U.S. hospitals and assess the extent to which this variation was attributable to differences in case mix and procedural volume.
Background As CAS is increasingly being used throughout the United States, assessing hospital variation in CAS outcomes is critical to understanding and improving the quality of care for patients with carotid artery disease.
Methods Hospitals participating in the National Cardiovascular Data Registry–Carotid Artery Endarterectomy and Revascularization Registry contributing more than 5 CAS procedures from 2005 through 2013 were eligible for inclusion. We estimated unadjusted and risk-standardized rates of in-hospital stroke or death for each participating hospital using a previously validated prediction model and applying hospital-level random effects.
Results There were 188 hospitals contributing 19,381 CAS procedures during the period of interest. Unadjusted and risk-standardized in-hospital stroke or death rates ranged from 0% to 18.8% and 1.2% to 4.7%, respectively. Operator and hospital volumes were not significant predictors of outcomes after adjustment for case mix (p = 0.15 and p = 0.09, respectively).
Conclusions CAS outcomes vary 4-fold among hospitals, even after adjustment for differences in case mix. Future work is needed to identify the sources of this variation and develop initiatives to improve patient outcomes.
Randomized trials have established the efficacy of carotid artery stenting (CAS) in standard- and high-surgical risk patients (1,2), and this has resulted in increased use of this novel therapy across the United States in recent years (3). Understanding the impact that these changes in practice patterns have had on patient outcomes is critical because the increased adoption of carotid endarterectomy nearly 2 decades ago resulted in significant variation in periprocedural stroke events (4).
CAS is a procedure with a well-established learning curve (5) and is performed by providers from a variety of medical specialties with patient selection practices and technical expertise that may differ (6). For these reasons, significant variation in CAS outcomes might be anticipated. Using the CARE (Carotid Artery Revascularization and Endarterectomy) Registry, we analyzed hospital-level variation in-hospital stroke or death (S/D) rates and assessed the extent to which this variation could be explained by differences in patient case mix as well as differences in procedural volume. Finding significant variation across hospitals could encourage further inquiry as to why such differences exist and lead to the widespread dissemination of best practices that could improve care and outcomes.
The CARE Registry includes patients receiving carotid revascularization with either carotid endarterectomy or CAS. The Registry uses a standardized dataset with written definitions (7). Hospitals reporting more than 5 CAS procedures from 2005 through 2013 were eligible for inclusion. CAS procedures for acute evolving stroke were excluded.
CAS risk model
A CAS risk model predictive of S/D was previously published and served as the basis of risk adjustment used in this analysis (8). This model was derived from 11,122 procedures performed between 2005 and 2011 in the CARE Registry and was internally validated by bootstrapping. Variables used in this model were age, previous stroke, symptomatic target lesion within 6 months, impending major surgery, atrial fibrillation, and no previous ipsilateral carotid endarterectomy. To account for clustering at the hospital level, model coefficients were re-estimated using a generalized linear model with hospital-level random effects, as has been described for other measures of hospital performance (9).
Patient and hospital characteristics are reported across low, average, and high tertiles on the basis of hospital-level observed S/D rates. Risk-standardized S/D rates were calculated for each hospital. These values were defined as the ratio of the number of events predicted to have occurred at a particular hospital to the expected number of events at an “average” hospital with similar case mix, multiplied by the mean unadjusted event rate for all included hospitals (9). Predicted events were estimated for each hospital using its own patient mix and hospital-specific intercept; expected events were estimated using each individual hospital’s case mix and average hospital intercept for all facilities included in this analysis. We estimated 95% confidence intervals using hospital-based bootstrap resampling (10).
To examine the impact that risk adjustment had on hospital performance, we quantified the percent of hospitals that were reclassified into different performance tertiles after risk adjustment. We also calculated the median odds ratio as a measure of between-hospital variation in outcomes (11). Finally, annual operator and hospital volume were each separately introduced into the hierarchical model before and after adjusting for case mix to determine whether these variables affected risk-standardized S/D rates. All statistical analyses were performed using SAS version 9.2 (SAS Institute, Cary, North Carolina).
There were 19,381 procedures from 188 hospitals included in this analysis. The dataset was very complete; <1% of each risk model variable had missing values. The mean unadjusted S/D rate across the entire cohort was 2.4%. Wide variation in procedural outcomes was present, with S/D rates ranging from 0% to 18.8% (Figure 1A). Corresponding 25th, 50th, and 75th percentile rates were 0%, 1.8%, and 3.7%, respectively. Individual components of the composite endpoint are shown in Table 1.
Characteristics by tertile
Hospitals were divided into low, average, and high tertiles on the basis of corresponding observed S/D rates (<0.3%, 0.3 to 3.1%, or >3.1%, respectively). Procedural characteristics of these tertiles are displayed in Tables 2 and 3⇓⇓.
Risk-standardized S/D rates
After adjusting for case mix, S/D rates ranged from 1.2% to 4.7% (Figure 1B). Low tertile hospitals had adjusted S/D rates of 1.2% to 2.2% compared with 2.5% to 4.7% in high tertile facilities. Corresponding 25th, 50th, and 75th percentile adjusted rates were 2.1%, 2.3%, and 2.7%, respectively. A plot of individual hospital rates was also constructed (Figure 2).
Overall, 63 hospitals (34%) shifted to different tertiles of performance after risk adjustment (Figure 3). Of the 62 hospitals in the unadjusted low tertile group, 37 (60%) remained in the low tertile group, but 25 (40%) shifted to the average group. In the unadjusted average tertile group, 6 (10%) shifted to the high tertile group, and 25 moved to the low tertile group (25%). In the unadjusted high tertile group, 7 (11%) shifted to the average tertile group, whereas the rest (n = 57, 89%) remained in the highest tertile group.
The median odds ratio was calculated to be 1.51 (95% confidence interval: 1.28 to 1.71), meaning that there was, on average, a 50% difference in the odds of experiencing S/D between 2 randomly selected hospitals treating the identical patient. Operator volume and hospital volume were not found to be significant predictors of outcomes after adjustment in case mix (p = 0.15 and p = 0.09, respectively).
In an analysis of 188 hospitals performing CAS in the United States as part of a large national registry, we found in-hospital outcomes vary 4-fold, and the odds of experiencing stroke or death differs by ∼50% (mean odds ratio: 1.51) for 2 randomly selected facilities treating an identical patient. These data suggest that substantial quality differences may exist among U.S. hospitals that offer CAS.
CAS quality assessment
The risk-standardized S/D metric developed here is valid, reliable, clinically meaningful, and feasible to implement in clinical practice. This measure should allow institutions and clinicians to benchmark their performance against others and, through the identification of best practices, has the potential to engender policies aimed at quality improvement. Outcomes assessment is but 1 component of quantifying health care quality, however, and additional work examining structure and process elements for carotid revascularization is needed to develop additional quality measures (12). Potential measures could include the development of uniform credentialing requirements for CAS operators or additional procedural metrics such as embolic protection device use, procedural times, and medication use (e.g., thienopyridine pre-loading, statins).
Epstein et al. (13) recently reported risk-adjusted 30-day mortality rates for 22,708 CAS procedures performed in 927 U.S. hospitals from 2009 through 2011. They found that mortality rates vary nearly 5-fold (1.1% to 5.1%) in Medicare beneficiaries receiving CAS, although only 13 hospitals (1.4% of the cohort) were classified as poor outliers on the basis of having 95% confidence intervals that excluded the national mean. The findings of our analysis, which are augmented with important clinical data and risk adjustment, are similar, demonstrating a 4-fold variation in S/D rates and <2% of facilities being identified as outliers using similar methods (data not shown). The potential advantages of our analysis are that it includes stroke in the composite endpoint, accounts for crucial patient characteristics that affect CAS outcomes (e.g., symptomatic status), and may be less vulnerable to coding and reporting errors.
Procedural volume considerations
Low CAS volumes have an important implication when quantifying hospital performance. The generation of risk-adjusted event rates for low-volume procedures inherently results in wide 95% confidence intervals, the boundaries of which have been traditionally used to identify outliers (10). In this scenario, the result is that the majority of facilities are found to perform “as expected”—the discriminatory capacity of a risk-adjusted outcome measure is diminished. In our analysis, only a small minority of such hospitals were identified as outliers using this methodology (<2%, data not shown).
Similar difficulties have been reported elsewhere. Dimick et al. (14) examined 7 surgical procedures for which risk-adjusted mortality was advocated as a quality measure. For coronary artery bypass grafting, the mean mortality rate was found to be 3.5%, and a minimal caseload of 219 procedures was needed to detect facilities with a mortality rate twice that of the national average. Only 61% of hospitals providing coronary artery bypass grafting met this volume by 1 year. For hip replacement, with a mortality risk of <1%, nearly 2,700 procedures would be required to detect a doubling of the mortality rate, a volume achieved in <1% of hospitals even when pooling procedures over 5 years.
Sources of CAS variation
In our analysis, procedural volume was not found to be an important predictor of outcomes after adjustment in case mix. This stands in contrast to previous reports that demonstrated an association between volume and CAS outcomes (5). There are several potential explanations for our findings. Therapeutic advances (e.g., embolic protection) and improved patient selection have resulted in steady improvement in CAS outcomes in recent years (15). These changes may have mitigated the association between volume and outcomes, particularly because our analysis included procedures from a more modern time period compared with previous reports. Our analysis was also limited to centers participating in a voluntary registry aimed at quality assessment and improvement. As such, the hospital participants may have more comprehensive measures in place to promote patient safety, or they may use more rigorous credentialing standards for providers performing CAS in their institutions. Finally, our sample size may not have been large enough to identify a statistically significant association between volume and outcomes.
The generalizability of our findings to non-CARE facilities may be limited as this is a voluntary registry in which participants may be more prone to quality assessment and improvement. Our analysis examined in-hospital events only; it is uncertain whether examining 30-day outcomes would have resulted in different findings. Our timeframe was broad and included a large number of diverse facilities. It is possible that patient selection practices for CAS may have varied among these institutions, and it is unclear whether examining a narrower, more recent period would have resulted in less observed variation. Finally, despite rigorous risk adjustment, we cannot exclude the possibility that unmeasured confounders may have influenced our results.
Using risk-standardized S/D rates, we identified significant variation in outcomes for hospitals participating in a national CAS registry. Future work is needed to identify additional sources of this variation and to develop initiatives to improve the quality of care for patients receiving CAS.
WHAT IS KNOWN? CAS is a relatively new procedure that has become widely used across the United States in recent years. It is not well established whether significant variation in outcomes exists among facilities performing CAS.
WHAT IS NEW? Rates of in-hospital stroke or death vary nearly 4-fold among hospitals participating in a large national CAS registry, even after adjusting for differences in case mix.
WHAT IS NEXT? Future work is needed to identify the sources of this variation and develop initiatives to improve patient outcomes.
This research was supported by the American College of Cardiology Foundation’s National Cardiovascular Data Registry (NCDR). The views expressed in this manuscript represent those of the author(s) and do not necessarily represent the official views of the NCDR or its associated professional societies identified at www.ncdr.com. The CARE Registry is an initiative of the American College of Cardiology Foundation, The Society for Cardiovascular Angiography and Interventions, the Society of Interventional Radiology, the American Academy of Neurology, the American Association of Neurological Surgeons/Congress of Neurological Surgeons, the Society for Vascular Medicine, and the Society of Vascular and Interventional Neurology. Dr. Aronow is a nonpaid consultant for Silk Road Medical, Inc. Dr. Rosenfield has received research grants from Abbott Vascular, Bard Peripheral Vascular, Medtronic/Invatec, and Atrium; has received consulting/advisory board fees from Abbott Vascular, Boston Scientific, Complete Conference Management, Harvard Clinical Research Institute, Contego, Micell, and Becker Ventures; has equity in Lumen Biomedical, Medical Stimulation Corp, Angioguard (Cordis), and Micell; and has served on the Board of Directors of VIVA Physicians (501C3). Dr. Normand is on the Board of Directors of Frontier Science Technology and Research Foundation. Dr. Spertus has received a research contract from the American College of Cardiology Foundation to analyze the NCDR Registry. Dr. Yeh is an investigator at the Harvard Clinical Research Institute. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- carotid artery stenting
- in-hospital stroke or death
- Received August 31, 2014.
- Revision received January 14, 2015.
- Accepted January 28, 2015.
- American College of Cardiology Foundation
- Goodney P.P.,
- Travis L.L.,
- Malenka D.,
- et al.
- ↵NCDR™ PVI Registry™ (Peripheral Vascular Intervention Registry) v1.0.1 Carotid Artery Stenting and Carotid Endarterectomy. Available at: http://cvquality.acc.org/w/media/QII/NCDR/Data%20Collection%20Forms/PVI%20RegistryDataCollectionFormCarotid101.ashx. Accessed April 2015.
- Hawkins B.M.,
- Kennedy K.F.,
- Giri J.,
- et al.
- Krumholz H.M.,
- Lin Z.,
- Drye E.E.,
- et al.
- Yeh R.W.,
- Rosenfield K.,
- Zelevinsky K.,
- et al.
- Larsen K.,
- Merlo J.
- ↵(2000) Measuring and improving quality of care: a report from the American Heart Association/American College of Cardiology First Scientific Forum on Assessment of Healthcare Quality in Cardiovascular Disease and Stroke. Circulation 101:1483–1493.
- Epstein A.J.,
- Yang L.,
- Yang F.,
- Groeneveld P.W.
- Safian R.D.