Accountable care organizations (ACOs) are new entities designed to achieve cost-savings by incentivizing higher quality, integrated care across the care continuum (Reference Fisher, McClellan and Bertko6). The Centers for Medicare and Medicaid Services (CMS) regulate adherence to rigorous quality measures by sharing the resultant cost savings between the regulatory agency and healthcare organization (Reference Fisher, McClellan and Bertko6). In 2016, there were 480 ACOs with 9.0 million beneficiaries in the United States (5). On average, ACOs have approximately 18,000 enrollees each, with a minimum of 2,000 enrollees and a maximum of 140,000 (Table 1) (5).
Table 1. General Characteristics of 2016 ACOs
Note. The 480 ACOs covering 9.0 million beneficiaries, adapted from Centers for Medicare and Medicaid Fast Facts 2016. One-sided risk refers to ACOs not being penalized by CMS if they experience losses, but as a result having a smaller share of savings, while two-sided risk refers to ACOs sharing in losses or gains with CMS.
ACO, accountable care organizations; SD, standard deviation.
The idea that coordinated care could achieve substantial cost savings was developed by Wennberg, Fisher, and colleagues who studied regional variation and found that lower-cost geographic regions were able to achieve comparable health outcomes at lower costs through coordinated care (Reference Wennberg, Fisher and Skinner15). Coordinated care involves multiple providers, such as primary care doctors, specialists, and hospitals, sharing accountability for coordinating patient care and clinical processes (1). The purpose is to ensure that patients get the right care at the right time by preventing duplicate tests and services, reducing hospitalizations, and limiting growth in healthcare costs (1). This concept became the foundation for ACOs, which were first implemented as a pilot program, Physician Group Practice Demonstration, which saw a moderate level of savings across all beneficiaries and even greater savings for vulnerable populations such as Medicare-Medicaid dual-eligibles (Reference Colla, Wennberg and Meara4).
ACOs were thus implemented nationally under the Patient Protection and Affordable Care Act as a new approach in healthcare delivery for fee-for-service beneficiaries, to reduce unnecessary costs by facilitating coordination among providers (2). Implementation of ACOs was done in two stages. First, a set of thirty-two Pioneer ACOs, healthcare organizations and providers with prior experience in coordinating care, piloted the program beginning in 2012 (Reference Nyweide, Lee and Cuerdon8). Then, the broader community of healthcare organizations could opt into the ACO model as a permanent program under the auspices of the Medicare Shared Savings Program (MSSP) (2).
ACOs follow a novel reimbursement structure under which, an annual benchmark expenditure rate is set by using actuarial projections and a multifactorial risk adjustment system (3). At the end of the year, an organization either shares in the savings produced by lower than expected expenditures or pays a penalty to CMS for expenditures above the benchmark (3). For organizations that qualify for savings, additional conditions of quality performance are set to realize the savings, such that high-quality scores can increase savings, but low-quality performance negates the savings (Reference Nyweide, Lee and Cuerdon8).
Due to the newness of the policy, empirical evidence on ACO adherence to prescribed quality metrics and the resultant cost savings is lacking. In this study, we seek to understand the implementation of quality metrics and compare cost savings with regard to economy and efficiency for Pioneer and MSSP ACOs compared with non-ACOs. Our second aim is to examine whether cost-savings (over non-ACOs) vary by category of ACO defined by a quality score. We expect to find greater cost savings in ACOs with higher quality care and among MSSP ACOs compared with Pioneer ACOs, as they have a lower level of coordination at inception and thus greater potential for improvement.
METHODS
Study Design
We used a cost minimization approach to assess the change in cost of care after implementation of either a Pioneer or MSSP ACO model. A cost minimization analysis is an economic evaluation method used to compare programs that have similar outcomes, making lowest cost the deciding factor in determining preferences (Reference Robinson11). It is a frequently used method to compare programs and pharmaceutical drugs at a health system level.
Data were extracted from publicly available CMS files for ACO expenditures, the number of assigned beneficiaries, and quality scores for performance years 1 and 2 (2012–13) (12). All costs were simulated for a 10-year period and expressed in 2015 USD.
Study Population
Each ACO received thirty-three quality scores across four quality domains: (a) Patient/Caregiver Experience; (b) Care Coordination/Patient Safety; (c) Preventive Health; and (d) At-Risk Population (10). Replicating CMS's methodology for calculating an aggregate quality score, values for each quality metric were summed by domain and divided by the number of measures in that domain (10). Then, the four domain scores were averaged into a composite quality score. Based on the distribution of the quality scores, ACOs were categorized as low quality if the composite quality score was less than 60 percent, medium if the composite score was between 60 percent and 75 percent, and high quality if their composite quality score was 75 percent or greater (Figure 1). Beginning in year 2, “dropout” was an additional category for institutions that discontinued their ACO programs. We used Markov modeling to calculate the probability of transitioning to a different quality level between years 1 and 2 (Figure 1).
Figure 1. Markov Model depicting transition states.
Analysis
To calculate cost savings by organization, the difference between benchmark expenditures and actual expenditures was generated, whereby a positive value indicated cost savings and a negative value represented losses. The benchmarks are ACO specific, risk adjusted predictions of Medicare Fee-for-Service Parts A and B expenditures for the assigned beneficiaries (7). The CMS Office of the Actuary calculates the expected costs for each ACO using expenditures of these individuals from the past three months (7). These data are supplemented with other sources to include individually identifiable payments made from the Medicare Trust Funds for beneficiaries under special programs, such as a demonstration, pilot, or time limited program (7).
The benchmark value was divided by the total number of beneficiaries assigned to the ACO, creating an aggregate amount saved per beneficiary. The organization-specific cost savings values were stratified by quality level (high, medium, low). Within each level, cost savings per beneficiary were averaged to formulate savings per quality level.
Using TreeAge Software (13), we calculated transition probabilities for quality levels from year 1 to year 2 and the cost savings associated with each level. Assuming a linear relationship over time, we ran a deterministic Markov simulation using 3 percent discounting and a 10-year time horizon (Figure 1). We used a health systems perspective, because ACOs are a conglomerate of various healthcare organizations and providers run by a single administrator that together form a health system. To test the uncertainty in the results, we conducted a probabilistic sensitivity analysis using 10,000 Monte Carlo simulations to determine the projected cost savings while incorporating variability within the data. For the probabilistic parameters, a gamma distribution was used to model costs, and a beta distribution was used to model transition probabilities between quality levels.
RESULTS
At baseline, there were thirty-two Pioneer ACOs: six low quality, twenty-two medium quality, and four high quality. In year 1, the average cost savings per beneficiary was USD 117.47 (SD: USD 258.80) for high quality ACOs, USD 176.42 (SD: USD 407.61) for medium quality ACOs, and USD 61.23 (SD: USD 451.71) in losses for low quality ACOs. In year 2, there were twenty-three Pioneer ACOs: 0 low quality, eleven medium quality, twelve high quality, and nine dropouts. Of the nine dropouts, in year 1, four were low quality and five were medium quality (Table 2).
Table 2. Inputs and Outputs of Markov Model to Analyze Cost Minimization Alternative ACO frameworks
ACO, accountable care organization.
Among the MSSP ACOs, at baseline, there were 220 ACOs: 18 low quality, 165 medium quality, and 37 high quality. In year 1, the average cost savings was USD 265.05 (SD: USD 698.08) for high quality ACOs, USD 28.92 (SD: USD 829.52) for medium quality ACOs, and USD 103.98 (SD: USD 773.85) for low quality ACOs. In year 2, there were 210 MSSP ACOs: seven low quality, 122 medium quality, eighty-one high quality, and ten dropouts. Of the ten dropouts, in year 1, one was low quality, eight were medium quality, and one was high quality (Table 2).
The number of ACOs in year 2 suggest systematic dropout. After year 1, there are few low quality ACOs remaining, 0 percent of remaining Pioneer ACOs and 3 percent of MSSP ACOs (Table 2). Among Pioneer ACOs, low quality performers were most likely to drop out of the program, while among MSSP ACOs the initial poor performers were more likely to transition to medium or high quality.
Running a Markov simulation after 10 years, high quality organizations saved a significant amount of money per beneficiary, USD 456 for Pioneer ACOs and USD 857 for MSSP ACOs. Medium quality organizations saved a moderate amount of money, USD 223 for Pioneer and USD 128 for MSSP ACOs, but low-quality organizations suffered financial losses, USD 41 for Pioneer and USD 107 for MSSP ACOs (Table 3). The long-term losses for low-quality organizations, despite some short-term gains, is likely because the standard deviation of cost savings per beneficiary is so large and the year 1 savings relatively small, such that in the long-term, there was no guarantee of savings, and greater likelihood of long-term costs.
Table 3. Net Cost Savings Per Beneficiary by Year and Quality Level
aNine pioneer ACOs dropped out from year 1 to year 2, 28% dropout.
bTen Medicare Shared Savings Program ACOs dropped out from year 1 to year 2, 4.5% dropout.
cThis number is negative because of the standard deviation is so large and the savings relatively small so that, in the long term, there was no guarantee of savings in fact making them unlikely.
These results were largely confirmed by probabilistic sensitivity analyses using 10,000 Monte Carlo simulations. Similar to the deterministic results, there is a significant difference in cost savings both by quality level and by ACO type; high quality MSSP ACOs save approximately double that of high achieving Pioneer ACOs (USD 816 for MSSP ACOs and USD 459 for Pioneer ACOs), while low quality MSSP ACOs lose nine times more money than Pioneer ACOs (USD 386 for MSSP ACOs and USD 40 for Pioneer ACOs) (Table 3). Given the greater uniformity among Pioneer ACOs compared with MSSP ACOs and their greater initial coordination of care, this result is in line with expectations. The only difference between the deterministic and probabilistic models occur among low quality MSSP ACOs; the deterministic model expects that low quality MSSP ACOs will lose USD 107 per beneficiary, but the probabilistic framework predicts losses of USD 386 savings per beneficiary. This variability then translates to differences in the overall effect of the program; under a deterministic model MSSP ACOs are expected to save USD 878 compared with USD 536 under a probabilistic model.
DISCUSSION
From this analysis, we find that in ACOs, quality and cost per beneficiary are inversely related; when quality goes up, costs go down. Higher quality organizations, Pioneer ACOs and high quality MSSPs, which are characterized by high levels of coordination of care, save the most money. These results suggest that reducing costs and improving quality are simultaneously achievable as a result of integrated care which leads to greater systematic efficiency. This result has also been demonstrated by researchers at CMS (Reference Pham, Cohen and Conway9).
The differences between Pioneer and MSSP ACOs is largely attributable to differential entry into the programs. The Pioneer ACO model was for a small sample of select organizations with an existing infrastructure for coordination of care. They were chosen for their innovative practices; hence, at baseline, they were of uniformly higher quality. In contrast, the MSSP ACO program was open to all healthcare organizations and, therefore, has a much broader range of quality and cost data. The baseline higher quality of Pioneer ACOs means that low quality Pioneer ACOs do not lose as much money as MSSP ACOs, because there exists some infrastructure for coordinated care, reducing the proportion of truly low-quality Pioneer ACOs.
However, as higher quality organizations, Pioneer ACOs did not have as much room to improve. So, as quality of ACOs increased, MSSP ACOs saved more money than Pioneer ACOs. This was confirmed by sensitivity analyses that showed the maximum amount of money saved by an MSSP ACO was double that of the highest performing Pioneer ACO. This suggests that the value of investment of adopting a high quality ACO model can result in a new, more efficient, healthcare production curve for MSSP ACOs (Reference Weinstein and Skinner14).
A confounding factor is the systematic dropout by low quality organizations. Because, poor performers are subject to fiscal penalties, there seems to be systematic dropout for organizations that cannot rapidly improve in quality, making it difficult to determine whether low performing centers would have the ability to improve their quality, if forced to do so.
This study is limited by several factors. First, the results of this study are based on only two years of data, which may not provide a complete longitudinal trajectory of the improved quality and reduced costs from operating an ACO. However, the results from the sensitivity analyses suggest that several years into implementation, there continues to be substantial room for quality improvement and cost savings. Second, we are uncertain whether these results are generalizable to ACOs that organize in the future.
The field would benefit from future research on the effect of ACOs on subpopulations, such as Medicaid beneficiaries or disease-specific ACO models, to show if there are improved outcomes by cohort (e.g., diabetes, stroke, AMI, etc.). These models should include health outcomes data to determine the mechanism for cost savings; is coordinated care leading to better health outcomes that result in cost savings, or do organizations achieve cost savings by providing less care, which may have long-term consequences on the health of the populace? Preliminary data suggest that ACOs are achieving cost savings through better disease management, such as reducing preventable hospital admissions and readmissions from complications, in which case ACOs may be a successful strategy for incentivizing coordination of care while reducing overall costs (Reference Colla, Wennberg and Meara4).
This study is the first to date that uses Markov modeling to predict long-term savings and losses for the ACO reimbursement model. The results present initial evidence that better quality is associated with achieving greater cost savings. From a policy perspective, this suggests that stringent quality requirements, so that ACOs transition to being high quality, is an important component of realizing cost savings, because high quality ACOs consistently perform to a higher net monetary benefit. This in turn helps to achieve the ACA's broader policy goal of reducing healthcare costs.
In conclusion, ACOs are successful at their dual purpose of improving quality and reducing healthcare costs. Through systematic efficiencies and coordination of care, ACOs are able to deliver higher quality care to produce both short- and long-term savings with comparable results between Pioneer and MSSP organizations.
CONFLICTS OF INTEREST
The authors have nothing to disclose.
