Our Argument: The Threat of Legislative Override

Our argument for the threat of legislative override is based on the separation of powers literature (e.g., Ferejohn and Weingast Reference Ferejohn, John and Weingast1992). An override occurs when a court's ruling is modified in subsequent legislation or treaty revisions. According to Wasserfallen (Reference Wasserfallen2010, 1131), the potential for such a legislative reaction can cause the ECJ to balance legal considerations against the concerns of key legislative actors and particularly those of the MSGs. For example, Wasserfallen (Reference Wasserfallen2010, 1139) describes how the ECJ ruling in Grzelczyk (Case C-184/99) anticipated the formation of subsequent legislative coalitions by moderating its jurisprudence on student mobility.

In our 2008 article we argue that the court's responsiveness to such threats depends on the likelihood that a legislative coalition will form to counter a ruling. In the European Union, a legislative override requires that a decisive set in the council—all member states under unanimity, a supermajority under qualified majority—vote for a new policy. Revising a treaty requires unanimity. We assume that MSGs can construct such a coalition through logrolls. If a subset of governments considers an issue sufficiently important, it can trade concessions in other areas in exchange for legislative support from disinterested or otherwise opposed governments. In fact, a sizable portion of EU legislation involves logrolls, especially under unanimity (Aksoy n.d.).Footnote ³ Further, individual states, even small ones, have succeeded in building logrolls for treaty reform.Footnote ⁴ Thus, we argue, override is possible when at least one MSG wants to revise a court ruling.

Of course, the likelihood of success of a potential logroll depends on the initial distribution of MSG positions on the ruling. Consequently, we argue that the credibility of a threat of override depends on these initial conditions: The more MSGs expressing initial opposition to the ruling and the fewer expressing support, the more credible the threat. This conception of credibility is continuous.

This argument contrasts sharply with that of Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012). They dismiss the potential for logrolls and instead posit that override can only be achieved if enough MSGs explicitly support an override. They note (correctly) that, in the data, the distribution of MSG observations does not meet that threshold, and thus they dismiss override as a constraint on the ECJ.

Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 205) also claim that there is no evidence of an override of a significant case in the history of the ECJ. We disagree and provide an example later. It is important to note, however, that because our argument is about courts curtailing their behavior in anticipation of a possible override, providing more such examples or counting the number of instances of successful override is not informative. This is a standard issue with studying deterrence. If the court is responsive to, and accurate in its assessment of, threats of override, we should not see many instances of override.

Our example comes from the area of social security benefits.Footnote ⁵ In 1971, the European Community (EC) passed a regulation on the applicability of social security schemes to workers moving within the EC; it potentially extended member states’ social security provisions to non-national residents. In a series of rulings in the late 1980s and early 1990s, the ECJ ruled that national welfare policies enjoy “exportability,” and therefore a variety of national residency requirements for eligibility were invalid.Footnote ⁶ In the last of these rulings (Stanton Newton) in 1991, the ECJ issued a preliminary ruling against a national restriction. The restriction in question was supported by the two MSG observations (from the United Kingdom and Belgium). In 1992, the council amended regulation 1408/71 explicitly to insert language countering the effect of recent rulings by the ECJ. The court then capitulated. Conant (Reference Conant2002, 193) summarizes these events as follows: “The Member States overruled the ECJ with secondary legislation, which the ECJ could have overturned on the basis of Article 42 (51 EEC) of the treaty. The EJC declined this opportunity, however, when it chose to uphold the compatibility of the amended regulation with the treaty, leaving Member States with the last word in this particular dispute.”

Thus, it is possible for MSGs to override an ECJ ruling through legislation. In addition, in this example the override was successful even though MSG observations against the ruling did not surpass the legislative threshold.

Our Argument: The Threat of Noncompliance

We define MSG “compliance” with EU law as the promulgation of community norms within their national territory. This process typically requires implementation, application, and enforcement by national institutions and may require the state to refrain from action (Krislov, Ehlermann, and Weiler Reference Krislov, Ehlermann, Weiler, Cappelletti, Seccombe and Weiler1986). Our use of the term “compliance” is consistent with several studies of compliance with ECJ decisions (e.g., Beach Reference Beach2005; Krislov, Ehlermann, and Weiler Reference Krislov, Ehlermann, Weiler, Cappelletti, Seccombe and Weiler1986; Obermaier Reference Obermaier2009; Steiner Reference Steiner1993). To clarify the relationship between our and Stone Sweet and Brunell's argument, we summarize our theory.

Our argument about compliance comes from Carrubba (Reference Carrubba2005), which argues that many international regulatory regimes are created to realize mutually beneficial cooperation by overcoming collective action problems. A common example is a treaty to liberalize trade. Such an agreement almost inevitably presents governments with a form of a prisoner's dilemma. All of the participant states perceive a benefit from mutual adherence with the proposed agreement, but they have no incentive to follow the rules unilaterally. A regulatory regime can help overcome this challenge by allowing states to tie the threat of retaliation or punishment to violations of the regime's rules (see also Simmons Reference Simmons1998).

The success of such a regime hinges on whether governments can effectively monitor and punish each other. Carrubba (Reference Carrubba2005) argues that courts can facilitate monitoring and punishment by acting as a fire alarm (a centralized venue in which to challenge possible acts of noncompliance) and an information clearinghouse (a venue in which participants make arguments in an attempt to establish whether a contested action is punishable).

A court knows its decisions do not automatically gain compliance. Rather a litigating government's decision to comply depends on whether it anticipates retaliatory punishment. Thus, the threat of noncompliance increases as the likelihood of retaliatory punishment decreases. Assuming that a court wants to avoid noncompliance, it is more likely to rule for the government as the threat of noncompliance by that government increases. Importantly, Carrubba (Reference Carrubba2005) argues that the decision to punish depends on the domestic context of the litigant government. The governments created the regulatory regime for a reason; they want to realize mutually beneficial gains from overcoming a collective action problem. But sometimes, for political, social, and economic reasons, the costs of compliance will be unusually high, and compliance no longer will be mutually beneficial. Thus, when the cost of complying with a ruling exceeds the benefits, the litigant government may be let “off the hook.”

The empirical prediction (Hypothesis 2) in CGH (2008) follows. If a government can credibly threaten noncompliance (i.e., it is a physical option), the more third-party government support and the less opposition a litigating government receives, the less retaliation for noncompliance should be anticipated, and the more likely the court is to defer to the litigating government.Footnote ⁷

This model of a regulatory regime has several important empirical implications for our tests and for several of Stone Sweet and Brunell's (Reference Stone Sweet and Brunell2012) observations. First, the theory predicts that governments will be taken to court, be ruled against, and comply with court decisions on a regular basis. Second, we do not expect to see third-party governments routinely signaling their position on a case. The frequency of filings will depend on how often governments consider the cost of compliance to be especially high. Thus, the observed frequency of observations highlighted by Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012) is not dispositive. Third, the court in the model is regularly expected to adjudicate cases consistent with the substantive provisions of treaty or legal statutes defining the agreement. The governments created the regime because they wanted the rules enforced when doing so is mutually beneficial, and governments will have adopted rules so that the legally sanctioned application is generally mutually beneficial. Thus, we should observe many decisions made on the “legal merits.” Fourth, the capacity of the court to promote its own agenda is limited. Governmental willingness to punish noncompliance determines the boundaries of the application of the agreement, and when those boundaries are reached, the court will tend to be deferential. Finally, this model does not mean that the court cannot innovate. If the court identifies new ways of applying the law that go beyond the written mandate and that governments see as mutually beneficial, governments may support the ECJ ruling or even enshrine it in future treaties.

Compare this argument to Stone Sweet and Brunell's claims. They (Reference Stone Sweet and Brunnell1998) argue that pro-liberalizing ECJ rulings increase opportunities for cross-border trade. By taking advantage of these new economic opportunities, pro-liberalizing interests grow and bring more cases challenging trade barriers to the ECJ, which in turn lead to more liberalization, and so on. Although MSGs do not act as a gatekeeper in their argument, Stone Sweet and Brunell's argument could be reconciled with our theory quite easily. Suppose, as pro-liberalizing domestic interests grow, that they induce reelection-seeking MSGs to support more liberalization via the standard electoral constraint. By shifting these induced preferences, new opportunities for mutual benefits may arise. If properly identified by the ECJ, these opportunities may allow it to promote an innovative liberalizing agenda, although the ECJ still cannot move beyond the newly identified mutually beneficial opportunities.

Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 205) also argue that noncompliance begets more litigation, which serves as a vehicle for expanding the ECJ's ability to construct EU law. They see this mechanism as contrary to our theoretical argument. But our theory allows for noncompliance followed by innovative jurisprudence. In particular, consider rulings that expand EU law and suffer noncompliance, but that are supported by third-party governments. In that setting, the punishment (or the threat thereof) should eventually end the noncompliance, ensuring implementation of the court's novel interpretation.

Where we differ from Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012) is in not expecting the court to develop EU law that enjoys compliance in areas where noncompliance with its rulings is supported by third-party governments. In particular, we predict that the ECJ is less likely to issue a ruling that requires national government compliance as the number of MSGs supporting the noncompliance increases. Our theory further implies that, when such rulings occur in the face of credible threats of noncompliance, the court will typically not be successful in developing EU law beyond the wishes of the MSGs. In sum, although we see tensions between our arguments and those of Stone Sweet and Brunell, we do not see the need for hard lines to be drawn. These arguments frequently can be complementary.

Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 205) assert that the development of the court's jurisprudence has never been stunted by threats of noncompliance and emphasize that the absence of such an example is critical evidence against our argument. Once again, we are happy to provide an example to illustrate our point.Footnote ⁸ In two 1998 rulings, Kohll and Dekker, the ECJ determined that national laws requiring patients to obtain prior authorization for medical services delivered outside their home country violated EU law.Footnote ⁹ These rulings and the broader principles on which they were based had the potential to disrupt dramatically national health care systems. Numerous MSGs submitted observations in opposition to the court's position. We interpret these actions to mean that threats of noncompliance were likely credible. Indeed these rulings were met with widespread noncompliance (Obermaier Reference Obermaier2009, 84) and the continued threat of noncompliance for similar subsequent rulings. In response, the ECJ significantly tempered its position in later rulings to satisfy the concerns of member state governments and thereby increase compliance (Obermaier Reference Obermaier2009, 170). Obermaier (Reference Obermaier2009, 173) concludes, “Since the ECJ narrowed down the doctrinal content of the original rulings and limited the scope de facto to out-patient care, the actual effect diminished accordingly. Ultimately, the tremendous impact which was postulated by legal and political science scholarship, that is the massive change of the institutional configuration of domestic health care systems, did not come true.”

Thus, the potential for noncompliance with its rulings seems to have caused the ECJ to alter its original decisions to placate the MSGs’ interests. We also see that the noncompliance generated subsequent rulings, which is consistent with Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 205). Yet, the additional litigation did not serve to expand EU jurisprudence or to develop law beyond the desires of the MSGs. The result was quite the opposite. This illustrative example is useful for clarifying our argument and demonstrating its plausibility. However, as with the reasoning on threats of override, we do not consider the frequency of such events material to our argument. If our argument holds, the court should usually react to credible threats of noncompliance by tempering its rulings to reflect MSG interests and thereby ensure compliance. Therefore, we will not typically observe noncompliance when the court is influenced by such threats.

Scope

We test our hypotheses with data from all rulings (1987–97), which consist primarily of three types: infringements, annulments, and preliminary rulings. Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012) claim that noncompliance is never an issue in annulment rulings, and they question its saliency to preliminary rulings. We suspect this disagreement reflects, at least in part, different uses of the term “compliance.” We use “compliance” to mean the promulgation of a community norm within a national territory. Government compliance is therefore relevant when the promulgation of the norm in the ruling requires reform of existing national law or depends on some form of executive behavior. This kind of compliance is front and center in an infringement ruling, because the MSG is the defendant and the failure to comply with its legal obligations is the cause of action. On this we agree. Both annulments and preliminary rulings can also contest existing national laws or, more generally, require implementation by national institutions. In preliminary rulings, the ECJ ruling often articulates a general right, principle, or rule with application to the specific litigants in the national case (Steiner Reference Steiner1993, 4). Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 205) comment that national courts typically “comply” in the sense that the national court decision comports with the ECJ ruling; however, the national court can implement the ruling only in the instant case. The broader effect of the ruling may require changes to national law or administrative practices—that is, government compliance (Steiner Reference Steiner1993, 4). National courts cannot ensure the compliance of their governments. Indeed, according to Conant (Reference Burley, Anne-Marie and Walter Mattli2002, 69), when government compliance is in question in preliminary references, national officials often evade or ignore the ruling. It is this potential noncompliance that is at issue in our article. We do not, however, dismiss the potential effect that national courts could have on the credibility of threats of noncompliance; our Hypothesis 3 specifies and tests that expectation.

Similarly, noncompliance can be at issue in annulments where the action in question (e.g., a commission decision) may depend on a national government for implementation, especially when that government brings an annulment against the commission or the council. The bulk of these actions have addressed agriculture or competition policy (Bauer and Hartlapp Reference Bauer and Hartlapp2010), both of which areas often rely on national government implementation. For example, a commission determination that state aid is invalid (i.e., in breach of competition policy) requires the national government to terminate the current subsidy and desist from creating a substitute policy. In particular, member states are responsible for the recovery of state aid that the court deems unlawful (Jestaedt, Derenne, and Ottervanger Reference Jestaedt, Derenne and (co-ordinators)2006). The inherent conflict of interest for member state governments that both grant and are charged with recovering state aids raises an obvious concern with the implementation of commission decisions requiring recovery. According to a comprehensive survey of the enforcement of these decisions, member state compliance is particularly problematic after an ECJ ruling confirming the commission determination—that is, when the MSG loses an annulment action (Jestaedt, Derenne, and Ottervanger Reference Jestaedt, Derenne and (co-ordinators)2006, 538).

Member states also are intricately involved in the implementation of the common agricultural policy. For example, in the annulment action France v. Commission (C-366/88), the French government challenged a measure adopted by the commission that obligated national officials to cooperate with commission officials in specific aspects of supervising and monitoring the European Agricultural Guidance and Guarantee Fund. Clearly, the effectiveness of an ECJ ruling for the commission would have depended on the compliance of the French government.Footnote ¹⁰

Our use of “compliance” in annulment cases is not unique. Consider, for example, the conventional account of the annulment action brought by the United Kingdom in 1994 against the council on the Working Time Directive (Case C-84/94). The directive defined minimum rest periods and maximum weekly working hours. The United Kingdom then enjoyed an exception from EU social policies. Nevertheless, the council had adopted the measure under a treaty article that applied to the United Kingdom and that allowed the council to decide without the support of the United Kingdom. The UK government brought an action to annul this directive on the grounds that it had been adopted with the wrong legal basis.

The Advocate General's (AG) opinion in the case advised the ECJ to find the directive valid. In response, the UK government announced that it would not implement the directive if the court followed this recommendation: It threatened noncompliance. Pollack (Reference Pollack2003, 333) described the context this way: “[T]he defiant mood of John Major's government, and the significant financial and political costs likely to be imposed by an adverse ECJ ruling, posed a non-trivial risk of overt noncompliance by the UK. This possibility of noncompliance was highlighted, moreover, by the British government's reaction to the preliminary opinion” [emphasis added].Footnote ¹¹

Of course, not all rulings involve threats of MSG noncompliance. Annulment actions can involve one EU institution challenging an act of another institution or a private litigant challenging the act of an EU institution. In these settings, government compliance is less often relevant to implementation (e.g., see Partie Ecologiste ‘Les Verts’ v. European Parliament; Case 294/83). Similarly, preliminary rulings involving two private parties (a frequent event) also are less likely to raise compliance challenges. In short, compliance figures more prominently in cases with national government litigants than otherwise.

Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012) gloss over this distinction, although it is central to distinguishing between our first and second hypotheses. Although override is always a possibility, the threat of MSG noncompliance is not. Because we posit that the court should be more sensitive to threats of noncompliance than threats of override—they are easier to execute—and because threats of noncompliance are primarily present when governments are litigants, we predict that the court's decisions will be more sensitive to MSG observations when a government is a litigant than when one is not.

Measurement and Statistical Tests

We operationalized our measure of third-party government preferences by using a weighted function of government observations on cases (CGH 2008, 440). For the purposes of this discussion, the tests of our argument are the following. First, we tested for threats of override when governments are not litigants and for threats of noncompliance when they are. We found statistically significant relationships between government observations and ECJ decisions when governments are not litigants, and a much stronger relationship when they are (Table 1, first column). Second, we included the Advocate General as a control for the legal merits of the case and found the exact same relationships (CHG 2008, 446). In both analyses we controlled for the position of the commission.

TABLE 1. Probit Models of ECJ Ruling for the Plaintiff

Notes: Robust standard errors reported in parentheses. *p < .05, one-tailed test.

We make two points. First, in CGH (2008), our operationalization of government observations followed the logic of our theories. We measured the impact of both MSG support and opposition to litigant governments in the “net weighted observations for the plaintiff” and in the unweighted net measure used in the supplemental analyses presented in the Appendix of CGH (2008).Footnote ¹² Thus, we captured net support for a particular litigant's position. The more net support for a position, the more credible the threat to a ruling against that position, and therefore the more likely the court is to defer to that position.

Second, we treated government briefs as a proxy for the underlying concept we wanted to measure: government preferences on the outcome of the case. If a direct measure of an MSG's preferences were available, we would certainly use it. Nonetheless, these briefs can be credible signals of MSG preferences. Might briefs correlate with court decisions for reasons other than political pressure? Yes. In particular, briefs may reflect, support, and/or substantiate the side with the stronger legal argument. Accounting for these alternatives is critical, and we addressed this issue in two ways. First, if the briefs are simply a function of the legal merits, the influence of briefs on court decisions should not depend on whether a government is a litigant. However, we did observe such a conditional effect. We also included the Advocate General's position as a control for the merits of the case and found support for political constraints.Footnote ¹³

Second, we included the commission's position as a control, because we have theoretical reasons to believe its position might be correlated with both the court's decision and with the positions that governments take. Its position may be an indicator of the legal merits of the case; the commission's observation may serve as a “political bellwether” indicating to the court what would be acceptable to the MSGs (Burley and Mattli Reference Martinsen1993, 71), or the court might agree with or be responsive to a commission pro-integrationist agenda. Thus, the commission's position may be correlated with the ECJ's decision, and for several of these reasons (i.e., the merits argument or responsiveness to MSG interests), the commission's position could also be correlated with the position of the government briefs.

Thus, Stone Sweet and Brunell's (Reference Stone Sweet and Brunell2012) claims about our interpretation of MSG observations are incorrect. First, we agree that these briefs do not “directly evaluate the proposed explanation of this influence” and that one should not “presume that the outcome can only be explained by ‘political’ threats” (Stone Sweet and Brunell Reference Stone Sweet and Brunell2012, 206). We minimize the odds that our analysis suffers from these problems by accounting for plausible alternative explanations for a correlation between government briefs and court rulings. Second, we do not, as Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 206) claim, count all instances of congruence between the position of MSG observations and the ECJ ruling as evidence of a political constraint. Only congruence independent of these other various factors counts as evidence supporting our hypotheses.

Does Neofunctionalism Account for Our Evidence Regarding MSG Observations?

Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 211) assert that neofunctionalism predicts that MSGs have more influence on the ECJ's rulings when MSGs encourage the ECJ to punish a member state and when they are joined by the commission. Consequently, they contend that, in preliminary rulings, MSG observations for the plaintiff should be more influential than ones for the defendant, and the impact of MSG support for the plaintiff should be strongest when the commission also supports the plaintiff. They present what amount to cross-tabs (without test statistics) showing that, in preliminary rulings, the ECJ is more likely to side with the position supported by the MSGs if they support the plaintiff. The plaintiff's success rate is even greater if both MSGs and the commission support the plaintiff.

This does not constitute an appropriate test. First, their measure of the MSGs’ position does not provide a test of our hypotheses. Our expectation is that the ECJ is more responsive to MSG observations for the plaintiff or defendant as the number of such observations grows. We therefore constructed a continuous measure of net MSG support. Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012, 211) use a dichotomous measure, indicating only whether MSGs were net on the side of the plaintiff or defendant. This measure ignores important variation in the net number of MSGs on the litigants respective sides and fails to distinguish the economic or political clout of the observers. This variation is also relevant to their argument; MSG observations indicate potential punishment of a defendant state (Stone Sweet and Brunell Reference Stone Sweet and Brunell2012, 211).

Second, they do not control for the legal merits of the ruling. The commission and the MSG position may convey information about a ruling's legal merits. If so, the greater frequency of success of plaintiffs supported by these actors (Stone Sweet and Brunell Reference Stone Sweet and Brunell2012, 211) could simply indicate that the legal merits were more often on the plaintiff's side. We therefore control for the legal merits.

In Models 1 and 2 in Table 2 we report the results for our tests of Stone Sweet and Brunell's hypotheses with corrections for the two problems.Footnote ¹⁵ The statistical model includes separate variables for each litigant, which allows us to assess whether the effect of MSG observations is stronger when they support the plaintiff. These two variables are derived from the original variable, net weighted observations for the plaintiff, defined in CGH (2008, 440). Each variable is coded as zero in the absence of net observations supporting the specific litigant and as the absolute value of net weighted observations for that litigant otherwise. Thus, both measures are continuous. In Model 2, we add an interaction term to capture the conditional effect of MSG observations for the plaintiff when the commission supports the plaintiff. If Stone Sweet and Brunell's hypotheses are correct, we should find that the coefficient on MSG observations for the plaintiff is larger than the absolute value of the coefficient for MSG observations for the defendant and that the interaction term is positive.

TABLE 2. Probit Models of ECJ Ruling for the Plaintiff

Notes: Robust standard errors in parentheses. *p < .05, one-tailed test. The hypotheses all specify a directional expectation, which calls for a one-tailed test (Kelstedt and Whitten Reference Kellstedt and Whitten2009, 176).

The results do not support Stone Sweet and Brunell's expectations. First, as shown in Model 1, the size of the positive impact of an increase in pro-plaintiff MSG observations is not statistically different from the size of the negative impact of an increase in pro-defendant MSG observations (p > .26). Second, as shown in Model 2, we find that the impact of MSG observations for the plaintiff is not greater when the commission also supports the plaintiff. To see this, one must consider the coefficients from both the main effect for pro-plaintiff MSG observations and the interaction term (Brambor, Clark, and Golder Reference Brambor, Clark and Golder2006). The sum of these two coefficients (3.34 + −2.23 = 1.11) is smaller than the coefficient for MSG observations for the plaintiff when the commission does not favor the plaintiff (3.34). These findings do not support Stone Sweet and Brunell's (Reference Stone Sweet and Brunell2012, 211) claims in favor of neofunctionalism.

Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012) also assessed whether the ECJ is more likely to rule for the commission position or the MSG position when they are in conflict. They claimed (211) that the critical question is how the ECJ responds when the commission's brief is in opposition to the MSG observations and that neofunctionalism predicts that the commission position trumps the MSG observations. They do not provide appropriate tests for this claim: for reasons cited earlier.Footnote ¹⁶

Model 3 of Table 2 presents an appropriate test. It adds two interaction terms to the variables in Model 1,Footnote ¹⁷ each of which captures the condition when the commission and MSG observations are in opposition. The results are counter to those of Stone Sweet and Brunell (Reference Stone Sweet and Brunell2012). First, with the commission supporting the defendant, an increase in MSG observations for the plaintiff is associated with a greater increase in the likelihood of the court ruling for the plaintiff than when the commission does not support the defendant. The sum of the coefficients indicating the impact of MSG observations for the plaintiff when the commission supports the defendant (2.21 + 1.25 = 3.46; p < .01) is greater than the coefficient for government observations for the plaintiff (1.25) when the commission does not support the defendant. Second, when the commission supports the plaintiff, an increase in MSG observations for the defendant is associated with a decrease in the likelihood of a ruling for the plaintiff, and the magnitude of this effect is at least as large as when the commission does not support the plaintiff. The sum of the coefficients capturing the impact of MSG observations for the defendant when the commission supports the plaintiff (−0.36 + −1.22 = −1.58; p < .05) indicates a larger negative impact than the coefficient for government observations for the defendant (−0.36) when the commission does not support the plaintiff. That is, the presence of an opposing commission observation fails to temper the impact of MSG observations on the court. More generally, the three models presented in Table 2 do not support Stone Sweet and Brunell's claims that our data favor their neofunctionalist hypotheses.Footnote ¹⁸

Substantive Significance of the Commission, MSG, and AG positions

Because CGH (2008) focused primarily on estimating the effect of MSG observations, we did not assess or compare the impact of other factors on ECJ rulings. Yet clearly, our understanding of ECJ decision making would be enhanced by such an assessment. Stone Sweet and Brunell pursued one such assessment: a comparison of the substantive significance of commission and MSG observations. Our data can shed light on the broader question of what accounts for ECJ rulings.

Unfortunately, Stone Sweet and Brunell's statistical evidence does not make appropriate comparisons and omits consideration of a key factor in ECJ decision making. First, their Figure 1 (210) is a visual comparison of the change in the predicted probability of a ruling for the plaintiff associated with changes in net MSG observations and with the presence of an observation from the commission. This figure is based on the first model presented in Table 2 in CGH (2008). That model lacks both the interaction term with the government litigant used to capture threats of noncompliance and a control for the legal merits. This is not the appropriate statistical model.

Second, this sort of visual inspection is potentially misleading (Gelman and Hill Reference Gelman and Hill2007). The basic problem is that the predicted probabilities calculated at different values on the variable of interest (e.g., commission observations for plaintiff) must assume particular values for the other variables in the statistical model. The selection of these values can dramatically alter the predicted probabilities. For this reason, Gelman and Hill (2007, 100–3) recommend assessing the substantive significance at each observed value for the other variables and then averaging across all of these values to generate a summary of the substantive effect. This is called the “average predictive comparison.” In the context of our data, this measure requires estimating the change in the predicted probability of a pro-plaintiff ECJ ruling due to a shift (e.g., from its lowest to its highest observed value) in the independent variable of interest, averaged across all observed values of the other variables.

We can also evaluate the impact of the position of the AG, which serves as a proxy for the legal merits. Because students of the court routinely assert that the ECJ rules in response to the merits, our accounting of the factors shaping ECJ decision making should include the impact of the AG's position.

We use the probit model presented in the last column of Table 1 to generate average predicted comparisons for these three factors. This model is based on Model 2 in Table 2 in CGH (2008), which is reproduced in the first column of the table, but also includes a control for the position of the AG.Footnote ¹⁹ Consequently, this model allows us to assess the threat of noncompliance and the impact of the legal merits. We calculated average predictive comparisons on the variables of interest. The average increase in the likelihood of a pro-plaintiff ruling associated with a change from a commission observation for the defendant to one for the plaintiff is 0.28. The average change associated with a low to high shift for MSG observations is 0.31 without a government litigant and 0.52 with a government litigant.Footnote ²⁰ The average change associated with a change in AG position between supporting the defendant and the plaintiff is 0.53. These results are not specific to the time period studied in CGH (2008). In a larger data set (1960–1999) the average predictive comparisons support the same conclusions.Footnote ²¹

In sum, we find no evidence of a landslide in favor of any one factor: All three have substantively important independent effects on the direction of ECJ rulings. However, we are intrigued by the impact of commission observations. Because the statistical model controlled for the position of the AG, we are fairly confident that the observed impact of the commission does not simply reflect the legal merits on the specific legal question at issue. This rules out one plausible interpretation of the relationship between the ECJ rulings and the commission's position. However, as we discussed earlier, several plausible interpretations remain. Given the sizable impact of commission observations on rulings, identifying the appropriate interpretation is important to our understanding of ECJ decision making.