The Millennium Development Goals

The MDGs consist of eight goals (see Table 1) which were promulgated by the UN, agreed to by such international financial institutions as the World Bank, and adopted by a variety of bilateral aid organizations. Each goal is mapped into one or more “targets,” which, in turn, is mapped into several numerical indicators that serve as a basis for assessment of progress. For instance, the goal we examine here—(2) universal primary education—is mapped into the task of “ensur[ing] that, by 2015, children everywhere … will be able to complete a full course of primary schooling.” Progress toward this target is assessed by primary education enrollment and completion rates, as well as the literacy rate of fifteen-to-twenty-four year-olds, broken down by gender.Footnote ¹⁹

Table 1. Millennium development goals

Many of these goals were drawn directly from the UN's Millennium Declaration, which the General Assembly issued and approved as the culmination of the Millennium Summit, held in September of 2000.Footnote ²⁰ The more precise indicators, and several of the more general goals, were only agreed upon during a World Bank conference, “From Consensus to Action: a Seminar on the International Development Goals,” held in 2001.Footnote ²¹ The agenda for both the World Bank conference and the UN Summit, however, was determined by a pre-existing set of development targets—the International Development Goals (IDGs).

The IDGs were originally promulgated by the Organisation for Economic Co-operation and Development (OECD) in 1996 as part of an effort to shape the aid policies of member states. In content, they were very nearly identical to the MDGs. The targets were set by rich-world donors, with fairly minimal input from aid recipients, and were constructed to appeal to domestic audiences in the developed world.Footnote ²² In particular, the goals were intended to focus on concrete measures that appealed to a broad audience (e.g., poverty reduction through economic growth, gender equality). Moreover, these goals were thought to advance a degree of accountability on the part of recipient states.Footnote ²³

The aim of such precise numeric targets was explicitly to increase accountability, particularly for states that received development aid. (Only indicators for goal 8, the “global partnership for development,” included measures of donor generosity.) In laying out these goals, the Secretary General's report noted: “In order to help focus national and international priority-setting, goals and targets should be limited in number, be stable over time and communicate clearly to a broad audience. Clear and stable numerical targets can help to trigger action and promote new alliances for development.”Footnote ²⁴

We examine the effects of the MDGs through the lens of existing theories of political accountability. We focus on the goal pertaining to primary education, in part because the mapping between this goal and a numeric indicator was particularly precise, and in part because measures of closely related public goods (secondary and tertiary enrollment rates) were not targeted by the MDGs.

Global Assessment Power and Public Goods Production

The MDGs represented a public international commitment by governments to achieve gains in the provision of specific public goods. They increased governments’ incentives to provide these goods. Governments, therefore, would be likely to devote greater personnel, material resources, and effort toward providing these specific public goods once the MDGs were in effect. However, these incentives also have a secondary effect—governments are likely to divert resources from the production of other public goods to achieve MDG goals.

First, consider the effect of the MDGs on government incentives. Kelley and Simmons argue that the dissemination of international standards may exert social pressure on governments to achieve certain policy outcomes, and that such standards are particularly likely to do so if “they are based on systematic monitoring, are comparative (and especially quantitative), are wielded by a respected actor or group/organization of actors, and are widely disseminated.”Footnote ²⁵ Perhaps no set of international standards meets these criteria as well as the MDGs. These standards were (1) explicitly linked to precise statistical targets;Footnote ²⁶ (2) subject to annual monitoring by the UN and other donor organizations; (3) initially linked to the Millennium Declaration, and thus approved by the UN General Assembly and its constituent member states, and subsequently promulgated jointly by the World Bank, IMF, and OECD;Footnote ²⁷ and (4) widely disseminated through public statements by these agencies as well as bilateral lenders.

As Figure 1 represents, international standards like the MDGs may affect government behavior through a variety of channels, including international actors, domestic politics, and the personal or reputational concerns of individual leaders. Global standards may alter behaviors in the international or intergovernmental realm. The creation of a common set of international standards may facilitate decentralized enforcement through bilateral sanctioning or a withdrawal of bilateral benefits.Footnote ²⁸ This was a particular risk with regard to the MDGs, given that the goals were explicitly endorsed by most bilateral development agencies soon after their promulgation—with the US as a notable holdout.Footnote ²⁹ Alternatively, standards can be enforced through material sanctions (or withdrawal of benefits) by international bodies. Lebovic and Voeten, for instance, demonstrate that the World Bank enforced international standards of human rights, and was less willing to lend to countries that had been “shamed” by the UNCHR.Footnote ³⁰ This danger, again, would seem particularly acute for governments that failed to make progress on the MDGs, given the central role both the World Bank and IMF played in the construction of the goals.

Global indicators may also serve to inform residents of monitored states, in a readily digestible manner, about the quality of their government's performance with respect to certain policy goals. Individuals within these states might use such information to discipline government behavior either through the ballot boxFootnote ³¹ or through protest or collective action.Footnote ³² A well-developed literature finds that such information enables citizens to hold their leaders accountable for public goods provision,Footnote ³³ which, in turn, induces leaders to more actively provide public goods.Footnote ³⁴ Hence, government leaders should face greater domestic pressure to achieve specific public goods targets once the MDGs are promulgated.

Finally, international standards can alter government incentives by generating global norms of acceptable behavior.Footnote ³⁵ In defining basic standards of “development,” the MDGs may have altered leaders’ notions of what constitutes success and failure. Such definitions may shape leaders’ primitive preferences over public goods allocations—or shift their strategies for regime legitimation.Footnote ³⁶ Focusing on the MDG targeting maternal health and its impact in Zambia, Alice Evans documents extensive evidence of norm adjustment among elites as the primary mechanism by which this particular GPI influenced policy.Footnote ³⁷

It is our contention that the MDGs had the effect of increasing the incentive to provide the specific public goods delineated in the goals. Any or all of these mechanisms may be effective, and other, unlisted, mechanisms may also be at work. Although our main focus is on the equilibrium provision of targeted and nontargeted public goods, we are able to test some channels of influence, including foreign aid and domestic politics. Our findings relate to similar work by Helena Skagerlind who conducts a detailed process tracing of the effects of the MDG with regard to the empowerment of women.Footnote ³⁸ She finds some evidence that all three of these causal channels were in operation with the strongest results for aid conditionality and the socialization of government elites. Our findings for education indicate alternative channels of influence, highlighting the heterogeneity of these mechanisms across targets.

While we contend that the MDGs had the intended effect of increasing government incentives to provide specific public goods, we further contend that they had the unintended effect of causing governments to substitute away from efforts not delineated by the UN. As Holmstsrom and Milgrom argue, when a given agent is faced with a multiplicity of tasks, some of which are monitored and others of which are not, the agent will rationally devote a disproportionate level of effort to those goals that are subject to monitoring.Footnote ³⁹ We therefore focus on government's tendency to provide a public good listed in the MDGs—for example, primary education—relative to its tendency to provide a public good that is not listed—for example, secondary education. MDGs alter the relative provision of primary and secondary education, boosting primary relative to secondary. This tendency may be somewhat offset to the extent that various public goods are complements in production. For instance, increasing primary enrollment and completion rates perforce increases the pool of students eligible for secondary education. However, the substitution effect should dominate provided these complementarities in production are not too large.

Ironically, the very characteristics that make the MDGs valuable in terms of increasing accountability also render them particularly prone to this substitution effect. Because they focus on precisely defined quantitative metrics of performance, with regard to very specific public goods, they are particularly vulnerable to “gaming.” More nebulous or qualitative goals, which are less susceptible to monitoring and less useful for the types of comparisons that give global indicators their power,Footnote ⁴⁰ are also likely to be less prone to substitution effects. However, such weak assessments are also less likely to alter government behavior at all.

The Moderating Role of Information and Institutions

We expect that the intuition we describe may vary by context. Government institutions and the domestic informational environment moderate the effects of global assessment—both the direct effects on the provision of specific services and the indirect substitution effects on the provision of unassessed public goods. To be more precise, both effects are weakened in instances where governments are highly accountable to domestic audiences. Highly transparent states and democracies are less prone to alter the menu of public goods delivered by the government in response to global indicators than opaque autocracies.

The rationale for this contention is twofold. First, and most directly, domestic and international accountability are substitutes. Recall that one of the mechanisms through which global indicators might induce policy change works through such indicators’ effect on domestic politics. Global monitoring might serve to inform domestic interests of their government's performance with regard to a particular policy area, inducing domestic demands for improved performance.Footnote ⁴¹ Naturally, however, if the domestic information environment is relatively rich, such information is superfluous. Citizens are already aware of their government's performance and any pressure to change policy as a result is likely to have already made itself manifest. Moreover, if the issue area in question is one of high public salience, or has important implications for domestic welfare, governments are likely to be induced to disclose information in response to electoral pressures in democratic political systems.Footnote ⁴² Precisely because accountable governments face a strong domestic pressure to provide public goods, increased international demands are likely to have minor effects.

The second reason we expect global indicators to have a relatively minor effect on governments with a high level of domestic accountability is more subtle. Any shift in the composition of public goods in such regimes is likely to be politically costly. Governments in such states are already providing levels of public goods provision, in keeping with the demands of large swaths of the population, to stay in power. Any change in the composition of these public goods is likely to be politically costly, more so than for comparable autocracies who rely less on public goods provision to retain power. Moreover, because accountable states provide higher levels of public goods than autocratic states,Footnote ⁴³ they will need to alter service provision less to satisfy international demands even in the instances in which they choose to so comply with the MDGs.

Model

To illustrate our claims formally, we adapt a variant of the selectorate model of Bueno de Mesquita and colleagues.Footnote ⁴⁴ We compare two game forms: one with and one without the MDGs. The model demonstrates that (1) the MDGs (weakly) increase the provision of primary education and (2) that wherever this increase is strictly positive there is a substitution away from secondary education provision. The formalization further demonstrates that the effects are more likely to be present in unaccountable (autocratic, opaque) than accountable (democratic, transparent) states.

Model without MDGs

Consider an infinite horizon discrete time interaction between a leader L and her selectorate. Normalize the size of the selectorate to one. In each period of play, L faces a challenger C.Footnote ⁴⁵ To maintain her hold on power, L must maintain the support of a share $\omega \in \left(0,\,\displaystyle{1 \over 2}\right]$ of her selectors—a “winning coalition.” We conceive of ω as capturing the degree of accountability of L to the selectorate—the probability that L is accountable to (requires the support of) an individual selector randomly selected from the set of selectors.Footnote ⁴⁶ We map the term ω into empirical measures of democracy, government transparency, and their interaction. All actors share a common discount factor, δ.

In each period, the leader and challenger make simultaneous offers of public goods—here consisting of primary (p) and higher/secondary (h) education; they may also offer private transfers (t) to each member of the (potential or actual) winning coalition. A representative member of the selectorate has preferences across p, h, t, according to the additively separable utility function U(p, h, t) = U _p(p) + U _h(h) + U _t(t), with the standard assumptions: $ U_k^{\prime} \gt 0,\; U_k^{{\prime}{\prime}} \lt 0,\;U_k\left( 0 \right) = 0, \, U_k^{\prime} \left( 0 \right) = \infty,\; U_k^{\prime} \left( \infty \right) = 0, \; k \in \left\{ {p,h,t} \right\} $.

L and C make a decision about how to allocate a budget of size B > 0 between p, h, and t. If in power, her utility is defined as the residual sum left over after this provision is made. If not in power, her utility is normalized to 0.

$$U_j\lpar {\,p,h,t} \rpar = \left\{ {\matrix{ {B-p-h-\omega t \;{\rm if}\;{\rm in}\;{\rm power}} \hfill \cr {0 \;{\rm otherwise}.} \hfill \cr}} \right.\;{\rm for}\;j\in \lcub {L,C} \rcub $$

At the start of each period of play, L and C make their offers of p, h, and t. Then there is an election—a decision is made by members of the winning coalition whether to retain the leader L. To remain in power, L must retain the support of a share ω of the selectors.

Definition 1

Define an equilibrium to this game as the pair of triples as the pair of triples, $\lpar {p_C^{\ast}, h_C^{\ast}, t_C^{\ast}} \rpar $ for the challenger and (p*, h*, t*) for the leader. Define $U_C^{\ast} = U_p\lpar {p_C^{\ast}} \rpar + U_h\lpar {h_C^{\ast}} \rpar + U_t\lpar {t_C^{\ast}} \rpar $.

In equilibrium, C will tailor his offer to buy off members of the winning coalition—his entire budget will be devoted to this task. L will address this threat by offering a menu of public goods provision and transfers that guarantees members of the winning coalition a continuation value at least as high as that from backing C. This is feasible for the incumbent because she can credibly commit to provide private transfers t to members of the winning coalition into the future. The offers of the challenger lack future credibility, and are discounted accordingly. The incumbent thus always retains her office.

In the online appendix, we offer a model extension that relaxes the assumption that L’s promises are perfectly credible. We assume that members of her winning coalition may be replaced by members of the wider selectorate with a fixed probability in each period of play. However the probability of being removed by the incumbent is less than the probability of losing one's place in the winning coalition if the incumbent is removed and the regime toppled. Our comparative statics are qualitatively unchanged by this model extension.

Lemma 1

In any (Markov Perfect) equilibrium to this (no MDG) game, the fraction $\displaystyle{{p^{\ast}} \over {t^{\ast}}}$ is rising in ω.Footnote ⁴⁷

Lemma 2

Comparative statics: p* r ises and t* f alls with ω.

As is standard in these games,Footnote ⁴⁸ greater domestic accountability leads to greater public goods provision. This is true both relative to the amount of private transfers provided and in absolute amounts. Having established these preliminaries and laid out the basic game form, we can now turn to the central question: How does the existence of the MDGs affect this domestic equilibrium?

Model with MDGs

To capture the role of the MDGs, we consider a model isomorphic to the one above. The only difference lies in L’s utility function. We now assume that the international community offers the leader a benefit for exceeding a certain threshold of primary education, as the MDGs specify. (We remain agnostic about which of the theorized transmission mechanisms diagrammed in Figure 1 are at play. In our empirical results, we test the role of foreign aid via mediation analysis.) Let β > 0 denote this benefit and ${\bar p}$ denote the exogenously imposed threshold from the international community. We can then redefine the utility of the leader:

$$ \eqalign{ U_L\left( {\,p,h,t} \right) & = \left\{ {\matrix{ {B-p-h-\omega t + \beta {\rm {\cal I}}\left( p \right){\rm \; if}\;{\rm in}\;{\rm power}} \hfill \cr {0{\rm \; otherwise}.} \hfill \cr } } \right. \cr & {\rm where} \cr {\rm {\cal I}}\left( p \right) &= \left\{ {\matrix{ {1{\rm \; if\; }p \ge \bar p} \hfill \cr {0{\rm \; otherwise}.} \hfill \cr } } \right.} $$

As before, the incumbent leader will always successfully buy off her winning coalition and remain in power. Our focus is on her equilibrium offer of primary and secondary education levels, which we will denote p** and h**, respectively. Denote the equilibrium transfer to a given member of the winning coalition as t**.

Before we can proceed further, we require some definitions. Recall that $U_C^{\ast}, p^{\ast},h^{\ast}$ and t* are defined in Definition 1 and ${\bar p} $ is the exogenous MDG threshold. The choice of the triple (p, h, t) is endogenous.

The retention constraint is, as its name suggests, a restriction on the values of the menu of goods offered to the selectorate which must be satisfied by the incumbent if she is to retain office. In particular the benefits to the selectorate must be larger than those on offer from the challenger. The residual constraint requires the leader to offer a menu that spends no more than the available budget. The participation constraint recognizes that the leader's decision to adopt the MDGs is voluntary, in the sense that the leader pockets a (weakly) larger residual after adopting the MDG ${\bar p} $ and receiving the bonus β than was the case without the MDG (and choosing p*, h*, t*). The set of all possible triples (p, h, t) that satisfy all three of these constraints is denoted Θ(·), and is parameterized by the exogenous variables of the model.

If the benefits the international community offers for attaining the MDG goals are not too large, there is a maximal target the international community can set and expect compliance. If this target for primary education provision ${\bar p} $ is set too high, L will recognize that she cannot possibly satisfy international demands, her residual and participation constraints, and the need to stay in power. We define this value of the MDG goal as ${\hat p} $.

This of course holds for “modest” values of the international reward β. If the international reward is large enough, then the leader can buy off the winning coalition, and satisfy the international community and have plenty left over. To ensure that the equilibrium to MDG game is not degenerate—in the sense that the parameter space where the MDGs can potentially affect behavior and is not empty, we adopt the following restriction:

Given that the feasible set Θ(·) is not empty (and convex), we can find the element in this set (the optimal choice of t and h) that maximizes the leader's utility in the case that the leader chooses to adopt the MDG, ${\bar p} $:

We are now ready to characterize the equilibrium to the MDG game. Let the equilibrium to the MDG game be denoted $\lcub {\left( {p_C^{{\ast}{\ast}}, h_C^{{\ast}{\ast}}, t_C^{{\ast}{\ast}}} \right) ;\left( {p^{{\ast}{\ast}},h^{{\ast}{\ast}},t^{{\ast}{\ast}}} \right) } \rcub $. Recall the equilibrium to the No MDG game is denoted $\lcub {\left( {p_C^{\ast}, h_C^{\ast}, t_C^{\ast}} \rpar ;\lpar {p^{\ast},h^{\ast},t^{\ast}} \right) } \rcub $.

Proposition 1

Given an exogenous MDG goal of ${\bar p} $, the equilibrium to the MDG game is: For C, $\lpar {p_C^{{\ast}{\ast}}, h_C^{{\ast}{\ast}}, t_C^{{\ast}{\ast}}} \rpar = \lpar {p_C^{\ast}, h_C^{\ast}, t_C^{\ast}} \rpar $. For L,

$$\left( {\,p^{\ast\ast},h^{\ast\ast},t^{\ast\ast}} \right) = \left\{ {\matrix{ {\left( {\,p^\ast,h^\ast,t^\ast} \right)} & {{if}} & {\bar{\,p} \le p^\ast} \cr {\left( { {\bar{\,p}} ,{\bar{h}} ,{\bar{t}} } \right)} & {{if}} & {\bar{\,p}\in \left( {\,p^\ast,{\hat p}} \right)} \cr {\left( {\,p^\ast,h^\ast,t^\ast} \right)} & {{if}} & {\bar{\,p} \ge {\hat p}} \cr } } \right.$$

The promulgation of the MDGs thus can affect primary education provision, but will do so for only certain configurations of parameter values. Straightforwardly, if the government were already meeting the international community's demand that primary education exceed a certain threshold, the MDGs will have no effect on state behavior. Compliance was given ex ante. By contrast, if international standards are too high, the government will find it impossible to comply while simultaneously meeting her domestic constraints. Since failing to satisfy these domestic considerations will entail a loss of office, the government will disregard the international target.

But for intermediate values of ${\bar p} $—those that exceed the prior provision of primary education but are not so onerous that they inhibit the government's ability to satisfy domestic audiences—the MDGs induce a change in behavior. The government will meet the international community's demand and set p** = ${\bar p} $.

Comparison Across the Two Cases

We are now in a position to compare the equilibrium provision of primary and secondary education under the two different equilibria, which we interpret as the effect of the MDGs. Specifically, we will examine how this effect is moderated by the level of domestic accountability—that is, how this effect varies with ω.

While the MDG level ${\bar p} $ is exogenous with respect to the game form we described, we should bear in mind that the level is in fact set by the international community. It would make little sense for the community to set an MDG level that is in fact out of reach of the very countries and leaders that it is designed to incentivize. That is, for the low ω countries, it makes little practical sense to consider any ${\bar p} $ > ${\hat p} $. In what follows we make the following assumption:

Intuitively, this means that the MDGs are attainable for all countries (given the size of the reward, β) without causing failure in office.Footnote ⁴⁹ For any country, if β is large enough, Assumption 1 is satisfied.

Let us begin by looking at the direct effect of the MDGs—the effect on primary education provision. Formally, this is defined as the difference in the equilibrium primary education provision across the two models p** and p*.

The provision of primary education is weakly higher when the MDGs are present relative to when they are absent. Some polities increase their provision up to the standard set by the international community (${\bar p} $). Others leave their level of primary education unchanged.

Systematically, the effect of the introduction of the MDGs is smaller in more accountable (higher ω) states. Primary education provision is higher in states with greater domestic accountability ex ante. Some such states will meet international standards with or without the MDGs, and so the introduction of global performance indicators will have no effect. Other accountable governments may fall a little short of international standards, but they will do so by less than their less accountable contemporaries. Meeting the goals set by the international community requires less of such states.

We can now turn our attention to the substitution effect. We begin by documenting that, whenever the effect of the MDGs on primary education is strictly positive (i.e., where the effect is non-zero), their effect on the provision of secondary education is negative. When governments respond to the MDGs they do so by increasing primary, and reducing secondary, education provision. Everywhere else, the MDGs have no effect on public goods provision.

We can go further and characterize the circumstances under which the substitution effect is likely to emerge. Systematically, less accountable states are more likely to experience substitution than more accountable states.Footnote ⁵⁰ Recall from Proposition 2 that polities with higher levels of domestic accountability are less responsive to the MDGs with regard to primary education provision. When there is no direct effect of the MDGs on primary education, there can be no substitution effect on secondary education. By contrast, less accountable states have greater leeway to respond to international incentives. Because their provision of primary education is more responsive to the MDGs, they are more inclined to engage in substitution.

In what follows, we empirically test propositions 2 and 3 and lemma 4. We anticipate that MDG adoption increased the provision of primary education, and this effect was most evident in the least domestically accountable states (autocracies with low levels of transparency). This effect should be smallest in more accountable polities (democracies with high government transparency). We further test the existence of a substitution effect—a decline in the relative provision of secondary education following the promulgation of the MDGs. This substitution effect should likewise be most evident in the least accountable states.

Empirics

We test our formal intuition on a cross-national panel of country-level primary and secondary enrollment rates measured between 1980 and 2010. Our empirical tests of the relationship between MDG adoption and enrollment use several methods to reduce bias and spurious associations, including country and year random effects, county-specific time trends, and first-differenced and lagged measures of our dependent variables. Nevertheless, our empirical approach relies on documenting evidence consistent with our theoretical predictions and not making causal claims about MDG adoption.

Data

We draw our definitions of primary and secondary enrollment rates from the UNESCO Institute of Statistics.Footnote ⁵¹ Primary enrollment rates are defined as the total number of children of official primary-school age who are enrolled in primary education divided by the total population of official primary-school age. Secondary and tertiary enrollment rates are defined analogously and are combined in our data (and our analysis). The UNESCO data contain a nontrivial number of missing observations. We drop country panels with fewer than ten years of data and impute missing values for the remaining country panels using the Amelia II package for R. We implement multiple imputation techniques to account for uncertainty across imputed data sets.Footnote ⁵²

After dropping countries with fewer than ten years of data, we are left with 114 countries with data from 1980 to 2010. We merge this data set with the HRV index of transparency from Hollyer, Rosendorff, and VreelandFootnote ⁵³ which includes additional information—importantly the annual change in real GDP—drawn originally from the Penn World Table version 6.3Footnote ⁵⁴—and a measure of democracy—drawn originally from Cheibub, Gandhi, and Vreeland.Footnote ⁵⁵ The democracy measure extends only through 2008 so we impute the values for 2009 and 2010 from this value for each country. We also include a measure of per capita GDP (current USD) from the World BankFootnote ⁵⁶ as well as three different measures of aid: total aid, aid targeting primary education, and aid targeting secondary education. All measures of aid are transformed to reflect logged per capita estimates. Table 2 presents summary statistics of our key variables, separated by the full sample, countries with GDP per capita in 2000 under $1,000, and those with GDP per capita in 2000 under $13,000.

Table 2. Summary statistics for pre- and post-millennium declaration by sample

One issue with both the primary and secondary enrollment rate data is that they are subject to ceiling effects. While neither enrollment figure is strictly bounded above by 100 percent—if, for instance, children enroll in primary education at ages below six years, the primary enrollment figure may exceed 100 percent—in practice, neither figure can substantially exceed 100 percent. To diminish the risk that ceiling effects may produce nonlinearities in our relationships of interest, we first scale all enrollment figures relative to their maximum values (such that all fall on the [0, 1] interval) and then apply a logistic transformation to these data.Footnote ⁵⁷ We apply this transformation to all enrollment figures in our data set.

To measure government accountability, we rely on two measures. The first of these captures political institutions, namely democracy. We draw our measure of democracy from the Democracy and Development Revisited data set,Footnote ⁵⁸ which codes a binary indicator democracy ∈ {0, 1} equal to 1 if both legislative and executive posts are filled by meaningful elections.

Our second measure of accountability measures the informational environment of a given country-year. Our measure of this concept is the HRV index of government transparency.Footnote ⁵⁹ The HRV index is a continuous measure, derived from an item-response model of data missingness. It captures the government's tendency to disclose credible information, pertinent to economics and citizen welfare, to the public.Footnote ⁶⁰

Estimating the Direct Effect of the MDGs

We begin by estimating the direct effect of the MDGs on primary enrollment rates. That is, we start by parsing the evidence that the MDGs increased the growth rate of primary enrollment across countries. Proposition 2 contends that the effect of the MDGs should be positive, and should be larger in unaccountable states. To conduct this estimation, we estimate a varying intercepts multilevel model of the following form:

(1)$$\eqalign{ {\Delta} P_{i,t} & = \alpha _i + \beta _1Transparency_{i,t} + \beta _2Democracy_{i,t} \cr & + I\lpar {t \ge 2000} \rpar \lsqb {\gamma_1 + \gamma_2Transparency_{i,t} + \gamma_3Democracy_{i,t}} \rsqb \cr & + \beta _3{\Delta} rGDP_{i,t} + \beta _4Aid_{i,t-1} + \tau _t + \epsilon_{i,t} \cr & {\rm where} \cr & \alpha _i\sim {\rm {\cal N}}\lpar {\mu_a,\sigma_a} \rpar \cr & \tau _t \sim {\rm {\cal N}}\lpar {\mu_\tau, \sigma_\tau} \rpar } $$

where ∆ denotes the difference operator, i indexes country, and t indexes year. By first differencing the outcome term, we eliminate country-specific factors that drive the level of primary education as potential confounds from the specification. In controlling for country (α _i) and year (τ _t) random effects, we help to adjust for random variations in the growth rate in enrollment figures across countries and time. Our results are robust to the inclusion of linear and quadratic time trends.

Our hypotheses are reflected in the γ _j coefficients. Proposition 2 holds that γ ₁ > 0. Our moderating hypotheses from proposition 2, predicting that any such effect of the MDGs should diminish in transparent and democratic polities, imply that γ ₂, γ ₃ < 0. Results from the model specified in equation 1 are presented in Table 3.

Table 3. The relationship between the millennium declaration and the primary school enrollment rate.

Notes: Multilevel model analysis allowing for random effects by country and year. Lagged primary enrollment rate and country-specific time trends not shown. Standard errors presented in parentheses. *p < .10; **p < .05; ***p < .01.

Table 3 reveals that coefficients on the MDG indicator are consistently positive and statistically significant at conventional levels. Descriptively, these results indicate that the change in primary enrollment rates was, year on year, consistently more positive following the Millennium Declaration, conditioning on countries, years, and country-specific cubic trends. However, our motivating theory would posit that, failing to control for the confounding effects of transparency and electoral accountability, our estimating equation is inaccurate.

Column 2 updates the specification with these controls. Specifically, we control for transparency and accountability linearly, along with the change in real GDP per capita (not shown) as well as measures of aid, including total aid received, and aid targeting primary and secondary education. The resulting estimate on the MDG indicator maintains its significance and magnitude. However, we still are not properly specifying a linear model in line with our theoretical motivation. To accurately reflect our intuition, we turn to interaction effects, as displayed in columns 3 through 5.

Here we see suggestive evidence supporting our theoretical framework. Namely, across specifications the MDG indicator is significant and positive. We further note that the conditional effect of MDGs declines with both accountability and transparency. Figure 2 displays the marginal effects associated with the relationship between the Millennium Declaration and primary enrollment rates over different values of transparency and among democracies and nondemocracies. Column 5 combines the specifications and finds statistically significant relationships for only transparency.

Figure 2. Linear marginal effects plots characterizing heterogeneity in effect of MDG on primary enrollment rates over values of democracy (left) and transparency index (right)

Results from equation 1 are thus consistent with our proposition 2: (1) The MDGs are associated with an increase in primary enrollment rates, and (2) this effect is diminished in accountable (and particularly in transparent) states.

As an additional test, we replicate the model specified in equation 1, substituting secondary and tertiary enrollment rates for primary. These results offer some preliminary evidence for proposition 3 and lemma 4. Table 4 presents the results, with no evidence of a relationship between enrollment rates at higher levels of education and MDG adoption after controlling for aid flows.

Table 4. The relationship between the millennium declaration and the secondary school enrollment rate.

Notes: Multilevel model analysis allowing for random effects by country and year. Lagged secondary enrollment rate and country-specific time trends not shown. Standard errors presented in parentheses. *p < .10; **p < .05; ***p < .01.

The coefficient on MDGs is noisily estimated but commensurate in magnitude to the coefficient from the primary enrollment rate regression. It is interesting that we still see negative coefficients on the interaction terms with transparency although these are marginally significant and only half the size of the primary enrollment results.

lthough these results are consistent with our theoretical expectations, they do not demonstrate that the same countries that increase their primary enrollment rates post-2000 also increase their secondary enrollment rates. To test this relationship, we turn to characterizing the country-level substitution from secondary enrollment to primary in our data.

Estimating the Substitution Effect of the MDGs

Method 1: Estimating the Difference between Primary and Secondary Enrollment Rates

We begin our analysis of substitution by employing a simple measure of the relationship between primary and secondary education within a given country-year: the difference between secondary and primary enrollment rates (P _i,t − S _i,t). If our substitution claim is correct, this measure should increase following the promulgation of the MDGs—that is, the primary enrollment rate should increase relative to the secondary enrollment rate.

We thus estimate a model of the form:

(2)$$\eqalign{\lpar {P_{i,t}-S_{i,t}} \rpar = & \alpha _i + I\lpar {year \ge 2000} \rpar \cr & \lsqb {\gamma_1 + \gamma_2Transparency_{i,t} + \gamma_3Democracy_{i,t}} \rsqb \cr & + {\bf X}_{{\bf i},{\bf t}}{\bi \beta} + \epsilon _{i,t}}$$

where i denotes country, t year, α _i is a country random effect, and X_i,t β is a data vector and associated coefficients. Included as controls are the transparency and democracy measures, GDP per capita, and the earlier measures of aid.

Our hypotheses hold that γ ₁ > 0, and γ ₂, γ ₃ < 0. We present results from the model specified in equation 2 in Table 5. Consistent with our theoretical expectations, γ ₁ is positive across all specifications and significant at conventional thresholds, indicating that MDG adoption is associated with an increase in the enrollment rates separating primary and secondary students.

Table 5. Coefficients relating the millennium declaration to the difference between primary and secondary enrollment rates across countries.

Notes: Multilevel model analysis allowing for random effects by country and year. Change in real GDP per capita and country-specific time trends not shown. Standard errors presented in parentheses. *p < .10; **p < .05; ***p < .01.

Unlike the results presented for primary enrollment in isolation, we note that the transparency index interaction is no longer significant while democracy appears as a highly significant moderator when estimated in the fully interacted specification (column 6). Aid for primary education is also significant and positive while the estimate for secondary education aid is a precisely estimated 0.

However, the use of the difference between primary and secondary enrollment as an outcome measure induces considerable loss of information. We are unable to tell, for instance, whether the changes in the measure are driven by an absolute decline in secondary rates, by a surge in primary enrollment rates, or some combination of the two factors.

Given these concerns, we turn to two alternative specifications: the first is an error correction model and the second is a vector autoregression discussed in our appendix.

Method 2: Error Correction Specification

First, we turn to an error correction model (ECM). Developed to deal with cointegrated time series, ECMs postulate a long-term equilibrium relationship between covariates—here primary and secondary education.Footnote ⁶¹ We are interested in changes in this long-term equilibrium. Specifically, we estimate a model of the form:

(3)$${\Delta} P_{i,M,t} = \gamma \lsqb {P_{i,M,t-1}-\beta_{i,M}S_{i,M,t-1}} \rsqb + \delta _{i,M}{\Delta} S_{i,M,t-1} + \epsilon _{i,M,t}$$

where, as in equation 1, ∆ is the difference operator, i denotes country, t denotes year, P is the primary enrollment rate, and S the secondary. M ∈ {0, 1} is an indicator taking the value 1 after the year 2000.

The expression P _i,M,t−1 − β _i,M S _i,M,t−1 denotes the equilibrium relationship between primary and secondary education. When P _i,M,t−1 = β _i,M S _i,M,t-1, the system is in equilibrium, and primary enrollment rates will not tend to adjust. Given γ < 0 (which is the case in all estimates), when primary education exceeds its equilibrium target (P _i,M,t−1 > β _i,M S _i,M,t−1) it will tend to decline. By contrast, when primary education falls below its equilibrium level (P _i,M,t−1 < β _i,M S _i,M,t−1), primary enrollment rates will tend to rise.

Since we are interested in changes in the equilibrium relationship between primary and secondary education, we examine changes in the β _i,M parameter before and after the introduction of the MDGs, and variation in this term across countries, as given by:Footnote ⁶²

(5)$$ \eqalign{\beta _{i,M} & = \alpha + \pi _1Trans_i + \pi _2Dem_i + \pi _3Trans_i \times Dem_i \cr & + I\left( {year \ge 2000} \right)\left[ {\omega _1 + \omega _2Trans_i + \omega _3Dem_i} \right] \cr & + ln(GDPpc_{2000})_i + \eta _{i,M}} $$

Proposition 3 holds that ω ₁ > 0, implying that the equilibrium level of secondary education, relative to primary, declines once the MDGs go into effect. Lemma 4 holds that ω _j, j ∈ {2, 3} < 0—the substitution effect is diminished in accountable polities. Estimates from equation 5 are presented in Table 6.

Table 6. β substitution tests for equilibrium levels of secondary and primary education.

Notes: Multilevel model analysis allowing for random effects by country and year. Change in real GDP per capita and country-specific time trends not shown. Standard errors presented in parentheses. *p < .10; **p < .05; ***p < .01.

As suggested by Table 6, the Millennium Declaration corresponds to a divergence of primary and secondary enrollment rates, suggesting a substitution effect (as indicated by the statistically significant coefficient in column 1).

However, this substitution effect is mitigated by democratic accountability, with the interaction term in column 2 entering negative and marginally significant. A similar effect is documented with transparency, where the substitution effect is reduced in countries with higher levels of transparency. The coefficient on the transparency term is positive, indicating that, before the MDGs, transparent governments had a higher equilibrium level of primary relative to secondary education than unaccountable states. This is consistent with existing claims, notably by David Stasavage.Footnote ⁶³ Combining these specifications suggests that the moderating effect is driven primarily by transparency, not electoral accountability.

ECM models, however, impose strong parametric assumptions. We merely posit that primary and secondary enrollment rates exhibit a long-term structural relationship of the form documented by equation 3, and rely heavily on this equilibrium in our interpretation. In particular, Grant and Lebo argue that such general error correction models will be biased if the respective time series—here primary and secondary enrollment rates—are either not co-integrated, or are co-integrated of different orders.Footnote ⁶⁴ The coefficient γ, in that instance, will be inconsistent—and so too will our parameters of interest β.

To address this concern, we first examine the co-integration of these two time series. While the series appear co-integrated in global tests, running these tests country by country supports co-integration in only roughly half our sample of states. We therefore re-estimate our models, restricting our sample to only those states where the evidence clearly indicates that primary and secondary enrollment are co-integrated. Results are substantively similar to those in the full sample.

Mechanisms

Thus far we have developed a model that predicts GPIs can lead to unintended substitution behavior and have presented empirical evidence consistent with our predictions. First, we have shown that MDG adoption prompted an increase in primary school enrollment as was its intention but that the magnitude of this relationship is strongest in the least accountable polities in our data. Second, this increase in primary enrollment was accompanied by a decline in secondary enrollment, consistent with our theoretical prediction for substitution effects. Again, evidence of this substitution effect is strongest in the least accountable polities.

However, our formal model is agnostic with respect to the specific mechanisms by which GPIs operate (see Figure 1). We are uncertain about whether these results arise from the budgetary impacts of foreign aid and investment which may respond to GPIs (“transnational pressures”), the activation of civil society (“domestic politics”), or from a direct effect on policymakers themselves (“elite shaming”). The moderating effects of accountability documented earlier suggest a possible channel via domestic politics. We therefore test whether a transnational pressure channel via foreign aid also operates.

To conduct this analysis, we treat foreign aid as a mediating variable and use a two-stage regression framework as in Baron and Kenney.Footnote ⁶⁵ The first stage predicts changes in foreign aid flows as a function of MDG adoption, given in equation 6. We interact MDG adoption with the primary enrollment rate in the year 2000, reflecting the intuition that performance assessments like the Millennium Development Goals affect aid flows specifically for underperformers.

(6)$$\eqalign{ {\Delta} Aid_{i,t} & = \alpha _i + \beta _1Primary_{i,2000} \cr & + I\lpar {t \ge 2000} \rpar \lsqb {\rho_1 + \rho_2Primary_{i,2000}} \rsqb \cr & + \beta _1{\Delta} rGDP_{i,t} + \beta _2GDP_{i,2000} + \beta _3Transparency_{i,t} \cr & + \beta _4Democracy_{i,t} + \tau _t + \epsilon _{i,t} \cr & {\rm where} \cr & \alpha _i\sim { {\cal N}}\lpar {\mu_a,\sigma_a} \rpar \cr & \tau _t\sim { {\cal N}}(\mu _\tau, \sigma _\tau\rpar} $$

We save the coefficient on MDG adoption (ρ ₁) for use in our second-stage analysis. We test a number of different measures of foreign aid, plotted in the left panel of Figure 3. As illustrated, aid targeting primary education increases following MDG adoption, particularly toward countries with low initial primary enrollment rates. This first-stage pattern is consistent with the theorized mechanism of “transnational pressures” discussed in the introduction to this symposium and illustrated in Figure 1—aid flows do respond to the MDGs.

Figure 3. First stage (left panel) and overall (right panel) results from mediation analysis on the mechanism of foreign aid

In the second stage, we regress the change in primary enrollment on MDG adoption and the same aid measure, replicating the specification described in equation 1 and saving the β ₄ coefficient. Under restrictive assumptions,Footnote ⁶⁶ the product of these coefficients captures the average mediation effect between MDG adoption and primary enrollment rates that travels via changes in aid flows. To accommodate the multiply imputed data, we bootstrap these estimates and plot the distribution of the results in the right panel of Figure 3. Pseudo p-values are calculated as the share of bootstrapped estimates greater than 0.

As illustrated, mediation analysis offers no robust support for the theorized mechanism of foreign aid, despite the first-stage channel being active. While foreign aid targeting primary enrollment does respond to the assessments provided by the MDGs (left panel of Figure 3), there is no evidence that this is the mechanism by which policymakers reallocate resources. The magnitude of the average mediation effect (AME) is only a fraction of the overall coefficient and is only marginally significant for the full data. Our null finding is consistent with the results of Kijima and Lipscy who also find that, while assessment participation is associated with higher foreign aid inflows, these sources of aid are not meaningfully associated with higher net secondary enrollment.Footnote ⁶⁷ As such, we conclude that, in the context of education and the MDGs, transnational pressures are not the primary mechanism by which this particular GPI induces change.