One of the more striking examples of the power that global performance indicators exercise in world politics is the case of the Aid Transparency Index (ATI), an annual rating and ranking of international development assistance donor agencies by Publish What You Fund (PWYF).Footnote 1 PWYF was established as a nongovernmental organization (NGO) in 2008 to monitor the progress of international donors by disclosing where they spend their aid funds, on whom, and for what. PWYF is small, with nine staff members and a 2017 budget of less than £600,000.Footnote 2 Housed in a modest one-room office above an Italian restaurant on London's South Bank, PWYF has no direct material power with which to coerce change in the behavior of these large multilateral and bilateral donors.
Nonetheless, the ATI has contributed greatly to global aid transparency. The ATI has attained and exercised significant symbolic and normative power by defining clear indicators and benchmarks for donor transparency.Footnote 3 Its authority derives from its independence and its process of working with donors and external reviewers to construct and validate the annual ratings and rankings. The ATI catalyzes behavior change by publicly comparing and categorizing donors as “very good,” “good,” “fair,” “poor,” and “very poor” performers, thus invoking peer reputation and status concerns and mobilizing pressure for donor reforms.
This is important. Since the first High-Level Forum on Aid Effectiveness in Rome in 2003, a cascade of initiatives and organizations has sought to pry open the spigot of information on donor agencies' projects and programs.Footnote 4 The principles of transparency and open data are centrally embedded in the 2030 Sustainable Development Goals and the UN's “Data Revolution,”Footnote 5 in hopes of centralizing information and improving donor coordination, country-level development planning, and management.Footnote 6 Advocates also claim that transparency empowers the poor by providing opportunities for citizen voice and feedback. For example, in January 2018, two Ebola survivors from Sierra Leone sued their government, alleging that it had misdirected millions of dollars of foreign aid marked for the Ebola crisis.Footnote 7 Transparency, simply put, promises to make aid more inclusive, accountable, and effective. The notion that transparency coincides with public disclosure of greater information is not limited to the ATI. For example, Hollyer, Rosendorff, and Vreeland have taken greater public disclosure to be a key element of transparency in a series of scholarly publications.Footnote 8
However, to an aid industry long subject to public scrutiny, the push for transparency presents a double-edged sword. On the one hand, the transparency agenda promises to assuage NGOs, civil society organizations (CSOs), and national parliaments who have long demanded enhanced access to donor information as a tool for accountability. On the other, as our interviews with donor staff and management suggest, transparency also threatens to lift the veil on internal processes that might disrupt donors' relationships with borrowers, introduce onerous reporting burdens, and invite further interrogation from parties disaffected by aid programs. Not all donor agency staff agree that the ATI's thirty-nine indicators represent the “right path” toward transparency. PWYF's efforts to promote convergence on the ATI's particular standards are not simply a matter of pushing on an open door. The ATI constitutes a critical case study that can address key questions regarding under what conditions and through what mechanisms an NGO-produced global performance indicator (GPI) can influence powerful actors to do what they otherwise might not do on their own volition. Bisbee and colleagues' and Doshi, Kelley, and Simmons's contributions to this symposium demonstrate the impact of GPIs propogated in part by international donors (the Millennium Development Goals and Ease of Doing Business indicators, respectively) on the behavior of countries.Footnote 9 We explore whether the donor community is also itself influenced by GPIs. We find that the ATI does affect donors' transparency practices, but not evenly. Agencies that have aid distribution as their primary operational mandate (e.g., the US Agency for International Development) are much more likely to respond to the ATI than donors for whom the provision of official development assistance is secondary to their core mandate (e.g., the US Department of Defense). When agencies respond to the ATI, they do so because of the ATI's influence on policy elites. The ATI diffuses via networks of elite actors, consistent with Morse's findings in her analysis of the Financial Action Task Force in this symposium.Footnote 10
Like Doshi, Kelley, and Simmons's findings on the Ease of Doing Business indicators, we find that the ATI operates via social pressure.Footnote 11 In the case of the ATI, however, it is not public reputational sanctioning that drives agency responsiveness. Rather, elites worry about their status in their peer group of aid professionals and are susceptible to socialization around new norms. The very process of being closely monitored and regularly interacting with the PWYF team produces inter- and intra-organizational learning and norm diffusion, and professionalizes aid staff and management around ATI's standards. Secondarily, the ATI enhances domestic political pressure by equipping transparency reform proponents (particularly those with some material power over aid agencies) with critical information and clear standards to guide policy change.
In what follows, we provide a brief overview of the international aid transparency movement to provide context for our study of the ATI. We use quantitative analysis to understand variation in donor performance on, and donor response to, the ATI. Then we further examine mechanisms using qualitative interview evidence.Footnote 12 This qualitative research enables us to dig deeper into how the channels of influence work. Together, our quantitative and qualitative evidence informs our conclusion that the ATI does in fact influence donors, and does so primarily through elite channels of influence and secondarily through its indirect influence via political pressure of donors' principals.
The International Aid Transparency Movement and the ATI
The transparency movement represents a sea change for international aid. Ten years ago, if you wanted to find out how much development assistance was going to Kenya, to whom, and for what, you would have needed high-bandwidth access to the Organisation for Economic Co-operation and Development's (OECD) creditor reporting system and the ability to decipher the complex accounting jargon of elaborate spreadsheets. Even then, only highly aggregated data were available. Actual project documents, which might or might not have contained information on the subnational locations of aid activities, implementing partners, and details on project objectives, could be attained only in hard copy and for a fee, through a few select donors' public information centers. Borrower governments themselves had scant knowledge of where the aid was in their country. As one Malawian deputy minister of finance told us in 2010, “We don't really know where the aid is in our country, what [it] is doing, and who is doing it. How can we plan to properly spend government money to build schools, hire doctors, or provide services when we don't know if our donor partners are already doing this?”Footnote 13
By 2017, the aid information landscape had dramatically changed. It is easy now to go directly to the World Bank's website to find full project documents. Most major donors have created public dashboards, with infographics and interactive maps. The International Aid Transparency Initiative, established in 2008, is rapidly moving toward a common, publicly accessible database with standardized information on the precise geographical locations of aid, budget data, and activity-level project details. While significant challenges remain, the past decade has been nothing short of a revolution in aid data.Footnote 14 Achieving this transparency in the multibillion dollar global aid industry has not been an easy task. Donor agencies have enjoyed relative opacity for most of their existence. Past efforts to enact fundamental changes in national freedom of information acts and organizational information disclosure policies have been met with resistance and persistent delays.Footnote 15 Numerous published analyses and interviews point out pervasive problems of organizational inertia, staff's cultural fears surrounding transparency, and a myriad of technological and economic barriers to change.Footnote 16
PWYF was established in 2008 by International Aid Transparency Initiative (IATI) advocates, including founding CEO Karin Christiansen, with funding from the Hewlett Foundation and Open Society Foundation. Three short years later, at the Fourth High Level Forum on Aid Effectiveness in Busan, South Korea, in November 2011, most major donor countries and agencies—including many from the global south—committed themselves to reporting their aid information to a common standard.Footnote 17 A rich set of supranational initiatives (such as the EU Aid Transparency Guarantee and the Global Partnership for Effective Development Cooperation), national-level policies, open data systems,Footnote 18 and international NGOs and networks have since been created to advocate for open aid data.Footnote 19 PWYF's ATI plays a specific role in this advocacy movement by translating the broad goals of transparency into measurable standards of performance and using ratings and rankings to monitor and enforce donor agencies' compliance with these international commitments.
The ATI works explicitly through engagement with donors and independent experts to collate annual data. Publish What You Fund publishes the evaluation criteria for a given year's index and engages in a three-month dialogue with every aid agency prior to the finalization of annual ATI scores. There were annual releases of the ATI from 2011 to 2016.Footnote 20 While the methodology of the ATI has been modified slightly over time, the ATI has always focused on publishing specific data regarding aid flows at the activity level—that is, the details regarding particular interventions and projects. While the ATI is primarily disseminated to the public via hierarchical rankings of agencies into categories (good, fair, poor, etc.), these rankings draw from a continuous scale drawn from a series of indicators. There are currently thirty-nine indicators in the ATI, which cover information such as project title, description, budget, and objectives of interventions. Table 1 provides greater detail.
Table 1. 2016 ATI indicators and weights
Source: Publish What You Fund. 2016a. 2016 Aid Transparency Index.
The comprehensive index largely evaluates national government agencies and international organizations, as well as a few foundations (e.g., Hewlett and Gates). The primary focus is bilateral aid agencies (e.g., the US Agency for International Development or the UK Department for International Development) and major multilateral aid-focused organizations (e.g., the UN Development Program or the World Bank), who together account for the majority of ODA. The index also includes agencies that disburse significant amounts of foreign aid, even though development assistance is not their primary mandate. For example, the 2016 index includes six US government agencies, only three of whom have ODA as a primary mandate.Footnote 21
When and How Does the ATI Influence Aid Donors?
A recent survey of staff within US development agencies revealed that over 75 percent of respondents thought the ATI had a “very positive impact” on their own agency's transparency efforts.Footnote 22 To what extent does observable agency behavior support this claim? If indeed the ATI influences organizational practices, under what conditions do we observe this impact? How exactly does the ATI influence targeted donors? Key informants and our examination of agency-level documents suggest that not all donors are equally concerned about, or responsive to, the ATI's assessment of their agencies' performance.Footnote 23 One key mediating factor appears to shape agencies' reaction to the ATI: whether their primary mandate is to provide ODA.
In the introduction to this symposium Kelley and Simmons suggest there are several possible channels of influence for GPIs.Footnote 24 In the case of the ATI, our interviews indicate that two channels are at play.Footnote 25 First, the ATI influences donors by inciting external political pressure, particularly by providing critical information to key domestic stakeholders who may then use the ATI's scores to monitor, sanction, and reward aid agencies. Second, the ATI plays a direct role in shaping the interests and behavior of elites within aid organizations by translating broad political mandates and commitments regarding transparency into distinct operational policies that can be enacted through internal reforms.
Donors act as the agents of principals, notably national legislatures (in the case of bilateral agencies, such as USAID or the US Department of Defense) or multinational executive boards (in the case of multilateral agencies, for example, the World Bank, or foundations such as the Hewlett Foundation). In this relationship, principals (member states or national parliaments) can exercise oversight and control over agents (donor agencies) through mechanisms such as mandated audits or threats of changes to financial appropriations. Here, as principal-agent theory hypothesizes,Footnote 26 the ATI reduces critical information asymmetries and provides clear assessments of agents' relative performance. With such information, principals can more easily detect and sanction agents' deviant behavior. This is consistent with previous scholarship, which has shown that aid agencies are sensitive to demonstrating success to principals, with some agencies much more concerned with appearing successful to principals than others.Footnote 27 The mere presence of the ATI incites agencies to be more proactive in transparency reforms, often in anticipation of increased principal oversight and control even when principals do not actually make overt gestures to this end.Footnote 28
Our interviews also suggest that the ATI shapes what transparency means to the elite professionals who staff aid agencies. This channel focuses on aid professionals' logic of appropriateness rather than the more traditional payoffs that might accrue to organizational reputational changes, such as greater funding, access to markets, or private investment. Scholars often frame donor organizations, in our view correctly, as part of an “aid industry.”Footnote 29 Professionals in that industry see themselves as part of a broader community of peers. By influencing the meaning of what it is to be a “good” aid agency and thus “good” aid professional, the ATI influences aid professionals' actions.
Both the political pressure and direct elite response channels imply a scope condition for the ATI's influence. We hypothesize that when aid is an agency's primary mandate (versus a secondary mandate or goal), the agency will be more sensitive to the delegitimizing effect of poor ATI scores. Agencies that do not primarily provide aid are less likely to be staffed by individuals who see themselves as part of a broader professional aid community, nor are these agencies' political principals likely to express concern over agencies' level of aid transparency.
We hypothesize that both channels—political pressure and direct elite response—operate, yet we remain agnostic about which of the two channels is more influential. We construct a quantitative test for which of these is the more influential mechanism. We believe leveraging variation in agency insulation from the pressure to respond to principals, or (as we call it) an agency's relative independence, is a way of getting purchase on which of these channels is the dominant means through which the ATI influences agencies. We hypothesize that the political pressure and direct elite response channels have conflicting implications for whether more or less independent agency will be more responsive.
If political pressure is the stronger channel, then less independent agencies—agencies that are more susceptible to political pressure—should be more responsive to the ATI, as measured via their yearly net change on the ATI ratings and rankings. This channel rests on the ATI enabling the materially weak PWYF and its key allies in the aid transparency movement to capture principals' “power of the purse” and executive or legislative authority over donor agencies. Consequently, if principals are indeed paying attention to and taking action on the ATI's information, then aid agencies with higher degrees of dependence on principals' financial contributions should be especially sensitive to the ATI's effects.
However, if direct elite response is the stronger channel, then agencies that have more relative independence to engage in needed reforms are likely to be more responsive to the ATI. This is because more independent agencies can react faster to emerging standards around transparency policies and to the ATI's professional norm diffusion and socialization effects. Less independent agencies may hold more limited capacity to enact wide-sweeping operational reforms around transparency and data reporting without the consent and resources of their principals.
To restate our argument: we hypothesize that the ATI influence's donor agencies, prompting them to alter their information disclosure practices. The ATI achieves this both via reducing information asymmetries for political principals (political pressure) and by constructing meaning and inducing competition not tied to direct payoffs for the professionals who staff donor agencies (direct elite effects).
We explore these hypotheses using a mixed-methods approach. We employ regression analysis using a panel data set of ATI scores with the unit of observation as the agency-year. In addition to each agency's overall and indicator-by-indicator score from 2011 to 2016, our data set includes independent data on aid agency disclosure and transparency practices from 2006 to 2013 compiled by AidData at our request.Footnote 30 The AidData data allow us to model the presence of the ATI as a treatment, examining whether (and which) agencies are responding to the ATI.
Then we utilize 465 semistructured key informant interviews between 2010 and 2017, primarily in the US, UK, Malawi, Uganda, Kenya, Nepal, and Honduras. These interviews were conducted with staff and management of donor organizations' headquarters and mission offices, national parliamentary and US Congressional staff, relevant think tanks, civil society groups, and international nongovernmental organizations. In addition, we analyzed a wide array of primary and secondary materials, including donor organization press releases, policy documents, parliamentary and congressional hearings and legislation, and the research and advocacy materials of NGOs, CSOs and think tanks.
Quantitative Results: Does the ATI Alter Donor Behavior?
To present evidence on whether the ATI has systematically affected donor practices, and for whom, we include agency independence in regression models to provide suggestive evidence that the elite channel of influence is more important than the political pressure channel in explaining the ATI's effects. To explore differing organizational response to “treatment” by the ATI, we build a panel at the agency-year of ati scores (and thus ATI coverage), complementing this with historic data on agency transparency practices. We also include a calculation of agency independence and an indicator for whether the aid agency's primary purpose involves giving foreign aid (e.g., USAID) or not (e.g., the US Department of Defense) in the data set. These agencies are hereafter referred to as “aid” and “non-aid” agencies to signal their primary mandate. Forty-five of the eighty-four agencies that appear in the ATI at some time between 2011 and 2016 are coded as aid agencies.Footnote 31 Online appendix Table A1 lists every covered agency, their country, their years of ATI coverage, and our assignment of the indicator for whether giving foreign aid is the agency's primary mandate. Table 2 provides summary statistics of key variables.
Table 2. Summary statistics of selected variables
To calculate agency independence, we build on Gilardi's work on Western European regulatory agencies.Footnote 32 The Gilardi Index is explicitly focused on agencies' relative independence from political authorizers' control and influence. Gilardi developed the index to allow cross-national calculation of formal agency independence from politicians for a variety of agencies. Independence is coded as a time-invariant measure based on the best available data. We were able to find consistent information for just four of Gilardi's twenty-one indicators of agency independence: agency head's term of office, source of budget, whether independence is formally stated, and whether the head of the agency is of cabinet rank.Footnote 33 Each of these four measures is scored between 0 and 1 using Gilardi's coding scheme. For example, agencies whose head has a fixed term of office receive one point, agencies whose term of office is six to eight years receive .8 points, and so on. While this opens up possible measurement error regarding changes in independence within organizations over time, we expect this measurement error should be orthogonal to the primary analysis and thus add noise (reducing power) rather than lead to spurious inferences. The measure constructed using the Gilardi method is broad, incorporating both aid and non-aid agencies. The online appendix details the full coding scheme.
A visual examination of ATI scores and changes over time underscores the heterogeneity of agency response to the ATI. The wide differences in donors' behavior are demonstrated in Figure 2, which shows the variation in realized scores, and Figure 3, which compares each agency's score in its most recent year of ATI coverage to the agency's score in its first first year of ATI coverage.
Figure 1. Histogram of overall ATI scores, 2011–2016
Figure 2. Net change by agency on ATI over coverage period
Figure 3. Marginal effect of treatment by independence for non-aid agencies
Table 3 uses the ATI overall score data to examine differential performance on the ATI with and without country, year, and country*year fixed effects, allowing us to examine intra-country differences in ATI performance. The results are quite stable with and without these fixed effects. Agencies whose primary purpose is to give foreign aid perform better in the ATI ratings than those for whom foreign aid is a secondary task. This is true looking both across all agencies and within a given country's set of covered agencies.Footnote 34
Table 3. Performance on the ATI
Notes: OLS; Standard errors in parentheses, clustered by year. *p < .10; **p < .05; ***p < .01.
Table 3 suggests that for non-aid agencies, greater independence has no association with higher ATI scores. For aid organizations, however, the picture is quite different. More independent aid organizations score better on the ATI than less independent aid organizations from the same country.
Of course, differential performance on the ATI by aid agencies of varying independence does not mean that it is the ATI that has affected aid agencies' disclosure and transparency practices. Aid agencies, particularly those with more independence, may simply be more apt to disclose information, regardless of the ATI's influence.
Our primary econometric test exploits intertemporal variation in ATI coverage across agencies. The ATI covers a significant majority of consequential aid agencies—that is, those that provide between 84 and 94 percent of global official development assistance between 2011 and 2016.Footnote 35 Of the eighty-four agencies in our sample to ever be included in the ATI, fifty-eight are included in the first full year of the ATI (2011). In 2012, seventy-two are covered. In 2013, sixty-seven agencies receive ATI scores. The empirical strategy in Table 4 exploits this within-organization and across-time variation, including both agency and year fixed effects in examining agency responsiveness to the ATI.
Table 4. ATI as treatmentFootnote 39
Notes: Standard errors in parentheses, clustered by year. *p < .10; **p < .05; ***p < .01.
Table 4 shifts to modeling inclusion in the ATI in a given year as a binary treatment variable. If an agency exists in the ATI in a given year, this variable takes the value of 1. Alternatively, if an agency is not covered by the ATI in a given year it takes the value of 0.Footnote 36 Intra-agency variation in coverage years, when combined with agency fixed effects, allows Table 4 to better identify whether inclusion changes within-agency transparency practices.
Since ATI scores exist only in years where an agency is covered by the ATI, we draw on disclosure quality measures from AidData's historic aid flow reporting data described earlier to separately measure transparency practices.Footnote 37 These data commence in 2006, five years prior to the ATI's first year of coverage and two years prior to the launch of PWYF. They run through 2013, thus overlapping with the first three waves of the ATI (in 2011, 2012, and 2013).Footnote 38
To construct the dependent variable in Table 4, we use a subset of the AidData historic data. This is quite similar in thrust to the ATI's “activity level” component (see Table 1), which focuses on the completeness of individual aid activity reporting. We also employ AidData's measures of the percent of projects for a given donor in a given year that provide a project title, a project description, report the source of the project's funding, and describe the type of flow (e.g., a grant as opposed to a loan). We take the simple average of these four measures, and call this the aiddata activity scale.Footnote 40
This scale is a distinct measure of a subset of items on which the ATI focuses.Footnote 41 It begins before the ATI and is available even when agencies are not covered by the ATI (and thus in agency-years when the binary ATI treatment variable takes a value of 0). We can thus examine whether inclusion on the ATI is in fact associated with a change in a covered agency's disclosure behavior, leveraging the variation in agency years of coverage we discussed earlier. In models with both year and agency fixed effects, Table 4’s analysis indicates whether within-agency performance rose in years where the agency was covered by the ATI (over and above secular time trends).
Table 4 strongly suggests that inclusion in the ATI changes aid agencies' behavior. Models 3 and 4 indicate that the quality of covered agencies' reporting at the “activity level”—the level of individual projects and interventions—improves when agencies are covered by the ATI. This finding is robust in the case of only dedicated aid agencies, however. When year fixed effects are included, the disclosure practices of aid agencies are positively correlated with coverage by ATI assessments, but there is no evidence that this holds for non-aid agencies.
This relationship is further conditional on agency independence. Models 5 through 8 of Table 4 examine the role of agency independence for aid and non-aid agencies. While there is no evidence of a relationship between independence and response for non-aid agencies, for aid agencies, greater independence is associated with greater improvement on the AidData scale, ceteris paribus. Indeed, for the median aid agency on the independence scale (independence = .375), there is no statistically significant effect of ATI inclusion. For a firm at the seventy-fifth percentile, however (independence = .5), there is a ten-point treatment effect statistically distinguishable from 0.Footnote 42 This ten-point effect would raise the median aid agency (score = 89.1) to a near-perfect 99.1.Footnote 43
Figures 3 and 4 graphically represent the role of independence for aid and non-aid agencies respectively (Table 4, models 4 and 6), demonstrating the importance of agency independence for aid but not non-aid agencies.
Figure 4. Marginal effect of treatment by independence for aid agencies
The AidData Activity Scale captures the completeness of the information donors disclose about projects (titles, descriptions, financing agency names, and flow types) in percentage-point terms. One scale point is the equivalent of a one percentage point shift in performance on the underlying measures. The ten-point treatment effect for an aid agency at the seventy-fifth percentile of independence is equivalent to a shift from nondisclosure to full disclosure of ten percentage points of a given agency's projects. For example, in 2012 USAID had USD 17 billion of total disbursements and over 8,500 activities.Footnote 44 This means an increase of ten percentage points of transparency for USAID would cover about USD 1.7 billion of aid flows and 850 activities. For these 850 activities, observers could now know what USAID's programs were doing, where, and who was financing them. Individuals and civil society organizations in recipient countries could access information that would help them hold foreign donors and their own governments accountable. Individuals and politicians in the US could better understand where their tax dollars are going. Other donors could better understand what 1.7 billion dollars of USAID funding was doing, allowing for better coordination and planning.
The independence scale itself is a patchy measure—a mere echo of the original Gilardi measure on agency independence. As such, these econometrics provide strong evidence that ATI coverage is associated with changes in the disclosure practices of (some) covered agencies. However, this analysis can only be suggestive about the mechanisms underlying any ATI coverage effect. In sum, donors included in the ATI are more transparent, especially if they are more independent and designated aid agencies. The finding that greater aid agency independence is associated with greater change in response to ATI coverage suggests the ATI primarily works through aid agency elites: if agencies primarily responded to pressure from their principal(s), then more dependent (less independent) agencies should perform better on the ATI. But this does not mean that direct elite influence is the only channel of influence. To further explore how the ATI drives donors to change, we turn to a closer examination of the nature of the ATI's power and its key mechanisms of influence over donor agencies.
How Does the ATI Affect Donors? Qualitative Evidence on Mechanisms and Channels of Influence
Between 2010 and 2016, we conducted 465 open-ended and semistructured interviewsFootnote 45 with aid donors, governments, NGOs and CSOs, academics, and other subject matter experts. We conducted these interviews primarily in seven countries: the United States, United Kingdom, Malawi, Uganda, Kenya, Nepal, and Honduras (see Table 2 and the online appendix). Many of these interviews broadly focused on aid transparency in donor agencies, with some focused on specific sectors (such as climate change, agriculture, and education). In most cases, to avoid leading questions, we did not ask directly about the ATI or its effects. Instead, we asked about general pressures for donor-level transparency, general data needs and uses in resource allocation decisions, and awareness and use of open aid data at the agency or country level. Some interviewees referred to the ATI explicitly. In other cases, interviewees referred to the “index” associated with the International Aid Transparency Initiative or Publish What You Fund. We subsequently coded all the interviews to assess both awareness of the ATI and, where awareness existed, the overall perceptions regarding the ATI's influence over donors' transparency behavior.
As Table 5 indicates, 186 (40%) of all interviewees indicated they were familiar with the ATI. Of these, 125 thought the ATI had a positive influence in shaping donor agencies' transparency behavior, whereas sixty-one reported that they did not think the ATI had any influence or simply expressed that they had no opinion. Ninety-three of these 125 who reported a positive influence for the ATI were based in London, Washington, DC, or another major donor country where donor aid agencies are headquartered.
Table 5. Perceptions of ATI influence on donor transparency behavior
*Includes interviews that were conducted under agreement of nonattribution or deep background (off the record only), per IRB protocol. In these instances, the names are not listed in the online appendix.
** Other includes international and national nongovernmental organizations, civil society groups, think tanks, and academic subject matter experts.
***ATI awareness includes explicit reference or implicit reference (for example, the “index used by” the International Aid Transparency Initiative or Publish What You Fund). See the online appendix for interview list.
General awareness of the ATI was largely confined to interviewees who worked in donor headquarters (e.g., London, Stockholm, Brussels, Washington, DC) or with NGOs, think tanks, and academic research programs that pay explicit attention to aid transparency issues. ATI awareness was significantly lower among government and donor staff in borrowing countries. In these instances, respondents were more aware of domestic donor aid transparency initiatives, such as their respective aid management platforms. In a few cases, respondents conflated aid transparency with general government transparency.Footnote 46 The handful of respondents in aid-receiving countries that signaled awareness of the ATI were individuals in government or donor agencies that were explicitly involved in reporting to domestic aid information management platforms or those who worked with domestic aid-transparency advocacy groups that had connections to transnational aid-transparency advocacy groups (such as Development Initiatives).
Direct Elite Response to the ATI's Social Power
Donor agencies clearly care about their reputation and perceived legitimacy, even when such status is not explicitly linked to material rewards or sanctions. In interviews at donor headquarters and in official organizational documents, management and staff nearly universally report that their agencies use ATI ratings and rankings to draw positive attention to themselves, direct negative attention to others, or signal their own good intentions and commitments. Landing in the “very good” category grants bragging rights, and large legitimacy gains, which organizations value independently from any direct link to financial sanctions or rewards.Footnote 47 Consistent with GPIs more broadly, the very act of the ATI's regularized monitoring triggers reactivity, with target actors changing their behavior (if not their underlying interests) in reaction to being evaluated, observed, and measured.Footnote 48
The ATI's ability to incite status and reputational concerns has two effects on organizations. The ATI's peer rankings serve to motivate poorly performing donor organizations to communicate renewed commitments and refocus organizational resources on transparency reforms.Footnote 49 At the same time, the ATI peer rankings and release of annual reports provide opportunities for well-performing organizations to signal to external constituents that they have made good on transparency promises and, in some instances, achieved compliance with international commitments and open data standards. According to fifteen interviews with donor agencies and NGOs in the US and UK, a fair amount of institutional rivalry reinforces organizations' desire for status and positive reputations. This appears to especially resonate in countries with multiple aid agencies that jostle for favorable positions in the eyes of the same political authorizers.Footnote 50 This is clearest in the case of the US. In our observations and interviews with donor staff and external aid experts in DC, there is a palpable rivalry between the Millennium Challenge Corporation (MCC) and the US Agency for International Development (USAID), both of whom have mandates that focus solely on official development assistance and “vie for the top spot in the ATI.”Footnote 51
The peer pressure invoked by the ATI motivates key reforms in US agencies that have performed poorly in past rankings, particularly at USAID and the US Department of State. In the first months after then-Secretary of State Hillary Clinton announced that the US would become a signatory to IATI in November 2011, these agencies (especially State) argued “vigorously through back channels” against an aggressive timeline and benchmarks for implementation.Footnote 52 They argued this was because of the anticipated high costs of required changes in underlying data technology systems. Agencies also argued they needed time to build capacity and buy-in from staff to report to new standards and dashboards, including the newly established US Foreign Assistance Dashboard. Interviews with approximately a dozen actors within US aid agencies, the US General Accounting Office, and US Congress reveal that much of the rationale behind this argument fell by the wayside when the Millennium Challenge Corporation quickly enacted an ambitious transparency agenda and vaulted to number one on the ATI in 2013.
Policy and behavior change in response to the ATI need not emerge solely from shifts in professional status. As Kelley and Simmons note, GPI processes can alter identity and professional norms through knowledge production and socialization.Footnote 53 The ATI shapes how the concept of aid transparency is collectively understood and acted upon.Footnote 54 Our interview with Publish What You Fund's Sally Paxton and public comments made by donor representatives at annual ATI launches suggest that circulation of elite staff between organizations and the sharing of “best practices” spurred by the ATI's annual rankings, publications, and public discussions foster interorganizational learning and diffusion of new policies.Footnote 55 For example, Sheila Herrling, vice president for the MCC, declared that the MCC learned directly from the transparency initiatives of other donors, including PLAN USA.Footnote 56
Annual ATI releases also diffuse learning by providing critical information on the experiences of peer institutions in building organizational cultures around transparency reforms, overcoming technological barriers, and building staff capacity in needed areas. This is done through donor-level narrative reviews, press releases, and open discussion forums. For example, interviews with DFID staff in London and USAID staff in DC affirm that the ATI has helped to “prove” that organizational change in possible, and PWYF's donor-level reports have provided key insights into how to approach difficult organizational reform.Footnote 57 This is an effect quite synergistic, but distinct, from that of peer naming and shaming. In the words of Brookings scholar George Ingram, such interorganizational learning helps organizations to “stop hugging data” and to release more of the information they gather.Footnote 58 The success of the IDA (World Bank), MCC, DFID (United Kingdom), and UNDP in scoring very highly on the ATI led other agencies such as USAID, IFC (World Bank), SIDA (Sweden), and GIZ (Germany), to adopt disclosure policies on sensitive areas such as procurement and finance that they previously had been reluctant to pursue. These first movers tend to be more independent agencies on the Gilardi independence scale. This is also true for within-country measures. For example, in the US, the MCC scores higher on the independence scale than USAID does, and was the “first mover” on improved transparency practices. In sum, some agencies performing well on the ATI are able to demonstrate to other organizations that implementing seeming “costly” or “risky” transparency reforms are, contrary to expectations, neither costly nor risky.Footnote 59
Similarly, the process of constructing the annual ATI itself is critical to understanding its social power and influence over elites within donor agencies. The inclusive nature of the ATI review process, which provides opportunities for target organizations to participate in the collection of data and validation of results, lends considerable authority to Publish What You Fund and the ATI. Because donors are directly involved in reporting to the index, and have opportunities to review the data before the index is finalized, the results are rarely openly contested.Footnote 60
Nearly two dozen interviews with the staff of PWYF and donor staff based in their agency's headquarters revealed that the process of collating the ATI results every year provides further opportunities for organizational learning and diffusion of transparency norms. During the review process, which takes several months and involves repeated interaction with PWYF, donors provide their own assessment of their performance on the ATI's indicators. When these donor-generated assessments conflict with scores generated by PWYF staff and independent reviewers, PWYF and the donor discuss what exactly is expected to achieve a full score on each of the ATI's indicators and how the donor may achieve such scores via full compliance with reporting standards.Footnote 61 Thus, according to PWYF staff in both London and DC, donors are compelled to reflect on their progress toward transparency. Because the ATI grants some points in donors' overall score for organizational commitments and implementation plans even in the absence of actual policy changes, the review period becomes an opportune time to double down on public statements in support of transparency reforms.Footnote 62 According to George Ingram, Brookings Institution senior fellow and co-lead of the Modernizing Foreign Assistance Network, the cost of inattention to this process is a stagnant or bad score for the agency, which leaves agencies' stakeholders and peers within the aid community with a lingering sense of “what have you done lately?”Footnote 63
The period of review also enhances the reactivity effects of the ATI's monitoring.Footnote 64 A donor agency may report that they are fully compliant with the ATI's expectations regarding information disclosure policies (i.e., they will argue they deserve a score of 100 on that measure). However, two sets of independent reviews plus PWYF's assessment can catch where such policies are weak—for example, if the disclosure policy provides few appeals mechanisms or puts limitations on the acceptance of third-party information. Among the USAID and PWYF staff we interviewed, the process reifies the sense that the annual review is deep and rigorous, with layers of independent analysis from subject matter experts to serve as a check against agencies' self-assessments.Footnote 65 In the case of USAID, staff reported that this prompted the organization to steer away from rhetorical commitments that might otherwise be used to “game” the assessment process (a form of shallow behavioral change), instead focusing on meaningful policy and operational changes.
The ATI also empowers elites to mobilize support for reforms by clearly defining what transparency looks like and setting specific benchmarks for success. According to John Adams, staff member at DFID and chair of the IATI Technical Secretariat, references to the ATI help champions of transparency reforms to persuade reticent staff of the merits of policy change.Footnote 66 Specifically, according to staff in charge of transparency reforms at USAID, the MCC, and GIZ, the ATI's detailed set of indicators reduces uncertainty on the part of agency leaders in terms of identifying precisely what policies and practices need to change to meet expectations set in international commitments and national law. Some donors' transparency strategies are, in fact, directly oriented around the standards in the ATI.Footnote 67 For example, the Millennium Challenge Corporation (ranked first in the 2014 ATI), states that “through the Aid Transparency Index process, Publish What You Fund and other advocacy groups have made specific recommendations to MCC in the interest of moving the field of aid transparency forward, particularly regarding how to prioritize improvements to IATI data.”Footnote 68 The ATI, in essence, became the MCC's “blueprint for reform.” This dynamic shows up in other interviews and in internal and published organizational strategy papers and operational policies at USAID and SIDA. Notably, we see it even when there continues to be disagreement within organizations on the importance or fit of the ATI's ideals and standards with the organization's overall transparency agenda and core values.Footnote 69
USAID's response to the ATI illustrates the central role that elite channels play. In July 2015, after struggling in prior years in the ATI rankings, USAID published a strategy paper on open data depicted as the agency's “roadmap” to transparency.Footnote 70 The strategy paper, also known as the IATI Implementation Cost Management Plan (CMP—Phase 2), explicitly stated that one of the strategy's four central goals was to increase USAID's ATI score.Footnote 71 A month later, Alex Thier, then-chief for the Bureau of Policy, Planning, and Learning at USAID, stated “after we implemented Phase One of the CMP, our Publish What You Fund Aid Transparency Index … increased more than twenty points and moved USAID from ‘Fair’ to ‘Good.’ This was an exciting, tangible way to demonstrate our progress, and this success raised awareness around the Agency on these important efforts.”Footnote 72 This internally driven, elite-led reform effort is consistent with the quantitative evidence we presented earlier regarding agency independence.
The ATI's Power via Political Pressure
While nearly 80 percent of our interviewees discussed direct elite responses to the ATI, approximately 65 percent of those who perceived the ATI as positive also noted that the ATI reduces information asymmetries and induces greater principal attention to donor organizations' disclosure practices. The ATI creates awareness and support among political principals for aid transparency via lobbying and advocacy campaigns of PWYF and like-minded members of the epistemic community. As such, the ATI is a useful tool for resolving information asymmetries that hinder principal oversight and control. In providing detailed, regular data on agencies' transparency performance, the ATI essentially acts as an information intermediary for politicians who may have neither the capacity nor inclination to closely monitor agent behavior.Footnote 73 According to one senior staff member in the US House Foreign Affairs Committee, the ATI is “great for letting us know when there's a problem [with US aid agencies] … We don't have time to follow that stuff that closely.”Footnote 74 We also found that principals use the ATI's detailed information on where donors lag in transparency performance to inform specific policies within national legislation on open data standards. National “aid transparency guarantees” in the UK and Sweden and the 2016 US Foreign Aid Transparency Act make specific recommendations that align with the indicators and goals of the ATI.
In other instances, it is quite apparent that the ATI is empowering third-party actors. In the US, for example, a thriving group of think tanks, academics, and NGOs pays close attention to the ATI.Footnote 75 The rankings and ratings inform their analytical reports, lobbying, and activism.Footnote 76 Interviews with senior congressional staff in the US House Foreign Affairs Committee and the US Senate Foreign Relations Committee reveal that a great deal of their information on the transparency performance of US aid agencies comes from this epistemic community, with frequent reference to the ATI as a primary source of evidence.
Some of the competitive pressure between agencies we described has links to political pressure. According to three off-the-record interviews with congressional staff, the US Senate has also brought attention and importance to the ATI. The MCC's success, and the need for other US agencies to “catch up,” has been prominently discussed in congressional hearings. For example, this was noted in the very first question asked to Dana Hyde in her 2013 confirmation hearing as MCC CEO, by Senator Markey (D-MA). As Markey put it, MCC's success on the ATI was “a very impressive record” and she asked “how do we keep it going, and how do we transfer that transparency to all of these other venerable institutions [e.g., USAID]?”Footnote 77 While only two interviewees admitted that the MCC's success was an official reason for the significant shift in USAID's approach to a more aggressive IATI implementation plan shortly thereafter, nearly all of the USAID, US State Department, and other US government interviews (20 out of 25) remarked that this competition had a lot to do with getting the attention of top USAID management and putting data transparency reforms “on the front burner.”Footnote 78
Consistent with our hypotheses, political pressure appears to play a stronger role with respect to less independent aid agencies. For example, close observers of the US system note that staff from USAID, which has relatively little financial autonomy vis-à-vis Congress, fear that the ATI's score may influence the way their political masters decide to appropriate funds.Footnote 79 By contrast, interviews on the evolution of the transparency initiative at the World Bank, whose funds come from a more diverse set of sources (including trust funds and profits from nonconcessional lending and bonds) make no mention of the shadow of appropriations, even when asked directly.Footnote 80 The World Bank has consistently placed in the top ten of all donors and in the ATI's “very good” category. Our interviews also reveal that staff in less independent agencies do not always wait for clear threats or actual principal exercise of oversight and control. Instead, they can act in anticipation of principals' possible greater oversight and control, exhibiting a desire to “get ahead of the game” by taking proactive steps to implement data reporting standards that would be fully compliant with IATI and aligned to the metrics in the ATI.Footnote 81
Conclusion: The ATI's Social Power and Influence
The qualitative and quantitative empirics jointly suggest that both political pressure and direct elite channels play a role in the ATI's influence. The elite channel appears to be the more influential of the two. The ATI provides information to political principals; it also provides information to elite bureaucrats inside donor agencies. These elites are the primary drivers of agency changes as the ATI has created a clear standard of what it means to be “transparent,” invoking normative and reputational power for these policy elites.
GPIs can enable weak actors to influence powerful actors in world politics. The ATI is a remarkable case in point. Created nearly ten years ago by a small NGO with no direct material power, the ATI now sets best practices in aid transparency. The ATI exercises influence via political pressure and elite channels to invoke important socialization, learning, and peer pressure effects that discernibly shape many donors' transparency policies. Agencies with aid as their primary mandate are more responsive to the ATI's assessments, particularly when those agencies are relatively independent and thus more able to act in response to a change in aid professionals' priorities and understandings. The ATI thus demonstrates that a GPI creator (in this case, Publish What You Fund) can substantially alter behavior within a relatively tightly knit professional community.
The case of the ATI suggests that there may be advantages to focusing on elite channels and construction of meaning in professional communities. At the same time, this case also suggests that a GPI that operates via social pressure will be unlikely to influence those who overtly reject the importance of the goal (such as China or the United Arab Emirates) or who are not part of the community in which social pressure operates. In this case of the ATI, organizations that are primarily focused on other matters and only incidentally disburse aid are not part of the professional community. The ATI's efforts to affect change in these agencies may be of limited usefulness.
In sum, the ATI has drawn critical attention to donors' aid transparency commitments and performance. It is now an industry leader in assessing aid transparency and setting donor accountability standards. The ATI alters what transparency means in practice, even as it encourages greater disclosure. It does more than merely assess transparency practices: it defines norms of transparency.
At the same time, the ATI's influence is not always viewed in a positive light, even when it pushes donor agencies toward transparency. For example, when asked if the ATI presented any concerns or risks, five interviewees in the US (in donor agencies and NGOs) pointed out that the systems put in place in response to the ATI did not always represent the quickest or most efficient route to full transparency for their organization.Footnote 82 It is interesting that conformity around the ATI's transparency norms has taken hold, despite emerging reservations about the appropriateness of its assessment criteria and specific indicators. This qualitative observation may portend a varying level of normative power over time as the ATI ages. Contestation over the operationalization of “transparency” may lead to some discrediting or distancing from the ATI. It might also create pressure to alter the index to include other indicators and weights, or perhaps even enable the rise of GPI competitors that reflect changing norms regarding the structure and goals of aid transparency.Footnote 83
The ATI has been a key part of the transparency revolution in international development aid. PWYF's indicators have helped to set a clear standard for the over 600 governmental and nongovernmental signatories of the International Aid Transparency Initiative. For an NGO that still occupies just one room, it shows that GPI creators do not necessarily need material power to influence the behavior of the powerful. David may move Goliath with a well-aimed slingshot fired in conflict. But David can also induce Goliath to change by orienting Goliath's attention toward the behavior David wishes to alter.
Supplementary Material
Supplementary material for this article is available at <https://doi.org/10.1017/S0020818319000122>.
Acknowledgments
Many thanks to Rucheta Singh, Grace Chao, Susannah Horton, and particularly Yunhui Lin for research assistance. We are also thankful to Publish What You Fund for their time and clarification; AidData, and particularly Brooke Russell and Brad Parks, for assistance in generating critical additional data; Dennis Vega and Jessica Klein of the US Department of State's Office of US Foreign Assistance Resources; Sally Paxton at Friends of Publish What You Fund; and numerous officials from aid agencies, parliaments, think tanks, and NGOs who graciously agreed to interviews. Many thanks to Sam Asher, James Bisbee, Stephen Chaudoin, Laura Henry, George Ingram, Judith Kelley, Eddy Malesky, John Marshall, Beth Simmons, the IO reviewers and editors, and participants in the May 2016 and September 2016 APSA Assessment Power in World Politics conferences as well as the 2017 Political Economy of International Organizations conference for valuable comments and guidance. Last but far from least, many thanks to the editorial team and professional staff at International Organization for their time and efforts, and to anonymous reviewers for their helpful comments in improving this manuscript.
