Policy diffusion and diffusion mechanisms

Policy diffusion can be defined as the process whereby policy choices in one unit are influenced by policy choices in other units (Braun and Gilardi Reference Braun and Gilardi2006; Simmons et al. Reference Simmons, Dobbin and Garrett2006; Gilardi Reference Gilardi2012; Graham et al. Reference Graham, Shipan and Volden2013). This definition is very broad and is based on an even more general definition put forward by Strang (Reference Strang1991, 325) in sociology: “[diffusion refers to] any process where prior adoption of a trait or practice in a population alters the probability of adoption for the remaining non-adopters”. This definition is applicable to many different types of units (countries, subnational states, cities, public organisations, firms, etc.) and to institutions and many types of (political) behaviour, in addition to policies. However, the definition is also precise because it emphasises that diffusion is characterised by interdependence. Thus, there is a clear distinction between diffusion and convergence, which represents the policies of different units becoming more similar over time (Bennett Reference Bennett1991; Knill Reference Knill2005). Although convergence can be caused by interdependence, it can also result from units reacting to similar, independent pressures, like people opening umbrellas when it starts to rain. By contrast, interdependence is the key defining component of diffusion.

A related concept is that of policy transfer, the “process by which knowledge about policies, administrative arrangements, institutions and ideas in one political system (past or present) is used in the development of policies, administrative arrangements, institutions and ideas in another political system” (Dolowitz and Marsh Reference Dolowitz and Marsh2000, 5). There is clearly a significant overlap between this definition and that of policy diffusion. Marsh and Sharman (Reference Marsh and Sharman2009, 271) argue that “these literatures share an overlapping conceptual core and a complementary interest in a related class of empirical phenomena”, whereas Graham et al. (Reference Graham, Shipan and Volden2013, 679) think that policy transfer studies can be considered a subset of the general policy diffusion literature. Following these authors, we refrain from drawing a sharp distinction between transfer and diffusion, and we regard the former as a special case of the latter.

There is consensus that diffusion is a product of interdependence. However, interdependence can take many different forms. The literature usually refers to these as “mechanisms”. The exact terminologies vary, but diffusion mechanisms can be grouped into three broad categories: learning, emulation and competition. Some scholars would add coercion to this list, but we disagree. Coercion means that a given unit adopts policy following pressure from powerful countries or international organisations (Gilardi Reference Gilardi2012, 461). EU and IMF conditionality are cases in point. Although coercion can certainly influence policy adoption, diffusion implies that no central actors are coordinating the spread of a policy. For this reason, although we recognise that some authors would consider that coercion is in fact a diffusion mechanism, we choose not to include it in our analysis. In any case, our specific analyses are unaffected by this choice because we examine each mechanism separately.

The first mechanism that we consider is learning, which is defined as a process where policies in one unit are influenced by the consequences of similar policies in other units. In other words, policy adoption in one unit is more likely if the policy has been successful elsewhere (Meseguer Reference Meseguer2004; Braun and Gilardi Reference Braun and Gilardi2006; Volden Reference Volden2006; Gilardi Reference Gilardi2010; Jensen and Lindstädt Reference Jensen and Lindstädt2012). There are different forms of success. Success can be related to (a) the goals that the policy is designed to achieve, (b) the challenges of its implementation and/or (c) its political support. When considering the adoption of a policy, policymakers can learn from others about each of these dimensions. For instance, Volden (Reference Volden2006) showed that the United States was more likely to imitate health insurance programmes targeting needy children that managed to increase insurance rates while keeping costs low, whereas Gilardi (Reference Gilardi2010) found that, under some circumstances, policymakers were more likely to imitate a policy if it had been shown to enhance the re-election outcomes of those who enacted it.

In contrast to learning, emulation is not related to the objective consequences of a policy. Instead, the symbolic and socially constructed characteristics of policies are crucial (Cao Reference Cao2009; Greenhill Reference Greenhill2010; Krook and True Reference Krook and True2010; Frenandez and Lutter Reference Fernandez and Lutter2013). Inspired by sociological institutionalism, the conceptualisation of this mechanism implies that units have to conform to their normative environment. Thus, some policies will enjoy high acceptance, regardless of whether or not they “work”. By contrast, others will be taboo, even though they could possibly be beneficial. Another way to see this mechanism is that the “burden of proof” changes over time as a function of social acceptance. When considering a radical policy innovation, the burden of proof rests on its advocates; however, when it becomes widely accepted, it is the opponents of the policy who have to make a compelling case to prevent its adoption. For example, Greenhill (Reference Greenhill2010) argues that international governmental organisations enhance the spread of human rights by fostering the development of norms through the socialisation of their members. In this view, the material consequences of respecting human rights carry less weight than the pressure to conform to a norm within a given peer group.

Finally, competition occurs when units react to one another in an attempt to attract or retain resources. Tax competition is the prototypical example (Basinger and Hallerberg Reference Basinger and Hallerberg2004; Cao Reference Cao2010; Genschel and Schwarz Reference Genschel and Schwarz2011), although competitive dynamics can also be found in many other areas of economic policy, such as capital account and exchange rate policies (Simmons and Elkins Reference Simmons and Elkins2004), bilateral investment treaties (Elkins et al. Reference Elkins, Guzman and Simmons2006) and market-oriented infrastructure reforms (Henisz et al. Reference Henisz, Zelner and Guillén2005). For instance, Simmons and Elkins (Reference Simmons and Elkins2004) found that a country is more likely to liberalise its international economic policies following similar reforms among its competitors, defined as countries with which it shares similar trade relationships.

Conceptual structure and measurement validity

We now turn to a more systematic discussion of the concept of policy diffusion. We rely on the approach put forward by Goertz (Reference Goertz2006), which conceives of concepts as “theories of the ontology of the phenomenon under consideration” (Goertz Reference Goertz2006, 27) – that is, concepts tell us what something is, not just what it looks like. Goertz (Reference Goertz2006) argues that the conceptual structure consists of three levels: basic, secondary and indicator. The structure of the policy diffusion concept is shown in Figure 1. The basic level is the concept as used in theoretical statements – in our case, “policy diffusion”. Here, it is helpful to distinguish between the “background” and “systematised” concept (Adcock and Collier Reference Adcock and Collier2001). The background concept of policy diffusion is the general idea that policies spread, whereas the systematised concept corresponds to the definitions made by Strang (Reference Strang1991) and Simmons et al. (Reference Simmons, Dobbin and Garrett2006) and discussed in the “Policy diffusion and diffusion mechanisms” section, which understand diffusion as a consequence of interdependence. In Figure 1, the basic level corresponds to the systematised concept. Next, the secondary level provides the constitutive dimensions of the basic level policy diffusion concept. They are the three diffusion mechanisms, connected to the basic concept with the === sign and OR operator to denote the ontological relationship, that is, policy diffusion exists when at least one of the three mechanisms is present. Finally, the indicator level is where the secondary level is operationalised, such that we can determine whether a specific instance belongs to the concept or not. It follows straightforwardly from the definitions in the “Policy diffusion and diffusion mechanisms” section that learning means being influenced by successful policies; emulation means copying “appropriate” policies; and competition means following the policies of competitors.

Figure 1 Structure of the policy diffusion concept, based on Goertz (Reference Goertz2006, 27–67).

An important question is whether measures derived from the indicator-level concept are valid. According to Adcock and Collier (Reference Adcock and Collier2001, 530), “valid measurement is achieved when scores […] meaningfully capture the ideas contained in the corresponding concept”. A minimal measurement validation can be established by answering two questions (Adcock and Collier Reference Adcock and Collier2001, 538): “First, are key elements omitted from the indicator? Second, are inappropriate elements included in the indicator?”. It follows straightforwardly from our discussion that a valid measure of learning must include information on the success (or lack thereof) of a policy in other units; that a valid measure of emulation must indicate how adoption of a policy by other units affects its appropriateness; and that a valid measure of competition must identify the policies of other units competing with one another.

In our meta-analysis, we focus on six indicators, identified inductively from their usage in the literature. They belong to the indicator level in Figure 1. Some are relational, whereas others consider the status (presence, success) of policies in other countries.

Methodology

Our analysis considers 114 articles published between 1990 and 2012. Our sampling procedure is explained in Appendix A1, and the selected articles are listed in Appendix A7. The units of analysis, however, are not the articles but the diffusion mechanisms that they study. As an article can investigate more than one mechanism, the number of observations in the analysis (152) is greater than the number of articles. We coded each mechanism binarily based on the conceptualisation discussed in the “Policy diffusion and diffusion mechanisms” and “Conceptual structure and measurement validity” sections, which often corresponded to the labels used by the authors, but sometimes did not. For learning, we also considered “lesson-drawing”; for emulation, we also considered “norms”, “isomorphism”, “imitation”, “mimicry” and “socialisation”; and for competition, we also considered “race-to-the-bottom” and “California effect”. Our coding rules are described in Appendix A2. The next step was to code how the mechanisms were operationalised based on the indicators discussed in the “Conceptual structure and measurement validity” section. Moreover, we coded whether the study focused on cities/towns, regions, federal states or countries; whether it focused on an economic policy or other types of policies; and whether the analysis was quantitative or qualitative. Coding was executed by a single research assistant, and a sample of coded mechanisms was checked to test for intercoder reliability, which proved to be satisfactory as explained in Appendix A3.

In the first step, we analyse these data first with a simple cross-tabulation; however, then we move to fsQCA,Footnote ¹ which allows us to carry out a more systematic meta-analysis. Meta-analyses are essential for taking stock of previous research, producing cumulative knowledge and developing a progressive research programme (Lipsey and Wilson Reference Lipsey and Wilson2000; Cooper et al. Reference Cooper, Hedges and Valentine2009a, Reference Cooper, Patall and Lindsay2009b; Borenstein et al. Reference Borenstein, Hedges, Higgins and Rothstein2011). Strictly speaking, our approach is in between classic meta-analyses and systematic reviews (Exadaktylos and Radaelli Reference Exadaktylos and Radaelli2012). That is, we do not aim to combine the results of each study in a single measure of effect size or another statistical measure, as in classic meta-analyses. Rather, we aim to extract patterns from a sample of studies to understand the logic behind methodological choices and to assess their coherence regarding the conceptualisation and operationalisation of diffusion mechanisms.Footnote ² QCA is particularly suitable for this type of meta-analysis because it identifies parsimonious regularities (Sager Reference Sager2006; Dunlop et al. Reference Dunlop, Maggetti, Radaelli and Russel2012). QCA is not limited to hypothesis testing, but rather can serve several purposes (Rihoux and Lobe Reference Rihoux and Lobe2009). It can be used to describe cases in a synthetic way, to check data coherence and gain knowledge about individual cases, to explore data and develop new insights and to elaborate new theories. Although the set-theoretic relations uncovered with QCA are often interpreted in causal terms, this is by no means the only possible application of this approach. In this article, we use QCA as a heuristic tool for exploring, mapping and finding systematic patterns in the way diffusion mechanisms are operationalised.

The first step in fsQCA is defining the conditions and outcome and arranging them in a “truth table”, that is, a data matrix summarising all the combinations of conditions.Footnote ³ Cases are seen as configurations of conditions (i.e. independent variables, according to QCA terminology). These configurations can be minimised to the shortest possible logical expression leading to the outcome of interest – in our case, one of the three diffusion mechanisms.Footnote ⁴ This expression is the QCA solution.Footnote ⁵ Its reliability and validity are assessed by two criteria – consistency and coverage (Ragin Reference Ragin2006). Consistency represents the degree to which cases sharing a given combination of conditions agree in leading to a given outcome, that is, how closely the subset relationship is approximated, whereas coverage measures the proportion of cases following a specific path, that is, the empirical relevance of a consistent subset.Footnote ⁶

We carry out a fsQCA for each of our three diffusion mechanisms (learning, emulation and competition). In each analysis, the conditions refer to the six elements of operationalisation and three elements of design discussed earlier. All conditions are coded binarily except “high level of aggregation”, which is calibrated using a four-point scale: country level (1), state level (0.67), regional (0.33) and local (0).

Findings

Table 1 presents a cross-tabulation of each conceptualisation and operationalisation of diffusion mechanisms. As discussed in “Conceptual structure and measurement validity” section, geographic proximity is a catch-all indicator that cannot be linked unambiguously with a specific mechanism; joint membership ignores the nature of actual interactions, but is often used as an indicator for shared norms or information flows among members; policy success can be linked directly to learning; structural equivalence could fit with both competition and emulation depending on the context; the number of previous adopters is often employed in connection with normative pressures, although it does not identify norms directly; and, finally, trade flows are linked with competitive relationships, but they can also capture more general connections between countries.

Table 1 Conceptualisation and operationalisation of diffusion mechanisms

On the one hand, we can observe that the concept of learning is mainly operationalised with a measure of success, with 18 occurrences. However, proximity is also used in 10 cases, interaction in seven cases and similarity in six to make sense of interdependent decision-making oriented towards learning. Emulation is almost equally operationalised with interaction (14 occurrences), similarity (12 occurrences) and proximity (11 occurrences). Finally, competition is mainly operationalised in terms of the similarity of entities (13 occurrences), and trade is the second most frequent way of operationalising this diffusion mechanism (five occurrences). It is worth noting that, in 31 cases (not reported in the table), no mechanism is explicitly conceptualised. On the other hand, if we look at Table 1 horizontally, we can observe that similarity, with 31 occurrences, is the most frequent operationalisation of interdependence in absolute numbers. Proximity and interaction, with 24 and 22 occurrences, respectively, are mainly used to operationalise the concepts of learning and emulation. These simple, descriptive data already allow us to note a high degree of heterogeneity in empirical analyses of diffusion mechanisms.

The cross-tabulation shows an overall lack of consistency in empirical diffusion research. We now turn to a more detailed analysis to uncover systematic patterns in the measurement of diffusion mechanisms. It could be that there is more coherence than Table 1 suggests, or that the heterogeneity is systematically skewed in some direction. QCA is useful for this purpose, because it is well suited “to bring to light similarities between cases that may, at first sight, seem quite different” (Rihoux and Ragin Reference Rihoux and Ragin2008, 15). Table 2 shows the results of the fsQCA; the corresponding truth tables are available in Appendix A5. Technical details on the analysis are discussed in Appendix A4.

Table 2 Fuzzy-set meta-analysis of three diffusion mechanisms

●=core condition (present); ○=core condition (absent); ●=peripheral condition (present); ○=peripheral condition (absent).

The analysis distinguishes between core and peripheral conditions. Core conditions are consistent components of both the parsimonious and the intermediate solution, whereas peripheral conditions are present only in the intermediate solution (Fiss 2007). In other words, the former are essential elements of a solution, whereas the latter are less directly related to the outcome. Table 2 uses the notation introduced by Ragin and Fiss (Reference Ragin and Fiss2008), wherein black circles indicate the presence of a condition and white circles indicate its absence. Large circles indicate core conditions, and small circles refer to peripheral conditions. Whereas traditional presentations of QCA results focus on logical expressions considered as a whole, this notation highlights individual conditions while allowing a synthetic overview of all solutions. The analysis produced six combinations of conditions for learning, six for emulation and four for competition. Each combination is a logical expression, including the conditions that, taken together, lead to the outcome. The combinations are equifinal. In other words, they represent different paths leading to the same outcome.

The analysis confirms the considerable heterogeneity shown in Table 1. There is no systematic pattern in the measurement of policy diffusion mechanisms. First, learning is operationalised in many different ways. The most relevant paths – namely, those with higher coverage – indicate that “success” is a core element (cf. combinations 2 and 3), in line with our previous cross-tabulation. However, the fsQCA reveals additional details. Learning tends to be studied at the level of countries or federal states, either with a focus on economic policies or with reliance on quantitative methods. An example is a study by Elkins et al. (Reference Elkins, Guzman and Simmons2006), which examines the cross-national diffusion of bilateral investment treaties and operationalised learning with the treaties’ demonstrated benefits. This approach is consistent with the conceptual discussion in the “Conceptual structure and measurement validity” section. However, learning is also operationalised in terms of critical mass in studies concerning local and individual actors (solution 1) or with the use of some measure of proximity (solution 4). Finally, in qualitative studies of policy diffusion within states and countries, learning is operationalised with indicators of similarity among the observed entities. Although “success” is very close to the conceptual core of policy learning, and “proximity” could be considered as a decent proxy of it in some contexts, “critical mass” and “similarity” appear to be second best solutions that do not operationalise the concept accurately. The observation that any given entity adopts a policy that was already adopted by similar entities does not provide adequate support for policy learning.

Second, the study of emulation is even more complex than that of learning. We decided to lower the consistency threshold to 0.65 to find a solution that also ensures empirical relevance in terms of coverage, which means that it is difficult to obtain a synthetic picture about the operationalisation of emulation even when accounting for a high level of heterogeneity.Footnote ⁷ Combinations of conditions are not only extremely diverse but also quite uncertain. Those including quantitative analyses at a high level of aggregation were the most frequent. They use critical mass or interaction (e.g. co-membership in a transnational network) as alternative core conditions for operationalising emulation (solutions 7 and 8). For instance, Jordana and Levi-Faur (Reference Jordana and Levi-Faur2005) interpreted their findings about the effects of prior decisions on the establishment of regulatory authorities in Latin America as evidence of emulation, whereas Cao (Reference Cao2009) operationalised emulation as the mechanism driving convergence of domestic economic policies through co-memberships in intergovernmental organisations with social and cultural functions. The analysis of emulation also conflates different dimensions, such as in paths 9 and 10 where success is used to indicate emulation in combination with interaction and with similarity, respectively. The other two paths also combine indicators of similarity and proximity as core conditions (11 and 12). Therefore, for emulation, all possible operationalisations of interdependence have been used with the exception of “trade”. These findings indicate that the same conceptual space is filled with indicators of very different types of interdependence. The distance between the concept of emulation and its operationalisation seems particularly vast. This conceptual overstretching is problematic because, on the one hand, it generates confusion about the validity of the empirical findings, and on the other, it reduces the analytical leverage of emulation-based diffusion theories. As Seneca wrote in the Epistulae Morales, II, 2, nusquam est qui ubique est (everywhere means nowhere).

Third, for competition, the overall level of coverage was quite low. This means that a consistent solution exists only for a relatively small number of cases. Trade is a dominant core condition present in three out of four combinations (13, 15 and 16). Quantitative methods are always present as peripheral conditions. The last two paths, which are also those with the highest coverage, are very similar and include economic policies as a core element of design, together with a high level of aggregation as another peripheral condition. An example is Linos’ (Reference Linos2011) study of the diffusion of family policies, whereby competition is operationalised as policy adoption in neighbouring countries weighted by similar types of exported products. However, some studies measure competition in a cruder way, simply with similarity (solution 14).

Table 2 also illustrates the mirror image of equifinality, namely, observational equivalence. The same indicator is frequently used to operationalise different mechanisms. This problem is particularly acute with “similarity”, which constitutes a core condition in combinations leading to each of the outcomes. This is a serious shortcoming: the same type of interdependence can be interpreted as evidence of different mechanisms of policy diffusion. Given this state of affairs, the risk of making contradictory inferences from the same data is quite high.

It is important to consider the possibility that the coherence of diffusion studies changed over time. In particular, it could be that, in the early days of policy diffusion research, mechanisms were conceptualised and operationalised less systematically, but a higher degree of consensus was achieved over time, such that a greater degree of coherence can be found in the field at present. However, as shown in Appendix A6, when we compare our model with additional analyses based on two subsamples for articles published until 2008 (84 cases) and since then (68 cases) the findings remain qualitatively unchanged. The same holds when comparing the two subsamples with each other. There is considerable heterogeneity within and across mechanisms. In both periods, mechanisms are operationalised in many different, sometimes contradictory ways. What is more, many individual solutions exist for the same outcomes in both periods. Nevertheless, a certain dynamic is perceptible. When moving from the earlier to the later period, the overall number of paths decreases from 17 to 12. Interestingly, this change is due in large part to a reduction in the number of consistent operationalisations of learning (from seven to four) and competition (from five to two). On the contrary, the number of different operationalisations of emulation increases from five to six. Therefore, it seems that the study of learning and competition has become somewhat more coherent, although significant heterogeneity clearly persists. A similar improvement is not apparent for emulation, where richness and pluralism come at the expense of conceptual clarity and measurement coherence.

Recommendations

We do not want to impose a single way of doing things. Sometimes there are very good reasons for disagreement within the social sciences. For instance, different epistemological positions produce a variety of empirical applications that range from ideographic to nomothetic explanations. This is very beneficial for the pluralism and richness of the social sciences. However, there are also less valid reasons for disagreeing. To some extent, this is the case in the empirical study of policy diffusion mechanisms. Indeed, the results of our meta-analysis indicate that the accumulation of knowledge in the study of policy diffusion is potentially problematic. Existing measures of diffusion mechanisms are unstable, unspecified and occasionally overlapping. The operationalisations of these mechanisms are quite incoherent across studies in the sense that the same concept is operationalised in many different ways, and even contradictorily. Likewise, the same operationalisation is frequently applied to make sense of diverse underlying concepts. These problems are particularly pressing in the study of emulation and, to a lesser extent, for learning and competition as diffusion mechanisms. This state of affairs may produce confusing interpretations and endanger scientific cumulativeness.

Our analysis underscores three main issues: conceptual clarity, the connection between concepts and measurement and the importance of research design. These points are quite general, but our analysis shows that they should be considered more carefully in future research.

First, researchers should pay great attention to conceptual clarity, while resisting the temptation to reinvent the wheel. We fully concur with Graham et al. (Reference Graham, Shipan and Volden2013, 700):

One of the main causes of the heterogeneity we uncovered is undoubtedly the fact that the literature still uses different labels for the same idea and the same label for different ideas. As long as this problem persists, it is unlikely that empirical analyses will become more consistent with one another, thus preventing the accumulation of knowledge. This does not mean that there is no room for conceptual innovation, nor that arguments cannot be adapted to a specific context. In fact, researchers should be strongly encouraged to do these things. However, it is paramount that conceptual innovations and refinements are situated within established categories, and that new jargon is only introduced in exceptional cases. In most instances this will be unnecessary. One can very well add further nuances to a mechanism while adhering to consolidated labels. Specifically, we suggest that the conceptual structure shown in Figure 1 is a useful template for studies applying a diffusion argument but whose purpose is not to refine diffusion theory, and should be the starting point for studies that do want to make theoretical improvements. In this context, it is important that researchers make it clear whether their goal is to make a contribution to the diffusion literature itself or to use the insights of diffusion research to learn something new about other phenomena (Gilardi Reference Gilardi2015).

Second, researchers should take all precautions to avoid the existing mismatch between concepts and measurement (Adcock and Collier Reference Adcock and Collier2001). Again, this is a necessary step to overcome the inconsistencies that are pervasive in the literature, as we have shown. This problem may be because of the tendency to use imperfect proxies – indicators that do not measure the underlying concept accurately – in the absence of more appropriate empirical data. An example is the use of indicators of similarity to operationalise policy learning without any measures that are more closely related to the properties of this mechanism – an evaluation of policy success for example. To avoid these problems, we recommend that scholars reflect explicitly and systematically upon the relationship between conceptualisation and operationalisation. The conceptual scheme in Figure 1 can help here, too. More specifically, we advise against including many poorly measured diffusion mechanisms in a single study. Constructing a convincing measure of a single mechanism is difficult. The priority should be to construct a good measure for the main mechanism under consideration, rather than including as many mechanisms as possible.

Third, more creativity is needed when constructing research designs. The marginal payoff of standard research designs, especially quantitative designs, is decreasing sharply. Cross-national time-series cross-section analyses, where the main explanatory variable is a spatial lag constructed using some measure of proximity or similarity, can reliably establish the presence of interdependence, but they are quite blunt when it comes to identifying specific mechanisms. Therefore, we encourage researchers to develop original research designs that allow them to investigate a given mechanism more accurately and reliably. Often, this means that a comprehensive analysis of several diffusion mechanisms will not be possible. However, we argue that a cleaner analysis of a single mechanism is preferable to a messier analysis of several. In other words, an innovative research design is often required to achieve good measurement. If no good indicator for a given mechanism is available using conventional research designs, as is often the case, an improvement of the overall research design is in order. For example, if one wants to study learning but no success measures are available or can be constructed, our advice would be to rethink the research design itself (for instance, by shifting the analysis from the cross-national to the subnational level) instead of settling for unsatisfactory measures that will give insufficient leverage.

In sum, our analysis has shown significant inconsistencies in empirical analyses of diffusion. To overcome them, we have put forward a conceptual scheme that scholars can use to apply a diffusion argument or to sharpen specific theoretical arguments. We also recommend that scholars prioritise the quality of measurement rather than the comprehensiveness of the analysis. In many cases, this will require the construction of original, innovative research designs instead of the replication of widely used templates.