Introduction
Norms lie at the center of an immense collection of research in international relations and comparative politics over the past thirty years. Commonly defined as “standard[s] of appropriate behavior for actors with a given identity” (Finnemore and Sikkink Reference Finnemore and Sikkink1998, 891), norms structure the behavior of actors by providing a “logic of appropriateness” rather than a “logic of consequences” (March and Olsen Reference March and Olsen1998). While early work developed insightful models on norm emergence and diffusion, such as the “norm life cycle” (Finnemore and Sikkink Reference Finnemore and Sikkink1998), “boomerang” (Keck and Sikkink Reference Keck and Sikkink1998) and “spiral” (Risse, Ropp, and Sikkink Reference Risse, Ropp and Sikkink1999) models, subsequent research greatly expanded our understanding of norms by problematizing the static meaning of norms, their assumed linear progression, and the “liberal bias” in the study of Western liberal norms (Bloomfield Reference Bloomfield2016). Despite the substantial theoretical development of norms research, comparatively little attention is devoted to how we measure international norms (Finnemore and Sikkink Reference Finnemore and Sikkink1998; Goertz and Diehl Reference Goertz and Diehl1992; Simmons and Jo Reference Simmons and Jo2019).
Two approaches to measuring norm adoption are common in existing research. First, a large subset of the literature uses case-study research designs, where the units of analysis are often individual countries. This approach allows researchers to carefully evaluate the relationship between a range of observable behaviors and the overarching norm with which they correspond. Second, scholars using statistical methods (for example, when studying norm diffusion) frequently operationalize norm adoption by using a single policy, law, or institution. Consequently, we observe a trade-off between nuance and scope in how we measure norms, resulting in a disconnect between measurement and theory.
In this article, I argue that item-response theory (IRT; also known as latent response theory) provides a framework for improving the connection between our empirical and theoretical study of norm adoption. These types of models are widely used to measure relationships between latent attributes and observed outcomes. For instance, such models are used to position individuals on ideological continuums (the latent attribute) through their responses to a set of survey questions (the observed outcomes). I argue that these models can be similarly applied to the adoption of international norms, with the degree of norm adoption (the latent attribute) being revealed through the adoption of a (theoretically informed) set of policies/laws (the observed indicators). Further, we can evaluate how policies and laws change over time in their capacity to distinguish the degree of norm adoption, as the standard of behavior associated with the norm shifts. I build on previous methodological work (Fariss Reference Fariss2014; Reference Fariss2018; Martin and Quinn Reference Martin and Quinn2002) in two important ways. To generate insight on the range of behaviors that constitute the norm, and how these behaviors (and standards) change over time, I estimate the difficulty and discrimination parameters for each observed indicator in each year (rather than a single estimate covering all years). Second, I link the specification of the dynamic latent prior for each country to domestic regime change, which operationalizes the anticipated temporal patterns of change and stability.
To demonstrate the proposed approach, I assess the lesbian, gay, and bisexual (LGB) equality norm.Footnote 1 I use a set of 13 domestic policies/laws for 196 countries (1990–2017) and generate estimates of each country’s latent position (degree of norm adoption) in each year. The results illuminate the changing substance of the norm and provide evidence of a gradual shift from norm polarization towards global norm adoption. Given the continued importance of work on international norms, this approach advances a large body of scholarship by improving the measurement of a fundamental concept and illuminating new areas of research.
Integrating Item-Response Theory and International Norms Research
Item-response theory is well suited to the task of measuring the adoption of international norms. Although we cannot directly observe the position of a subject along a latent dimension, IRT models provide a framework for assessing how a subject’s position is related to the observed responses among a set of indicators.Footnote 2 Our understanding of norm adoption naturally operates in a similar fashion; the degree of norm adoption varies and cannot be directly observed, yet we can observe the behaviors associated with the norm. To be clear, we may be interested in a specific behavior, but the study of norms often requires us to consider several observable implications of the norm.
I focus specifically on norm adoption by states to assess the plausibility of the strategy. The state is the “natural referent” and relevant target for many types of international norms (Cloward Reference Cloward2016, 10). Further, norm adoption by the state (i.e., institutionalization) is productively distinguished from other aspects, such as the effectiveness of implementation or broad attitudinal shifts. While each are important, there is theoretical value in considering the components separately to facilitate the study of the relationship between them.Footnote 3
The application of IRT models to norm adoption requires the identification of indicators (akin to survey questions in the ideology example). Using international legal agreements, resolutions, and treaties related to a single norm offers one potential avenue, but such an approach is of limited use and generalizability. International institutionalization is not a necessary condition of international norms (Finnemore and Sikkink Reference Finnemore and Sikkink1998, 900; see also Clapp and Swanston Reference Clapp and Swanston2009; Orchard Reference Orchard2016). Such indicators may also be of limited use in assessing norm adoption over time, as a one-off vote on issue x in year t provides no insight on the norm in other years. As noted by Simmons and Jo (Reference Simmons and Jo2019), the literature has converged towards measuring norms through a combination of state practices or behavior and social responses to violations of the norm. Moreover, the distinction between domestic and international norms is often blurred in practice, as the creation of either domestic or international norms may precede and shape the other (Clapp and Swanston Reference Clapp and Swanston2009; Gest et al. Reference Gest, Armstrong, Carolan, Fox, Holzer, McLellan, Mogan and Talib2013).
Using a theoretically informed set of state policies/laws thus offers a useful way to evaluate norm adoption over time, as states can adopt or repeal policies/laws in any year. Such an approach also allows for the substance of the norm to shift, as one manifestation of norm evolution is found in the adoption of new state behaviors. As illustrated in Figure 1, the distribution of countries along the latent dimension (representing degree of norm adoption) provides insight into dynamic models of norm evolution. Beyond norm “rejection” or “adoption,” we may observe shifts towards “polarization” (division into two distinct groups of actors) or “rejection/backlash” (distancing from the norm, as observed through a shift over time from adoption towards rejection) (Nuñez-Mietz and García Iommi Reference Nuñez-Mietz and Iommi2017; Symons and Altman Reference Symons and Altman2015).
Figure 1. Stylized Distributions of Latent Estimates
Note: Each panel represents the stylized distribution of latent scores across countries in a single year, corresponding with potential theoretical expectations of norm evolution.
The IRT model developed here builds on previous scholarship using Bayesian dynamic models to measure changing standards of accountability in human rights practices (Fariss Reference Fariss2014). Simpler additive approaches model the latent dimension (norm adoption) as the sum of policies/laws in a given year, assuming equally weighted indicators and no error (Fariss Reference Fariss2018, 250). Instead, dynamic Bayesian approaches easily allow us to investigate indicator weights, temporal dependence in latent estimates, and measurement uncertainty. In other words, this approach allows for policies/laws that are not equally “difficult” for countries to adopt while also recognizing that our estimate of the degree of norm adoption is uncertain and conditioned by our beliefs about a country’s norm adoption in previous years. Importantly, the policies/laws are expected to be related to each other through their shared relationship with the latent dimension (norm adoption), but they are otherwise assumed to be independent.
The data thus consists of country-year observations where i = 1, ..., N indexes countries, t = 1, ..., T indexes year, and j = 1, ... J indexes policies/laws. The probability distribution, where
$ F\left(\cdot \right) $
identifies the logistic cumulative distribution function, is
$$ P\left[{y}_{it j}=1\right]=F\left({\alpha}_{jt}+{\beta}_{jt}{\theta}_{it}\right). $$
For ease of interpretation,
$ \alpha $
represents the “difficulty” of a positive response for a policy/law given a country’s position on the latent dimension;
$ \beta $
represents the ability of a policy/law to discriminate between different positions on the latent dimension; and
$ \theta $
represents the latent dimension (degree of norm adoption). The likelihood function is
$$ L\left(\alpha, \beta, \theta |y\right)=\prod \limits_{i=1}^N\prod \limits_{t=1}^T\prod \limits_{j=1}^J\left[F{\left({\alpha}_{jt}+{\beta}_{jt}{\theta}_{it}\right)}^{y_{it j}}\ast {\left(1-F\left({\alpha}_{jt}+{\beta}_{jt}{\theta}_{it}\right)\right)}^{\left(1-{y}_{it j}\right)}\right]. $$
Following previous work (Fariss Reference Fariss2014), the prior for each difficulty parameter is modeled as
$ {\alpha}_{jt}\sim N\left(0,1\right) $
and for each discrimination parameter as
$ {\beta}_{jt}\sim Gamma\left(4,3\right) $
.Footnote
4 But this model differs from others in two important ways. First, the
$ \alpha $
and
$ \beta $
parameters are estimated in each year, which allows the difficulty of the policies/laws to change over time. Second, the dynamic prior for the latent estimate is tied to domestic regime changes. Using data from the Archigos project (Goemans, Gleditsch, and Chiozza Reference Goemans, Gleditsch and Chiozza2009) to identify country-years with political leadership changes, I specify priors of
$ {\theta}_{i1}\sim N\left(0,{\tau}_1\right) $
for the first year in each country’s regime and for years t = {2, ..., T} it is
$ {\theta}_{it}\sim N\left({0}_{i,t-1},{\tau}_2\right) $
. The prior for each variance is
$ \tau \sim Gamma\left(1,.1\right) $
. These changes improve the connection between the model specification and our theoretical understanding of norms. We expect some state behaviors to appear less difficult as the norm is more widely adopted. It is also possible for countries to rapidly change position on the latent dimension, notwithstanding their dependence on previous years. Further, linking the dynamic latent prior to domestic regime change operationalizes the expected temporal patterns of change and stability that correspond to the political dynamics of regime change. This imposes a structure that is between fully unstructured independent priors and fully smoothed dynamic priors.Footnote
5
Lesbian, Gay, and Bisexual Equality Norm
Data
To evaluate the LGB norm, I constructed a dataset of 13 indicators using reports from the International Lesbian, Gay, Bisexual, Trans, and Intersex Association (ILGA). This organization is a leading global advocate for LGBTI rights and has been a key actor in the promotion of the LGB equality norm (Kollman Reference Kollman2009). The resulting dataset includes 13 laws and policies across 196 countries between 1990–2017Footnote 6, based primarily on information from the 2017 State Sponsored Homophobia Report (Carroll and Mendos 2017).Footnote 7 While the report does include more than 13 possible indicators, it was necessary to exclude some items that could not be consistently evaluated over time.Footnote 8 The 13 items included in the dataset are: homosexuality is legal, equal age of consent for heterosexual and homosexual activity, existence of promotion (propaganda) or morality laws (reverse coded), restrictions on LGB NGOs (reverse coded), employment protection, hate crime legislation concerning sexual orientation, legislation concerning incitement to hatred based on sexual orientation, partnership recognition for same-sex couples, marriage recognition for same-sex couples, joint adoption by same-sex couples, second-parent adoption by same-sex couples, constitutional prohibition of discrimination based on sexual orientation, and a ban on “conversion therapy.” Using the year when the policy or law was enacted, each country received a positive response (1) if the specified law was in place in that year and a negative response (0) if it was not.Footnote 9
Results
I first consider the individual country latent estimates, where larger estimates correspond with a greater degree of norm adoption (Figure 2). Scholarship on the LGB norm has identified both the leadership of the Netherlands (Kollman Reference Kollman2007) and resistance of Uganda (Nuñez-Mietz and García Iommi Reference Nuñez-Mietz and Iommi2017). The results align with our expectations, where the estimate for the Netherlands is consistently large while the estimate for Uganda is consistently small. Moreover, the estimate for Uganda gradually decreases over time despite not necessarily changing its policies/laws. This is consistent with the changing standard of behavior associated with the norm and thus the position of Uganda on the latent dimension relative to other countries. In other words, the adoption of new policies/laws by countries in the late 1990s and 2000s (e.g., same-sex marriage, constitutional protection) beyond legalizing homosexuality or creating employment protections was indicative of a changing standard of appropriate behavior associated with the norm. We can also see that the model captures rapid changes, as illustrated by the both the Netherlands and the United Kingdom. In 1994, the Netherlands adopted employment protections and legislation on incitement to hatred, two very rare (difficult) behaviors for the time. In turn, we can observe a large increase in the latent estimate from 1993 to 1994. We can also observe that the relative distance between the United Kingdom and the Netherlands increased until the mid-2000s, as the standard of the norm changed but the United Kingdom did not adjust its behavior. In the late 2000s, however, a series of policy and legal changes brought the United Kingdom in line with the Netherlands and other norm leaders.
Figure 2. Individual Country Latent Estimates
Note: Individual country median latent estimates (degree of norm adoption) with 95% credible intervals. Larger latent estimates correspond with a greater degree of norm adoption.
We can also evaluate how the distribution of the median country latent estimates changes over time. Figure 3 compares the global distributions from 2001, 2009, and 2017.Footnote 10 Symons and Altman (Reference Symons and Altman2015) argue that the LGB norm is characterized by polarization rather than outright adoption or rejection. The results in Figure 3 broadly support this claim, but also provide evidence for a gradual shift towards adoption. While the global distribution was starkly polarized in 2001, the results in 2017 are indicative of general realignment towards greater adoption of the norm.
Figure 3. Distribution of Median Latent Estimates
Note: Global distribution of median country latent estimates (degree of norm adoption). Larger latent estimates correspond with a greater degree of norm adoption.
Finally, we can consider how the “difficulty” of each policy/law changes over time (Figure 4).Footnote 11 We can interpret the difficulty parameter as how “easy” it is for a country to adopt a given policy/law. Given the parameterization of the model, larger estimates correspond with “easier” policies/laws. Two aspects of Figure 4 are noteworthy. First, the general pattern of difficulty parameter estimates fits our expectations. Legalizing homosexuality, for example, is “easier” than a constitutional protection. Second, the difficulty estimates for several policies/laws become easier between 2001 and 2017, which is consistent with the norm becoming more widely adopted. While employment protection and hate crime legislation experience the largest changes, policies/laws affecting family structures noticeably shift as well.
Figure 4. Difficulty Parameter Estimates
Note: Median difficulty parameter estimates with 95% credible intervals for each policy/law. Larger estimates correspond with “easier” policies/laws.
Conclusion
In this note, I argue that item-response theory (IRT) provides a framework for measuring the degree of international norm adoption and develop a modified Bayesian dynamic IRT model to do so. Using the lesbian, gay, and bisexual (LGB) equality norm and a dataset of 13 theoretically informed domestic policies/laws (across 196 countries between 1990–2017), I demonstrate that we can empirically assess the substantive, geographic, and temporal shifts in the norm. More specifically, I evaluate how individual countries vary over time with respect to the degree of norm adoption, how the distribution of norm adoption across countries can vary, and how the substance of the norm (i.e., the standard of behavior) shifts dynamically. The results generated from this approach can not only directly contribute to statistical research on international norms (through the direct inclusion of the latent estimates in statistical models); they can also be used to strengthen case selection and theory development in multimethod research designs.
The substantive example focused on the LGB norm due to both the broad attention to state-centric norms in the literature and the availability of research on the LGB norm to validate the results. However, generalizing this approach to measuring norm adoption is relatively straightforward. The theoretical basis of the model—an informed set of policies or laws serve as indicators of norm adoption—can be applied to a variety of state-based human rights norms, environmental norms, or security norms. Future research can use this approach to advance dynamic theories explaining norm evolution, such as the determinants of norm backlash, or explore norm adoption among other types of actors (for example, international organizations; Sommerer and Tallberg Reference Sommerer and Tallberg2019). Methodologically, scholars might consider how subnational variation in policies can also be used to inform our estimates of norm adoption by states. Scholars may identify additional extensions by adapting alternative IRT models to different structures of international norms, such as the use of multidimensional measurement models to capture international norm clusters. Given the widespread use of latent response theory in other areas of political science and the extensive scholarship on international norms, this paper highlights the significant potential in greater interaction between both.
Supplementary Materials
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S0003055420000854.
Replication materials can be found on Dataverse at https://doi.org/10.7910/DVN/XS4P50.
Comments
No Comments have been published for this article.