1.1 Local power holders

We begin by noting that studies involving extensive data collection at the individual level almost always require the authorization of national and local power holders. In our own research outlined here, permissions were obtained at the ministry level and from local authorities in each enumeration area. In turn, the enumeration teams were issued certification letters, and the governments helped to publicize the survey and encourage participation via radio announcements in various languages. As Fujii (Reference Fugii2012) notes, it is both implausible and reckless to collect data from local contexts in the developing world without authorization, as local power holders will certainly know of the presence of researchers and maintain a right to sanction the research activities, particularly those undertaken by outsiders. Importantly, the local incumbent power may not always be the formally recognized government; Atran et al., (Reference Atran, Axelrod, Davis and Fischhoff2017) stress that in the context of studying terrorism, official governments can in some cases be too weak to provide consent, having abdicated that role to rebel opposition or terrorist groups in certain areas. This is a point to which we return later in the paper. Irrespective of who holds de facto local power, however, we suggest that the process of obtaining and publicizing local authorization creates a broader awareness in enumeration areas that the sanctioning power is in some sense implicated in the research.

In the study areas of Burkina Faso, Chad, and Niger exploited for this study, the formally recognized governments constitute the local power holders. Burkina Faso and Niger each score above 5 on the Polity IV democracy index, suggesting relatively democratic institutions and governments that exercise control with at least moderate support and legitimacy. Chad is less democratic but exerts fairly heavy-handed military control over citizens, mitigating the potential for rebel or extremist groups to gain de facto control (Marchal, Reference Marchal2016). All three countries take part in the regional G5 Sahel Cross-Border Joint Force aimed at undercutting terrorist activity,Footnote ¹ and, while extremists carry out insurgent attacks in the region, no rebel or extremist groups hold territorial control in the study area. As of our data collection, the extremist groups that are active in Burkina Faso, Chad, and Niger by and large have not operated openly; instead, they infiltrate communities and operate in the shadows or with one-off attacks (Antwi-Boateng, Reference Antwi-Boateng2017).Footnote ² For the purposes of hypothesis testing below, we thus treat the formally recognized governments as the local incumbent power holders.

1.2 Individuals under study

From the perspective of individuals under study, we note three tendencies relevant to response patterns over time. First, especially in rural or developing contexts, community members are typically well aware of the population-based surveys conducted around them. Hershfield et al., (Reference Hershfield, Rohling, Kerr, Hursh-Cesar, Bulmer and Warwick1983) note that the presence of enumerators tends to be more conspicuous than researchers assume, such that word of the research activities often precedes their arrival at households. That public opinion surveys like the Afrobarometer typically record the presence of community members listening in on ostensibly private survey interviews further provides an indication of the extent to which individual data collection is in some sense a community-wide activity. As knowledge of the activities spreads, later respondents provide attitudinal data from a different informational baseline than their counterparts reached earlier in the enumeration period.

Second, participants in public opinion surveys in the developing world often maintain fairly deep skepticism regarding the provenance of survey questionnaires and the confidentiality of their answers. According to Afrobarometer data, over one-third of respondents from Round 6 (and over half from Round 5) believe that enumerators for this non-partisan, non-governmental enterprise were sent by the government, despite a clear introductory statement to the contrary.Footnote ³ Longstanding skepticism about the secrecy of votes in elections—sometimes justified—can similarly undermine community members' confidence in other ostensibly confidential activities such as survey enumeration (Kalinin, Reference Kalinin2016). In fact, Singer et al., (Reference Singer, Hippler and Schwarz1992) note that more elaborate assurances of confidentiality in the collection of individual-level data can be counterproductive, increasing participants' concerns that their responses may be revealed. In short, participants in population-based research often assume that their responses may be known to local authorities.

Third, we assume that community members understand the capacity of local power holders to punish. Trust in police forces in our region of study remains low, owing in part to what is perceived as excessive use of force and arbitrary applications of punishment without due process (Goldsmith, Reference Goldsmith2005). Contexts controlled by rebel or terrorist groups can be much worse: the Taliban notoriously applied rigid punishment against women for minor dress code violations (Goodson, Reference Goodson2001), and as Boko Haram expanded in northern Nigeria, community members who challenged the group's authority were summarily killed (Mohammed, Reference Mohammed and Pérouse de Monclos2014). In response, up to 1.7 million persons are estimated to have moved internally (UNHCR 2017), indicating a clear understanding of the incumbent power's ability to punish.

To summarize the individual-level perspective, knowledge of population-based data collection tends to spread as those activities advance; meanwhile, participants often assume that their answers may be known to local authorities who possess the capacity to punish community members deemed to be deviant or uncooperative. In this context, we suspect that fear—or at least a desire to acquiesce—drives participants to increasingly offer attitudes consistent with the perspective of local incumbent power holders as word of the data collection spreads. This would be especially true regarding sensitive topics that implicate the local power holders, including security matters, religious divisions, and violence.

1.3 The content and context of study

Researchers have long understood the importance of contextual factors for survey outcomes. As social interactions, face-to-face data collection can be affected by respondents' experiences, perceptions regarding interviewers, and local environmental factors (Tourangeau and Yan, Reference Tourangeau and Yan2007; Krumpal, Reference Krumpal2013), and all of those pressures are amplified when survey topics are perceived as sensitive or taboo (Berinsky, Reference Berinsky2004). For example, researchers have demonstrated changes in response patterns as a function of interview language (Lee and Pérez, Reference Wray2014), interviewer ethnicity (Adida et al., Reference Adida, Ferree, Posner and Robinson2016), interviewer race (Davis and Silver Reference Davis and Silver2003, Finkel et al., Reference Wray1991), and interviewer gender (Johnson and DeLamater, Reference Wray1976). Blaydes and Gillum (Reference Blaydes and Gillum2013) find that when Egyptian women are surveyed regarding the sensitive issue of personal piety, their responses shift toward the devout when enumerators wear Islamic headscarves.

Broader political factors can also shape response patterns and the willingness or capacity of respondents to discuss sensitive topics truthfully. Robinson and Tannenberg (Reference Robinson and Tannenberg2019), for example, find that in autocratic countries, respondents practice pro-regime self-censorship on items relating to trust. Corstange (Reference Corstange2014, Reference Corstange2016) shows that survey refusals increase when sponsors are identified as foreign embassies as opposed to independent research institutions.

All of these contextual effects can be summarized as forms of social desirability bias, whereby respondents provide altered, more socially appropriate answers as a strategic or subconscious reaction to either the enumerator or the broader political context (Holtgraves, Reference Holtgraves2004). In response to these challenges, scholars have increasingly adopted subtle, experimental means to access sensitive attitudes (see, e.g., Rosenfeld et al., Reference Rosenfeld, Imai and Shapiro2016). Importantly, Blair et al. (Reference Blair, Imai and Lyall2014) find that responses to experimental measures approximate true attitudes in a way that standard survey questions cannot, adding confidence to the assumption that participants do not simply game the surveys. This is not to suggest, however, that answers will remain stable across respondents as information on the data collection spreads.

We argue that, over the course of population-based data collection on sensitive topics, a grapevine effect takes place. Grapevine effects have been cited in previous research to describe the legal transfer of responsibility for defamation as claims spread (Douglas, Reference Douglas2015) and the informal transfer of purchase information among consumers (TARP 1981). Here, we suggest that survey research on sensitive topics inspires community conversation, both when the topic is raised overtly and even when the subtlety of experimental measures masks the intent behind specific questions.Footnote ⁴ Respondents learn, not from their own experience with a survey question but from word-of-mouth in their communities, that the data collection in some way involves sensitive matters (such as, in this case, support for violent extremism). As a result, subsequent respondents are likely to be more attuned to the consequences of their responses than earlier ones, as fear of reprisal for inappropriate answers spreads.

To summarize the study's hypothesis, we expect that respondents who take part later in the collection of sensitive survey data on violent extremism will give answers systematically different from earlier respondents in their communities, moving in the direction of support for the local incumbent power. We would not expect this pattern to emerge in every survey—those with widely dispersed respondents, for example—but where respondents are queried in fairly close proximity, the word may travel more quickly than the enumeration. In the context of this study, we thus anticipate that expressed attitudes will tilt in the favor of government interests and against violent extremist groups as data collection proceeds. We surmise that this occurs because participants who become aware of the ongoing sensitive survey fear retribution for inappropriate answers from those in power.

The process by which this grapevine effect unfolds can be captured in qualitative reports. Our enumeration teams cited three common behaviors in the households they visited. First, despite requests for private interviews and efforts to situate the respondent away from onlookers, an enumeration supervisor reported that anywhere from two to several bystanders observed many of the interviews.Footnote ⁵ Children and the head of household were often present, but neighbors also sometimes followed proceedings before leaving the compound. Second, in the rural communes and tightknit urban neighborhoods in which the data collection took place, the local well or water pump serves as a center for social engagement, informal networking, and gossip. “If you raised the name of Al Qaeda in a survey with me, I promise you I would discuss it the next day at the water pump,” the enumeration supervisor noted in a follow-up interview.Footnote ⁶ Third, those who gain knowledge of the survey and its general contours, either from observing an earlier interview, hearing about it at the water pump, or learning through other channels, would be likely to share the news with other members of their households. Enumerators were often greeted with “we've been waiting for you to come by our house,”Footnote ⁷ suggesting that awareness of the surveys indeed spread through communities, and references to extremist groups were a likely reason why. We test for evidence of such a grapevine effect below.

2.1 Overt measures of support for violent extremism

To test our claims regarding a grapevine effect, we first evaluate overt expressions of support for violent extremism, where the effects should be most evident. To measure the outcome of support for violent extremism, we create a composite index from four survey items,Footnote ⁸ each coded on a 3-point scale with higher values indicating greater support:

• • a) Do you agree or disagree with the following statement: Al Qaeda's violent actions are permitted under Islamic law?

• • b) Do you agree or disagree with the following statement: Violence in the name of Islam can be justified?

• • c) When do you think that violence is an effective method to solve problems: often, sometimes, or never?

• • d) Do you personally feel that using arms and violence against civilians in defense of your religion can be often justified, sometimes justified, or never justified?

The key explanatory variable is the number of days since the first survey interview within the respondent's commune. Focusing on the passage of time within communes provides a measure appropriate to the geographically constrained logic of a grapevine effect, whereby word-of-mouth would likely travel from one household to another within a community and the process would start anew in other surveyed communities. We use OLS regression with standard errors clustered by commune (our PSUs), along with commune dummy variables to account for different start dates and other between-commune factors that may confound the estimation of within-commune grapevine effects.

As Table 2 shows, in bivariate analyses, the number of days since the first interview in a commune has a statistically significant, negative effect on overt support for violent extremism in both surveys. That is, respondents who are surveyed later in the data collection period within a commune are systematically less likely to express overt support for violent extremism, a pattern in keeping with the preferences of their local incumbent powers (the governments fighting terrorist groups in the region). The results cannot be easily attributed to social or political changes that might covary with the period of data collection: the surveys took place over several weeks and months and with different start times in each commune, and yet a within-commune pattern of decreasing support for violent extremism persists. Additionally, by including commune-level fixed effects, we account for any potential time-invariant factors that could confound the results, such as proximity to terrorist bases or exposure to previous attacks.

Table 2. Overt support for violent extremism (OLS)

Note: Standard errors clustered by commune in parentheses; commune dummies included in models but not reported; *** p < 0.01 ** p < 0.05 * p < 0.10.

To address other potential alternative explanations, we add a series of controls. Most importantly, variation in support for violent extremism may be a function of who lives where within each commune. Previous research suggests that wealthier and better-educated individuals are more supportive of violent extremist groups (Blair et al., Reference Blair, Fair, Malhotra and Shapiro2013); if those better off individuals live in clustered areas and are more likely to be surveyed earlier in the data collection period, our explanation would be spurious. Similarly, employment status may covary with proximity to local institutions and services and may also affect attitudes regarding the disruption caused by terrorism. Thus, if the household identification process led enumerators closer to or further from those centers over subsequent days, what may appear to be a consequence of word-of-mouth learning as the days progress could instead be driven by differences in employment status.

Table 3 adds controls for wealth, education, and employment. We measure wealth using an additive index, Wealth, denoting how many out of 13 possible household items such as a refrigerator, TV, and radio respondents have in their households. Education is measured on a 10-point scale ranging from no formal education to a postgraduate degree. Employed takes the value of 1 if a respondent reports being employed and 0 otherwise. If the pattern we presented above is simply a function of the type of respondent surveyed earlier and later in the data collection process, we should find that the number of days since the start of data collection loses its significance at the expense of those variables. As Table 3 shows, however, the coefficient on the variable for survey timing remains negative and significant, indicating that support for extremism continues to decrease as data collection time elapses, even with those controls.Footnote ¹⁰

Table 3. Overt support for violent extremism—alternative explanations (OLS)

Note: Standard errors clustered by commune in parentheses; commune dummies included in models but not reported; *** p < 0.01 ** p < 0.05 * p < 0.10.

The indexed measure of support for violent extremism does not easily lend itself to substantive interpretation, so we consider responses to the discrete components. Fifty-five percent of respondents at the outset of Survey 1 provided a neutral or positive response to at least one of the four measures of extremism and violence.Footnote ¹¹ By the end of data collection within communes, however, that number had fallen to 18 percent, a decrease of 37 percentage points over approximately a one-month period or less within communes. Support for violent extremism in Survey 2 similarly declined by 30 percentage points.

2.2 Experimental measures of support for violent extremism

While respondents may be particularly likely to alter their stated answers in response to overt survey questions, we are also interested in evaluating whether the grapevine effect may influence even those data collection methods that aim to circumvent strategic responses. To measure support for violent extremism in a subtle, unobtrusive manner, we apply an endorsement experiment to gauge individuals' support for violent Islamist groups without asking them to directly reveal their attitudes about this group. The use of specific extremist group labels as indicators of support for violent extremism more generally has been established elsewhere (see Fair et al., Reference Fair, Littman and Nugent2018). The endorsement experiment was designed as follows. First, respondents were randomly assigned to control and treatment groups, with one half of the sample assigned to each. Respondents in the control group were asked the following question:

The treatment group heard the same question but with the inclusion of an extremist group endorsement; respondents were additionally told that “It is likely that Al-Qaeda in the Islamic Maghreb (AQIM), an Islamist group, will oppose this program.”Footnote ¹² We measured support for AQIM as the difference across the treatment and control groups in the proportions of individuals who did “not at all” approve of the polio vaccination.

Figure 1 below shows a descriptive plot of the endorsement experiment for the first survey.Footnote ¹³ It can be seen that across all three countries, substantially more individuals in the treatment group were “not at all” in support of the policy, which we infer as covert support for AQIM. On average, approximately 18 percent of our sample supported AQIM covertly, with these figures relatively consistent across the three-country contexts.

Figure 1. Descriptive Plot of the Endorsement Experiment (Survey 1).

Note: “A Lot” indicates strong support for the polio vaccination program, which we interpret as a rejection of AQIM (since the treatment reported that AQIM is opposed to the initiative). “Not at all” indicates a rejection of the vaccination program, or support for AQIM.

Next, we test whether covert support for violent Islamist organizations is a function of the timing of interviews. We measure time as the number of days that passed since the first interview was conducted in the respondent's commune (thus, 1–28 for Survey 1 and 1–38 for Survey 2). We reverse-code the treatment responses to transform the outcome variable from “support for the polio vaccination program” to “support for the violent Islamist organization.” We then dichotomize the measure so that “not at all” supporting the vaccination program is coded as 1 (support for the Islamist organization) and moderate or strong support for the vaccination program is coded as 0 (meaning little or no support for the Islamist organization).

In addition to variables for the treatment (the Islamist endorsement) and the number of survey days elapsed, we include an interaction term that interacts survey timing with the endorsement treatment. This represents the additional impact of time on the vaccination policy response among the experimental group versus the control group, and thus serves as a test of whether respondents are more or less likely to support the extremist group as the survey progresses in their commune.

Table 4 shows regression results measuring support for the Islamist group in each of the two surveys, with controls included. Following Freedman (Reference Freedman2008), we use Ordinary Least Squares regressions both because randomization alone does not justify the logit linking function and because of the ease of interpretability of OLS results, though we also include logistic analyses in Appendix Table A3. As noted, standard errors are clustered by commune. A negatively signed coefficient indicates a decrease in respondents' level of support for the Islamist group.

Table 4. Covert support for AQIM (OLS)

Note: Standard errors clustered by commune in parentheses; commune dummies included in models but not reported; *** p < 0.01 ** p < 0.05 * p < 0.10. The experiments were administered to a random subset of respondents.

The results suggest that not even covert, experimental measures are immune to grapevine effects. The treatment variable measuring support for AQIM in the presence of the endorsement is positive and significant, as expected. Further, while the interaction of treatment and days is not significant in Survey 1, it is negatively signed and significant in Survey 2, indicating that those receiving the AQIM endorsement increasingly sided with the government and not the terrorists as data collection in their communes progressed, despite the disguised means of inquiring about their allegiances.

For interpretative purposes, Figure 2 illustrates the linear impact of the treatment on individuals' likelihood of supporting AQIM in Survey 2. While in the initial days of the data collection, the treatment increases support for the Islamist organizations (i.e. respondents who hear the Islamist group's position are more likely to “not at all” support the vaccination program), that effect steadily declines over the course of the data collection. By the end of the period of data collection in each commune, receiving the treatment has no statistically significant bearing on respondents' likelihood of expressing support for the extremist groups.

Figure 2. Support for AQIM in Survey 2 (95 percent CIs).

Importantly, time appears to affect individuals' responses in the treatment group but less so those in the control group, a finding consistent with the hypothesis. The overall results suggest that support for Islamist groups shifts away from the Islamist groups and thus in the direction of the anti-extremist governments, representing the local incumbent power holders. We argue that this pattern should be anticipated, as individual respondents become increasingly fearful about countering the local power holder during what many perceive as non-confidential inquiries.

While the commune-level fixed effects employed in our models account for any unobserved confounders in the broader context and in time-invariant differences between communes, we also add calendar day fixed effects to the models as a robustness check. The results remain consistent with our expectations. See Appendix A7 for details.

4.1 Support for Hizballah's disarmament among Beirut residents: Corstange (Reference Corstange2016)

In a 2016 study, Daniel Corstange discusses the results of a survey experiment of 2481 households in the greater Beirut metropolitan area, in which he manipulates the putative sponsor of the survey and asks local residents about their views regarding support for Hizballah. The study includes both overt, direct measures of support for Hizballah and measures of support for Hizballah following a covert manipulation. It was conducted within a month of the formation of a Hizballah-led cabinet, between June 29 and July 14, 2011.Footnote ¹⁸ One complexity in the study context is that, while Hizballah had formally taken control, non-Shi'a neighborhoods in Beirut remained in a stance of conflict vis-à-vis Hizballah.

We test whether responses changed in the direction of the local power—either pro- or anti-Hizballah depending on the sectarian make-up of the community—as data collection unfolded. While our analyses do not undermine the central findings of the Corstange paper, they do suggest that a grapevine effect may emerge in population-based studies beyond our own. Like the analyses of our own survey data, we find strong support for such an effect using the overt measures and suggestive patterns even with the use of experimental manipulations.

To replicate the overt, direct measure of support for Hizballah in the Corstange (Reference Corstange2016) study, we use the same dependent variable, a composite index of four questions regarding Hizballah's disarmament, the Syrian conflict, the neutrality of the international Supreme Tribunal for Lebanon (STL, which has implicated Hizballah members in domestic assassinations), and the desirability of abolishing the STL. All four items are coded on a 5-point scale, with higher values indicating greater support for Hizballah; we use the numerical average for the composite measure. We include available controls similar to our own, the key independent variable of days since the first interview in a locality, and an additional interaction term to account for the effect of days specific to Shi'a-majority neighborhoods.Footnote ¹⁹

As Table 5 shows, the average effect of time since the first interview in a neighborhood is insignificant. However, in Shi'a-majority neighborhoods, where Hizballah is the recognized local power, we see a different effect: the interaction of days and Shi'a neighborhood indicates that respondents in those areas increasingly provide responses in support of Hizballah as the data collection in their neighborhood advances.Footnote ²⁰

Table 5. Overt support for Hizballah (Corstange, Reference Corstange2016)

Note: OLS regressions with standard errors clustered by neighborhood in parentheses; neighborhood dummies included in models but not reported; *** p < 0.01 ** p < 0.05 * p < 0.10.

Figure 4 illustrates this point. The effect of time in non-Shi'a neighborhoods is marginally negative in slope, as might be expected from sectarians who do not accept the authority of Hizballah. In Shi'a neighborhoods, however, responses move in a positive, pro-Hizballah direction as data collection within each neighborhood progresses.

Figure 4. Effect of Days on Hizballah Support in Shi'a versus Non-Shi'a Neighborhoods (Corstange, Reference Corstange2016).

The Corstange study also includes an experimental treatment aimed at eliciting the effects of a Hizballah antagonist, the United States, in an unobtrusive manner. The treatments—embedded in enumerator introductions—expose respondents to a variety of putative sponsors, including academic and governmental, and the United States and non-U.S. sponsors.Footnote ²¹ Among the 2100 respondents who consented to the survey, 381 individuals were assigned to the control group, while 434 individuals were told that the survey was conducted on behalf of “the American Embassy.” Other treatments included “the University of Maryland” and “the Canadian Embassy.”Footnote ²² Again, the outcome variable is a composite index that captures local residents' support for Hizballah.

We have no theoretical expectations regarding the starting point of responses at the outset of data collection. However, if a grapevine effect exists in the data, we should expect to see that both treated and control responses from Shi'a neighborhoods tilt in the pro-Hizballah direction as data collection progresses (because both respond to the same outcome measure of Hizballah support), and that those exposed to the United States treatment display an even stronger tendency in this direction after having been cued to this antagonist of the incumbent local power.

The limited sample size for the experimental component of the study does not leave us room to detect statistically significant results, but Figure 5 illustrates a pattern in keeping with a potential grapevine effect. In it, we plot OLS regression results for the effects of time on Hizballah support among treated and control respondents living in Shi'a and non-Shi'a neighborhoods. The top two lines represent control and treatment responses in the Shi'a-majority areas. They indicate that support for Hizballah gradually rises among the control group and that the responses of treated participants increase even more starkly in this direction. The bottom two lines represent the control and treatment responses in non-Shi'a neighborhoods; interestingly, while the control group respondents remain steady in their (clearly inferior) support for Hizballah, evidence of increasing support for Hizballah over the course of data collection among those treated with the American cue again emerges, perhaps as fear of Hizballah reprisal seeps even into those neighborhoods.

Figure 5. Plotted Experimental Results in Shi'a and Non-Shi'a Neighborhoods (Corstange, Reference Corstange2016).

Note: Standard errors clustered by neighborhood.

Corstange (Reference Corstange2016) shows that, in the context of systematic opting-out of survey participation by some respondents, purported U.S. sponsorship makes people in Beirut more supportive of U.S.-favored policies and less supportive of Hizballah interests. Our grapevine analysis additionally suggests that, while the average treatment effect remains robust, responses in Shi'a neighborhoods gradually tilt in a relative pro-Hizballah direction as data collection progresses. Corstange's conclusion may thus be considered a conservative one, as it persists despite the apparent tendency of some respondents to alter their responses out of fear of the local Shi'a power.