The introduction to this symposium highlights that collaborative methodologies (CM) invite “people affected by the research puzzle […] to participate in concept formation and methodological decision making, regardless of whether the methodologies are quantitative or qualitative, or positivist or interpretivist” (Firchow and Gellman Reference Firchow and Gellman2021; emphasis added). There are, however, practical issues that limit opportunities for meaningful collaboration with participants in large-N surveys. The collaboration may not be as thorough and meaningful as in smaller-sample surveys due to time and accessibility constraints. This article focuses on how these limitations become even more apparent when fielding a survey in a politically unstable environment. Practical constraints and political volatility made it unfeasible to use CM in the implementation of our survey of political trust in Haiti; however, in various ways, we worked “with” rather than “on” local participants. We discuss how collaboration was achievable and ultimately mattered. The highly uncertain political environment led us to rely on a Haitian research team to better evaluate local understanding of trust and to effectively design our study. Collaboration with the Haitian researchers helped us to address the idiomatic language, norms, and cultures of local communities as well as the variation in local awareness of peace-building organizations.
Practical constraints and political volatility made it unfeasible to use CM in the implementation of our survey of political trust in Haiti; however, in various ways, we worked “with” rather than “on” local participants.
In the autumn of 2019, we fielded a survey in Haiti with the aim to examine trust among Haitians in external organizations such as the United Nations (UN) and international non-governmental organizations (INGOs) as contrasted to local community organizations (LCOs) and local churches. Haiti was flooded with foreign intervention following political upheaval in the 1990s and even more so after the devastating earthquake in 2010. The failures of foreign interventions in Haiti have been documented extensively, but little is known about how Haitians trust—or distrust—foreign organizations in contrast to local organizations. Collaboration with Haitians is clearly essential to deliver the local perspective, which was the aim of our study.
To measure what political trust means for Haitians, CM proposes to involve participants to define criteria and indicators that are accurate, complete, and relevant for them. When implementing a large-N survey, in practice, such involvement requires multiple iterations—for example, initial collaboration via focus-group meetings to design the survey to be subsequently fielded.Footnote 1 A highly uncertain and volatile political environment, however, risks framing the evaluation of political trust and priming responses. To explain our rationale for not directly involving the survey participants in our study via CM, we illustrate how political trust is shaped in Haiti via positive and negative framing. Questions about political trust prime respondents to give negatively rather than positively worded responses, which indicates that participants are fundamentally affected by instability in the country. In addition to creating obvious practical problems, (political) instability thus undermines the collaborative measurement of trust. Therefore, in our case, the applicability of CM for large-scale population surveys remained limited.
FIELDING THE SURVEY
When researchers investigate unfamiliar regions, the knowledge and experience of local research teams is essential (Asiamah, Awal, and MacLean Reference Asiamah, Awal and MacLean2021). We relied on the long-standing cooperation between Athena Kolbe and a team of Haitian social workers and graduate students. Athena has extensive experience in organizing surveys in Haiti. The Haitian research team provided valuable expertise in conducting social science surveys in Haiti. We provided the team with a thorough background of our study and Athena was responsible for their training in terms of interviewing and sampling strategies. The Haitian team provided valuable local insights that we would have been unable to grasp otherwise. We designed the survey questionnaire, defined the sampling frame, and secured ethical approval. These tasks reflect both technical expertise and practical consideration as well as our broader theoretical interest in the study. Ultimately, we did not simply want to measure trust but also to understand it as it related to factors affecting levels of trust and possible implications of its lack.
Collaboration with Haitian researchers was important for several reasons. First, the Haitian research team translated the survey instrument from English to Haitian Creole and, in the process, helped to refine the question wordings. Second, local knowledge was vital to obtain a truly representative sample. Poor infrastructure and security concerns often limit and bias the access of Western researchers. Third, the team suggested examples of LCOs and INGOs to be used in the survey.Footnote 2 Local knowledge enabled them to identify organizations that were sufficiently known across Haiti and to avoid naming those that had been embroiled in particular scandals. Mentioning the latter would have introduced bias by priming the responses with these scandals, and doing so may have left the wrong impression that the survey was part of a public relations effort. Fourth, discussions with Haitian researchers convinced us to increase the number of indicators of trust. The team suggested adding indicators that it considered relevant and fitting to local conditions.
CHALLENGES OF CM WHEN FRAMING TRUST
CM builds on the idea that reliance on local understanding increases the validity of one’s research. Trust does not always have a similar meaning, and Haitians probably use different, context-specific indicators for trust. The perceptions, experiences with, and expectations of a particular organization determine not only how much respondents trust that organization but also their criteria or indicators for trust.
Reflecting our theoretical understanding of trust as relational, the survey was designed to allow for comparisons within the Haitian population (as trustors) across different types of organizations (as trustees). Accordingly, appropriate indicators of trust had to balance shared understanding across Haiti but also differentiation between trustors (i.e., variation among Haitians in their propensity to trust) and differences in perceived trustworthiness (i.e., variation across different peace-building organizations). Therefore, simply asking about trust (konfyans in Creole) in an organization such as the UN may not be informative. The Haitian understanding of what it means to trust an organization may correspond imperfectly with our—Western or theoretical—conceptualization. Firchow (Reference Firchow2018, 109) argued for the importance of localized and contextual indicators when people describe peace in their life. For similar reasons, Flores (Reference Flores2021) proposes to develop “everyday democracy indicators.” At the same time, highly fine-grained or localized indicators are unlikely to represent the broader set of indicators relevant across Haiti. The promising iterative process as outlined by Levy and Firchow (Reference Levy and Firchow2021) was, in our case, infeasible because of budget and time constraints due to the volatile environment. There is a need not only to balance local and more universal indicators but also to consider carefully how accurately collaborators—in our case, the Haitian researchers—represent the population of the study: that is, Haitians in general. Our large-N survey of Haitians required a medium level of abstraction in measuring the central concept of trust, balancing contextual indicators with a more universal extension (Sartori Reference Sartori1970, 1044). Also, our purpose was to have absolute control of the survey instrument to ensure that the questions pertained to the framing literature of trust in peace-building organizations. Moreover, because trust is multidimensional (Hardin Reference Hardin2002; Mayer, Davis, and Schoorman Reference Mayer, Davis and Schoorman1995), it is important to understand which indicators respondents base their trust on when evaluating a particular organization.
Positive or negative primes are likely to produce different indicators—for example, reliability draws attention to the possible benefits an organization may bring whereas corruption emphasizes losses. These effects result from equivalency framing that presents “the same critical information in either a positive or a negative light” and has been shown to affect individual perceptions (Levin, Schneider, and Gaeth Reference Levin, Schneider and Gaeth1998, 150). We expected that negative wording or frames prime trust perceptions more strongly by means of direct and indirect mechanisms. They do so directly because negative words (e.g., corruption and arrogance) prime on losses or on what organizations failed to deliver, wherein losses generally provoke stronger reactions. Indirect effects occur because traumatic experience of a conflict generally leads to low levels of trust in postconflict societies such as Haiti (Hutchison and Johnson Reference Hutchison and Johnson2011). Experiencing substantial violence increases the risk that people perceive the state and other institutions as unable to provide security (De Juan and Pierskalla Reference De Juan and Pierskalla2016).
We recognize the value of CM to identify nuanced experiences from local people reflecting concerns or feelings that cannot be addressed adequately in a survey instrument designed solely by the researchers. Because conflict is arguably an explanatory factor of trust, any measurement of trust involving collaboration with participants experiencing conflict will be biased toward negative indicators of trust. In conflict or highly volatile postconflict environments, it is challenging to directly involve CM because political trust tends not only to be low but respondents also are more responsive to negative indicators of trust generating participants’ bias in the study. This means that negative wording has a higher impact in postconflict societies due to the preexisting negativity of the traumatic experience. Negative wordings also prime respondents with negative—possibly even traumatic—experiences and lower their levels of trust accordingly.
In conflict or highly volatile postconflict environments, it is challenging to directly involve CM because political trust tends not only to be low but respondents also are more responsive to negative indicators of trust generating participants’ bias in the study.
EVIDENCE FROM HAITI
Our survey instrument was implemented nationwide across Haiti amid a deteriorating political situation. Approximately 2,000 respondents were probed about their perceptions of international and local institutions with a focus on political trust. We used different survey items to examine how wording affects the perception of political trust. Respondents also were asked about their overall experience with international and local organizations, as well as how they felt about the current situation in Haiti (Dorussen, Bakaki, and Kolbe Reference Dorussen, Bakaki and Kolbe2021).
The use of negatively or positively worded survey items can identify whether this aspect of survey design affects how people report political trust. Participants were asked about four different types of organizations (i.e., the UN, INGOs, LCOs, and local churches) that operate across Haiti and to associate them with different words or short characterizations. Approximately 98% of our sample was aware of the international organizations (i.e., the UN, Action Aid, and Save the Children) and local organizations (i.e., Fonkoze, Fokal, and local churches) that we provided as examples. Also, 45% of respondents noticed an international institution working in the area and 94% noticed a local institution in their region.
Participants received the survey questions and answered items in random order without any indication about what we considered positive or negative wordings. They were asked which words “best describe” each organization, and they could select multiple items for each organization. We included 16 positive items next to four negative items. The considerably larger number of positive items actually provided a difficult test for our study because it essentially meant that we could have obtained more answers for positive wordings. In effect, the structure and the wording of the questions gave considerable freedom to respondents to indicate which particular aspects of trust they deemed relevant for a specific organization.Footnote 3
The first four items listed in table 1 are those that we classify as negative wordings; the remaining items are positive wordings. Table 1 ranks the words based on the frequency that respondents associated them with a particular organization, from highest to lowest. We found that negative words such as “corrupt” and “arrogant” were most commonly associated with all types of organizations; in particular, local churches and the UN were perceived as corrupt. Local churches were less commonly perceived as arrogant compared to other types of organizations. INGOs more often were perceived as unpredictable whereas LCOs were more commonly viewed as incapable.
Table 1 Ranking of Wordings Associating Aspects of Organizational Trust
Notes: Percentage of total number of respondents; N=1,547; sample included only those respondents who evaluated all four types of organizations; original (Haitian Creole) wordings in italics; highest percentage per row in boldface. Respondents were allowed to give positive responses to multiple indicators; therefore, column and row totals do not sum to 100%.
Respondents were more inclined to ascribe positive wording to LCOs, which were perceived as representing Haitians, respecting Haitian autonomy, and being approachable. The differences in responses could be quite stark: for example, 66.15% agreed that LCOs respect Haitian autonomy whereas only approximately 1% agreed that this applied to the UN. The only positive wording regularly associated with the UN was competent: 30.06% described it as competent compared to 12.29% for INGOs and less than 10% for LCOs and local churches. INGOs and local churches were associated with communicating effectively, and respondents more often described them as reliable compared to the UN and LCOs. Haitians thus were primed toward negative wordings, which reflects lack of trust in foreign organizations as well as disillusionment with local organizations.
To compare the general preference for positive and negative wordings, we generated indices for the positively and negatively worded survey items for each organization. We then calculated the relative frequency by which respondents selected the provided survey wording: the number of times a positive or negative wording was chosen divided by the total number of positive or negative wordings. When describing the UN, the results show that respondents selected, on average, only about 3% of the positive wordings but, on average, more than 40% of the negative wordings available. When describing INGOs, they selected, on average, 8% of the positive wordings and, on average, 47% of the negative wordings available. Respondents selected more positive wordings when considering LCOs, on average, at 23.5% and about 31%, on average, the negative wordings. When referring to local churches in Haiti, our respondents selected 11% of the positive wordings and, on average, about 30% of the negative wordings. As shown in figure 1, these preferences significantly differed across most organizations presented because the confidence-interval bars do not overlap. The only exception is that negative wordings for LCOs and local churches did not significantly differ. Although respondents had negative feelings about all organizations, they reported the strongest negative reaction to INGOs.
Figure 1 Positive and Negative Assessment of Survey Wording
Notes: N=1,547; vertical bars show confidence intervals calculated at 99%.
CONCLUSIONS
The Everyday Peace Indicators (EPI) research approach (Firchow Reference Firchow2018) advocates for the use of locally derived evaluation frames and for relying on local stakeholders and respondents to identify indicators of (dis)trust. This approach proposes to use CM initially with local stakeholders via focus-group meetings. It is our intention to incorporate focus-group discussions in future research to allow for direct engagement with the local community and to eventually provide a thorough understanding of positive and negative framings of political trust. Participatory bias will be limited in selective focus groups with stakeholders who are better informed on the issue of concern. However, focus groups provide qualitative information that also has limitations in terms of generalizations—even within the Haitian population, given that Haiti is a diverse country.
We relied on the expertise of the local Haitian research team to refine our research design. This helped us to obtain comprehensive and accurate information on political trust as relevant for the general Haitian population. The large-N survey provides substantial evidence of the sources of trust (and distrust) in the population. Negative wordings provoked the strongest responses, but there also was notable variation in the choice of positive wordings across organizations (e.g., “competent” for the UN and “respecting Haitian autonomy” for LCOs). We would have not been able to obtain these findings with a smaller group of participants or if we had collaborated directly with participants largely affected by the research questions.
Contributions to this symposium illustrate how CM can be applied across various methodological approaches. Collaboration with the people affected also is relevant and possible in population surveys, but it poses unique challenges. First, there will be more heterogeneity among the research population. For practical reasons, CM will be able to engage with only a relatively small group of participants. It is not only more difficult to capture the full heterogeneity with a small subset of participants, it also is key that the participants in CM are not a biased representation of the full population. It is obvious that the team of Haitian researchers on whom we relied were not a random sample of the Haitian population, but they had valuable knowledge and experience about conducting (survey) research in Haiti. This was an important and relevant tradeoff. If time and budget had permitted, applying CM in multiple places could have identified various relevant indicators that indeed would have been interesting (Levy and Firchow Reference Levy and Firchow2021). Potentially, EPI would have been more reflective of different experiences than the Haitian research team.
Collaboration with the people affected also is relevant and possible in population surveys, but it poses unique challenges.
Second, in important ways, our research population also will change over time. Temporal heterogeneity is particularly important in politically volatile situations such as Haiti and causes problems when there is a need to scale up (Levy and Firchow Reference Levy and Firchow2021)—for example, when indicators identified by focus groups may no longer be appropriate when the larger survey is fielded.
Third, concepts are not theory free, as illustrated by our finding that political instability as experienced by participants biases their preferred measures of trust. This does not invalidate the use of CM. Rather, it highlights the necessity to remain aware of the risks of priming respondents and framing survey questions.
DATA AVAILABILITY STATEMENT
Replication materials are available on Harvard Dataverse at https://doi.org/10.7910/DVN/D6W8W1.
