Interest in mixed-methods research in political science appears to be at an all-time high. Advice on best practices for combining at least two methodological approaches from distinct traditions into one research design has increasingly appeared in articles, books, and textbooks published in the past two decades (Ahram Reference Ahram2013; Collier and Elman Reference Collier, Elman, Box-Steffensmeir, Brady and Collier2010; Coppedge Reference Coppedge1999; Creswell and Clark Reference Creswell and Plano Clark2011; Fearon and Laitin Reference Fearon, Laitin, Brady, Box-Steffensmeier and Collier2008; Goertz Reference Goertz2017; Greene Reference Greene2008; Reference Greene2007; Humphreys and Jacobs Reference Humphreys and Jacobs2015; Johnson, Onwuegbuzie, and Turner Reference Johnson, Onwuegbuzie and Turner2007; Lieberman Reference Lieberman2010; Reference Lieberman2005; Onwuegbuzie and Johnson Reference Onwuegbuzie and Burke Johnson2006; Schneider and Rohlfing Reference Schneider and Rohlfing2013; Seawright Reference Seawright2016; Seawright and Gerring Reference Seawright and Gerring2008; Tashakkori and Teddlie Reference Tashakkori and Teddlie2010; Weller and Barnes Reference Weller and Barnes2014). By 2013, almost half of scholarly publications using OLS regression referred to mixed methods, compared with only 8% of such publications in 2000 (Seawright Reference Seawright2016, 14). Entire journals devoted to mixed methods have gained prominence, most notably Journal of Mixed Methods Research (Sage Publications). The American Political Science Association (APSA) has an organized section for Qualitative and Multi-Method Research, founded in 2003, and increasingly more scholars are teaching mixed-methods techniques in graduate courses, APSA short courses, and methods-focused summer programs, including the Institute for Qualitative and Multi-Method Research in Syracuse.
The expansion in the use of mixed methods—which often (but not always) takes the form of combining quantitative analyses with qualitative case studies—inspired a lively debate in the discipline. Some scholars question the extent to which mixed-methods research designs actually enhance causal inference, whereas others doubt the utility or even the feasibility of combining methods in a single study (Ahmed and Sil Reference Ahmed and Sil2012; Chatterjee Reference Chatterjee2013; Goertz and Mahoney Reference Goertz and Mahoney2012; Kuehn and Rohlfing Reference Kuehn and Rohlfing2009). Even among advocates of mixed-methods approaches, disagreements arise about exactly how one should combine various methods and the extent to which mixing methods contributes to the development of robust, cogent, and parsimonious theories. Triangulation—in which a repurposed hypothesis is tested sequentially using the techniques of at least two distinct methodologies (Jick Reference Jick1979; Tarrow Reference Tarrow1995)—remains the dominant approach to mixed methods in political science. However, other mixed-methods approaches—including integration (Seawright Reference Seawright2016); Bayesian analysis (Humphreys and Jacobs Reference Humphreys and Jacobs2015); nested analysis (Lieberman Reference Lieberman2005); and complementarity, initiation, and expansion (Greene, Caracelli, and Graham Reference Greene, Caracelli and Graham1989)—reflect areas of both neglected theory and emerging praxis. From this clash of perspectives, important questions arise: What is the specific contribution of each method in a mixed-methods analysis? What type of leverage in causal inference do researchers gain by simultaneously going “deep” and “broad”? What is the appropriate way to connect two or more methods in a single research design?
This symposium takes the view that mixed-methods research designs can provide valuable contributions to causal inference and theory development. However, at the same time, the growing use of mixed methods creates a need for venues that allow for a deeper, more sustained discussion of how different methodological tools and traditions can enhance our research. Specifically, more needs to be done to bridge the gap between methodological prescription and scholarly practice. Currently, there is no clear position on how to properly conduct mixed-methods research because scholars disagree about best practices. Perhaps more problematically, researchers often neglect to follow available guidance. Moreover, many unanswered questions related to useful yet underutilized mixed-methods techniques remain.
The five featured articles address these issues by discussing guidelines for researchers in terms of causal inference, contrasting actual research practices with theories of best practices, and exploring unanswered questions related to underutilized methodological tools.
The five featured articles address these issues by discussing guidelines for researchers in terms of causal inference, contrasting actual research practices with theories of best practices, and exploring unanswered questions related to underutilized methodological tools. These articles originated in the context of the Southwest Workshop on Mixed Methods Research (SWMMR), an annual workshop dedicated to advancing the theory and practice of mixed methods in the social sciences. Founded by four assistant professors in 2014, the SWMMR held its inaugural meeting at the University of New Mexico in November 2015. The second workshop took place in October 2016 at the University of Arizona. In October 2017, the SWMMR will be hosted at the University of California, Riverside. Funding for this initiative is from various sources, including the Methodology, Measurement, and Statistics Program at the National Science Foundation; the Consortium on Qualitative Research Methods; the Robert Wood Johnson Foundation; the Latin American and Iberian Institute of the University of New Mexico; and the political science and government departments at the University of New Mexico, University of Arizona, and University of California, Riverside. The five articles in this symposium represent the type of work that the SWMMR seeks to advance.
Barnes and Weller provide guidance for the proper conduct of mixed-methods research focused on causal inference. They consider how to achieve analytic transparency in the context of mixed-methods research by making two related contributions. First, they translate the general call for analytic transparency into a series of three questions that can frame the discussion of the role of process-tracing case studies in a mixed-methods research agenda. Second, they consider the value added of small-N methods based on their purported contribution to the larger research project and how they combine with the large-N results. Taken together, these two aspects clarify analytic transparency without imposing a single perspective on all scholarship.
The articles by Koivu and Hinze and by Niedzwiecki and Nunnally use meta-analyses to examine the actual use of mixed methods in research designs. Koivu and Hinze examine the gap between practice and prescription by asking to what extent the actual practice of case selection diverges from theories of appropriate case-selection techniques. Using bibliometric analysis, the authors identify several trends in case-selection strategies. They find that researchers rely heavily on most-similar research designs that select on the dependent variable, combine case-selection techniques in an ad hoc manner, and rarely identify the population of relevant cases from which they have chosen their specific case studies. Koivu and Hinze identify an additional factor that may influence case selection: logistical considerations, such as language skills or in-country networks. They conclude that practitioners should clarify the goals of their case selection and be more aware of underutilized strategies, whereas methodologists should address proper techniques for combining case-selection procedures. Moreover, a threshold rule should be applied to cases chosen for logistical reasons.
Niedzwiecki and Nunnally also study the actual use of mixed methods in political science. In focusing on the literature about welfare states, the authors find that few published works incorporate mixed methods. They argue that this is a missed opportunity because the few studies that combined multiple methodologies in a single research design significantly advanced theories of welfare-state formation and outcomes.
As more researchers work with mixed methods, the onus of recognizing and implementing good practices should be shared equally among methodologists and practitioners. This requires that we extend the well-worn paths of standard mixed-methods approaches.
The Harbers and Ingram article and the Cyr article address unanswered questions in the use of spatial-econometric analysis and focus groups, respectively. Harbers and Ingram argue that most mixed-methods designs still assume that case selection will be regression-based; in those regressions, units are assumed to be identical and distributed independently of one another. They show how tools from geospatial and spatial-econometric analysis can be leveraged to determine the appropriate unit of analysis and to select cases for subsequent qualitative analysis. In doing so, the authors identify promising ways for integrating mixed-methods literature with spatial-analysis literature.
Cyr’s article demonstrates that focus groups are particularly useful when incorporated into mixed-methods research designs. Cyr explains that focus groups can and should be used in conjunction with quantitative methods when researchers want to study intersubjectively created or complex concepts, make contextualized comparisons, or explore mechanisms or relationships that are first identified in large-N analyses. Focus groups are particularly useful for these objectives because the data they produce are inherently social and emic in nature.
Overall, cutting-edge mixed-methods research has advanced the systematic analysis of “big questions” in political science, including those that pertain to political regimes, party systems, interstate conflict, and development. However, much work still needs to be accomplished. As more researchers work with mixed methods, the onus of recognizing and implementing good practices should be shared equally among methodologists and practitioners. This requires that we extend the well-worn paths of standard mixed-methods approaches. We must move beyond triangulation to incorporate other means of connecting quantitative and qualitative analyses, including integration, complementarity, expansion, and initiation. We must surpass standard regression-based approaches to quantitative methods and case-based approaches to qualitative methods. We must interrogate the ontological implications of the increase in mixed-methods research designs and consider the practical consequences, especially related to training future political scientists. Certainly, we must improve dialogue between methodologists and practitioners so that mixed-methods theories translate into practice. Mixed-methods research may still be the road less traveled. Nevertheless, as its use grows, a sustained effort to improve theory and praxis could make all the difference.
