With this article we are contributing to a conversation about Critical Frame Analysis (CFA) as a feminist research method. CFA was developed within the context of two collaborative and comparative research studies of gender equality policies in the European context, MAGEEQ (www.mageeq.net) and QUING (www.quing.eu). Since the introduction of CFA in these projects, many scholars have used the method—some affiliated with these projects as well as others. This contribution is a first reflection on CFA and a call for more extensive reflections on methodologies developed in feminist work. We use reflection on CFA's origins, mixed with illustrations taken from research articles by authors who have been affiliated with the projects and others, and self-criticism based on two of our own studies. These reflections underpin our conclusions about the ongoing potential of CFA and the necessity and urgency of more thorough attention to methodological issues related to the use of CFA.
THE BEST OF TWO WORLDS: AIM AND ORIGIN OF CRITICAL FRAME ANALYSIS
CFA was developed to analyze and address discursive power dynamics connected to policy making (Verloo Reference Verloo2005). The methodology is designed to disclose and study the different representations that sociopolitical actors offer about policy problems and solutions in policy documents (Verloo and Lombardo Reference Verloo, Lombardo and Verloo2007). Built on insights from communication research, social movement theory, and critical policy studies, CFA was introduced to move beyond the methodological shortcomings of both quantitative and qualitative discursive research.
The MAGEEQ project (2003–2006) set out to describe and analyze comparatively and systematically how gender equality is framed as a policy problem across Europe (Verloo Reference Verloo2005). Existing comparative methods presented a methodological puzzle on how to move beyond either simple word counts or establishing codes before the analysis. Varieties of discourse analysis on the other hand score high on finding unexpected elements, but their results are almost impossible to compare across studies and researchers. Leaning heavily on social movement scholars, CFA started “from the general assumption that a policy (proposal) will always contain an implicit or explicit representation of a problem (diagnosis), connected to an implicit or explicit solution (prognosis) and a call for action” (Verloo Reference Verloo2005, 22). CFA's answer to the methodological puzzle then is to analyze crucial dimensions of frames based on a set of sensitizing questions for diagnosis, prognosis, and call for action, rather than constructing a hierarchical set of codes (as in content analysis) or typologies of frames (as in some forms of discourse analysis). The “critical” in CFA stands for explicitly paying attention to the voice of actors (authors of texts and references in texts) and to their varying power in diagnosis, prognosis, and call for action. When CFA is applied from a feminist perspective, gender+ is central in the power dimension.
The MAGEEQ project applied CFA on a set of texts based on policy process analysis. Open codes (answers to sensitizing questions) were used to characterize the text, and a computer program (KWALITAN) was used to store and organize the codes. The codes were subjected to a round of revision within the 25-person research team. Together, the revised codes form a “supertext”: a structured and systematic summary of the analyzed text, comparable to other such supertexts. Then the codes were synthesized across texts, again in rounds of discussion, to describe frames (coherent combinations of diagnostic and prognostic codes) across policy fields and across countries. This enabled comparative analysis of frames.
The MAGEEQ project saw the potential of CFA predominantly in its ability to detect unexpected elements and inconsistencies because of its open coding, and its analytical capacity to expose policy inclusion and exclusion related to the different roles and voice given to actors in diagnosis and prognosis. The ability to compare was a second strong asset. The limitations of CFA were the time-costly data harmonization and aggregation of codes and frames, as well as an acknowledgement that the comparison potential is still limited (Verloo and Lombardo Reference Verloo, Lombardo and Verloo2007). MAGEEQ's technical problems with the database were substantial, but the program's ability to generate supertexts from the codes was extremely helpful.
DEVELOPING A MORE ADVANCED SYSTEM
The much larger QUING project (2006–2011) further developed CFA by adding several innovations: syntactic coding, a database for storing texts and codes, with added code hierarchies building options (Dombos et al. Reference Dombos, Andrea Krizsan and Zentai2012). As in MAGEEQ, the qualitative coding was in English, and codes were stored linked to the original texts.
The paradox in the further development of CFA in QUING was about simplicity versus sophistication. QUING CFA is more detailed and electronic than MAGEEQ CFA but kept the open coding to capture as much meaning as possible. The electronic database and its to-be-developed tools intended to help keep under control the work of comparing 2086 texts. Within the QUING project, codes of the 2086 texts were synthesized to frames at text and at issue level, and a comparative analysis of frames across issues and countries was successfully made, as well as a broad analysis of voice in the texts (Krizsan et al. Reference Krizsán, Tamás, Erika, Linda, Jasminka, Martin, Roman, Ana, Birgit and Mieke2009). It proved more difficult to analyze the intersectional dimension of the texts and of the frames, and this was done by the old-school method of making separate country reports, aggregated in a final intersectionality report (Verloo et al. Reference Verloo, Walby, Armstrong and Strid2009). Similarly, explanatory analyses were conducted only for certain issues and for small-scale comparisons.Footnote 1 Code hierarchies were made for a limited set of code categories (mainly actors, norms, and domains) and proved to be extremely time consuming. The code hierarchies have not been used extensively so far (but see Van der Haar and Verloo Reference van der Haar and Verloo2013). After the QUING project, the database has been maintained and is still used by former QUING researchers, but it is not open to others. While the QUING database has been developed using Open Source software, additional finances would have been needed to offer such a database as a tool for new projects.
CFA APPLIED: POTENTIAL FOR IMPROVEMENT
Our Google Scholar search (February 10, 2015) showed frequent use of CFA in research: we found 396 citations to three key publications on the method particularly (Verloo Reference Verloo2005, Reference Verloo2007; Dombos et al. Reference Dombos, Andrea Krizsan and Zentai2012), testifying that CFA is widely referred to. Roughly half of the citations found were by scholars who were not part of the MAGEEQ and QUING research teams. More than half of the research articles by non-team members refer to CFA in their methods section, usually to position their work in relation to forms of discursive analysis or to refer to Verloo's definition of the concept of frame. Our observation is that there are many single case studies that analyze country or organization policies in a restricted time frame. (All studies using CFA are referenced in the following section).
We found that studies vary in information on the data used. An appendix of texts analyzed is presented in Duarte Hidalgo (Reference Duarte Hidalgo2013), Fajardo (Reference Fajardo2014) and Paterson (Reference Paterson2011), but not in Elias (Reference Elias2013), Allwood (Reference Allwood2013), or Krook and True (Reference Krook and True2012). Some scholars focus only broadly on the four overall dimensions of diagnosis, prognosis, voice, and call for action, without using CFA's intersectionality component (Fajardo Reference Fajardo2014). Others do use this intersectionality component (Duarte Hidalgo Reference Duarte Hidalgo2013) and present details on the CFA method. It is also not uncommon to see articles saying they are “drawing on CFA” without giving further detail as to how it has been applied (Krook and True Reference Krook and True2012). These illustrative examples testify to the high potential for improvement of the use of the CFA method: while it is rather popular to use it or refer to it, we have found no step-by-step description of its use (at times even the corpus of texts analyzed is not clarified), and there is no discussion of or reflection on the use and potential of the method.
Reflecting on two studies that we contributed to ourselves, we observe that we did not use the full potential of the method either, nor did we include a critical methodology section on the CFA method. In the study of changes in policy frames on gender and migration in 17 Dutch policy texts and debates between 1995 and 2005, Roggeband and Verloo (Reference Roggeband and Verloo2007) apply CFA on MAGEEQ data, focusing on diagnosis, prognosis, voice, and call for action. The findings are explained by relating them to the context dynamics in Dutch politics. This analysis does include attention to intersectionality and voice to a certain degree, by analyzing political parties separately and relating their standpoints to the specific political constellation and climate. The paper lacks methodological reflection, however, and does not further theorize voice through attention to the absence and dominance of particular political actors in policy making.
More recently, we conducted a cross-country analysis of categories used in policy-related texts based on the supertexts and code hierarchies from the QUING database, looking at four policy issues across three types of policy texts (policy documents, laws, and civil-society documents). In our analysis of the labels used for problem holders and target groups in gender equality documents by policy and civil society actors, we zoomed in on the gendering and intersectionalizing of these actors. The database and the code hierarchies made on actors proved to be very useful for this. Our cross-country analysis of 29 countries and at the EU level did not include any contextualization. Moreover, we also did not further analyze those who actually wrote the documents (voice) or those who are given a voice through references in the texts, and we did not specifically analyze the potential reasons or implications of the appearance of gendered+ categories across the three types of texts. While contextualization, voice analysis, and explanatory and impact analysis would have contributed positively to the quality of the analysis, unlocking the full potential of CFA, the article format is unfortunately more suited to addressing a limited set of questions. Therefore we choose to showcase CFA's potential to analyze the range and role of actors addressed by gender equality policies.
From the selected works we discussed above we may conclude that using elements from CFA already provides rich analyses. Scholars using CFA appear to choose to conduct either in-depth small-N studies or more general large-N comparative work. That is, the complexity of the method seems to force scholars to either highlight its ability to provide a deep and specifically contextualized analysis or its comparative potential. This suggests that CFA would benefit most from a rethinking of its “best of two worlds” promise.
CONCLUSION
Concluding our observations and reflections, we argue that CFA's potential is still high: in its various forms CFA enables a systematic and analytic comparison of discursive content in policy making. That is and remains extremely valuable. To further understand the political dynamics of policy making, more is needed, especially combining CFA with contextual and process-tracing data that can explain what happens and why. Given that comparisons over time in one case study often make it easier to collect and present context data, it may be no surprise that we found many case studies and fewer examples of (large) multicountry comparisons.
We also saw that methodological criticism or explicit amendments to the method are rare and would argue that this hampers the chances of the method to grow and develop. A first impression of the list of studies is that there are very few analyses using the data on voice (and agency), and this certainly is the case for the two examples of studies that we contributed to. This means that the critical potential is underused.
We hope that this short reflection essay encourages researchers to continue using CFA, to include more methodological information and discussion when using CFA, to add contextual analysis, and to pay attention to the voice dimension to strengthen the critical potential of CFA. We acknowledge the problem of the lack of a basic open access database that currently hinders non-QUING researchers from contributing to further developing this tool.
