Democracy has become a commodity that can be exported. Promoting democratization or defending human rights are privileged channels for the exportation of political technologies, economic recipes, or juridical models. No longer providing the basis for the critique of power, they have become the main language of global power.
(Guilhot Reference Guilhot2005, 8; emphasis added)Freedom House is a New York based organization that produces annual scores measuring the level of democracy in each country around the world. First published in 1973 as The Comparative Survey of Freedom, these measures rank countries as “free,” “partly free,” or “not free,” depending on where they fall on a scale of 1 to 7 (respectively) along two dimensions – political rights and civil liberties. Today, this Survey is the most widely used indicator of democracy by both academics and the U.S. government alike.
Despite political scientists’ reliance on the now prominent Survey, a full account of its origins is noticeably absent from the discipline’s literature. Most work done on the scores is instead devoted to a discussion of their methodological shortcomings, and suggestions for overcoming them (Bollen Reference Bollen1990, 2000; Coppedge Reference Coppedge2002, 2011; Munck 2002, Reference Munck2009). Freedom House’s own retrospective account of the history of its famed indicator varies markedly from the manner in which the scores were actually crafted. The organization now claims that the scores were “devised by leading social scientists” (plural). Footnote 1 However, both archival documents and publications written by the Survey’s creator, political scientist Raymond Gastil, contradict this account. According to these sources, Gastil alone was responsible for the entirety of the Survey’s content, and he created the scores on the basis of his own “hunches and impressions” (Gastil Reference Gastil1990, 26). Rather than see this as a defect of the Survey, however, Gastil believed it to be one of the tool’s strengths. In his view, the involvement of other individuals in the production of the scores would have impeded streamlined decision-making. Instead of settling potential disputes between area specialists over a specific country’s rating, Gastil had only his own opinion to consider, a process he believed to be much preferable.
In addition to Freedom House’s current claim that the Survey was produced by multiple social scientists, the organization also touts the methodology behind its scores as complex and scientific. The recent remarks of the newly appointed President of Freedom House illustrate this commonly held view. Mark P. Lagon, a Georgetown Ph.D., Footnote 2 said in a press release announcing his appointment: “the research reports published by Freedom House are state of the art. They’re objective, sophisticated and subtle, in the sense that they look at changes in freedoms through a particularly powerful lens. They are the benchmarks for a large number of institutions around the world.” Footnote 3 Freedom House therefore still continues to expend tremendous effort publicly extolling the methodological soundness of the scores.
However, not only was Raymond Gastil the sole individual producing these rankings, but he also admitted to doing so in a non-quantitative way. That is to say, the scores could have just as easily been assigned letters rather than numbers. Attaching a numeric value to a country’s global standing, however, has served to both obscure the scores’ non-quantitative nature and legitimize them as scientific. Gastil’s account of the Survey’s composition is quite different from current notions about the quality and complexity of the scores.
Given the historical evidence and Raymond Gastil’s own words, it is clear that Gastil alone manufactured the Survey without the use of a sophisticated, rigorous, scientific methodology. Paradoxically, these scores are now the most widely used indicator of democracy by both political scientists and American foreign policy makers. How, then, did social scientists and politicians come to regard these measures as valid and reliable when the Survey originated with one man’s “hunches and impressions” (Gastil Reference Gastil1990, 26)? In addition, what are the consequences of the Survey’s rise to prominence for political science and for the production of the global political order?
In this piece I challenge Freedom House’s retrospective description of the Survey as having been the result of a “methodology” of “leading social scientists.” This account is inaccurate, and attributes a scientific, quantitative character to the scores that they really did not possess. In what follows, I trace the development of these scores using archived documents housed in the Mudd Manuscript Library at Princeton University. I argue that political scientists and policy makers alike have overlooked what most would consider to be fatal flaws in the conception and creation of The Comparative Survey because the indicator fits the dominant narrative in political science, namely, a neopositivist approach to research, and lends legitimacy to and reinforces the dominant global order. Because of this fit, a retrospective history of scientific reliability and validity of the Freedom House scores, rather than an accurate account of their origins, has been told. The Survey’s story has been retold in order to bend the measures to the criteria and standards championed by neopositivist social scientists.
I here examine the processes surrounding the creation of the Survey by political scientist Raymond Gastil. I then use archived documents to illustrate the Survey’s double promotion, both in political science and American politics, emphasizing the way in which scientific knowledge is used as an instrument for the promotion and diffusion of democracy. In doing so, I argue that the scope of influence and success of the scores is primarily due to the fact that the Survey fit both the dominant narrative in political science and the U.S. foreign policy regime at the time.
The influence of Freedom House
While the prominence of the Annual Survey conducted by Freedom House in academia is generally accepted, some solid evidence of the importance of these measures is worth providing. In academic social science, one common indicator used to assess the impact of a work is the Social Sciences Citation Index. Footnote 4 One feature of this tool enables researchers to see how many times a specific work has been cited by others within a particular time period, and also provides links to the aforementioned papers. Those works that are cited on numerous occasions, such as Anthony Downs’ An Economic Theory of Democracy (1957), are often said to be cornerstones of the field. The same applies to the yearly measures of worldwide democracy produced by Freedom House. For example, the organization’s Annual Survey, Freedom in the World, was cited 820 times during the year 2009 alone. Footnote 5 In 1978 (the year the Survey was first published in book form), however, the Comparative Survey was cited only 52 times. Footnote 6 That is to say, the reliance of academics on this Survey when doing scholarly work has dramatically increased over time, as illustrated by the number of times the scores are now cited in published works. These figures not only show the prominence of the measures within academia, but also indicate their increasing use in the field over time.
Another way of measuring the influence of works within the discipline of political science is to note their presence on the comprehensive examination readings lists of the top Ph.D. programs in the nation. The works that appear on such lists are generally seen as canonical in various subfields of political science; that is, one must be familiar with them and able to discuss their arguments as part of the process of being awarded a Ph.D. in political science. According to the 2009 U.S. News & World Report, the comparative politics Ph.D. program at Princeton University was ranked second in the country. Footnote 7 While generalizations cannot be made from this one particular case, and I do not provide a survey of all the top programs in the country, Princeton’s comprehensive reading is a useful illustrative example of the influence of Freedom House on academia. Princeton’s 2010 general examination reading list in comparative politics is divided into sections, one of which is devoted to “Democratization.” Footnote 8 This subfield’s list includes references to 17 different articles and books, 13 of which were written after 1978 (when the Comparative Survey first became available in annual edition form), and of those, 8 cite the democracy measures created by Freedom House. That is to say, 61.5 percent of the literature listed in the democratization subfield of Princeton University’s Ph.D. comparative politics comprehensive examination reading list written after 1978 relies in some way on Freedom House’s Comparative Survey. This figure reveals the degree to which Freedom House exerts its influence upon the academic field of political science in the United States.
In addition, two of the most authoritative figures in the fields of comparative politics and democratization studies in particular have attested to the authority of the Comparative Survey of Freedom. Both Larry Diamond (Reference Diamond2015) and Ronald Inglehart (2005, 2011) have recently stated that Freedom House scores are “widely used in both academic research and policy assessments” and are “the most widely used indicator of democratic freedom,” respectively. The fact that these two men, considered heavy weights in the discipline, publicly recognize the hegemonic status of the scores lends further support to my claim.
Academics are not alone in paying attention to Freedom House scores. They are also given a great deal of weight by the U.S. government. One way to gauge the influence of the information produced by Freedom House on the state of democracy throughout the world is simply to examine the frequency with which prominent members of the organization are called to testify before Congressional committees on matters of international importance. For example, in 2011 alone, former Freedom House Executive Director Footnote 9 David J. Kramer testified before Congress on four separate occasions. Before the House Committee on Foreign Affairs, Kramer addressed the state of democracy and freedom in Eastern Europe, as well as speculation surrounding human rights abuses in Belarus. Footnote 10 Twice in 2012 he also advised the Senate Committee on Foreign Relations on U.S.-Russian relations and foreign policy options, as well as those involving Belarus. Footnote 11 On May 6, 2014, Kramer was one of five individuals who spoke before the Senate Committee on Foreign Relations at a hearing titled “Ukraine – Countering Russian Intervention and Supporting a Democratic State.” The other testimonials came from high-ranking representatives Footnote 12 of the Department of State, Department of Treasury, Department of Defense, and the Center for Eurasian, Russian, and East European Studies at Georgetown University. Footnote 13 It should also be mentioned that prior to holding the position of Executive Director at Freedom House, Kramer worked for the State Department for many years. Footnote 14 From March 2008 to January 2009, immediately preceding his position at Freedom House, Kramer served as Assistant Secretary of State for Democracy, Human Rights, and Labor following his nomination by President George W. Bush. Footnote 15 David J. Kramer resigned as Executive Director of Freedom House in October of 2014, and was replaced by Mark P. Lagon in January of 2015. While Lagon is definitely more representative of academia than past Executive Directors (he holds a Ph.D. in Government from Georgetown), he has a history of government service as well. Lagon is a former Deputy Assistant Secretary of State, and he directed the Office to Monitor and Combat Trafficking in Persons between 2007 and 2009. Footnote 16 By mid-July of 2015, Lagon made his first appearance before the Senate Committee on Foreign Relations, giving a report on the state of human rights worldwide. In his testimony Lagon made the case that “our [U.S. national] strategic and economic interests are inextricably linked with the protection and promotion of human rights.” Footnote 17 The aforementioned instances of Freedom House officials testifying before Congress is only partially indicative of the influence of the organization on the U.S. government more broadly. Oftentimes in such Congressional proceedings and governmental reports the results of the Freedom House Comparative Survey are relied upon when making foreign policy and aid decisions, without calling upon anyone from the NGO to testify. Footnote 18
Given the significant impact of The Comparative Survey on both academic work and American foreign policy decisions, a careful study of the measurement tool’s origins is in order.
Raymond Gastil and the creation of the survey
Before discussing Gastil’s claims that the Survey did not utilize quantitative methods, it is useful to first examine what social scientists generally mean by this concept. According to King, Keohane and Verba, “quantitative research uses numbers and statistical methods. It tends to be based on numerical measurements of specific aspects of phenomena; it abstracts from particular instances to seek general description or to test causal hypotheses; it seeks measurements and analyses that are easily replicable by other researchers” (KKV Reference King, Keohane and Verba1994, 3). Throughout their work the authors emphasize the use of “systematic statistical analysis” and “precisely defined statistical methods” in quantitative research (KKV Reference King, Keohane and Verba1994, 4, 6).
While statistical analysis involving numeric figures differentiates quantitative from qualitative research, both forms can be considered scientific if conducted in the appropriate manner. For King, Keohane, and Verba, in order to engage in scientific endeavors, one’s research should possess four characteristics. First, the researcher should seek inference. Second, the researcher should make her/his procedures public knowledge. The authors describe at length the importance of transparent methodologies.
Scientific research uses explicit, codified and public methods to generate and analyze data whose reliability can therefore be assessed … [However,] many proceed as if they had no method – sometimes as if the use of explicit methods would diminish their creativity. Nevertheless, they cannot help but use some method. Somehow they observe phenomena, ask questions, infer information about the world from these observations, and make inferences about cause and effect. If the method and logic of a researcher’s observations and inferences are left implicit, the scholarly community has no way of judging what was done. We cannot evaluate the principles of selection that were used to record observations, the ways in which observations were processed, and the logic by which conclusions were drawn. We cannot learn from their methods or replicate their results. Such research is not a public act (KKV Reference King, Keohane and Verba1994, 8).
Third, in scientific research one’s conclusions are always uncertain. While King, Keohane, and Verba recognize that this uncertainty is a part of all research, this uncertainty should be addressed. These estimates involve standard errors for quantitative researchers and “carefully worded judgements” from qualitative ones (KKV Reference King, Keohane and Verba1994, 152). The authors go on to assert that “[a] researcher who fails to face the issue of uncertainty directly is either asserting that he or she knows everything perfectly or that he or she has no idea how certain or uncertain the results are. Either way, inferences without uncertainty estimates are not science as we define it” (KKV Reference King, Keohane and Verba1994, 9). Lastly, scientific research does not concern particular content, but the methods and rules used by the scholar. Using this general framework for understanding quantitative methods and social science, I now turn to the origins of The Comparative Survey of Freedom.
Raymond Gastil created the Freedom House scores that are used so widely today. A “Harvard-trained” Footnote 19 political scientist, Gastil specialized in the Middle East, and received both Fulbright and Ford fellowships to conduct research in the region. Footnote 20 Later, Gastil taught anthropology courses at the University of Oregon. During his lifetime Gastil worked for the Hudson Institute, the Battelle Seattle Research Center, the United States Agency for International Development (USAID), and the Organization for Economic Co-operation and Development (OECD). He is most widely known, however, for his creation of the Comparative Survey of Freedom in 1973. In 1977 he was officially appointed as the Director of the Survey, Footnote 21 and remained in this position until 1989.
The Comparative Survey of Freedom was first published in the January/February edition of the 1973 Freedom at Issue. This year also marked the unveiling of the Map of Freedom, a 20-foot visual representation of the Survey’s findings that graced the east wall of the Wendell Willkie Memorial Building, the then New York headquarters of Freedom House (Freedom at Issue 1973, 3). Gastil later noted that the idea of creating such a map was a key component of Sussman’s initial conception of the Survey’s purpose. The dividing of countries among three tiers, “free,” “partly free,” and “not free,” was “primarily [used] as a means of summarizing the data for presentation as a Map of Freedom …. The seven-point scales have always been Footnote 22 the heart of the survey, with the three-point generalized status of freedom little more than a heuristic device for printing maps or adding up doubtful totals for free and unfree peoples” (Gastil Reference Gastil1990, 28). This Map of Freedom is a visual representation of the Survey with each country color coded according to their status. It is still available, although it is now an online interactive map (by hovering one’s mouse over a particular country one is able to see their numeric level of freedom). Footnote 23 Originally, however, it was a large display across the walls of Freedom House’s New York headquarters – the Wendell Willkie Memorial Building – until the sale of the building in 1985. The Survey was here described as “an extensive analysis of institutional freedom or repression within each nation” and in this regard “the most detailed examination yet undertaken” (Gastil 1973, 1; emphasis added). Gastil explained its theoretical underpinnings and methodology in fewer than seven pages of the bi-monthly periodical in 1973.
Gastil began his discussion of the new survey by describing the state of global freedom following WWII, which he portrays as relatively bleak (save the “direct military occupation [under which] nations such as West Germany, Italy, and Japan developed democratic systems”) (Gastil 1973, 2). Not only does Gastil applaud this method of democratization; he also explains that he sees the Survey as a means of countering the claim that “we [the United States] have no business interfering in, or even judging, the activities of foreign governments in their countries” (Gastil 1973, 2). The Survey therefore rejected this “scientific relativism” in favor of an overt agenda aimed at democratization on a global scale – at least the advancement of a limited, procedural conception (Gastil 1973, 2). Gastil’s explanation of his intentions help to illustrate the trajectory of the Survey, which would become a useful tool in the emerging narrative of global democracy promotion.
As to the actual production of the scores, archival sources (including correspondence, the minutes of meetings, etc.) make it clear that Raymond Gastil alone was responsible for the content of the Survey. Though the Comparative Survey was the brainchild of Leonard Sussman, who had been Executive Director of Freedom House since Field’s retirement in 1967, Gastil actually produced the scores. The division of labor between Sussman and Gastil was quite similar to that of a salesman and a manufacturer, respectively. Sussman was the visionary, Gastil was the engineer.
Correspondence between Sussman and Gastil reveals Gastil’s role as the sole creator of the measures. Because Sussman was the official head of Freedom House and the one who came up with the idea of creating the Survey, Gastil often explained his rationale for the ratings to Sussman. The language he used in doing so, however, makes it clear that Gastil alone was producing the scores, even as late as 1983. In correspondence between the two, Gastil talked about making changes to the Survey in the first person singular (“I” changed these scores, etc.). Footnote 24 Further, Sussman presented Gastil as the creator of the Survey during meetings of the Freedom House Executive Committee. On January 30, 1978, it was reported in the minutes of such a meeting “Mr. Sussman stated that Dr. Gastil had completed the manuscript known by its working title, ‘Survey Yearbook’.” Footnote 25 Perhaps some of the most definitive evidence that Gastil was the only person generating these measures was his reaction to a specific individual who wrote Sussman about the Survey’s rating of Southeast Asia. Apparently, Sussman had received a number of letters from Dr. William R. Kintner, then President of the Foreign Policy Research Institute in Philadelphia. On February 1, 1979, Dr. Kintner wrote that “sometime ago I asked Kamol Somvichian, who is currently on the faculty of Swarthmore College, to give me his appraisal of comparative political freedom in Southeast Asia. Enclosed is a copy of his letter to me dated January 30.” Footnote 26 Sussman passed the correspondence on to Gastil as the Director of the Comparative Survey, and Gastil responded on February 5, 1979: “Dear Bill, Leonard passed me another letter [from] (Somvichian) Footnote 27 relative to the Thai rankings. I have several thoughts in this regard. First, why do you keep sending these to Leonard? If you are sincerely interested in offering information for the Survey why don’t you send the information directly to me?” Footnote 28 This frustrated exchange not only illustrates that Gastil was the only one creating the Freedom House scores at this time, but also that Sussman clearly was not. Footnote 29
After Raymond Gastil left Freedom House in 1989, he published an article in Studies in Comparative International Development the following year entitled “The Comparative Survey of Freedom: Experiences and Suggestions.” He described the Survey in general as “a loose, intuitive rating system for levels of freedom or democracy” (Gastil Reference Gastil1990, 25). Gastil admitted that “it was years before its author [himself] understood that the survey was essentially a survey of democracy” (ibid., 26). That is to say, while Gastil initially understood the scores to be a measure of global freedom, he later came to the realization that the ranking system represented the level of democracy within countries. Therefore, though Gastil did not originally approach the Survey as measuring democracy, he later came to the conclusion that the concept was synonymous with freedom. Gastil also claimed that the Survey may not even truly be capturing democracy (ibid.). In particular, “detailed comparative literature on political systems and behavior opens up problems that the relatively superficial survey has not addressed” (ibid.; emphasis added). That is to say, simply assigning a numeric score to a country does not necessarily tell one much about the actual status of freedom in that country. He went on state that labeling all Western democracies as 1-1 Footnote 30 (or “Free”) obscures the differences between them. This prevents researchers from developing a more substantive definition of democracy, or examining the extent to which said country is actually free. This is a major criticism of the Survey, given that Gastil has stated at various points in time that the primary purpose of the scores is comparison between countries.
Examining in detail the literature on the United States, Great Britain, Sweden, Switzerland, or Japan, raises many doubts as to the degree to which political competition is really open and fair, or ‘meaningful.’ Although the media appear ‘fair,’ are they not constrained by an unspoken consensus representing powerful economic and bureaucratic forces in the society that makes effective criticism and mobilization for change impossible? Similarly, how do we compare party systems? In most party systems, the choice of candidates is made the top down, with little more than ratification at local levels. The United States has been the only major democracy with a primary system. Yet in our system, the advantages for incumbents in the House of Representatives are such that only flagrant public abuses or incompetence makes possible an effective challenge. Our ineffective party system and lack of party discipline means that most senators and representatives cannot be held responsible by the voters for the success or failure of successive government. To weigh such criticisms and differences in a more detailed survey with more rating levels would require more than marginal growth in the present resources of the survey (ibid., 26–27).
Also contained within this article, published in a scholarly journal, is Gastil’s public admission that the Survey is both cursory and non-quantitative. In fact, Gastil deemed the Survey “superficial,” “relatively superficial,” and “highly personal” at multiple points in the piece (ibid., 26).Footnote 31
Gastil’s explanation of the methodology of the Survey in this article reinforces his earlier claims, particularly his response to the criticisms of Jonathan Power in 1977, that it is both un-scientific and superficial. While Gastil never uses the exact term “un-scientific” to refer to the creation of the scores, saying he did not employ quantitative methods and that the Survey is “superficial” implies this in my view (ibid.). In a letter to Sussman dated November 23, 1983, Gastil addressed recent changes he made to the Survey. He writes: “Obviously there is no scientific means of perfectly placing countries that vary in so many ways and for which we have such incomplete information … my hope would be to be right not on the absolute placing of a country but on its relative placing.” Footnote 32 Gastil went on to add that “we [Freedom House] have never claimed that every judgment on every country is accurate … In regard to the seven point scales our standard has been that no country should be more than one point off what a more careful analysis would determine” (ibid.) These comments also imply that Gastil was assigning scores to countries based on his perception of a limited amount of information.
Gastil’s comments about the Survey seven years later echo those he made in this previous correspondence. In 1978 Gastil responded to a request from an educational center to use the Comparative Survey by stating: “It should also be clear in your presentation that our work is quantitative only in the sense that numbers are used to designate points along continua. The categories would as well be labeled A – G as (1) to (7). The important point is the relative rather than absolute standing of nations.” Footnote 33 In another letter the following year Gastil explicitly stated that “The Survey does not employ quantitative methods fashionable in political science.” Footnote 34 In 1990 Gastil expands on his description of his creation of the Comparative Survey, but he does not necessarily cover any new critical ground. The difference, however, is that this later admission was made publicly, rather than in private correspondence. Gastil did not publicly portray his process of arriving at the scores as “highly personal,” “superficial,” and “intuitive” when he was still working in an official capacity at Freedom House (Gastil Reference Gastil1990, 25–26). Instead, Gastil was quiet on the subject and let Sussman take the lead as marketer of the Survey. And Sussman pitched the Survey as “a universal standard for measuring the level of human rights in every nation and dependent territory.”
While the ratings were the result of Gastil’s intuitions, which he later freely admitted, he also seemed to find nothing wrong with this process. Instead, he takes comfort from the fact that “the few times that the monograph or journal literature has actually been researched [in order to compare the survey results to more in-depth country reports], remarkably little has been found that affected the ratings” (ibid., 26). Again, Gastil readily accedes these points, perhaps due to the fact that he did not see the Survey’s rankings as quantitative because he did not employ any formal methodology in order to arrive at them.
In reflecting on the origins of the Survey’s data in 1990, Gastil pondered, quite rightly, whether or not any undertaking of this kind is possible without making subjective decisions about a particular country’s situation, and democracy in general.
Are we not after all engaged in a form of judgment? We could assume that we are not judging, that we are simply setting up a list of criteria, devised at a particular point in history, and seeing how other societies measure up to them. But if this is so, then how are we going to mark the divisions on our ruler when it involves societies that seem to excel, or believe they do, our top marks? (ibid., 41).
Clearly, the survey’s methodology is and has always been based on “judgment calls” (ibid., 28). Gastil used Poland’s rankings to illustrate this point.
For example, one year Poland might be ‘6-5, Not Free,’ while the next it might be ‘6-5, Partly Free.’ The author had a picture in his mind as to what a five and six were on each scale, and also a picture of what Not Free and Partly Free were. Since these were all continuous scales, the author reasoned that a ‘high five’ and a ‘high six’ might be considered Not Free, while a ‘low five’ and a ‘low six’ could be Partly Free. Footnote 35 The logic was fine, but the result confusing (Ibid.; emphasis added).
Again, Gastil’s description of his process in creating the scores, based on “a picture in his mind,” clearly reveals that he alone was chiefly responsible for the Survey. Even if Gastil was using government sources to inform his decisions about the scores he assigned to countries (as he claims), he was certainly not an “expert” on every country, let alone the region to which they belonged. At best, such a method would lead to perfunctory choices about how to rate countries with which one had only a passing familiarity.
These judgment calls are also reflected in the leeway granted an investigator who sits down with the “simplest of checklists” Footnote 36 in order to determine a country’s rating (ibid., 27). Below are the checklists included in the article for both political rights and civil liberties (see Tables 1 and 2). Gastil claimed that these were used to assess the level of freedom in the countries included in the Comparative Survey.
Table 1. The Comparative Survey’s Checklist for Political Rights
Adapted from (Gastil Reference Gastil1990, 30). In this article Gastil did not go into detail regarding what the features on this checklist meant for him, or how he determined these features were indeed present in the countries examined by the Survey.
Table 2. The Comparative Survey’s Checklist for Civil Liberties
Adapted from (Gastil Reference Gastil1990, 36–37). These criteria loosely follow what Dahl identifies as “Some Requirements for a Democracy among a Large Number of People,” in a table in Polyarchy (Dahl Reference Dahl1971, 3). These requirements include “the opportunity to: formulate preferences; signify preferences; and have preferences weighted equally in conduct of government,” and their corresponding “institutional guarantees” (Dahl Reference Dahl1971, 3).
Gastil also addressed critics of the Survey who over the years called for the creation of a more accurate, scientific ranking system. He claimed that “the unevenness of the sources of available information over time and its incompleteness has been a major reason why the survey never acceded to the occasional suggestion that a more transparent system be devised in which many subindicators would be rated numerically, and then summed up for an overall rating” (ibid., 29).
Gastil even briefly responded to the criticism of a right-wing bias, particularly relevant during the time he was writing. This claim, he argued, “is based on opinions about Freedom House rather than a detailed examination of survey ratings” (ibid., 26). He clearly differentiated between the organization and the Survey, which remained under his direction until 1989. Footnote 37
Gastil partly blames the lack of institutional resources (personnel, finances, etc.), at least in the early stages of the project for the shallowness of the Survey.
With little or no staff support, the author has carried out most of the research and ratings. Lack of a research staff is not as overwhelming a fault as it might appear. By working alone the author has not had to integrate the judgments of a variety of people. The hunches and impressions that are so important in a survey of this kind would be almost impossible to keep on the same wave lengths if one had an Asianist, Africanist, and Latin Americanist to satisfy before the ratings were finalized for each year (Ibid.).
Gastil therefore paints a very different picture about the origins of the Survey than do Freedom House itself, most academic social scientists, or the U.S. government. For example, Freedom House, in the “Our History” section of its website, currently states the following:
In 1973, Freedom House launched what is now its flagship publication, Freedom in the World, an annual survey of global political rights and civil liberties. Employing a methodology devised by leading social scientists, the survey analyzes and rates every country in the world on a series of fundamental freedom indicators. Its results always highly anticipated, it provides policymakers, journalists, and the public a comprehensive view of the global state of freedom [emphasis added]. Footnote 38
Given what Gastil had to say about the creation of the Survey, making it clear that he alone was responsible for all of its content, it seems quite deceptive on the part of Freedom House now to claim that the scores were “devised by leading social scientists ” (plural). This description presents the Survey’s creation very differently than Gastil did in his Reference Gastil1990 article, and lends a false air of credibility to the scores as a result. Not only was Gastil the only person coming up with these rankings, but he admitted that he did so in a non-quantitative sense (they could just as well have been assigned letters rather than numbers). Freedom House’s retrospective description of the Survey as having been the result of a “methodology” of “leading social scientists” attributes a scientific, quantitative character to the scores that they really did not possess.
The manner in which Gastil created The Survey clearly falls short of the general conception of quantitative methods and scientific research expressed by King, Keohane, and Verba. Gastil acknowledged this both privately and, after his departure from Freedom House, publicly. The numeric indices were, at best, a show-of-science, and this is precisely what would become important for their use and acceptance among scholars. Because the scores appeared to employ social science techniques favored by behavioralists (numeric measurements being among the more important of these), they were adopted by the community.
Even if one could claim that the Survey has become more quantitative or “scientific” over time, the way in which the measures originated would still undermine their credibility given that a country’s previous years’ ratings highly influence its current standing. Gastil explained this aspect of the measures:
The work of the survey in subsequent years has consisted of following news about a country in a variety of sources, and occasionally changing ratings when the news did not fit the established rating level. In effect, the author developed rough models in his mind as to what to expect of a country at each rating level, reexamining his ratings only when current information no longer supported this model (Gastil Reference Gastil1990, 27; emphasis added).
Ultimately, “the original intention was to produce, with relatively few manhours, an orienting discussion of variation in levels of freedom” (Gastil Reference Gastil1990, 25). The focus, then, was not on creating an authoritative measure of global democracy. The Survey, which was initially created in order to start conversation and be easily translated into a map, has become one of the discipline’s principal measurements of democracy. How, then, did social scientists come to regard these measures as valid and reliable when the Survey originated with one man’s “hunches and impressions” (Gastil Reference Gastil1990, 26)?
Double promotion: the scores, political science, and American politics
Though the promotion of the Survey certainly increased its visibility among government officials and academics who could help catapult the indicator to fame, the fact that the scores fit dominant narratives in both fields proved far more important to their success.
With respect to the first arena, Freedom House scores and the conception of democracy they embody represent the views dominant among the foreign policy makers of the United States government (Robinson Reference Robinson1996). Democracy promotion is purported to be a primary motive of American actions abroad (Guilhot Reference Guilhot2005); therefore, the concept itself wields a great deal of power – wars have been carried out in its name. The assignment of numbers to the Freedom House scores has served the American government because this guise of scientific objectivity allows their use in the promotion of democracy across the globe (in whatever form that might take) to go unquestioned. In turn, Freedom House’s relationship with the U.S. government has helped to increase the visibility and use of its democracy indicator.
Replacing the Balance Sheets of Freedom Footnote 39 with The Comparative Survey, a numeric indicator, allowed Freedom House scores to fit into the dominant narrative in American political science as well. That narrative, behavioralism, emphasizes an approach to the study of politics that utilizes large-N datasets in order to reveal causal relationships within social phenomenon and (ideally) arrive at general laws (Adcock Reference Adcock, Robert Adcock and Stimson2007; Hauptmann 2016).
Democracy promotion and the global political order
Promotion of a particular brand of democracy as a part of American foreign policy wields ideological power. The advancement of this conception by American academics and organizations alike, particularly those involved with important international forums such as the United Nations, further embeds this specific definition in the global democracy promotion network. When academics and democratization experts, including Freedom House officials, are consulted on and approve of democracy promotion strategies in American foreign policy, these strategies gain an air of legitimacy and scientific objectivity.
There has been a close “fit” in the post-World War II period between US foreign policy and the mainstream academic community. In particular, modernization and political culture/development theorists provided intellectual guidelines and legitimization for foreign policy, and also contributed important theoretical and practical elements – including developing a new generation of democratization theory – to the development of the new political intervention (Robinson Reference Robinson1996, 42).
Robinson (Reference Robinson1996) argues that democracy promotion provides a seemingly ideologically neutral banner under which the United States has carried out a foreign policy aimed at maintaining and perpetuating its own interests abroad. Specifically, he examines relations between the United States and the Third World in the 1980s and 1990s, and the manner in which specific U.S. policies and the global economic system have disadvantaged the latter.
Using an approach to critical theory informed by Gramsci’s concept of hegemony, Robinson illustrates how democracy promotion is linked to “consensual domination” globally (ibid., 6). In place of its prior support for dictatorships in the Third World that were amenable to its interests, the policy of the United States government has now become one of “democracy promotion.” Robinson’s discussion of the rise of neo-conservatism, and the movement’s success in pursuing what it deemed American national interests abroad under the banner of democracy promotion, is essential to my argument that the changes that took place at Freedom House under Executive Director Leonard Sussman in the 1970s are reflective of this larger political context in the U.S. at the time. Democratization has lent American foreign policy the appearance of humanitarianism, and has allowed the U.S. to maintain “control of the world’s resources, labor, and surpluses,” while substituting the coercive means for doing so with more consensual ones (ibid., 16).
Robinson attributes some of this shift in American foreign policy to globalization and the accompanying increase in social interactions between people that often leads to a re-imagining of political structures. The United States, becoming aware of the impetus for political change created by globalization, purposefully began to execute a foreign policy centered on democracy promotion in order “to gain influence over and try to shape their [democratic movements’] outcomes in such a way as to preempt more radical political change, to preserve the social order and international relations of asymmetry” (ibid., 318–319). The logic behind Robinson’s claim here is that globalization produces “highly fluid social relations, ‘stirs’ masses of people to rebel against authoritarian forms of political authority, and thus calls forth new political structures to mediate social relations within and between nations in the world system” (ibid., 318). Radical political systems are those that would threaten the interests of the elite – the class of individuals that currently benefits from the neoliberal global structure. The United States, Robinson argues, honed in on the push for change globalization produced among the populace in Third World countries in the 1980s, and refocused American foreign policy under the guise of democracy promotion. In reality, however, these policies sought to maintain the inequality of the current world order. In an examination of four case studies (Nicaragua, Haiti, Chile, and the Philippines), Robinson illustrates how interventions on the part of the United States have actually stunted the process of democratization in Third World countries. In limiting and controlling movements toward more democratic governments in these regions, the United States has been able to ward off attempts to promote dramatic changes within Third World governments that would threaten the interests of the “transnational elite” (ibid., 319).
According to Guilhot, democratization experts are part of this transnational elite, acting as “key regulatory actors of globalization” (Guilhot Reference Guilhot2005, 7). Democracy promotion has become professionalized and distinctly associated with American ideals. The advancement of democratization “no longer provid[es] the basis for the critique of power … [but has] become the main language of global power” (ibid., 8). Democratization rhetoric and policies are another means by which developed countries exercise economic and political power over developing ones. The ability of American political scientists to advocate a specific conception of democracy as the appropriate goal for countries across the globe has allowed these “double agents” (to use Guilhot’s striking term) to promote distinctly ideological goals under the guise of scientific objectivity. The use of numerical scores presented as objective findings has extreme ramifications, as “numbers have often been an agency for acting on people, exercising powerFootnote 40 over them. … Where power is not exercised blatantly, it acts instead secretly, insidiously” (Porter Reference Porter1995, 77).Footnote 41 Freedom House scores are used by the media, academics, and government institutions because they represent “expert” information; however, that information is both funded and approved by “primary sources and agents of power”Footnote 42 (Chomsky Reference Chomsky and Herman1988, 2). Using a numeric form cloaks the subjectivity of measures by protecting them from charges that they are ideological.Footnote 43
Guilhot’s (Reference Guilhot2005) The Democracy Makers makes a compelling argument regarding the way in which democracy promotion became the dominant discourse in international relations. The author draws special attention to the role of “democracy makers” in this process – those academics and human rights activists who have become “professionals” whose expertise is sought after and are gainfully employed by the democratization industry. Freedom House officials, and in fact the scores themselves, are often cited and called upon to lend legitimacy to American foreign policy decisions, and are therefore an important part of this professional democratization network. In particular, Guilhot is interested in the manner by which a discourse that was often used to challenge dominant regimes is now used in order to legitimize them. Whereas radical political movements once used the language of democratization to protest the rule of the global elite, this language is now used to suppress such movements.
Guilhot’s work is largely about the process by which democracy came to be more than just an international norm, but something to which everyone is entitled, “a universal right” (Guilhot Reference Guilhot2005, 2). In order to understand the institutionalization of democracy along these lines, Guilhot explores the role of those who both promote and protect the concept. This group, however, has branched out from the traditional advocates (such as human rights activists, dissidents, and NGOs) to include “think tanks, philanthropic foundations, state administrations, international organizations such as the United Nations or the World Bank, private consulting firms, professional associations, activists, lawyers and, last but not least, academic scholars” (ibid.). All are important actors whose contributions have helped to build a dominant democratization discourseFootnote 44 – one associated with a particular set of concepts and language. Powerful networks of democracy promotion experts have been created as a result. In this way
NGOs contribute to the establishment and the enforcement of global standards. In the field of political and civil liberties or economic corruption, such standards already exist and are in part policed through the ‘rating’ practices of Freedom House or Transparency International, two NGOs which rank countries according to their record in these fields and which have managed to achieve international credibility. Every year, the publication of their rankings makes it to the headlines of the most important newspapers (Ibid., 4).
Guilhot concludes that democracy networks have therefore become used as instruments of control by the dominant global powers, particularly by the U.S. government. In fact, “the success of this agenda lies precisely in its ambivalence, that is, in its capacity to lend itself to different interpretations and to accommodate different strategies, whether those of genuinely concerned activists and dissidents, or those of State Department planners” (ibid., 9).Footnote 45 Therefore, what has been seen by many to be a newly developed kind of scientific expertise is in fact a construct used to maintain and perpetuate the power of global “organized interests” (ibid., 7).
An abundance of energy was expended by Executive Director Leonard Sussman to ensure that Freedom House became one source of “scientific expertise” used in democracy promotion efforts. Insofar as the division of labor between Gastil and Sussman worked, Sussman was responsible for marketing the Survey. This included ensuring that the measures garnered the attention of the media, public officials, and academic institutions. The latter group’s endorsement of the Survey provided it with an important legitimacy that Sussman would use when pitching the scores to government officials for use in foreign policy decisions. Additionally, Sussman had to guarantee financing in order to showcase the Survey in various ways, and to continue the project. In a memo Sussman wrote to the Executive Committee on October 6, 1976, Sussman appealed for greater financial backing of the Survey, claiming that the “number and diversity of places to air our views are limited only by staff and dollar resources. It is often deeply frustrating to resist accepting some commitments. [Sussman goes on to recommend] several major steps we could take that would immediately enhance our effectiveness and visibility.”Footnote 46 At this point in time Gastil was still working in some capacity at the Battelle Research Center in Seattle, and one of the proposals Sussman put before the Committee was a full-time position at Freedom House for Gastil as the Survey’s Director. Sussman also appeared before Congress and wrote to several public officials (including then Secretary of State Henry Kissinger), inquiring about the government’s need for a measure of human rights around the world. He attempted to peddle the Survey for this use (in the mid-1970s), illustrating his role in the project as marketer.
On Thursday, December 4, 1975, Leonard Sussman testified before the Subcommittee on Foreign Assistance and Economic Policy of the Senate Foreign Relations Committee.Footnote 47 Generally speaking, the amendment about which Sussman testified dealt with a proposal to link foreign aid with the level of human rights in recipient countries. Sussman’s testimony here was both interesting and important in a number of ways. First, Sussman, while not necessarily being critical of governmental policy, argued against this linkage. That is to say, the Executive Director of Freedom House, an organization that purports to be one of the most important advocates in the advancement of freedom and basic human rights globally, suggested that the amount of aid a nation receives from the United States not be dependent in any way on the level of freedom among its citizens. He claimed to take this position for a number of reasons. First, “in my view,” the Board of Trustees of Freedom House would largely be divided on the issue.Footnote 48 Secondly, he urged the government to reconsider linking assistance with human rights violations, as “there may be overriding American national security considerations.”Footnote 49 This stance seems counter to the original purpose and agenda of Freedom House, and reflects the organization’s increasing concern with U.S. national security during Sussman’s tenure, one that would override its commitment to democracy.Footnote 50 Thirdly, “a consistent pattern of gross violations may not be established, though occasional or even aberrant examples of such violations may be apparent.”Footnote 51 That is to say, Sussman asserted that a few sporadic human rights abuses were not sufficient cause to deny a country U.S. foreign assistance. Ultimately, much of what Sussman said before Congress on the issue ran counter to the democratic principles with which Freedom House had long been associated. Rather than speak at length about this particular amendment, which he never fully endorsed, Sussman used his testimony to advance the Comparative Survey. What is truly needed, he argued, is a governmental office that can make such determinations on the level of human rights on a country-by-country basis. Enter the Comparative Survey.
After giving a brief background on Freedom House, Sussman tailored his representation of the Comparative Survey and the organization itself to what the State Department may have been looking for – a seemingly legitimate way to make aid allocation decisions. “By means of our Comparative Survey of Freedom we have established and continually strive to refine a universal standard for measuring the level of human rights in every nation and dependent territory” (ibid.). Sussman argued that the Survey measures the concept of human rights, and he used the term interchangeably with freedom and democracy. Sussman claimed that the Comparative Survey established “criteria for a standard of individual freedom that goes beyond recording gross violations [of human rights] and describes as well the degree of movement toward a generally free society” (ibid.). These criteria of freedom could therefore be used to ascertain the level of human rights in any given country. He was also careful to argue that the Survey does “not set forth the American political system as the standard, nor do we assume western liberal democracy is necessarily the highest form of political development, though it does presently provide varied forms of comparatively free societies” (ibid.; emphasis added). In addition, Sussman presented the Survey as an objective measure of freedom. “It has been argued that it is not possible to create an objective standard to test the conduct of all nations with regard to conditions of human rights. There are, it is said, vastly differing cultures, traditions and juridical, social, economic and political systems. Freedom House rejects this view” (ibid.). In his sales pitch Sussman clearly articulated what he believed the State Department needed in order to properly make such decisions as the amendment in question would require – The Comparative Survey of Freedom. At the same time, Sussman was also careful to claim that the Survey was not attached to an American conception of democracy. However, Sussman’s explanation of his motives behind this testimony before Congress illustrate the organization’s prevailing regard for U.S. national interests.
Sussman continued his pitch by writing to Henry Kissinger (then Secretary of State). In a letter dated March 29, 1976, Sussman stated that, contrary to what was called for in the Congressional amendment on which he testified, Freedom House could not sanction the withdrawal of aid to influence political outcomes in other countries. However, Freedom House was “particularly interested in the procedures by which the Department of State, presumably through a new Office of Human Rights, will inform the Congress of the level of human rights in the relevant countries.”Footnote 52 This new “law, however, would empower the Department to secure relevant data from nongovernmental as well as governmental sources. Freedom House, we submit, may be of particular service in this analytical and reportorial process” (ibid.) Again, Sussman began his solicitation with a retelling of Freedom House’s history as an organization long dedicated to freedom and democracy. The majority of his letter, however, focused on the merits of The Comparative Survey, which he argued
is indeed recognized by scholars in Europe, Japan and, of course, throughout the United States. A recent study at the Institute of Political Science, University of Florence, Italy concluded that our Survey is the most reliable empirical analysis of freedom. A Georgetown University monograph comes to the same conclusion. A number of your academic colleagues at Harvard, Columbia, Berkeley and Stanford regularly employ our analyses and express favorable judgments of its value. … The press in the United States, Europe and Asia, moreover, is accustomed to accepting our Survey as the established criterion of the level of freedom around the world. (Ibid.; emphasis added)
It is clear here that Sussman saw a great opportunity for the Survey, and a chance to form a lasting relationship with the United States government. Sussman also added that “Of no small consequence, foreign governments often turn to us for criteria and judgments” (ibid.; emphasis added).Footnote 53
The State Department did indeed respond to Sussman’s push for the Survey’s use in aid allocation decisions. J.M. Wilson, Jr. (the U.S. State Department’s Coordinator for Humanitarian Affairs),Footnote 54 who responded to Sussman at Kissinger’s request, clearly envisioned a relationship in the future between the government agency and Freedom House. Specifically, Wilson told Sussman that the Comparative Survey would be used by the State Department to determine whether or not countries would receive (or continue to receive) U.S. aid.
We accordingly anticipate many difficult problems in implementing the human rights provisions of the International Security Assistance and Arms Export Control Act of 1976. We appreciate the extremely useful work that Freedom House has been doing for many years in monitoring the status of political and civil freedoms around the world and we hope to maintain a close professional relationship with your organization. It is our present intention to use Freedom House’s reports, as well as those of Amnesty International, the International Commission of Jurists, information available from the UN Human Rights Commission and many other sources in the public domain in dealing with our human rights reporting responsibilities as suggested in the new legislation.Footnote 55
Wilson closed the letter by stating that the Deputy Coordinator for the Human Rights Office would be visiting New York soon, and Wilson had asked him to meet with Sussman in order “to gain the insights I know Freedom House has to offer us” (ibid.). Leonard Sussman’s persistence and “salesmanship” had clearly paid off, and the Survey’s influence on important decision-making apparatuses in the United States government was on its way to being established.
Fitting into the dominant narrative in political science
The rise of behavioralism in the discipline was prompted by some scholars’ call “for analytical frameworks that could give systematic order to the findings of empirical work” (Adcock Reference Adcock, Robert Adcock and Stimson2007, 155). Though there were certainly earlier pushes for the use of a more streamlined, scientific methodology in the social sciences (Charles Merriam, for example), these arguments began to firmly take ground in political science in the early 1950s (Adcock Reference Adcock, Robert Adcock and Stimson2007). Early behavioralists felt that the discipline needed reform, and in order to accomplish this reform “it was believed helpful, as Easton put it, to leave ‘premature policy science’ behind and adopt a ‘pure science’ conception of political science” (ibid., 155).Footnote 56 The most successful endeavor of the behavioral movement proved to be its “promotion of techniques for the systematic collection and analysis of data” (ibid., 156). In other words, behavioralism’s lasting legacy is more an emphasis on the manner in which research should be conducted, rather than particular subject matter (Hauptmann Reference Hauptmann2012, 154).
The manner of this research has perhaps been best codified in the work by King, Keohane, and Verba, considered by many to be crucial for understanding the way in which social science “ought” to be conducted – Designing Social Inquiry (1994). In this work the authors (often referred to collectively as KKV), “argue that the logic of good quantitative and good qualitative research designs do not fundamentally differ” (KKV Reference King, Keohane and Verba1994, ix). In their first chapter, entitled “The Science in Social Science,” the authors posit that the “precisely defined statistical methods that undergird quantitative research represent abstract formal models applicable to all kinds of research, even that for which variables cannot be measured quantitatively” (ibid., 6). KKV’s “rules of inference” represent the dominant narrative in political science today, a narrative whose origins lie in the rise of behavioralism (ibid.).
Though behavioralism had been the dominant narrative in American political science for quite a while then, by the time The Comparative Survey of Freedom was first published in 1973, the indicator clearly reflected the mode of social research the movement had advocated for. What the scores ultimately represented, a large-N dataset, would become utilized by many social scientists. Freedom House would become a forum for data production and collection. One of the first organizations dedicated to the dissemination of such data was ICPSR, whose staff recognized the Survey’s academic potential almost immediately.
The Inter-University Consortium for Political and Social Research was established in 1962; it is now best known “as the data dissemination arm of the American National Election Studies (ANES).”Footnote 57 This institution has grown to become one of the most important sources of data for scholars in the social sciences, and also functions as a training site. Its summer programs teach students to conduct research in a manner consistent with the use of large datasets.Footnote 58 It is not surprising, therefore, that Sussman corresponded with members of the institution in the early years of the Comparative Survey.
Prior to the Survey’s publication in book form, Leonard Sussman was approached by Robert Beattie, the then Assistant Director of the International Politics section at ICPSR. Thus, while Sussman definitely anticipated the importance of the Survey for Freedom House’s relationship with the United States government, its use as data for social scientists was not an immediate motivation for the creation of the scores. In 1976 correspondence, Sussman stated that “in a less direct way, our Survey has an impact by virtue of the data it provides for scholars in the human rights field (admittedly few in number) and political and social scientists generally.”Footnote 59 Therefore, it was actually the data collecting institution of ICPSR that first saw the niche these ratings could fill in the discipline. Dated March 20, 1977, the correspondence represents a recognition on the part of ICPSR that “this project is one of great importance for scholars and policy analysts interested in monitoring the international system,” and suggested that
perhaps we might cooperate with your organization in making the freedom data more widely available to researchers around the world. The ICPSR is an organization composed of some 250 institutions of education and research in the U.S. and abroad. One of our activities is to acquire social science data for rediffusion in computer-readable form to individuals at member institutions and elsewhere.Footnote 60
The institution clearly saw the value of disseminating the Comparative Survey, and Beattie even offered, if Freedom House had not already done so, to “punch the data onto IBM cards, and … add standard country codes to the data so they may be merged with other data. … Moreover, as you continue this project over time analysts will have available a file for time series analysis. All in all the prospects seem very exciting” (ibid.). This correspondence demonstrates that at a very early point, even before the Comparative Survey emerged from under the wings of Freedom House’s journal Freedom at Issue to appear in book form, institutions such as ICPSR recognized the utility of the scores for other researchers. The Survey, a ranking system that Gastil freely admitted after his departure from the organization was based largely on his own intuitions, was being set up for large scale dissemination as early as 1977. ICPSR was most interested in obtaining the Survey so that it could be compared alongside other measures and variables. The fact that the Survey was collected over the span of many years made it amenable to time series analysis as well. The Comparative Survey conducted by Gastil between 1972 and 1976 (prior to its publication in book form) eventually became available through ICPSR in 1984.Footnote 61
Conclusions and implications
In 1973, two men set Freedom House on a course that would ultimately lead to its international renown and far-reaching influence. The Comparative Survey of Freedom, conceived by Sussman and created by Gastil, has now become the benchmark for global standards of democracy. While Sussman certainly had such grand ambitions in mind when he came up with the idea for the scores and subsequently marketed them to American politicians and social scientists, his motivations and actions alone cannot account for the prominence of the Survey. Though this promotion of the Survey certainly increased its visibility among government officials and academics who could help catapult the indicator to fame, the fact that the scores fit dominant narratives in both fields proved far more important to their success.
Those who would argue that the Freedom House scores are not as prominent within the field of political science as I here assert have only to look to Larry Diamond’s book In Search of Democracy. Published in July of 2015, the work makes a number of generalizations about global democratic trends. “Drawing on the most recent data from Freedom House, it assesses the global state of democracy and freedom, as of the beginning of 2015, and it explains why the world has been experiencing a mild but now deepening recession of democracy and freedom since 2005.”Footnote 62 Diamond is considered to be one of the leading authorities on the study of democratization. A professor at Stanford University, he oversees the Center on Democracy, Development, and the Rule of Law (CDDRL), and is a co-founding editor of the Journal of Democracy.Footnote 63 He has done consulting work for both the International Forum for Democratic Studies of the National Endowment for Democracy and the United States Agency for International Development (USAID). Diamond has edited or co-edited 36 books on democracy.Footnote 64 And although Larry Diamond is but one scholar, he is prominent in the field of democratization studies. Most notably, his position as editor of the Journal of Democracy allows him to influence the type of work that appears in “the world’s leading publication on the theory and practice of democracy.”Footnote 65 This journal “ranks among the most influential of social-sciences journals; it is one of the most cited social-sciences journals, according to the Journal Citation Reports compiled by the Institute for Scientific Information (the Social Sciences Edition covers 1,800 leading social-sciences journals)” (ibid.). In addition to Diamond’s Reference Diamond2015 book, the Journal of Democracy also published a book that year titled Democracy in Decline? in honor of the journal’s twenty-fifth anniversary. The work is a collection of short essays written by “eight of the world’s leading public intellectuals and scholars of democracy.”Footnote 66 Of the six chapters in this book, all but one cite the Freedom House scores as good, reliable data. The chapters that refer to the Freedom House scores were written by Francis Fukuyama, Robert Kagan, Philippe C. Schmitter, Steven Levitsky and Lucan Way, and Larry Diamond.Footnote 67 If Larry Diamond and other leading democracy scholars are currently using Freedom House scores in their work, this is a strong indication that these measures continue to remain extremely relevant.
In this piece I have sought to draw attention to the fact that scholars often treat measures as ahistorical when in reality they are not (Oren Reference Oren2003; Roelofs Reference Roelofs2003; Parmar Reference Parmar2004). In addition to the Comparative Survey, many other measurement tools used by political scientists should be historicized as well. Oren (Reference Oren2003) briefly addressed the ideological orientation embedded in the POLITY dataset, which was directed by Ted Gurr of the University of Maryland who also carried out other studies specifically “commissioned by the CIA” (Oren Reference Oren2003, 171). While the compiling, trading, and discussing of data, and the collaborative works undertaken as a result of these data, are quintessential components of what it means to be an academic in the social sciences, the processes by which data are conceived and collected should be transparent and open. According to KKV (Reference King, Keohane and Verba1994) this is a key characteristic of scientific research. The field of political science, in particular, should be reflective in examining the measures upon which it so heavily relies.
Archival sources cited
The George Field collection
“To Complete the Record” (1970), Series X: Writings, 1962–1990, George Field Collection, Box 4, Seeley G. Mudd Manuscript Library, Princeton University.
“To Complete the Record – Part II” (1985), Series X: Writings, 1962–1990, George Field Collection, Box 4, Seeley G. Mudd Manuscript Library, Princeton University.
“The End of a Dream” (1988 & 1990), Series X: Writings, 1962–1990, George Field Collection, Box 4, Seeley G. Mudd Manuscript Library, Princeton University.
Series III: Correspondence; 1934–1985; Box 2; George Field Collection of Freedom House Files, Mudd Manuscript Library, Princeton University.
Series XI: Our Secret Weapon; 1942–1943; Box 6; George Field Collection of Freedom House Files, Mudd Manuscript Library, Princeton University.
Series VI: Financial Files; 1949–1969; Boxes 2 & 3; George Field Collection of Freedom House Files, Mudd Manuscript Library, Princeton University.
Series XII: Legal Files; 1985; Box 6; George Field Collection of Freedom House Files, Mudd Manuscript Library, Princeton University.
The Freedom House collection
Series 1, Subseries 1: Bylaws and Minutes: 1941–1994; Freedom House Records.
Minutes: 1941–1969; Box 1; Folders 2-27
Minutes: 1970–1988; Box 2; Folders 1–19
Executive Committee Minutes: 1945–1968; Box 2; Folders 2-27
Executive Committee Minutes: 1969–1988; Box 3; Folders 1-20
Series 2, Subseries 1: Executive Directors; 1940–1995; Freedom House Records.
George Field Files: 1940–1973
The Balance Sheet of Freedom; 1948–1970; Box 27, Folder 8
Field, George: Personal; 1962 July–1966; Box 30, Folder 1
Leonard Sussman Files: 1965–1991
Balance Sheet of Freedom; 1968–1969; Box 39, Folder 10
Correspondence; 1967–1975; Box 39, Folder 17–18
Correspondence; 1976–1986; Box 40, Folder 1-2
Comparative Survey of Freedom; 1971–1977; Box 40, Folder 11–12
Comparative Survey of Freedom; 1978–1979, 1983; Box 41, Folder 1
Testimony; 1975 & 1981; Box 55, Folder 17
Series 2, Subseries 3: Financial Records; 1941–1992; Freedom House Records.
Annual Reports; 1943 1955 1977–1992; Box 75, Folder 13
Series 4, Subseries 3: Conferences and In-Service, 1942–1994, Freedom House Records.
Conferences; 1942–1995.
News Media/Gov’t Consultants Conference; 1973; Box 98, Folder 13
News Media/Gov’t Consultants Conference; 1973; Box 99, Folder 1-2
Series 7: Serials and Pamphlets; 1942–1999; Freedom House Records.
Balance Sheet of Freedom; 1953–1970; Box 126, Folder 6-7
Emily Zerndt is Visiting Assistant Professor of Political Science at Grand Valley State University. She is currently working on a book manuscript based on her dissertation, “The House that Propaganda Built: Historicizing the Democracy Promotion Efforts and Measurement Tools of Freedom House.” Her research and teaching interests include international relations, democratization, human rights, and democratic political theory.
