Many researchers in political science would probably subscribe to the idea that our research should be characterized by a “scientific” approach that prioritizes concise theoretical reasoning, rigorous methodological techniques, and robust results. This characterization should affect the language that political scientists use to write about their research. That is, scientific articles should convince their audience through scientific quality without the need to “market” a particular approach or a new result by emphasizing its importance through the use of positive language. By “positive language,” we mean the tendency of researchers to explicitly include positive evaluative statements in their writing—for example, by describing their contributions as “promising,” “novel,” or “important.”
Yet, even in the “hard” sciences, scientific language often deviates from the principle of objectivity and takes a decidedly positive tone. A recent article by Vinkers, Tijdink, and Otte (Reference Vinkers, Tijdink and Otte2015) studied the change in the use of positive words in medical journals. Counting the frequency of positive words in the journal abstracts over time, the authors documented a considerable increase in their use. This trend has led to discussion about the appropriate use of scientific language in the natural sciences. To counter “superlative scientific writing,” some journals actively discourage the use of exaggerated, positive language and even ban selected words in titles and abstracts (Scott and Jones Reference Scott and Jones2017). Where does political science stand regarding this question? Do we avoid “the use of impressionistic or metaphorical language, or language which appeals primarily to our senses, emotions, or moral beliefs?” (The Writing Center at UNC–Chapel Hill, 2017). This article reports the results of an analysis we conducted similar to that of Vinkers, Tijdink, and Otte (Reference Vinkers, Tijdink and Otte2015) for the discipline of political science. To document changes in the scientific language in our discipline over time, we analyzed the use of positive and negative words in the three main general political science journals.
To document changes in the scientific language in our discipline over time, we analyzed the use of positive and negative words in the three main general political science journals.
This article is one of the few to do so. Despite the discipline’s strong interest in the language of politics and major progress in analyzing it computationally (see, e.g., Lucas et al. Reference Lucas, Nielsen, Roberts, Stewart, Storer and Tingley2015), the language of political science has received little attention. Despite an extensive literature search, we found minimal existing work that is closely related to the question we studied. There is significant reflection within the discipline about topics that political scientists cover (Miller, Tien, and Peebler Reference Miller, Tien and Peebler1996), the relative role of empirical versus normative work (Sigelman Reference Sigelman2006), patterns of coauthorship among political scientists (Fisher et al. Reference Fisher, Cobane, Vander Ven and Cullen1998), and the scientific impact of political science research (Gleditsch Reference Gleditsch1993). However, there are few reflections about the language used to present this research. One exception is a critique of the increasing methodological terminology in the discipline due to the rise of quantitative approaches (Margolis Reference Margolis1971). More relevant for our purpose was a study on the use of scientific English beyond the discipline of political science, which assessed the extent to which scientific abstracts contain evaluative statements (Stotesbury Reference Stotesbury2003). This study uncovered considerable differences between scientific fields: abstracts in the humanities typically include more evaluations compared to the natural sciences, with the social sciences ending up between the two. If the social sciences tend to include more evaluations in their abstracts, does this mean that the trend toward more positive words found in the medical sciences is amplified? The remainder of this article presents data and analysis to answer that question.
DATA, APPROACH, AND GENERAL PATTERNS
Our approach followed Vinkers, Tijdink, and Otte (Reference Vinkers, Tijdink and Otte2015) to trace the frequency of distinctly positive and negative words in political science journal abstracts over time. To obtain a general pattern based on the use of language across the entire discipline, we followed earlier work (Fisher et al. Reference Fisher, Cobane, Vander Ven and Cullen1998) and selected the three main political journals: American Political Science Review (APSR), American Journal of Political Science (AJPS), and Journal of Politics (JOP).
From the JSTOR archive, we obtained abstracts for all articles published through 2014. Because journals started using abstracts at different times, our period of observation was limited to the years 1977–2014 for APSR and AJPS and 1982–2014 for JOP. Our final dataset consisted of 5,528 abstracts: 1,748 from APSR, 1,929 from AJPS, and 1,851 from JOP. Using this dataset, we then searched for positive and negative words according to the list provided by Vinkers, Tijdink, and Otte (Reference Vinkers, Tijdink and Otte2015). Because their work applies to scientific articles published in a different field, we did not expect all words to be equally relevant in political science. For example, the word “cure” applies specifically to medical research. Therefore, we restricted the set of positive and negative words to those that appear more than five times during our period of observation (for the complete list, see appendix table A.1). Our text search ignored case and, therefore, was able to find the specified words regardless of where they appear in a sentence.
Using our dataset and the list of positive and negative words, we calculated annual values of two pairs of variables: First was the share of abstracts that use at least one positive or one negative word. The specified words (whether positive or negative) are rarely used overall, and we did not expect to find more than one in a given abstract. However, the length of abstracts may be changing over time, which can partly affect this measure. For this reason, we also used alternative indicators: the share of positive and negative words of all words contained in the abstracts. These numbers, obviously, were much smaller than the first set of indicators.
This section presents aggregate results over all words before discussing more refined analyses in the following section. Figure 1 (left panel) shows the smoothed trend in the share of abstracts with at least one positive or negative word. Two findings are immediately obvious from the figure. First, the use of positive words increased significantly during our period of observation. Whereas in 1982, less than 5% of all abstracts contained a positive word, in 2014, this percentage increased to more than 20%. Second, there is no similar trend in the use of negative words. We observed fluctuations during the 1990s but, overall, the use of negative words remained at around 8%. Figure 1 confirms that these trends are not the result of changes in the length of abstracts over time. If we calculate the percentage of positive and negative words of the total number of words (right panel), the trends remain almost unchanged. Appendix figure A.1 shows the same results without the JOP abstracts, which allowed us to extend the observation period to 1977–2014. The trends remain similar to those identified previously.
Figure 1 Trends in the Use of Positive and Negative Words over Time: Share of Abstracts with at Least One Positive/Negative Word (Left) and Share of Positive/Negative Words Overall (Right)
These findings confirm that political science experienced changing practices in the use of language over time. Words implying positive connotations about scientific research become increasingly frequent over time, whereas we found no such tendency in the use of negative words. Thus, it is not the case that scientific writing in our discipline uses more evaluations in general (whether positive or negative) over time; rather, it emphasizes primarily the perceived positive aspects.
POTENTIAL EXPLANATIONS
What accounts for the increased use of positive words in political science writing? First, we must ensure that these changes are not simply reflections of changing language practices overall. As suggested by Vinkers, Tijdink, and Otte (Reference Vinkers, Tijdink and Otte2015), we compared our results to the occurrence of positive and negative words in general writings using data from Google Books (Heyman Reference Heyman2015; see also https://books.google.com/ngrams). The project provided word frequencies extracted from numerous fiction and non-fiction books that were obtained by Google from libraries and publishers. If these books were not available in electronic format, they were scanned and processed with text-recognition software before being added to the text-analysis corpus. Using the public website, users can obtain data on the use of specific words over time (i.e., as they appear in publications of the respective year). The Google Books text-analysis corpus is not without problems and has limitations. We followed the recommendation of Pechenick, Danforth, and Dodds (Reference Pechenick, Danforth and Dodds2015) and used the “English fiction” corpus provided by the project, which largely excludes professional texts and therefore is better suited to measure trends in general, non-scientific language. Data coverage ends in 2008, which is why we show results only for the period 1974–2008. Figure 2 shows the proportion of positive words in Google Books, which is the same quantity shown in figure 1 (right). To make the figures comparable, we used the same y-axis range as in figure 1 (right). As the figure shows, the proportion of positive words in general language was even slightly decreasing during the observation period. Thus, the trend we observed cannot be explained by changes in the use of general language and seems to be specific to scientific texts.
One of the words with the most pronounced increase is “novel,” which was used only once in the first five years of our study period (1982–1986) but 40 times in the last five years (2010–2014).
Figure 2 Trends in the Use of Positive Words in General Language According to Google Books
Second, the trends we observed could simply reflect changes in the scientific practice in political science, in particular toward more positivist and empirical approaches. As discussed by Margolis (Reference Margolis1971), this brought with it a “new language” of political science, which may account for the increased frequency in the words we examined. To find out to what extent this applies to our results, we took a closer look at the results separately for each word (figure 3). Due to the low overall frequencies of these words, the trends displayed considerable fluctuations, although a general increase can be seen for most of them. Indeed, some of the words in our analysis are closely related to the rise of quantitative methods (e.g., “robust”); however, many others are not. One of the words with the most pronounced increase is “novel,” which was used only once in the first five years of our study period (1982–1986) but 40 times in the last five years (2010–2014). We saw similar changes for “unique” and “innovative”; these words have no obvious relationship to quantitative research. They constitute positive evaluative judgments and cannot be tied to particular scientific approaches that have seen more adoption in the discipline over time.
Figure 3 Trends in the Use of Specific Positive Words over Time: Share of Abstracts that Contain the Respective Word at Least Once
Third, changes in writing style could be responsible for the trends we observed. Perhaps political scientists have attempted to move away from an overly technical language to emphasize the scientific or policy implications of their research rather than the scientific execution. Again, a closer look at the trends—specifically, positive words—helps us to assess this explanation. In particular, words such as “unique” and “novel” provide little reason to believe that a change in writing style should be responsible for the increase in positive words. This holds particularly because we examined only the abstracts of scientific articles, in which space is limited and authors are required to summarize the essence of an article in only a few words. Thus, if political writing style indeed were changing in the ways mentioned previously, it is unlikely that we would observe this in the abstracts of articles.
Therefore, we suspect that an explanation similar to that provided by Vinkers, Tijdink, and Otte (Reference Vinkers, Tijdink and Otte2015) for the medical sciences also applies to political science, at least partly. The authors concluded that the use of positive language increases the chances of publication. Increasing pressure and competition in scientific publishing may provide incentives to deviate from the commonly accepted need for objective language. Moreover, research that is presented with a distinctly positive tone may stand a better chance of convincing others of its importance and surviving the peer-review process. If this explanation holds, the reason for the increase in the use of positive words is found in the discipline as a whole—that is, not only by the authors using these words but also by the reviewers and editors who favor manuscripts using this language. This does not mean that the positive framing is misplaced—it well could be true that the research we publish is as “novel” and “innovative” as it claims to be. What we observed, however, is that authors increasingly feel the need to explicitly tell the reader that the research really is innovative. However, using our approach, we cannot prove with certainty that this explanation holds, and further work (e.g., comparing abstracts of published and rejected papers over time) is required.
CONCLUSION
This article documents a marked increase in the use of positive words in the language of political science in recent decades. This trend is unlikely to be explained by a changing use of language, more generally, or by a broadening of the set of scientific approaches or a new writing style used in our field. We therefore suspect that selection mechanisms in academic publishing partly account for the trend, whereby manuscripts framed in a positive tone are more likely to be published. At the same time, however, this trend is much less dramatic than in other disciplines. In the medical sciences, Vinkers, Tijdink, and Otte (Reference Vinkers, Tijdink and Otte2015) found an increase in the use of positive words by a factor as much as 10, whereas our analysis revealed only a four-fold increase for the same period.
Although the identification of this trend was not difficult, we must be more cautious in its interpretation. Should we consider the increased use of positive words in political science a problem? For once, this trend could indicate researchers’ increasing efforts to clearly highlight the added value of their work. Given the growing number of academic publications, it can be difficult to pin down if and how new research moves beyond what we already know; therefore, this should be a welcome trend. At the same time, however, it may be worrisome that authors increasingly seem to include positive judgments in their work. We could argue that the evaluation of scientific innovation and potential should be left to reviewers and the scientific readership. Still, given the modest magnitude of this development in political science compared to other fields, we do not believe that, at present, it constitutes an issue of major concern.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049096518000124.
