History, the mother of truth! —The idea is staggering…Historical truth, for Menard, is not “what happened”; it is what we believe happened.
Jorge Louis Borges (Reference Borges2000, 41)“History is not what it used to be”; “history is the interpretation of the significance that the past has for us.” Historians use aphorisms such as these to address the fact that we often know little about the past and that what we do know is partly or sometimes fully invented—termed in this article as the “problem of history.” As Goldthorpe (Reference Goldthorpe1991, 221–22) stated 30 years ago, “[h]istorical facts should be recognised as no more than ‘inferences from the relics’…often complex and indeed often quite speculative, which are drawn from relics that are manifestly incomplete, almost certainly unrepresentative, and in various other ways problematic” (see also Lustick Reference Lustick1996, 605).Footnote 1
The limited factual basis of our historical knowledge characterizes even some of the historical events with which we are most familiar. As a first example, consider Islam in the seventh century AD. There are virtually no contemporary written sources about Muhammed or the first four caliphs. What we know is based mainly on later Abbasid narrative sources or scattered remarks in contemporary Byzantine sources (Brown Reference Brown2011, 66–7, 87). Therefore, among historians, there has been a lively debate about whether Islam was first conceived as a branch of Judaism or Christianity rather than as a separate monotheistic religion, and also about whether the Quran was compiled around the time of Muhammed or only much later (Crone and Cook Reference Crone and Cook1977; see also Brown Reference Brown2011, 93–95; Silverstein Reference Silverstein2010).
As a second example, consider the military and political exploits of Alexander the Great in the period 336–323 BC. In history books that cover this subject, these exploits are extremely detailed. We are told what Alexander said before making important decisions,Footnote 2 of his relationships with many people (both friends and foes), the dates and places of his whereabouts, whom he fought along the way, and the precise outcome of each battle. This staggering level of detail is a result of the fact that most of what we know about Alexander is based on narrative biographies. However, it often is ignored that these biographies date to the period between 30 BC and the third century AD; that is, they were written at least three centuries after Alexander lived. Furthermore, they overwhelmingly were composed in Italy, often by Roman authors (Bowden Reference Bowden2014, 4–5). These narratives clearly draw on then-extant older historical sources, possibly written by eyewitnesses or chroniclers who had consulted them. Nonetheless,
[i]t is clear that, to greater or lesser extent, the surviving accounts have been shaped to appeal to a contemporary readership of Greeks and Romans living in a world governed by powerful emperors, for whom Alexander might serve as a model for how to rule, or how not to rule. Fundamentally, the Alexander of the narrative sources is a Roman Alexander (Bowden Reference Bowden2014, 4–5).
In many other situations, what we know is based on either evidence that is clearly outdated or extremely scarce and probably biased source material. Medieval historian Chris Wickham (Reference Wickham2016, 122) has this to say about much of the economic history on Europe before 1300: “Many current accounts present as ‘fact’ claims that go back to speculations made by pioneers in economic history in the 1960s and often well before, which have never been seriously tested.” In other cases, the description of entire periods is based on one singular, surviving source. A good example is the Histories of Amnianus Marcellinus from around AD 395. Not only is this the only existing narrative source on the Germanic invasions of the Western Roman Empire; it also has come down to us via one copy preserved at the monastery of Fulda in Germany. As Brown (Reference Brown1997, 309) states in his book on the Early Middle Ages, without this source, “[w]e would know next to nothing about the early stages of what we have come to know as the ‘barbarian invasions.’ Whole chapters of this book could not have been written.” Moreover, the one source that we do have often is clearly biased. A good example is the History of the Franks written by Gregory of Tours, long sections of which might be fully invented (Wickham Reference Wickham2009, 12–18).Footnote 3
Nonetheless, some social scientists freely cite descriptions such as those of Gregory of Tours as solid historical fact (e.g., Acemoglu and Robinson Reference Acemoglu and Robinson2019, 157, 160–62) without any discussion or mention of the reservations that trained historians have about Gregory’s History. This exemplifies a more general problem: social scientists who lift evidence from the work of historians often take a cavalier approach to prior debates about historiography and to the reservations that historians have about the truth value of their own historical analyses (see examples in Goldthorpe Reference Goldthorpe1991; Kreuzer Reference Kreuzer2010; Lustick Reference Lustick1996; Møller and Skaaning Reference Møller and Skaaning2021). However, even social scientists who display a keen awareness of the nature and limits of historical sources face the problem of the frailty of historical knowledge, although they do so with open eyes. Thus, when Stasavage (Reference Stasavage2020, ch. 7) analyzed the use of political consent in early Islam, he was forced to do so based on Quranic verses or later narrative sources, which well might have been reinvented or at least influenced by later political conflicts.
HOW SHOULD POLITICAL SCIENTISTS ADDRESS THE PROBLEM OF HISTORY?
The “golden rule” for political scientists conducting historical analysis is this: Read historians and read them carefully! (Kreuzer Reference Kreuzer2010; Møller and Skaaning Reference Møller and Skaaning2021). But how should political scientists, engaging with this work, respond to the fragile basis of much historical knowledge: the inconvenient fact that every historical account is an incomplete and possibly biased interpretation?
But how should political scientists, engaging with this work, respond to the fragile basis of much historical knowledge: the inconvenient fact that every historical account is an incomplete and possibly biased interpretation?
Twenty-six years ago, Lustick (Reference Lustick1996) published a pathbreaking article on how political scientists can approach disagreements within historiography that addressed important aspects of this question. Lustick proposed four strategies that guide the way we choose sources on which to base our historical narratives: “be true to your school,” “explaining variance in historiography,” “quasi-triangulation,” and “explicit triage” (Lustick Reference Lustick1996, 615–16). His more general aim was to convince political scientists to systematically factor in differences in historiography—or at least to make them aware of the obvious risk of “selection bias” that arises when scholars are free to take whatever narrative they find useful for their work as an accurate description of the underlying historical reality.Footnote 4
Lustick’s attempt to make historically minded political scientists aware of these issues was praiseworthy, and there is much to be gained from his strategies for dealing with selection bias (see Møller and Skaaning Reference Møller and Skaaning2021 for a critical discussion). However, as I have made clear, the problem of political scientists (and social scientists more generally) dealing superficially with the nature of historical knowledge persists. It calls for a more general modification in the way we “do history,” a shift in cognitive style that foregrounds the contradictions and uncertainties of historiography or the tenuous nature of historical knowledge more generally. I propose a general approach for how to do this, exemplified by three more particular guidelines.
A SHIFT IN COGNITIVE STYLE
The overall idea is simple: We need to be much more critical about the data handed down to us by historians, to recognize historical uncertainty when we sift the historical evidence, to signal this clearly when we present our historical research, and to avoid making strong claims on weak evidentiary foundations.
The overall idea is simple: We need to be much more critical about the data handed down to us by historians, to recognize historical uncertainty when we sift the historical evidence, to signal this clearly when we present our historical research, and to avoid making strong claims on weak evidentiary foundations.
This is a shift in cognitive style that takes historical uncertainty as a given and is based on having an open mind and using critical faculties when interpreting the historical evidence. Recognizing—and living with—uncertainty does not mean being ignorant. When we enlist evidence produced by historians, we are captives of what they, as a profession, know. This is why we must delve deeper into historiography to understand the internal debates of historians and to factor in the diversity of opinion among them—and only then bring a political scientist’s perspective on the evidentiary claims that we glean from this scholarship. We should not let convenience rule by simply invoking the first piece of evidence that we find. On the contrary, the shift in cognitive style is meant to shield us against the “confirmation bias” that results from applying strong theoretical expectations empirically: that is, the risk of one-sidedly finding what we are seeking when perusing the historical evidence (Lustick Reference Lustick1996; Møller and Skaaning Reference Møller and Skaaning2021).
Three Particular Guidelines
Three more particular guidelines can be used to exemplify the approach that I recommend. First, be up to date. We must constantly investigate whether historical knowledge has progressed to avoid enlisting evidence that clearly is outdated (Møller and Skaaning Reference Møller and Skaaning2021, 113–15). This does not mean that the most recent publication by a historian is, by default, the most accurate. As in many other scholarly fields, historians often have a type of “herd mentality.” They adopt implicit or explicit theories, concepts, or methodological trends that influence their interpretations, only to be amended by subsequent research.Footnote 5
However, these trends are visible only to those who have taken the time and effort to understand how historical knowledge has progressed. Moreover, the dearth of historical evidence sometimes means that one finding can significantly increase or decrease confidence in prior inferences about the past. Returning to the example of early Islam, the 2015 discovery that the “Birmingham Quran manuscript” (i.e., two leaves of early Quranic manuscript on parchment) can be radiocarbon dated to between 568 AD and 645 AD undermined many of the revisionist theories about a gradual development of Islam that was brought to an end only around 700 AD or even later (see Wickham Reference Wickham2016, 50, fn. 16). The early dating of the Birmingham Quran manuscript—supported by a few other early pieces of parchment, including the Sana’a manuscript found in 1972 and dated to before 671 AD—provides strong evidence against two revisionist claims: first, that the Quran was written long after the death of Muhammed; and, second, that early Islam should be viewed as a branch of Judaism rather than an independent monotheistic religion. This demonstrates how historians and archeologists sometimes stumble on a form of “airtight-alibi” evidence that makes it possible to at least rule out several theories.Footnote 6
Second, be specific. Avoid loose and composite concepts (e.g., feudalism) that pervade much historical research by social scientists (e.g., Anderson Reference Anderson1974; Blaydes and Chaney Reference Blaydes and Chaney2013). These flimsy concepts aggravate the problem of history by allowing scholars to infer general knowledge about theoretical variables in a way that is unconvincing when we critically examine the operationalization. For instance, loose concepts can easily lead scholars to make false historical analogies between events, developments, and political, social, or economic institutions that, at most, share some form of family resemblance (Møller Reference Møller2016).
Of course, we sometimes must use relatively general concepts or variables in historical analysis because we cannot meaningfully theorize without them, as pointed out long ago by Przeworski and Teune (Reference Przeworski and Teune1970, 25). However, if it is at all meaningful, it is better—in a transparent way—to test our theories against more specific and easily operationalized variables than against loose and composite constructs that are scored in more speculative ways. A helpful illustration is how recent research on European state-formation has used ruler tenure as a dependent variable (Blaydes and Chaney Reference Blaydes and Chaney2013; Kokkonen and Sundell Reference Kokkonen and Sundell2014). Based on historical sources, we know much about when monarchs took over the throne and when they died or were deposed; therefore, this type of data can be coded with relatively high accuracy. Another example is recent research that uses the geographic location of ecclesiastical institutions (e.g., monasteries) to score theoretically relevant explanatory variables, which also can be accomplished with high precision based on extant sources (Doucette Reference Doucette2021).
However, there are two important caveats. First, when sources are incomplete, we should not mistake specificity for accuracy. As historians often insist, “the absence of evidence is not evidence of absence” (Scott Reference Scott2017, 173), and this has consequences for the weight we attribute to the presence of evidence. A good example is recent attempts to code urban self-government in Italy based on the first mention of “consules” (e.g., in Milan in 1097) (Stasavage Reference Stasavage2010). As Wickham (Reference Wickham2015, 15) has pointed out, “[t]hese references are all entirely chance citations…we do not know the date of the passage to a consular regime in a single Italian city.” Second, we risk committing what Fischer (Reference Fischer1970, 90) termed the “quantitative fallacy”—that is, the idea that “facts are important in proportion to their susceptibility to quantification.”
Third, be cautious. This advice applies to the interpretation of findings in historical analysis in general; however, my main point is that the language we use when we present our findings signals our reservations about the solidity of the evidentiary record. When this record is not solid, we can use linguistic qualifiers such as “believed,” “commonly held,” “it has been argued,” and “evidence seems to indicate.” Scott (Reference Scott2017, xiii), for instance, notes that “[w]here the evidence is thin and I stray into speculation, I try to signal this as well.” As hinted at previously, Stasavage (Reference Stasavage2020) also is good at emphasizing that many of his observations are based on relatively scarce, relatively old, or relatively biased source material.
Historians often are skillful at this type of signaling, so we can draw on their use of language, illustrated by two examples. Bartlett (Reference Bartlett1993, 194) has the following to say about the Livländische Reimchronik of c. 1290: “…a late and lively but not totally misleading source.” In his History of the Crusades, Tyerman (Reference Tyerman2006, 237) refers to the memoirs of Usamah of Shaizar: “…whose stories are frequently too good to be precisely true.” However, Bartlett and Tyerman both proceed to invoke certain aspects of these texts as evidence, based on a critical approach to their truth value. Political scientists thus can gather inspiration from how historians often are able to recognize historical uncertainty without being paralyzed by it (see Wickham Reference Wickham2009, 12–18, for more general considerations about how to do this).
An Illustration
The purchase of this general approach and of the three particular guidelines can be illustrated by an example: Political Order and Inequality by Boix (Reference Boix2015). This impressive book presents a general theory of the conditions of political order and then tests it using a large variety of historical evidence. Boix (Reference Boix2015, 15) enlists new quantitative data on social stratification, biogeographical conditions, and political institutions, supported by qualitative historical comparisons, and he attempts to carry out a “strictly analytical” test of his theoretical propositions—for instance, through the clever use of instrumental variables estimation.
However, whereas Boix (Reference Boix2015, 15, 93) carefully explains how he uses archeological, ethnographic, and biogeographical data to test causal propositions, he says little or nothing about his secondary use of historical data in the form of more qualitative narratives and historical comparisons. In parts of his book where these forms of evidence are used, he often references relatively old and relatively sweeping historical work without stating any reservations. This is especially pronounced in chapter 4 (on warfare), where he uses narrative historical data to investigate the effects of different military technologies. For instance, Boix’s (Reference Boix2015, 143–47) analyses of the introduction of the horse in West and East Africa and of the consequences of the advent of the stirrup and heavy cavalry in the Carolingian Kingdom/Empire and the Byzantine Empire (Boix Reference Boix2015, 149–57) are both based mainly on literature from the 1960s to the 1980s. Boix ignores the fact that the period since the 1980s has seen an explosion of historical data about, for example, the European Middle Ages, resulting mainly from new archeological findings (Wickham Reference Wickham2005, 1).
This same period also has witnessed a ferocious debate about the empirical validity of the feudal model, which Boix (Reference Boix2015, 149–53)—without further explanation—uses to understand the Carolingian development.Footnote 7 Moreover, Boix says nothing about the loose and composite nature of this construct. Thus, this type of empirical evidence is fundamentally different from the specific quantitative measures that Boix enlists in other parts of his book—for instance, geographical distances, data on climatic conditions, and archeological data on height differences between members of the elite and commoners.Footnote 8
Considering the care that he takes to validly interrogate anthropological and archaeological statistical data, it is curious how nonchalantly Boix bases key empirical observations on outdated narrative interpretations and loose concepts (e.g., feudalism) that many historians have abandoned, without any reflection on whether newer research corroborated these older claims or on the inherent uncertainty of our historical knowledge about the early medieval context. This brings us to the third guideline. With a few praiseworthy exceptions (e.g., Boix Reference Boix2015, 136, fn. 2, 138, 150–51), Boix’s historical narratives mainly read as if he is describing solid facts rather than possibly biased or uncertain historical interpretations. Moreover, these “facts” sometimes are derived from work that is clearly theoretically influenced. For instance, Boix references Anderson’s (Reference Anderson1974) Marxist synthesizing of historical work from the decades after World War II for evidence about how nobles east of the Elbe enserfed peasants, oriented their economies toward the grain-export market, and allied with their monarchs against townsmen (Boix Reference Boix2015, 146, 161). It would have been helpful to signal to readers that this type of historical evidence is weak—or, at most, suggestive.
Considering the undisputed merits of Boix’s bold and captivating book, this is an example of how we—as a profession—stand to benefit from changing our cognitive style by treating the work of historians in a more self-conscious and critical way, just as we do with other sources of data. Often, this more thorough, critical, and systematic approach can be an eye-opener. Scott (Reference Scott2017, x) describes how his endeavor to review the history of the earliest states in the Fertile Crescent quickly made him skeptical about several unexamined assumptions that “those of us who have not been paying much attention to the new knowledge of the past two decades or so are apt to have taken for granted.” There is every reason to believe that Boix (Reference Boix2015) would have encountered similar new insights had he based his use of narrative historical data on more recent evidence and if he had thought more carefully about which type of narrative evidence is more valid—rather than rehashing the knowledge that historians and social scientists held more than a generation ago.
CONCLUSIONS
The main purpose of this article is to make clear to historically minded political scientists just how fragile and speculative the evidentiary record with which they engage often is. To avoid building “castles in the sand,” political scientists must recognize this and respond to the problem of history. I argue that this calls for a more general shift in cognitive style in which we take the problem seriously in all stages of historical research and we are open about the limitations of the historical evidence. Three simple guidelines are proffered to illustrate this approach: (1) we constantly must update our historical knowledge; (2) if it is at all meaningful, it is better to be specific in historical research; and (3) it is better to be cautious when making claims based on scarce evidence. I also emphasize that even these guidelines come with certain caveats and tradeoffs. This merely underlines the need for the general shift in cognitive style toward a more skeptical approach. Be Bold, Be Bold, But Not Too Bold.
Three simple guidelines are proffered to illustrate this approach: (1) we constantly must update our historical knowledge; (2) if it is at all meaningful, it is better to be specific in historical research; and (3) it is better to be cautious when making claims based on scarce evidence.
ACKNOWLEDGMENTS
This is the third and last in a series of short articles about the challenges that face political scientists who engage with history. I again am indebted to Svend-Erik Skaaning and to an anonymous reviewer for helpful comments. As in the two other articles, my own prior work hardly lives up to the standards that I advocate. Indeed, the guidelines presented in the three articles, to a large extent, reflect my own trial-and-error–prone development as a political scientist doing historical analysis. Apropos, any specific errors in this article are my own.