The use of models has come to dominate much of the scientific study of politics. We use models, mathematical or otherwise, to investigate and illuminate causal mechanisms, generate comparative statics, and understand the conditions under which we expect certain outcomes to occur. While our use of models has grown dramatically, our understanding of the role or function that models play in the scientific enterprise has not kept pace.
Throughout the 1970s and 80s, models were treated as tools for conceptual exploration, and little was to be gained by testing either their assumptions or their predictions. Moe, for instance, argued that rational choice models are devoid of empirical content, and Achen warned that mathematical models “are to be used and not believed.”1
Theoretical modelling sometimes informed data analysis and vice versa, but with few exceptions, the field was divided into theorists and empiricists.As models in political science became more prevalent, the uses to which we put those models shifted significantly, and today, the emphasis is on using models to generate testable predictions that serve as hypotheses for subsequent data analysis, which in turn is interpreted as a test of the model. The field has created a hierarchy of models where those that are “tested” are valued more highly than those that are not tied to a regression analysis. While much ink has been spilled arguing for this test-the-model approach to the study of political science, little attention has been paid to justifying and rationalizing the method. On the rare occasions that justification has been attempted, the results have been maddeningly vague. Why test predictions from a deductive, and thus truth-preserving, system? What can be learned from such a test? If a prediction is not confirmed, are assumptions already known to be false to blame? These questions are never addressed in a satisfactory way.
Lack of a suitable justification for model testing is not the only reason to revisit the role of models in political science. The emphasis on model testing has led to a distortion of both the modelling process and the art of data analysis. Preferring models that predict well has led the field to devalue the other important contributions that models can make to understanding the political world. The emphasis on using data analysis as a means for testing models turns regressions into hypothesis-testing machines and ignores the role that data analysis can play in producing empirical generalizations that may serve as a spur to further modelling efforts.
Rethinking our use of models in political science is of critical importance, as a generation of scholars is being trained to think about science in an outdated and inadequate manner. The success of the Empirical Implications of Theoretical Models (EITM) project has highlighted the fact that while political scientists are using models more than ever before, we are still thinking in old ways. The goal of this paper, then, is to modernize political science. Following the work of Suppes, van Fraassen, Suppe, and others, we define a model as an object, not a linguistic entity, and thus neither true nor false.2
We specify five uses to which models may be put and argue that model evaluation should proceed in the much the same fashion that mechanical models are evaluated in the physical sciences—good models are useful for particular purposes. This perspective frees political science from the deleterious effects of an over-emphasis on prediction.What we propose is more than a change in language; we argue for a change in how political scientists construct, perceive, and utilize models. This model-based approach provides new ways to think about the purposes of models in political science, the relationship between models and the empirical world, and the language of scientific inquiry.
What Is a Model?
Before discussing how models are used, it is necessary to define explicitly what a model is. We do so first through an analogy. Maps are models.3
Giere 1999.
The question to ask of a map is not whether it is true or false, but whether the map is similar to the world. The standard objection to this line of reasoning is that two objects may be similar or dissimilar in almost an infinite number of ways. Such objections may be met by specifying in what manner, to what degree, and for what purpose the map is similar to the world. Is the model similar enough to the world to be useful for a specific purpose? A map of a subway line may have little similarity to the geography of a city, and yet be similar enough to the rapid transit system to make it a necessary appendage for all commuters. Thus, as Giere argues, “Maps necessarily reflect the interest of map makers and map users.”4
Ibid., 43.
If theories were linguistic entities, it would make sense to ask whether they are true or false, just as any statement can be true or false. What we argue is that political scientists should think of models more like representational objects and less like linguistic entities.5
This is not to say that models in political science have a corporeal existence as maps do; you cannot hold a model in your hand. Rather, a model is a kind of system that has some degree of similarity with a natural, or real-world system.
Morton 1993 refers to this characteristic of models as “purpose-relativity.”
To achieve this end, we consider the predicate or semantic (referring to the relation between language and reality, as opposed to syntactic, which refers to the relation between verbal symbols) view of theories most closely associated with the work of Patrick Suppes, Frederick Suppe, Bas van Fraassen, and Ronald Giere.7
Suppes 1967; Suppe 1977, 1989; van Fraassen 1980; Giere 1990. While the semantic and predicate views are theoretically distinct, the two terms are often used interchangeably in practice with little harm. We use the term “semantic.”
Salmon 1988.
Brady 2004, 296. Interestingly, one of the earliest experimenters with a semantic or model-theoretic approach to theories was none other than John von Neumann of game theory fame; Suppe 1977.
The most important feature of the semantic approach is that models, not theories, are central to the scientific enterprise.10
The precise formulations we use are drawn mostly, but not entirely, from Giere 1984, which is a textbook, and its presentation is consequently somewhat simplified. The encapsulation is elegant enough, however, to be liberally cited in van Fraassen 1989, Grunbaum and Salmon 1988, and Hausman 1992.
Suppes 1967, 63.
Salmon 1988 (6) writes that a theory is “a set of models”; Costa and French 1990 (249) write that a scientific theory “can be considered in terms of a description of its set of models”; Downes 1992 (143) writes that scientific theories “consist of families of (mathematical) models”; and Giere 1984 (83) writes that a scientific theory is the “conjunction” of claims regarding models and natural systems.
A model is a kind of system whose characteristics are specified by an explicit (and sometimes elaborate) definition. As a model is characterized by a definition, by construction it cannot be falsified. For example, “a Newtonian Particle System is a system that satisfies the three laws of motion and the law of universal gravitation.”13
Giere 1984, 81.
The question we should ask of a model is whether or not it is similar in certain respects, and for certain uses, to a system in the real world.14
While we have been writing primarily in terms of formal mathematical models, there is nothing in the semantic approach that restricts its application to such models. This misconception, however, appears to be widespread among opponents of the semantic view; Lloyd 1988. It may well be easier to understand the semantic view when considering mathematical models, but we need not be concerned that adoption of the semantic view will, in some way, threaten a “unified logic” of scientific inference in political science. Scholars who make use of “informal models” (for lack of a better term) may also benefit by thinking in terms of similarity between their models and real-world systems.
Thus, “a model may or may not be suitable … but it is neither true nor false, or confirmable, or directly subject to what is usually called induction.”15
Hutten 1954, 296.
The Purposes of a Model
As models should be assessed for their usefulness for a specific purpose, it is therefore necessary to detail the purposes to which a model can be put and how we can judge whether or not a model is useful. At the most basic level, models can serve in any one (or more) of five different roles: foundational, structural, generative, explicative, and predictive.16
We are open to other possible uses of models. For instance, many of the early social choice models may be seen as normative. We thank Loren King and James Johnson for this point.
The Purposes of a Model
It is this last function of models, prediction, with which we are most familiar, although actual examples of models whose main purpose is prediction are quite rare. Most of these models can be found in the area of national election forecasting.17
For example, see the models used by Lewis-Beck and Rice 1992 and described by Campbell and Garand 1999.
Campbell 2004, 733.
In most research, though, a prediction is a hypothesis deduced from a model. This kind of prediction is often seen as the “crucial activity” in science, and models are judged by how well they predict.19
Brady 2004.
Laver 1997, 6.
The spatial model, first employed in an electoral setting by Downs, is a case in point.21
Downs 1957.
Krehbiel 1998.
As this example demonstrates, the correlation between predictive ability and usefulness is far from perfect. Thus, models can prove useful in other ways. Highly abstract models, for example, may exist to provide foundational results for applied modelers.23
Gibbard and Varian 1978 (665) describe such models as “ideal models” that are the “description of some ideal case which is interesting either in its own right or by comparison to reality.”
Arrow 1963.
Baron and Ferejohn 1989.
Banks and Duggan 2000.
Models that play an organizational or structural role need not be as abstract as those that play a foundational role. Models of this kind provide a “framework wherein information may be defined, collected and ordered.”27
Haggett and Chorley 1967, 24.
Achen 1992.
Ibid., 206. See our extended example drawn from Achen's more recent work.
Models may also serve a generative function. The purpose of these models lies in their “fertility”—the ability to generate interesting and non-obvious statements about the phenomenon of interest. Laver clearly views rational choice models as performing this function and forcefully argues that the purpose of rational choice theory is “the search for interesting and non-trivial tautological arguments.”30
“Tautological” in Laver's sense means “determined by the assumptions and [a] system of logic”; Laver 1997, 6.
Romer and Rosenthal 1978.
Finally, models may also serve an explicative function, which refers to the fact that we can use models to explore the putative causal mechanisms underlying phenomena of interest. One such method is to ask counterfactual or “what-if” questions.32
“What-if” questions are often rather loosely referred to in political science as “counterfactuals.” For several applications from political science, see Tetlock and Belkin 1996.
Knight 2001, 10.
For majority rule in an n-member, odd-sized legislature, q = (n + 1)/2; for unanimity, q = n.
Laver 1997.
As models may be used for a myriad of purposes, predictive success cannot be, as it too often is, the only metric by which we assess models. Prediction may, in fact, be orthogonal to a model's goals. Given that models are purpose-relative, proper evaluation must begin with a determination of the model's intent. Once that has been established, we can then ask whether or not the model achieves its stated purpose. Thus, the purpose to which a model is put generates its own metric of success. Models that generate a significant number of interesting statements or counterfactuals are successful, as are models that are routinely used as building blocks for further inquiry. At the same time, it is quite possible to write down models that generate uninteresting, vapid, or narrow results. Models that are useful should survive regardless of their ability to predict, and models that are not useful should simply die from neglect, again, regardless of their ability to predict.36
The obvious exception here are models for which prediction is the main goal. Such models comprise a small proportion of the political science literature.
The Political Scientific Method
In order to draw a clear distinction between the understanding of models outlined in the previous section and the state of the discipline in political science, we need to characterize and critique what serves as the scientific method for much of political science. The technical name for this method is hypothetico-deductivism (H-D), and at first glance, it may appear that we are offering a caricature of political science. However, we follow the discussion with evidence from the documentary record that political scientists do write and think in this way.
The H-D approach comprises the following:
- a hypothesis H set up for testing or examination;
- an observation sentence O implied by H along with theoretical background statements, mixed statements, boundary conditions, etc.; and,
- an experiment or examination of the world where we observe either O or ∼O.37
Kyburg 1988, 65.
If we observe ∼O, then we have refuted H. If we observe O, then we have confirmed H or, at the very least, failed to refute H.38
An equivalent formulation is given by Glymour 1980b, 322: “A sentence h is confirmed by a sentence e with respect to a theory T if e is true and h&T is consistent and h&T entails e but T does not entail e.”
Putnam 1991, 123.
The approach, on its face, appears hard to argue with. What could be more straightforward than deriving a prediction from a model and testing it? As we and many others argue, however, the straightforwardness of the H-D approach collapses as soon as one begins to push past its intuitive appeal.40
Attempts to justify the supposedly intuitive H-D procedure had, by 1980, become so complex that one leading philosopher entitled an article “Hypothetico-Deductivism Is Hopeless” (Glymour 1980a), and another was left to argue rather impotently about H-D's ineffable rightness (Grimes 1990). Yet another has argued the problem lies with first-order logic, and that a solution to the problem requires the abandonment of this branch of logic (Waters 1987).
The shortcomings of the H-D approach include, in no particular order, the Quine-Duhem problem, underdetermination, non-deductive predictions, the tacking problem, the problem of quantitative generalization, the paradox of the ravens, and the grue paradox. The Quine-Duhem problem concerns locating the blame in cases of disconfirmation; H-D does not help us understand whether the theory of interest is false, or an auxiliary theory is false. Underdetermination refers to the logical truth that for any finite collection of evidence, there are infinitely many inconsistent hypotheses which entail that evidence. Non-deductive prediction refers to the fact that most hypotheses in science are statistical and thus not deductive. The tacking problem notes if some evidence confirms a hypothesis, then that evidence also confirms the conjunction of the original hypothesis and any other hypothesis whether it is relevant or not. Hempel's paradox of the ravens demonstrates that non-black, non-ravens confirm “all ravens are black” to the same degree as black ravens. The grue paradox refers to the fact that all observed green emeralds confirm the statement “all emeralds are grue,” where grue is defined as observed before 2010 AD and green and observed after 2010 AD and blue. Most of these problems are rather technical in nature and need not concern us here.
What can one learn about a deductive model from testing an implication of that model? Two states of the world are depicted in table 2. The assumptions of a model are either true or false. If the assumptions of the model are true, then the predictions of the model must be true because deductive systems are truth-preserving, and testing is therefore unnecessary and redundant. As Doron and Sened write, “Deductive reasoning has one clear advantage over inductive reasoning—it need not be verified in the empirical world. In fact, it need not even reflect real world phenomena. Its falsification comes about as a result of internal inconsistency.”41
Doron and Sened 2001, 146.
Possible States of the World
If the assumptions of a model are false, then the predictions of the model may be either true or false. It is not true that if the assumptions are false, then the predictions must be false. (Deductive systems are not falsity-preserving.) Thus, even assuming that models can be true or false, whether or not the prediction is borne out in the data analysis does not inform us about the truth of the model. The accuracy of the predictions is not connected in any necessary way to the model and therefore their accuracy cannot reflect on the model.
Of course, political scientists never know the true state of the world, so it could be argued that false predictions do indicate that the assumptions are false. This argument would be quite valid except that political scientists are well aware that almost all assumptions are false. Data analysis therefore cannot inform a researcher as to whether or not a model is “confirmed.” We should note that it is no defense to argue that our assumptions are “approximately true” or “true enough.” The truth-preserving properties of a deductive model do not hold when the assumptions are approximate. In the context of a deductive model, “approximately true” equals false.
Testing a prediction or implication deductively derived from a model cannot help us to learn about the model itself. A true prediction cannot tell us that the model is correct, and a false prediction cannot tell us something that we already know.42
Some argue that predicting “novel facts” lends additional confirmation. The proponents of “epistemic difference,” however, have yet to overcome serious objections. See White 2003.
Morton 1999, 161.
Are political scientists really doing H-D, or have we set up a straw man? Before turning to the evidence, we reiterate that the models to which we refer need not be formal mathematical models. It is quite common for political scientists to derive predictions for testing from informal or verbal models or theories. Examples include Zaller's study of public opinion, Huth and Allee's study of territorial conflict, or Norris's comparative study of electoral rules.44
Thus, while our use of the term “model” may appear to restrict our arguments to the small group of political scientists who use formal models, we are in fact addressing a substantial portion of the discipline.Hypothetico-deductivism is so ingrained in the thinking of most political scientists that it often goes unrecognized. We demonstrate the dependence of our field on H-D in two ways. First, we cite the writings of some of the most well-respected scholars in political science, whose work often serves as models for others. Second, we report the results of a survey of recent articles in the top three political science journals. The evidence shows that H-D is the prevailing method of model testing in political science.
Walt and Bawn, for example, argue that the success or failure of a theory hinges on its “empirical validity” established by testing the predictions derived from a model.45
Fiorina views the potential for falsifiability as the sine qua non of a model's scientific status,46Fiorina 1994.
Bueno de Mesquita and Lalman 1992, 22.
Further examples of the influence of hypothetico-deductivism are easily found throughout the discipline. Keith Krehbiel, in a study of legislative committee organization, writes, “to reap the benefits of a joint theoretical-empirical approach, the remainder of this study seeks to conform with orthodox tenets of positive social science. Postulates are empirically motivated. Theoretical assumptions are explicit and precise. Theoretical results follow logically from the assumptions. Empirical predictions are extracted from theoretical results. And empirical predictions are refutable.”48
Krehbiel 1991, 15.
John Zaller adopts a similarly axiomatic approach to developing testable propositions about public opinion: “The method of this book, then, is to develop deductive implications of the four basic axioms for a given, highly specific set of conditions; review evidence indicating whether or not these implications are empirically correct; and present new evidence as necessary and possible to resolve outstanding empirical questions.”49
Zaller 1992, 51.
Cameron, Segal, and Songer derive comparative statics from their model and “test them with a random sample of search-and-seizure cases appealed to the Burger Court between 1972 and 1986.” Their empirical findings “support the theory.”50
Cameron, Segal, and Songer 2000, 101, 107, 113.
Clark and Hallerberg 2000, 323, 324, 326.
Pippa Norris writes that rational-choice institutionalism “makes certain simple assumptions about the self-interested aims of rational actors and then seeks to outline and test the predictions that flow logically from these premises.”52
Norris 2004, 252.
Geddes 2003, 86.
Even a scholar who often writes about methodological issues, such as Chris Achen, uses the language of hypothetico-deductivism liberally. Throughout a recent article on political socialization, Achen writes of deductive propositions being “validated” or assessed.54
Achen 2002.
Some might object that while political scientists often write in terms of hypothetico-deductivism, they often think about models in more complex terms. This objection raises two problems. First, the only evidence we have regarding what political scientists think is what they write. Second, the objection assumes that how one writes has no influence on how one thinks. While this assumption is almost certainly false, writing in terms of the HD model can have baleful effects on research even if scholars think in more sophisticated ways. For instance, writing an article in the H-D format can skew both the kinds of models we construct and the kinds of data analysis we perform. The reason lies in the fact that the H-D perspective views prediction as the crucial activity in science.55
Brady 2004.
A recent debate over editorial decisions at the American Journal of Political Science demonstrates this point. See our discussion on integrating models and data.
Others might claim that quoting leading scholars in the field is anecdotal. To address this claim, we conducted a survey of the three leading generalist journals in political science: American Political Science Review, American Journal of Political Science, and Journal of Politics. Of the 738 articles in these journals between 2001 and 2005, we randomly sampled 10 percent, or 74 articles. Using very conservative coding rules, we identified nearly half of the articles (46%) as using or promoting H-D.57
To be coded as practicing H-D, authors had to either (1) derive, formally or not, testable hypotheses from a model or theory and then make claims about the state of the model or theory based on the outcomes of those tests, or (2) make a statement that model testing should be accomplished by testing hypotheses derived from the model. Political philosophy articles were excluded from the sample. An independent research assistant classified 65 percent of the sample as H-D, and nearly 40 percent of the sample was classified independently as H-D by both the authors and the research assistant.
A modern take on the H-D approach to scientific inference in political science is the EITM project.58
As the essays from the National Science Foundation workshop on EITM make clear, there are numerous and often conflicting definitions of EITM. For some, EITM simply means using empirical generalizations to “discipline” formal theory and vice versa. For others, EITM begins with a puzzle and culminates in the “testing” of a model; National Science Foundation 2002, 19, 25.
Morton 1999, 280.
Aldrich and Alt 2003, 309.
Granato and Scioli 2004, 315.
If there is any remaining doubt that the authors cited above are writing from an H-D perspective, note that the proponents of the hypothetico-deductive method similarly exhort us “to formulate one's hypotheses clearly, to draw testable implications from them by deductively valid reasoning, and to put these implications to experimental test without fear or favor.” Kyburg goes on to claim that these exhortations are particularly prevalent “in courses in methodology associated with the social and behavioral sciences.”62
Kyburg 1988, 61. How political scientists came to use the H-D method is explained easily when we reflect on the considerable confusion that exists in political science between theories and models. A theory, according to logical positivism (the philosophical basis for H-D), is a logical calculus along with a set of correspondence rules. The logical calculus is assessed by deriving an implication and testing it against data. It just so happens that the mathematical models used by social scientists are also in the form of a logical calculus. Over time, models assumed the place that theories had held in the logical positivist programme. This usurpation had a significant negative consequence—political scientists assumed that their models should be tested in the same way that the logical positivists claimed theories should be tested.
Those who require direct evidence of the connection between the logical positivist programme and the early rational choice modelers in political science need look no further than the writings of William Riker, a pioneer in the use of rational choice theory in the study of politics. Riker (1977, 13), citing the leading logical positivists (Nagel and Hempel, among others), argues for an explicitly “positivistic” or axiomatic view of science. To explain an event, according to Riker (1990, 167), is to “subsume it under a covering law that, in turn, is encased in a [deductive] theory.” The idea of a covering law comes from Hempel and Oppenheim's (1948) “deductive-nomological” model of explanation, which was the first complete version of hypothetico-deductivism; Brady 2004, 296.
An Example
A detailed example is the clearest way of illustrating the differences between the semantic approach and the current state of the discipline in political science. Presenting such an example is complicated significantly by the fact that despite a recent mention by Brady, and despite having made inroads into biology, psychology, economics, as well the health sciences and physics, the semantic view has yet to inform any discussions of political science.63
Brady 2004; in biology Beatty 1980, Thompson 1983, 1986, 1988, and Lloyd 1988; in psychology Hardcastle 1994; in economics Hausman 1992; in health sciences and physics Suppe 2000.
Achen recently proposed a rational choice model of political socialization that seeks to understand why children may try on their parents' party identification while at the same time gleefully rejecting their parents' advice on music and style.64
Achen 2002.
The puzzle is how a new voter, with no experience of her own to guide her, can rationally choose a party. (Bayesian updating takes place in the normal fashion after this initial choice.) The party benefits accruing to a voter, according to the model, depend on the voter's position in society, and the societal positions of parents and children are correlated. Thus, a new voter can draw upon the experience of her parents and estimate her future benefits. She can therefore choose a party rationally based on this estimate and update as she gains experience.
The interesting thing about Achen's paper, for the purposes of this article, is that with the exception of some language, there is no pretense of strictly following the dictates of hypothetico-deductivism. Achen's paper is an archetypal example of the semantic approach to model-based reasoning, though he almost certainly never intended it as such. As we demonstrate, the choices Achen makes in this paper are perfectly reasonable from a semantic perspective and quite puzzling from an H-D perspective.65
Achen even refers to “Bayesian voter theory,” which is simply a collection of Bayesian models investigating party identification, the effects of political campaigns, and vote choice.
The key insight of the semantic approach is that models are objects and thus are neither true nor false. Therefore, those scholars discussed earlier who are asking whether or not their models are “confirmed” by the tests that they run are asking the wrong question. The question they should be asking is whether or not their model is similar to the world in a specific way for a specific purpose. It is this question that drives Achen's piece.
At no point in his article does Achen make the claim that his Bayesian model is “confirmed” in the normal sense. Indeed, he specifically notes that “detailed empirical veridicality” is not the goal of the model, and he readily admits that his assumptions are “somewhat inaccurate.” Rather, the goal of the model is to account for what we already know about “parental transmission of party identification to children” for the specific purpose of understanding why children behave differently in the political arena than in the social arena.
The first step is to write down a model that is similar to the world in a particular way. For Achen, this means writing down a “stylized model” that generates qualitative predictions “matching the main empirical generalizations.” One purpose of Achen's model is therefore clearly structural. He is concerned with accounting for well-known features of political socialization, including the correlation between the partisanship of parents and their children, the relatively larger number of Independents among young voters, and the decay in partisan alignments over time. There is no pretense here that the model just happens to predict these findings; the model was designed to predict these findings. The fact that the resultant model does predict these findings is in no sense a test of the model. Rather, it is an indication that the model is similar to the real world in a particular way for a particular purpose (understanding the actions of new voters). Achen's claim that his model is more useful than the social psychological model for understanding intergenerational transmission of party identification (PID) is in this vein.
Achen is quite content to state in what ways his model is not similar to the real world. For instance, the model ignores the empirical claim that realignment is continuous (secular realignment) and not intermittent. Indeed, Achen notes that “greater realism” would be necessary for data analysis. Empirical verification of the model is not at issue, however. What is at issue is how well the model captures particular features of the real world.
If a model were simply to capture, in a coherent way, known facts about the world, it would make a contribution. One of the advantages of model-based reasoning, however, is the ability to use the model to further explore reality. Achen's model leads to Proposition 5, which states, “All else equal, the greater the changes in party policy across generations, the more centrist the initial PIDs of young voters will be.” Achen notes that this proposition has never been investigated. Unlike the proponents of “epistemic difference,” Achen makes no claims that empirical investigation of this novel proposition will “confirm” his model. Rather, the point is that the model, which is based on empirical findings, has led to new avenues of empirical investigation and is thus generative as well as structural. This kind of give-and-take between theoretical models and empirical findings is consistent with the semantic approach, but has no place under the rules of H-D, where deductive hypotheses are put to empirical test “without fear or favor.”66
Kyburg 1988, 61.
To make the point more forcefully, Achen's paper simply does not fit into the “deductive theory, prediction, test” approach to scientific research. Yet, it is unlikely that anyone would argue with either the paper's scientific status or the paper's contribution to the literature. The semantic approach requires the researcher to state the purposes of a model and its similarity and dissimilarity to the world. For Achen, the purpose of the model is to elucidate the intergenerational transmission of party identification. The model is similar to the world in that it explains three known empirical generalizations: the correlation between parents' and children's partisanship, the greater partisan independence of the young, and the decay of partisan alignments over time. The model is dissimilar to the world in that it is unconcerned with secular realignment and ignores the non-instrumental preferences that voters might possess. The model is successful in that it achieves its specific explanatory purpose and directs us to explore the effect of changes in party policy across generations on the centrality of the initial party identification of young voters.
Integrating Models and Data
The Achen example provides some insight as to how political scientists should approach their research once they have abandoned the practices of hypothetico-deductivism. In what follows, we distill these insights into a set of four “rules” for social scientific research. These rules are meant to be neither exhaustive nor overly strict, but instead are designed to serve as guidelines for integrating models and data.
1. Be clear about the purpose(s) your model is intended to serve. Any political situation may be modelled in an almost infinite number of ways and for a variety of purposes. Thus a researcher should strive to address the question, “why this model and not another?” Answers might reference any of the purposes of a model listed in table 1. One model might be chosen over another because it predicts particularly well. In another case, a model that lays bare a heretofore unrecognized causal mechanism might be chosen despite its lack of predictive success. A third model may be used because of its great fecundity in generating new avenues for research. In this way, the success of the modelling enterprise can be judged on its own merits and not on the artificial merits of H-D.
2. Abandon the goal of “model testing” as currently practiced. “Model testing” implies using statistical analysis to determine the truth or falsity (or any of the synonyms that political scientists use, such as “supported,” “confirmed,” “verified,” or “validated”) of a model, but as discussed earlier, the truth or falsity of a model is not the question. Rather, the point is demonstrating that the model is useful in a particular way. For the maker of a subway map, this work would entail demonstrating that the average commuter could successfully use the map to get from point A to point B. For the maker of a structural model, the work would entail showing that a model really does organize the data into a particular framework for understanding.
Whether or not this kind of demonstration requires data analysis depends. For the Achen model discussed in the previous section, data analysis was not required to make a compelling case for the usefulness of his model. One reason that Achen can get away without data analysis in his structural model is that the field of political socialization is peppered with a number of strong empirical generalizations. In a research area with fewer such generalizations, data analysis would be required to make a compelling case.
3. Include a data analysis only when the purpose(s) of your model is served by it. Not all models require an accompanying data analysis. That this point is widely misunderstood was dramatized by the recent reviewer-inspired decision (and subsequent retraction) by the editors of the American Journal of Political Science to publish formal theory only when accompanied by data analysis. Of the five different types of models listed in table 1, only predictive models need be accompanied by some form of data analysis. The reason is, of course, that prediction is the metric by which these kinds of models are judged.
The above does not mean that outside the realm of predictive models data analysis is useless; it means only that data analysis is not a necessity. Krugman writes that a model is good “if it succeeds in explaining or rationalizing some of what you see in the world in a way that you might not have expected.”67
Krugman 1994, 49.
4. Treat data analysis as more than an endpoint. On those occasions where models and data are integrated, too often the model is carefully developed over the first nine-tenths of the paper while an inconsequential data analysis is tacked on as the final one-tenth of the paper, no doubt to appease reviewers. Seeing data analysis simply as an endpoint is an unfortunate consequence of a focus on model testing. In place of an openness to new empirical findings that a model might incorporate and explain is a search for the “three stars” that indicate that the central prediction of the model is “confirmed” or “supported.” This mode of research can hardly be referred to as the integration of models and data.
True integration of models and data is not easy. Showing that a model is similar to the world in a particular way for a particular purpose often has description, as opposed to inference, as its goal, and to some political scientists, description is a dirty word evoking atheoretical accounts devoid of conceptual bite. When guided by theory, however, description becomes a powerful tool both for assessing the usefulness of a model and for opening new avenues for theoretical exploration.
Adoption of a model-theoretic approach to political science means more than simply changing the language of research. It means changing the ways in which we think about and practice political science.68
Language and practice are, of course, endogenous. How we speak and write about research informs the practice of political science.
See Morton 1999 for the current state of the discipline.
Conclusion
Political science is a model-based enterprise, but the discipline's understanding of the role and function of models remains rooted in the past. Under the current view, researchers more-or-less treat models as if they are truthful representations of reality and then test them by deriving predictions and using the accuracy of those predictions to judge whether the models are correct. Good models predict well, and bad models predict poorly.
This process, however, rests on two misapprehensions about the nature of models. The first is that models are either true or false, and the second is that we can learn whether or not our models are true or false. We argue instead that models are objects and thus neither true or false. We argue that models should be assessed for their usefulness for a particular purpose, and not solely for the accuracy of their deductive predictions. Models may be foundational, structural, generative, explicative, and/or predictive, and must be judged on the appropriate metric. We argue that viewing models in this way allows a wider range of evidence to be considered and frees empirical investigation from slavish devotion to statistical significance.
The implications of seeing models in this way are far reaching. Pure formal theory can be recognized as being as deeply substantive as any applied formal theory generating testable predictions. Constructing a model to match known facts can be recognized as a useful scientific activity, and a model that generates new directions for empiricists to follow is to be hailed. Our model-based approach highlights the centrality of models in scientific reasoning, avoids the pitfalls of hypothetico-deductivism, and offers political scientists a new way of thinking about the relationship between the natural world and the models with which we are so familiar. Furthermore, our approach brings true integration of models and data to the EITM project, which is currently training the next generation of scholars. Where an emphasis on H-D model testing narrows the range of acceptable research, our approach expands the ways in which models and data may be integrated and frees political science from the methodological straitjacket imposed by current practice.
The leading political scientists have always appreciated the importance of understanding, and when necessary, reimagining, the foundations upon which our work is built. We see this paper as being squarely within this tradition, and we hope to be engaged constructively by others willing to reexamine the underpinnings of our discipline.
