ORIGINS

Quantitative methods began appearing in the social sciences in the nineteenth century. The work was painfully difficult in that pre-Internet, pre-computer, even pre-adding machine era, and the initial applications used only simple statistical measures, though often with considerable substantive sophistication (Gow Reference Gow1985). By the early twentieth century, the University of Chicago pioneered the behavioral revolution. There, Harold Gosnell introduced randomized experiments and factor analysis to the discipline, and he made extensive use of regression analysis. Future Nobel Prize winner, Herbert Simon, studied with him. By the 1950s and 1960s, prominent scholars like Simon, Bill Riker, Don Stokes, and others were doing statistical and theoretical derivations for political science applications, not just applying tools from other disciplines.

As the computer age began, quantitative analysis became much easier, and younger scholars took to it in large numbers. High-quality academic survey research, for which the American National Election Studies (ANES) was the gold standard, also appeared in this era. Under Warren Miller’s leadership, the Inter-University Consortium for Political and Social Research (ICPSR) supplied these and other data to individual researchers at no cost. Suddenly, one could do in half an hour what had taken the pioneers many weeks. An explosion of important substantive findings resulted. Of course, qualitative and historical methods were then, and continue to be, intellectually important. They predominate in certain subfields and rightly so. But by mid-century, it was clear that political science was on its way to becoming a discipline in which the majority of research was done using quantitative research tools.

In parallel with quantitative applications, methodological research grew in importance during this period. The number of methodology articles in the discipline’s journals rose dramatically after the mid-1960s, although from a small base.^{Footnote 2} The American Political Science Review was the primary outlet for methodology articles until the mid-1970s, when Stokes established the Workshop section in the American Journal of Political Science. Political Methodology, edited by John Sullivan and George Markus, emerged in 1974. By the early 1980s, the methodology field was showing signs of health and growth.

Applied to much of this era, however, the term “methodology field” is a misnomer. There was no methodology field. US political science departments offered American Politics, Comparative Politics, International Relations, and Political Theory. Until the '60s, virtually no one was appointed as a full-time methodologist. Apart from a handful of pioneering institutions, full-service departments offered extremely few such jobs. Almost no one took graduate exams in methodology and formal theory. Before the 1960s, most graduate methods training was primarily practical, geared to teaching graduate students to use the limited available statistical software. But in this period, there began to develop a group of mostly young scholars who did proofs in class and who thought of themselves primarily as methodologists. Yet the great majority of them had been hired to teach something else.

The clash of outmoded academic structures versus the teaching and research identity of that group of scholars created discontent. When Steven Rosenstone used his discussant role at an APSA meeting to propose that an organization of political methodologists be formed, the idea struck like a lightning bolt. It sounded great. But there were so few scholars involved, none was yet well known, and research funds were scarce. Would enough people sign up? Would they pay money to belong to an APSA section? How could an annual section meeting be funded? Where would money be found so that graduate students could be subsidized to attend? It all seemed impossibly challenging. A very large investment of time and energy, followed by organizational collapse, seemed the most likely outcome.

Nonetheless, due to the hard work of everyone involved, along with key support from more senior scholars such as Warren Miller, those challenges were met. Those who attended methods panels at the annual APSA conventions were invited to apply to attend a first meeting, and they were asked to spread the word to graduate students. In 1984, the first summer meeting of the Society for Political Methodology took place in Ann Arbor with support from ANES. About a dozen scholars attended and actively participated in the full program.^{Footnote 3} At the end of that meeting, the chair bravely announced that “the first annual meeting of the Society for Political Methodology has concluded.” In fact, no one knew how a second meeting would take place, much less an annual event. Yet the group met in Berkeley the following year with increased attendance. The APSA Political Methodology section became established, and under John Jackson’s leadership, National Science Foundation support was obtained for the summer meetings, which have taken place annually ever since.

Scholarly communication was updated as well. Political Methodology evolved into Political Analysis under the editorships of Chris Achen and Jim Stimson, with the first volume of the new journal published in 1989. The society began publication of The Political Methodologist (TPM) newsletter in 1988 with Gary King as editor. The Summer Program of ICPSR under Hank Heitowit’s leadership responded quickly to the rise of the section, offering an expanded list of advanced courses designed to convey a broad range of specialized topics to the profession. The Sage Quantitative Applications in the Social Sciences (QASS) series also produced a series of successful popular monographs on a variety of methods topics, designed to gently introduce the basics of the new methodologies to a wide audience, primarily students. These early successes resulted in a broader range of methods publications in political science journals, as well as increased APSA section membership and growing participation in the summer meetings. Annual meetings now attract well over 150 attendees, methodology is a standard field at most research departments, and methods positions have become a standard career track in the discipline. With a few important exceptions, the early methodologists focused primarily on American politics, but comparative politics and international relations subfields now draw substantial methodological attention and interest as well. Particular topics now have the critical mass to create their own APSA sections, including political networks, political forecasting, and the increasingly sophisticated field of qualitative methods, which often has a substantial multi-method component.

Thus, the section has succeeded far beyond the dreams of the first generation. In a great many respects, the methodology section now enjoys robust health. But with that health have come new challenges, challenges not addressed in the first decade or two, when simply getting the organization set up and solvent consumed the time of everyone involved and perhaps made them less aware than they should have been of the new tasks arising.

The focus on the mathematical side of political methodology in those early years, while probably necessary at the time, did set a pattern and create a set of expectations about the field. Certain kinds of work counted as real methodology, while others did not. Even critical parts of quantitative research, such as the formulation, conduct, and dissemination of opinion surveys, were largely set aside, though statistical theories of survey sampling and respondent weighting were well advanced among statisticians. A great many other aspects of data creation and their statistical properties, such as those appearing in the rapidly developing event data field in international relations, similarly received only a little attention, as did the challenges of case selection and qualitative inference that loom large in comparative politics. In consequence, the professional identity of political methodologists took on a certain character. That character has endured.

CONSTRUCTING IDENTITY

In the wake of the early years of self-definition, the work of political methodologists has not changed much in the past several decades. Methodologists identify research questions and theories in political science that have encountered methodological difficulties of a statistical kind, and they develop new strategies, tools, and estimators to better answer those questions. Political methodologists also teach methods courses in their departments and help integrate their graduate students into the methods community. They often serve as consultants for their colleagues, helping them to identify the best methodological approach for their research project and assisting them in carrying it out. Political methodologists publish research in journals such as Political Analysis and develop software to enable a broader set of political scientists to employ the statistical approaches they advance. In short, the general character of research and teaching activities for our field are very similar today to what they were in the 1980s when the society’s first summer meetings occurred.

What has changed is the range of what methodologists are expected to know. In the 1960s, graduate students were fortunate if their department offered two methods courses. By the 1990s, a typical graduate program might have offered up to four methods courses, with an emphasis on research design, introductory statistics, regression models, and some advanced topics that built upon regression, such as simultaneous equation models or time series analysis. Much else was missing. For example, maximum likelihood estimation (MLE) was taught as the basis for estimating probit and logit models (typically with some mention that ordinary least squares regression analysis with normally distributed errors was also MLE), but except in special cases, the computing power needed to do MLE was not available until the advent of personal computers. Hence MLE was not a central part of political science graduate courses until the early 2000s. To get additional training in MLE and other specialized techniques, graduate students had to attend the ICPSR Summer Program or seek out courses in other disciplines on their campuses.

Many programs today have a much wider coverage of topics in the methods sequence. For example, the University of Iowa’s Department of Political Science requires four courses (research design/philosophy of science, introductory statistics/math, regression/MLE, and advanced models) and offers a number of elective courses beyond those topics (e.g., advanced computing in R/STATA, Bayesian statistics, text analysis, network analysis, spatial models, and time series analysis) as well as several shorter workshops on specific methods topics (e.g., geographical information systems [GIS]). In most programs, too, causal inference and experimentation are more likely to form the conceptual foundation for methods training than they once did—a help to conceptual firmness if they do not become a straitjacket. Wise political methodologist graduate students still take courses in other departments on campus and seek out summer or other training programs to advance their skills, but they can get much further within their own departments than they once could.

The shift in approaches and topics in political methodology reflects both advances in related disciplines and new substantive problems arising in our own discipline. Interest in some topics, such as Bayesian statistics, stemmed initially from the increased attention to them in related disciplines, especially statistics and economics. On the other hand, interest in topics such as event counts, event history, multilevel models, and network analysis was driven more by the topics we study in political science. Much of the work of methodologists continues to be informing political scientists about developments in adjacent fields and adapting their models to our needs. As Lewis-Beck (Reference Lewis Beck, Box-Steffensmeier, Brady and Collier2008) notes, most methods publications in political science take a “missionary” approach, introducing and applying techniques from other disciplines to our own. This kind of work is quite valuable because it is critical to know how particular tools work with our data. Which covariates need to be balanced? Which regression discontinuities work and which do not? Careless application of statistical tools by methodologists who lack substantive knowledge is just as dangerous as the reverse error.

Valuable as missionaries are, however, they do not meet all needs. Political methodologists who act more as “theologians,” creating “methodological tools from within political science itself” are less common (Lewis-Beck Reference Lewis Beck, Box-Steffensmeier, Brady and Collier2008, 824), though their numbers are slowly growing. And one can certainly point to methodological research areas where political scientists have played a central role, such as ecological inference, the dynamics of opinion change, and the scaling of legislative votes. Some of the most interesting advances in political methodology have occurred when they were motivated by political questions, and of course, that is precisely the kind of work that is of interest to the rest of the profession.

Maintaining the deep substantive interests that lead political methodologists to consequential research is not easy. Even within the topical bounds that political methodologists have set for themselves, the depth of technical specialization required for newly minted political methodologists to be competitive on the job market is daunting. Sometimes their graduate training is in another discipline entirely. Knowledge of politics and political data risks being driven out, remedied only in part by working with more knowledgeable colleagues in substantive fields. The consequence, as one of us has written, is that “political methodologists often look past key inferential problems in the discipline because we know too little about the issues that raise them” (Achen Reference Achen2014, 1).

Graduate students facing these conundrums must make choices, and the issues are particularly acute for those wanting to be methodologists. What is the right balance between technical sophistication and deep theoretical and empirical knowledge about political science that will advance the science? Questions of that kind are not easily answered, and the decisions will inevitably and properly differ from one student to another. But what we want to insist on here is that there is no hierarchy based purely on mathematical skill. The methodological identity coming out of the early years of the society is only part of what will be needed. Scientific advances are going to come from all sorts of people. Telling a beginning graduate student who is not at the top of the class in doing statistical derivations that “you’ll never be a methodologist” is not just bad teaching, but a sign that the speaker knows too little about the discipline and about science generally. Albert Einstein was very well trained in mathematics, but what he knew deeply was physics. To flesh out his theories, he needed help from colleagues with more mathematical skill and training than he had. Their names have been forgotten.

DIVERSIFYING POLITICAL METHODOLOGY

The founding members of the Society for Political Methodology were primarily white men interested in American mass political behavior, and the initial orientation of the society reflected those interests. Like the political science profession as a whole, the political methodology community has become a more diverse group over time, with resulting changes in substantive interests and methodological concerns. For example, the special hurdles of studying minority populations of all kinds are now better recognized. American political behavior is no longer the dominant substantive interest among quantitative researchers, who are often focused on international relations, the electoral politics of other continents, or on political institutions anywhere in the world. How well has the society incorporated new members, new perspectives, and new preferences about professional interactions at conferences?

We focus here on gender diversity because that is what we are best prepared to discuss.^{Footnote 4} Diversity patterns in political methodology are important because they provide insights into how methods scholars not only define what it means to be a member of their group, but also whether that definition is narrowly or widely constructed. Does increased training at the department level for political methodology make the field more accessible to a diverse group of scholars? Or do the widening expectations for expertise and skill sets for political methodologists further stymie our efforts to diversify our membership?

In figure 1, we present data on the percentage of female paper presenters at the summer political methodology meetings for all years of the organization’s conferences (1984–2016). The average is 12%, with a range from zero (1985, 1988) to 22% (1994, 1999). The mean has increased over time by an average of about a third of a percent per year, although we note that there is no clear trend in the last decade. Of course, the number of participants has increased dramatically as well, from 16 attendees in 1984 to more than 175 in the past few years. Thus while there are many more females participating in the summer conferences than there once were, they still constitute only 20% of the total presenters. This figure is somewhat below the averages for other political science organizations of similar size with a quantitative orientation.^{Footnote 5}

Figure 1 Female Percentage of Paper Presenters at Political Methodology Summer Meetings, 1984–2016

Enhancing diversity in the society became a more serious priority when Janet Box-Steffensmeier became the society’s president in 2005. She helped to create funding for the Visions in Methodology (VIM) conferences by embedding those monetary requests in the overall National Science Foundation grant application that supported the society. The VIM conferences are designed to mentor women in political methodology and help recruit more female attendees to the summer political methodology meeting (Barnes, Beaulieu, and Krupnikov Reference Barnes, Beaulieu and Krupnikov2014; Dion Reference Dion2014). Box-Steffensmeier also hosted the first VIM conference at Ohio State University, and she helped recruit other scholars to host nine additional conferences. The diversity committee for the society was created in 2006, although its influence and activism have varied over time. As directors of the ICPSR Summer Program, Hank Heitowit and Bill Jacoby also helped to create funding awards for women and underrepresented minorities to attend ICPSR (the Box-Steffensmeier and Garcia awards).

One of the potential barriers to recruiting a diverse pool of participants is arguably the stereotypically-male argumentative and competitive nature of the summer meeting (Shannon Reference Shannon2014, 3). Too often, beginning scholars have found it necessary to defend themselves by declaring that they were not actually methodologists, hence not part of the amateurish status-seeking.^{Footnote 6} As with any conference, a less welcoming atmosphere influences scholars’ interest in returning after they have attended their first meeting. Dion and Mitchell (Reference Dion and Mitchell2012) studied this pattern from 1984–2010; participants were treated as repeat attendees if they attended at least two summer conferences. Overall, the female repeat attendance rate (37.5%) was lower than the male repeat attendance rate (44.4%). However, women’s repeat attendance rate increased by about 10 percentage points in the later time period analyzed (2002–2010 compared to 1984–1992). This suggests that efforts to promote diversity have helped bring women back to the political methodology conferences. The greater size of the group helped create more cooperative interactions. The larger number of poster presentations also fostered a better atmosphere for junior scholars to present their research.

One of the recent diversity efforts was the organization of women’s dinners on the first night of the summer meetings (Dion Reference Dion2014). While some scholars enjoyed the opportunity to network with other women, others lamented the lack of senior women at the dinners. This was somewhat ironic given that the absence of senior women reflected a systematic decision by earlier female conference participants to exit the society. The result was an unfortunate dynamic where some women sought to distance themselves from other women in the community (Morrow-James and Box-Steffensmeier 2014, 18).

On the other hand, the VIM conferences were successful in building a female political methodology network. A study by Barnes, Beaulieu, and Krupnikov (Reference Barnes, Beaulieu and Krupnikov2014) found that VIM participants viewed themselves more as methodologists than did a female control group, they had broader conceptions of what it means to be a methodologist than either the male or female control groups, and they gave invited talks and submitted research articles to journals at higher rates than women who had not participated in VIM. These findings are similar to the patterns that Blau et al. (Reference Blau, Currie, Croson and Ginther2010) identify in their analysis of the CEMENT mentoring program in economics. Both studies point to the importance of mentoring activities in political methodology to improve the diversity of our community (Boehmke Reference Boehmke2014).

Applications that were submitted to the first few VIM conferences produced an interesting pattern. Many of those applicants had never attended a summer political methodology meeting. Why were female scholars comfortable submitting their research to VIM but not the society’s more general meeting? Partly this reflects the reputation that the society had for not being particularly open and inviting, especially in its earlier years. It may also reflect the lack of networking and mentoring opportunities for women in political methodology compared to men (Shannon Reference Shannon2014).

To investigate gender differences in participation, the society supported a randomized controlled trial that involved inviting both men and women to apply to the summer political methodology meeting (Unkovic, Sen, and Quinn Reference Unkovic, Sen and Quinn2016). The authors found that while women who were selected into the treatment group (and hence received two emails encouraging them to apply) were more likely to apply, they did not have a higher chance of being accepted to the conference. This resulted in part from the supporting reference letters that many of these female applicants were unable to secure.

These results demonstrate the importance of thinking broadly about recruitment strategies for both women and minority populations. Our traditional approaches, stemming from professional identities formed almost half a century ago, may be flawed.^{Footnote 7} As many universities have learned, being open to underrepresented groups and offering scholarship money do not suffice to raise minority presence on campus. People need to feel connected and welcomed, with a campus atmosphere that is supportive and not dominated by other groups. Most social scientists know the decades-long body of research that backs those findings. We are quick to see and condemn the blindness of business and workplace organizations that fail to adapt to changing norms in a changing America. Now the trick will be to open our own eyes wider, too, and to build a broader and more inclusive definition of what it means to be a political methodologist.