It is a great honor to have been invited to deliver the Ithiel de Sola Pool lecture. Professor de Sola Pool had a tremendous influence on scholarship and practice. His work describes the ways in which evolving technologies fundamentally alter communicative expectations and social outcomes. Years after their publication, works such as Technologies of Freedom (Reference De Sola Pool1984) continue to provide insight about how the networked communications are changing the world. Professor de Sola Pool's work is of great public value. In what follows, I want to focus our attention on the value of social science today.
Figure 1 Ithiel de Sola Pool Lecture Given by Arthur Lupia
Photo by Mike Moffa.
What is the public value of social science research and is it so valuable that Congress should continue to fund it? In recent years, these questions have gained increased attention. Much of this attention comes in response to a series of proposals to curtail federal funding of social science research. These proposals have focused on the National Science Foundation (NSF). Several elected officials have asked why NSF funds the social and behavioral sciences at all (Cantor Reference Cantor2013). Others have sought to limit the kinds of social and behavioral science research that NSF can fund (Coburn Reference Coburn2011). In 2013, an effort to apply special criteria to NSF political science funding was successfully attached to a continuing resolution and is law, at this writing (see the description in Farrell Reference Farrell2013).
These actions have been widely criticized. Science and Nature, two of the most widely read general science journals, offered unusually strong critiques. Nature's editorial (2012) emphasized political motivations underlying reversals to Congress' long-standing support of social science (also see RK Wilson Reference Wilson2013). Nature referred to one proposed amendment to limit political science funding as “no different in principle to the ideological infringements of academic freedom in Turkey or Iran.” Science published a front-page article titled “Is Any Science Safe?” (Prewitt Reference Prewitt2013). The article describes the broad public benefits of social science and the ways in which proposed congressional limitations could threaten military, medical, and governmental effectiveness (see also Lupia Reference Lupia2012). The popular press has run similarly themed stories (e.g., Krugman Reference Krugman2013; Noah Reference Noah2013).
Most social scientists, however, have remained silent about these challenges. Many are hoping that other people will make political questions about federal funding of social science go away. This is not going to happen. Questions about the public value of social science and whether Congress should pay for it will continue because at least two countervailing forces fuel divergent views about how to answer these questions.
One force is the sheer size and influence of modern social science. Today, social scientists conduct research on more topics in more parts of the world using more methods and speaking to more audiences than in any previous era. In the United States, this research is supported not just by NSF, but by a wide range of governmental agencies. At the federal level, social science is critical to the effectiveness of important operations in the Departments of Defense, State, and Homeland Security just to name a few. Social science plays a similar role in many state and local governments. It informs and influences numerous decisions.
The other force is the cost of modern social science. Social scientists are experts in drawing effective and actionable conclusions about a wide range of phenomena. Effective research of this kind requires well-prepared scholars. Such preparation often entails extensive training. This training involves teams of highly skilled people devoting themselves to the skill development of others. In terms of time and effort, such training can be quite costly. Social scientists are also asked to produce precise technical instruments. Data collection and analyses, performed by entities such as the United States Census, and strategic and tactical planning in military operations, cost tens of millions of dollars and thousands of person-hours to produce.
When funding for social science training and products comes from governmental sources, questions follow about whether the benefits of these efforts justify the taxpayers' expenditure. Skepticism about the public value of social science arises, in part, because its social benefits are not always apparent. Among the factors that make the value of social science difficult to perceive are translation problems and the distinction between basic and applied social science.
On the topic of translation, much social science is written in technical language. This language, while allowing more precise descriptions of concepts and relationships than ordinary language, can make it difficult for nonscientists to understand why they should pay for it. While technical work is often needed to unlock the code of important social phenomena, social scientists often fail to communicate how such work benefits society (Lupia Reference Lupia2013). Social scientists are not routinely trained to effectively communicate the value of their technical findings. Many lack career incentives to make their work understandable to a broader audience. To the extent that social scientists want governments to fund their work, organizations that train and employ social scientists should devote more attention to effective presentational skills.Footnote 1
The other issue is that some social science does not seem to have an immediate and obvious link to critical social problems (Isaac Reference Isaac2013). For example, some research is aimed at uncovering basic mechanisms and processes in the social world. Social scientists need to be more proactive in explaining how these “building block” findings connect to explanations of important phenomena and to potential solutions for associated problems. Of course, there is great value in basic research. Our society has been transformed by it in many ways. No one can deny this. At the same time, there are also studies with little or no potential for affecting high-value public outcomes. No one can deny this either. The challenge for social science is help ourselves and others better understand the difference between the two activities.
So, when people frame proposals to reduce social science funding as an indefensible attack on reason, I am not convinced that this framing is the best way to address the quandary facing our prospective funders. For when taxpayers and their elected representatives ask about the value of seemingly obscure research whose public payoff is never or not well explained, they are exercising due diligence (Cantor and Smith Reference Cantor and Smith2013). While scholars are free to pursue such research on their own time, the public has a right to decide to pay for other things instead. When social scientists cannot make effective arguments about the public value of their work, the task for potential supporters to make the case for government funding becomes increasingly difficult, and the job for people who wish to characterize social science as folly becomes easier. This is our problem to solve. We cannot wait for others to make it go away. For us, government funding of social science cannot be a proposition to defend only when it is threatened. Conveying the relevance of social science to high-value outcomes is an activity in which our disciplines must be perpetually engaged.
In that spirit, I offer an argument about the public value of social science and a nonpartisan rationale for congressional funding. In these remarks, I work from two premises. My first premise is that support for social science funding is not something to which social scientists are entitled. As former APSA President Matthew Holden explained in his eloquent address to the association in 1999 “not everyone sees the value of our work as we do” (Holden Reference Holden2000). From this premise, I offer a framework for articulating the public value of social science. My second premise is that nothing in the Constitution requires Congress to fund social science. From this premise, I argue that the rationale for congressional funding must come from another part of the Constitution and I offer a framework for doing so.
WHAT IS THE PUBLIC VALUE OF SOCIAL SCIENCE RESEARCH?
Decision makers often face complex challenges. On topics ranging from security, health, and the economy, many decision makers in the public and private sector have incentives to seek the best available information in their quests for greater effectiveness and efficiency. By providing needed information in a timely and usable format, public investments in credible forms of data collection and analysis can be crucial to the success of a wide range of public and private decision makers.
Social scientists examine a wide range of phenomena about which policy makers and private sector decision makers care. Social science has developed a range of powerful methods for collecting and analyzing many kinds of data (see, e.g., Prewitt, Schwandt, and Straf Reference Prewitt, Schwandt and Straf2012). Social science has produced methods that many people can use to rigorously assess the plausibility and accuracy of competing explanations of critical events. When individuals and institutions around the world have questions about the validity and reliability of causal claims in social and behavioral domains, social science provides much of the knowledge base for the best available answers.Footnote 2
In many cases, however, decision makers have access to more information than they can possibly use. When decision makers can choose from multiple sources of information, the quality of their decisions can depend on which information they choose to believe and which information they choose to ignore (see, e.g., de Sola Pool and Kochen Reference De Sola Pool and Kochen1978; Lupia and McCubbins Reference Lupia and McCubbins1998; Gigerenzer, Todd, and ABC Research Group Reference Gigerenzer, Todd and Group1999). Those whose goals and ambitions involve working for a government agency, moreover, receive information from various constituents and interest groups who offer wide-ranging opinions about the consequences of certain actions and about what government should do. These opinions give our democracy great energy, but when it comes to developing and implementing effective ways to serve the public, knowing which claims to believe can be the difference between success and failure.
One attribute that distinguishes social scientific explanations from others is that social science offers an increased capacity for honesty in attempts to characterize social and behavioral phenomena. For some readers, this description may seem odd. After all, some social scientific analyses are so complex that they complicate more than they clarify. As Mark Twain (Reference Twain1906) once said, “There are three kinds of lies: lies, damned lies, and statistics.”
What gives social science its distinctive potential to expand our capacity for honesty is its norm of procedural transparency. I contend that strong answers to questions about the public value of social science can be built on information about how its claims are produced. Social science produces descriptive inferences, causal inferences, and interpretations of events that are more than just data collections. Credible social science conclusions are drawn from methods grounded in rigorous introspection about what individuals can and cannot claim to know about the world. When researchers are transparent about the procedures that they use to produce and evaluate their conclusions, they give these conclusions a meaning that others can inspect for themselves. Such processes give social scientists the ability to tell decision makers in the public and private sector things about the world that they could not have discovered on their own or are unlikely to have been able to rigorously verify on their own.Footnote 3
Social science's increased capacity for honesty comes from researchers' willingness and ability to publicize the path from a particular set of observations to a particular conclusion about the social world. Indeed, the focal expectation among many social science research traditions is that the meaning of a conclusion should not depend on who is making the claim or on irreproducible procedures. As King, Keohane, and Verba (Reference King, Keohane and Verba1994, 6–9) describe, the hallmarks of scientific research are that “the goal is inference”, “the procedures are public”, “the conclusions are uncertain”, and “the content is the method.” So, if recipients of social scientific analyses view the procedures as legitimate, then they have a basis for valuing the resulting claims.
This capacity for honesty is particularly valuable when vested interests seek to interpret important events in self-interested ways. Consider, for example, the many interpretations that follow monthly releases of economic data. The current government's supporters often claim the data as evidence of their success, while the government's opposition alleges that the data signals the government's failure (C. Wilson Reference Wilson2013). At such moments, societies benefit from being able to differentiate false stories from explanations that are consistent with transparent logic applied to the best available evidence. Social science can provide such benefits.
While social science can provide an understanding of important social phenomena that is precise and credible, it does not always do so. Social scientists are paying increasing attention to questions of research transparency (see, e.g., Lupia and Elman Reference Lupia and Elman2014). Across the social sciences, there have been many cases in which research claims cannot be replicated and, in a few cases, instances of fraud (Bartlett Reference Bartlett2012). Scholars who seek to have social science fully leverage the legitimacy that comes from being transparent about how knowledge claims are produced are pursuing a series of projects that make sharing data and information about scientific procedures easier and more appealing to scholars. The Reproducibility Project in psychology (Open Science Collaboration 2013) and Data Access and Research Transparency (DA-RT) in political science (Elman and Kapieszewski Reference Elman and Kapieszewski2014; Lupia and Alter Reference Lupia and Alter2014) are examples of broad collaborations that can help scholars document how they know what they claim to know.
Social science not only lays claim to having valuable procedures for generating knowledge claims with high truth content, it is also willing and able to lay those procedures bare. Transparency, in turn, generates self-correcting mechanisms, where replication uncovers errors, and the possibility of replication encourages rigor. This openness distinguishes science from many other sources of information about social and behavioral topics (e.g., news reports, interest group claims). Social science gives decision makers the ability to make better decisions about what kinds of claims to believe. When social science provides such opportunities in the context of important social decisions, it provides value to society by helping the public and private sectors evaluate their feelings and beliefs with respect to credible data and replicable logic.
Government support of social science provides opportunities for its citizens to convert a wide range of information about the world into actionable insights. Today many people in the private and public sector use such knowledge to improve productivity and efficiency in many socially valuable domains. In addition to informing government activity, social science helps a diverse range of nongovernmental organizations, faith-based institutions, community groups, and educational endeavors achieve their missions more effectively. These organizations use science to understand numerous complex issues. In cases where intuition or misreadings of past events can cause us to make bad decisions, social science can help us evaluate these beliefs with logic and evidence.
Consider, for example, the role of social science in what we can learn about military strategy from the war in Iraq. In 2003, troops from the United States and other nations invaded Iraq. Within a few weeks, these military forces assumed control of Baghdad and Iraq's president Saddam Hussein was removed from power. While many people hoped that the invasion would produce a relatively quick and stable transition of power in Iraq, the months and years following the invasion became increasingly characterized by insurgency and a wave of sectarian violence. This was a tragic sequence in so many ways.
In 2007, the United States embarked on a new military strategy in Baghdad. The foundation of the strategy was a “surge”—a massive increase in the number of troops on the ground in Iraq supplemented by increased training of Iraqi security forces. In 2008 and 2009, violence in Baghdad and other important areas of Iraq decreased significantly. As elected officials, members of the military, and the public attempted to understand the sequence of events in Iraq, questions lingered about the relationship between the surge and the fall in violence. Although some leaders claimed that the surge caused the decline, others pointed to a different factor. Much of the violence in Baghdad was the result of Sunnis attacking Shias and vice versa. As these attacks escalated a migration pattern commenced. Whereas before the invasion Sunnis and Shias often lived in close proximity to one another, the violence produced significant sectarian unmixing. Sunnis moved out of neighborhoods where they were outnumbered. Shias did the same. Some analysts claim that it was not the surge that reduced violence, but rather the rise of homogeneous ethnic enclaves that, by reducing sectarian contact, reduced sectarian violence.
A social science study conducted by Nils B. Weidmann and Idean Salehyan and funded in part by the Air Force Office of Scientific Research sheds important light on military strategy in response to ethnic strife (Weidmann and Salehyan Reference Weidmann and Salehyan2013). From a range of different sources, Weidmann and Salehyan accumulated detailed data on the ethnic composition of Baghdad neighborhoods, changes in settlement patterns across these neighborhoods over time, and changes in the number of violent acts in each neighborhood over time. Weidmann and Salehyan used a computational model to examine correlations between the timing and nature of violent acts and the movement of various populations within and across Baghdad. This evaluation led them to identify violence-migration correspondences that were consistent with the best available data. With such results in hand, Weidmann and Salehyan simulated the effects of various policing strategies. These simulations allowed them to produce and evaluate estimates of whether and how a surge could be effective.
Their findings shed important light on Baghdad before and after the surge. Their data showed that most attacks were against nearby sectarian rivals. If a person was among a sectarian minority within neighborhoods where attacks were occurring, they searched for safety by moving to areas where their sect was in the majority. Examined many different ways, their analysis makes clear that violence-induced sectarian migration played a significant role in reducing Baghdad violence.
Their findings also provided an important insight about the surge. Timing matters. Their work supports the hypothesis that if a surge is going to be pursued, it is much more effective when it is done in the early stages of a conflict. The intuition is that if a surge is delayed, then individuals and groups that are prone to violence will accumulate a rationale for violent action (revenge for attacks against their members) and will have made efforts to organize their resources for increased effectiveness at violence. When groups have such rationales and resources, the troops have to work much harder to reduce the level of violence.
Findings such as those of Weidmann and Salehyan, while deeply rooted in research, are not simply academic matters. Military actions entail considerable risk for the servicemen and women who are sent to implement a strategy. When the sacrifices of our servicemen and women are at stake, it is not enough to rely on intuition or wishful thinking about the effectiveness of the mission on which they are to embark. At these moments, these individuals and our nation benefits from distinguishing false stories about cause-and-effect from explanations that are consistent with precise logic and the best available evidence. This is the type of activity that social science is exceptional at identifying and supporting.
Social science is also proving useful in informing the public and private sectors about the most efficient and effective ways to improve public health. One such example, in which I was involved, shows how even a small amount of funding can have a large effect. In 2001, NSF funded an interdisciplinary infrastructure project titled Time-Shared Experiments for the Social Sciences (TESS http://www.tessexperiments.org/). Diana Mutz and I were the founding principal investigators.
TESS invited researchers to run innovative experiments on large national respondent samples. At that time most social science experiments were conducted on undergraduates or residents of college towns. TESS gave investigators an opportunity to evaluate social scientific research hypotheses on a much more diverse group of people (Loftis and Lupia Reference Loftis and Lupia2008).
In the project's early days, I was approached by a young doctor who wanted to know whether these methods could be used to help physicians across the country better understand and more effectively serve their patients. We discussed a variety of options and soon he sent a proposal to TESS. The proposal went through TESS' review process and within months we were fielding the study. The question pertained to the best way to make newly approved vaccines available to patients (Davis and Fant Reference Davis and Fant2005).
At the time of the study, employer-sponsored health plans were the dominant source of health insurance for children and adults. A typical plan was the result of a negotiation between an employer and a health insurance provider. The employer and the provider would agree on a contract that set rates and benefits for a fixed period of time—often several years.
During the contractual period, the Centers for Disease Control and Prevention would regularly identify new vaccines with public health benefits. These new vaccines were rarely named in the existing contracts. As a result, most Americans would have to wait several years before their insurance would cover the cost of newly approved vaccinations.
The TESS study by Matthew M. Davis and Kathryn Fant examined the public health consequences of this situation. Their study compared citizens' willingness to pay out of pocket for new vaccines to an alternate method of providing the benefit. In this alternate method, citizens were offered what economists call an “options contract.” David and Fant asked people if they would be willing to pay a small additional amount per month ($3–$6) for immediate access to newly approved vaccines.
The study revealed important information. While fewer than half of the participants were willing to pay out of pocket for new vaccines, more than three-quarters of respondents would pay for the options contract. This finding is noteworthy not just for the public health benefits arising from increasing vaccination rates, but also because of the actual costs to insurance companies of providing such coverage. The additional premium of $3–$6 per month that was offered to participants in the study was an intentional overestimate of the amount that it would cost an insurance provider to offer the option. In fact, the best available estimate of the maximum impact of a new vaccine on a monthly premium was 1/12 the cost of the options contract that three-quarters of the participants accepted.Footnote 4 In other words, the study shows that many people would be willing to pay health insurance providers $3–$6 per month for a service that would significantly improve public health and cost the providers pennies per month to provide.
This study, while deeply rooted in research, is not simply an academic matter. Finding ways to provide health care in efficient and cost-effective ways is critical to the health of many companies and, many would argue, to the nation as a whole. When the health of our citizens is at stake, it is not enough to rely on intuition or wishful thinking about the effectiveness of various health provision strategies. At these moments, these individuals and our nation benefits from distinguishing false stories about cause-and-effect from explanations that are consistent with precise logic and the best available evidence. NSF has been exceptional at identifying and supporting this type of activity. Moreover, the doctor who ran this TESS study continues to integrate social scientific methods and public health issues in his work. He now leads the National Poll on Children's Health (http://mottnpch.org/) that is helping many doctors and patients better understand one another as they work together to improve childrens' health outcomes.
SHOULD CONGRESS FUND SOCIAL SCIENCE RESEARCH?
In this final section, I want to address a topic that I have been asked about many times in recent months. As you may know, questions have been raised about how and whether Congress should fund social science research. In recent months, I have been told that a bill had been drafted that would eliminate the entire social and behavioral sciences directorate at NSF. At the same time, the State Department, the Department of Homeland Security, the Department of Defense, and the Department of Energy, to name a few, are all seeking social science expertise to help them accomplish their missions. So there are clearly different points of view about the value of social science research to American taxpayers. With such differences in mind, let us address this question: What is the role of Congress in funding social science research? While the answer that I will give to this question can be applied to other scientific activity, I will focus on social science because that is where the recent debate is focused.
A productive way to think about the question is to first remind ourselves about the role of Congress and then to think about whether and how funding social science research helps Congress do its job. Congress' job, as stated in the Constitution is to “establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity.” To understand the role of Congress in funding social science research, we must understand when and how does scientific research help Congress achieve the Constitution's goals.
Science plays an important role in American life. Scientific discoveries help individuals, firms, and governments improve the lives of people at home and abroad. Science makes our factories more efficient, our farms more bountiful, our military stronger, and our doctors better able to save lives.
For many decades, America has been the global leader in science. America's science leadership did not happen by accident. America's science leadership owes a huge debt to congressional foresight. In particular, Congress' creation of the NSF presented America with widely admired abilities to produce research in the public interest. Through rigorous peer-review processes, NSF brings wisdom to complex topics by supporting research that evaluates competing scientific claims. In many cases, NSF chooses which proposals to fund through highly competitive processes. In each competition, it puts forward a question of national interest and asks researchers to bring the best available scientific methods to bear on them. NSF does this because it knows that knowledge often advances faster when scientists are free to rigorously evaluate competing ideas. Freedom and competition provides the foundation from which scientific innovation and leadership emerge.
As the previous sections show, social scientific research has provided great value to the nation. But should Congress pay for it? Let me propose that Congress is not obligated to spend a single cent on scientific research. Its obligation is to the American people under the framework of the Constitution. So the question we have to ask is whether Congress choosing to fund social science research is essential to its obligation to the American people.
It is possible, for example, that social science is valuable, but that Congress need not fund it. This proposition merits serious consideration. We have to recognize that a term like “political science” sounds odd. To the general public, much of government and policy seems anything but scientific. Many of us, however, have seen that using science to understand critical elements of how modern societies organize ourselves to meet important challenges, and why we sometimes fail to do this, is serious business. Ample evidence suggests that supporting social science is critical to any modern society that wants to become or remain effective and great.
However, some people claim that you do not need science to understand government and policy because people can explain these things without science. That's partially right. Americans explain politics, policy, and government in many different ways. However, people who tell such stories often emphasize the world as they want to see it and are less interested in objective evaluations of whether their stories are true. When people's lives and livelihoods are at stake, our nation benefits from distinguishing false stories from explanations that are consistent with precise logic and the best available evidence.
Other people claim that the private sector will fund basic research on science and government. It is true that many firms use the social sciences to inform and advance their operations. But most firms have very specific interests and lack a mandate to support broadly beneficial science. Economists refer to such situations as “free rider problems.” Free rider problems explain why advanced societies use government (rather than waiting around for voluntary contributions) to fund justice systems, interstate highways, and soldiers. As a public good, basic social science is unlikely to be provided in sufficient quantity by the private sector. So, when it comes to promoting science of broad public value, leading nations have chosen to combat free rider problems by empowering national science foundations—and, for decades, our nation has led the way in this regard. But should Congress continue such funding?
Given that social science can be expensive to produce, we should expect those who pay for it to ask questions about the return on their investments. To those questions, we must be able to explain the many ways in which social science provides insights that are useful for governance. It helps communities, cities, states, and nations better serve their citizens, and it helps us advance multiple national priorities, like homeland security and job creation. Social science does all of this by increasing our ability to operate from knowledge, rather than being captive to untested stories, when designing and evaluating government programs. This is why social science plays an increasingly critical role in how America “establishes Justice, insures domestic Tranquility, provides for the common defence, promotes the general Welfare, and secures the Blessings of Liberty to ourselves and our Posterity.”
At the same time, we should prepare for continuing controversy. Social scientists often research topics about which others have strong opinions. This is particularly true of social scientists who study politics and policy (Lupia Reference Lupia2000)—as many people in these domains do not like to be told that data and evidence support other ways of viewing topics about which they are passionate. In the face of such opinions, social scientists can expect questions about whether science describes social phenomena better than media commentators, interest groups, and politicians. While societies often benefit from the exchange of views that come from these and other sources, societies also benefit by basing policy decisions on the best available logic and evidence. The ability to offer such logic and evidence in a credible, legitimate, and rigorous manner is what social science is built to do. Honest, empirically informed, and technically precise analyses of the past provide the strongest foundation for knowledge and can significantly clarify the future implications of current actions. For this reason, supporting social science is critical to any nation that seeks to achieve its greatest aspirations with ever-increasing effectiveness and efficiency.
ACKNOWLEDGMENTS
This essay is based on the 2013 Ithiel de Sola Pool Lecture, which was delivered at the APSA Annual Meeting (Chicago, Illinois. August 30, 2013). I thank Henry E. Brady and Samuel L. Popkin for graciously introducing me at the lecture. I thank John H. Aldrich, Cheryl Boudreau, John G. Bullock, James N. Druckman, Colin Elman, Adam Seth Levine, Spencer Piston, and Gisela Sin for insightful comments on a previous draft of this essay.
