1 Introduction
Although there are no reliable data on how many U.S. police departments have proactive patrol units, the strategy has been widespread in various forms since the 1990s. Proactive policing, broadly defined, is any policing strategy based on the notion that “by proactively regulating minor offenses, the police can reduce both serious crime and fear of crime among the community by sending a message that crime will not be tolerated” (Nix & Rojek, Reference Nix and Rojek2014). Proactive patrol units engage in policing activities aligned with this idea. These tactics can include, but are not limited to, the aggressive use of traffic tickets to search vehicles for contraband and firearms, frequent use of in-person stops to search persons for contraband and firearms, and generally increasing staff levels in high-crime areas in order to “increase the number of citations and/or arrests to deter more serious crime” (Nix & Rojek, Reference Nix and Rojek2014).
A strategy of proactive policing, like nearly all policy interventions, comes with both benefits and costs, both direct and indirect. The most direct benefits of proactive policing are that it allows the police to arrest individuals who pose a danger to the community, as well as to confiscate illegal drugs and firearms. Making these arrests and confiscating contraband directly benefit community safety. The benefits of proactive policing would be all the greater if it also deterred more serious crimes, reduced fear of crime among community members, or led to greater public or private investment in particular neighborhoods.
With respect to costs, one direct cost of proactive policing is that it requires more police officers performing different tasks, as compared with reactive policing, and thus imposes costs on police departments. Because proactive policing demands significant investment of officer time, implementation will often require either shifting of existing officers into proactive policing (and thus away from other priorities), or, alternatively, maintenance of current staffing levels for other priorities while adding proactive policing patrols (thus imposing labor costs on departments). In addition, departments will bear the costs of training officers for engagement in proactive policing and ensuring compliance with a department’s internal rules and applicable legal rules governing policing tactics. Proactive policing will also impose direct costs on individuals who are stopped by policing as part of the proactive policing but would not have been stopped otherwise. Proactive policing can impose several types of costs on individuals who are stopped: costs in time, money, and psychological harm. Indirect costs of proactive policing include the costs borne by the public, such as erosion of police–community relations and increased arrests and incarcerations of community members.
The costs of proactive policing will not fall on all individuals or communities equally. Instead, these costs will fall predominately on individuals and communities who tend to have more frequent involuntary encounters with police. Studies of past proactive policing efforts provide strong evidence that the costs of proactive policing policies fall disproportionately on some demographic groups. A seminal study of New York City’s “stop-and-frisk” policy, for example, found that “persons of African and Hispanic descent were stopped more frequently than Whites, even after controlling for precinct variability and race-specific estimates of crime participation” (Gelman, Fagan & Kiss, Reference Gelman, Fagan and Kiss2007, p. 1). Proactive policing – at least as it was practiced in New York City – also results in stops of substantially more men than women and stops of substantially more young people than middle-aged and elderly people (New York Civil Liberties Union, 2017). For nearly any proactive policing policy that a department considers adopting, the policy’s costs will fall predominately on those who are young, male, or non-White (or those who fall into several of those demographic groups). In this issue, Cohen (Reference Cohen2017) sets out the impacts that racially targeted police encounters have on the individuals targeted, on other members of the targeted individuals’ demographic group, and on society as a whole.Footnote 1
The full effects of introducing proactive policing, including these benefits and costs, cannot be estimated with certainty without a fully randomized control trial (RCT), in which some neighborhoods are randomly assigned to proactive patrols and other similar neighborhoods are randomly assigned to standard patrols. While RCTs are the gold standard in social science research (Jones & Podolsky, Reference Jones and Podolsky2015), they can be extremely challenging to conduct in the context of police reform. The key research design challenge of RCTs evaluating policing policy is that they require the police to treat similarly situated high-crime neighborhoods differently for no other reason than those neighborhoods’ randomization into either the treatment or control group. While typically possible in theory, such a randomization can create serious challenges in practice. Communities may object to a policy that allocates benefits and burdens (or perceived benefits and burdens) at random (Boruch, Reference Boruch2015); policymakers may be hesitant to implement policies through a randomization protocol; and, relatedly, policymakers confident in the likely effects of a possible reform and feeling pressure to take action may not wish to take the more incremental steps necessary for a valid RCT. For these reasons, an RCT will often not be possible in practice. Researchers may thus wish to evaluate proactive enforcement strategies in the absence of an RCT.
This article seeks to inform researchers of police practices, both within police departments and in academia, of the most accurate ways to fully evaluate the benefits and costs of various police reforms in the absence of an RCT, without falling victim to the many pitfalls of simple regression-based or before-and-after studies. Because police departments may be hesitant to treat similarly situated high-crime neighborhoods differently, studies that use existing data will often be researchers’ best options for evaluation of police policy reforms, including various forms of proactive policing.
There are two basic frameworks for approaching this evaluation challenge: a quasi-experimental approach and a nonexperimental retrospective approach. In this article, I discuss each approach in turn, highlighting advantages and disadvantages, along with what each would entail in terms of police cooperation and data collection. I also discuss threats to validity for each approach and how those threats could be addressed.
2 Quasi-experimental approaches
The key virtue of an RCT is that, on average, there are no differences between treatment and control units on observed or unobserved background covariates (Imbens & Rubin, Reference Imbens and Rubin2015, p. 40). While this covariate balance is impossible without active randomization, a quasi-experiment can emulate this setup in at least two ways for evaluation of proactive policing.
First, some neighborhoods could be assigned to proactive enforcement while others are assigned to standard patrol during the course of a staggered rollout of proactive enforcement across a city. Such a strategy may not be feasible for a municipality that already has a municipality-wide practice of proactive policing, but it could be effective in several other cases. Some examples include a municipality considering a move to proactive policing in the first instance, a municipality considering expansion of small-scale proactive policing to a larger geographic area, or a municipality considering expansion of proactive policing through the introduction of particular new tactics. Staggered rollout of proactive enforcement would have the most important advantage of an RCT (simultaneous comparison between treated and nontreated neighborhoods) without the most daunting disadvantage (the forced assignment, by researchers, of some neighborhoods to treatment and others simultaneously to control). Moreover, given that new policies are often rolled out on a staggered basis for reasons having nothing to do with research design, policymakers are more likely to be willing to cooperate with researchers on a staggered rollout than to allow for a conventional RCT.
Such a research design, if implemented, lends itself most naturally to a difference-in-differences (DID) analysis. A DID analysis is appropriate
“in problems with multiple subpopulations – some subject to a policy intervention or treatment and others not – and outcomes that are measured in each group before and after the policy intervention. To account for changes over time unrelated to the intervention, the change experienced by the group subject to the intervention (referred to as the treatment group) is adjusted by the change experienced by the group not subject to treatment (the control group). The underlying assumption is that the time trend in the control group is an adequate proxy for the time trend that would have occurred in the treatment group in the absence of the policy intervention” (Athey & Imbens, Reference Athey and Imbens2006).
Probably the best-known example of a DID analysis is an evaluation of the effect of a minimum wage increase on employment levels when the minimum wage increased in New Jersey but not in neighboring Pennsylvania (Card & Krueger, Reference Card and Krueger1994). That study leveraged an increase in the minimum wage in New Jersey in 1992, and compared fast-food restaurant employment levels before and after the minimum wage increase in New Jersey and in Pennsylvania. In the crime context, the best-known DID study is that of Di Tella and Schargrodsky (Reference Di Tella and Schargrodsky2004), where the authors were able to compare an increase in police patrols in specific areas of Buenos Aires with other areas that had simultaneously not received the same increase, finding that certain types of crime decreased in more heavily policed neighborhoods. In the proactive policing context, DID analysis can allow scholars to compare crime rates – and perhaps also other variables of interest, such as public attitudes toward the police – in neighborhoods that were subject to proactive policing with otherwise similarly situated neighborhoods that were not.
A quasi-experimental design could also allow researchers to discern a policy’s effects by using slightly different proactive enforcement approaches for different neighborhoods in the city. For example, in two similar neighborhoods in different parts of the city, the police could assign one to a “high-staff” proactive patrol and one to a “low-staff” proactive patrol, with a greater density of proactive enforcement officers assigned to the former neighborhood compared with the latter. Alternatively, police could focus on one type of proactive enforcement in one neighborhood and another type in a different neighborhood. Doing so could enable one to discern the differential effects of, for example, increased foot patrols compared with increased traffic enforcement. Even though assignment would not necessarily be random, this would allow for a dose–response comparison between multiple neighborhoods (Farrelly, Davis, Haviland, Messeri, & Healton, Reference Farrelly, Davis, Haviland, Messeri and Healton2005); that is, if the average effect of proactive patrol is indeed crime reduction, it should be that the higher-staff patrol has a greater crime reduction effect than a lower-staff patrol. Moreover, if particular strategies are more effective at crime reduction than others, a comparison between them would similarly allow researchers to observe those differences.
The most important threat to the validity of these quasi-experimental designs for evaluation of the effects of proactive policing is the risk of spillover effects. Spillover effects occur when the experimental or quasi-experimental treatment affects the outcomes of control units as well as treatment units. In this case, if proactive enforcement is by design confined to certain areas or neighborhoods of the city, but it affects crime outcomes in other areas or neighborhoods of the city as well, the control unit outcomes are being affected by the treatment, and a spillover is taking place. In colloquial terms, spillovers are a problem because they muddy the difference between treatment and control units so that a comparison between them is no longer a clean one. In technical terms, they are a problem for causal inference because they represent violations of the stable unit treatment value assumption (SUTVA), which requires that, for causal statements to be made, “the potential outcomes for any unit do not vary with the treatments assigned to other units” (Imbens & Rubin, Reference Imbens and Rubin2015, p. 10).Footnote 2
For proactive policing, the risk of the police’s efforts spilling over into neighborhoods adjacent to those assigned to the new patrol can be substantial. Most obviously, patrol officers are usually expected to respond to any calls that take place nearby, so an increased police presence in one neighborhood may affect the level of policing in nearby neighborhoods. In addition, at the level of police resource allocation, allocation of more or better officers to some neighborhoods may have spillover effects on other neighborhoods, depending on how the police department goes about staffing. Moreover, at the level of criminal activity, there are perhaps some criminals who would, upon witnessing increasing police presence in some neighborhoods, shift their criminal activities to neighborhoods with a lower police presence. There are thus several respects in which a quasi-experimental design poses risks of spillover effects.
There are at least two ways to minimize spillover effects, although it is impossible to eliminate them entirely. First, if researchers are involved in the initial rollout of proactive enforcement, they can (1) be clear that officers should remain in their assigned patrol areas and (2) choose, if faced with choosing between beginning proactive enforcement in otherwise similar neighborhoods, to implement proactive enforcement first in high-crime neighborhoods that border low-crime neighborhoods (the logic being that police are not likely to be looking for illegal guns or drugs in a low-crime neighborhood, so proactive enforcement in a nearby high-crime neighborhood is unlikely to affect police activity there). If researchers are not involved in the initial design, they can use arrest and confiscation data to try to estimate the extent to which proactive enforcement has spread beyond the assigned neighborhoods, and accordingly deflate their estimated treatment effect (Chen, Mu, & Ravallion, Reference Chen, Mu and Ravallion2009), and they can observe the extent to which police were likely to have been using proactive enforcement strategies in nearby neighborhoods based on historical crime rates in those neighborhoods.
Ultimately, a quasi-experimental approach is very similar to an RCT. Unless researchers are fortunate enough to discover that a staggered rollout took place before they became involved in an evaluation, or that proactive enforcement was taking place in some high-crime areas but not in others, a quasi-experiment requires prospective planning around where and how to assign officers to proactive enforcement, very similar to what would take place in the RCT context. Police department requests for or researcher initiatives to conduct fully nonexperimental approaches are likely to be more common, and so I discuss evaluations of those sorts in the next section.
3 Nonexperimental retrospective approaches
In a nonexperimental retrospective research design, evaluators are faced with simply assessing the change in crime rates before and after the introduction of a proactive enforcement policy, and trying as best as possible to take account of all of the factors that could bias the results in any direction. In practice, this means using regression of the outcome variable or variables of interest (such as measures of the crime rate) on the treatment indicator (treated or untreated by proactive enforcement) and all covariates of interest that could potentially confound the analysis. This sort of analysis is plagued by a number of threats to validity. Most generally, the researcher must be confident that all possible confounders have been taken into account and ignorability thus achieved, even though “[i]n general, one can never prove that the treatment assignment process in an observational study is ignorable” (Gelman & Hill, Reference Gelman and Hill2006, p. 184). Beyond this general threat to validity in all observational regression analysis, there are four additional threats to validity for the specific case of assessing crime rates before and after the introduction of proactive enforcement. I discuss each in turn below.
3.1 Simultaneous policy changes
If increases in proactive enforcement occurred simultaneously with other policing policy changes, it would be extremely difficult to know whether observed changes in the crime rate were due to proactive enforcement or to these other changes. Many cities considering proactive enforcement might also, for the same reasons that they are considering the introduction of proactive enforcement, be considering the introduction of other anti-crime measures, such as the introduction of new technologies or the hiring of additional staff. To obviate the threat that simultaneous policy changes would pose to an evaluation of the effectiveness of proactive enforcement, researchers can, first and foremost, explain to police departments that the introduction of a raft of reforms all at once will make it nearly impossible to evaluate independent effects of the different reforms. At the very least, departments must be advised to keep close track of the new reforms – specifically, where they have been implemented and how aggressively they have been implemented – so as to at least be able to estimate different effects of specific reforms. Alternatively, researchers could alter the question of interest to reflect the joint effect of all reforms, thereby defining away this particular problem.
3.2 Overall temporal and seasonal trends in the city or regional crime rate
Researchers need to have accurate estimates of ex ante crime rate trends in the city and region so as to know how much of an observed ex post change in crime rates might be due to these broader trends. There are well-documented seasonal fluctuations in the crime rate (Andresen & Malleson, Reference Andresen and Malleson2013, p. 43), for example, and there are also significant long-term crime trends which transcend geography and are not well understood (Blumstein & Wallman, Reference Blumstein and Wallman2006). Without thorough knowledge of these trends, it would be impossible to have confidence in any estimate of the effect of a policy change.
3.3 Hawthorne effects
Hawthorne effects occur when a population knows that it is the subject of a research study, and thus alters its behavior – in particular, alters its behavior in a way that is prosocial vis-à-vis its perception of the goals of the study (McCarney et al., Reference McCarney, Warner, Iliffe, van Haselen, Griffin and Fisher2007, p. 1). In this case, police officers who know that the proactive enforcement policy is going to be the subject of researchers’ scrutiny could take a number of steps to alter measured outcomes in ways that they might not have had they not known about the research, including downgrading crimes from felonies to misdemeanors, electing not to make arrests when not absolutely necessary, making more arrests than may be strictly necessary, or being especially vigilant about the possibility of citizen complaints (Greenberg, Reference Greenberg2014). A fully retrospective observational study might avoid this problem if police did not know that a research evaluation would take place at the time reforms were introduced.
3.4 Consequential under- or over-reporting of (some types of) crime
In some cases, aggressive or violent police actions have been shown to decrease the rate of citizen reporting for some types of crimes. The only case in which this has been definitively documented, however, was an extreme case of police violence (Desmond, Papachristos & Kirk, Reference Desmond, Papachristos and Kirk2016). A corollary issue is that it is not uncommon for individuals who are chronically in contact with law enforcement to use law enforcement as an instrument to act out their personal agendas, such as by calling the police on friends or family members with whom they have disputes (Venkatesh, Reference Venkatesh2009). In sum, proactive enforcement might lead to under-reporting of crimes when individuals are concerned that the police might take aggressive action against friends, neighbors, or relatives. Conversely, proactive enforcement might lead to over-reporting of crimes where individuals see an increased police presence in their neighborhoods. If individuals perceive that officers are eager to make arrests, individuals might be more likely to use the police as instruments in interpersonal disputes. These possible effects on citizen behavior would complicate efforts to measure the effects of proactive enforcement alone, rather than the effects of proactive enforcement coupled with corresponding changes in citizen behavior.
There are strategies, however, for researchers to account for possible changes in citizen reporting behavior brought about by the implementation of proactive policing practices. To measure possible under-reporting, researchers can take advantage of 911/call-for-service data for at least one year before the policy change in order to estimate any changes in reporting that may have resulted from the introduction of proactive enforcement. Qualitative research can also be used to interview residents of high-crime neighborhoods and ask whether they have or have seen others change what they might report to the police due to the new policy (Brunson, Reference Brunson2007). To measure possible over-reporting, researchers could take advantage of a 911 dispatcher’s coding of various incidents. Dispatchers code incidents by assigning a category based on the type of incident, in light of what is told to the dispatcher on the 911 call. This coding could allow researchers to keep track of incidents that are first coded as family fights or domestic disputes, and which are resolved with administrative codes for preserving the peace, civil matter, or arrests for previously outstanding warrants. These types of commencement/resolution combinations should not increase as a result of proactive enforcement, so any increases that do take place are evidence that citizen behavior has changed in light of proactive enforcement.
4 Relevant nonexperimental data
Fortunately, neither the quasi-experimental nor the nonexperimental retrospective research designs research requires data that are not regularly collected by police agencies, although some additional data would be useful. I have mentioned various data sources in the above discussion, but I list them in more detail below. With one exception (noted below), these data require a strong partnership with a police department that is willing to share detailed current and historical crime data.
-
∙ Crime data. Data on stops, arrests, confiscations, and fines, including detailed location information, are absolutely crucial to either the quasi-experimental or the nonexperimental retrospective research designs. Additionally, crime report data that reflect what types of incidents officers responded to and how those incidents were resolved reflect the “dosage” of proactive enforcement (how much proactive enforcement behavior is in fact taking place) and whether proactive enforcement has shifted the types of calls or incidents to which the police respond.
-
∙ Officer assignment data. Data on where officers were on patrol and when, including which incidents they responded to, are important for assessing whether any complaints about proactive enforcement are coming from the conduct of a small number of officers who could potentially be reassigned (Terrill & Ingram, Reference Terrill and Ingram2016).
-
∙ Historical and regional crime data. These data are necessary for the nonexperimental retrospective evaluation, because researchers need to be able to assess whether observed changes are due to historical or regional trends rather than policy changes. The most appropriate data source for this purpose would be the FBI’s Uniform Crime Reporting program.
-
∙ 911/call-for-service data. These data are needed to estimate whether over- or under-reporting is taking place relative to before the introduction of the proactive enforcement policy.
-
∙ Interview data. These data are not strictly necessary, but interviews with the residents of affected neighborhoods would be useful for an understanding of how the proactive enforcement policy is being perceived, and whether it might be leading to changes in crime or reporting behavior.
5 Conclusion
Researchers seeking to evaluate a proactive enforcement policy without an RCT have two options: a quasi-experimental evaluation and a nonexperimental retrospective evaluation. Ultimately, a quasi-experimental evaluation may be nearly as difficult and costly as an RCT, unless researchers find that a natural experiment took place before they arrived. A nonexperimental retrospective evaluation will almost always be easier, but will require the use of a wider range of data to eliminate the many threats to validity faced by such an analysis. The key threats to validity for an observational study of proactive policing are simultaneous policy change, regional and historical crime trends that may wash out policy effects in initial before-and-after summary statistics, Hawthorne effects, and under- or over-reporting of certain types of crimes as a result of the new policy. Researchers may estimate the size of the cross-cutting effects generated by these outside factors by using crime data, police patrol assignment data, historical and regional crime data, 911/call-for-service data, and interview data. Moreover, even a study design that overcomes these threats will need to account for the distributional effects of proactive policing, given that its costs and benefits fall on different individuals.
Proactive enforcement, like all police practices, should be subject to rigorous analysis to determine its benefits and costs. While the full range of benefits and costs may be difficult to measure, the effect of proactive enforcement on crime rates is, happily, more readily measured than other questions of interest in police policy. The ability to clearly identify both the intervention (proactive enforcement in particular areas) and one major effect (crime rates in those areas) makes benefit-cost analysis a potentially viable possibility. Through either a quasi-experimental or a purely randomized design, scholars can estimate proactive enforcement’s effects on crime rates, and incorporate that estimate into a broader analysis of whether the benefits of proactive enforcement outweigh the costs, given the nature of those benefits and costs, and the populations on whom they fall.
Acknowledgments
I am grateful to the editors and referees of the Journal of Benefit-Cost Analysis for helpful comments on this paper, and to Barry Friedman and the Policing Project at NYU Law School for hosting the conference for which this paper was initially written.
