Legal Context

The populations in which violence risk is assessed vary across many disparate legal contexts. In the mental health system,Reference Desmarais, Van Dorn and Johnson ² civil commitment on the ground of “dangerousness,” commitment as a sexually violent predator, and the tort liability of psychiatrists and psychologists for their patients’ violence often turn on issues of risk assessment. In the justice system, risk assessment is increasingly being used to inform decisions about sentencing and parole. Risk assessment for violent terrorism is also becoming increasingly common.

The law regulating the process of violence risk assessment has become much more developed in the U.S. in recent years. Some laws specify risk factors that may and may not be used to estimate risk (eg, race is Constitutionally proscribed as a risk factor, whereas gender and age are generally permitted).Reference Monahan ³ Other laws allude to specific likelihoods of violence necessary to trigger preventive actions. For example, a Virginia statute allows for the civil commitment of a person with mental illness if “there is a substantial likelihood that, as a result of mental illness, the person will, in the near future, cause serious physical harm to himself or others.” The material used to train professionals in the law elaborates on the meaning of substantial likelihood: a “‘one-in-four’ estimated risk of serious harm in the near future is sufficient, particularly when the harm being threatened is potentially fatal.… A ‘substantial risk’ is not meant to mean ‘more likely than not’ (51%),”Reference Cohen, Bonnie and Monahan ⁴ (p133).

Assessment Approaches

No distinction in the history of risk assessment has been more influential than Paul Meehl'sReference Meehl ⁵ cleaving the field into “clinical” and “actuarial” (or statistical) approaches. Subsequent research is fairly characterized in a comprehensive review by Ægisdóttir etalReference Ægisdóttir, White and Spengler ⁶ : “One area in which the statistical method is most clearly superior to the clinical approach is the prediction of violence.” In recent years, however, a plethora of instruments has been published that are not adequately characterized by a simple clinical–actuarial dichotomy. Rather, the risk assessment process now exists on a continuum of rule-based structure, with completely unstructured (“clinical”) assessment occupying one pole of the continuum, completely structured (“actuarial”) assessment occupying the other pole, and several forms of partially structured assessment lying between the 2.Reference Skeem and Monahan ⁷

The violence risk assessment process, in this regard, might usefully be seen as having the 4 components shown in Table 1: (1) identifying empirically valid (and legally acceptable) risk factors, (2) determining a method for measuring (“scoring”) these risk factors, (3) establishing a procedure for combining scores on the risk factors, and (4) producing an estimate of violence risk. It is possible to array 5 current approaches to violence risk assessment according to whether the approach structures (ie, specifies rules for generating) none, 1, 2, 3, or all 4 of these components of this process. Purely “clinical” risk assessment structures none of the 4 components. The clinician selects, measures, and combines risk factors, and produces an estimate of violence risk, as his or her clinical experience and judgment indicate.

Table 1 Four types of risk factors

Adapted from Kraemer etal.Reference Kraemer, Kazdin and Offord ¹

Performing a violence risk assessment by reference to a standard list of risk factors that have been found to be empirically valid (eg, age, past violence, substance abuse), such as the lists provided in psychiatric texts structures 1 component of the process. Such lists function as an aide memoir to identify which risk factors the clinician should attend to in conducting his or her assessment, but they do not further specify a method for measuring these risk factors. As TardiffReference Tardiff ⁸ has stated, “Some factors may be more important than others for the individual patient,” (p5).

The “structured professional judgment” approach exemplified by the HCR (Historical-Clinical-Risk Management)-20Reference Webster, Douglas, Eaves and Hart ⁹ structures 2 components of the violence risk assessment process: the identification and measurement of risk factors, which may be scored as 0 if absent, 1 if possibly present, or 2 if definitely present. A revised version of this instrument—HCR-20 (Version 3)—has recently been released.Reference Douglas, Hart, Webster and Belfrage ¹⁰ Structured professional judgment instruments do not go further and structure how the individual risk factors are to be combined in clinical practice.Reference Douglas, Hart, Groscup and Litwack ¹¹ Approaches to risk assessment that structure 3 components of the risk assessment process are illustrated by the Classification of Violence Risk (COVR).Reference Monahan, Steadman and Silver ¹² This instrument structures the identification, measurement, and combination of risk factors (via a classification tree design). But those who developed this instrument do not recommend that the final risk assessment reflect only the combined scores on the assessed risk factors. Given the possibility that rare factors influence the likelihood of violence in a particular case—and that, precisely because such factors rarely occur, they will never appear on an actuarial instrument—a professional review of the risk estimate is advised (while realizing that clinicians may overidentify “rare” factors). This professional review “would not revise or ‘adjust’ the actuarial score produced by the COVR, but would likely be of a more qualitative nature (eg, ‘higher than,’ or ‘lower than’ the COVR estimate).”Reference Monahan ¹³ A validation of the COVR in Sweden has recently been published.Reference Sturup, Kristiansson and Monahan ¹⁴

The best known forensic instrument that structures all 4 of the components of the violence risk assessment process—ie, that is completely actuarial—is the Violence Risk Appraisal Guide (VRAG). This instrument not only structures the identification, measurement, and combination of risk factors, but it also specifies that once an individual's violence risk has been actuarially characterized, the risk assessment process is complete. As Quinsey etal have stated, “What we are advising is not the addition of actuarial methods to existing practice, but rather the replacement of existing practice with actuarial methods,”Reference Quinsey, Harris, Rice and Cormier ¹⁵ (p197). A revision of the VRAG has recently been published.Reference Rice, Harris and Lang ¹⁶

Does One Approach Predict Better than Another?

Of these 5 approaches, the unstructured (“clinical”) one rests on the least empirical support. In one major study of this approach, Lidz etalReference Lidz, Mulvey and Gardner ¹⁷ concluded that

We know of no research that systematically compares the predictive utility of strategies that structure none, 1, 2, 3, or all 4 components of the process. Relevant data are available, however, on approaches that structure 2 or more components. Recent debates about whether it is more appropriate to structure clinical judgment (eg, HCR-20) or replace it altogether (eg, VRAG) have prompted a number of horse races that compare the predictive efficiency of one risk assessment instrument against another.

Taken together, these studies provide little evidence that one validated instrument predicts violence significantly better than another. In a recent meta-analysis of 28 studies that controlled well for methodological variation across studies, Yang etalReference Yang, Wong and Coid ¹⁸ found that the predictive efficiencies of 9 risk assessment instruments (including the HCR-20 and VRAG) were essentially “interchangeable,” with point estimates of accuracy falling within a narrow band (ie, AUC = .65 to .71). This meta-analysis yields much the same message as meta-analyses of alternative psychotherapy techniques. As pronounced by the Dodo bird at the end of the race in Lewis Carroll's Alice in Wonderland, “Everyone has won, so all shall have prizes,” (Carroll, L. Alice's Adventures in Wonderland. London: MacMillan; 1866. p34).

Why might different instruments perform equally well in predicting violence? One persuasive explanation is that they tap “common factors” or shared dimensions of risk, despite their varied items and formats. In an innovative demonstration, Kroner etalReference Kroner, Mills and Morgan ¹⁹ printed the items of several leading instruments on strips of paper, placed the strips in a coffee can, shook the can, and then randomly selected items to create 4 new tools. The authors found that the “coffee can instruments” predicted violent and nonviolent offenses as well as the original instruments. Factor analyses suggested that the instruments tap 4 overlapping dimensions: criminal history, an irresponsible lifestyle (eg, poor engagement in school/work), psychopathy and criminal attitudes (eg, entitlement), and substance abuse–related problems. Despite surface variation, instruments may generally tap “a longstanding pattern of dysfunctional and aggressive interpersonal interactions and antisocial and unstable lifestyle that are common to many perpetrators of violence,”Reference Kroner, Mills and Morgan ¹⁸ (p759).

The strongest risk factors for violence seem to be shared not only among risk assessment instruments, but also across key groups. In particular, an increasing body of research suggests that only a small proportion of violence committed by people with mental illness—perhaps as little as 10%—is directly caused by symptoms.Reference Skeem, Manchak and Peterson ²⁰ Most people with mental illness share leading risk factors for violence with their relatively healthy counterparts.

Are Empirically Based Instruments Useful for Individuals?

One issue that has generated much recent controversyReference Hart, Michie and Cooke ²¹ is the argument that the margins of error surrounding individual risk assessments of violence are so wide as to make such predictions “virtually meaningless.” As stated by Cooke and Michie, “On the basis of empirical findings, statistical theory, and logic, it is clear that predictions of future offending cannot be achieved, with any degree of confidence, in the individual case,”Reference Cooke and Michie ²² (p259).

This position has been vigorously contested, both with respect to the overall argument and application of statistics to make it. For example, Hanson and HowardReference Hanson and Howard ²³ demonstrate that the wide margin of error for individual risk assessments is a function of having only 2 possible outcomes (violent or not violent), and therefore conveys nothing about the predictive utility of a risk assessment tool, which must be judged by other criteria. Because all violence risk assessment approaches, not just actuarial approaches, yield some estimate of the likelihood that a dichotomous outcome will occur, none are immune from Hart etal'sReference Hart, Michie and Cooke ²¹ argument (as they recognize). Indeed, their argument “if true, … would be a serious challenge to the applicability of any empirically based risk procedure to any individual for anything,”Reference Hanson and Howard ²³ (p277).

Our view is that group data theoretically can be, and in many areas empirically are, highly informative when making decisions about individual cases.Reference Scurich, Monahan and John ²⁴ Consider 2 examples from other forms of risk assessment. In the insurance industry, “Until an individual insured is treated as a member of a group, it is impossible to know his expected loss, because for practical purposes that concept is a statistical one based on group probabilities. Without relying on such probabilities, it would be impossible to set a price for insurance coverage at all,”Reference Abraham ²⁵ (p79). In weather forecasting, “Extensive statistical data are available on the average probability of the events [meteorologists] are estimating” and therefore when meteorologists “predict a 70% chance of rain, there is measurable precipitation just about 70% of the time,” ²⁶ (p46).

Along these lines, Faigman etalReference Faigman, Monahan and Slobogin ²⁷ have recently analyzed how the law deals with what they refer to as group-to-individual (G2i) inference in scientific expert testimony, including scientific expert testimony on violence risk. They conclude:

In the view of Faigman etal, evidence-based scientific expert testimony on an individual's violence risk when offered by qualified psychiatrists or clinical psychologists should have no trouble being admitted in court.

Should Risk Assessment and Reduction Be Separated?

In the U.S., correctional agencies that manage a staggering number of youth and adults are increasingly endorsing structured risk assessment approaches and treatment programs that reduce reoffending by targeting risk factors such as anger, poor self-control, and antisocial attitudes. In this context, companies have begun to market complex (and poorly validated) assessment systems that explicitly include treatment-relevant variables in their risk estimates and ostensibly serve the risk reduction enterprise better than simple actuarial tools.

This has sparked debate about whether the pursuit of risk assessment and risk reduction should be separated or integrated. BairdReference Baird ²⁸ favors separation, arguing that the addition of treatment-relevant variables to otherwise parsimonious risk equations that emphasize past (mis) behavior will dilute their predictive utility. Andrews,Reference Andrews ²⁹ on the other hand, argues that some treatment-relevant variables are risk factors and should be integrated in risk estimates. His view is that efficient prediction can be achieved by statistically selecting and combining a few highly predictive risk factors, but tools that sample risk domains more broadly and include treatment-relevant risk factors can be equally predictive.

Monahan and SkeemReference Monahan and Skeem ³⁰ observed that this debate has been exacerbated by confusion about what a treatment-relevant risk factor is, exactly. They differentiated among 4 different types of risk factors for violent re-offending or recidivism (see Table 2):

Table 2 Increasingly structured approaches to violence risk assessment

Note: HCR-20 = Historical Clinical Risk-20; COVR = Classification of Violence Risk; VRAG = Violence Risk Appraisal Guide. Adapted from Skeem and Monahan.Reference Skeem and Monahan ⁷

All 4 types of risk factors are relevant to risk assessment, but only causal risk factors are relevant to risk reduction. Put simply, treatment-relevant risk factors are causal risk factors. Unless a variable risk factor has been shown to be causal, there is little reason to assume that reducing the risk factor will reduce violence. This fact is rarely recognized in current discourse. Few risk factors for violence have been shown to be causal, but variable risk factors have been shown to predict proximate violence and are the best point of reference the field presently has to offer for risk reduction.

Choosing Among Valid Instruments

Given a pool of instruments that are well-validated for the groups to which an individual belongs, our view is that the choice among them should be driven by the ultimate purpose of the evaluation. If the ultimate purpose is to characterize an individual's likelihood of future violence relative to other people, then choose the most efficient instrument available. This is appropriate for a single event decision in which there is no real opportunity to modify the risk estimate based on future behavior.Reference Heilbrun ³¹ If the ultimate purpose is to manage or reduce an individual's risk, then value may be added by choosing an instrument that includes treatment-relevant risk factors.Reference Howard and Dixon ³² (Although an integrated instrument would be most parsimonious, we can easily envision a 2-stage process in which a risk-assessment step was followed by an independent risk-management step.) This choice is appropriate for ongoing decisions where the risk estimate can be modified to reflect ebbs and flows in an individual's risk over time. Beyond focusing risk reduction efforts, these instruments could provide incentive for changing behavior. (A parole board cannot advise an inmate to “undo” his past commission of an assault, but can advise him to develop employment skills.)

This view comes with a 3 important caveats. First, techniques that include treatment-relevant risk factors will add no value to simpler approaches unless the risk assessment is followed by a period of control over the individual, during which those factors are translated into an individual supervision and treatment plan (rather than simply filed away), and systematically targeted with appropriate services (rather than ignored in resource allocation). Risk reduction cannot be achieved through risk assessment alone, regardless of the approach applied. Second, treatment-relevant variables can and do appear in statistically derived risk assessment instrumentsReference Monahan ¹³ ; an instrument's degree of structure cannot be equated with its relevance to risk reduction. Third, even well-validated instruments offer little direct validity data for the treatment-relevant variables they include. It is not enough to demonstrate that a variable is a risk factor for violence; here, it must further be shown that the variable reduces violence risk when successfully changed by treatment (ie, is a causal risk factor).Reference Kraemer, Kazdin and Offord ¹ This is a crucial issue to address in future research, if tools continue to be sold on the promise of informing risk reduction.Reference Kroner and Yessine ³³

Future Directions

The violence risk assessment field may be reaching a point of diminishing returns in instrument development. It has been long been argued that there may be a “sound barrier” to predictive validity in this area, such that the correlation between risk estimates and criterion measures will rarely exceed .40.Reference Menzies, Webster and Sepejak ³⁴ In this regard, Paul AppelbaumReference Appelbaum ³⁵ has stated that “predictive assessments are the most challenging evaluations performed by mental health professionals,” (846). He speculates on the reasons underlying this great challenge:

While the “sound barrier” for predictive accuracy in the case of violence may prove to be somewhat higher than it is now,Reference Singh and Petrila ³⁶ there is no question that “the contingencies of life” will place an upper limit on what can be achieved in many risk assessment contexts. The most promising candidates for incremental advances in violence risk assessment may include violent victimization,Reference Sadeh, Binder and McNiel ³⁷ implicit measures,Reference Knock, Park and Finn ³⁸ patient self-perceptions,Reference Skeem, Manchak, Lidz and Mulvey ³⁹ and the incorporation of risk factors from the neurosciences.Reference Aharonia, Vincent and Harenskia ⁴⁰ If we are approaching a “sound barrier” in the risk assessment domain, there clearly are miles to go before we can rest on the risk reduction front. We hope that in the future, psychology and psychiatry shift more of their empirical attention from predicting violence to understanding its causes and preventing its (re)occurrence.

Disclosures

John Monahan has nothing to disclose. Jennifer Skeem received research support from the National Institute of Health.