Scrolling through the results of the Supreme Court Forecasting Project in July 2003, I regarded nearly every prediction as within the realm of plausibility until I reached the statistical model's forecast of the outcome in Lawrence v. Texas, the gay rights case. The model predicted that the Supreme Court would affirm by a 5–4 vote the judgment of the Texas Court of Appeals that the state's law criminalizing gay sex was constitutional. This was such a gaffe that it led me to question whether all the fancy modeling was worth the effort. (Two of the three experts correctly predicted a 6–3 reversal; the third made the same mistake as the model.)

I have since learned that the model's erroneous prediction was produced in part by a programming error in its internal computer code.¹

Once that was fixed—without change to the model's basic design—its prediction in Lawrence was a 5–4 reversal. This is more plausible than the model's original prediction, but the fact that such a “bug” infected the original prediction highlights one problem with relying on machines to forecast cases. Moreover, even after correcting the programming error, the model still missed Justice Kennedy's vote to reverse the lower court in Lawrence—a significant mistake given the fact that he wrote the majority opinion. And he had, after all, earlier indicated his empathy for the gay rights claim in his majority opinion in Romer v. Evans (1996), rejecting the discriminatory anti-gay regime imposed by a Colorado constitutional amendment.

Of the major cases of the 2002 term, Lawrence v. Texas was arguably the most predictable. If that seems a rash statement, let me provide a bit of context. The petition for certiorari was filed on July 16, 2002, by Paul M. Smith, a former Supreme Court law clerk, partner in the Washington, DC, office of Jenner and Block, and experienced Supreme Court advocate. It laid out the case for overturning not only the Texas Homosexual Conduct Law and the judgment supporting it, but also for repudiating the Supreme Court's 1986 precedent in Bowers v. Hardwick that had rejected a due-process–right-to-privacy attack on a criminal sodomy law in Georgia.

Texas waived its right to respond to the petition—a common government counsel strategy for dealing with frivolous cases, but a surprising move in light of this appeal's obvious substantiality. On September 9, the Supreme Court indicated its interest in the case by ordering the state to respond. The Texas brief in opposition was filed on October 21 and the case was immediately relisted for the justices' next available conference. It was then promptly granted on December 2.

What does this narrative demonstrate? Of the more than 2,000 certiorari petitions that passed through the Court during this period, this was one that caught the justices' attention early. The decision to grant was immediate, in contrast to those petitions that require multiple conferences before four votes coalesce for a grant of certiorari. More significantly, and less subject to placement on a grid, there would have been absolutely no reason for the court to take this case in the absence of a potential majority to reverse. Any justice content with the state of the law on gay rights would certainly not have voted to grant Lawrence. Those who thought either that the Texas law appeared problematic in its own right or that it might be a vehicle for reconsidering Bowers would not have risked a grant in the absence of a clear indication of majority support. In my article on the grant in Lawrence, I recalled a similar moment in the prior term when the Court placed itself in a position to reconsider its precedent permitting execution of the mentally retarded (Atkins v. Virginia [2002], overruling Penry v. Lynaugh [1989]). I observed that while Chief Justice Rehnquist and Justices Scalia and Thomas were likely to uphold the Texas law, the other justices had “shown themselves to be open to arguments on behalf of civil rights for gays” (December 3, 2002, A26).²

The only uncertainty about Lawrence, in fact, was whether the Court would strike down the Texas law on the basis of equal protection, which Justice O'Connor eventually chose in her separate concurrence, or whether it would issue a broad due-process ruling and overturn Bowers, as of course it did. Among close watchers of the Court, there was no doubt that the eventual bottom line would be to reverse. To the extent that a statistical model is unable to take into account the sort of contextual factors that created that nearly universal consensus, it will not be a satisfactory substitute for human judgment—even if it scores well on most of the cases, most of the time.

But instead of simply throwing stones, I will offer myself as another test case. The Supreme Court press corps often indulges in predictions. For years, there was a lively betting pool every year on when the term would end—an entertainment that was eventually abandoned due to the chief justice's unvarying success in having the term end on the last Thursday in June.³

Reporters also bet among themselves on the outcome of cases—friendly wagers (the standard stake being 38 cents, for reasons that are lost to history) usually made during the walk down the stairs from the courtroom following an oral argument session. There were few such bets during the past term because there was general agreement on the likely outcome of nearly all cases. The only formal bet I made, in fact, was that the government would prevail in the Copyright Term Extension Act case, Eldred v. Ashcroft, with no more than two dissenting votes. This proved to be a winner when only Justices Breyer and Stevens dissented from Justice Ginsburg's opinion for the Court.

But more than this intramural sport, what really counts are those predictions that a reporter is bold enough to share with readers. I frequently make predictions in my accounts of oral argument, part of my effort to raise the story above the level of the formulaic “The Supreme Court heard arguments today …” and to give the reader my best judgment on what is likely to happen next—the oral argument, after all, being only a snapshot of the decisional process.

Inspired by this project, I reviewed all my argument stories for the 2002 term and evaluated the quality of the predictions. First, two caveats. These are postargument predictions, while the forecasting project made its predictions before argument. That gives me a substantial advantage. Also, I do not cover the arguments in every case. I apply a rough triage to each term that tends to work out as follows: one-third of the cases I ignore altogether as too technical to be of general interest; for another third, I cover only the actual decision, for reasons not germane here; and for the remaining third, I cover both the argument and the decision. So there may be some built-in bias in my case selection, although, I would guess that the cases I choose to cover tend to be the most closely fought, and thus less predictable, cases of each term.

For the 2002 term, I covered 27 arguments. I included some form of prediction about the outcome for 17. (One of those was the Nike case, which ended in a dismissal; while I flagged the jurisdictional problems in my argument story, I suggested that Nike was likely to win on the merits. [April 24, 2003]) I avoided predictions for 10 other cases. Of the 16 predictions in cases that proceeded to decision, 12 (75 percent) proved correct; two were wrong; and two were “mixed.”

Of the 10 cases for which I avoided predictions,⁴

it appears to me in retrospect that on three occasions, I was either too diffident or insufficiently perceptive, and that I should have found a way to make predictions in Miller-El v. Cockrell (the Court voted 8–1 that the Fifth Circuit should have issued a certificate of appealability); Scheidler v. National Organization for Women, in which the Court ruled for petitioners by a vote of 8–1; and one of the Megan's Law cases, Connecticut Department of Public Safety v. Doe, in which the court's decision to uphold the state law against a due process challenge was unanimous.

All in all, not too humiliating a record. But because that is perhaps a judgment best left to others, I will put my cards on the table for those who want to compare the predictive abilities of one who has covered the court daily for many years with those of the statistical model and the ad hoc expert panels. I will describe my predictions chronologically through the term.

First, in Federal Communications Commission v. NextWave Communications, the lead paragraph of my story said the court “gave a skeptical hearing” to the government's defense of its confiscation of the bankrupt company's licenses and that “the only justice who appeared at all receptive to the government's was Stephen G. Breyer” (October 9, 2002, C1). Indeed, NextWave won the case 8–1, with Justice Breyer dissenting. (All three panels got the outcome wrong.)

In Eldred v. Ashcroft, the copyright extension case, I wrote that the lawyer challenging the extension “faced an uphill battle” and that “the justices appeared to agree that there should be a limit somewhere, but not that they should be the ones to impose it” (October 10, 2002, C1). The extension was upheld. The model got the outcome right; the two experts were wrong.

In United States v. Bean, I wrote that “most justices appeared to agree with the government's position” that federal judges lacked jurisdiction to reinstate felons' firearms privileges” (October 17, 2002, A25).⁵

As all predicted, the government won the case; both experts correctly forecast the unanimous vote.

It was clear from the back-to-back arguments in the California “three-strikes” cases that the state would prevail. In retrospect, I should have been more predictive. I limited myself to a somewhat oblique lead paragraph, saying that the crimes these hapless defendants had committed to earn their life sentences “were not particularly shocking” but that the justices did not appear to find the sentences shocking either (November 6, 2002, A20). The state won both cases, 5–4. The model failed this test, predicting a 6–3 win for the petitioner in Ewing v. California, while all three experts correctly forecast his loss.

My predictive language in Norfolk & Western Railway Co. v. Ayers was tepid: I said “the justices responded somewhat skeptically” to the two lawyers arguing for reversal (November 7, 2002, C18). And the court in fact affirmed in this liability case by a 5–4 vote. Both the model and the experts forecast reversal, as I would have when the case was granted; this case was won in the briefing and argument.

Washington Legal Foundation v. Legal Foundation of Washington (renamed Brown v. Legal Foundation of Washington by the time of decision), the lawyers' trust account case, is a prime example of a case where Justice O'Connor's vote would forseeably make the difference. I said, “The outcome may depend on Justice Sandra Day O'Connor, who sided with the majority four years ago in finding the interest to be the clients' property. Justice O'Connor seemed skeptical today about the further step of declaring that an unconstitutional taking had occurred” (December 10, 2002, A32). She did change sides, resulting in a 5–4 majority to sustain the program.

I count Virginia v. Black, the cross-burning case, as one of my “mixed” outcomes. I wrote that following an emotional outburst by Justice Clarence Thomas, “while the justices had earlier appeared somewhat doubtful of the Virginia statute's constitutionality, they now seemed quite convinced that they could uphold it as consistent with the First Amendment” (December 12, 2002, A1). In fact, while the Court did affirm the notion of criminalizing cross-burning, it struck down the Virginia statute as not meeting the appropriate evidentiary test for intentional intimidation.

I give myself a failing grade on the punitive damages case, State Farm v. Campbell. I saw no center of gravity in the argument, and wrote that on the question of punitive damages, “consensus on the court appeared as elusive as ever” (December 12, 2002, A36). The decision turned out to be a decisive (6–3) corporate victory, and all the forecasts did better than I did in predicting it.

Nevada Dept. of Human Resources v. Hibbs, the Family and Medical Leave Act case, was the second of my “mixed” results. The outcome here, rejecting the state's 11th Amendment immunity claim, was missed by all the forecasts and was perhaps the term's biggest surprise. I offered a hint that Justice O'Connor might be moving away from her states' rights position in the earlier 11th Amendment cases: “Only one member of the consistent 5-to-4 majority in those cases, Justice Sandra Day O'Connor, offered a hint that she saw this case as perhaps different in a significant way” (January 16, 2003, A25). And so she did—but so also did Chief Justice Rehnquist, to everyone's great surprise.

My biggest forecasting failure came in Ryan v. Telemarketing Associates, the Illinois telemarketing regulation case. I predicted that the state would lose, as did all three experts; it won unanimously, albeit with strictures placed on the state's regulatory zeal (March 4, 2003). The model correctly predicted the result.

It seemed apparent after the argument in Federal Election Commission v. Beaumont that the Court would reverse the Fourth Circuit and uphold the broad prohibition on corporate contributions to federal campaigns, as predicted in my story (March 26, 2003, A15).

There was even less doubt of the eventual outcome after the lopsided argument in Lawrence v. Texas than there had been when the court granted the case. The lead sentence of my argument story began: “A majority of the Supreme Court appeared ready today to overturn a Texas homosexual conduct law …” (March 27, 2003, A18).

The term's other landmark cases, the two challenges to affirmative action at the University of Michigan that were argued the following week, did not present quite so clear a target. Yet as the April 1 argument approached, it appeared to me, largely on the basis of the briefing, that the university was in a stronger position that many people had anticipated when the case was granted. I suggested as much on March 30. After the arguments, I wrote: “[I]t appeared to many in the packed courtroom that affirmative action would survive its most important test in 25 years….” (April 2, 2003, A1).

I made a strong prediction in the California Holocaust insurance case, American Insurance Assoc. v. Garamendi, that the court would find the state law preempted; the 5–4 margin was narrower than I expected (April 24).

Finally, the public housing trespass case, Virginia v. Hicks, was always in an odd posture—a facial First Amendment challenge brought by someone who evinced no desire to speak. The argument made it clear that the Court would avoid a broad ruling. Was there a First Amendment problem with the ordinance? “Perhaps, but not in this case” May 1, 2003, A28). The court unanimously rejected the facial challenge but kept the door open for an appropriate plaintiff.

As these reflections probably make clear, I have no particular desire to cede my own responsibilities to either a statistical model or to outside experts. But as I begin my 23rd term covering the Supreme Court for a general audience, I expect to continue learning from both.