I

In a now famous visit to the London School of Economics in November 2008, Queen Elizabeth II asked why nobody had noticed the turmoil on the international markets before the crisis occurred (Pierce Reference Pierce2008). According to the Huffington Post, following this question a group of eminent economists apologized to the Queen and blamed ‘a failure of the collective imagination of many bright people … to understand the risks to the system as a whole’. Nevertheless, the economists' failure to forecast and address the recent crisis is more than an isolated event. Similar drawbacks have been highlighted in the past as well. Davies and McGoey (Reference Davies and Mcgoey2012) argue that ignorance is sometimes a convenient excuse for the absence of reaction to potential crises, and ultimately for the denial of responsibility. Undoubtedly, lack of forward-looking imagination partly explains why economists systematically miss the outbreak of crises. This article, however, argues that lack of imagination is but one manifestation of a more general problem ingrained in the way economists identify crises.

The standard approach in economics uses statistical techniques to understand the world.Footnote ² In particular, economists concerned with crises typically rely on statistics to infer crisis probabilities and draft policy responses. The legitimacy of such a frequentist approach, however, depends on specific assumptions. First, the object under study – here, the crisis – is well defined. Second, any observed crisis is attributed to the appropriate reference class (as crises of different natures exist). Third, the sample used to infer probabilities is not biased. This article argues that none of these assumptions is actually met in practice. Let us develop our point with a metaphor.

Consider two friends, Jack and Jill (J&J), who frequently play games. They have played chess ten times and from the evidence we have been able to gather, we estimate that Jill won eight times and Jack twice. And they have played tennis ten times, with estimates giving Jill four wins and Jack six. Today J&J are playing a new board game whose rules have never been tested before. What is the probability of Jack winning? This seemingly simple question raises issues regarding definition, reference class and sample bias.

First, the issues pertaining to definition are the following. What counts as a win or as a success? Perhaps we have misclassified the past events, because we only see the physical movements of J&J and do not know the winning conditions. Is the new game unique and are all past elements therefore irrelevant? What do we know about the next game J&J will play?

Second, we have the reference-class problem. The computed probabilities will depend on the chosen reference class. One could argue that the probability of Jack winning the new game is 0.4, since Jack wins 40 per cent of games. Alternatively, one could say that the probability is 0.2, because Jack wins 20 per cent of board games. Eventually, one could also consider the probability as undefined since J&J have never played the new board game.

Last, there is an issue associated with sample biases. Do we really know that there have been ten chess games and ten tennis games? Perhaps we really know that Jack has won two chess games and six tennis games, but do not have a great deal of evidence about how many games J&J have played. Did J&J report the results correctly? Perhaps they forgot to report unfinished games or games that ended in a tie. Should we just use the frequencies so far of Jack winning, or the frequencies of Jack winning in the past and in the future? Should we exclude the chess game for which Jack reported having a headache? How do we imagine future games? It might be the case that the new game is so different from the previous ones that past games do not inform us on winning likelihood. In this case, alternative forms of reasoning, imagination in a sense, may prove more fruitful than a frequentist approach.

These three types of problem are mostly ignored by economists. They are in fact inextricably intertwined. Typically, existing studies fail to define crises unambiguously. The classifications found in the literature change through time and are inconsistent with each other. As a result statistical analyses are likely to suffer from a reference-class problem. Ambiguous definitions and badly defined reference classes increase the likelihood that studies will rely on biased samples. Eventually, using past-frequentist approaches is more likely to lead to a failure to imagine new types of crises. We contend that only a more conceptual approach to crises can overcome these limitations. To clarify its arguments, this article builds heavily on historical examples.

The rest of the article is organized as follows. The next section discusses how economists identify and classify crises. Section iii shows how the reference-class problem may lead to biased conclusions. Section iv presents the biases stemming from the ambiguity regarding crisis definitions and the reference-class problem. Section v concludes.

II

As in J&J's case, the ability to define correctly what represents a success or a failure is crucial for determining probabilities of occurrence of a given event. Regarding economic crises one may first ask: what are economic crises? How should we classify them? Economists are mostly unable to answer these questions. Instead, they tend to subjectively adapt their views on crises to recent episodes. As pointed out by Lo (Reference Lo2012), there is no agreement on the basic facts related to the latest crisis, let alone on its starting date. Empirical evidence is fuzzy and interpretations as well as conclusions exhibit significant heterogeneity. The problem of definition is compounded by the fact that crises shape economic discourses (Flandreau Reference Flandreau2012). This in turn affects the generally agreed-upon narratives on specific crises and therefore the way they are perceived. Classifications of economic crises come in many forms. Some authors track ‘abnormal’ data, and declare that above or below a certain threshold there is a crisis. For instance, for Reinhart and Rogoff (Reference Reinhart and RogofF2004) a free-falling episode is defined by a 12-month inflation threshold of 40 per cent or more. For Reinhart and Rogoff (Reference Reinhart and RogofF2011), an inflation crisis is an episode during which inflation exceeds 20 per cent per year, while hyperinflation means inflation above 500 per cent per year.

Defining crises by means of thresholds and outliers has intuitive appeal. It gives the impression that crises share certain features. This is, however, valid if and only if the underlying variables are driven by an objective and invariant probability distribution. It may be wondered to what extent all so-called hyperinflations do indeed stem from a unique distribution. Is there any reason to believe that the 1946 Hungarian hyperinflation was generated by the same underlying distribution as Zimbabwe's hyperinflation in 2008? In some instances, the underlying structural relationship is obviously unstable. For example, Barkbu et al. (Reference Barkbu, Eichengreen and Mody2012) stress the changes in both crises and the IMF programmes over time. Alternatively, one could argue that each relevant hyperinflation episode is the result of a conjunction of economic circumstances driven by country-specific factors. Some Latin American countries have long lived with double-digit inflation rates, while European countries would view such a situation as a disaster. Threshold-based definitions of crises run the risk of being ad hoc. They lack theoretical grounds. For instance, data-driven thresholds for delineating hyper-inflationary episodes are arbitrary.Footnote ³ Why would crises start at 20 per cent inflation? Why not 19 per cent or 21 per cent? Although robustness checks can at least partially address this issue, threshold-based definitions of crises are both inelegant and unconvincing.

Most importantly, the threshold-based approach inevitably neglects crises averted just before their outbreak. Looking only at symptoms is insufficient to provide preventive cures. In medicine such an approach would rule out research aimed at understanding why some patients contract a disease while others remain immune. In the hyperinflation example, scrutinizing countries that managed to combat a borderline-high inflation rate could be as insightful as restricting the analysis to those who fell into the trap.

The standard empirical approach to crises is based on statistical thresholds rationalized by the supposed ‘anomalies’ associated with them. From a probabilistic standpoint, this hardly makes sense because extreme events are naturally consistent with low-probability outcomes. This is the motivation for using, say, 5 per cent thresholds in hypothesis testing. If economic crises were merely below- or above-threshold outcomes driven by otherwise-standard theory, their representation would require no special attention. In contrast, what economists have in mind when speaking about crises are most likely specific features that make the underlying model behave differently than during ‘normal’ times. Therefore, threshold-based econometrics is inappropriate for dealing with crises. Rather, if crises were special, then asymptotic theory based on standard models would be discarded.Footnote ⁴ As a result, econometrics alone is unable to properly address and classify crises. There is a strong need for a theory that does not rely on past observations but rather on an ex ante conceptual approach.

A refinement of the threshold-based approach consists in combining it with expert knowledge. In this line of thought, Schularick and Taylor (Reference Schularick and Taylor2012) use a two-step process to identify banking crises. First, they select ‘events during which a country's banking sector experiences bank runs, sharp increases in default rates accompanied by large losses of capital that result in public intervention, bankruptcy, or forced merger of financial institutions’ (Schularick and Taylor Reference Schularick and Taylor2012, p. 1038). Second, they send their selected dates to international experts for confirmation. Even though this refined approach has strong points, it still fails to provide an ex ante definition of crises. The assumption behind this approach is that expert knowledge is based on some obscure – and most likely incommunicable – theoretical fundaments. Indeed, if theory were clear, it would allow any economist to identify crises with no need for external confirmation.Footnote ⁵

III

While a consensual definition is still lacking, many articles provide stylized facts and classify crises. As expected, classifications vary across studies because observed crises differ across a wide range of features, including the role of fundamentals, the relative importance of banks and securitized debts,Footnote ⁶ the relative importance of private and sovereign debts, the exchange rate regimes and history, and the underlying structure and dynamics (Portes Reference Portes, Hunter, Kaufman and Krueger1999).Footnote ⁷ As a consequence, studies are hardly comparable. Empirical studies classify crises based on past occurrences and therefore ignore both averted and potential crises. A consensual classification should be general enough to encompass all types of imaginable crisis situations.Footnote ⁸ Besides the issues related to definition, researchers wishing to determine the probability that a crisis may occur should also engage in a reflection on the reference-class problem.Footnote ⁹

The reference-class problem relates to the ex ante classification of uncertain events. In a probabilistic framework, observations are outcomes driven by an underlying generating distribution. Hence, ex ante classification is necessary for assessing theories using observations. In addition, valid classes must constitute a partition of all imaginable outcomes. Applied to economic crises, the reference-class problem involves imagining and classifying all possible scenarios for future crises.

The reference-class problem formulated by Reichenbach (Reference Reichenbach, Reichenbach and Cohen[1935] 1978, p. 374) can be summarized as follows:

Most importantly, the probability associated with an event ‘can change depending on how it is classified’ (Hájek Reference Hájek2007, p. 563). In the case of J&J's game the decision to refer to one class of events or the other would lead to very different outcomes. If the probabilities of winning the new game were assessed on the basis of wins in general, we would expect Jack to win 40 per cent of the new games. In contrast, if the reference class is board games, the win expectation would only be 20 per cent. Choosing a reference class may thus have serious implications in terms of occurrence assessments.

In practice, establishing the probability of an economic crisis comes down to delineating the reference classes to which such crises belong. Based on these classes, one could then try to evaluate the probabilities associated with each of them. The reference-class problem epitomizes the fact that there are always many possible classes. As Eagle (Reference Eagle2004, p. 393) puts it:

Grouping crises together is an issue that some authors deal with by considering each crisis as unique. The ‘this time is different’ syndrome underlined by Reinhart and Rogoff (Reference Reinhart and RogofF2011) emphasizes that economic analysts tend to consider each new crisis as the sole member of a new class. This view was defended in 1913 by Wesley Mitchell, one of the founders of the National Bureau of Economic Research (Morgan Reference Morgan1990). Like Taleb's (Reference Taleb2007) black swans, crises are rare and dramatic events, far removed from the repetitive events that probability statements are about, such as dice throws, roulette games, or even plane crashes. In roulette, getting a 17, for example, is rather rare (probability: 1/37), but no more exceptional than any other outcome. It is a rare event, not a dramatic one. On the contrary, plane crashes are dramatic events, but unfortunately frequent enough to infer their probability with an admissible confidence range (and subsequently make recommendations for lowering their occurrence).

But any event, be it a dice throw or a plane crash, is unique in time and place. Only abstraction makes it possible to move from single cases to a repeated idealized event. For example, we ignore space–time modifications when speaking of ‘throwing the same dice ten times’. In doing so, we deliberately erase some ‘insignificant conditions’ in order to gather similar events and search for general principles. ‘Categorical repetitivity of the world's facts is a classificatory abstraction [through which] rational knowledge begins, [i.e.] when facts are reduced into classes as symbol of conceptual categories, with the search for laws which transcends the unique event’ (Scardovi Reference Scardovi and Agazzi1988, p. 59).

Single cases do not make science. But to what extent is categorization legitimate? At what point do we lose the specificity of a phenomenon under study by putting it in a predefined class? In games of chance, ignoring the ‘nitty-gritty causal details’ (Millstein Reference Millstein2003, p. 1321) of concrete situations (physical forces, etc.) in favour of symmetry considerations to elaborate scientific theories seems reasonable. Moreover, concerning roulette or plane crashes, everyone knows precisely what is at stake; no one will include ‘18’ in the ‘getting a 17’ event; nor will they include car accidents in the plane crash statistics.

The same is not true for economic crises as each so-called crisis is typically associated with a single-event story (Bernal et al. Reference Bernal, Oosterlinck and Szafarz2010). Consequently, the issue goes far beyond ‘details’ that can be ignored in order to study crises. The question ‘How to define a crisis?’ cannot be addressed without previously delineating what matters, and what does not, in the classification of crises.

In addition, the limitations of the frequentist approach are pointed out by several authors. Svetlova and Fiedler (Reference Svetlova, Fiedler, Dejuãn, Febrero and Marcuzzo2011) trace the critique back to the eighteenth-century philosopher David Hume, who questioned inductive methods and the common assumption that the future will reproduce the past. Knight (Reference Knight1921) distinguished risk from ‘true’ uncertainty. According to his view, risk relates to measurable uncertainty whereas true uncertainty is not quantifiable. Therefore, probabilities are meaningful only in case of risk, and not in case of true uncertainty. Building on this approach, Zeckhauser and Viscusi (Reference Zeckhauser and Viscusi1990) add ignorance to the picture, which results in a three-case classification. First, in situations characterized by risk, both the states of the world and their probabilities are known. Second, in uncertain situations, the states of the world are known but it is impossible to assign probabilities to them. Third, when ignorance prevails, the states of the world are no longer identified. In the same vein, John Maynard Keynes, George Shackle and Friedrich Hayek raised doubts about the application of frequentist probability measures to imperfect knowledge of future events (Svetlova and van Elst Reference Svetlova, van Helst, Gross and McGoey2015).

In a nutshell, the frequentist approach to crisis probability is flawed.Footnote ¹⁰ This approach considers that the probability of event X is to be deduced from the relative frequency of X within a reference class. It implies a sort of ‘backward causation from future results to current chances’ (Hájek Reference Hájek1997, p. 216). Such inference acts as if probability statements about X were counterfactually dependent on the future behaviour of X. But the probability of crises does not depend on whether they will actually occur in the future. For instance, crises linked to future technology, by definition, never occurred before the technology in question was invented. Therefore, this is not the right approach for estimating the probability of a crisis.

As long as crises are not properly defined within unambiguous reference classes in which inference is possible, seeking to establish the probability of a crisis is pointless. In modern economic theory, observations are considered as outcomes of a generating probability distribution. However, associating probability distributions with single events is far from obvious. Philosophers actually quarrel about the existence of such probabilities (Settle Reference Settle1977; Milne Reference Milne1986; Levy Reference Levy2007; Bauman Reference Bauman2005, Reference Bauman2008).

Assigning a single event to two different classes can bring polar consequences. In this respect, the discussion on the nature of the 1931 German crisis provides a good example. Chang and Velasco (Reference Chang and Velasco2001) distinguish three generations of crisis models: the first-generation model considers crises as an inescapable outcome of fiscal imbalances and fixed exchange rates. In the second-generation model, central banks may decide to abandon an exchange rate peg when defending it is too costly in terms of employment. The third-generation model stresses the role of financial institutions and the domestic banking system. Should the 1931 German crisis be analysed with the first- or third-generation model? According to Temin (Reference Temin2008), this difference matters not only for theoretical analysis but also for policy purposes. Indeed, if the German crisis of 1931 was just a currency crisis, the blame would rest on the government's shoulders. Alternatively, if this crisis qualifies as a twin crisis, bankers would bear the responsibility.Footnote ¹¹ Thus, the classification of the 1931 crisis affects both economic interpretation and policy recommendations.

The role of the Empire effect on colonial borrowing costs provides another example for which choices made regarding the reference class dramatically alter the results. First, Obstfeld and Taylor (Reference Obstfeld and Taylor2003) and Ferguson and Schularick (Reference Ferguson and Schularick2006) treat bonds issued by colonies essentially as sovereigns. To analyse the impact of colonial status on sovereign borrowing they include a dummy variable in a model aimed at explaining the spreads between sovereign bonds and the British consol. Such an approach assumes that colonial ‘sovereign’ bonds and bonds issued by traditional sovereigns belong to the same class of assets, namely sovereign bonds. Empirically the authors get contradictory results: in one instance being a colony would lower borrowing costs in the other case it would not. Second, according to Accominotti et al. (Reference Accominotti, Flandreau and Rezzik2011) colonial ‘sovereign’ bonds do not belong to the same reference class as traditional sovereign bonds. The colonial status embeds implicit benefit from an imperial guarantee. As a result, belonging to the Empire has a structural effect rather than a marginal one.

Many other examples of reference-class problems may be found in the literature. For instance, Barkbu et al. (Reference Barkbu, Eichengreen and Mody2012) discuss the impact of grouping the Russian crisis of 1998 with the subsequent Latin American and Turkish crises and conclude that ‘any taxonomy of crisis episodes is controversial’ (Barkbu et al. Reference Barkbu, Eichengreen and Mody2012, p. 423). The Eurozone sovereign debt crisis provides an additional example of a reference-class problem. In the 1990s leading scholars such as Eaton and Fernandez (Reference Eaton and Fernandez1995) considered that sovereign debt crises only concerned emerging markets as major economies seemed safe from default. This amounts to distinguishing between different classes for ‘sovereign debt defaults in emerging markets’ and a more general class ‘sovereign debt defaults’. There is a priori no correct class even though the probabilities of default would change dramatically depending on the preferred definition.

IV

The ambiguity related to the definition of crises and the reference class may generate additional sample biases. In the case of J&J's game, how interrupted games are treated affects our ability to predict future winners. Omitting a tennis game during which Jack won four sets out of five may bias our expectations. The same argument holds for crises which were averted. In many instances economists just discard data because they consider it abnormal, but is such an approach warranted? Similarly, in J&J's game should the chess game for which Jack claimed to have had a headache be excluded? Eventually, if the new game played by J&J is so different, can we rely on our past observation to predict an outcome, or should we rely solely on our deductive analysis and imagination? Basically, are new forms of crises predictable on the basis of past data?

In the current state of economic knowledge, the onset of crises remains largely unpredictable. Admitting this fact instead of changing theory after each crisis will avoid the pervasive ‘this time is different’ syndrome (Reinhart and Rogoff Reference Reinhart and RogofF2011). To illustrate this point, we put forward some common distortions stemming from ad hoc analyses of crises. In particular, we pinpoint the consequences of neglecting averted crises. Such ‘unborn’ crises, often referred to as ‘peso problems’, are typically acknowledged in specific contexts, but ignored when it comes to classifying crises on a large scale. We then discuss the sample bias implied by the decision to treat some data as abnormal before discussing the role of imagination.

First, ex post views on crises prevent researchers from factoring in events that were perceived as dramatic when they occurred but failed to leave a long-lasting footprint. In particular, the so-called ‘peso problem’ relates to ex post implications of a disruptive event that was expected to happen, but actually did not (Veronesi Reference Veronesi2004). As Sill (Reference Sill2000, p. 4) puts it: ‘Peso problems can arise when the possibility that some infrequent or unprecedented event may occur affects asset prices. The event must be difficult, perhaps even impossible, to accurately predict using economic history.’ In practice, this problem creates sample biases in econometric analyses of crises. Indeed, a peso problem occurs when positive probabilities were initially assigned to ultimately missing events. The actual observations used to calibrate or estimate models do not properly account for the real-time evidence, and predictions prove inaccurate.

Second, biases may result from deliberate choices made by researchers. For instance, many economists simply discard wartime data. They implicitly assume that such data are not generated by the same economic mechanisms as ‘normal’ (i.e. peacetime) data. Be that as it may, focusing on ‘normal data’ raises the issue of defining ‘normality’. The assumption that wartime data is ‘abnormal’ from a historical standpoint may seem ludicrous. As pointed out by Lerner (Reference Lerner1954, p. 506), ‘economists have left whole areas of valuable statistical information unexamined because they have considered wartime figures “abnormal”’. Sixty years after this statement, most authors continue to omit wartime observations when dealing with long-term economic series. This can result in severe sample biases, including survivorship bias.

Third, even if all economic crises were intrinsically similar in nature, the current approaches in economics would be too biased and incomplete to address this issue. Indeed, crisis investigations fail to capture the whole universe from which crises are drawn. The typically backward-looking approach economists tend to adopt through econometric investigation lacks imagination about the emergence of new crises that may be the first of their kind. Such new crises can result from new technologies or new ideologies. Within an ever-evolving human society, the shapes of economic crises inevitably change.

Lack of imagination, exclusion of ‘abnormal data’, and the peso problem may lead to inappropriate understanding of the nature of economic crises.Footnote ¹² Based on real-life situations, the remainder of this section shows how these three issues reinforce the reference-class problem.

The term ‘peso problem’ was coined in 1976 following Milton Friedman's comments on the Mexican peso market (Sill Reference Sill2000). At the time, traders were speculating on a large devaluation of the peso. Accordingly, the currency sold at a discount on the forward market with respect to the spot market. Eventually, however, the peso did not devalue. Hence, spot and forward prices give the wrong impression that the market was inefficient during the speculative period. This biased conclusion is mainly driven by the impossibility of observing traders' real-time expectations.

More generally, a peso problem designates a situation in which traders pay attention to the possible occurrence of a dramatic event that ultimately fails to occur. In such situations, market prices are driven by pessimistic, but most likely rational, expectations. However, ex post there is no observable crisis associated with the price decline, which therefore seems irrational. Ex ante, however, being pessimistic might have made perfect sense. What would have happened if the ‘Y2K bug’ had materialized or if the Cuban missile crisis had turned into a world war?

Empirical findings have been criticized for failing to identify peso problems. Krasker (Reference Krasker1980) uses this argument to refute the common belief that the mark/pound exchange rate was irrationally priced during the German hyperinflation of 1922–3. Burnside et al. (Reference Burnside, Eichenbaum, Kleshchelski and Rebelo2011) attribute the claimed profitability of carry trade speculation to a peso problem.Footnote ¹³ Notwithstanding, the economic literature on crises tends to overlook the peso-problem issue. This may be due to the difficulty in determining not only (unrealized) dramatic events but also the moment when traders started to factor them into their expectations.

Economic history shows that identifying peso-problem situations is extremely hard, especially because analysts know the eventual outcome and suffer from an ex post bias. For instance, several papers attempt to understand market anticipations regarding conflict outcomes.Footnote ¹⁴ In some cases, sharp market movements are not associated with events that historians view as dramatic. According to Willard et al. (Reference Willard, Guinnane and Rosen1996) the most important break in the prices of greenbacks during the US civil war corresponds to the retreat of Confederate General Jubal Early from the suburbs of Washington, an episode largely downplayed by modern historians. Samples of crises built from ex post historical appreciation thus most likely omit crises perceived as such in real time. This is due to interpretations made with knowledge of subsequent events. As pointed out by Frey and Waldenström (Reference Frey and Waldenström2007, p. 3), ‘this knowledge may bias the evaluation of the events, and may lead to “facts” being overlooked or overemphasized, as the case may be’.

In sum, peso problems may harm an empirical analysis involving unobservable expectations.Footnote ¹⁵ They lead researchers to conclusions plagued by missing variables associated with unrealized dramatic events. Circumventing peso problems requires gathering relevant information on real-time circumstances and expectation formation. This is mostly unfeasible for large cross-section studies. In particular, regarding the economic analysis of crises, peso problems raise challenging issues. Should researchers ignore crises that had been feared but never occurred? Otherwise, how should they classify these unborn crises? Since ex post biases are inevitable, peso problems should be addressed from a theoretical perspective.

Economists often discard wartime data, considering it ‘abnormal’. But history shows that the ‘abnormal’ wartime environment can in some instances be the norm. During the period of Louis XIV's reign when he ruled alone (1661-1715), France was at war for more than 60 per cent of the time. For the Napoleonic period (1800-15), the figure is close to 80 per cent. Figures are even higher in the case of Philip II of Spain: during his reign (1556-98), the empire was at peace for just one year (Drelichman and Voth Reference Drelichman and Voth2011). Furthermore, although wars usually take place in a well-defined time frame, the beginning and end dates of crises are often blurred and economists fail to reach a consensus on them (Lo Reference Lo2012). Studies should thus be comprehensive enough to encompass all periods. Omitting ‘abnormal’ data also prevents an analysis of economic crises in their natural context.

Additionally, disregarding wartime data can trigger a survivorship bias. We illustrate this observation using the so-called ‘equity premium puzzle’. In their seminal paper, Mehra and Prescott (Reference Mehra and Prescott1985) argue that the size of the equity premium observed from US equities cannot be explained by standard general equilibrium models unless agents exhibit unrealistically high levels of risk aversion. This puzzle subsequently became a key issue in finance and macroeconomics (Kocherlakota Reference Kocherlakota1996). A convincing solution to this puzzle is proposed by Jorion and Goetzmann (Reference Jorion and Goetzmann1999), drawing on the survivorship bias associated with sample selection. The authors point out that most empirical studies concentrate on uninterrupted markets only, and equity returns are therefore much higher than if all markets were taken into account. To prove their point, Jorion and Goetzmann (Reference Jorion and Goetzmann1999) build a database comprising both interrupted and uninterrupted markets. They find that the equity premium is indeed significantly lower in interrupted markets. The deliberate omission of ‘abnormal’ data may thus be sufficient to create a long-lasting ‘puzzle’.

The 1918 repudiation by the Soviets of the bonds issued by the Tsarist regime shows how hard it is to imagine new forms of crises before they occur. This bond crisis most likely represents the first historical example of ideologically motivated debt repudiation.

At the end of the 1890s France allowed Russia to issue a huge number of government bonds on the Paris stock exchange. Political reasons played a key role. As tensions with Germany escalated, the French government wanted to make sure that its Russian ally would remain faithful. The Russian government, knowing the importance of external funding, heavily bribed French politicians (including the finance minister), the head of the stock exchange and the press (Raffalovitch Reference Raffalovitch1931). As a result, by 1914 Russian bonds were trading on the Paris Bourse for an amount estimated to represent a staggering 4.5 per cent of French wealth (Ukhov Reference Ukhov2003). When the Soviets managed to take power, they refused to recognize Tsarist debt and decided, in February 1918, to repudiate all former Russian debts. Despite this extreme statement, bondholders kept hoping to be reimbursed. More than two years after the repudiation, some bonds were still traded above 50 per cent of par (Oosterlinck and Landon-Lane Reference Oosterlinck and Landon-Lane2006).

French investors who had bought Russian bonds were certainly not irrational. When the repudiation occurred they had legitimate reasons to hope they would still be reimbursed. At the time, there was no historical precedent in which a country had, solely for political reasons, repudiated a huge amount of debt without reaching a settlement sooner or later. Even in the dramatic hours following the French Revolution, the legality of the royal debt was not questioned. The revolutionaries defaulted on two-thirds of the debt but bondholders nonetheless recovered part of their investment (Aftalion Reference Aftalion1996). Scholars working on crises in 1917 could not possibly have included the probability of an as-yet unknown form of crisis in their work: the Soviet Revolution. The literature on sovereign debts considers three main reasons why countries would repay their debt: fear of loss of reputation (and thus loss of access to the capital market), fear of military intervention and fear of trade retaliation. The Russian case is exceptional inasmuch as the Soviets did not fear any of these threats and thus the repudiation completely fell out of the range of considered outcomes.

The problems presented in the Russian case are in fact quite common. When the first commodity bubble, centred on tulips, imploded in the seventeenth century, observers were unable to understand that such a crisis could have occurred at all. They blamed wild speculation and irrationality. In the absence of a precedent in which commodity prices would have followed a bubble-shaped pattern, they were at a loss to include such a crisis in their expectations. The same may be said for the South Sea and Mississippi bubbles. In both cases, schemes to exchange government bonds for company shares ended up in a new form of crisis. Even the 1929 stock market crash may be viewed as a totally new form of crisis. World-famous economists such as Irving Fisher completely failed to foresee the crisis, which would eventually lead to a dramatic drop in the prices of all the shares traded on the exchange.Footnote ¹⁶

The recent sovereign debt crisis in the Eurozone provides another enlightening example. The risks related to the creation of a single-currency zone with no lender of last resort were hard to fathom. In fact, some scholars believed that the advent of the Eurozone had allowed countries to overcome their ‘original sin’ problem, i.e. their incapacity to issue debt abroad in their own currency. Eichengreen et al. (Reference Eichengreen, Haussmann, Panizza, Eichengreen and Hausmann2005) stressed the positive effects of the euro on sovereign debt issuance. Their measure of original sin for Euro-land countries experienced a steep fall from (0.53 to 0.09) following the introduction of the euro. In contrast, when the crisis occurred, struggling countries proved unable to print more euros to reduce the debt burden.

Economics has been called into question by the 2007–8 financial crisis, which most researchers failed to predict. In our view, this failure is due more to a lack of imagination than to inadequate models. If anything, the blame should be placed on restrictive interpretations of model outcomes, not on the models themselves. In addition, failure to predict crises accurately is not specific to economics; it is also frequent in the natural sciences (Franz Reference Franz2011).

We argue that the only way to overcome the three issues exemplified in this section is to start from theoretical models comprehensive enough to address the reference-class problem. Such models should allow researchers to use all available data without distinction. While sample biases and peso problems are likely to affect any empirical study, they can be largely avoided when empirical analysis follows from a proper theoretical framework.

V

In the natural sciences, explanatory power and realism are deeply related. The success of theories in physics relies on empirical confirmation, mainly through experimentation. In that context, the explanatory power is bound to the realism of the model: what the model explains is the reality under study. Since the models are empirically verifiable or falsifiable, their explanatory capacity merges with a descriptive and predictive capacity. The models concurrently explain and describe the world as it is, and predict how it will be. Inescapably, models evolve with time, but this is the best researchers can do. As Batisty and Domotor (Reference Batisty, Domotor and Ciprut2008, p.169) put it: ‘True, a model that works may still prove to be inadequate in some respects in the future. But accepting the mere possibility of such a future discovery does not in any way undermine our rationality as scientists in our commitment to the best-working model we currently have.’

For obvious reasons, economists are unable to run crisis experiments. Instead, they need to rely on theoretical models and historical data. When experimentation is unfeasible, the simultaneity of explanation and prediction is possible only if the phenomena under study are stable and repetitive, like the movements of planets, which is not the case for economic crises. Like epidemiologists who have announced the possibility of global pandemics, which ultimately did not materialize (mad cow disease, avian flu), economists find themselves in an uncomfortable situation when dealing with crises (Franz Reference Franz2011). Weighing the risk of panic against the accusation of being incapable of predicting future crises is a challenging trade-off.

For natural scientists, realism is the most important issue. Unification-type explanations are more suited to the social sciences than are the deductive-nomological explanations of the natural sciences (De Scheemaekere Reference De Scheemaekere2009). Explanations indeed increase our understanding of the world by reducing the number of independent phenomena that we have to accept as ultimate. Hence, an economic theory has a significant explanatory capacity insofar as it describes the economic world with fewer independent phenomena – and thus makes this world, other things equal, more comprehensible (Friedman Reference Friedman1974). This is especially relevant when data are unreliable.

It might be of great interest for a theory to account for once-expected large changes that did not ultimately occur (peso problem situations). Given the radical uncertainty underlying financial crises, the balance between the urge to predict (realism) and the need to understand (explanatory value) should tip in favour of the latter, because what matters ultimately are the human decisions taken for reducing the occurrence of crises, not the (impossible) accurate prediction of the next crisis. In economics, the link between a theoretical model and its empirical verification rests on the soundness of human judgement, not the precision of a mechanical measurement.

To this day there is no consensus regarding the definition and classification of crises. Economic studies are deeply rooted in the frequentist approach. Typically, they use an ex post threshold-based definition of crises. This implicitly assumes that all crises are simply tail observations driven by the same underlying probability distribution as the ‘normal situation’. This contradicts the intuition that crises exhibit features that make the underlying model behave differently than during ‘normal’ times. This approach is also plagued by the reference-class problem formulated by Reichenbach (Reference Reichenbach, Reichenbach and Cohen[1935] 1978). Attributing a given crisis to one or another class can lead to dramatically different conclusions, including for policy recommendation.

Focusing on past crises and disregarding crises that were eventually averted is akin to building a medical science by ex post examining incurable patients only, not those who either did not get sick or were successfully cured in due course. Moreover, economists lack imaginative models to apprehend new forms of crises. Rather, they tend to restrict their models to either ‘normal’ or ‘abnormal’ past periods. This distinction ignores the fact that reality is a succession of both normal and crisis periods. In particular, crises start in normal periods. Whereas in the natural sciences, models can predict the existence of particles before they are empirically discovered, such as the Brout–Englert–Higgs boson, or diseases before their outbreak, like disease mutations, existing economic models fail to consider potential innovations in crises. Potential crises come from the theory itself, not from ex post – and typically ad hoc – considerations. Models that are supposed to hold only under ‘normal conditions’, leaving the possibilities for crises aside, are too narrowly conceived.