1. Introduction

Levins (Reference Levins1966) claimed scientific modeling can maximize at most two of three virtues: generality, realism, and precision. Models sacrificing generality (SG) make precise quantitative predictions about specific systems and maximize realism by representing as many system details as possible. Models sacrificing realism (SR) make unrealistic assumptions so scientists can describe systems with general, mathematically-tractable equations that produce precise quantitative predictions.

Qualitative models that sacrifice precision (SP) abandon quantitative accuracy and focus on qualitative relations between model variables. Loop analysis, one method of qualitative modeling systematically developed by Richard Levins (Reference Levins1974, Reference Levins, Cody and Diamond1975, Reference Levins, Rapport, Contanza, Epstein, Gaudet and Levins1998; Puccia and Levins Reference Puccia and Levins1985), consists of the analysis of signed digraphs (directed graphs) representing whether increases in individual variables induce qualitative increases or decreases in other variables, or leave them unchanged.

Levins (Reference Levins1966) did not explicitly define ‘generality’, ‘realism’, and ‘precision’, which prompted Orzack and Sober (Reference Orzack and Sober1993) to criticize the clarity of his distinctions between modeling strategies. They also criticized that the kind of qualitative testing involved in qualitative modeling is conceptually and methodologically problematic since:

1. 1. Grounds for accepting qualitative predictions are often unstated; and,

2. 2. Unlike quantitative testing, qualitative testing cannot determine how well models account for data.

This paper defends qualitative modeling against this criticism. Section 2 briefly considers some weaknesses of SG and SR modeling of complex systems that qualitative modeling avoids. After defining Lyapunov stability, Section 3 illustrates how loop analysis, which is an example of only one kind of qualitative modeling,Footnote ¹ evaluates the stability of mathematical models of ecological communities. Based on this analysis, Section 4 argues Orzack and Sober's (Reference Orzack and Sober1993) criticism is unwarranted. Specifically, the basis of (1) as a criticism of qualitative testing is dubious, and (2) presupposes an overly narrow view of qualitative modeling, and the function of models within science in general.

2. Limitations of Quantitative Modeling of Complex Systems

For Levins, successful scientific research requires several different modeling strategies, each exhibiting strengths and weaknesses relative to different purposes and contexts. In the context of modeling complex systems to understand their dynamics, Levins thought SG and SR modeling, unlike qualitative modeling, face significant difficulties.

Levins (Reference Levins1966, 421) considered three disadvantages of SG models:

1. 1. Their construction requires measurement of an intractable number of parameters, many requiring several years to measure precisely.

2. 2. Even if these parameters could be measured, the resulting differential or difference equations would be analytically insoluble and exhaust the numerical solution capabilities of computers.

3. 3. Even if soluble, their solutions, “would have no meaning for us.”

Disadvantage 1 concerns the practical problem of maximizing model realism by maximizing the number of parameters measured. This is usually infeasible within the monetary and temporal constraints of scientific research. Disadvantage 2 concerns the unmanageability of such complicated models. The sheer size and intractability required to maximize realism and precision made Levins skeptical SG models would provide scientifically useful representations of complex systems.Footnote ²

Disadvantage 3 concerns the limited explanatory power of SG models. Complicated models that are difficult to manage are also difficult to comprehend. Without a clear comprehension of their structure and dynamics, SG models often frustrate rather than enhance understanding.Footnote ³ Botkin (Reference Botkin1977) emphasized this when criticizing the “systems approach” to ecosystem modeling which, historically, was the target of Levins’ (Reference Levins1966) criticisms of SG modeling.

SR modeling faces different disadvantages. Unlike SG models, SR models focus on small sets of salient system components and thereby avoid Disadvantages 1–3. This restricted focus is achieved through idealization: ignoring some system components and interactions, treating interactions as instantaneous, representing discrete components with continuous variables, etc. Scientists can then represent real-world systems with idealized but thoroughly studied, well understood mathematical models that produce precise quantitative predictions. The underlying assumption, Levins (Reference Levins1966) suggested, is that differences between predictions and observations will identify what idealizations preserve accurate description.Footnote ⁴

Unrealistic idealizations, however, make it uncertain whether modeling results demonstrate properties of the represented system or are byproducts of unrealistic idealizations. Since it is often unclear what properties are primarily responsible for system dynamics, idealizations may significantly mischaracterize its most important features. Consequently, an enhanced sense of understanding conveyed by an SR model may fail to be about the system it is intended to represent. This especially troubled Levins since he believed biologists often uncritically emulated mathematically sophisticated models of physics to ensure their modeling was mathematically rigorous, with the consequence that, “theoretical work often diverged too far from life and became exercises in mathematics inspired by biology rather than an analysis of living systems” (Reference Levins1968, 4). Levins understood that all modeling involves idealization. His affinity for qualitative modeling, however, stemmed from its avoidance of the substantial kinds of idealizations SR models make. Loop analysis, Section 3 shows, achieves mathematical rigor with a comparatively minimal sacrifice of realism.

3. Qualitative Stability Criteria and Loop Analysis

Consider a system represented by n differential equations:

where c _j are parameters, x _k are variables, $1\leq j\leq m$, and $1\leq i$, $k\leq n$. At equilibrium,

In general, systems at equilibrium can be globally or locally stable. Informally, a system at a globally stable equilibrium returns to equilibrium following any disturbance. It is improbable any ecological systems are globally stable, but probable some are locally stable. Local stability depends upon how systems behave in the local neighborhood of an equilibrium. To analyze this behavior, the equations of (1) can be linearized:

where x(t) is the vector of n variables $x_{1},\,\ldots,\,x_{n}$ and A is the n×n matrix of constant real coefficients a _ij derived from the nonlinearized interaction terms of the Jacobian matrix, $c_{ij}=\partial F_{i}/\partial x_{j}$ evaluated at equilibrium.

By definition, an equilibrium x* is Lyapunov stable iff

where x(t) is a solution of (2) with initial conditions x(t ₀), and $\vert \cdot \vert $ designates a Euclidean distance metric. Informally, (3) says x* is Lyapunov stable if a system beginning in a neighborhood of x* remains near it after perturbation. By definition, x* is asymptotically Lyapunov stable iff (3) and $\mathbf{x}(t) \rightarrow \mathbf{x}^{*}$ as $t\rightarrow \infty $.

Whether x* is locally asymptotically Lyapunov stable (hereafter ‘stable’) depends upon the eigenvalues of A. These are scalar values λ such that $\mathrm{det}\,(\mathbf{A}-\lambda \mathbf{I}) =0$, the roots of the characteristic polynomial of A. Lyapunov (Reference Lyapunov[1892] 1992) proved x* is stable iff

where Reλ_i(A) designates the real part of λ_i, the ith eigenvalue of A.

The Routh-Hurwitz stability criterion (Gantmacher Reference Gantmacher1960) extends Lyapunov's theorem and is the basis of Levins’ account of stability in loop-theoretic terms. It states (4) holds iff every Hurwitz determinant,

( $i=1,\,\ldots,\,n$), is positive, where the b _i are the coefficients of $\mathrm{det}\,(\mathbf{A}-\lambda \mathbf{I}) $.

In models of ecological communities, a _ij from (2) represents the effect of species j on species i.Footnote ⁵ Its quantitative value represents the effect's magnitude and its sign represents whether the effect is positive or negative. Unfortunately, Lyapunov's theorem and the Routh-Hurwitz criterion provide little help in assessing whether communities are stable. The problem is that determining whether they are satisfied usually requires information about the quantitative values of the a _ij, which is typically unavailable and cannot be feasibly obtained. Measuring all these values, for instance, requires measurement of n ² species interactions, each of which would require numerous manipulative experiments.Footnote ⁶ Coefficient signs are more easily determined. For instance, for competitors, a _ij, $a_{ji}< 0$ and for predator and prey, $a_{ij}> 0$ and $a_{ji}< 0$.

Since coefficient signs often can be determined, hypotheses about ecological communities often can be evaluated without quantitative data. One such hypothesis is whether community stability depends on the quantitative strengths of species interactions or their qualitative structure in the community alone. Whereas SG and SR modeling requires quantitative data which is unavailable in this context, the sacrifice of quantitative precision makes qualitative modeling well suited to analyze this hypothesis.

Loop analysis is based on equivalence between matrices of constant coefficients and digraphs, first anticipated by Wright (Reference Wright1921). For systems represented by (2), $x_{1},\,\ldots,\,x_{n}$ correspond to digraph vertices. If A represents qualitative information about species interactions, a _ij takes values +1, −1, or 0 to represent positive, negative, or null effects of species j on species i. These values determine what vertices are connected by edges and the edges’ direction and effect. If $a_{ij}=1$, $x_{j}\rightarrow x_{i}$ designates positive effect; if $a_{ij}=-1$, $x_{j}$

$x_{i}$

$a_{ij}=0$

corresponds to the digraph in Figure 1. A loop is a series of directed edges from one vertex to itself crossing no intermediate vertices more than once. The number of edges is a loop's length and disjunct loops share no vertices. $a_{HP}a_{PH}$, and a _NN, for instance, are disjunct loops of length 2 and 1.

Figure 1.

The equivalence between matrices and digraphs entails a correspondence between matrix determinants and loops. For example, if

$\mathrm{det}\,(\mathbf{A}) =a_{11}a_{22}-a_{12}a_{21}$, which is the difference between the product of length 1 loops and length 2 loops in the digraph of A. Levins (Reference Levins, Cody and Diamond1975, 20) generalized this relationship to n-order matrices:

where L(m, n) is the product of n coefficients forming m disjunct loops and L_m,n is the set of all such products in the digraph of A. L(2, 4), for instance, is the product of the four coefficients of two disjunct loops. With this generalization, Levins (Reference Levins, Cody and Diamond1975, 21) defined “feedback at level k” in n-variable systems:

where $1\leq k\leq n$. Notice $F_{1}(\mathbf{A}) =\sum_{i=1}^{n}a_{ii}$, the sum of the diagonal elements of A, the length 1 loops.

According to Puccia and Levins (Reference Puccia and Levins1985), feedback is a process by which changes in variables induce changes in other variables that then affect the variables originally changed. Positive feedback enhances change: increase in variables induces further increase, and decrease induces further decrease. Negative feedback counteracts change: increase induces decrease, and decrease induces increase.Footnote ⁷

With this definition of feedback, Levins formulated the Routh-Hurwitz criterion in loop-theoretic terms. (See Puccia and Levins Reference Puccia and Levins1985, Chapter 6.) An n-variable system is stable iff $(\forall k< n) \,(F_{k}< 0),$ and

The first condition requires negative feedback at every level and the second requires stronger feedback at lower levels than higher ones. For instance, for 3 variable systems the second condition requires $F_{1}F_{2}+F_{3}> 0$.

One drawback of this loop-theoretic criterion is that quantitative data are often required to determine whether it is satisfied. The system represented in Figure 1, for example, is stable iff $a_{NN}a_{HP}> a_{NP}a_{HN}$, which typically depends on their quantitative values. Furthermore, the probability quantitative data will be required, and the number of variables for which it will, increases with the number of model coefficients (Dambacher et al. Reference Dambacher, Luh, Li and Rossignol2003).

This limitation of, ultimately, the Routh-Hurwitz criterion prompted searches for completely qualitative stability criteria, which economists Quirk and Ruppert (Reference Quirk and Ruppert1965) found first. They defined the sign-pattern of a matrix A as its pattern of coefficients (+1, −1, 0). B is sign-similar to A iff their sign-patterns are identical. A is sign-stable iff the eigenvalues of every matrix sign-similar to it satisfies (4). That is, sign-stable matrices remain stable with any specification of the quantitative values of their entries preserving their sign-pattern. Quirk and Ruppert (Reference Quirk and Ruppert1965) proved a real n×n matrix $\mathbf{A}=[ a_{ij}] $ with negative diagonals, i.e., (∀i) ( $a_{ii}< 0$), is sign-stable iff

and

Other more complicated necessary and sufficient conditions were found (see Jeffries Reference Jeffries1974 and Logofet Reference Logofet1993), but they were similarly biologically unrealistic (Jefferies Reference Jeffries1974). The restriction to negative diagonals requires every species be self-damping, which entails stable communities cannot contain species exhibiting Allee effects, and (7) requires communities not contain competitors or symbionts; both are extremely implausible. It is probable, therefore, that community stability depends on the quantitative strengths of species interactions, not merely their qualitative structure.

As this limited exposition indicates, loop analysis is a rigorous method of scientific analysis. Establishing (7) and (8) and other conditions entail and are entailed by sign-stability, for instance, requires sophisticated mathematical proof. Before their critique of qualitative modeling, however, Orzack and Sober (Reference Orzack and Sober1993, 538), characterized a qualitative model as one that, “makes only a qualitative prediction,” and, “In this sense, these models are not mathematical.” Senses differ about the exact meaning of the term, but their claim is indefensible if intended to support the criticism that qualitative modeling, unlike quantitative modeling, lacks rigor because it is not ‘mathematical’.Footnote ⁸ Quantification is not necessary for rigor in science any more than in mathematics. This was understood by Levins, who considered loop analysis to be a form of qualitative mathematics of functions only specified as increasing/decreasing, or convex/concave, without quantitative precisification (Levins Reference Levins, Rapport, Contanza, Epstein, Gaudet and Levins1998). Similarly, Simon (Reference Simon, Fishwick and Lucker1991, vi) characterized qualitative analysis as, “the mathematics of monotonic transformations, or … the mathematics of ordinally measured quantities.” Although not explicitly stated, the supposition that rigorous scientific modeling requires quantification seems to underlie Orzack and Sober's (Reference Orzack and Sober1993) criticisms of qualitative modeling.Footnote ⁹

4. Orzack and Sober's Criticisms of Qualitative Modeling

After criticizing Levins’ (Reference Levins1966) distinctions between modeling strategiesFootnote ¹⁰ and concept of robustness, Orzack and Sober (Reference Orzack and Sober1993) argued that the qualitative testing involved in qualitative modeling is conceptually and methodologically problematic. Specifically, they criticized that, although qualitative testing can be useful,

1. 1. “Grounds for accepting qualitative predictions are often left unstated. One consequence is that investigators sometimes use contradictory criteria to judge the same model. … Although this is also a potential problem in quantitative testing, [this] approach usually leads biologists to state test criteria explicitly” (Reference Orzack and Sober1993, 542).

2. 2. “The most important defect in qualitative testing, however, is that it fails to allow one to answer the most important question about a particular model: How well does that model explain the data? Qualitative testing may show some models are incompatible with data, but only quantitative testing of quantitative models can determine what one if any sufficiently explains the data” (Reference Orzack and Sober1993, 542).

Regarding the adaptationism debate, Orzack and Sober (Reference Orzack and Sober1993, 543) explained that quantitative models of traits can be tested with data to determine whether, “natural selection has been so important in a trait's evolution that nonselective forces may be safely ignored.” Qualitative testing, however, is incapable of such determinations since it, “fails to allow one to discriminate between the claim that natural selection is an important cause of what we observe and the claim that natural selection suffices as an explanation for the trait.” Orzack and Sober (Reference Orzack and Sober1993, 543) concluded their criticisms of qualitative optimality models of adaptation with a general indictment: “there is no more compelling reason to reassess the view of models endorsed in Levins’ Reference Levins1966 paper than the fact that the idea of qualitative modeling has hindered the development of an unbiased assessment of the truth [of adaptationism].”

As a general critique of qualitative compared to quantitative modeling, the first problem is that the data required for quantitative testing often do not exist and cannot be collected feasibly. Besides ecological communities, this is often true when system dynamics are complex, and for this reason Levins stressed the utility of loop analysis in modeling other complex systems, such as social systems (Puccia and Levins Reference Puccia and Levins1985). In some scientific fields, furthermore, the relevant data are essentially qualitative (e.g., ordinal) and interpreting them quantitatively is indefensible. Social sciences that model agent behavior by analyzing strictly ordinal information about preference rankings are a clear example. Thus, even if some scientific issues, like the adaptationism debate, can be properly assessed only by quantitative testing with adequate quantitative data, this does not justify a blanket critique of qualitative modeling.

In general, criteria by which models should be evaluated, for instance in assessing whether models can be simplified, should reflect the, “reality to be described [and] the state of the science” (Levins Reference Levins1966, 422). Comparative criticism of modeling strategies, therefore, is only justified when both strategies are appropriate for the context. Criticism 2 fails to appreciate that qualitative modeling is well suited to scientific fields in which quantitative modeling is infeasible or inappropriate.

The second problem is that Criticism 1 is dubious as a criticism of qualitative modeling. Orzack and Sober (Reference Orzack and Sober1993) only briefly discussed the difficulty, but cited Orzack Reference Orzack1990 in support. Orzack (Reference Orzack1990) analyzed several studies of Werren's (Reference Werren1980) extension of Hamilton's model of mating competition.Footnote ¹¹ Werren (Reference Werren1980) tested his model, as did subsequent studies, by plotting observations of the second sex ratio of parasitic wasps against predicted values and visually assessing their “fit.” Based on this assessment, Werren (Reference Werren1980, 1158) judged, “The sex-ratio data show the trend predicted,” without presenting any supporting statistical tests. Subsequent studies with similar results that also presented no statistical tests, however, suggested different conclusions. Orzack (Reference Orzack1990) correctly identified two problems with this method of model evaluation: visual appearance of fit depends significantly on presentation scale, and judgments about what can be concluded from such fit vary dramatically.

As criticisms of qualitative modeling, however, these problems are misplaced. Werren (Reference Werren1980) specified his model's mathematical form, measured the variable needed to make quantitative predictions, and tested those predictions against quantitative data. In Orzack and Sober's (Reference Orzack and Sober1993, 535–536) classification of model types, Werren's (Reference Werren1980) model is instantiated and generates quantitative point predictions; it is not qualitative. Orzack's (Reference Orzack1990) criticisms, therefore, target only one, obviously flawed, type of qualitative test, visual inspection of fit, applied to a quantitative model.

In fact, Orzack and Sober's (Reference Orzack and Sober1993) view of qualitative testing seems limited to this kind of test. After Criticism 1, which they believed highlights a general deficiency of qualitative testing, they continued, “Qualitative assessment of fit can also be highly dependent upon the manner of graphical presentation” (Reference Orzack and Sober1993, 542). Qualitative testing should not, however, be narrowly identified with problematic visual assessments of fit. As Section 3 and other examples make clear, qualitative testing can be a rigorous method of scientific analysis. What Orzack (Reference Orzack1990) revealed was bad scientific methodology, not methodological problems with qualitative modeling. The appropriate test of a significant fit between observations and quantitative predictions is regression analysis, in Werren's (Reference Werren1980) case, nonlinear regression. That regressions were not performed justifies suspicion that results were not significant and this was the reason these studies appealed to visual assessments. Application of poor qualitative tests when quantitative data are available and quantitative statistical tests should be performed does not, however, support a general indictment of qualitative modeling.

The final problem is that Criticism 2 presupposes an unjustifiably narrow view of the function of models within science. Unlike Criticism 1, Criticism 2 focuses on the limitations of what qualitative models can show even if they are properly testable. Even if, contrary to Criticism 1, testing qualitative models is no more problematic than testing quantitative models, Criticism 2 suggests some scientific issues may be resolvable by the latter but not the former. Regarding adaptationism, for instance, Orzack and Sober (Reference Orzack and Sober1993, 543) claimed the fundamental flaw of qualitative models is their inability to provide sufficient explanations of traits.Footnote ¹² This is a limitation of qualitative modeling, but it should not be exaggerated. First, as Section 3 shows, it does not apply to all scientific questions. Second, it does not demonstrate qualitative modeling has hindered resolution of the adaptationism debate, or other debates it alone cannot resolve. Besides making qualitative predictions, qualitative models help scientists understand phenomena, which is crucial in developing scientific explanations and, ultimately, resolving such debates. The understanding qualitative models provide is particularly useful since, unlike SG models, it is generalizable and, unlike SR models, it reflects realistic assumptions about the system represented.Footnote ¹³

Most accounts of scientific explanation agree that enhancing understanding is essential to explanation but differ about the kind of understanding required. Without broaching this issue, it should be noted that Sober takes a broad view of the kinds of understanding involved in scientific explanation, unlike causal accounts that narrowly require understanding of the causal processes underlying phenomena. Sober (Reference Sober1983), for instance, cogently argued that equilibrium explanations are counterexamples to such causal accounts. Considering differences between causal and equilibrium explanations of 1:1 sex ratios, he explained (Reference Sober1983, 207):

Sober's argument supports rather than contradicts the view that (qualitative) loop-theoretic explanations of community stability are scientifically sound. Satisfaction of the conditions of Quirk and Rupert's proof, for example, entails a system is sign-stable, but is consistent with an infinite number of quantitative specifications of its coefficient values. Loop analysis does not pinpoint particular ‘causal scenarios’ of stable system behavior as actual but does ‘situate’ them within certain qualitative constraints on how system components interact. Why then is Sober critical of qualitative modeling a decade later if, as his argument seems to affirm, loop analysis and other qualitative modeling methods enhance scientists’ understanding and help them develop scientific explanations?

One answer may be a narrow conception of qualitative modeling and implicit assumption it lacks rigor already discussed. A second may lie in a general tension between Levins’ conception of the principal goal of qualitative modeling and the narrow view of scientific modeling Orzack and Sober's (Reference Orzack and Sober1993) analysis seems to presuppose. For Puccia and Levins (Reference Puccia and Levins1985, 4), qualitative modeling, “stresses qualitative understanding as the primary goal, rather than numerical prediction.” This prioritizes enhancing understanding over the fact that only quantitative modeling can, for some debates, fully resolve them, as emphasized by Orzack and Sober (Reference Orzack and Sober1993). In contrast, Orzack and Sober (Reference Orzack and Sober1993) seem to assume that enhancing understanding is a secondary objective of modeling. That qualitative models, for instance, “play an important role in modern efforts to understand adaptation” (Orzack and Sober Reference Orzack and Sober1993, 543), which was mentioned but not elucidated, did not mitigate their general criticism that, “the idea of qualitative modeling has hindered the unbiased assessment of the truth” about adaptationism.

The narrow view of scientific modeling that motivates this criticism is indefensible. Loop analysis, for instance, helps pinpoint what model coefficients should be measured to answer specific questions, identifies plausible hypotheses about system behavior and provides reasons to reject others, and enhances understanding of ecological systems with complex dynamics. Besides the discussion in Section 3, two additional examples are illustrative:

1. 1. If a digraph is not sign-stable, the feedback equations (6) specify what interaction coefficients stability depends upon and identify qualitative restrictions on their quantitative values. Since most quantitative values cannot be measured feasibly in complex systems, loop analysis helps focus limited resources on measuring those needed to evaluate system stability.

2. 2. Loop analysis provides a method for determining how equilibrium values of model variables will respond to changed extrinsic conditions (Puccia and Levins Reference Puccia and Levins1985, Chapter 3). For example, if climate change reduces the level of N in Figure 1, loop analysis can determine the conditions under which equilibrium values of P or H will increase or decrease. Sometimes this can then be used to identify correlations between variables without quantitative data (Puccia and Levins Reference Puccia and Levins1985, Chapter 4). If known correlations differ from predicted ones, the model structure can be modified to represent the system more realistically.

Examples of these kinds show how qualitative modeling assists in scientific research by broadly contributing to better representation and understanding of modeled systems. That it has a different but complementary focus compared with quantitative modeling does not indicate conceptual or methodological deficiency.