2 The Cartesian problem of induction

Recall the following instance of full enumerative induction:

Note that, even if the ravens observed in the past, present, and future are all black, it still leaves open whether all ravens are black—the true answer might be “yes” and, unfortunately, it might be “no.” This latter possibility may be called the Cartesian scenario of induction, for it can be materialized dramatically by a Cartesian-like demon who always hides a white raven behind the inquirer. In the other possibility, the inquirer still sees only black ravens without a counterexample—and fortunately all ravens are black. This is the normal counterpart to the Cartesian scenario of induction. Those two scenarios, the Cartesian and the normal, are empirically indistinguishable for the inquirer (but perhaps not for the demon), and the hypothesis “all ravens are black” is underdetermined in that it is true in one of the two scenarios and false in the other. As we will see very soon, that causes trouble for many formal epistemologists, including both Bayesians and learning theorists.

For Bayesians, any justification of full enumerative induction requires justifying a prior probability distribution of credences that disfavors the Cartesian scenario of induction and favors its normal counterpart. To see why, consider the tree depicted in Figure 1. Branches represent possible worlds, or possible histories of inquiry. Moving straight up means observing a black raven; veering to the right means observing a nonblack raven. Let $\mathtt {Yes}$ denote the hypothesis that all ravens are black, which is true in the branch marked with ‘ $\mathtt {Yes}$ ’. Let $\mathtt {No}$ denote the hypothesis that not all ravens are black, which is true in the branches marked with ‘ $\mathtt {No}$ ’. The two vertical branches represent the Cartesian scenario and its normal counterpart, respectively; the Cartesian one is marked with ‘ $\mathtt {No}$ ’ and its normal counterpart is marked with ‘ $\mathtt {Yes}$ ’. Now, construct a Bayesian prior by distributing probabilistic credences $p_1, p_2, \ldots , p_{{\textrm {normal}}}, p_{{\textrm {cartesian}}}$ to the branches in the way depicted in Figure 1, while making sure that those credences are nonnegative and sum to one. With that Bayesian prior, the inquirer updates her credences as follows. When she moves straight up in the tree, she observes more and more black ravens and thereby rules out more and more branches that veer to the right. Then, with successive updates by conditionalization, the ratio of her posterior credence in $\mathtt {Yes}$ to that in $\mathtt {No}$ converges to the prior ratio of $p_{{\textrm {normal}}}$ to $p_{{\textrm {cartesian}}}$ . If this ratio is much greater than 1, the prior is inductive; much smaller than 1, counterinductive; close to 1, skeptical.

Fig. 1 A Bayesian prior over some possibilities.

So, to justify the full version of enumerative induction, a Bayesian needs to provide some reason for having a prior that is inductive rather than counterinductive or skeptical. But Bayesians seldom, if ever, try to do so. Possible reasons are not hard to see. Think about objective Bayesians first, who think that a probabilistic prior is epistemically permissible just in case, roughly, it is flat or somehow flat enough to guard against a certain kind of unwarranted bias (which they specify). Objective Bayesians seem to have never mentioned the Cartesian scenario of induction and the inductive priors; see, for example, [Reference Carnap and Schilpp4, Reference Jaynes19, Reference Williamson43]. It might be simply because they were already too busy with justifying some less-than-full versions of enumerative induction. Or it might be because they did not find a way to justify an inductive prior as a sufficiently flat prior, for a naïve view of sufficient flatness appears to require that the prior ratio of $p_{{\textrm {normal}}}$ to $p_{{\textrm {cartesian}}}$ be close to $1$ , leading to a skeptical prior.

On the other hand, subjective Bayesians (such as [Reference de Finetti6]) would say, roughly, that an inductive prior, if probabilistic, is epistemically permissible—but only because they think that any probabilistic prior is epistemically permissible.Footnote ⁶ So they are committed to the thesis that the counterinductive priors and the skeptical ones are epistemically permissible, too, as long as they are probabilistic. To make this commitment explicit is to invite the usual worry that “anything goes” on the subjective Bayesian account. This worry, which is all too familiar, has already arisen from many other case studies of inductive inferences. So subjective Bayesians seem to have no motivation to mention the case of full enumerative induction because this case does little to assuage (or aggravate) the familiar worry.

Learning theorists fare no better. When the adoption of an inductive principle is justified in formal learning theory, it is typically justified as a necessary condition for achieving a certain epistemic ideal: the ideal of finding the truth at least in the long run (and possibly as fast as possible) in every possible history of inquiry under consideration. This epistemic ideal is often called identification in the limit [Reference Gold12, Reference Putnam35]. When applied to the problem of whether all ravens are black, the ideal of identification in the limit sets an extremely high standard: finding the truth both in the Cartesian scenario and in its normal counterpart. But to find the truth in one of those two scenarios is to fail to do so in the other. So the epistemic standard in question is too high to be achievable and thus too demanding to serve as an evaluative standard. That is, formal learning theorists lose their evaluative standard in this case.

Those seem to be some of the reasons why so little has been done to justify enumerative induction in its full version. So we are still left with the problem of how we may develop an explicit argument for full enumerative induction, against counterinduction, and against the skeptical policy. I call this problem the Cartesian problem of induction, because of the role played by Cartesian scenarios of the sort mentioned above. The point of pursuing this problem is not to respond to every conceivable kind of inductive skeptic. The point is, rather, to push ourselves to the limit—to explore the extent to which we can justify various kinds of inductive inferences with an explicit argument.

That problem might appear to have a simple solution: just adopt the view that scientists should care about, not the colors of all ravens, but only the colors of the ravens that are or will actually be observed by us. This solution looks simple, but are not straightforward to defend. It presupposes a criterion of what scientists should care about, a quite stringent criterion that rules in only the objects that are or will actually be observed. A more lenient criterion rules in all objects that are observable, whether or not they will actually be observed (e.g., [Reference van Fraassen41]).Footnote ⁷ An even more lenient criterion rules in all objects that are physical, whether or not they are observable. It would be interesting to see how the most stringent criterion of the three might be defended in favor of the simple solution to the Cartesian problem of induction. But I believe that, before we can have an overall assessment of the competing solutions, it is important to explore and develop at least one positive solution for those who adopt the intermediate or lenient criterion of what scientists should care about. That is what I set out to do in this paper.

This paper aims to develop the first systematic solution to the Cartesian problem of induction. My proposal, put in a slogan, is that Bayesians go learning-theoretic and learning theorists be truly learning-theoretic. The crux is to revisit the spirit by which learning theory was created in the 1960’s, give it a clear formulation, and use it to explain how normative considerations about possible futures can impose a significant constraint on the short run and even the present, including Bayesian priors.

The next section—Section 3—presents my positive solution to the Cartesian problem of induction, with a focus on the philosophical ideas and a sketch of the supporting theorems. The mathematical details are then developed in Sections 4–8. The Bayesian version of my account is presented in Section 9. To declare the style in use: Emphasis is indicated by italics, while the terms to be defined are presented in boldface.

3 A solution

An empirical problem can be understood to have three components: First, it poses a question with some potential answers as competing hypotheses. Second, it considers some possible bodies of evidence that the inquirer might receive. Third, it comes with a presupposition taken for granted by the inquirer. For example, we have:

Call this empirical problem the hard raven problem.

The hard raven problem can be represented by the tree in Figure 2. The observation of a black raven is represented by a datum $\texttt {+}$ ; a nonblack raven, $\texttt {-}$ ; a nonraven, $\texttt {0}$ . A possible state of the world is represented by an entire branch, which is specified by two items: first, the infinite data stream produced in that state (represented by an infinite sequence of circular nodes); second, the hypothesis true in that state (as indicated in the box). The figure also highlights some Cartesian scenarios of induction. To be sure, each state is associated with an infinite data stream $(e_1, e_2, \ldots , e_n, \ldots )$ , but that does not really represents a state in which the scientist will be immortal—it represents, rather, a state in which the scientist would happen to have evidence $(e_1, e_2, \ldots , e_n)$ if the inquiry were to unfold up to sample size n, for any positive integer n.

Fig. 2 A tree representation of the hard raven problem.

A learning method for the hard raven problem is a mapping that sends each of the finite data sequences under consideration to one of the competing hypotheses, $\mathtt {Yes}$ or $\mathtt {No}$ , or to a question mark. Think of such a learning method as an instruction that receives data and recommends one of the three qualitative attitudes toward the general hypothesis: belief, disbelief, and suspension of judgment. (The account to be presented below has a Bayesian version that allows a learning method to output probabilistic attitudes.)

Which learning methods are the best for tackling the hard raven problem? I would like to address this issue by revisiting the root of learning theory: Putnam’s [Reference Putnam and Schilpp34, Reference Putnam35] and Gold’s [Reference Gold11, Reference Gold12] pioneering works. But before that, I would like to go all the way back to Plato’s Meno.

3.3 Exploring modes of convergence

The hard raven problem is so hard that the ideal of everywhere convergence is even too high to be achievable. This is because, for any learning method, if it enjoys convergence to the truth in a Cartesian scenario of induction, it must fail to do so in its normal, empirically indistinguishable counterpart. With the impossibility to achieve everywhere convergence for the hard raven problem, the existing literature of learning theory stops there. But that seems to give up too soon. If everywhere convergence is unachievable, we should not try to achieve it. We only need to achieve the best we can have and, before that, we need to look for what can be achieved—following the two guidelines formulated above, which is what I meant when I said “be truly learning-theoretic.”

So let’s explore some modes of convergence. Everywhere convergence is only one of the many convergence criteria that concern the question of where convergence happens. We can also consider another question, which concerns how convergence happens. Have a look at Figure 3, in which various modes of convergence to the truth are arranged by two dimensions. The dimension that stretches to the upper right concerns “where.” The other dimension, which stretches to the upper left, concerns “how.” I introduce three modes for each of the two dimensions, so in combination there are nine modes to be considered in this paper.Footnote ¹⁰ Let me explain those modes of convergence in greater detail, focusing on their philosophical interpretations and leaving rigorous definitions to subsequent sections.

Fig. 3 Modes of convergence to the truth, arranged by two dimensions.

Given that no learning method can make it everywhere, let’s see whether there exists a learning method that can at least make it almost everywhere. The challenge is to find a good, rigorous explication of the informal concept of “almost everywhere.” I propose to proceed this way: first motivate a certain closeness relation between states, which defines a concept of open neighborhoods of a state in a state space, which then defines what it is for a region to cover almost everywhere in a state space. To be more specific:

Step 1. Consider an arbitrary state s as a branch depicted on the left side of Figure 4. An alternative state is said to be empirically closer to s if more empirical data are needed to distinguish it from s.

Fig. 4 The empirical topology on a space of data streams or states.

Step 2. Given a space of states, X, an open neighborhood of a state s in X is the set of states in X that are empirically close to s to at least a certain degree, as depicted on the right side of Figure 4.Footnote ¹¹

Step 3. Following the standard treatment in geometry and topology, a region within a space X is called topologically negligible (also called nowhere dense) if it can be constructed by removing an open set within every open neighborhood of every point in space X, and possibly removing more. Intuitively, we can think of a topologically negligible region as a slice of “hyper” Swiss cheese that is incredibly full of open holes. A region is said to cover almost everywhere in X iff it excludes only a topologically negligible region within X.

Now I can define a new epistemic ideal: Say that a learning method M for an empirical problem converges to the truth almost everywhere if M converges to the truth almost everywhere on the space of the states that make h true—for every hypothesis h considered in the problem. The idea is that we hope to make it almost everywhere—whichever hypothesis is true.

Here is another mode of convergence that concerns where convergence happens. Say that a learning method converges to the truth on a maximal domain if there exists no learning method that converges to the truth in the same states and in more states.

Now let’s consider how convergence might happen. Say that a learning method achieves perfectly monotonic convergence to the truth if, whenever it outputs one of the competing hypotheses (rather than suspends judgment), it outputs the truth and would then continue to have the same true output were the inquiry to unfold indefinitely. That sounds like a very high ideal. A lower one is this: a learning method is said to achieve stable convergence to the truth if, whenever it outputs a true competing hypothesis, it would then continue to have the same true output were the inquiry to unfold indefinitely. What if it outputs a falsehood? Stable convergence is silent about that case, while perfectly monotonic convergence rules out that case. So “perfectly monotonic” implies “stable” but not the other way round. To be stable is to be somewhat monotonic but not necessarily perfectly so. Plato would probably love stable convergence to the truth, for it basically requires that, whenever the inquirer forms a belief in the true hypothesis, this belief is not merely a true opinion but has been “stabilized” or “tethered” to the truth, attaining the epistemic status that Plato values in Meno.

Finally, by ‘no requirement’ in Figure 3, I mean no requirement on how to converge.

The above finishes the sketch of the three modes of convergence on each of the two axes in Figure 3. So there are nine “combined” modes of convergence arranged into a two-dimensional lattice structure, in which some modes are ordered higher than some others. Here I adopt the convention of the Hasse diagram, according to which a higher node is one that we can reach by going up along edges without going down. A mode, if ordered higher, is mathematically stronger; it implies all the modes ordered lower in the lattice.

4 Learning theory: The basics

This section reviews some definitions familiar in formal learning theory.

The state space $\mathcal {S}$ captures the presupposition of the problem—it is the set of all possible ways for the inquiry to unfold indefinitely without violating the presupposition.

Definition 4.2. A learning method for a problem ${\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ is a function

$$ \begin{align*}M: \mathcal{E} \to \mathcal{H} \cup \{\texttt{?}\} \,,\end{align*} $$

where $\texttt {?}$ represents suspension of judgment. Given each data sequence $(e_1, \ldots , e_{n}) \in \mathcal {E}$ , the output of M is written $M(e_1, \ldots , e_{n})$ .

Definition 4.5. Let M be a learning method for a problem $\mathcal {P} = {\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ . M is said to converge to the truth in a state $(h, \vec {e}) \in \mathcal {S}$ if

$$ \begin{align*} \lim_{n \to \infty} M(e_1, \ldots, e_{n}) &= h \,, \end{align*} $$

namely, there exists a positive integer k such that, for each $n \ge k$ , we have that $M(e_1, \ldots , e_{n}) = h$ . M is said to converge to the truth everywhere for $\mathcal {P} = {\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ if it converges to the truth in every state in the state space $\mathcal {S}$ .

Then it is routine to prove the following negative result:Footnote ¹⁶

5 A topological conception of “almost everywhere”

When convergence to the truth cannot be achieved everywhere, let’s see whether it can be achieved at least almost everywhere. This section explicates the concept of “almost everywhere” in terms of the concept of open neighborhoods, which is in turn explicated by the concept of closeness.

Consider two data streams $\vec {e}$ and $\vec {e}\,'$ . Suppose that they are identical up until sample size n, that is, $e_i = e^{\prime }_i$ for each $i \le n$ but $e_{n+1} \neq e^{\prime }_{n+1}$ . Note that the larger n is, the later the point of departure is and the more data one needs to distinguish those two data streams. So, the larger n is, the harder it is to empirically distinguish those two data streams, and the closer $\vec {e}$ is to $\vec {e}\,'$ —closer in the empirical sense (recall the left side of Figure 4). Accordingly, we have:

This allows us to define the empirical closeness between states, which can then be used to define open neighborhoods of states, as follows:

It is routine to verify that the empirical topological base of any state space is indeed a topological base as standardly defined.Footnote ¹⁸

Of particular interest is the case in which the set of states in question, X, is the set of states that make a fixed hypothesis true. Consider an arbitrary hypothesis h in a problem $\mathcal {P} = {\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ . Let $|h|$ be the set of states that make h true; or formally:

$$ \begin{align*} |h| &= \{ (h', \vec{e}\,') \in \mathcal{S} : h' = h \}. \end{align*} $$

Let me introduce a convenient way to express the basic open neighborhoods in $|h|$ . Choose a finite data sequence $(e_1, \ldots , e_n)$ . Construct the set of the states in $|h|$ whose data stream extends $(e_1, \ldots , e_n)$ . If this set is nonempty, we have a basic open neighborhood. All basic open neighborhoods in $|h|$ can be constructed that way. More formally, define

$$ \begin{align*} |(e_1, \ldots, e_n)| &= \{ (h', \vec{e}\,') \in \mathcal{S} : \vec{e}\,' \mbox{ extends } (e_1, \ldots, e_n) \}. \end{align*} $$

We can understand $|h|$ and $|(e_1, \ldots , e_n)|$ as the propositions expressed by a hypothesis and a body of evidence, respectively (conditional on the presupposition of the problem).Footnote ¹⁹ It is routine to verify that the basic open neighborhoods in $|h|$ are exactly the nonempty sets taking this form: $|h| \cap |(e_1, \ldots , e_n)|$ , which is the set of the states that make h true and have a data stream that extends $(e_1, \ldots , e_n)$ .

Once we have a criterion of what counts as a basic open neighborhood in a space X, it can be used to define a topological conception of “negligible.” The idea, put informally, is that a subset of a space X is said to be negligible if it can be constructed from what may be called a hyper hole-punching procedure:

1. 1. Start from the entire topological space X in question.

2. 2. For any point $x \in X$ , and any open neighborhood N of x (however small N may be), punch a hole by removing a basic open set within N.

3. 3. If you wish, remove some more points.

Examples of topologically negligible sets include straight lines in the two-dimensional Euclidean space (for a straight line can be constructed by removing an appropriate open disc from every open disc). This idea can be put formally as follows:

With the above definitions, we have:

This is a lemma for proving the main result of this paper. Note that I do not wish to infer from “that is topologically negligible” to “we should ignore that.” To infer that way is to ignore all possible states of the world, because every state forms a singleton that is topologically negligible. And it is absurd to ignore all states. A good justification for full enumerative induction has to be formulated in a more careful way, such as the learning-theoretic argument in Section 3.4.

6 Mode of convergence: “Almost everywhere”

The preceding section is purely explicatory: it provides an explication of the concept of “almost everywhere,” and gives no epistemology at all. Epistemology starts from here: using the above explication of “almost everywhere” to define an epistemic ideal.

I propose this definition as an explication of the (informal) epistemic ideal of convergence to the truth in almost all possible states. Perhaps there are other good explications of “almost all”—they should be explored. If we can develop a better explication than the present one, we should use it instead. But I tend to think that the present proposal is at least a natural starting point. If the present explication turns out to be only one of many good standards of “almost all,” then we should try to see whether it is possible to meet all of those standards simultaneously.Footnote ²¹ The reason is learning-theoretic: let’s achieve them all, if the best possible is to achieve them all.

The above definition makes possible a series of positive results. Here is the first one:

Proof. By Lemma 5.4, the topological space $|\mathtt {No}|$ has the following negligible region:

$$ \begin{align*}C \;=\; \{(\mathtt{No}, \vec{e}): \vec{e} \mbox{ is a}\ \texttt{+}/\texttt{0}\ \mbox{sequence} \},\end{align*} $$

which consists of the Cartesian scenarios of induction. So it suffices to give an example of a learning method that converges to the truth in

every state in $|\mathtt {Yes}|$ and

every state in $|\mathtt {No}| \smallsetminus C$ .

The following learning method does the job:

$M^*$ : “Output hypothesis $\mathtt {Yes}$ if you haven’t seen a nonblack raven ( $\texttt {-}$ ); otherwise output $\mathtt {No}$ .”

This finishes the proof.

It should be noted that the above is only a first step toward justifying full enumerative induction. For there remains a subproblem, which may be called the problem of counterinduction. Let me illustrate with an example.

The above example suggests that, to rule out every counterinductive method, we need to look for additional modes of convergence.

7 Modes of convergence: “Stable” and “maximal”

Before one’s opinion converges to the truth, it might be false, or it might be true but to be retracted as evidence accumulates. But when one’s opinion has converged to the truth, it is “tied” to the truth and will not “run away,” which seems to be epistemically valuable. Hence the following definition:

Definition 7.1. A learning method M is said to have converged to the truth by the n-th stage of inquiry in a state $s = (h, \vec {e})$ if

$$ \begin{align*} M(e_1, \ldots, e_{i}) &= h \quad\mbox{for each}\ i \ge n. \end{align*} $$

Then we can distinguish the following two epistemic ideals:

Stable convergence is sort of monotonic but not necessarily perfectly so, while perfect monotonicity is quite demanding. Indeed, when a learning method for a problem achieves everywhere convergence to the truth with perfect monotonicity, it is basically what a computability theorist would call an effective procedure for solving that problem if we ignore whether the learning method in question is a computable function. That combination of modes, “everywhere” + “perfectly monotonic,” is a great thing to have whenever achievable. But it is too demanding to be achievable for any problem that is inductive in nature, such as the hard raven problem. In fact, we have a stronger negative result:

Here is another desirable mode of convergence:

Then we have:

The idea of proof is explained in Appendix A.1, which I have tried to make as instructive as possible for those new to learning theory. The results of this section are proved in Appendix A.2.

Given Propositions 4.6, 7.3, and 7.5, the first main result follows immediately:

The above result is the mathematical premise of the learning-theoretic argument for full enumerative induction, as formulated above in Section 3.4. The impossibility of everywhere convergence to the truth implies that convergence to the truth has to be sacrificed in some possible states, preferably only on some negligible domain. Some, but which? The above result gives an answer: to achieve all those three modes of convergence to the truth (i)–(iii), we have no alternative but to sacrifice convergence to the truth in exactly the Cartesian scenarios of induction, at least when we are tackling the hard raven problem.

It should be noted that, like many results in learning theory, the above result is sensitive to the problem under discussion. If we turn to a problem that is even harder than the hard raven problem—too hard to allow the achievement of almost everywhere convergence—then the account I propose here will be silent on that problem. See Appendix A.5 for an example. Learning-theoretic epistemology provides evaluations that are sensitive to the empirical problem under discussion.

The rest of this paper is devoted to two interesting extensions of the above account: First, my argument against counterinduction can be generalized to a justification for a kind of Ockham’s razor. Second, every important result in the above can be easily modified to obtain a Bayesian counterpart.

8 “Almost everywhere” + “stable” $\Rightarrow $ “Ockham”

One of the results in the above (clause 1 of Proposition 7.5) has been used to justify a norm that says when to follow this methodological principle:

When to follow that principle? At least when tackling the hard raven problem. This section presents a strengthened result, which can be used to justify a norm that says when one ought to follow a certain version of Ockham’s razor, whose informal idea can be expressed as follows:

Let me start by defining the version of Ockham’s razor I have in mind.

Definition 8.1. Let a problem $\mathcal {P} = {\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ be given. A data sequence $(e_1, \ldots , e_n)$ and a hypothesis h are said to be compatible if the propositions they express have a nonempty overlap, which means that there exists a state in $\mathcal {S}$ that makes hypothesis h true and produces data sequence $(e_1, \ldots , e_n)$ . For each hypothesis $h \in \mathcal {H}$ , let $\mathcal {E} (h)$ denote the set of data sequences in $\mathcal {E}$ that are compatible with h (so $\mathcal {E} (h)$ captures the data-fitting capacity of h). The empirical simplicity order, written $\prec $ , is defined on $\mathcal {H}$ as follows: for all hypotheses h and $h' \in \mathcal {H}$ ,

$$ \begin{align*} h \prec h' & \quad \mbox{iff}\quad \mathcal{E} (h) \subset \mathcal{E} (h'). \end{align*} $$

Or in words, h is simpler then $h'$ iff h “fits” strictly less data sequences than $h'$ does. Say that h is no more complex than $h'$ if $h' \nprec h$ .

In the hard raven problem, for example, the inductive hypothesis $\mathtt {Yes}$ is simpler than the counterinductive hypothesis $\mathtt {No}$ .

The version of Ockham’s razor I have in mind asks one to accept a hypothesis h only if, first, h is no more complex than necessary for fitting the available data and, second, h will continue to be accepted until it is refuted by the accumulated data. More formally, we have:

In the hard raven problem, to comply with the razor condition is exactly to be never counterinductive.

Then we have the second main result, which I call the Ockham stability theorem:Footnote ²⁶

So, given almost everywhere convergence, stable convergence is the weakest normative standard that is strong enough to enforce Ockham’s tenacious razor—strong enough, thanks to the 1 $\Rightarrow $ 2 side; weakest, thanks to the 2 $\Rightarrow $ 1 side.

The 1 $\Rightarrow $ 2 side of the Ockham stability theorem has an application: it can be used to prove, as an immediate corollary, the “never be counterinductive” part of Proposition 7.5. For, when tackling the hard raven problem, to be never counterinductive is exactly to comply with the razor condition.

The result of this section is proved in Appendix A.3.

9 Learning-theoretic Bayes

Instead of tethering one’s qualitative beliefs to the truth, learning-theoretic epistemology can talk about tethering one’s probabilistic degrees of belief to high accuracy. The epistemological idea is the same; it is just a change of the doxastic modeling in use. So it should not be surprising that the important results presented above all have their Bayesian counterparts, as the following shows.

Definition 9.2. A probabilistic prior for a problem $\mathcal {P} = {\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ is a probability function $\mathbb {P}$ defined on $\sigma $ -algebra $\mathcal {A}_{\mathcal {P}}$ with $\mathbb {P}(e_1, \ldots , e_n)> 0$ for each data sequence $(e_1, \ldots , e_n) \in \mathcal {E}$ . $\mathbb {P}$ is said to (have its posteriors) converge to the truth in a state $s = (h, \vec {e}) \in \mathcal {S}$ if

$$ \begin{align*} \lim_{n \to \infty} \mathbb{P}(h {\,|\,} e_1, \ldots, e_{n}) &= 1, \end{align*} $$

that is, for any $\epsilon> 0$ , there exists a positive integer k such that, for each $n \ge k$ , $\mathbb {P}(h {\,|\,} e_1, \ldots , e_{n})> 1-\epsilon $ . $\mathbb {P}$ is said to converge to the truth everywhere for problem ${\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ if it converges to the truth in each state in $\mathcal {S}$ . $\mathbb {P}$ is said to converge to the truth almost everywhere for a problem $\mathcal {P} = {\big ( \mathcal {H}, \mathcal {E}, \mathcal {S} \big )}$ if, for each hypothesis $h \in \mathcal {H}$ , $\mathbb {P}$ converges to the truth almost everywhere on the topological space $|h|$ .

In the above definition of stable convergence, you can replace the occurrence of $1/2$ by any threshold you take to stand for high probability, and do the same for the Bayesian results below. But for concreteness, let me stay with the threshold $1/2$ in this paper.

Although the converse does not hold,Footnote ²⁸ we might be able to formulate a stronger version of tenacity or a weaker version of stability in Bayesian terms in order to restore the equivalence between conditions 1 and 2. But that will not be attempted here. For there is no loss in application to epistemology: to justify Ockham’s razor, what is really needed is just the implication relation from the epistemic ideal expressed by 1 to the methodological principle expressed by 2, which shows that the latter is necessary for achieving the former. The converse does no justificatory work. Showing that Ockham’s razor achieves a certain epistemic ideal does not suffice to argue that one has to follow Ockham’s razor, for there might be other ways to achieve that ideal.

The results of this section are proved in Appendix A.4.