Barbey & Sloman (B&S) disentangle and systematize the various explanations of base-rate neglect/facilitation. They present strong arguments in favor of the hypothesis that the nested subset structure is responsible for facilitation effects. My comments try to further clarify the implications of natural sampling. Throughout the article, the authors adopt the terminology of “natural frequencies” used by Gigerenzer and his group. The adjective “natural” was transferred from “natural sampling.” Let's therefore start with the origin of the latter concept.
The notion “natural sampling” was introduced by Aitchison and Dunsmore (Reference Aitchison and Dunsmore1975) in their excellent book on statistical prediction analysis. In estimating probability parameters, frequencies are informative if and only if they are the outcome of a random sampling process and there is no missing data. Sampling is non-natural if, for example, sample sizes are planned by an experimenter. I used the term “natural sampling” in the Bayesian analysis of binomial sampling (Kleiter Reference Kleiter, Fischer and Laming1994) in the technical sense of Aitchison and Dunsmore. For several Bernoulli processes and beta prior distributions for the binomial probability parameters, natural sampling results in a mathematically nice property: The posterior distribution turns out to be also a beta distribution, and, most important in the present context, the distribution does not depend on the between-group frequencies (base rates) but only on the frequencies within each group (e.g., hit and false alarm rates). As the between-group frequencies are estimators of base-rate probabilities, this result describes a situation in which the base rates are irrelevant. Under properly defined natural sampling conditions, base-rate neglect is rational. The result was generalized to multinomial sampling in combination with Dirichlet priors (Kleiter & Kardinal Reference Kleiter, Kardinal, Mammitzsch and Schneeweiß1995) and used to propagate probabilities in Bayesian networks (Kleiter 1996). Recently, Hooper (Reference Hooper2007) has shown that some claims about the generality of beta posteriors in Bayesian networks made in my 1996 paper are only asymptotically valid.
If base rates are irrelevant in a “normative” model, then base-rate neglect in psychological experiments is not necessarily an error but may be rational. If Bayes' Theorem is written in “frequency format,” even elementary school math shows that the base rates in the numerator and in the denominator get cancelled when the within-group frequencies add up to the between-group frequencies. This property fitted extremely well within Gigerenzer's approach. In the early 1990s when Gerd Gigerenzer was at Salzburg University, during one of the weekly breakfast discussions held among Gigerenzer and members of his group, the mathematical result of base-rate cancellation was communicated and it was immediately taken up and integrated into his work. Natural sampling requires random sampling, additive frequencies in hierarchical tree-like sample/subsample structure (i.e., complete data), and a few more properties that belong to the statistical model. The notion of “natural frequencies” seems, in addition, to involve sequential sampling and thus acquires an evolutionary adaptive connotation.
The additivity in natural sampling goes hand in hand with the subset structure, the favorite explanation in the target article. The close relationship between natural sampling and the subset structure may have led to a confounding of the two in the past. If frequencies (and not subset structures) are the cause of facilitation effects, then critical experiments should investigate non-natural sampling conditions (Kleiter et al. Reference Kleiter, Krebs, Doherty, Gavaran, Chadwick and Brake1997). Frequencies should still have a facilitating effect. Unfortunately, instead of non-natural sampling conditions, often “single-case probabilities” are taken for comparison to demonstrate the base-rate facilitation with natural sampling conditions.
How common are natural sampling conditions in everyday life? I have severe doubts about the ecological validity and the corresponding evolutionary adaptive value. From the perspective of ecological validity, it is important that the base-rate neglect has often been demonstrated for categories with low prevalence, such as rare diseases. Consequently, the prevalence of base-rate neglect will also be low. Base-rate effect certainly depends upon the actual numbers used in the experiments, a property not discussed in B&S's review.
The cognitive system of an intelligent agent capable of uncertainty processing and judgment requires competence in at least six domains. (1) Perception and processing of environmental information, such as numerosity, cardinalities of sets, relative frequencies, descriptive statistics of central tendency, variability, and covariation. (2) Understanding of randomness, of not directly observable states, of alternatives to reality and hidden variables, of the non-uniformities in the environment, and of the limited predictability of events and states. (3) Introspection of one's own knowledge states, and weighting and assessing one's own incomplete knowledge by degrees of beliefs (subjective probabilities). (4) An inference engine that derives conclusions about the uncertainty of a target event from a set of uncertain premises. Typical inference forms are diagnosis, prediction, or explanation. The conclusions often concern single events. The probabilities can be precise or imprecise (lower and upper probabilities, or second order probability distributions). Recently, classical deductive argument forms have also been modeled probabilistically (Oaksford & Chater Reference Oaksford and Chater2007; Pfeifer & Kleiter Reference Pfeifer and Kleiter2005). (5) Modeling functional dependencies/independencies which are basic to causal reasoning. (6) Understanding of the knowledge states of other persons – a prerequisite for the effective communication of uncertainty in social settings.
Many base-rate studies present frequency information (belonging to item [1] in the list given above) and observe whether the subjects use “Bayes' Theorem” as an inference rule (belonging to item [5]). Bayes' Theorem degenerates to a rule for cardinalities, formulated not in terms of probabilities but in terms of frequencies (see Note 2 in the target article). This can of course be done, but we should be aware that we are dealing with the most elementary forms of uncertain reasoning, not involving any of the other items listed above. Moreover, if the response mode requires frequency estimates and not the probabilities of single events, another important aspect of uncertain reasoning is lost. If subjects are poor in the judgment of single event probabilities they have an essential deficit in uncertainty processing.
Conditional events and conditional probabilities are at the very heart of probability theory. Correspondingly, the understanding of conditional events and conditional probabilities should be central to investigations on human uncertain reasoning. Considering base-rate tasks in natural sampling conditions alone, misses this point completely. The B&S structural subset hypothesis shows that conditional probabilities are not needed in this case, and that structural task properties are the main cause of facilitation effects.
