Elqayam & Evans (E&E) bring a timely focus to the numerous pitfalls that can arise in studying human reasoning through a normativist research programme. While recognising many of these pitfalls, we nevertheless contend that normative accounts provide an invaluable reference point for understanding individual differences in performance and need not invoke the prescriptive notion that people “ought” to reason in a particular way. Instead, we argue that normative views can usefully inform theorising about the cognitive processes individuals employ without necessarily limiting the nature or scope of the processes considered. Our recent individual differences research seems to have key points of contact with the descriptivist programme recommended by E&E as a counterpoint to the normativist one. At the same time, there appear to be important points of departure that limit our ability to commit fully to their pure, descriptivist vision. Our commentary aims to explore some key points of alignment and non-alignment.
To begin, we fully agree with E&E that there is a fallacy in inferring that the mere observation of normative responding is diagnostic of underlying analytic processing (i.e., normative responses can be taken neither as necessary nor sufficient in identifying analytic processing). A case in point relates to the phenomenon of belief bias, where, for example, normative responses to syllogisms with valid-believable and invalid-unbelievable conclusions can provide no evidence whatsoever as to whether analytic processing has occurred, because either a heuristic or an analytic process would give rise to an identical (normative) response. Any attempt at identifying analytic strategies within this paradigm would need to focus on invalid-believable and valid-unbelievable conflict problems. With valid-unbelievable items the dominant response is normative conclusion acceptance, despite conclusion unbelievability. This suggests that analytic processing may be occurring with such items, with a key proposal being that reasoners attempt to “disconfirm” unbelievable conclusions by searching for a single, counterexample model (see, e.g., Evans' Selective Processing Model [SPM]; Evans Reference Evans, Garcìa-Madruga, Carriedo and González-Labra2000). Because valid-unbelievable problems have no counterexample models, an analytic process would then lead to normative conclusion acceptance. For invalid-believable problems the dominant response is non-normative conclusion acceptance. The SPM explains this by proposing that reasoners attempt to “confirm” believable conclusions by searching for a single supporting model, which is readily available. Overall, this paradigm suggests a highly nuanced picture whereby normative and non-normative responding may arise from analytic or non-analytic processes.
Despite interpretational difficulties, we suggest that the presence of normative responses with particular problems can provide researchers with a vital stimulus to search for converging evidence for analytic processing, and should not be ignored. For example, according to the SPM, normative responses should never be seen for invalid-believable items; yet they do arise and it seems advisable for researchers to explore their underlying basis. Particularly useful in such research is the acquisition of fine-grained process-tracing evidence such as that derived from think-aloud protocols (Evans et al. Reference Evans, Newstead and Byrne1993; Lucas & Ball Reference Lucas and Ball2005) and inspection-time and eye-tracking methods (Ball et al. Reference Ball, Phillips, Wade and Quayle2006; Stupple & Ball Reference Stupple and Ball2008; Stupple & Waterhouse Reference Stupple and Waterhouse2009). Our recent process-tracing studies of belief bias reveal that individuals who give logical responses to invalid-believable conflict syllogisms process these problems significantly more slowly compared to non-conflict problems and to individuals who respond non-logically. Coupling such evidence with neuroscientific data (De Neys et al. Reference De Neys, Vartanian and Goel2008; Goel & Dolan Reference Goel and Dolan2003) indicating the involvement of the right lateral prefrontal cortex (a region specialised for cognitive monitoring) in normative responding, naturally leads to a view of individuals who respond logically to conflict problems as deploying analytic strategies sensitive to normative considerations. If such evidence additionally correlates with individual-difference measures that are predictive of normative responding (e.g., high working memory capacity, generation of alternatives and need for cognition; Torrens et al. Reference Torrens, Thompson and Cramer1999), then we assert that such triangulation points inescapably to an association between normative responses and analytic processing.
Whether or not this latter research process based around methodological triangulation is descriptivist under E&E's view is, however, unclear, given the guiding role of normative considerations in the whole endeavour – right down to defining responses as “logically correct” or otherwise from the outset. We acknowledge that belief bias research is not immune from issues that pertain to selecting an appropriate normative benchmark. Indeed, Todd and Gigerenzer (Reference Todd and Gigerenzer2000) have argued that it may not be rational to prefer logic to belief. We also agree that it is problematic to equate the appropriate norm with the response given by the most cognitively gifted reasoners (Stanovich & West Reference Stanovich and West2000b). Nevertheless, we see great merit in examining the cognitive processes employed by such gifted individuals. Moreover, De Neys (Reference De Neys2006) has demonstrated that reasoners attempt to adhere to a normative standard irrespective of their cognitive ability, which suggests that these standards hold some genuine descriptive value. Indeed, we have obtained preliminary data indicating that normative training in a matching bias paradigm (as used by Stupple & Waterhouse Reference Stupple and Waterhouse2009) serves to increase logical responding and slow response times. Enhancements in logical responding have also been demonstrated with clarified quantifiers (Schmidt & Thompson Reference Schmidt and Thompson2008) and training in the concept of logical necessity (Prowse-Turner & Thompson Reference Prowse Turner and Thompson2009). These modifications to performance could be construed as merely bringing responses into closer alignment with an existing norm, but we would argue that there is not only merit in exploring methods of enhancing reasoning performance as measured against normative standards, but also in examining the associated cognitive processes and individual differences arising from such methods.
We therefore seek clarification as to where normativism stops and descriptivism starts. For us the situation seems more like a continuum than a dichotomy. We also admit to confusion over E&E's claim that normative standards may be legitimate in a meliorist research programme designed to enhance reasoning skills but are inappropriate as a benchmark in research on underlying cognitive processes. This seems to lead to a conundrum if we wish to examine the underlying cognitive processes arising in response to the interventions deployed in meliorist research. If post-intervention responses align with normative ones, then how are we to go about understanding such changes unless we accept that they reflect normative principles?
Elqayam & Evans (E&E) bring a timely focus to the numerous pitfalls that can arise in studying human reasoning through a normativist research programme. While recognising many of these pitfalls, we nevertheless contend that normative accounts provide an invaluable reference point for understanding individual differences in performance and need not invoke the prescriptive notion that people “ought” to reason in a particular way. Instead, we argue that normative views can usefully inform theorising about the cognitive processes individuals employ without necessarily limiting the nature or scope of the processes considered. Our recent individual differences research seems to have key points of contact with the descriptivist programme recommended by E&E as a counterpoint to the normativist one. At the same time, there appear to be important points of departure that limit our ability to commit fully to their pure, descriptivist vision. Our commentary aims to explore some key points of alignment and non-alignment.
To begin, we fully agree with E&E that there is a fallacy in inferring that the mere observation of normative responding is diagnostic of underlying analytic processing (i.e., normative responses can be taken neither as necessary nor sufficient in identifying analytic processing). A case in point relates to the phenomenon of belief bias, where, for example, normative responses to syllogisms with valid-believable and invalid-unbelievable conclusions can provide no evidence whatsoever as to whether analytic processing has occurred, because either a heuristic or an analytic process would give rise to an identical (normative) response. Any attempt at identifying analytic strategies within this paradigm would need to focus on invalid-believable and valid-unbelievable conflict problems. With valid-unbelievable items the dominant response is normative conclusion acceptance, despite conclusion unbelievability. This suggests that analytic processing may be occurring with such items, with a key proposal being that reasoners attempt to “disconfirm” unbelievable conclusions by searching for a single, counterexample model (see, e.g., Evans' Selective Processing Model [SPM]; Evans Reference Evans, Garcìa-Madruga, Carriedo and González-Labra2000). Because valid-unbelievable problems have no counterexample models, an analytic process would then lead to normative conclusion acceptance. For invalid-believable problems the dominant response is non-normative conclusion acceptance. The SPM explains this by proposing that reasoners attempt to “confirm” believable conclusions by searching for a single supporting model, which is readily available. Overall, this paradigm suggests a highly nuanced picture whereby normative and non-normative responding may arise from analytic or non-analytic processes.
Despite interpretational difficulties, we suggest that the presence of normative responses with particular problems can provide researchers with a vital stimulus to search for converging evidence for analytic processing, and should not be ignored. For example, according to the SPM, normative responses should never be seen for invalid-believable items; yet they do arise and it seems advisable for researchers to explore their underlying basis. Particularly useful in such research is the acquisition of fine-grained process-tracing evidence such as that derived from think-aloud protocols (Evans et al. Reference Evans, Newstead and Byrne1993; Lucas & Ball Reference Lucas and Ball2005) and inspection-time and eye-tracking methods (Ball et al. Reference Ball, Phillips, Wade and Quayle2006; Stupple & Ball Reference Stupple and Ball2008; Stupple & Waterhouse Reference Stupple and Waterhouse2009). Our recent process-tracing studies of belief bias reveal that individuals who give logical responses to invalid-believable conflict syllogisms process these problems significantly more slowly compared to non-conflict problems and to individuals who respond non-logically. Coupling such evidence with neuroscientific data (De Neys et al. Reference De Neys, Vartanian and Goel2008; Goel & Dolan Reference Goel and Dolan2003) indicating the involvement of the right lateral prefrontal cortex (a region specialised for cognitive monitoring) in normative responding, naturally leads to a view of individuals who respond logically to conflict problems as deploying analytic strategies sensitive to normative considerations. If such evidence additionally correlates with individual-difference measures that are predictive of normative responding (e.g., high working memory capacity, generation of alternatives and need for cognition; Torrens et al. Reference Torrens, Thompson and Cramer1999), then we assert that such triangulation points inescapably to an association between normative responses and analytic processing.
Whether or not this latter research process based around methodological triangulation is descriptivist under E&E's view is, however, unclear, given the guiding role of normative considerations in the whole endeavour – right down to defining responses as “logically correct” or otherwise from the outset. We acknowledge that belief bias research is not immune from issues that pertain to selecting an appropriate normative benchmark. Indeed, Todd and Gigerenzer (Reference Todd and Gigerenzer2000) have argued that it may not be rational to prefer logic to belief. We also agree that it is problematic to equate the appropriate norm with the response given by the most cognitively gifted reasoners (Stanovich & West Reference Stanovich and West2000b). Nevertheless, we see great merit in examining the cognitive processes employed by such gifted individuals. Moreover, De Neys (Reference De Neys2006) has demonstrated that reasoners attempt to adhere to a normative standard irrespective of their cognitive ability, which suggests that these standards hold some genuine descriptive value. Indeed, we have obtained preliminary data indicating that normative training in a matching bias paradigm (as used by Stupple & Waterhouse Reference Stupple and Waterhouse2009) serves to increase logical responding and slow response times. Enhancements in logical responding have also been demonstrated with clarified quantifiers (Schmidt & Thompson Reference Schmidt and Thompson2008) and training in the concept of logical necessity (Prowse-Turner & Thompson Reference Prowse Turner and Thompson2009). These modifications to performance could be construed as merely bringing responses into closer alignment with an existing norm, but we would argue that there is not only merit in exploring methods of enhancing reasoning performance as measured against normative standards, but also in examining the associated cognitive processes and individual differences arising from such methods.
We therefore seek clarification as to where normativism stops and descriptivism starts. For us the situation seems more like a continuum than a dichotomy. We also admit to confusion over E&E's claim that normative standards may be legitimate in a meliorist research programme designed to enhance reasoning skills but are inappropriate as a benchmark in research on underlying cognitive processes. This seems to lead to a conundrum if we wish to examine the underlying cognitive processes arising in response to the interventions deployed in meliorist research. If post-intervention responses align with normative ones, then how are we to go about understanding such changes unless we accept that they reflect normative principles?