Lievens and Motowidlo (Reference Lievens and Motowidlo2016) argue compellingly that situational judgment tests (SJTs) measure job-relevant general domain knowledge, conceptualized as implicit trait policies (ITPs). ITPs are defined as a person's knowledge about the utility of expressing certain traits. They develop through the feedback a person receives when acting in accordance with their trait profiles in different environments (work, life, leisure). Positive feedback reinforces the knowledge that behavior in accordance with one's own traits is appropriate, and negative feedback reinforces the knowledge that an approach that differs from one's trait tendencies may be more effective. As such, ITPs represent a person's knowledge about the effectiveness of behaviors across a variety of contexts.
Job knowledge has been recognized as an important determinant of job performance throughout the history of industrial and organizational (I-O) psychology, and because it is more proximal to the performance context, it generally accounts for more variance in performance than cognitive ability (Schmidt & Hunter, Reference Schmidt and Hunter1998). Indeed, more than 30 years ago, Hunter (Reference Hunter1986) described how the effect of ability on job performance is mediated by job knowledge. Sadly, the common wisdom until recently was that knowledge tests are not useful in selection because applicants generally do not possess relevant job knowledge (Schmidt & Hunter, Reference Schmidt and Hunter1998). The promise of SJTs for measuring domain-general job knowledge—general job-relevant knowledge that even novices possess—makes these assessments an exciting area for future research in selection (Motowidlo & Beier, Reference Motowidlo and Beier2010). But even with their promise in the practice of selection, questions remain about the place of ITPs measured by SJTs in the nomological net of knowledge constructs. The purpose of this commentary is to highlight what we see as an important area of future research—placing ITPs in a nomological net of knowledge constructs.
A taxonomy of job-relevant knowledge can be described as a two-by-two matrix, which crosses procedural and declarative type knowledge with the content of this knowledge being either task or contextual (see Table 1, based on Beier, Young, & Villado, in press). From the description of SJTs and relevant ITPs described by Lievens and Motowidlo, SJTs fit best under procedural knowledge related to contextual performance (the lower left quadrant in the table). Although one can imagine an SJT developed to measure task-specific knowledge (e.g., presenting a situation in which an engine is in need of repair to assess mechanical knowledge), we focus our discussion here on SJTs relevant to procedural knowledge in the contextual domain because ITPs have been discussed mostly as relevant to this area. In addition to ITPs, additional knowledge constructs occupy the lower left quadrant, namely, tacit knowledge (Sternberg & Wagner, Reference Sternberg and Wagner1993) and emotional intelligence (Joseph & Newman, Reference Joseph and Newman2010). There may be additional constructs related to ITPs that would also occupy this space, but to make our point succinctly, we limit our discussion to these constructs.
Table 1. Taxonomy of Job-Relevant Knowledge
Tacit Knowledge
Tacit knowledge was advanced by Sternberg and Wagner (Reference Sternberg and Wagner1993) as a form of practical intelligence that resembles domain-general knowledge. For instance, Sternberg and Wagner describe tacit knowledge as “the practical know-how one needs for success on the job” (Sternberg & Wagner, Reference Sternberg and Wagner1993, p. 2). Moreover, like the development of ITPs, tacit knowledge is thought to develop through life experience, in particular, through adaptive responses to the myriad situations encountered throughout the lifespan (Stemler & Sternberg, Reference Stemler, Sternberg, Weekley and Ployhart2006).
Currently it is unclear how the construct of tacit knowledge differs from the type of knowledge that develops from ITPs. Perhaps tacit knowledge is a more general construct. One might consider, for instance, that ITPs represent a specific type of tacit knowledge related to personality trait expression. In the introduction of the tacit-knowledge construct 20 years ago, however, Sternberg and Wagner (Reference Sternberg and Wagner1993) discussed the importance and development of work-specific tacit knowledge and described tacit knowledge as partly job general rather than job specific. They stated, “Tacit knowledge increases, on average, with job experience, but is not a direct function of job experience” (Sternberg & Wagner, Reference Sternberg and Wagner1993, p. 3). Moreover, measurement of tacit knowledge is generally done through situational judgment (Stemler & Sternberg, Reference Stemler, Sternberg, Weekley and Ployhart2006), and validities for tacit knowledge and job performance are typically aligned with those of SJTs and job performance (i.e., correlations in the .20 to .40 range; Sternberg, Wagner, Williams, & Horvath, Reference Sternberg, Wagner, Williams and Horvath1995). In summary, tacit knowledge and ITPs are both measured with SJTs, and they both show the same range of validity coefficients for predicting job performance. Although theory about ITP development is perhaps more robust than theories of tacit-knowledge development, which describe the latter as a function of experience (Sternberg & Wagner, Reference Sternberg and Wagner1993), a more inclusive theory and more research is needed to examine how these constructs overlap and are related to the nomological net of general domain knowledge.
Emotional Intelligence
Ability models of emotional intelligence typically describe it as a person's ability to perceive, understand, and regulate emotion (Joseph & Newman, Reference Joseph and Newman2010). Most models of emotional intelligence do not address how this ability develops. Nonetheless, emotional intelligence can easily be conceived of as a type of knowledge that—similar to knowledge related to ITPs—develops through the feedback a person receives related to their interpersonal interactions in their environment (at work or in life more generally). For instance, Joseph and Newman (Reference Joseph and Newman2010) posit that facets of emotional intelligence “can be conceptualized as accumulated knowledge structures” (Joseph & Newman, Reference Joseph and Newman2010, p. 57), and they state that the “ability to understand emotion represents a body of knowledge concerning which emotions are appropriate in a given context” (Joseph & Newman, Reference Joseph and Newman2010, p. 59). Moreover, like assessments of tacit knowledge, most ability-based measures of emotional intelligence are essentially SJTs that can be developed by asking subject matter experts to describe situations in which they have observed particularly effective or ineffective behavior related to managing and understanding others’ emotions. One publicly available example of SJT items designed to assess emotional intelligence is found in the Situational Tests of Emotion Management (STEM) and Situational Test of Emotion Understanding (STEU; MacCann & Roberts, Reference MacCann and Roberts2008). Although the overall correlation between emotional intelligence and job performance tends to be smaller in magnitude than overall SJT–job performance relations (McDaniel, Hartman, Whetzel, & Grubb, Reference McDaniel, Hartman, Whetzel and Grubb2007; Van Rooy & Viswesvaran, Reference Van Rooy and Viswesvaran2004), both correlations represent small to moderate effects. As with tacit knowledge, understanding how ITPs measured by SJTs compare with emotional intelligence constructs both empirically and theoretically is an area ripe for research.
In conclusion, we applaud Lievens and Motowidlo for focusing on examining why SJTs are powerful predictors of job performance and for highlighting the importance of domain-general job knowledge in I-O psychology generally and in selection contexts more specifically. We hope that our commentary piques the interest of researchers for examining knowledge of all types and for further developing the nomological net of job-related knowledge.
Lievens and Motowidlo (Reference Lievens and Motowidlo2016) argue compellingly that situational judgment tests (SJTs) measure job-relevant general domain knowledge, conceptualized as implicit trait policies (ITPs). ITPs are defined as a person's knowledge about the utility of expressing certain traits. They develop through the feedback a person receives when acting in accordance with their trait profiles in different environments (work, life, leisure). Positive feedback reinforces the knowledge that behavior in accordance with one's own traits is appropriate, and negative feedback reinforces the knowledge that an approach that differs from one's trait tendencies may be more effective. As such, ITPs represent a person's knowledge about the effectiveness of behaviors across a variety of contexts.
Job knowledge has been recognized as an important determinant of job performance throughout the history of industrial and organizational (I-O) psychology, and because it is more proximal to the performance context, it generally accounts for more variance in performance than cognitive ability (Schmidt & Hunter, Reference Schmidt and Hunter1998). Indeed, more than 30 years ago, Hunter (Reference Hunter1986) described how the effect of ability on job performance is mediated by job knowledge. Sadly, the common wisdom until recently was that knowledge tests are not useful in selection because applicants generally do not possess relevant job knowledge (Schmidt & Hunter, Reference Schmidt and Hunter1998). The promise of SJTs for measuring domain-general job knowledge—general job-relevant knowledge that even novices possess—makes these assessments an exciting area for future research in selection (Motowidlo & Beier, Reference Motowidlo and Beier2010). But even with their promise in the practice of selection, questions remain about the place of ITPs measured by SJTs in the nomological net of knowledge constructs. The purpose of this commentary is to highlight what we see as an important area of future research—placing ITPs in a nomological net of knowledge constructs.
A taxonomy of job-relevant knowledge can be described as a two-by-two matrix, which crosses procedural and declarative type knowledge with the content of this knowledge being either task or contextual (see Table 1, based on Beier, Young, & Villado, in press). From the description of SJTs and relevant ITPs described by Lievens and Motowidlo, SJTs fit best under procedural knowledge related to contextual performance (the lower left quadrant in the table). Although one can imagine an SJT developed to measure task-specific knowledge (e.g., presenting a situation in which an engine is in need of repair to assess mechanical knowledge), we focus our discussion here on SJTs relevant to procedural knowledge in the contextual domain because ITPs have been discussed mostly as relevant to this area. In addition to ITPs, additional knowledge constructs occupy the lower left quadrant, namely, tacit knowledge (Sternberg & Wagner, Reference Sternberg and Wagner1993) and emotional intelligence (Joseph & Newman, Reference Joseph and Newman2010). There may be additional constructs related to ITPs that would also occupy this space, but to make our point succinctly, we limit our discussion to these constructs.
Table 1. Taxonomy of Job-Relevant Knowledge
Note. Implicit trait policies are most relevant to the lower left quadrant (procedural knowledge related to contextual performance), bolded in the table.
Tacit Knowledge
Tacit knowledge was advanced by Sternberg and Wagner (Reference Sternberg and Wagner1993) as a form of practical intelligence that resembles domain-general knowledge. For instance, Sternberg and Wagner describe tacit knowledge as “the practical know-how one needs for success on the job” (Sternberg & Wagner, Reference Sternberg and Wagner1993, p. 2). Moreover, like the development of ITPs, tacit knowledge is thought to develop through life experience, in particular, through adaptive responses to the myriad situations encountered throughout the lifespan (Stemler & Sternberg, Reference Stemler, Sternberg, Weekley and Ployhart2006).
Currently it is unclear how the construct of tacit knowledge differs from the type of knowledge that develops from ITPs. Perhaps tacit knowledge is a more general construct. One might consider, for instance, that ITPs represent a specific type of tacit knowledge related to personality trait expression. In the introduction of the tacit-knowledge construct 20 years ago, however, Sternberg and Wagner (Reference Sternberg and Wagner1993) discussed the importance and development of work-specific tacit knowledge and described tacit knowledge as partly job general rather than job specific. They stated, “Tacit knowledge increases, on average, with job experience, but is not a direct function of job experience” (Sternberg & Wagner, Reference Sternberg and Wagner1993, p. 3). Moreover, measurement of tacit knowledge is generally done through situational judgment (Stemler & Sternberg, Reference Stemler, Sternberg, Weekley and Ployhart2006), and validities for tacit knowledge and job performance are typically aligned with those of SJTs and job performance (i.e., correlations in the .20 to .40 range; Sternberg, Wagner, Williams, & Horvath, Reference Sternberg, Wagner, Williams and Horvath1995). In summary, tacit knowledge and ITPs are both measured with SJTs, and they both show the same range of validity coefficients for predicting job performance. Although theory about ITP development is perhaps more robust than theories of tacit-knowledge development, which describe the latter as a function of experience (Sternberg & Wagner, Reference Sternberg and Wagner1993), a more inclusive theory and more research is needed to examine how these constructs overlap and are related to the nomological net of general domain knowledge.
Emotional Intelligence
Ability models of emotional intelligence typically describe it as a person's ability to perceive, understand, and regulate emotion (Joseph & Newman, Reference Joseph and Newman2010). Most models of emotional intelligence do not address how this ability develops. Nonetheless, emotional intelligence can easily be conceived of as a type of knowledge that—similar to knowledge related to ITPs—develops through the feedback a person receives related to their interpersonal interactions in their environment (at work or in life more generally). For instance, Joseph and Newman (Reference Joseph and Newman2010) posit that facets of emotional intelligence “can be conceptualized as accumulated knowledge structures” (Joseph & Newman, Reference Joseph and Newman2010, p. 57), and they state that the “ability to understand emotion represents a body of knowledge concerning which emotions are appropriate in a given context” (Joseph & Newman, Reference Joseph and Newman2010, p. 59). Moreover, like assessments of tacit knowledge, most ability-based measures of emotional intelligence are essentially SJTs that can be developed by asking subject matter experts to describe situations in which they have observed particularly effective or ineffective behavior related to managing and understanding others’ emotions. One publicly available example of SJT items designed to assess emotional intelligence is found in the Situational Tests of Emotion Management (STEM) and Situational Test of Emotion Understanding (STEU; MacCann & Roberts, Reference MacCann and Roberts2008). Although the overall correlation between emotional intelligence and job performance tends to be smaller in magnitude than overall SJT–job performance relations (McDaniel, Hartman, Whetzel, & Grubb, Reference McDaniel, Hartman, Whetzel and Grubb2007; Van Rooy & Viswesvaran, Reference Van Rooy and Viswesvaran2004), both correlations represent small to moderate effects. As with tacit knowledge, understanding how ITPs measured by SJTs compare with emotional intelligence constructs both empirically and theoretically is an area ripe for research.
In conclusion, we applaud Lievens and Motowidlo for focusing on examining why SJTs are powerful predictors of job performance and for highlighting the importance of domain-general job knowledge in I-O psychology generally and in selection contexts more specifically. We hope that our commentary piques the interest of researchers for examining knowledge of all types and for further developing the nomological net of job-related knowledge.