Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-02-06T15:44:58.824Z Has data issue: false hasContentIssue false
  • English
  • Français

In Defense of the Situation: An Interactionist Explanation for Performance on Situational Judgment Tests

Published online by Cambridge University Press:  23 March 2016

Alexandra M. Harris ,
Lane E. Siedor ,
Yi Fan ,
Benjamin Listyg  and
Nathan T. Carter
Alexandra M. Harris*
Affiliation:
Department of Psychology, University of Georgia
Lane E. Siedor
Affiliation:
Department of Psychology, University of Georgia
Yi Fan
Affiliation:
Department of Psychology, University of Georgia
Benjamin Listyg
Affiliation:
Department of Psychology, University of Georgia
Nathan T. Carter
Affiliation:
Department of Psychology, University of Georgia
*
Correspondence concerning this article should be addressed to Alexandra M. Harris, Department of Psychology, University of Georgia, Athens, GA 30602. E-mail: alexandra.harris25@uga.edu
Rights & Permissions [Opens in a new window]

Extract

Whereas Lievens and Motowidlo (2016) propose a model of situational judgment test (SJT) performance that removes the “situation” in favor of conceptualizing SJTs as a measure of general domain knowledge, we argue that the expression of general domain knowledge is in fact contingent on situational judgment. As we explain, the evidence cited by Lievens and Motowidlo against a situational component does not inherently exclude the importance of situations from SJTs and does overlook the strong support for a person–situation interaction explanation of behavior. Based on the interactionist literature—in particular, the trait activation theory (TAT) and situational strength literatures—we propose a model that both maintains the key pathways and definitions posited by Lievens and Motowidlo and integrates the situational component of SJTs.

Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 9 , Issue 1 , March 2016 , pp. 23 - 28
Copyright
Copyright © Society for Industrial and Organizational Psychology 2016 

Whereas Lievens and Motowidlo (Reference Lievens and Motowidlo2016) propose a model of situational judgment test (SJT) performance that removes the “situation” in favor of conceptualizing SJTs as a measure of general domain knowledge, we argue that the expression of general domain knowledge is in fact contingent on situational judgment. As we explain, the evidence cited by Lievens and Motowidlo against a situational component does not inherently exclude the importance of situations from SJTs and does overlook the strong support for a person–situation interaction explanation of behavior. Based on the interactionist literature—in particular, the trait activation theory (TAT) and situational strength literatures—we propose a model that both maintains the key pathways and definitions posited by Lievens and Motowidlo and integrates the situational component of SJTs.

Interactionist explanations of work behavior have received increasing attention and support in the employee selection literature and stem from a long history of research on person–situation models of personality (e.g., Mischel, Reference Mischel1968). The ability to evaluate situational demands predicts performance across assessment types, including structured interviews (Melchers, Bösser, Hartstein, & Kleinmann, Reference Melchers, Bösser, Hartstein and Kleinmann2012) and assessment centers (Jansen et al., Reference Jansen, Melchers, Lievens, Kleinmann, Brändli, Fraefel and König2013). Further, the ability to identify criteria for performance evaluation (broadly conceptualized as situational cues) has been posited as a key explanation of the criterion-related validity for selection assessments (Kleinmann et al., Reference Kleinmann, Ingold, Lievens, Jansen, Melchers and König2011). Given the evidence supporting a person–situation interaction account of performance in assessment centers and structured interviews, it is reasonable to posit that situations also play a significant role in SJTs.

Two recent studies provide the foundation for Lievens and Motowidlo's removal of situations from their SJT performance model. First, Krumm et al. (Reference Krumm, Lievens, Hüffmeier, Lipnevich, Bendels and Hertel2015) suggested that removal of situational prompts from SJT items necessarily eliminates all situational descriptors in the item. We disagree with this conclusion and argue that both the broader testing scenario (e.g., applying for a particular job) and the content of SJT response options retain situational descriptors. In an example item given by Krumm et al., two of the response options are “(a) Have your supervisor decide, because this would avoid any personal bias” and “(b) Arrange for a rotating schedule so everyone shares the chore” (2015, p. 415). Even with the removal of the item prompt, the responses communicate a situation in which the participant must decide how to delegate a chore among coworkers. Thus, we question the degree to which these results support a situation-free model of SJT performance. Second, although Rockstuhl, Ang, Ng, Lievens, and Van Dyne (Reference Rockstuhl, Ang, Ng, Lievens and Van Dyne2015) may suggest that SJTs do not currently assess situational judgment sufficiently, their findings are presented as supportive of a situational judgment component of SJTs. They also show that those SJTs that explicitly tap into such judgment have incremental validity over other forms of SJTs. Consequently, we argue that exclusion of situational factors in models of SJT performance discounts the otherwise overwhelming support for a person–situation interaction in explaining performance on selection assessments.

Recently, the literature regarding the person–situation model for personality has explored the model of TAT (Tett & Burnett, Reference Tett and Burnett2003; Tett & Guterman, Reference Tett and Guterman2000), which proposes that a trait will only be expressed to the extent that a trait-relevant situational cue calls for it. Tett and Burnett (Reference Tett and Burnett2003) refer to such a cue as a “demand” or an opportunity to express a trait that is valued in the given situation. Lievens and Motowidlo assert that general domain knowledge is “knowledge about the utility of expressing various traits” and “reflects effects of fundamental socialization processes and personal dispositions” (p. 5). We agree with this definition. However, we argue that TAT posits a situationally moderated relationship between personality and performance consistent with general domain knowledge as defined by Lievens and Motowidlo. That is, individuals high in general domain knowledge accurately apply implicit trait policies (ITPs) to evaluate the effectiveness of a particular response. In terms of TAT, individuals high in general domain knowledge will express the most valued trait-relevant behavior given the available situational cue. Based on the congruence of general domain knowledge with TAT, we propose a model of SJT performance that incorporates situational cues.

A Situational Model of SJT Performance

Our model (see Figure 1) extends TAT to include the crucial mechanisms pointed to by Lievens and Motowidlo. Paths 1 and 2 represent the TAT foundations of our model, indicating that personality trait levels determine SJT performance to the extent that situational cues activate or restrict trait expression. The strength of the moderating effect of situational cues on the relationship between personality and performance is in part determined by the accurate application of the ITP (Path 3), representing the expression of general domain knowledge as indicated by a star in Figure 1. The relationship between personality and the situation is also a function of the accurate application of specific job knowledge (Path 4) given the situation. In the absence of situational cues, individuals would not be able to draw on the appropriate ITP or job knowledge required for effective performance. That is, SJT performance is increasingly a function of personality traits (reflecting the idea of personality as a generalized representation of how persons behave across situations; McCrae, Reference McCrae2001) to the extent that (a) an SJT does not convey situational information, (b) the ITP is applied improperly, and/or (c) job knowledge is applied improperly.

Figure 1. Model of situational judgment test (SJT) performance determinants utilizing trait activation theory to explain the moderating effects of situational cues. Pathways connecting personality trait to general experience and personality trait to specific job experience are assumed but have been removed for the sake of clarity.

Notably, apart from the above incorporation of situational cues, all other pathways and processes in our model match Lievens and Motowidlo. Paths 5, 6, and 7 show that personality trait levels, general experience, and general mental ability combine to determine ITPs regarding trait effectiveness. Therefore, the developmental socialization process of general domain knowledge as represented in our model is consistent with Lievens and Motowidlo. Finally, paths 8, 9, and 10 show that specific job knowledge is a function of ITP, general mental ability, and specific job experience, respectively. Thus, specific job knowledge and ITPs function identically in terms of both their developmental processes and their dependence on situational cues to exert influence on SJT performance.

The Role of Situational Strength

One of the most explored extensions of TAT research is the concept of situational strength (Tett, Simonet, Walser, & Brown, Reference Tett, Simonet, Walser, Brown, Christiansen and Tett2013), which we believe further supports the role of situational cues in moderating the relationship between personality and SJT performance. Situational strength can be broken into four facets that together compose the force of a situation on behavior (Meyer, Dalal, & Hermida, Reference Meyer, Dalal and Hermida2010). As we explain below, the most salient to our model is clarity, which suggests that ambiguous situations exert almost no influence on trait expression, resulting in individuals behaving in the way most consistent with their personalities.

Recently, Judge and Zapata (Reference Judge and Zapata2015) integrated TAT and situational strength perspectives, showing that both the trait relevance and strength of situations impact the personality–performance relationship. Importantly, our model invokes both concepts in the broad “situational cues” term and is consistent with the integrated model of trait activation and situational strength. Perhaps most crucially, Judge and Zapata note that “individuals in trait-relevant situations likely realize that their innate tendencies are beneficial” (p. 1153). To the degree that persons recognize the benefits and drawbacks of their innate tendencies, their ITPs will be correct, reflecting high standing on general domain knowledge. However, there must also be some degree of clarity regarding the valued behavior in a situation for persons to have the opportunity to express their general domain knowledge. Thus, we argue that moderately strong situational cues are required for expression of general domain knowledge.

We hold that the necessity of situational strength for the expression of general domain knowledge is supported rather than disconfirmed by the studies cited by Lievens and Motowidlo. Consistency in SJT performance for versions with and without prompts (Krumm et al., Reference Krumm, Lievens, Hüffmeier, Lipnevich, Bendels and Hertel2015) might reflect the previously noted issue that situational cues are still present and clear in the response options of SJTs, such that the two versions do not reflect a substantial difference in situational strength (i.e., general domain knowledge is important in both measures). This interpretation of the role of situational strength is consistent with Rockstuhl et al.’s (Reference Rockstuhl, Ang, Ng, Lievens and Van Dyne2015) finding that SJTs that explicitly ask respondents to judge the situation provide incremental validity over SJTs that do not, as well as consistent with studies showing the importance of situational cues in other assessment formats (Jansen et al., Reference Jansen, Melchers, Lievens, Kleinmann, Brändli, Fraefel and König2013; Melchers et al., Reference Melchers, Bösser, Hartstein and Kleinmann2012). Further, we suggest that all SJTs invoke situational cues that are stronger than in self-report personality tests, which may account for the incremental validity of SJTs above and beyond personality tests (Meriac, Hoffman, Woehr, & Fleisher, Reference Meriac, Hoffman, Woehr and Fleisher2008). Finally, our application of an interactionist perspective is consistent with past successful applications of the interactionist perspective of TAT to improve assessment centers (Haaland & Christiansen, Reference Haaland and Christiansen2002; Lievens, Chasteen, Day, & Christiansen, Reference Lievens, Chasteen, Day and Christiansen2006; Lievens, Schollaert, & Keen, Reference Lievens, Schollaert and Keen2015). Research should similarly seek to embrace an interactionist perspective to the benefit of SJTs.

Conclusion

Lievens and Motowidlo propose reformulating the measurement of situational judgment as the measurement of general domain knowledge; we see this proposal as a paradox. On the basis of existing theoretical perspectives on interactionist explanations of behavior and empirical research confirming the importance of situational strength in other assessment forms, we suggest that an individual's ability to demonstrate general domain knowledge is contingent on the use of situational cues to differentiate persons in their ability to identify and express the most valued trait. Thus, if general domain knowledge does indeed represent the new frontier in SJTs, the consideration of situational cues and their strength will be all the more important and should not be ignored.

References

Haaland, S., & Christiansen, N. D. (2002). Implications of trait-activation theory for evaluating the construct validity of assessment center ratings. Personnel Psychology, 55, 137–163. doi:10.1111/j.1744-6570.2002.tb00106.x CrossRefGoogle Scholar
Jansen, A., Melchers, K. G., Lievens, F., Kleinmann, M., Brändli, M., Fraefel, L., & König, C. J. (2013). Situation assessment as an ignored factor in the behavioral consistency paradigm underlying the validity of personnel selection procedures. Journal of Applied Psychology, 98, 326–341. doi:10.1037/a0031257 CrossRefGoogle ScholarPubMed
Judge, T. A., & Zapata, C. P. (2015). The person-situation debate revisited: Effect of situation strength and trait activation on the validity of the Big Five personality traits in predicting job performance. Academy of Management Journal, 58, 1149–1179. doi:10.5465/amj.2010.0837 CrossRefGoogle Scholar
Kleinmann, M., Ingold, P., Lievens, F., Jansen, A., Melchers, K., & König, C. (2011). A different look at why selection procedures work: The role of candidates’ ability to identify criteria. Organizational Psychology Review, 1, 128–146. doi:10.1177/2041386610387000 CrossRefGoogle Scholar
Krumm, S., Lievens, F., Hüffmeier, J., Lipnevich, A. A., Bendels, H., & Hertel, G. (2015). How “situational” is judgment in situational judgment tests? Journal of Applied Psychology, 100, 399–416. doi:10.1037/a0037674 CrossRefGoogle Scholar
Lievens, F., Chasteen, C. S., Day, E. A., & Christiansen, N. D. (2006). Large-scale investigation of the role of trait activation theory for understanding assessment center convergent and discriminant validity. Journal of Applied Psychology, 91, 247–258. doi:10.1037/0021-9010.91.2.247 CrossRefGoogle ScholarPubMed
Lievens, F., & Motowidlo, S. J. (2016). Situational judgment tests: From measures of situational judgment to measures of general domain knowledge. Industrial and Organizational Psychology: Perspectives on Science and Practice, 9, 3–22.CrossRefGoogle Scholar
Lievens, F., Schollaert, E., & Keen, G. (2015). The interplay of elicitation and evaluation of trait-expressive behavior: Evidence in assessment center exercises. Journal of Applied Psychology, 100, 1169–1188. doi:10.1037/apl0000004 CrossRefGoogle ScholarPubMed
McCrae, R. R. (2001). 5 years of progress: A reply to Block. Journal of Research in Personality, 35, 108–113. doi:10.1006/jrpe.2000.2294 CrossRefGoogle Scholar
Melchers, K. G., Bösser, D., Hartstein, T., & Kleinmann, M. (2012). Assessment of situational demands in a selection interview: Reflective style or sensitivity? International Journal of Selection & Assessment, 20, 475–485. doi:10.1111/ijsa.12010 CrossRefGoogle Scholar
Meriac, J. P., Hoffman, B. J., Woehr, D. J., & Fleisher, M. S. (2008). Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings. Journal of Applied Psychology, 93, 1042–1052. doi:10.1037/0021-9010.93.5.1042 CrossRefGoogle ScholarPubMed
Meyer, R. D., Dalal, R. S., & Hermida, R. (2010). A review and synthesis of situational strength in the organizational sciences. Journal of Management, 36, 121–140. doi:10.1177/0149206309349309 CrossRefGoogle Scholar
Mischel, W. (1968). Personality and assessment. New York, NY: Wiley.Google Scholar
Rockstuhl, T., Ang, S., Ng, K., Lievens, F., & Van Dyne, L. (2015). Putting judging situations into situational judgment tests: Evidence from intercultural multimedia SJTs. Journal of Applied Psychology, 100, 464–480. doi:10.1037/a0038098 CrossRefGoogle ScholarPubMed
Tett, R. P., & Burnett, D. D. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88, 500–517. doi:10.1037/0021-9010.88.3.500 CrossRefGoogle ScholarPubMed
Tett, R. P., & Guterman, H. A. (2000). Situation trait relevance, trait expression, and cross-situational consistency: Testing a principle of trait activation. Journal of Research in Personality, 34, 397–423. doi:10.1006/jrpe.2000.2292 CrossRefGoogle Scholar
Tett, R. P., Simonet, D. V., Walser, B., & Brown, C. (2013). Trait activation theory: Applications, developments, and implications for person–workplace fit. In Christiansen, N. D. & Tett, R. P. (Eds.), Handbook of personality at work (pp. 71–100). New York, NY: Routledge.Google Scholar
Figure 0

Figure 1. Model of situational judgment test (SJT) performance determinants utilizing trait activation theory to explain the moderating effects of situational cues. Pathways connecting personality trait to general experience and personality trait to specific job experience are assumed but have been removed for the sake of clarity.