Many phenomena in I-O psychology are multidetermined

Because many of the phenomena in which I-O psychologists are interested are multidetermined (involving dispositional, situational, organizational, and societ al sources of influences) and involved in complex causal chains (including mediation and moderation), the idea of methodological fit implies that to address these complex issues, multivariate data-analytic methods are required. Although several examples can be given, one that recently gained attention is balanced need satisfaction. More precisely, individuals with balanced satisfaction in the needs for autonomy, competence, and relatedness should experience higher levels of well-being than people with the same aggregate level of, yet less balanced, need satisfaction (Sheldon & Niemiec, Reference Sheldon and Niemiec2006). Importantly, the role of need (im)balance cannot be tested using descriptive statistics because it necessitates an inherently multivariate approach (Gillet et al., Reference Gillet, Morin, Huart, Colombat and Fouquereau2020).

Much of our data have a nested data structure

Because people typically work in teams, and teams in organizations, I-O data often have a nested data structure. The same is true when repeated measurements are nested within individuals. Such data structures create dependencies in the data (e.g., the commitment of employees working within the same group is likely to be more similar than that of employees from different groups; Schreurs et al., Reference Schreurs, Hofmans, Wille, Murphy and Tosti-Kharas2021). These dependencies need to be considered to properly analyze such data, necessitating multilevel methods (Morin et al., Reference Morin, Blais and Chénard-Poirier2021). Two examples show how crucial methodological fit is for multilevel data. In educational psychology, the big-fish-little-pond effect (Marsh et al., Reference Marsh, Kuyper, Morin, Parker and Seaton2014) shows that the effect of classroom levels of achievement on the academic self-concept (negative) differs from that of individual levels of achievement (positive) due to social comparison processes. Likewise, McCormick et al.’s (Reference McCormick, Reeves, Downes, Li and Ilies2020) meta-analysis shows that, when comparing between-person and within-person associations (repeated measurements), associations are different across levels of analysis 24.1% of the time. Thus, when dealing with nested data, one needs to take these dependencies into account, which necessitates multilevel or time-structured models. This means that raw scores and descriptive statistics alone might not be sufficiently informative.

Psychological phenomena are usually dynamic

I-O research is often interested in phenomena that develop, evolve, and change over time. These phenomena can have a beginning, a development, an evolution, and a completion, and many of them (e.g., self-esteem) are known to present trait and state components (Perinelli & Alessandri, Reference Perinelli and Alessandri2020). Job attitudes change over time, performance is dynamic, and affect at work is incredibly fluctuating, just to mention a few examples. Expecting these changes and fluctuations to be accurately represented by simple descriptive statistics is, at best, unrealistic and becomes impossible when facing nonlinearity and interindividual variation in shape (e.g., Navarro et al., Reference Navarro, Rueff-Lopes and Rico2020). Even when considering the relatively simple case of mediation, Murphy (Reference Murphy2021) implicitly assumes that we can test mediation from a static perspective, which ignores the fact that most of our theories assume the presence of dynamic psychological processes that unfold over time. For this reason, accurate tests of mediation require longitudinal designs (Cole & Maxwell, Reference Cole and Maxwell2003) and the ability to disaggregate sources of within-person versus between-person variation.Footnote ¹

The questionable assumption of population homogeneity

Most I-O studies implicitly assume that a single set of “averaged” parameters can be used to describe the population. Yet, awareness is growing that this assumption is often too simplistic (Meyer & Morin, Reference Meyer and Morin2016). Hofmans et al. (Reference Hofmans, Wille and Schreurs2020) argue that several of our theories imply population heterogeneity rather than homogeneity: People can hold distinct configurations on a series of indicators reflecting their career orientation (McLarnon et al., Reference McLarnon, Carswell and Schneider2015), their motivation (Tóth-Király et al., Reference Tóth-Király, Morin, Bőthe, Rigó and Orosz2021), or their commitment (Meyer & Morin, Reference Meyer and Morin2016), and these can follow distinct longitudinal trajectories (Fernet et al., Reference Fernet, Morin, Austin, Gagné, Litalien, Lavoie-Tremblay and Forest2020). To detect heterogeneity, person-centered analyses are needed (Meyer & Morin, Reference Meyer and Morin2016). For instance, Solinger et al. (Reference Solinger, van Olffen, Roe and Hofmans2013) revealed that the bond between newcomers and their organizations could develop in different ways, whereas Morin et al. (Reference Morin, Maïano, Marsh, Nagengast and Janosz2013) demonstrated the indissociable nature of self-concept levels and stability. Identifying profiles of employees is also more naturally aligned with managers’ tendencies to think in terms of categories (Mäkikangas & Kinnunen Reference Mäkikangas and Kinnunen2016). The person-centered approach is thus particularly helpful for guiding intervention strategies that are tailored to the needs of distinct types of employees, an approach that has yielded benefits for burnout intervention (Hätinen et al., Reference Hätinen, Kinnunen, Mäkikangas, Kalimo, Tolvanen and Pekkonen2009). If we want to relax the often-unrealistic assumption of population homogeneity, we need to embrace complex data analyses. Moreover, heterogeneity cannot be inferred from the inspection of descriptive statistics and correlations, which suggests that the interpretation of such sample-level statistics can be misleading.

Many theories, even simple in appearance, cannot be empirically tested without complex data analyses

Many good theoretical models seek to capture the complex nature of human reality,Footnote ² which requires complex data analyses. For instance, Lawler’s (Reference Lawler1992) theory of empowerment—highlighting the role of complementariness and coherence in leaders’ empowerment practices—has long been used in textbooks and as a guide for intervention. However, a proper test of this theory has been lacking until Chénard-Poirier et al. (Reference Chénard-Poirier, Morin and Boudrias2017) found partial support for its propositions using a hybrid mixture regression approach. Similarly, to test the dynamic model of the psychological contract properly, dual regime models are required. Indeed, dual regime models “mimic the theoretical processes underlying the elicitation of violation feelings via two model components: Abinary distribution that models whether an event in one’s work environment leads to a crossing of the acceptance limits of the psychological contract and a count distribution that models how severe the negative effect of this crossing is” (Hofmans, Reference Hofmans2017, p. 8). Last, combining many previous issues (person centered, multilevel, theoretical complexity), O’Neill et al. (Reference O’Neill, McLarnon, Hoffart, Woodley and Allen2018) identified distinctive patterns of task, process, and relationship conflict located at the team-level that “variable-centered” studies failed to uncover.

As illustrated, merely examining descriptive statistics is not sufficiently informative for many of our research questions, unless we are ready, as a field, to dramatically simplify our theories. In that sense, Murphy (Reference Murphy2021) seems to argue in favor of theoretical abstraction and simplicity (assuming a limited set of grand universal laws; Healy, Reference Healy2017), whereas we believe that there is also value in understanding complexity (Tsoukas, Reference Tsoukas2017). Life is inherently complex—it is interconnected, multifaceted, paradoxical, ever-changing—and advanced statistics help us come to grips with it.

Random measurement error

Due to the imperfect nature of our measures, manifest scores contain random measurement error, which attenuates our estimates of associations between variables. In most research settings, reliable measurement (i.e., true score variance: the total variance minus the variance due to random measurement error) is reflected in the covariance among ratings that is obtained across various indicators of the same construct, whereas the unique part of each indicator incorporates random measurement error. However, in nonlatent analytic models, these two sources of variance are conflated. Whereas latent variable models naturally separate them, allowing for tests of associations corrected for random measurement error, properly accounting for measurement error is far more complex than simply relying on latent variable models. For example, Marsh and Hau (Reference Marsh and Hau1996) demonstrated that test–retest correlations based on manifest variables tend to be upwardly biased by the failure to account for longitudinal correlated uniquenesses among the matching indicators used repeatedly over time. Marsh et al. (Reference Marsh, Scalas and Nagengast2010a) similarly demonstrated the need to account for wording effects (e.g., negative wording, parallel wording) to achieve an accurate representation of the structure of our constructs. With multilevel data, measurement error due to “interitem agreement” occurs separately across levels of analysis, and “interrater agreement” between members of the higher level reality (e.g., a workgroup) are also likely to bias measurement (Morin et al., in press). To make matters worse, Marsh et al. (Reference Marsh, Seaton, Kuyper, Dumas, Huguet, Regner, Buunk, Monteil, Blanton and Gibbons2010b) demonstrated that rather than attenuating associations, multilevel sources of measurement error, in combination, could create artificial associations between constructs (referred to as phantom effects). Finally, despite their well-controlled nature, laboratory experiments are subject to the same challenges, including biased estimates of intervention effects (Breitsohl, Reference Breitsohl2019). These examples all demonstrate that descriptive results based on manifest variables can be misleading when working with unobservable constructs.

Disentangling distinct sources of variance

Recent developments have shown that measurement issues tend to be a lot more complex than previously believed (Morin et al., Reference Morin, Myers, Lee, Tenenbaum and Eklund2020). For example, statistical research has highlighted the need to account for distinct forms of true score variance in conceptually related and hierarchically ordered constructs (Morin et al., Reference Morin, Boudrias, Marsh, McInerney, Dagenais-Desmarais, Madore and Litalien2017). Exploratory structural equation modeling has been recommended as a way to account for the presence of conceptually related constructs by incorporating cross-loadings (Asparouhov et al., Reference Asparouhov, Muthén and Morin2015), whereas bifactor modeling has been recommended for hierarchically ordered constructs (Morin et al., Reference Morin, Boudrias, Marsh, McInerney, Dagenais-Desmarais, Madore and Litalien2017). In research involving ratings from different sources (e.g., teams, supervisor, assessment centers), more complex latent variable models and mixed effects models for cross-classified data are needed to isolate the multiple sources of variability that are present in these ratings (O’Neill et al., Reference O’Neill, McLarnon and Carswell2015). Such latent variable techniques have advanced our knowledge of measurement in a way that would have been impossible using descriptive statistics, in addition to help model the complexity of real-world phenomena.

Improved techniques for testing moderation

Moderator effects are central to many models and theories. Murphy (Reference Murphy2021) rightfully argues that tests of moderation suffer from several issues, including low reliability (and associated low power) of the interaction term. However, rather than taking a step back and reverting to descriptive statistics, it is equally valuable to take a step forward and work with latent variable models, which make it possible to tackle moderation in a way that accounts for unreliability (Marsh et al., Reference Marsh, Hau, Wen, Nagengast, Morin and Little2013). Once again, why not use the strengths of our complex data-analytic methods, which offer clear solutions to many of the issues that we face in our field.