Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-02-06T20:46:57.206Z Has data issue: false hasContentIssue false

Reboot Hu et al.’s review of the ICT literature, but only after updates are installed

Published online by Cambridge University Press:  22 September 2021

Philip Bobko*
Affiliation:
Gettysburg College
Denise Potosky
Affiliation:
The Pennsylvania State University, Great Valley School of Graduate Professional Studies
*
*Corresponding author. Email: pbobko@gettysburg.edu
Rights & Permissions [Opens in a new window]

Abstract

Type
Commentaries
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of the Society for Industrial and Organizational Psychology

We note fundamental concerns about Hu et al.’s (Reference Hu, Barber, Park and Day2021) narrative review of information communication technology (ICT) research including their identification of “important trends” (p. 2), reflections on “key constructs, findings, and theories” (p. 2), and their attempt to “bring together diffuse research areas” (p. 4). Their attention to this timely topic is appreciated, but we suggest a revised and renewed effort by SIOP members to synthesize such research.

Our suggestion is based on two overall observations. First, a substantial amount of key literature appears to have been overlooked in Hu et al.’s (Reference Hu, Barber, Park and Day2021) review effort. Therefore, any clarifying taxonomies of research models (and associated conclusions) are misspecified and potentially inaccurate and misleading. Second, in their attempt to clarify diffuse research areas, Hu et al. created two ambiguously overlapping categories. As well, some key terms are undefined, causing potential conceptual confusion. In addition, it is only toward the end of the article that they note that ICT research is multidisciplinary. Consequently, the article does not appear to provide a useful foundation upon which to advance interdisciplinary ICT research, let alone applied research by industrial-organizational (I-O) psychologists. We expand on these concerns (missing research; lack of due process) in turn. To borrow from Hu et al.’s metaphoric title, we suggest that their well-intentioned effort should be closed and rebooted, but only after some important updates are installed.

Missing research/theory

In empirical research, single studies and meta-analytic summaries can suffer from issues of model misspecification (Bobko & Roth, Reference Bobko, Roth and Murphy2003; James, Reference James1980). Of course, no study is capable of including every variable, nor is any review capable of including every relevant study. But if too many key variables or studies are missing, then underlying analyses are likely flawed and important constructs/themes may be overlooked. We suggest that such issues have occurred in major ways in Hu et al.’s (Reference Hu, Barber, Park and Day2021) review.

The focal article is missing important references to both classic and contemporary literature regarding the influence of technological changes on people’s work and nonwork interactions. For example, one might consider work by Reisman (Reference Reisman, Larrabee and Meyersohn1958), who described the problems of a “postindustrial society” and a consequent abundance of spare time due to the reduction in work hours and increased productivity provided by advancements in technology (see also Kahn & Wiener, Reference Kahn and Wiener1967). As another example of earlier missing work, Michael (Reference Michael1962) focused on computers and work automation as a threat to people’s sense of purpose. As well, Marvin (Reference Marvin1988) described the profound influence, social disruptions, and invasion of privacy caused by electronic communication inventions in the 19th century (e.g., phone, electric light, movies, telegraph). In his examination of inventions in the modern history of technology, Edgerton (Reference Edgerton1999) distinguished between technology innovation and technology use. It is possible that Hu et al. (Reference Hu, Barber, Park and Day2021) may not have intended to provide a comprehensive review so much as offer a perspective that would prompt a response in Industrial Organizational Psychology (IOP). But by failing to consider the extensive literature on the influence of technology at large, as well as available current frameworks (see examples below), those authors missed an opportunity to build on a conversation that has been occurring since at least the 1950s.

More specifically to I-O psychology, a recent article in this journal (Morelli et al., Reference Morelli, Potosky, Arthur and Tippins2017) has also provided a call for more theorizing and I-O psychology models of technology. Although that article used the context of selection/assessment, it did provide three conceptual frameworks that could be useful in synthesizing ICT research. Hu et al. (Reference Hu, Barber, Park and Day2021) make no reference to this focal IOP paper or the literature that is used to develop its conceptual frameworks.

Hu et al. (Reference Hu, Barber, Park and Day2021) do mention some books and articles when describing their two-category framework and other ideas, but the contributions of these works to the review are unclear. For example, at first mention of their two-category system (technology behavior and technology experience), they cite Rice and Leonardi (Reference Rice, Leonardi, Putnam and Mumby2013). However, that review (of multiple research literatures) identified 13 themes that are relevant to ICT, which are not addressed in Hu et al. (Reference Hu, Barber, Park and Day2021). Rice and Leonardi also emphasize that communication theory and research has much to offer ICT research, because communication is the constructive process of organizing (e.g., Putnam & Nicotera, Reference Putnam and Nicotera2010). This notion does not appear in Hu et al.

In a related concern, Hu et al. (Reference Hu, Barber, Park and Day2021) suggest that I-O psychologists might apply communication theory to better understand the utility and uniqueness of ICT concepts. However, instead of reviewing the communication literature, they describe “three example theoretical approaches related to communication” (p. 371). Yet, theories “related” to communication are not necessarily communication theories, and Hu et al.’s rationale for choosing these three examples is unclear. Indeed, in the communication literature, a fundamental premise is that “information exchange” (e.g., Shannon & Weaver’s [Reference Shannon and Weaver1949] model) does not adequately characterize “communication,” i.e., the creation, interpretation, and exchange of symbols to construct meaning. This distinction seems central to a complete review of ICT, but is absent in Hu et al. Also, specific theories or frameworks of the dyadic communication process (cf., Jones, Reference Jones2013) would be helpful. Barnlund’s (Reference Barnlund, Akin, Goldberg, Myers and Stewart1970) transactional model of communication, although not cited, is one possibility. Or, Potosky (Reference Potosky2008) delineated four features of any communication medium (i.e., transparency, social bandwidth, interactivity, and perceived surveillance) that could help clarify key issues related to ICT.

We offer another example of missing literature. Despite a stated focus on individuals, Hu et al. (Reference Hu, Barber, Park and Day2021) mention studies on virtual teams, including team-level concepts of cohesiveness and performance. Most of their citations span the years 1999 to 2009, but there is a substantial, more recent literature on teams and technology, virtual teams, and team communication. For example, the American Psychologist (Reference McDaniel, Salas and Kazak2018) includes several summaries of teams research, which include rich reference lists. In addition, a web search will generate many other articles on communication in virtual teams, types of communication platforms, virtual leadership, and so on. Indeed, one such article (Kirkman et al., Reference Kirkman, Gibson, Kim and Kozlowski2012) provides categories and themes that might help the clarification process that Hu et al. attempted to conduct.Footnote 1 Also, in work on human–computer interactions, there is a robust examination of computer-supported cooperative work (CSCW; see Grudin, [Reference Grudin1994], for a review), including the psychosocial effects of “groupware” and “group decision support systems,” which is clearly relevant to the use of technology in work groups, albeit missing in Hu et al.

Even more broadly, we suggest that research in education and instructional design will have value in synthesizing ICT research. Indeed, well before the recent pandemic, these efforts have considered hybrid and virtual learning platforms—with respect to delivering course content, engaging consumers, sharing knowledge, and developing best practices. Future efforts might incorporate this missing work.

Lack of due process in creating clarity

If Hu et al.’s (Reference Hu, Barber, Park and Day2021) intent was to conduct a “review,” “bring together diffuse research areas,” and provide “clarity regarding theoretical mechanisms” (p. 371), one would expect (a) definitions of key concepts/terms (e.g., a definition of technology), (b) thorough or strategically targeted consideration of relevant literatures (i.e., I-O, management, communication, education, and information systems), and (c) an analysis that leads to a synthesizing taxonomy or model. Instead, what appears after 27 pages are statements that ICT research spans many literatures (I-O psychology, human–computer interactions, information systems) and a statement that the field needs interdisciplinary collaboration. This seems to reverse the logic of how a review is conducted.

As a related concern, the authors propose two categories for technology research (i.e., behavioral and psychological experience perspectives) prior to describing findings from their review of some literature. However, forming categories without analyzing the relevant extant literatures may do little to organize information that the authors perceive as scattered. Although the “who, what, where, when, why, and how” questions they pose represent a formula for gathering information, the derivation of the behavioral perspective in terms of where/when/what and then psychological experiences as the how/why is not apparent. Indeed, it seems likely that all questions apply to both categories (origin of the categories notwithstanding).

The two proposed categories may actually increase confusion rather than clarify ICT research. The technology behavior perspective is described in terms of psychological experience, that is, “how use of ICTs (i.e., behaviors) have influenced employees’ psychological experiences” (p. 371). The technology experience perspective is explained as psychological states or responses “that may arise from technological work events or situations” (pp. 371–396), which seems to imply that the states/responses are produced by the behavioral use of technology. Because “actual” behavior and perceptions/experiences of that behavior seem inherently intertwined, sorting ICT studies into one or the other category seems conceptually subjective if not methodologically unsound.

Separating individuals’ technology use behaviors from their perceptions, motives, and situated experiences with technology may prove especially difficult to align with research paradigms in the social sciences, where use and experience occur synergistically. For example, theorizing on sociomateriality (Leonardi, Reference Leonardi, Leonardi, Nardi and Kallinikos2012; Orlikowski, Reference Orlikowski2007) indicates that the material aspects of technology are shaped by its use, and, simultaneously, the social aspects of its use are shaped by the design of digital technology. Other instances of this synergy occur in the social sciences, and the circularity inherent in the definitions of the two categories may thus be more theoretically accurate than realized.

Another process concern is that Hu et al. (Reference Hu, Barber, Park and Day2021) do not offer definitions for some of their constructs—not even for basic terms such as “technology” or “information communication technology (ICT).” This deficiency is surprising, as Hu et al. suggest that a lack of clarity in the field is “likely driven by a lack of awareness of both key terms and definitions” (p. 371). By comparison, Morelli et al. (Reference Morelli, Potosky, Arthur and Tippins2017) and Rice and Leonardi (Reference Rice, Leonardi, Putnam and Mumby2013) provide exemplary definitions of technology and ICTs, respectively.

Further, terms such as “technology,” “information technology,” and “information communication technology,” which are used almost interchangeably in their article, do not consistently refer to the same concept(s). Not all technology involves computers (e.g., consider phones, pencils, etc.), not all IT involves communication (e.g., database management), and not all ICT is web enabled (e.g., landline phone, fax, radio, pager, etc.). The authors’ exclusive focus on internet technology in the context of ICTs is limiting. To illustrate, Hu et al. (Reference Hu, Barber, Park and Day2021; citing Day et al., 2012) note a “new” (p. 371) phenomenon of ICT hassles, which are technology-related events that interfere with one’s ability to effectively conduct work (e.g., “computer freezes, Internet problems, and computer viruses”). Negative perceptions of technology in use are not new, however, and there is little reason to associate these annoyances uniquely with computers or the Internet. For example, using a pencil to exchange ideas via written words (or to complete a survey) might be considered the use of an ICT—and sharpened points breaking, finding a sharpener, or having the wrong lead number pencil, and so forth can also be annoying. The frequency of computer freezes, Internet problems, and viruses combined might even be less than the frequency of pencil problems. Our “point” (pun intended) is that ICT hassles are not new. Furthermore, as sociomateriality theory suggests, “hassles” are perceived and experienced by the individual while using a technology, not inherent in the technology itself.

Hu et al. (Reference Hu, Barber, Park and Day2021; citing Barber & Santuzzi, 2015) also claim that “workplace telepressure is another new concept” (p.371); that is, the preoccupation with, and urge to respond quickly to, work-related ICT messages. However, we observe that long before the visual/aural notifications of social media and web-enabled messages, people have felt compelled to answer a ringing phone even when engaged in a face-to-face conversation. Workplace telepressure is interesting, but it does not seem particularly new. There is also considerable research on work interruptions associated with instant messaging or emails (e.g., Garrett & Danziger, Reference Garrett and Danziger2007; Grandhi & Jones, Reference Grandhi and Jones2015; Jackson et al., Reference Jackson, Dawson and Wilson2003; Wajcman & Rose, Reference Wajcman and Rose2011). Puranik et al. (Reference Puranik, Koopman and Vough2020) provide an integrative review of work interruptions, including those associated with technology.

The above concerns seem to be at odds with Hu et al.’s (Reference Hu, Barber, Park and Day2021) conclusion that their defrag and reboot process will help make ICT research “easy to understand and ‘actionable’” (p. 371). Indeed, their results are not particularly actionable. For example, Hu et al. consider the notions of pressing organizational practices, policies, and legislative constraints—a laudable focus. However, they note that monitoring policies “may” have positive effects and “may have negative effects,” and they note that, given the moderator of culture, the beneficial effect of right-to-disconnect legislation “is unclear.” It is possible that the fundamental concerns noted above contributed to these equivocations.

Conclusion

Hu et al.’s (Reference Hu, Barber, Park and Day2021) perception that ICT research is “scattered” is understandable. However, they are not the first to encourage researchers to develop theories “that clarify the unique role of ICT concepts in I-O psychology research” (p. 371), nor does their review adequately provide that synthesis. The review is missing relevant literature from a variety of domains and periods as well as definitions of key constructs. In addition, the two seemingly a priori categories (behavioral and psychological experience) do not effectively set the stage for future research in I-O psychology.Footnote 2 Further, Hu et al.’s conclusions are nonactionable (Pearce & Huang, Reference Pearce and Huang2012).

Hu et al. (Reference Hu, Barber, Park and Day2021) suggest that researchers “reboot” efforts to synthesize ICT literature. We agree, but first some “updates” need to be installed. We suggest that I-O psychologists who attempt this task more carefully define their terms, acknowledge the reviews that have already been done, more thoroughly consider multiple, relevant literatures (e.g., in psychology, management, communication, information science, and education), and develop taxonomies via induction or deduction. Overall, we see utility in Hu et al.’s paper as motivating future synthesis efforts rather than directly clarifying disjointed research in ICT. Perhaps we do need to reboot, but let’s first update using our best systematic understanding of the interdisciplinary body of knowledge we have to work with.

Footnotes

Contributions equal; order of authorship is alphabetic.

1 Additional articles consider how computer technologies and sensors can help assess communication processes (e.g., Mathieu et al., Reference Mathieu, Wolfson and Park2018; Stewart et al., Reference Stewart, Vrzakova, Sun, Yonehiro, Stone, Duran, Shute and D’Mello2019). Such missing literature seems to be particularly relevant to I-O psychology, given its rich history of interest in measurement.

2 The “defrag” metaphor that is used in their title seems unintentionally appropriate. Defragmentation places things more closely in available, adjacent electronic space; it does not provide links between files, nor does it synthesize any information that is stored. Somewhat ironically, the process of defragmentation keeps all the empty spaces together, thus creating larger gaps in the overall storage space than before.

References

Barnlund, D. C. (1970). A transactional model of communication. In Akin, J., Goldberg, A., Myers, G., & Stewart, J. (Eds.), Language behavior: A book of readings in communication (pp. 4361). Mouton.CrossRefGoogle Scholar
Bobko, P., & Roth, P. (2003). Meta-analysis and validity generalization as research tools: Issues of sample bias and degrees of mis-specification. In Murphy, K. (Ed.), Validity generalization: A critical review (pp. 6790). Lawrence Erlbaum Associates.Google Scholar
Edgerton, D. (1999). From innovation to use: Ten eclectic theses on the historiography of technology. History and Technology, an International Journal, 16(2), 111136.CrossRefGoogle Scholar
Garrett, R. K., & Danziger, J. N. (2007). IM= Interruption management? Instant messaging and disruption in the workplace. Journal of Computer-Mediated Communication, 13(1), 2342.CrossRefGoogle Scholar
Grandhi, S. A., & Jones, Q. (2015). Knock, knock! Who’s there? Putting the user in control of managing interruptions. International Journal of Human-Computer Studies, 79, 3550.CrossRefGoogle Scholar
Grudin, J. (1994). Computer-supported cooperative work: History and focus. Computer, 27(5), 1926.CrossRefGoogle Scholar
Hu, X., Barber, L., Park, Y., & Day, A. (2021). Defrag and reboot? Consolidating information and communication technology research in I-O psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 14(3), 371–396.Google Scholar
Jackson, T., Dawson, R., & Wilson, D. (2003). Reducing the effect of email interruptions on employees. International Journal of Information Management, 23, 5565.CrossRefGoogle Scholar
James, L. (1980). The unmeasured variables problem in path analysis. Journal of Applied Psychology, 65, 415421.CrossRefGoogle Scholar
Jones, R. (2013). The communication process. Communication in the real world: An introduction to communication studies, Chapter 1.2. The Saylor Foundation. https://open.lib.umn.edu/communication/ Google Scholar
Kahn, H., & Wiener, A. J. (1967). The Year 2000: A framework for speculation on the next thirty-three years. Macmillan.Google Scholar
Kirkman, B. L., Gibson, C. B., & Kim, K. (2012). Across borders and technologies: Advancements in virtual teams research. In Kozlowski, S. W. J. (Ed.), The Oxford handbook of organizational psychology (Vol. 2, pp. 789858). Oxford University Press.Google Scholar
Leonardi, P. M. (2012). Materiality, sociomateriality, and socio-technical systems: What do these terms mean? How are they related? Do we need them? In Leonardi, P. M., Nardi, B. A., & Kallinikos, J. (Eds.), Materiality and organizing: Social interaction in a technological world (pp. 2548). Oxford University Press.CrossRefGoogle Scholar
Marvin, C. (1988). When old technologies were new: Thinking about electric communication in the late nineteenth century. Oxford University Press.Google Scholar
Mathieu, J., Wolfson, M., & Park, S. (2018). The evolution of work team research since Hawthorne. American Psychologist, 73, 308321.CrossRefGoogle ScholarPubMed
McDaniel, S., Salas, E., & Kazak, A. (Eds.). (2018). The science of teamwork [Special issue]. American Psychologist, 73.CrossRefGoogle Scholar
Michael, D. N. (1962). Cybernation: The silent conquest. Center for the Study of Democratic Institutions.Google Scholar
Morelli, N., Potosky, D., Arthur, W., & Tippins, N. (2017). A call for conceptual models of technology in I-O psychology: An example from talent-based technology assessment. Industrial and Organizational Psychology: Perspectives on Science and Practice, 10(4), 634653.CrossRefGoogle Scholar
Orlikowski, W. J. (2007). Sociomaterial practices: Exploring technology at work. Organization Studies, 28, 14351448.CrossRefGoogle Scholar
Pearce, J., & Huang, L. (2012). Toward an understanding of what actionable research is. Academy of Management Learning & Education, 11, 300301.CrossRefGoogle Scholar
Potosky, D. (2008). A conceptual framework for the role of the administration medium in the personnel assessment process. Academy of Management Review, 33, 629648.CrossRefGoogle Scholar
Puranik, H., Koopman, J., & Vough, H. C. (2020). Pardon the interruption: An integrative review and future research agenda for research on work interruptions. Journal of Management, 46(6), 806842.CrossRefGoogle Scholar
Putnam, L. L., & Nicotera, A. M. (2010). Communicative constitution of organization is a question: Critical issues for addressing it. Management Communication Quarterly, 24(1), 158165.CrossRefGoogle Scholar
Reisman, D. (1958). Work and leisure in post-industrial society. In Larrabee, E. & Meyersohn, R. (Eds.), Mass leisure (pp. 363385). Free Press.Google Scholar
Rice, R., & Leonardi, P. (2013). Information and communication technologies in organizations. In Putnam, L. L. & Mumby, D. K. (Eds.), The SAGE handbook of organizational communication (pp. 425448). Sage.Google Scholar
Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. The University of Illinois.Google Scholar
Stewart, A. E., Vrzakova, H., Sun, C., Yonehiro, J., Stone, C. A., Duran, N. D., Shute, V., & D’Mello, S. K. (2019). I say, you say, we say: Using spoken language to model socio-cognitive processes during computer-supported collaborative problem solving. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), Article 194.CrossRefGoogle Scholar
Wajcman, J., & Rose, E. (2011). Constant connectivity: Rethinking interruptions at work. Organization Studies, 32(7), 941961.CrossRefGoogle Scholar