It is safe to assume that an accurate performance appraisal (PA) is an important prerequisite to an effective performance management (PM) system, because with accurate PA information, management, teams, and employees can engage in the process of identifying and developing a wide range of job-relevant knowledge or skills to improve job performance. However, researchers and practitioners alike must continue to push for PA to be something other an administrative ritual; the ideal goal for PA is for it to contribute to a reliable process that can offer practical help to organizational operations, including PM. As Pulakos, Mueller Hanson, Arad, and Moye (2015) have pointed out, supervisors are concerned about demotivating or disengaging employees by providing PA ratings that are too much lower than the highest rating or ranking that is available, so having ratings that are clustered at the high end of the rating scale is quite common across organizations (Bretz, Milkovich, & Read, 1992).

The focal article (Hyland, Lee, & Mills, 2015) has reviewed the literature and has generally concluded that research on mindfulness training has been supportive and suggestive of the potential of these interventions to benefit organizations and people in the workplace. We generally agree with this conclusion but have some suggestions regarding future research and the implementation of mindfulness interventions. Our first suggestion is that organizations exercise caution about simply jumping on the mindfulness bandwagon without thinking through the decision to implement mindfulness interventions. The wide variety of mindfulness techniques can be very different in terms of the techniques used, the potential to be perceived as intrusive, and the outcomes associated with the different mindfulness interventions. We suggest that, when mindfulness interventions are implemented in organizations, the fit of the intervention with the specific goals of the organization be considered as well as alternative interventions. In addition, we suggest that the benefits to employees and employers be clearly articulated, as there may be possible conflicts between these constituencies.

We believe that Hyland, Lee, and Mills (2015) introduced important considerations concerning the importance of mindfulness for research and practice in industrial and organizational (I-O) psychology, and we applaud their effort. We also feel that some equally important points were neglected or given scant attention. We amplify their introduction in three ways. We discuss (a) how construct confusion is common when new constructs are introduced and how paying attention to the type of meditation practice used to induce mindfulness will enhance construct clarity, (b) how using forms of meditation that employ physical movement and other activities to induce mindfulness expand the choices for intervention and training, and (c) how meditation and mindfulness provide a valuable bridge linking positive psychology and I-O psychology theory and practice.

Lapierre et al. (2018) provide a compelling set of reasons why conducting research in organizational settings should include investing time and energy to foster and promote enhanced relationships with organizational stakeholders. We agree that managing research partnerships carefully is paramount to creating the conditions necessary to implement rigorous methodology and collect the data needed to test theory-based hypotheses. However, we believe that Lapierre et al.’s argument should be taken further.

We expected that the commentary process would provide valuable feedback to improve our ideas and identify potential obstacles, and we were not disappointed. The commentaries were generally in agreement that synthetic validity is a good idea, although we also received a fair amount of suggestions for improvements, conditional or tempered praise, and explicitly critical comments. We address the concerns that were raised and conclude that we should move forward with developing a large-scale synthetic validity database, incorporating the suggestions of some of the commentators.

We were asked by the editor of this journal about our reactions to the Ryan and Ford (2010) article. More specifically, she asked (a) to what degree “organizational psychologist” was part of our identities in our roles as the senior-most human resources executive in our respective organizations and (b) which of the scenarios described at the end of the Ryan and Ford article would most benefit organizations and the people who work in them (rather than examining which would be most beneficial to the profession itself).

In the past few years, the term “21st century skills” has gained increasing popularity in educational research and business practice. Neubert, Mainert, Kretzschmar, and Greiff (2015) advocated for the utility of assessing 21st century skills in industrial–organizational psychology (I-O psychology) and its superiority over assessing basic psychological constructs, using application-oriented constructs, or conducting job analysis to determine relevant skills for individual work settings. We argue, however, that the issues identified and discussed in the focal article are rather threefold and that, to integrate 21st century skills into I-O psychology and to bridge organizational science and practice, we need to have (a) a standard framework of clearly defined constructs, (b) innovative assessments, and (c) evidence for validity generalization. We elaborate below.

Chamorro-Premuzic, Winsborough, Sherman, and Hogan (2016) describe a variety of new selection approaches (e.g., “scraping” of social media information, gamified assessments) in the staffing domain that might provide new sources of information about people. The authors also mention advantages and downsides of these potentially “new talent signals.”

The lengthy and complex focal article by Tett, Hundley, and Christiansen (2017) is based on a fundamental misunderstanding of the nature of validity generalization (VG): It is based on the assumption that what is generalized in VG is the estimated value of mean rho ($\bar{\rho}$). This erroneous assumption is stated repeatedly throughout the article. A conclusion of validity generalization does not imply that $\bar{\rho}$ is identical across all situations. If VG is present, most, if not all, validities in the validity distribution are positive and useful even if there is some variation in that distribution. What is generalized is the entire distribution of rho ($\bar{\rho}$), not just the estimated $\bar{\rho}$ or any other specific value of validity included in the distribution. This distribution is described by its mean ($\bar{\rho}$) and standard deviation (SDρ). A helpful concept based on these parameters (assuming ρ is normally distributed) is the credibility interval, which reflects the range where most of the values of ρ can be found. The lower end of the 80% credibility interval (the 90% credibility value, CV = $\bar{\rho}$ – 1.28 × SDρ) is used to facilitate understanding of this distribution by indicating the statistical “worst case” for validity, for practitioners using VG. Validity has an estimated 90% chance of lying above this value. This concept has long been recognized in the literature (see Hunter & Hunter, 1984, for an example; see also Schmidt, Law, Hunter, Rothstein, Pearlman, & McDaniel, 1993, and hundreds of VG articles that have appeared in the literature over the past 40 years since the invention of psychometric meta-analysis as a means of examining VG [Schmidt & Hunter, 1977]). The $\bar{\rho}$ is the value in the distribution with the highest likelihood of occurring (although often by only a small amount), but it is the whole distribution that is generalized. Tett et al. (2017) state that some meta-analysis articles claim that they are generalizing only $\bar{\rho}$. If true, this is inappropriate. Because $\bar{\rho}$ has the highest likelihood in the ρ distribution, discussion often focuses on that value as a matter of convenience, but $\bar{\rho}$ is not what is generalized in VG. What is generalized is the conclusion that there is validity throughout the credibility interval. The false assumption that it is $\bar{\rho}$ and not the ρ distribution as a whole that is generalized in VG is the basis for the Tett et al. article and is its Achilles heel. In this commentary, we examine the target article's basic arguments and point out errors and omissions that led Tett et al. to falsely conclude that VG is a “myth.”