Introduction
Rational Emotive Behaviour Therapy (REBT; Ellis, Reference Ellis2001) is based on the theoretical premise that dysfunctional cognitive, emotional, behavioural, and physiological responses, or ‘consequences’ (C), are not the direct product of the adverse activating events experienced (A), but are rather the result of evaluative or appraisal beliefs (B) about these activating events. Evaluative beliefs are thus hypothesized to be the key aetiopathogenetic variables in the development of cognitive-emotional-behavioural-physiological reactions.
REBT theory outlines two main classes of evaluative beliefs; rational beliefs and irrational beliefs. Rational beliefs reflect flexible and non-extreme evaluations of life events whereas irrational beliefs reflect rigid, absolutistic, and extreme evaluations of life events (Dryden & Neenan, Reference Dryden and Neenan2004). The primary irrational belief process is termed ‘Demandingness’ beliefs, (absolutistic imperatives directed towards oneself, others, and life conditions).
Demandingness beliefs are hypothesized to give rise to a set of secondary irrational beliefs which include: ‘Catastrophizing’ beliefs (an event is evaluated in extremely negative terms); ‘Low frustration tolerance’ beliefs (a person hugely underestimates their own ability to tolerate the discomfort of not having their demand met); and ‘Depreciation’ beliefs (global negative evaluations of the self, others, and/or the world). The rational counterparts to these beliefs include: ‘Preference’ beliefs (desires or wishes rather than demands); ‘Non-catastrophizing’ beliefs (balanced and realistic evaluations of the badness of an adverse life event); ‘High frustration tolerance’ beliefs (recognition of one's capacity to cope with, and withstand, unpleasant life events); and ‘Acceptance’ beliefs (acceptance of one's own, or others, fallibility as human beings).
Activation of a set of irrational beliefs in response to a life adversity is expected to lead to dysfunctional negative emotional consequences (along with associated maladaptive behaviours or behavioural tendencies, distorted negative automatic thoughts, and disturbing physiological arousal). Alternatively, responding to the same unpleasant event with a set of rational beliefs is predicted to give rise to functional negative emotional consequences (along with associated adaptive behaviours or behavioural tendencies, non-distorted automatic thoughts, and non-disturbing physiological arousal) (see David et al. Reference David, Montgomery, Macavei and Bovbjerg2005a).
A distinguishing theoretical feature of REBT theory (Ellis, Reference Ellis1994) relates to its prediction of a binary model of emotional distress (ED). REBT theory is therefore unique in the field of psychotherapy as all other theoretical models assume a unitary model of ED. The unitary model of ED assumes that distress is experienced along a continuum which ranges from low to high levels of ED, regardless of the particular emotion being considered. As such, from the perspective of the unitary model of emotions, functional and dysfunctional emotions are considered to differ quantitatively.
By contrast, the binary model of ED assumes a qualitative rather than a quantitative distinction between functional and dysfunctional emotions. In other words, functional and dysfunctional emotions are not predicted to be distinguished on the basis of the intensity with which the emotion is experienced but rather by the underlying cognitive architecture of the emotional response, along with the subjective phenomenological experience of the emotion, and the associated behavioural consequences of the emotion (Ellis & DiGiuseppe, Reference Ellis and DiGiuseppe1993). A number of recent research findings have offered support for the cognitive, emotional and behavioural response styles that can be predicated using the binary model of emotions (e.g. David et al. Reference David, Schnur and Belloiu2002, Reference David, Schnur and Birk2004, Reference David, Szentagotai, Kallay and Macavei2005b; DiLorenzo et al. Reference DiLorenzo, David and Montgomery2011). Despite the recent empirical support for the binary model of emotions, there is still no scientific consensus on the superiority of either the unitary or binary models and therefore many within the REBT community continue to favour the unitary approach to conceptualizing ED (Wessler, Reference Wessler1996).
To provide a method of investigating the predictions of the binary model of emotions, researchers developed the Profile of Emotional Distress (PED; Opris & Macavei, Reference Opris and Macavei2007). The PED is first self-report measure of ED constructed upon a binary model of ED. The scale was designed to measures four emotional categories (sadness, concern, anxiety, depression) which are expected to reflect the distinctions between functional and dysfunctional affective responses.
Opris & Macavei (Reference Opris and Macavei2007) initially investigated the validity and reliability of the PED within a large sample (n = 701) of the Romanian general population. Results suggested that the PED possessed satisfactory internal reliability (Cronbach's α = 0.94 for the full scale and each of the four subscales demonstrated internal reliability values >0.75). Subsequent analysis indicted good concurrent and discriminant reliability. In an effort to establish the construct validity of the scale, the authors performed a principal component analysis (PCA) among both a clinical (n = 32) and a non-clinical (n = 122) sample with results revealing two factors. The first was termed ‘General distress’ and included all items of both functional and dysfunctional distress, while the second factor was termed ‘Functional distress’ and included only the functional negative distress items. A number of methodological issues undermine the results of this study. For example, PCA is method that simply allows for the reduction of a large body of data, it does not allow for the testing or falsification of a particular model. Within a PCA framework there are no objective statistical criteria to determine the solution with the optimal number of factors (see Bollen, Reference Bollen1989). The small sample sizes employed for such analysis further undermines the reliability of such results.
Consequently, the latent structure of the PED has yet to be established and formulating an appropriate scoring system scheme for this questionnaire remains problematic. Moreover, given that the PED was developed to capture the qualitative distinctions between functional and dysfunctional emotions, and its intended use in research programmes using this paradigm, it is necessary that a comprehensive evaluation of the underlying factor structure of PED be performed. Establishing the latent structure of the PED (Opris & Macavei, Reference Opris and Macavei2007) is therefore a prerequisite not only for identifying accurate assessments of validity and reliability, but also for establishing its use within a variety of research contexts. Research has demonstrated that treating a multidimensional measure as unidimensional can result in unstable estimates of reliability (Shevlin et al. Reference Shevlin, Miles, Davies and Walker2000).
Therefore, the primary aim of the current study is to test a series of theoretically plausible factorial solutions within an alternative models’ framework using CFA techniques as well confirmatory bifactor modelling producers (see Yung et al. Reference Yung, Thissen and McLeod1999; Reise et al. Reference Reise, Morizot and Hays2007, Reference Reise, Moore and Haviland2010). Confirmatory bifactor modelling is a conceptually distinct alternative to traditional CFA models in which the covariance among PED items is explained in terms of a single general ED factor reflecting the overlap across all items, and independent (uncorrelated) method factors reflecting the unique covariance that occurs among a particular groups of items (concern, sadness, anxiety, depression). Reise et al. (Reference Reise, Moore and Haviland2010) argue that bifactor models should always be used as a baseline comparison model rather than the traditional one-factor model given that a bifactor model is capable of retaining a unidimensional conceptualization while also acknowledging the unintended and meaningless covariance that can occur between particular items in a scale due to wording effects and can thus present spurious evidence of multidimensionality. Additionally, the current study also seeks to better establish the reliability of the PED through the use of composite reliability analysis.
Methods
Participants and procedures
The sample for the current study consisted of 313 (males: n = 212; females: n = 101) emergency-service personnel (police, military, and related emergency-service officers) recruited from active duty while serving in either the Republic of Ireland or the Republic of Kosovo. All participants chosen for inclusion in the current study had English as a primary language. Participants ranged in age from 23 to 65 years (mean = 38.18, s.d. = 8.70). Participation in the current study was voluntary and no inducements or obligations were used. Each participant was assured about confidentiality and those who chose to take part in the research project had the option of completing either an anonymous self-administered paper-and-pencil version of the questionnaire or an electronic version which was delivered and returned via email. The majority of respondents chose the paper-and-pencil option (63.26%, n = 198).
Instruments
The PED (Opris & Macavei, Reference Opris and Macavei2007) is a 26-item self-report measure of ED which is theoretically associated with Ellis’ (Reference Ellis1994) binary cognitive model of ED. The PED measures functional and dysfunctional emotions within two major categories: concern/anxiety and sadness/depression. Six adjective items are used to measure concern, anxiety, and sadness respectively, while eight items are employed to measure depression. Participants were asked to rate how often they experienced each emotion over the past 2 weeks by selecting either (1) ‘not at all’, (2) ‘a little’, (3) ‘moderately’, (4) ‘quite a bit’, and (5) ‘extremely’. Possible scores range from 26 to 130, with higher scores indicating higher ED.
Analysis
The dimensionality of the PED was investigated through the use of conventional CFA techniques, along with the utilization of a confirmatory bifactor modelling approach (see Yung et al. Reference Yung, Thissen and McLeod1999; Chen et al. Reference Chen, West and Sousa2006; Reise et al. Reference Reise, Morizot and Hays2007, Reference Reise, Moore and Haviland2010).
Model 1 is a one-factor model in which all 26 items load on a single latent ED variable. Model 2 is an intercorrelated four-factor solution measuring Concern (six items), Anxiety (six items), Sadness (six items), and Depression (eight items). This model represents the intended structure of the scale and is congruent with the binary model.
Model 3 is a two-factor model represented by a functional negative emotional distress (F-NED) factor and a dysfunctional negative emotional distress (D-NED) factor. This model is also in line with theoretical predictions of the binary model and within this model 12 items load onto the F-NED factor (items measuring sadness and concern) and 14 items load onto the D-NED factor (items measuring anxiety and depression). Model 4 is consistent with a unitary model of emotions and reflects an alternative two-factor solution. This model includes an Anxiety factor (12 items measuring concern and anxiety) and a Depression factor (14 items measuring sadness and depression) (see Fig. 1). Model 5 is a bifactor conceptualization in which all 26 items load onto a single ED factor. This model also includes four grouping factors [Concern (six items), Anxiety (six items), Sadness (six items), Depression (eight items)] which exist at the same conceptual level as the general ED factor.
Fig. 1. Bifactor model of the Profile of Emotional Distress Scale. C, Concern; S, Sadness; A, Anxiety; D, Depression; ED, Emotional distress.
The overall fit of each model and the relative fit between models were assessed using a range of goodness-of-fit statistics and assessment of the appropriateness of the model parameters. The χ2 statistic assessed the sample and implied covariance matrix and a good fitting model is indicated by a non-significant result. However, the χ2 statistic is strongly associated with sample size, and as such good models tend to be over-rejected. Therefore Tanaka (Reference Tanaka1987) suggested that a model should not be rejected simply on the basis of a significant χ2 result. Accordingly, it is recommended that researchers examine the ratio of the χ2 value to the degrees of freedom (d.f.), and according to Kline (Reference Kline1994), any model with a χ2:d.f. ratio of less than 3:1 indicates a good fitting model. The Comparative Fit Index (CFI; Bentler, Reference Bentler1990) and the Tucker–Lewis Index (TLI; Tucker & Lewis, Reference Tucker and Lewis1973) are measures of how much better the model fits the data compared to a baseline model where all variables are uncorrelated. For these indices values >0.90 indicate a reasonable fit while values >0.95 indicate a good model fit (Bentler, Reference Bentler1990; Hu & Bentler, Reference Hu and Bentler1999). In addition, two more absolute indices are presented; the standardized root mean square residual (SRMR; Jöreskog & Sörbom, Reference Jöreskog and Sörbom1981) and the root mean-square error of approximation (RMSEA; Steiger, Reference Steiger1990). Ideally these indices should be <0.05; however, values <0.08 also suggest adequate fit (Bentler, Reference Bentler1990; Jöreskog & Sörbom, Reference Jöreskog and Sörbom1993; Hu & Bentler, Reference Hu and Bentler1999). Furthermore, Akaike's Information Criterion (AIC; Akaike, Reference Akaike1974) was used to evaluate the alternative models, with the smaller value indicating the best fitting model. The CFI, RMSEA and AIC all have explicit penalties for model complexity. These models were specified and estimated using Mplus version 6.0 (Muthen & Muthen, Reference Muthen and Muthen1998–2010) with robust maximum-likelihood estimation.
Results
The mean PED score for the entire sample was 53.53 (s.d. = 24.96). Scores ranged from 26 to 129.
Model results
Table 1 reports the fit indices for the five alternative models. On the basis of the χ2:d.f. ratio, CFI, TLI, RMSEA, and SRMR results, Model 5 (the bifactor model) was found to be the most accurate representation of the underlying latent structure of the PED. The χ2:d.f. ratio of 3:1 and SRMR value of 0.05 indicate good model fit while a RMSEA value of 0.08 and CFI and TLI values >0.90 suggest an adequate model fit. This model also displayed the lowest AIC value further indicating its statistical superiority.
Table 1. Confirmatory factor analysis (CFA) and bifactor model fit indices for the alternative models of the Profile of Emotional Distress (PED)
N = 313.
χ2, Chi square goodness-of-fit statistic; d.f., degrees of freedom; CFI, Comparative Fit Index; TLI, Tucker–Lewis Index; RMSEA, root-mean-square error of approximation; CI, confidence interval; SRMR, standardized square root mean residual; AIC, Akaike's Information Criterion; F-NED; Functional negative emotional distress; D-NED, Dysfunctional negative emotional distress.
* Indicates χ 2 are statistically significant (p < 0.001).
The adequacy of this model can also be determined in relation to its parameter estimates. All the factor loadings for the general factor of ED were high, positive, and statistically significant (p <0.001) (see Table 2). Further inspection of the factor loadings for each of the grouping factors (Concern, Anxiety, Sadness, Depression) provides critical information regarding the appropriateness of including these factors in the scoring scheme of the PED. Reise et al. (Reference Reise, Moore and Haviland2010) advise that when items load strongly onto a general factor, and comparatively weaker on each of the grouping factors, this provides overwhelming support for consideration of a unidimensional scoring scheme. Alternatively when items load as strongly, or more strongly, onto each of the respective grouping factors as they do on the general factor, creation of subscales is then appropriate.
Table 2. Standardized and unstandardized factor loadings (and standard errors) for each Profile of Emotional Distress item on the Emotional distress factor
All factor loadings are statistically significant (p < 0.001).
As outlined in Table 3, factor loadings for each grouping factor were markedly lower compared to the general ED factor with a number of items displaying non-significant loadings on their respective grouping factors. These results demonstrate that there is little value in considering the distinct grouping factors as substantively meaningful and creation of subscales based on these four factors should be avoided. The PED is best conceptualized as a unidimensional measure of ED, once the effects of item heterogeneity have been controlled for.
Table 3. Standardized and unstandardized factor loadings (and standard errors) for the four grouping factors of the Profile of Emotional Distress
Factor loadings are statistically significant: * p< 0.01, ** p< 0.001.
Composite reliability
The use of traditional measures of internal reliability such as Cronbach's α have been criticized within a latent variable modelling context given the propensity to over- or under-estimate scale reliability (see Raykov, Reference Raykov1998). In order to provide a more rigorous assessment of the internal reliability of the PED the current study investigated the composite reliability of the measurement properties of the scale. Composite reliability was calculated using the formula
\begin{equation*}
p_c = \frac{{\left( {\sum\nolimits_{i = 1}^m {\lambda i} } \right)^2 }}{{\left( {\sum\nolimits_{i = 1}^m {\lambda i} } \right)^2 + \left( {\sum\nolimits_{i = 1}^m {\left( {\theta i} \right)} } \right)}},
\end{equation*}where ρc is the reliability of the factor score, λi is the standardized factor loading, and θi is the standardized error variance. Values >0.60 are generally considered acceptable (Bagozzi & Yi, Reference Bagozzi RP1988; Diamantopoulos & Winklhofer, Reference Diamantopoulos and Winklhofer2001). The results show that the ED factor exhibited excellent composite reliability (ρc = 0.98). In contrast, the composite reliability for the four grouping factors were lower, and in the case of the Sadness and Depression factors, the reliabilities were unacceptably low (Concern, ρc = 0.66; Anxiety, ρc = 0.80; Sadness, ρc = 0.25; Depression, ρc = 0.55). These results provide further indications that the distinct grouping factors are of little relevance, and that the PED is best conceptualized as a unidimensional measure of ED.
Discussion
This study provided the first comprehensive assessment of the factor structure of the PED by testing five alternative models using CFA and confirmatory bifactor modelling procedures. The PED was developed to capture the qualitative distinctions between functional (concern and sadness, respectively) and dysfunctional (anxiety and depression, respectively) negative emotional responses. The absence of any reliable psychometric data meant it was unclear whether the PED was effectively capturing the hypothesized qualitative distinctions among these negative emotions, or whether an alternative factorial solution would offer a more accurate and parsimonious account of the latent structure of the scale. Given that the PED was the first scale developed in line with the binary model of emotions, and intended for use in research contexts evaluating the competing predictions of the unitary and binary models of emotions, a thorough investigation of the latent structure was indeed warranted.
Many researchers (e.g. Chen et al. Reference Chen, West and Sousa2006; Reise et al. Reference Reise, Moore and Haviland2010) have argued that a significant limitation of factor analytical research is the use of a traditional one-factor model when attempting to assess unidimensionality. This type of model structure is rarely expected or discovered to adequately explain the covariation among the observable indicators of a scale given the necessity of using heterogeneous item sets in order to capture the diverse aspects of a single psychological variable. Using a one-factor solution as the foundational model in any comparative work is believed to be misguided. Thus, Chen et al. and Reise et al. have recommended that a bifactor model be considered a baseline model of unidimensionality given the ability of a bifactor conceptualization to model unidimensionality while also accounting for appearances of multidimensionality. The basis for this is homogeneous item sets developed to capture the diverse elements of the latent variable of interest. Bifactor modelling therefore has the capacity to determine whether these grouping factors have any statistical relevance or whether they are better conceptualized as rather unimportant method effects.
In line with these recommendations, a bifactor model conceptualization was investigated as a possible explanation of the latent structure of the PED. This model included a general factor of ED in which all 26 items load onto this factor, and four grouping factors (Concern, Anxiety, Sadness, Depression) reflecting the distinct item sets. Each item therefore was allowed to load onto the ED factor and its respective grouping factor. This bifactor model emerged as the only viable factorial solution exhibiting acceptable model fit values across all fit indices.
Inspection of the model parameters provided considerable evidence for a unidimensional conceptualization of the PED. All 26 items loaded strongly onto the ED factor, with the majority of items displaying factor loadings in excess of 0.60 thus generally satisfying the strict criteria outlined by Hair et al. (Reference Hair, Anderson, Tatham and Black1998). By contrast, factor loadings for each of the four grouping factors were consistently low, with a number of items not reaching the level of statistical significance. These results provide unequivocal evidence that a large proportion of the variation within each observable indicator is attributable to a single ED latent variable, rather than as a result of any of the four grouping factors. It is therefore strongly recommended that on the basis of current results the PED be considered a unidimensional measure of ED, and that researchers avoid the construction of subscales in the scoring of the PED in subsequent research efforts.
In order to provide a thorough evaluation of the reliability of the PED, composite reliability analysis was conducted. The ED factor was found to possess extremely good internal reliability while the four grouping factors displayed noticeably lower reliability values, and in the case of both the Sadness and Depression factors, reliability was poor. These results provide further indication that within the current sample, the development of subscales is unwarranted.
The current analysis was conducted within a specialized sample and therefore current results may not be widely generalizable. Future studies will need to retest the factor structure of the PED among more diverse population groups in order to develop a more robust picture of the true underlying latent structure of this measure. It should be noted, however, that models 2 and 4 were found to be an almost adequate fit and therefore should still be considered alongside the bifactorial model as potential model solutions in future studies.
In conclusion, the current study provides initial evidence of the underlying factor structure of the PED and suggests that the PED is best conceptualized as a unidimensional measure of ED which includes four grouping/method factors that exist due to item heterogeneity. These findings indicate that the PED is not a valid method of capturing the qualitative distinctions between functional and dysfunctional negative emotions as described in REBT theory and its use is therefore questioned when investigating predictions of the binary model of emotions. However, the PED does appear to be a valid measure of ED, possessing excellent internal reliability, and of good practical value given its short length and ease of completion.
Declaration of Interest
None.
Learning objectives
(1) To understand the impact of the binary model of emotions in the interpretation of the Profile of Emotional Distress (PED).
(2) To determine the factor structure of the PED, a novel measure of emotional distress associated strongly with Rational Emotive Behaviour Therapy (REBT). Identification of factors may help support the argument for a binary model of emotion regarding interpretation of the PED.
(3) To investigate the internal consistency of the PED through the use of composite reliability, a more statistically sophisticated approach than traditional measures such as Cronbach's α.
Comments
No Comments have been published for this article.