Recent estimates of the extent of mental illness revealed that about one in three individuals meet criteria for the diagnoses of at least one mental health disorder at some point of their lives (WHO International Consortium in Psychiatric Epidemiology, 2000). Studies have suggested that in a one-year time span at least one in five people is likely to be diagnosed with a mental disorder. Taking into account the wide spectrum of psychopathologies and the impact these have on populations’ health and functionality, several surveys and empirical reviews estimated that anxiety and mood disorders (namely unipolar major depression) are among the most common mental disorders, with a lifetime prevalence of about 25% of the general population (Antony & Swinson, Reference Antony and Swinson1996). Even though depression and anxiety tend to be under-diagnosed (e.g., Gwynn et al., Reference Gwynn, McQuistion, McVeigh, Garg, Frieden and Thorpe2008), they are among the most prevalent diagnoses in clinical admissions and treatments (Ng et al., Reference Ng, Trauer, Dodd, Callaly, Campbell and Berk2007; Page, Hooke, & Morrison, Reference Page, Hooke and Morrison2007).
According to the Eurobarometer survey conducted in 2006 (Eurobarometer, 2007), chronic anxiety and depression tend to represent about 10% of the reasons why European citizens undergo a long-term medical treatment. In Portugal, this prevalence rises up to 17%, with 13% of the interviewed subjects reporting that they had or have chronic anxiety and/or depression. The first Portuguese mental health epidemiological study revealed that almost 23% of the individuals in the study reported a diagnosable mental illness during the previous year (Caldas de Almeida, Reference Caldas de Almeida2010).
Several measures, models and classifications have been presented with the purpose of explaining and contributing to the diagnosis of depressive and anxious conditions and symptoms. The challenge is that depression and anxiety are usually defined as distinct at the conceptual, terminological and phenomenological levels, but empirical overlap has been observed in clinical and research settings (Lovibond & Lovibond, Reference Lovibond and Lovibond1995b). This overlap has been observed in self-reported data and among clinicians’ ratings of depression and anxiety (see Watson et al., Reference Watson, Weber, Assenheimer, Clark, Strauss and McCormick1995) resulting in several theoretical and empirical studies to explain this apparent comorbidity or co-occurrence (Antony, Bieling, Cox, Enns, & Swinson, Reference Antony, Bieling, Cox, Enns and Swinson1998; Beck & Perkins, Reference Beck and Perkins2001; Widiker & Coker, Reference Widiger, Coker, Hersen and Turner2003).
According to Clark and Watson (Reference Clark and Watson1991) the apparent comorbidity of depression and anxiety appears to reflect some level of mixed symptomatology, as well as an evident overlap of syndromes, which concurred to the need for a “new” diagnostic category of mixed anxiety-depression, presently contemplated in a DSM-IV appendix. Such framework suggests the existence of a common (nonspecific) general distress construct, but also assumes the distinct phenomena of each disorder. This tripartite model proposes that depression and anxiety symptoms should be divided in three factors. First, a general distress or negative affectivity factor underlies symptoms that are commonly experienced by both depressed and anxious individuals (e.g. distress, irritability) and cannot contribute to its discrimination. Second, a depression factor is best expressed by low, or absence of, positive affect and symptoms of anhedonia. It is important to appreciate that the tripartite models posits that negative affect and positive affect are independent of each other (i.e., uncorrelated). Although intuitively it may seem that this is unlikely to be the case, empirical examination of measures of positive and negative affect indicate that they are independent or at least quasi-independent (Crawford & Henry, Reference Crawford and Henry2004). Third, symptoms of somatic tension and physiological hyperarousal are relatively specific to an anxiety factor. A comprehensive assessment of these constructs provides the identification of clusters of common and unique elements of these syndromes (Clark & Watson, Reference Clark and Watson1991) and contributes to the explanation of the overlap between anxiety and depression (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998).
One of the existent psychological instruments that aims to achieve better discrimination of depression and anxiety is the Depression, Anxiety and Stress Scales (DASS; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b). Several authors (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998; Brown, Chorpita, Korotitsch, & Barlow, Reference Brown, Chorpita, Korotitsch and Barlow1997; Lovibond, Reference Lovibond1998; Norton, Reference Norton2007) argued that these scales are fairly consistent with some of the constructs proposed by Clark and Watson (Reference Clark and Watson1991), namely the low positive affect and physiological hyperarousal dimensions. The objective of the DASS development was to create a self-report measure that could differentiate adequately anxiety from depression, and cover the full range of its core symptoms (Lovibond & Lovibond, Reference Lovibond and Lovibond1995a). Initially, items were defined in terms of clinical consensus and redefined empirically. No external criteria was taken into consideration, which resulted in the exclusion of some symptoms usually applied in the depression diagnosis according to the DSM-IV-R, namely changes in appetite, sleep disturbance, guilt, tiredness, loss of concentration, loss of libido, crying and irritability (Crawford & Henry, Reference Crawford and Henry2003; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b). A third factor emerged during the scale development, comprising non-discriminating anxiety and depression items, such as tension, nervous arousal, irritability, difficulty relaxing and agitation. This third scale was labelled stress due to the notorious similarity of this cluster of symptoms to the nature of stress as suggested by Selye (Reference Selye1974).
The DASS is a commonly used measure designed to assess the unique and unrelated symptoms of depression and anxiety, as well as a third syndrome – stress –, among clinical and non-clinical populations. Two versions of these scales are available: a full (42 items – three 14 item scales) and a short (21 items – three 7 item scales) version. DASS-21 items were selected from the full version of the DASS, on basis of assuring that all scales subcomponents were considered and evaluated (Lovibond & Lovibond, Reference Lovibond and Lovibond1995b, p. 339), specifically: (a) depression: dysphoria, hopelessness, devaluation of life, self-deprecation, lack of interest/involvement, anhedonia and inertia; (b) anxiety: autonomic arousal, skeletal musculature effects, situational anxiety and subjective experience of anxious affect; and (c) stress: difficulty relaxing, nervous arousal, agitation, irritability and impatience. Clinical diagnosis was not the original purpose, but the scale developers stated that the DASS might contribute to the identification of a locus of emotional disturbance as part of a broader clinical assessment and diagnosis, albeit these negative emotional states are represented as a non categorical dimensional constructs (Widiger & Coker, Reference Widiger, Coker, Hersen and Turner2003) in the present instrument. The popularity of the DASS can be mostly attributable to its free access (public domain), shortness (Crawford & Henry, Reference Crawford and Henry2003), self-report format, existence of user-friendly scores that can be analysed/discussed with the patient (Ng et al., Reference Ng, Trauer, Dodd, Callaly, Campbell and Berk2007), and existence of a website (http://www2.psy.unsw.edu.au/groups/dass/) containing the main results/conclusions and available translations.
Empirical evidence suggested that both versions of the DASS revealed good internal consistency for the scales and for the total score (Apóstolo, Mendes, & Rodrigues, Reference Apóstolo, Mendes and Rodrigues2007; Bados et al., Reference Bados, Solanas and Andrés2005; Crawford & Henry, Reference Crawford and Henry2003; Henry & Crawford, Reference Henry and Crawford2005; Norton, Reference Norton2007; Page et al., Reference Page, Hooke and Morrison2007; Pais-Ribeiro, Honrado, & Leal, Reference Pais-Ribeiro, Honrado and Leal2004a) and presented adequate convergent and discriminant validity among clinical (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998; Apóstolo, Mendes, & Azeredo, Reference Apóstolo, Mendes and Azeredo2006; Brown et al., Reference Brown, Chorpita, Korotitsch and Barlow1997; Ng et al., Reference Ng, Trauer, Dodd, Callaly, Campbell and Berk2007) and non-clinical samples (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998; Apóstolo, Rodrigues, & Oliveira, Reference Apóstolo, Mendes and Rodrigues2007; Bados et al., Reference Bados, Solanas and Andrés2005; Brown et al., Reference Brown, Chorpita, Korotitsch and Barlow1997; Crawford & Henry, Reference Crawford and Henry2003; Henry & Crawford, Reference Henry and Crawford2005; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b; Norton, Reference Norton2007). This last psychometric property has been mainly evaluated through the examination of the correlation values with other instruments, namely BDI (Beck Depression Inventory), BAI (Beck Anxiety Inventory), PANAS (Positive and Negative Affect Schedule), HADS (Hospital Anxiety and Depression Scale) and STAI-T (trait version of the State-Trait Anxiety Inventory). Some preliminary studies demonstrated that such results are similar across racial groups (Norton, Reference Norton2007). Some studies found that these scales consistently indicated good sensitivity to patients’ scores showing changes from admission to discharge in clinical settings (Page et al., Reference Page, Hooke and Morrison2007; Ng et al., Reference Ng, Trauer, Dodd, Callaly, Campbell and Berk2007). However, a ceiling effect has been observed on the depression scale. Of the analysed patients, 89% scored the scale’s maximum and yet none scored the highest score of the BDI (Page et al., Reference Page, Hooke and Morrison2007).
Regarding the construct validity of the DASS versions, several studies have conducted exploratory (EFA) and/or confirmatory factor analysis (CFA) in order to determine which factor structure best fits their data. The overall results demonstrated that the three-factor solution tends to be the most adequate, despite what alternative models have suggested. Some authors argued that stress and negative affect can be considered synonymous, leading to the omission of the (substantive) stress scale (Tully, Zajac, & Venning, Reference Tully, Zajac and Venning2009). Others demonstrated that correlated errors between items of the same subscales (Crawford & Henry, Reference Crawford and Henry2003; Page et al., Reference Page, Hooke and Morrison2007) and/or cross-loading items (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998; Brown et al., Reference Brown, Chorpita, Korotitsch and Barlow1997; Clara, Cox, & Enns, Reference Clara, Cox and Enns2001; Page et al., Reference Page, Hooke and Morrison2007) provided a better fit to the data. Henry and Crawford (Reference Henry and Crawford2005) suggested that a quadripartite structure (general psychological distress plus orthogonal factors of depression, anxiety and stress) fitted the data better than all competing models analysed. Some authors also verified that a second-order factor model fitted the data somewhat better than a first-order factor model (Daza, Novy, Stanley, & Averill, Reference Daza, Novy, Stanley and Averill2002; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b). Finally, a two-factor model which distinguishes depression from anxiety/stress dimensions has also been suggested (Lovibond & Lovibond, Reference Lovibond and Lovibond1995b), even though only two empirical studies supported this two-factor structure (Apóstolo et al., Reference Apóstolo, Mendes and Azeredo2006; Duffy, Cunningham, & Moore, Reference Duffy, Cunningham and Moore2005).
Two Portuguese versions/translations of the DASS are available (Apóstolo et al., Reference Apóstolo, Mendes and Azeredo2006; Pais-Ribeiro, Honrado, & Leal, Reference Pais-Ribeiro, Honrado and Leal2004a, Reference Pais-Ribeiro, Honrado and Leal2004b), but none performed a CFA on their data neither evaluated the competing models of the latent structure of the DASS-21. Moreover, these studies predominantly used convenience samples of undergraduate students, which compromised the external validity of the results.
Therefore, the main aim of the present study was to assess the psychometric properties (factorial validity and reliability) of the Depression, Anxiety and Stress Scales (DASS-21) in a large Portuguese community sample.
Method
Participants
Participants were 1020 individuals randomly recruited from the northern and central regions of Portugal. The mean age of the sample was 36.74 (SD = 11.90) years, with an age range from 18 to 75 years. Of these participants, 585 (57.4%) were women and 435 (42.6%) were men. The mean years of education was 11.13 (SD = 4.15). Regarding the sample’s remuneration levels, 252 (24.7%) reported receiving one minimum monthly salary (MMS; nearly 485€), 523 (51.3%) more than one to two MMS, 164 (16.1%) more than two and three MMS, while the remaining 81 (7.9%) individuals reported incomes higher than three MMS. Data were collected using a cross sectional street intercept survey method. All participants were informed of the study’s objectives and provided an informed consent.
Instruments
Participants responded to a Portuguese version (Pais-Ribeiro et al., Reference Pais-Ribeiro, Honrado and Leal2004b) of the DASS-21 (Lovibond & Lovibond, Reference Lovibond and Lovibond1995a) using a 4-point scale from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time), with possible scores ranging from 0 to 21 for separate scales (7-items), and from 0 to 63 for the total score (21-items). The DASS asks respondents to rate the frequency/severity of experienced negative emotions over the last week. Total scores of the DASS-21 must be doubled in order to be compared to the DASS-42 scores and suggested severity ratings. This interpretation is primarily based on the cut-off values originated from percentile scores, specifically: “normal” (0–77), “mild” (78–86), “moderate” (87–94), “severe” (95–97), and “extremely severe” (98–100), which are available in the DASS manual (Lovibond & Lovibond, Reference Lovibond and Lovibond1995a).
Procedure
Eleven measurement models were specified and tested according to the several competing factor models found in the literature review (e.g. Crawford & Henry, Reference Crawford and Henry2003; Henry & Crawford, Reference Henry and Crawford2005; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b; Tully et al., Reference Tully, Zajac and Venning2009).
Model 1 concerns a simple one-factor model. Models 2 and 3 represent, respectively, a correlated (oblique) and uncorrelated (orthogonal) two-factor model which combines the anxiety and stress scales alongside with the depression scale. Models 4 and 5 comprise the correlated (oblique) and uncorrelated (orthogonal) three-factor structure suggested by Lovibond and Lovibond (Reference Lovibond and Lovibond1995b). Model 6 is based on the original three-factor model plus correlated errors between items of the same subscales. Model 7 is a second-order factor model underlying the three measured scales (factors). Model 8 is a quadripartite model that includes one general distress factor in which all items are allowed to load, plus three orthogonal factors corresponding to the original scales, as suggested by Henry and Crawford (Reference Henry and Crawford2005). Model 9 represents the same factor structure of the previous model, plus correlated errors between items of the same subscales. Models 10 (without correlated errors) and 11 (with correlated errors) are identical to Models 8 and 9, but omit the stress factor and only allow these scale’s items to load on the general factor. The last two models question the significance of the stress scale as a coherent and legitimate construct (Crawford & Henry, Reference Crawford and Henry2003; Henry & Crawford, Reference Henry and Crawford2005).
Statistical analysis
Descriptive statistics (SD and range) were calculated for all the DASS’s scores and total score. Skewness and kurtosis coefficients were computed for univariate normality analysis purposes. Item-scale associations and scales intercorrelations were calculated using the Pearson product-moment coefficient. The reliabilities of the measured scales and total score were estimated using the Cronbach’s alpha.
The EQS software (version 6.1) was selected to analyse the competing models of the latent structure of the DASS-21. Given that the pre-analyses of the data revealed a considerable multivariate kurtosis (Mardia’s normalized coefficient equal to 108.13), CFA was performed on the covariance matrices, using the robust maximum likelihood method of estimation. According to Curran, West, and Finch (Reference Curran, West and Finch1996) the Satorra-Bentler method (Satorra & Bentler, Reference Satorra, Bentler, von Eye and Clogg1994, Reference Satorra and Bentler2001) is a good approach for dealing with multivariate non-normality.
The evaluation of the adjustment of the tested models was assessed using several fit indices, namely: the Satorra-Bentler scaled chi-squared statistic (S-Bχ²; Satorra & Bentler, Reference Satorra and Bentler2001), the robust comparative fit index (*CFI; Bentler, Reference Bentler1990), the standardized root mean squared residual (SRMR), and the robust root mean squared error of approximation and its 90% confidence interval (*RMSEA, 90%IC; Browne & Cudeck, Reference Browne, Cudek, Bollen and Long1993). Usually, indices greater than .90 for *CFI and lower than .10 for SRMR and .08 for *RMSEA, are interpreted as indicating “acceptable” fit (Bentler, Reference Bentler1990; Bollen, Reference Bollen1989; Browne & Cudeck, Reference Browne, Cudek, Bollen and Long1993). Hu and Bentler (Reference Hu and Bentler1999) suggested more stringent cut-off values of .95 for *CFI, .08 for SRMR and .06 for RMSEA. The comparison of competing models was performed according to the scaled χ² difference test statistic provided by Satorra and Bentler (Reference Satorra and Bentler2001). A computer program specifically developed for this purpose (see Crawford & Henry, Reference Crawford and Henry2003, p. 116) was used to test the difference between the fit of two measurement models. For this result, the normal χ² value is also required and, therefore, was also presented. Definitions, assumptions and characteristics of these indices are beyond the scope of this article and can be consulted elsewhere (e.g. Kline, Reference Kline2010).
Results
Descriptive, correlational and reliability analysis
Table 1 shows descriptive statistics (range, means and standard deviations) and univariate normality measures (skewness and kurtosis) for the DASS-21 scores and total score. The doubled values of the DASS-21 scores are also presented.
Table 1. Descriptive and univariate normality analysis of the DASS-21 and doubled DASS-21 scores
Mean DASS-21 values varied between 3.13 ± 3.22 (anxiety) and 5.89 ± 3.59 (stress), while the mean total score was 12.44 ± 9.16. The doubling scores ranged from 6.27 ± 6.43 to 11.77 ± 7.17, with a mean total score of 24.88 ± 18.32. Univariate normality coefficients ranged between .878 and 1.624 for skewness and between 1.343 and 3.088 for kurtosis. These positive skewness and kurtosis values (univariate normality) concur to the obtained value of multivariate normality (Mardia’s coefficient of 108.13), and indicate some moderate departure of the normality assumption (Kline, Reference Kline2010; Schumacker & Lomax, Reference Schumacker and Lomax2004).
In order to determine the item-scale associations and scales intercorrelations, Pearson’s correlation coefficients were calculated. Within item-scale values varied from .59 to .77 for the depression scale, from .58 to .71 for the anxiety scale, and from .62 to .78 for the stress scale. All 21 items associated higher with the hypothesized scale, even though some moderate to large correlations were observed with the remaining scales. The zero-order scales intercorrelations were .70 for depression–anxiety, .67 for depression–stress, and .69 for anxiety-stress. The DASS-21 scales equally associated with the total score (r = .89).
In order to examine the scales and total score reliability, Cronbach’s alpha estimates were computed. Alpha was .84 for depression, .80 for anxiety, .83 for stress, and .92 for the total DASS-21 score.
Confirmatory factor analysis
As an initial procedure, a scale by scale CFA was performed to determine its specific construct validity before several CFA were calculated to test the hypothesized models. These analyses showed that the three one-factor models presented an excellent fit to the data: depression (*CFI = .977, *RMSEA = .040); anxiety (*CFI = .946, *RMSEA = .056); and stress (*CFI = .977, *RMSEA = .045).
In Table 2 are summarized the results of the eleven CFA performed to examine the factorial structure underlying the DASS-21 responses.
Table 2. Summary of fit indices from CFA
Note:
All χ² and S-Bχ² values are significant at p < .001.
The results revealed that Models 1, 2, 3 and 5 were unable to fit the data, being the *CFI and the SRMR the most penalizing indices. The poorest fit indices were observed for Models 3 and 5 which represented orthogonal two and three-factor structures, respectively. Regarding the hypothesis that the DASS-21 measures a single factor (Model 1), the various fit indices also demonstrated that such a premise was untenable. The three-factor structure suggested by Lovibond and Lovibond (1995) and its nested second-order model led to an improved, however, only adequate fit to the data (Models 4 and 7, correspondingly). The original three-factor model produced the following correlations between factors: .86 for depression-anxiety, .77 for depression-stress, and .84 for anxiety-stress. In addition, a test of an “alternative” three-factor model, plus correlated errors between items of the same subscales (Model 6), revealed a better fit to the data, ΔS-Bχ²(6) = 25.48, p < .001 than the previous models, even though the lower *CFI value suggested the possibility of an improvement in fit.
Hence, the following factor structures were examined in order to determine if the intercorrelations between the DASS-21 scales could be accounted by an underlying general distress factor (Models 8 and 9) and to test if the stress scale constitutes a legitimate construct (Models 10 and 11). When the differences between models with or without correlated errors between items of the same subscales (Model 8 vs 9 and 10 vs 11) were computed, results revealed that allowing correlated errors did not lead to an improvement of fit for the first comparison, ΔS-Bχ²(6) = 11.59, p > .05, although Model 11 indicated a fairly better fit than Model 10, ΔS-Bχ²(6) = 28.00, p < .001. However, when the maintenance or exclusion of the stress factor was compared (Model 8 vs 10 and 9 vs 11), it was possible to verify that the removal of the stress dimension was associated with a significantly poorer fit to the data, ΔS-Bχ²(7) = 89.75, p < .001, and ΔS-Bχ²(7) = 85.17, p < .001, respectively. Additionally, S-Bχ² difference tests demonstrated that the quadripartite models (8 and 9) were statistically better-fitting than Model 6, which was the most adequate three-factor model, ΔS-Bχ²(12) = 71.83, p < .001, and ΔS-Bχ²(18) = 88.32, p < .001, respectively.
Thus, it is possible to verify that Models 8 and 9 represented a better, and somewhat equivalent, fit to the data. However, a more close inspection of the Model 9 output revealed that only one (item 1 – item 12) of the six imposed correlated errors was significant (p < .05) and several correlations were negative, albeit weakly so. Therefore, we consider that Model 8 was the best-fitting model (*CFI = .940, *RMSEA = .038) based on parsimony and clarity criteria, demonstrating that depression, anxiety and stress represent legitimate and consistent constructs, although they share a substantial latent component of general psychological distress (or negative affectivity).
The standardized factor loadings for the general and specific factors are presented in Table 3. A graphical representation of this model can be consulted elsewhere (Henry & Crawford, Reference Henry and Crawford2005, p. 235).
Table 3. Standardized factor loadings for Model 8
All items loaded higher on the general distress factor (M = .581) than on its specific factor (M = .324), with loading values ranging from .380 to .747 for the general factor and from .219 to .472 for the specific factor they intended to represent. Concerning the loading values for the specific scales, the items of the anxiety dimension revealed lower mean values (M = .289) than the depression (M = .333) and stress (M = .349) related items.
Discussion
The present study was conducted in a large Portuguese community sample with the aim of assessing the factor structure and reliability of depression, anxiety and stress scales (DASS-21). To the best of our knowledge, this is the first study to investigate the psychometric properties of these scales in Portuguese samples, using a CFA approach.
Initial reliability and one-scale factor models demonstrated that these analysed scales provide reliable and valid measures of the related syndromes. Cronbach’s alpha was equal or higher than .80 for all dimensions, which is similar to previous studies using Portuguese versions (Apóstolo et al., Reference Apóstolo, Mendes and Azeredo2006; Apóstolo, Mendes, et al., Reference Apóstolo, Mendes and Rodrigues2007; Apóstolo, Rodrigues, & Oliveira, Reference Apóstolo, Mendes and Rodrigues2007; Apóstolo, Ventura, Caetano, & Costa, Reference Apóstolo, Ventura, Caetano and Costa2009; Pais-Ribeiro et al., Reference Pais-Ribeiro, Honrado and Leal2004b) or other linguistic versions (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998; Bados et al., Reference Bados, Solanas and Andrés2005; Henry & Crawford, Reference Henry and Crawford2005) of the DASS-21. These reported values tend to be lower than the internal consistency estimates of the full version (DASS-42), since alpha values are affected by underlying number of items per scale. Some authors argue that for some applied settings (e.g., individual assessment purposes), recommended reliability values of the scales should be higher than .80 (e.g., Abell, Springer, & Kamata, Reference Abell, Springer and Kamata2009; Nunnally & Bernstein, Reference Nunnally and Bernstein1994). Therefore, our results suggest that the depression, anxiety and stress scales measured by the DASS-21 can be used in separate or combined forms in order to contribute to the broader clinical assessment of such syndromes. Originally, it was not designed as a self-report diagnostic instrument since, for instance, several symptoms usually applied in the depression diagnosis were omitted (Crawford & Henry, Reference Crawford and Henry2003; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b). A ceiling effect has been observed in the depression scale (Page et al., Reference Page, Hooke and Morrison2007), but there is empirical evidence that its use in clinical settings is appropriate. It has been demonstrated that these scales provide a valid routine clinical outcome measure, since they present good sensitivity to changes in clinical diagnosis from admission to discharge (Ng et al., Reference Ng, Trauer, Dodd, Callaly, Campbell and Berk2007).
One of the main aims of the DASS’s development was to create self-report measures that could contribute to the differentiation of anxiety and depression, covering the full range of its core symptoms (Lovibond & Lovibond, Reference Lovibond and Lovibond1995a, Reference Lovibond and Lovibond1995b). In the present study, the scales intercorrelations varied between .67 for depression–stress and .70 for depression–anxiety, indicating a common shared variance of about 49% for depression and anxiety measures. Even though such findings are similar to some of the previous studies (e.g. Apóstolo, Mendes et al., Reference Apóstolo, Mendes and Rodrigues2007, Reference Apóstolo, Ventura, Caetano and Costa2009; Norton, Reference Norton2007; Page et al., Reference Page, Hooke and Morrison2007), important differences should be noted. First, previous studies have reported higher correlations between stress and the remaining factors, than between depression and anxiety (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998; Brown et al., Reference Brown, Chorpita, Korotitsch and Barlow1997; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b). In our study, the highest intercorrelation was observed between depression and anxiety. Even though such differences are minor, it is possible to suggest that such (clusters of) syndromes might reveal different patterns of associations across peoples of different backgrounds and/or cultures (Good & Kleinman, Reference Good, Kleinman, Tuma and Maser1985). Therefore, variations of the discrimination between the affective/interpersonal and somatic items are expected. The knowledge and explanation of such cognitive and interpretative constructions of the emotional disorders is an important issue concerning clinical practice and inherent psychiatric assumptions (Lu, Bond, Friedman, & Chan, Reference Lu, Bond, Friedman and Chan2010). Second, the high correlation value between depression and anxiety tends to counteract the objective of developing scales that could “provide maximum discrimination between the two scales” (Lovibond & Lovibond, Reference Lovibond and Lovibond1995b, p. 336), even though a value of “+.50 may well be an irreducible minimum correlation between self-report scales designed to measure depression and anxiety” (p. 342). Our results suggested that about 50% of these scale’s variance was shared, which might express common underlying causes under the umbrella of a more general dimension of “negative affectivity”, and not necessarily overlapping clinical constructs. It might also provide some evidence for a mixed anxiety-depression classification (Clark & Watson, Reference Clark and Watson1991; Fawcett, Cameron, & Schatzberg, Reference Fawcett, Cameron, Schatzberg, Stein, Hollander and Rothbaum2010) in which the apparent comorbidity of depression and anxiety appears to reflect the common (nonspecific) phenomena of each disorder. Third, we observed substantially higher factor correlations (CFA of Model 4) than its respective scale correlations (Pearson coefficient). This is explained by the fact that the previous associations in the CFA were measured without error, while the scales intercorrelations were attenuated by measurement error and the unique variance associated with each item (Crawford & Henry, Reference Crawford and Henry2003).
Taking into consideration the empirical and theoretical arguments presented in the literature, eleven different competing models of the latent structure of the DASS-21 were submitted to CFA. Initial results demonstrated that the one and the two-factor models, as well as the three orthogonal factor solution, did not provide an adequate fit to the data, and therefore, were neglected. However, some studies (Apóstolo et al., Reference Apóstolo, Mendes and Azeredo2006; Duffy et al., Reference Duffy, Cunningham and Moore2005) have suggested that the DASS responses were best represented by two factors, but the present study did not supported such assumption. Previous literature has consistently identified the three-factor solution as the most adequate model (Antony et al., Reference Antony, Bieling, Cox, Enns and Swinson1998; Apóstolo, Mendes et al., Reference Apóstolo, Mendes and Rodrigues2007; Brown et al., Reference Brown, Chorpita, Korotitsch and Barlow1997; Clara et al., Reference Clara, Cox and Enns2001; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b; Norton, Reference Norton2007; Pais-Ribeiro et al., Reference Pais-Ribeiro, Honrado and Leal2004a, Reference Pais-Ribeiro, Honrado and Leal2004b), although few alternative competing structures have been proposed (Henry & Crawford, Reference Henry and Crawford2005). Regarding the three-factor and the second-order factor models, results indicated an adequate fit to the data, with some fit improvement when correlated errors between items of the same subscales were considered. However, both the *CFI and *RMSEA suggested that a greater fit to the data could be achievable (Hu & Bentler, Reference Hu and Bentler1999; Kline, Reference Kline2010). Therefore, the following tests addressed the question regarding the quadripartite fit model, calculated with (or without) correlated errors and with (or without) a defined stress scale. Thus, although Henry and Crawford’s (Reference Henry and Crawford2005) basic quadripartite model had the best fit, unlike these authors we did not find that allowing correlated errors significantly improved the fit of this model. Such assumption was based primarily in parsimony and clarity criteria, since the majority of the errors correlations were non-significant and its coefficient values were not in accordance to the hypothesized (i.e., negative parameter values) relationships. Several studies suggested the existence of a common factor underlying the scales constructs with limited testing of that assumption (Henry & Crawford, Reference Henry and Crawford2005; Szabo, Reference Szabó2010; Tully et al., Reference Tully, Zajac and Venning2009). Taken as a whole, our results suggested that the stress scale was a coherent and legitimate dimension, and that the quadripartite structure represents an excellent well-fitting model. These results clearly support the assumption of a common general distress factor (Clark & Watson, Reference Clark and Watson1991; Lovibond & Lovibond, Reference Lovibond and Lovibond1995b), although at a “fundamental” level, the scales of depression, anxiety and stress constitute measurable and legitimate dimensions of each specific emotional syndrome. Regarding the standardized factor loadings, it was also possible to observe higher general factor loadings than specific factor loadings, which might contribute to the reduction of the correlation between the depression and anxiety factors. Among a normative sample of Australian adolescents, Tully et al. (Reference Tully, Zajac and Venning2009) demonstrated that the correlation between these psychological dimensions was considerably reduced, when a negative affectivity factor was included in a tripartite factor structure.
Such results provide important implications for the assessment and diagnosis of mental health problems among the Portuguese population. First, the present results suggest that the DASS-21 is a reliable and valid instrument to be used among the Portuguese adult population. Moreover, this assumption can also be extended to the millions of Portuguese immigrants spread all over the world who might be at risk of suffering from some form of mental disorder. Some studies have demonstrated that the migration process can influence the immigrants’ mental well-being (e.g., Bhugra, Reference Bhugra2004). Therefore, the existence of a valid, free, short and user-friendly screening instrument is a major contribution to the identification of potential mental health problems, in order to provide adequate referrals to treatment services. It is important to emphasize that mental illness and the utilization of mental health institutions and services, namely psychiatric, is much stigmatized in the Portuguese society, which might contribute to an under-diagnosis of mental disorders. Second, taking into consideration that such identification should be grounded on the use of psychometrically sound instruments, we are hopeful that the present results are of substantial interest and utility, allowing the DASS-21 to help clinical decisions or to be used in future research studies among the Portuguese population.
Some limitations must be considered regarding our results. Generalizations require caution because our random sample is representative of the northern and centre regions of Portugal, but it is not representative of the general Portuguese population. In addition, the present study only presented information regarding the psychometric properties of a Portuguese version of the DASS-21; therefore, future studies need to develop normative data, based on representative samples of the Portuguese population. As suggested by some authors (Crawford, Garthwaite, & Slick, Reference Crawford, Garthwaite and Slick2009; Crawford, Cayley, Wilson, Lovibond, & Hartley, Reference Crawford, Cayley, Wilson, Lovibond and Hartley2011) it would be best to present such data in the form of percentile norms. The point estimates of these percentiles could be supplemented with interval estimates that quantify the uncertainty arising from using a normative sample to estimate the standing of a score in the normative population (Crawford, Garthwaite, Lawrie, et al., Reference Crawford, Garthwaite and Slick2009).
In conclusion, the results from the CFA revealed that the Portuguese version of the DASS-21 showed good factorial validity, best represented by a quadripartite model that includes a common general distress factor and three orthogonal factors (depression, anxiety and stress scales). Reliability analysis demonstrated that these scales can be used in separate or combined forms in order to contribute to the clinical assessment of such syndromes.
Future studies should continue to investigate the psychometric properties of the DASS-21 for both nomothetic (e.g., research) and ideographic (e.g., clinical) purposes.
