Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-02-06T05:49:21.632Z Has data issue: false hasContentIssue false

Exploring emotions and cognitions in hoarding: a Q-methodology analysis

Published online by Cambridge University Press:  07 May 2020

Adam Postlethwaite
Affiliation:
Sheffield Health and Social Care NHS Foundation Trust, Sheffield, UK
Stephen Kellett*
Affiliation:
University of Sheffield and Sheffield Health and Social Care NHS Foundation Trust, Sheffield, UK
Nathan Simmonds-Buckley
Affiliation:
Barnet, Enfield and Haringey Mental Health Foundation NHS Trust, UK
*
*Corresponding author. Email: s.kellett@sheffield.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Background:

The cognitions and emotions of people prone to hoarding are key components of the dominant cognitive behavioural model of hoarding disorder.

Aims:

This study sought to use Q-methodology to explore the thoughts and feelings of people that are prone to hoarding, to identify whether distinct clusters of participants could be found.

Method:

A 49-statement Q-set was generated following thematic analysis of initial interviews (n = 2) and a review of relevant measures and literature. Forty-one participants with problematic hoarding met various study inclusion criteria and completed the Q-sort (either online or offline). A by-person factor analysis was conducted and subsequent participant clusters compared on psychometric measures of mood, anxiety, hoarding and time taken on the online task as proxy for impulsivity.

Results:

Four distinct participant clusters were found constituting 34/41 (82.92%) of the participants, as the Q-sorts of n = 7 participants failed to cluster. The four clusters found were ‘overwhelmed’ (n = 11 participants); ‘aware of consequences’ (n = 13 participants); ‘object complexity’ (n = 6 participants) and ‘object–affect fusion’ (n = 4 participants). The clusters did not markedly differ with regard to hoarding severity, anxiety, depression or impulsivity.

Conclusions:

Whilst the participant clusters reflect extant research evidence, they also reveal significant heterogeneity and so prompt the need for further research investigating emotional and cognitive differences between people prone to hoarding.

Type
Main
Copyright
© British Association for Behavioural and Cognitive Psychotherapies 2020

Introduction

The cognitive behavioural model of hoarding disorder (HD) (Steketee and Frost, Reference Steketee and Frost2003) describes how information-processing deficits, problematic beliefs about and emotional attachments to possessions and both cognitive and behavioural avoidance of discarding interact to create and then maintain the observable and overwhelming levels of clutter evident in problematic hoarding. However, a review of the role that emotions play in hoarding found that this component of the cognitive behavioural model was the least researched and therefore understood (Kellett and Holden, Reference Kellett, Holden, Frost and Steketee2014). The emotions experienced towards the often vast and undifferentiated arrays and piles of objects in hoarding can be both positive and negative (Mogan et al., Reference Mogan, Kyrios, Schweitzer, Yap and Moulding2012). In terms of negative feelings, when people prone to hoarding are requested to sort personal possessions in laboratory tasks, then strong feelings of anxiety and sadness become apparent (Grisham et al., Reference Grisham, Norberg, Williams, Certoma and Kadib2010). Clearly, emotions do not occur in a vacuum and so cognitions (e.g. high need for control over possessions, responsibility assumptions and beliefs that objects function as memory aids) can also drive hoarding behaviour (Frost et al., Reference Frost, Gabrielson, Deady, Dernbach, Guevara, Peebles-Dorin, Yap and Grisham2018).

Yap and Grisham (Reference Yap and Grisham2019) recently reviewed and ordered the evidence for the emotional attachment to objects in hoarding into five main areas: (a) object attachment (Kyrios et al., Reference Kyrios, Mogan, Moulding, Frost, Yap and Fassnacht2018), (b) objects functioning as symbols of comfort and safety (Przeworski et al., Reference Przeworski, Cain and Dunbeck2014), (c) anthropomorphism of objects (Timpano and Shaw, Reference Timpano and Shaw2013), (d) possessions functioning as extensions of self (Dozier et al., Reference Dozier, Taylor, Castriotta, Mayes and Ayers2017) and (e) reliance on possessions to trigger assumed lost autobiographical memories (Cherrier and Ponnor, Reference Cherrier and Ponnor2010). This high emotional attachment to possessions has been linked to the elevated levels of anxiety and depression found amongst people with problematic hoarding (Coles et al., Reference Coles, Frost, Heimberg and Steketee2003; Frost et al., Reference Frost, Steketee and Tolin2011). The evidence overall therefore suggests that hoarding-specific cognitions and deficits in emotion regulation (i.e. the ability to manage emotions via use of adaptive strategies) may play a role in problematic hoarding. Fernandez de la Cruz et al. (Reference Fernandez de la Cruz, Landau, Iervolino, Santo, Pertusa, Singh and Mataix-Cols2013) have found that individuals with problematic hoarding (with and without comorbid obsessive compulsive disorder) reported greater difficulties forming goals and also finding strategies for managing emotions compared with controls. Tolin et al. (Reference Tolin, Levy, Wootton, Hallion and Stevens2018) reported that even after controlling for baseline depression, anxiety and stress, people prone to hoarding had greater general emotion regulation difficulties compared with controls. Taylor et al. (Reference Taylor, Theiler, Nedeljkovic and Moulding2019) used qualitative methods to suggest that difficulties with emotion regulation were associated with both the onset and maintenance of hoarding, with the acquisition and possession of objects having a role in the management of emotion.

A research method well suited to the investigation of cognitions and emotions of people prone to hoarding would be that of Q-methodology (Stephenson, Reference Stephenson1935), which is a ‘qualiquantological’ method (Stenner and Stainton-Rogers, Reference Stenner, Stainton-Rogers, Tod, Nerlich, McKeown and Clark2004). The methodology accesses the individual opinions of participants on a particular, focused topic and uses a ‘by-person’ factor analytic approach to then identify clusters of individuals sharing common viewpoints (Watts and Stenner, Reference Watts and Stenner2005). This study is the first attempt to apply Q-methodology to people prone to hoarding, in the attempt to better understand the manner in which they think and feel about their possessions.

Method

Phase 1: Q-set generation

Ethical approval for this research study was obtained from the University of Sheffield ethics committee (reference no. 012409). The research instrument in Q-methodology is called a Q-set: emblematic, distinct and representative statements of the area under investigation drawn from academic literature, initial interviews and pilot studies (Watts and Stenner, Reference Watts and Stenner2005). A typical Q-set consists of between 40 and 80 statements (Stainton-Rogers, Reference Stainton-Rogers, Smith, Harre and Van Langenhove1995). For the current study, the Q-set was generated from interviews with a clinician and a patient with HD diagnosis and from a review of relevant research and also HD assessment measures. A panel of n = 3 trainee clinical psychologists then coded the interview transcripts and reviewed the relevant HD assessment measures (Coogan and Herrington, Reference Coogan and Herrington2011). A panel meeting agreed on statements that had blanket consensus. A total of 166 statements were prospected (156 from interviews and 10 from HD measures). Statements were then reduced to the 49 most relevant statements (i.e. 46 interview statements and three measure items). Statements were reduced to 49 due to the choice of pre-arranged frequency distribution for the subsequent Q-sort (Watts and Stenner, Reference Watts and Stenner2012). These items were reviewed by two clinicians with research and clinical experience with HD. The two domain experts (Valenta and Wigger, Reference Valenta and Wigger1997) rated each of the potential Q-set items in terms of relevance (i.e. 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant). Scores were then reduced to a dichotomy (Davis, Reference Davis1992) of either relevant (and received a score of 3 or 4) or not relevant (and received a score of 1 or 2). Acceptable levels of inter-rater agreement range from 0.70 to 0.80 (Selby-Harrington et al., Reference Selby-Harrington, Mehta, Jutsum, Riportella-Muller and Quade1994) and the level of inter-rater agreement was 0.76. The content validity index (CVI) was then used to calculate the proportion of statements rated as either ‘quite relevant’ or ‘highly relevant’ (Polit and Beck, Reference Polit and Beck2006). The CVI was 0.71, with a level of 0.80 proposed as being desirable (Polit and Beck, Reference Polit and Beck2006).

Phase 2: Q-sorting

The Q-sort task was completed by a participant sample (either online via Qualtrics or offline) and involved sorting the 49-item Q-set. The answer sheet used forced the Q-sort into a quasi-normal distribution shape (Watts and Stenner, Reference Watts and Stenner2012). Participants sorted the randomly shuffled statements along a 7-point scale. Each point could only house a specific number of statements: strongly disagree (3); disagree (5); slightly disagree (9); neither agree nor disagree (15); slightly agree (9); agree (5); and strongly agree (3).

Phase 3: by-person factor analysis

PQMethod was used for the analysis (Watts and Stenner, Reference Watts and Stenner2005) and consisted of an initial pairwise intercorrelation of individual Q-sorts to generate a by-person correlation matrix. A factor analysis was then undertaken to identify an optimal model of factors (Preacher et al., Reference Preacher, Zhang, Kim and Mels2013). Factors were only chosen if the eigenvalue was >1.00 and had at least two Q-statements loading on it alone; these were seen as factor exemplars (Brown, Reference Brown1996; Watts and Stenner, Reference Watts and Stenner2005).

Participants

The inclusion and exclusion criteria for the study were: (1) for a participant to self-identify as having problems with hoarding and (2) that Q-sorts would be excluded from the factor analysis if the participant was beneath the clinical cut-off score on both hoarding measures (see ‘Measures’ section below). As Q-methodology is not designed for hypothesis testing, the study was therefore not subject to sample size estimation, but there is some guidance as to the size of sample required. Webler et al. (Reference Webler, Danielson and Tuler2009) and also Cairns (Reference Cairns2012) recommended sample sizes of between 12 and 40, whilst samples of 30–60 participants have also been recommended (McKeown and Thomas, Reference McKeown and Thomas2013; Stainton-Rogers, Reference Stainton-Rogers, Smith, Harre and Van Langenhove1995). In practice, Q-method samples rarely exceed n = 50 (Brown, Reference Brown1993). The number of participants is typically smaller than the Q-set (Brouwer, Reference Brouwer1999). The initial sample size for the current study was n = 44. Offline participants were recruited via hoarding support groups (n = 10). The online version was distributed via national charities, on social media websites and through hoarding support forums. A total of n = 79 participants began the online version, with n = 34 (43%) completing.

Measures

Clutter Image Rating (CIR)

This pictorial measure indexes the extent of clutter within the home. It includes nine photographs for each of three rooms (kitchen, living room and bedroom) varying in the amount of clutter from a rating of 1 (no clutter) to 9 (severe clutter). A mean score is calculated across the three rooms, with a mean score of 4 or more being indicative of significant clutter requiring clinical attention (Muroff et al., Reference Muroff, Underwood and Steketee2014). The CIR has been shown to demonstrate good psychometric properties (Frost et al., Reference Frost, Steketee, Tolin and Renaud2008).

Saving Inventory-Revised (SI-R)

This is a 23-item self-report measure of three primary components of hoarding: difficulty discarding (7 items), compulsive acquisition (7 items) and clutter (9 items). A total score of 41 or more is indicative of caseness (Muroff et al., Reference Muroff, Underwood and Steketee2014) and the SI-R has been psychometrically validated (Frost et al., Reference Frost, Steketee and Grisham2004).

Hospital Anxiety and Depression Scale (HADS)

This is a 14-item self-report measure that detects anxiety and depression in clinical and non-clinical populations consisting of two subscales: anxiety (7 items) and depression (7 items). Total scores range from 0 to 21 and are categorised as normal (0–7), mild (8–10), moderate (11–14) and severe (15–21). Caseness is defined by a score of 8 or above on each of the anxiety and depression subscales (Bjelland et al., Reference Bjelland, Dahl, Haug and Neckelmann2002). The HADS has been shown to possess good psychometric properties (Mykletun et al., Reference Mykletun, Stordal and Dahl2001). All the items for each of the measures used were retyped to enable delivery by Qualtrics.

Decision-making

Time taken to complete the online Q-sort task and number of clicks were used as proxy measures for decision-making. Grisham et al. (Reference Grisham, Brown, Savage, Steketee and Barlow2007) have previously illustrated that (compared with community controls) people prone to hoarding have slower/more variable reaction time, increased impulsivity, and worse spatial attention.

Results

Sample characteristics

A total of 89 participants consented to take part in the Q-sort phase of the study: 79 online and 10 offline. Forty-four participants completed the study. For the online version, 45 participants (50.56%) discontinued the study during the online task. No offline participants dropped out during the task (see Fig. 1). The majority of participants were female (86%; n = 38/44). Duration of hoarding ranged from 4 to 50 years (mean = 23 years) with 86% (n = 38/44) having received a psychological intervention for their hoarding. Duration of hoarding for completers (mean = 209.21 months, median = 180) did not differ from non-completers (mean = 276.46 months, median = 240), U = 460.5, z = 1.894, p = 0.058. Completers and non-completers did not differ in terms of gender; χ2 (1) = 3.084, p = 0.079, HADS scores (U = 361.0, z = –0.209, p = 0.834), SI-R total score (U = 478.0, z = 0.899, p = 0.368) and CIR scores (U = 473.0, z = 1.200, p = 0.230).

Figure 1. Flow chart depicting participant retention and drop-out.

For the 44 participants that fully completed the study, HADS scores for the anxiety subscale ranged from 3 to 19 (mean = 11.62, SD = 4.14) with 36 participants (81.8%) meeting caseness for anxiety. Depression subscale scores ranged from 1 to 19 (mean = 10.78, SD = 4.23) with 35 participants (79.5%) meeting caseness for depression. Mean scores for the CIR ranged from 1.67 to 7.00 (mean = 3.97, SD = 1.49) with 32 participants (72.7%) meeting clutter caseness. Total SI-R scores ranged from 32 to 76 (mean = 57.52, SD = 12.28), with 40/44 participants (90.9%) meeting hoarding caseness on the SI-R. Three online participants did not meet caseness for hoarding on either of the hoarding measures and so were removed from the dataset for the subsequent Q-sort analysis. This meant that 41 Q-sorts were included in the factor analysis.

Analysis of Q-sort data

Analysis of the unrotated factors indicated 12 factors with eigenvalues of greater than 1, which explained 79% of the variance. Exploration of the first eight factors revealed that only two factors had two or more factor exemplars. A varimax rotation was conducted and a four-factor model was chosen as having the highest verisimilitude. The four factors explained 49% of the variance and Q-sort loadings for each of the factors are presented in Table 1. Thirty-four of the 41 Q-sorts (82.93%) were found to load significantly onto any single factor and were therefore classified as factor exemplars. This translated into four participant clusters consisting of n = 11 participants (factor 1), n = 13 (factor 2), n = 6 (factor 3) and n = 4 (factor 4). Correlations between factors were low and ranged between 0.0093 and 0.3665. Seven participants were excluded from analyses on the psychometric measures, as they failed to load onto any single factor, nor constitute a factor. Analysis indicated that 46 of the 49 (93.88%) statements in the Q-sort significantly discriminated between clusters. Statements 24 (cognition; ‘objects are predictable and are not able to let you down like people might’), 28 (cognition; ‘other people get frustrated by my hoarding’) and 44 (emotion; ‘I am often torn between needing to discard items and thinking they are still useful’) did not significantly distinguish between clusters. Factor arrays representing the viewpoints of each cluster are presented in Table 2, with Z-scores and Q-sort values demonstrating the levels of agreement between the participants within each factor.

Table 1. Q-sort loadings for each factor – asterisk denotes a participant

* Factor exemplars.

Table 2. Factor arrays showing both Q-sort values (Q-SV) and Z-scores (Z)

* Distinguishing statements at p < .0186.

denotes the three Q-statements that did not significantly distinguish between the clusters.

Two statements were found to statistically distinguish each factor from all other factors (p < .01). Statement 21 (emotion) ‘letting go of an item feels like letting a part of me go’ was rated differently by each cluster. Factor 2 participants strongly disagreed with this statement, whereas factor 4 participants strongly agreed with the statement. Factor 1 participants slightly agreed, whereas factor 3 participants slightly disagreed. Statement 33 (cognition) ‘my anxiety causes me to postpone addressing my hoarding’ was also rated consistently differently by each cluster. Factor 4 participants strongly disagreed with this statement, whereas factor 3 participants only slightly disagreed, factor 2 participants slightly agreed, and factor 1 participants agreed.

Factor 1: the ‘overwhelmed’ cluster (n = 11)

This cluster of participants was represented by 11 factor exemplars that explained 15% of the variance (see Table 2). The majority of the 11 participants (63.6%) completed the study online. All 11 participants met caseness for anxiety and seven (63.6%) met caseness for depression. All met hoarding caseness on the SI-R, with 8 (72.7%) also meeting clutter caseness on the CIR. ‘Overwhelmed’ participants strongly agreed (i.e. Q-sort value scores of +3) with statements that ‘thinking about discarding my possessions causes me to feel distressed’ (S11; cognition), ‘discarding my possessions makes me feel distressed’ (S12; emotion), and that ‘I’m felt embarrassed about the state of my home’ (S41; emotion). There was also agreement (+2) that ‘rediscovering items refreshes positive memories attached to them’ (S4; emotion), that ‘anxiety causes me to postpone addressing my hoarding’ (S33; cognition) and that ‘I avoid discarding possessions due to finding this process stressful’ (S49; emotion). This group of participants strongly disagreed (–3) that they got ‘a sense of companionship from my possessions’ (S31).

Of the statements mentioned, statements 11 (cognition) ‘thinking about discarding my possessions causes me to feel distressed’, 33 (emotion) ‘my anxiety causes me to postpone addressing my hoarding’ and 49 (emotion) ‘I avoid discarding possessions because it is too stressful’ were found to statistically distinguish the emotionally overwhelmed cluster from the other three clusters (p < .01). Participants in other clusters tended to neither agree nor disagree with statement 11, and whereas participants in this cluster agreed with statement 49, participants in the other three clusters slightly disagreed. Further distinguishing statements included a slight disagreement with cognitive statement 17 ‘I am able to see unique features in items’, whereas participants in other clusters tended towards agreement with this statement. Participants in other clusters held shared opinions on several statements which the ‘overwhelmed’ cluster did not. Participants in the ‘overwhelmed’ cluster neither agreed nor disagreed with S10 (emotion) ‘my hoarding is destructive to my relationships’ whereas those in other clusters expressed stronger opinions. ‘Overwhelmed’ participants also neither agreed nor disagreed with S6 (cognition) ‘I think about how I could use an object in the future’ and S2 (emotion) ‘itʼs exciting when I find bargains’, suggesting that these items were not significant in their hoarding, whereas the other clusters all showed agreement across these items.

Factor 2: the ‘aware of consequences’ cluster (n = 13)

This cluster of participants was best represented by 13 factor exemplars that explained 16% of the variance (see Table 2). The majority (76.9%) completed the study online. Eleven participants (84.6%) met caseness for anxiety and all met caseness for depression. All met SI-R caseness for hoarding and 12 (92.3%) met clutter caseness on the CIR. This cluster had the highest mean score on both the SI-R clutter and excessive acquisition subscales. The ‘aware of consequences’ cluster participants strongly agreed (+3) ‘fearing what will happen if someone comes to my home?’ (S18; emotion), with ‘hoarding is destructive to my relationships’ (S10; emotion) and questioned ‘why I have so much stuff’ (S9; cognition). This cluster also agreed (+2) that ‘I worry that others think I’m disgusting’ (S16; cognition), that ‘itʼs exciting when I find bargains’ (S2; emotion) and that ‘I get a buzz from acquiring new things’ (S3; emotion). ‘Consequences’ participants strongly disagreed (–3) with statement 21 (cognition) ‘letting go of an item feels like letting a part of me go’. Of the statements mentioned, statements 3, 9, 10, 16, 18 and 21 were found to statistically distinguish the ‘consequences’ cluster participants from the other clusters (p < .01). This cluster contained the only participants to agree with statements 10 (cognition) ‘my hoarding is destructive to my relationships’ and 16 (cognition) ‘I worry that others think I am disgusting’. Factor 2 participants also rated stronger disagreement with statement 21 (emotion) ‘letting go of an item feels like letting a part of me go’ than did participants of the other three clusters.

Factor 3: the ‘object complexity’ cluster (n = 6)

This cluster of participants was best represented by six exemplars that explained 10% of the variance (see Table 2). The majority (83.3%) completed the study online. Four (66.7%) met caseness for anxiety, and four (66.7%) met caseness for depression. Five (83.3%) met SI-R caseness for hoarding and four (66.7%) met CIR clutter caseness. ‘Object complexity’ cluster participants strongly agreed (+3) that ‘I feel guilty about throwing items away’ (S20; emotion). They also strongly agreed with the cognition statement ‘I think about the potential that objects have’ (S7) and about ‘how I could use an object in the future’ (S6; cognition). This cluster agreed with the cognitive statement (+2) ‘I sometimes feel I am rescuing objects’ (S35), but not because ‘when an object feels sad, I feel compelled to rescue it’ (S36; emotion). This cluster strongly disagreed (–3) with emotion statement 39 ‘I feel safe when I am with my possessions’, and disagreed with cognitive statement 31 ‘I get a sense of companionship from my possessions’. They slightly agreed (+1) with the statement that ‘I feel responsibility towards objects and that if they can be used then they should’ (S37; cognitive) and that ‘I sometimes feel I am being made to discard things’ (S14; emotion). They were the only cluster that expressed any degree of agreement towards these two statements. Of the statements mentioned, statements 7, 14, 35, 36 and 37 significantly distinguished ‘object complexity’ participants from the other clusters (p < .01).

Factor 4: the ‘object–affect fusion’ cluster (n = 4)

This cluster of participants was best represented by four factor exemplars that explained 8% of the variance (see Table 2). The majority (75%) completed the study online. Three (75%) met caseness for anxiety, and three (75%) met caseness for depression. All four met SI-R caseness for hoarding and clutter caseness on the CIR. This cluster had the highest mean score on SI-R clutter scale. ‘Object–affect fusion’ cluster participants strongly agreed with the statement (+3) that ‘letting go of an item felt like letting a part of me go’ (S21; emotion), that they experienced ‘a buzz from acquiring new things’ (3; emotion), and that ‘I think about how I could use an object in the future’ (S6; cognition). This cluster shared agreement (+2) with the statements that ‘I like being around my possessions’ (S30; emotion), ‘I get a sense of companionship from my possessions’ (S31; emotion), and these participants consisted the only cluster to express any degree of agreement with the later statement. Similarly, the ‘object–affect’ cluster shared slight agreement (+1) with statements 47 (emotion) ‘my possessions provide me with emotional comfort’ and 48 (emotion) ‘I love some of my belongings the way I love some people’, whereas the other three clusters expressed varying degrees of disagreement with these statements. Participants in this cluster expressed strong disagreement (–3) with statements 1 ‘if an object looks abandoned, I will feel compelled to rescue it’, 8 ‘I find it difficult to make decisions’, and 33 ‘my anxiety causes me to postpone addressing my hoarding’. Of the statements mentioned, only statement 6 (cognition) ‘I think about how I could use an object in the future’ did not significantly distinguish ‘object–affect’ participants from those in the other clusters (p < .01).

Factor comparisons

Table 3 contains the scores on the psychometric measures and time taken on the online version. Twenty-five participants were included in the time-taken analysis, as click data were only collected for online participants (n = 34); three of these did not meet caseness for hoarding and six were not factor exemplars. Two data points were removed from the time-taken data as they were deemed to be outliers. There were no significant between-group differences found for number of clicks or time taken. Similarly, no significant between-group differences were found between the clusters in terms of hoarding severity, anxiety, depression or impulsivity. Table 4 contains the results for the caseness and cluster analyses. The relationship between cluster and depression caseness (HADS-D) was significant [χ2 (3, n = 34) = 8.017, p = 0.046]. Post-hoc examination of the adjusted standardised residuals was then conducted (García-Pérez and Vicente, Reference García-Pérez and Vicente2003). Residual scores indicated that the ‘aware of consequences’ cluster was more likely to meet caseness for depression than participants from the other clusters. However, after conducting a Bonferroni correction for multiple comparisons (α = 0.00625), the effect became non-significant. No other significant associations were found between clusters and caseness: HADS anxiety [χ2 (3, n = 34) = 5.096, p = 0.165], SI-R [χ2 (3, n = 34) = 3.616, p = 0.306] and CIR [χ2 (3, n = 34) = 4.108, p = 0.250].

Table 3. Differences between the four participant clusters in terms of psychometric measures, time taken and clicks used

* Cases missing (not collected in offline version) or removed from analysis (outliers). HADS, Hospital Anxiety and Depression Scale; SI-R, Saving Inventory-Revised.

Table 4. Differences between the four participant clusters in terms of caseness of each of the psychometric measures

* Significant at p < 0.05. HADS, Hospital Anxiety and Depression Scale.

Discussion

This has been the first attempt to explore cognitions and emotions in people prone to hoarding using Q-methodology. Analysis identified four distinct participant clusters with differing profiles of quite distinct thoughts and feelings. Within each of the clusters, participants held common and shared perspectives. The implication of this is that the personal experience of hoarding is possibly more cognitively and emotionally heterogeneous than previously considered, and that people prone to problematic hoarding can vary considerably in terms of their thoughts and feelings.

The ‘overwhelmed’ cluster appeared to exemplify the attentional deficits and organisational problems suggested to contribute to hoarding, such as indecision and categorization problems (Frost and Gross, Reference Frost and Gross1993; Frost and Hartl, Reference Frost and Hartl1996). It has been shown that amongst people that hoard, indecisiveness is correlated with the core features of hoarding (Frost et al., Reference Frost, Steketee and Tolin2011). It has been suggested that this difficulty with decision-making and the resulting tendency to avoid or postpone making decisions is based in a pronounced fear of making mistakes (Warren and Ostrom, Reference Warren and Ostrom1988). The ‘overwhelmed’ cluster reported similar tendencies; for example, they reported that they often postponed addressing their hoarding and procrastinated regarding discarding possessions, as they found this behaviour too stressful. This is in line with findings that heightened emotional attachment to a possession interacts with concerns about making an incorrect decision concerning discarding that possession, with any mistaken discarding of valued objects being experienced as a particularly aversive and traumatic event by people that hoard (Tolin et al., Reference Tolin, Kiehl, Worhunsky, Book and Maltby2009).

The ‘aware of consequences’ cluster seems to reflect that these individuals were not ego-dystonically distressed by their hoarding behaviour itself, but rather by the negative social consequences generated (Mataix-Cols et al., Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010). For example, social services may express concerns about the health hazards of cluttered home environments, and family members might become distressed and agitated concerning the clutter (Wilbram et al., Reference Wilbram, Kellett and Beail2008). The mean scores on the SI-R acquisition scale may indicate that this cluster had more of a problem with acquiring than discarding. The consequences cluster was particularly concerned about how others perceived and felt about their hoarding. Frost and Gross (Reference Frost and Gross1993) reported similar themes related to embarrassment that led to avoidance of any social contact in the homes of people prone to hoarding. Participants in the consequences cluster similarly reported fearing what would happen if someone visited their home, worrying that people would judge them as socially unacceptable. Given the prominent role of domiciliary visits in hoarding treatment (Koenig et al., Reference Koenig, Leiste, Holmes and Macmillan2014), this is a particularly useful finding.

The ‘object-complexity’ cluster was characterised by beliefs that objects were uniquely and inherently useful. This cluster felt some sense of responsibility towards objects causing them to feel guilty upon discarding them, and also feeling aggrieved if and when they are forced into discarding an object. At the same time, these participants did not derive emotional comfort from their possessions, creating a complex emotional dilemma. Steketee et al. (Reference Steketee, Frost and Kyrios2003) found that the responsibility felt towards objects was a significant component of problematic hoarding. Individuals that hoard have previously reported difficulties aligned with those of the object complexity cluster, such as not wanting to ‘waste’ an object, it as will be useful in the future and feelings of marked guilt associated with discarding objects (Frost et al., Reference Frost, Steketee, Tolin, Sinopoli and Ruby2015). Furby (Reference Furby1978) highlighted object complexity as a specific contributor to possession behaviour, suggesting that this feature was central to problematic hoarding.

Participants within the final ‘object–affect fusion’ cluster derived emotional comfort from their possessions, enjoyed being with their possessions, and felt that letting go of a possession was like letting go of a part of them. This is highly similar to the concept of objectaffect fusion proposed by Kellett and Knight (Reference Kellett and Knight2003), by which a personʼs emotions associated with an object become merged with the object itself, such that the objects become symbolic tabernacles of affective information. Frost and Gross (Reference Frost and Gross1993) found that hoarders reported higher levels of emotional attachment to their possessions than non-hoarding controls. This insight suggests that these individuals not only feel emotionally attached to objects, but experience the objects as extensions of themselves, creating a sense of ‘itme’ confusion. Yap and Grisham (Reference Yap and Grisham2019) found that possessions functioned as a store for autobiographical memories, anthropomorphism, and insecure object attachment predicted hoarding even after accounting for depression, anxiety, and other non-sentimental hoarding beliefs.

In terms of future research, this paper implies that emotion regulation difficulties may be a shared and common problem across people that hoard (Fernandez de la Cruz et al., Reference Fernandez de la Cruz, Landau, Iervolino, Santo, Pertusa, Singh and Mataix-Cols2013; Tolin et al., Reference Tolin, Levy, Wootton, Hallion and Stevens2018). Emotion regulation is the process of changing the experience (or the expression of) feelings via situation selection/modification, attentional focus, cognitive change and response modification and therefore has obvious potential influence on hoarding behaviour. A recent qualitative study conducted by Taylor et al. (Reference Taylor, Theiler, Nedeljkovic and Moulding2019) has added to the hoarding and emotion regulation evidence by highlighting that people prone to hoarding have problems with naming feelings, harbour unhelpful attitudes towards feelings, avoid feelings and carry a personal sense of being deficient in their emotion regulation strategies. Further use of the Difficulties in Emotion Regulation Scale (Gratz and Roemer, Reference Gratz and Roemer2004) is indicated to explore the manner in which individual Q-statements may correlate with the five scales (acceptance, goals, impulsivity, strategy and clarity). For example, individuals with low distress tolerance could also be in the object complexity cluster and have an exemplar Q-statement (Q11) of ‘thinking about discarding my possessions causes me to feel distressed’ or score highly on the impulsivity subscale, be in the objectaffect cluster and have a factor exemplar of (Q3) ‘I get a buzz from acquainting new things’. Greater use of qualitative and qualiquantological methods (Stenner and Stainton-Rogers, Reference Stenner, Stainton-Rogers, Tod, Nerlich, McKeown and Clark2004) for exploring emotion regulation difficulties in HD is also called for, as is the evaluation of whether CBT for hoarding can improve and broaden emotion regulation. Q-methods can be used longitudinally and therefore it would be useful to follow up treated and untreated samples over time. Future Q-methods research should also integrate a measure of compulsive acquisition into their methods, such as the Compulsive Acquisition Scale (Faraci et al., Reference Faraci, Perdighe, Del Monte and Saliani2018). Future research also needs to control for the manner in which co-occurring mental health conditions may influence Q-item endorsement. Innovations in using bootstrapping methods offer much potential in advancing Q-method analysis (Zabala and Pascual, Reference Zabala and Pascual2016).

In terms of study limitations, despite opinion being divided (Weingarden and Renshaw, Reference Weingarden and Renshaw2015), some research has suggested that hoarding can carry significant shame and stigma (Schmalisch et al., Reference Schmalisch, Bratiotis and Muroff2010), and this may have impacted on willingness to participate. Participants were recruited primarily through hoarding support groups which may have introduced bias. Whilst the clusters identified differentiated the aspects of the hoarding that were distinct for that cluster, this does not imply that these were the only concern of that cluster. The size of the clusters in the third (object complexity) and fourth (object–affect fusion) clusters were small, bringing the reliability of these clusters into doubt. However, as Q-methods often recruit samples less than n = 50 and some participants do not load onto factors, then clusters of this size should be expected (Dennis, Reference Dennis, Strickland and Waltz1988). Q-methodology requires participants to be self-aware and previous research has suggested that people that hoard often lack insight (Kim et al., Reference Kim, Steketee and Frost2001; Tolin et al., Reference Tolin, Fitch, Frost and Steketee2010). It is acknowledged that the content validity index (CVI) score for the Q-items could have been higher (Lynn, Reference Lynn1986).

Half of the online participants dropped out during the Q-sort task and this was in comparison with zero in the offline group. Q-sorting is a complex task that is more demanding than completing a questionnaire, as attention needs to be given to sorting and resorting the Q-statements. It may be the case that this felt a more attentionally demanding task (Grisham et al., Reference Grisham, Brown, Savage, Steketee and Barlow2007) when completed online and therefore participants withdrew. Whilst there is evidence that online and paper versions of Q-methods are analogous (Nazariadli et al., Reference Nazariadli, Morais, Supak, Baran and Bunds2019), this may not be the case in clinical samples. It is acknowledged, nevertheless, that the high drop-out rate limits the generalisability of the current findings. It is also worth noting that offline participants tended to be over-represented in the ‘object–complexity’ and ‘object–affect’ clusters and this calls into question whether the method of study completion affected Q-sorting.

In conclusion, the results of this study provide evidence of cognitive and emotional heterogeneity within HD. The methods were capable of categorising the experiences of a sample of people prone to hoarding into four clusters and these clusters were reflective of current hoarding theory and research. This grounding of the elicited clusters in extant theoretical concepts has helped to develop a more nuanced understanding of cognition and emotions in HD. The heterogeneity evidenced suggests that HD treatment with CBT is more likely to be effective when is it matched to both the individual and the diagnosis.

Acknowledgements

The authors thank the participants and charities that helped with the project.

Conflicts of interest

Stephen Kellett, Adam Postlethwaite and Nathan Simmonds-Buckley have no conflicts of interest with respect to this publication.

Ethical statements

This project adhered to the APA ethical principles and code of practice. Ethical approval for this research study was granted by the University of Sheffield ethics committee (reference no. 012409).

Financial support

None.

References

Bjelland, I., Dahl, A. A., Haug, T. T., & Neckelmann, D. (2002). The validity of the Hospital Anxiety and Depression Scale. An updated literature review. Journal of Psychosomatic Research, 52, 6977. doi: 10.1016/S0022-3999(01)00296-3CrossRefGoogle ScholarPubMed
Brouwer, M. (1999). Q is accounting for tastes. Journal of Advertising Research, 39, 3540.Google Scholar
Brown, S. R. (1993). A primer on Q methodology. Operant Subjectivity, 16, 91138. doi: 10.15133/j.os.1993.002Google Scholar
Brown, S. R. (1996). Q methodology and qualitative research. Qualitative Health Research, 6, 561567. doi: 10.1177/104973239600600408CrossRefGoogle Scholar
Cairns, R. C. (2012). Understanding science in conservation: a Q method approach on the Galápagos Islands. Conservation and Society, 10, 217231.CrossRefGoogle Scholar
Cherrier, H., & Ponnor, T. (2010). A study of hoarding behavior and attachment to material possessions. Qualitative Market Research, 13, 823. doi: 10.1108/13522751011013945CrossRefGoogle Scholar
Coles, M. E., Frost, R. O., Heimberg, R. G., & Steketee, G. (2003). Hoarding behaviors in a large college sample. Behaviour Research and Therapy, 41, 179194. doi: 10.1016/S0005-7967(01)00136-XCrossRefGoogle Scholar
Coogan, J. & Herrington, N. (2011). Q methodology: an overview. Research in Secondary Teacher Education, 1, 2428.Google Scholar
Davis, L. L. (1992). Instrument review: getting the most from a panel of experts. Applied Nursing Research, 5, 104107 doi: 10.1016/S0897-1897(05)80008-4CrossRefGoogle Scholar
Dennis, K. E. (1988). Q-methodology: new perspectives on estimating reliability and validity. In Strickland, O. L & Waltz, C. F (eds), Measurement of Nursing Outcomes (pp. 409419). New York, USA: Springer.Google Scholar
Dozier, M. E., Taylor, C. T., Castriotta, N., Mayes, T. L., & Ayers, C. R. (2017). A preliminary investigation of the measurement of object interconnectedness in hoarding disorder. Cognitive Therapy and Research, 41, 799805. doi: 10.1007/s10608-017-9845-xCrossRefGoogle ScholarPubMed
Faraci, P., Perdighe, C., Del Monte, C., & Saliani, A. M. (2018). Reliability, validity and factor structure of the Compulsive Acquisition Scale (CAS). Clinical Neuropsychiatry: Journal of Treatment Evaluation, 15, 4249. doi: 10.1037/t19526-000Google Scholar
Fernandez de la Cruz, L., Landau, D., Iervolino, A.C., Santo, S., Pertusa, A., Singh, S., & Mataix-Cols, D. (2013). Experiential avoidance and emotion regulation difficulties in hoarding disorder. Journal of Anxiety Disorders, 27, 204209. doi: 10.1016/j.janxdis.2013.01.004CrossRefGoogle ScholarPubMed
Frost, R. O., Gabrielson, I., Deady, S., Dernbach, K. B., Guevara, G., Peebles-Dorin, M., Yap, K., & Grisham, J. R. (2018). Scrupulosity and hoarding. Comprehensive Psychiatry, 86, 1924. doi: 10.1016/j.comppsych.2018.06.011CrossRefGoogle ScholarPubMed
Frost, R. O., & Gross, R. C. (1993). The hoarding of possessions. Behaviour Research and Therapy, 31, 367381. doi: 10.1016/0005-7967(93)90094-BCrossRefGoogle ScholarPubMed
Frost, R. O., & Hartl, T. L. (1996). A cognitive-behavioral model of compulsive hoarding. Behaviour Research and Therapy, 34, 341350. doi: 10.1016/0005-7967(95)00071-2CrossRefGoogle ScholarPubMed
Frost, R. O., Steketee, G., & Grisham, J. (2004). Measurement of compulsive hoarding: Saving Inventory-Revised. Behaviour Research and Therapy, 42, 11631182. doi: 10.1016/j.brat.2003.07.006CrossRefGoogle ScholarPubMed
Frost, R. O., Steketee, G., & Tolin, D. F. (2011). Comorbidity in hoarding disorder. Depression and Anxiety, 28, 876884. doi: 10.1002/da.20861CrossRefGoogle ScholarPubMed
Frost, R. O., Steketee, G., Tolin, D. F., & Renaud, S. (2008). Development and validation of the Clutter Image Rating. Journal of Psychopathology and Behavioral Assessment, 30, 193203. doi: 10.1007/s10862-007-9068-7.CrossRefGoogle Scholar
Frost, R. O., Steketee, G., Tolin, D. F., Sinopoli, N., & Ruby, D. (2015). Motives for acquiring and saving in hoarding disorder, OCD, and community controls. Journal of Obsessive-Compulsive and Related Disorders, 4, 5459. doi: 10.1016/j.jocrd.2014.12.006CrossRefGoogle ScholarPubMed
Furby, L. (1978). Possession in humans: an exploratory study of its meaning and motivation. Social Behavior and Personality: An International Journal, 6, 4965. doi: 10.2224/sbp.1978.6.1.49CrossRefGoogle Scholar
García-Pérez, M. A., & Vicente, N.A. (2003). Cellwise residual analysis in two-way contingency tables. Educational and Psychological Measurement, 63, 825839. doi: 10.1177/0013164403251280CrossRefGoogle Scholar
Gratz, K. L., & Roemer, L. (2004). Multidimensional assessment of emotion regulation and dysregulation: development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment, 26, 4154. doi: 10.1023/B:JOBA.0000007455.08539.94.CrossRefGoogle Scholar
Grisham, J. R., Brown, T. A., Savage, C. R. Steketee, G. & Barlow, D. H. (2007). Neuropsychological impairment associated with compulsive hoarding. Behaviour Research and Therapy, 45, 14711483. doi: 10.1016/j.brat.2006.12.008CrossRefGoogle ScholarPubMed
Grisham, J. R., Norberg, M. M., Williams, A. D., Certoma, S. P., & Kadib, R. (2010). Categorization and cognitive deficits in compulsive hoarding. Behaviour Research and Therapy, 48, 866872. doi: 10.1016/j.brat.2010.05.011CrossRefGoogle ScholarPubMed
Kellett, S., & Holden, K. (2014). Emotional attachment to objects in hoarding: a critical review of the evidence. In Frost, R. O & Steketee, G (eds), The Oxford Handbook of Hoarding and Acquiring. New York, NY, USA: Oxford University Press.Google Scholar
Kellett, S., & Knight, K. (2003). Does the concept of object-affect fusion refine cognitive-behavioural theories of hoarding? Behavioural and Cognitive Psychotherapy, 31, 457461. doi: 10.1017/S1352465803004077CrossRefGoogle Scholar
Kim, H.-J., Steketee, G., & Frost, R. O. (2001). Hoarding by elderly people. Health & Social Work, 26, 176184. doi: 10.1093/hsw/26.3.176CrossRefGoogle ScholarPubMed
Koenig, T. L., Leiste, M. R., Holmes, R., & Macmillan, K. R. (2014). Multidisciplinary teams’ practice strategies with older adult clients who hoard. Social Work in Mental Health, 12, 8197.CrossRefGoogle Scholar
Kyrios, M., Mogan, C., Moulding, R., Frost, R. O., Yap, K., & Fassnacht, D. B. (2018). The cognitive–behavioural model of hoarding disorder: Evidence from clinical and non-clinical cohorts. Clinical Psychology & Psychotherapy, 25, 311321. doi: 10.1002/cpp.2164CrossRefGoogle ScholarPubMed
Lynn, M. (1986). Determination and quantification of content validity. Nursing Research, 35, 382386.10.1097/00006199-198611000-00017CrossRefGoogle ScholarPubMed
Mataix-Cols, D., Frost, R. O., Pertusa, A., Clark, L. A., Saxena, S., Leckman, J. F., Stein, D.J., Matsunaga, H., & Wilhelm, S. (2010). Hoarding disorder: a new diagnosis for DSM-V? Depression and Anxiety, 27, 556572. doi: 10.1002/da.20693CrossRefGoogle ScholarPubMed
McKeown, B., & Thomas, D. B. (2013). Person samples and the single case. In Quantitative Applications in the Social Sciences: Q Methodology. Thousand Oaks, CA, USA: SAGE Publications. doi: 10.4135/9781483384412.n3Google Scholar
Mogan, C., Kyrios, M., Schweitzer, I., Yap, K., & Moulding, R. (2012). Phenomenology of hoarding – what is hoarded by individuals with hoarding disorder? Journal of Obsessive-Compulsive and Related Disorders, 1, 306311. doi: 10.1016/j.jocrd.2012.08.002CrossRefGoogle Scholar
Muroff, J., Underwood, P., & Steketee, G. (2014). Group Treatment for Hoarding Disorder: Therapist Guide. London, UK: Oxford University Press.CrossRefGoogle Scholar
Mykletun, A., Stordal, E., & Dahl, A. A. (2001). Hospital Anxiety and Depression (HAD) scale: factor structure, item analyses and internal consistency in a large population. British Journal of Psychiatry, 179, 540544. doi: 10.1192/bjp.179.6.540CrossRefGoogle ScholarPubMed
Nazariadli, S., Morais, D. B., Supak, S., Baran, P. K., & Bunds, K. S. (2019). Assessing the visual Q method online research tool: a usability, reliability, and methods agreement analysis. Methodological Innovations. doi: 10.1177/2059799119832194CrossRefGoogle Scholar
Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know whatʼs being reported? Critique and recommendations. Research in Nursing & Health, 29, 489497. doi: 10.1002/nur.20147CrossRefGoogle ScholarPubMed
Preacher, K. J., Zhang, G., Kim, C., & Mels, G. (2013). Choosing the optimal number of factors in exploratory factor analysis: a model selection perspective. Multivariate Behavioral Research, 48, 2856. doi: 10.1080/00273171.2012.710386CrossRefGoogle ScholarPubMed
Przeworski, A., Cain, N., & Dunbeck, K. (2014). Traumatic life events in individuals with hoarding symptoms, obsessive-compulsive symptoms, and comorbid obsessive-compulsive and hoarding symptoms. Journal of Obsessive-Compulsive and Related Disorders, 3, 5259. doi: 10.1016/j.jocrd.2013.12.002CrossRefGoogle Scholar
Schmalisch, C. S., Bratiotis, C., & Muroff, J. (2010). Processes in group cognitive and behavioral treatment for hoarding. Cognitive and Behavioral Practice, 17, 414425. doi: 10.1016/j.cbpra.2009.08.005CrossRefGoogle Scholar
Selby-Harrington, M. L., Mehta, S. M., Jutsum, V., Riportella-Muller, R., & Quade, D. (1994). Reporting of instrument validity and reliability in selected clinical nursing journals, 1989. Journal of Professional Nursing, 10, 4756.CrossRefGoogle ScholarPubMed
Stainton-Rogers, R. (1995). Q-methodology. In Smith, J. A., Harre, R. & Van Langenhove, L. (eds), Rethinking Methods in Psychology (pp. 178192). London, UK: Sage Publications.CrossRefGoogle Scholar
Steketee, G., & Frost, R. (2003). Compulsive hoarding: current status of the research. Clinical Psychology Review, 23, 905927. doi: 10.1016/j.cpr.2003.08.002CrossRefGoogle Scholar
Steketee, G., Frost, R. O., & Kyrios, M. (2003). Cognitive aspects of compulsive hoarding. Cognitive Therapy and Research, 27, 463479. doi: 10.1023/A:1025428631552CrossRefGoogle Scholar
Stenner, P., & Stainton-Rogers, R. (2004). Q methodology and qualiquantology: the example of discriminating between emotions. In Tod, Z., Nerlich, B., McKeown, S. & Clark, D. (eds), Mixing Methods in Psychology. London, UK: Routledge.Google Scholar
Stephenson, W. (1935). Correlating persons instead of tests. Journal of Personality, 4, 1724. doi: 10.1111/j.1467-6494.1935.tb02022.xCrossRefGoogle Scholar
Taylor, J. K., Theiler, S., Nedeljkovic, M., & Moulding, R. (2019). A qualitative analysis of emotion and emotion regulation in hoarding disorder. Journal of Clinical Psychology, 75, 520545. doi: 10.1002/jclp.22715CrossRefGoogle ScholarPubMed
Tolin, D. F., Fitch, K. E., Frost, R. O., & Steketee, G. (2010). Family informants’ perceptions of insight in compulsive hoarding. Cognitive Therapy and Research, 34, 6981. doi: 10.1007/s10608-008-9217-7CrossRefGoogle Scholar
Tolin, D. F., Kiehl, K. A., Worhunsky, P., Book, G. A., & Maltby, N. (2009). An exploratory study of the neural mechanisms of decision making in compulsive hoarding. Psychological Medicine, 39, 325336. doi: 10.1017/S0033291708003371CrossRefGoogle ScholarPubMed
Tolin, D. F., Levy, H. C., Wootton, B. M., Hallion, L. S., & Stevens, M. C. (2018). Hoarding disorder and difficulties in emotion regulation. Journal of Obsessive-Compulsive and Related Disorders, 16, 98103. doi: 10.1016/j.jocrd.2018.01.006CrossRefGoogle ScholarPubMed
Timpano, K. R., & Shaw, A. M. (2013). Conferring humanness: the role of anthropomorphism in hoarding. Personality and Individual Differences, 54, 383388. doi: 10.1016/j.paid.2012.10.007CrossRefGoogle Scholar
Valenta, A. L., & Wigger, U. (1997). Q-methodology: definition and application in health care informatics. Journal of the American Medical Informatics Association, 4, 501510.CrossRefGoogle ScholarPubMed
Warren, L. W., & Ostrom, J. C. (1988). Pack rats: world class savers. Psychology Today, 22, 5862.Google Scholar
Watts, S., & Stenner, P. (2005). Doing Q methodology: theory, method and interpretation. Qualitative Research in Psychology, 2, 6791. doi: 10.1191/1478088705qp022oaCrossRefGoogle Scholar
Watts, S., & Stenner, P. (2012). Doing Q Methodological Research: Theory, Method and Interpretation. London, UK: Sage Publications.Google Scholar
Webler, T., Danielson, S., & Tuler, S. (2009). Using Q method to Reveal Social Perspectives in Environmental Research. Greenfield, MA, USA: Social and Environmental Research Institute.Google Scholar
Weingarden, H., & Renshaw, K. D. (2015). Shame in the obsessive compulsive related disorders: a conceptual review. Journal of Affective Disorders, 171, 7484. doi: 10.1016/j.jad.2014.09.010CrossRefGoogle ScholarPubMed
Wilbram, M. Kellett, S., & Beail, N. (2008). Compulsive hoarding: a qualitative investigation of partner and carer perspectives. British Journal of Clinical Psychology, 47, 5973. doi: 10.1348/014466507X240740CrossRefGoogle ScholarPubMed
Yap, K., & Grisham, J.R. (2019). Unpacking the construct of emotional attachment to objects and its association with hoarding symptoms. Journal of Behavioral Addictions, 8, 249258. doi: 10.1556/2006.8.2019.15CrossRefGoogle ScholarPubMed
Zabala, A., & Pascual, U. (2016). Bootstrapping Q methodology to improve the understanding of human perspectives. PLoS ONE, 11, e0148087. doi: 10.1371/journal.pone.0148087CrossRefGoogle ScholarPubMed
Figure 0

Figure 1. Flow chart depicting participant retention and drop-out.

Figure 1

Table 1. Q-sort loadings for each factor – asterisk denotes a participant

Figure 2

Table 2. Factor arrays showing both Q-sort values (Q-SV) and Z-scores (Z)

Figure 3

Table 3. Differences between the four participant clusters in terms of psychometric measures, time taken and clicks used

Figure 4

Table 4. Differences between the four participant clusters in terms of caseness of each of the psychometric measures

Submit a response

Comments

No Comments have been published for this article.