Introduction
Posttraumatic stress disorder (PTSD) remains one of the most prevalent and serious mental health conditions experienced by military personnel. Combat-related PTSD among US combat veterans is estimated to have a point prevalence of ~2–17%, with a lifetime prevalence of ~6–31% (Richardson, Frueh, & Acierno, Reference Richardson, Frueh and Acierno2010). Given the prevalence and clinical impact of PTSD, an accurate diagnostic conceptualization of the disorder is crucial but is the subject of debate in the scientific literature (Hoge et al., Reference Hoge, Yehuda, Castro, McFarlane, Vermetten, Jetly and Rothbaum2016; O'Donnell et al., Reference O'Donnell, Alkemade, Nickerson, Creamer, McFarlane, Silove and Forbes2014). Traditionally, theoretical conceptualizations of the PTSD factor structure have been modeled using confirmatory factor analysis (CFA). However, limitations of this approach, along with a shifting paradigm in the field, have lead researchers to explore alternative approaches, such as the burgeoning technique of network analysis.
An assumption of the current diagnostic construct is that PTSD contains an underlying latent construct which is expressed through an array of symptoms and should therefore be modeled through CFA. CFA is a multivariate statistical approach to quantitative data that evaluates theoretical models containing underlying latent constructs determined from the observed variables. As such, it is the analysis of choice when validating the factor structure of a disorder. CFA research has found that the DSM-5 structure of PTSD, for example, often demonstrates poor fit (Armour, Műllerová, & Elhai, Reference Armour, Műllerová and Elhai2016) while largely confirming the structure of the International Classification of Diseases 11th Revision (ICD-11) conceptualizations of PTSD (Armour et al., Reference Armour, Műllerová and Elhai2016; Böttche et al., Reference Böttche, Ehring, Krüger-Gottschalk, Rau, Schäfer, Schellong and Knaevelsrud2018). Alternative factor structures have also been proposed (Armour et al., Reference Armour, Műllerová and Elhai2016) that range from two (Asmundson, Wright, McCreary, & Pedlar, Reference Asmundson, Wright, McCreary and Pedlar2003) to seven factors (Armour et al., Reference Armour, Tsai, Durham, Charak, Biehn, Elhai and Pietrzak2015). Although CFA has served as a helpful tool in assessing factors of PTSD and aiding in diagnostic conceptualization, the more complex models of PTSD produced using CFA are criticized for over fitting and lacking theoretical parsimony (Marshall, Schell, & Miles, Reference Marshall, Schell and Miles2013). Marshall et al. (Reference Marshall, Schell and Miles2013) postulated that due to the complexity and variability in the clinical presentation of PTSD, it is unlikely that one model will accurately and consistently characterize maladaptive responding to traumatic events. Furthermore, CFA does not provide information regarding the influence of specific symptoms.
Although CFA has been the preeminent analysis for studying the factor structure underlying the diagnostic conceptualization of PTSD, advancements in analytical approaches have coincided with the emergence of foundational debates regarding conceptualizations of psychopathology. New methodologies have allowed for the exploration of the dynamic interplay between symptoms rather than symptoms existing within static categories. This has led some researchers to suggest that CFA is inherently limiting due to its reliance on a common cause hypothesis. A common cause hypothesis posits that symptoms are the manifestation of an overarching latent construct and that the co-variations among these equally representative symptoms are reflective of that latent factor. In this way, the philosophy of psychopathology follows the ‘disease model’ of physical medicine and views psychological symptoms as consequences of an underlying disorder. However, an alternative approach is to view a disorder as the emergent property of a ‘network’ of symptoms that tend to interact in characteristic patterns (Borsboom & Cramer, Reference Borsboom and Cramer2013).
Although the common cause hypothesis has been the basis for the modern understanding of psychopathology, supported by CFA analyses, in an effort to address the shortcomings of the common cause hypothesis, recent research has shifted to network analysis. A network model of PTSD is encouraged by evidence that symptoms in PTSD are not present in a way reflective of an underlying construct. For example, researchers have identified multiple unique profiles of symptoms (Horn et al., Reference Horn, Pietrzak, Schechter, Bromet, Katz, Reissman and Herbert2016; Pietrzak et al., Reference Pietrzak, el-Gabalawy, Tsai, Sareen, Neumeister and Southwick2014), and found that certain symptom clusters have stronger associations than others (Elhai et al., Reference Elhai, Contractor, Tamburrino, Fine, Cohen, Shirley and Galea2015; Price & van Stolk-Cooke, Reference Price and van Stolk-Cooke2015; Price, Legrand, Brier, & Hébert-Dufresne, Reference Price, Legrand, Brier and Hébert-Dufresne2019), and those specific symptoms are related to different levels of impairment and distress (Resick & Miller, Reference Resick and Miller2009).
Based on partial correlation coefficients, network analysis allows for the identification of symptom relationships without seeking latent factors. Thus, instead of conceptualizing mental illness as a phenomenon caused by a latent factor, network analysis models the dynamic interplay of related symptoms as the disorder, rather than a manifestation of a larger construct. The resulting network is evaluated in terms of stability of the network as well as the ‘centrality’ of specific symptoms. Centrality is based on several metrics which reflect how integral a specific symptom is to a given network, the more central a symptom is to the network; the more an increase in that symptom will predict changes in the network as a whole. For the PTSD diagnosis, consensus regarding the most central symptoms is still being established. Extant studies broadly identify fear-based symptoms reflecting reactivity, and negative-emotionality-based symptoms as central to PTSD networks both in military populations (Armour, Fried, Deserno, Tsai, & Pietrzak, Reference Armour, Fried, Deserno, Tsai and Pietrzak2017; Phillips et al., Reference Phillips, Wilson, Sun, VA Mid-Atlantic MIRECC, Morey, Van Voorhees and Tupler2018; Ross, Murphy, & Armour, Reference Ross, Murphy and Armour2018; Segal et al., Reference Segal, Wald, Lubin, Fruchter, Ginat, Yehuda and Bar-Haim2019; von Stockert, Fried, Armour, & Pietrzak, Reference von Stockert, Fried, Armour and Pietrzak2018) and in civilians (Bartels et al., Reference Bartels, Berliner, Holt, Jensen, Jungbluth, Plener and Sachser2019; Benfer et al., Reference Benfer, Bardeen, Cero, Kramer, Whiteman, Rogers and Weathers2018; Bryant et al., Reference Bryant, Creamer, O'Donnell, Forbes, McFarlane, Silove and Hadzi-Pavlovic2017; Hoffart, Langkaas, Øktedalen, & Johnson, Reference Hoffart, Langkaas, Øktedalen and Johnson2019; McBride, Hyland, Murphy, & Elklit, Reference McBride, Hyland, Murphy and Elklit2020; McNally, Reference McNally2016; Papini, Rubin, Telch, Smits, & Hien, Reference Papini, Rubin, Telch, Smits and Hien2020).
Network analysis has been implemented not only to identify interactions within a single disorder, but across multiple disorders to observe specific comorbidity of symptoms (Boschloo et al., Reference Boschloo, van Borkulo, Rhemtulla, Keyes, Borsboom and Schoevers2015; Djelantik, Robinaugh, Kleber, Smid, & Boelen, Reference Djelantik, Robinaugh, Kleber, Smid and Boelen2020; Ross et al., Reference Ross, Murphy and Armour2018). Network analysis has also been used to demonstrate a change in one symptom predicting change in others (Greene, Gelkopf, Fried, Robinaugh, & Lapid Pickman, Reference Greene, Gelkopf, Fried, Robinaugh and Lapid Pickman2020; Hoffart et al., Reference Hoffart, Langkaas, Øktedalen and Johnson2019; Papini et al., Reference Papini, Rubin, Telch, Smits and Hien2020). This latter approach may assist in the identification of specific symptom leverage points which, when successfully reduced, have a large effect on quieting the overall network of the disorder, and are therefore priority targets during therapy.
Although CFA and network analysis differ in their respective conceptualizations of psychopathology and the utility of their findings, there are methodological limitations inherent in both approaches. One potentially important limitation in understanding symptom relationships is order effects. Self-report and clinician-administered assessments of PTSD and psychopathology, in general, are almost always administered in the same order as they are theorized to be conceptually related based on established definitions. While these groupings appear logical, analytical approaches attempting to objectively validate the relationships between these symptoms are subject to a type of confirmation bias as items close together are often endorsed similarly. Researchers argue that participants filling out these measures endorse proximal items similarly not just because of the content of items, but for reasons such as item anchoring or priming (Marshall et al., Reference Marshall, Schell and Miles2013). An understanding of the variance attributable to authentic symptom relationship and method bias is a crucial step to unlocking core symptom relationships that inform diagnostic conceptualizations and future treatment approaches.
The issue of order effects in CFA research of PTSD was first articulated by Marshall et al. (Reference Marshall, Schell and Miles2013) concluding that order effects contributed to previous findings of the inadequate fit of the DSM-IV model. Although this research points to the potential impact of order effects, the study employed a statistical examination of order effects rather than a more direct methodological approach (Witte, Domino, & Weathers, Reference Witte, Domino and Weathers2015). In an attempt to address these shortcomings, Witte et al. (Reference Witte, Domino and Weathers2015) examined the fit of PTSD models including the Detailed Assessment of Post-Traumatic Stress (Briere, Reference Briere2001), which presents PTSD symptoms in a different order. The results do not replicate Marshall et al. (Reference Marshall, Schell and Miles2013) findings and the researchers concluded that the model fit did not significantly differ based on item presentation order.
The two studies outlined above represent the best assessments to date of item order effects in the PTSD literature; however, they both contain significant limitations. Marshall et al. (Reference Marshall, Schell and Miles2013) did not vary the order of the PTSD checklist (PCL) item presentation. Witte et al. (Reference Witte, Domino and Weathers2015) used items containing vastly different wording than the PCL. Therefore, to date, no study has assessed PCL items in an order other than the order traditionally presented, whether using CFA or network analysis. Although we expect CFA and network analysis to both be vulnerable to order effects present in the survey responses, it is important to quantify the potential impact of such effects on the strength and stability of resulting networks. Network analysis may be particularly vulnerable to order effects as it relies on relationships between symptoms. If the weight of these relationships is based at least partially on order effects, interpretations regarding the identification of core PTSD symptom relationships may be based on methodological interference rather than accurately reflecting clinical disease processes; this has implications for selecting the most salient treatment targets.
The current study aims to fill previous knowledge gaps regarding order effects related to PTSD diagnosis on three levels. First, it employs a direct methodological approach in presenting the PCL items in different orders. Second, it uses standard PCL wording. Third, it uses a more robust network analysis approach rather than CFA to model PTSD symptomatology.
Methods
Participants
The data used in this study were collected in 2008 as part of a larger study of service member health and well-being (Land Combat Study), approved by the institutional review board of the Walter Reed Army Institute of Research (WRAIR). Confidential cross-sectional data were collected from 6002 active component US soldiers from one Army infantry division, six months after their units returned from deployments to Iraq or Afghanistan. Soldiers were recruited by coordinating with unit commanders who made soldiers available for large group briefings that included a general description of the study, use of the data, and consent procedures. Approximately 40% of soldiers from participating units were present for the briefing, and those unable to attend had other work-related duties, were on leave, ill, in training, or were on temporary duty elsewhere. Approximately 86% of the soldiers who attended the recruitment sessions consented to participate in the study (n = 5145).
The sample characteristics are provided in Table 1. Education ranged from ‘some high school’ to ‘graduate degree’ with the majority of individuals reporting a high school diploma or equivalent (51%). Years in the military ranged from 0 to 39 (median = 3, interquartile range (IQR) = 5). Most soldiers reported Iraq as their most recent deployment (74.1%) and the median career months of deployment was 15 (IQR = 22). The three most frequently endorsed combat events were receiving small arms fire (73.4%), knowing someone seriously injured or killed (81%), and receiving incoming artillery (85.6%).
Table 1. Sample characteristics
a Prevalence is based on DSM-IV criteria.
Study design
There were two versions of the study questionnaire, which were identical except for the order of the health questions. In both versions, the demographics, deployment history, and combat experiences questions were asked prior to the psychological and physical health section. However, in the ‘ordered’ version the measures of depression (PHQ-9), anxiety (GAD-7), PTSD (PCL), somatic symptoms (PHQ-15), and tailored post-deployment reactions were arranged as distinct scales with standard response options. In the ‘random’ version, the same 101 items of PHQ-9, GAD-7, PCL, PHQ-15, and post-deployment reactions were randomized and given the response options of the PCL (i.e. not at all to extremely). The number of items between PCL questions ranged from 2 to 21 (M = 4.88, s.d. = 6.43), and no PCL items were presented without being separated by at least one non-PCL item. This randomization resulted in two versions of the survey, one in the standard ordered format and one with the items in the randomized order.
The questionnaires were packaged in envelopes, which were bundled in packs of 50 with alternating random and ordered versions. The study team was blind to which version soldiers were handed. Among the soldiers who consented, about one-half each completed the ordered (2577) and random (2568) versions. Random and ordered survey administrations were combined into one dataset and only participants who met DSM-IV criteria for PTSD were included in network and CFA analyses.
Measures
PTSD checklist
The PCL-civilian (Weathers, Litz, Herman, Huska, & Keane, Reference Weathers, Litz, Herman, Huska and Keane1993) was used to assess traumatic stress symptoms. The PCL is a 17-item scale measuring the 17 symptoms of the DSM-IV criteria for PTSD. Respondents endorse the degree to which they have been bothered by specific symptoms in relation to any stressful life experience in the past month on a five-point scale (1-not at all to 5-extremely). Individuals scoring a three or higher were determined to meet the criteria for that symptom. DSM-IV criteria were used to determine those meeting criteria for PTSD.
Analytic plan
Partial correlation network estimation
Network estimation was done using the R package qgraph to create a graphical Gaussian model for both the ordered and random samples. Given that we conducted CFAs with PTSD symptoms as ordinal, polychoric correlation matrices were used for network estimation. Network estimation is conducted using partial correlation coefficients represented by connections (edges) between the symptoms (nodes) after controlling for all other connections in the network. The qgraph package selects an optimal extended Bayesian information criterion (EBIC) λ value to derive the best fitting and most parsimonious model. Using the glasso function of the qgraph package allows for the least absolute shrinkage and selection operator (LASSO) regularization which was used to arrive at the best network representation of the data. The LASSO technique is common practice in the network analysis literature to control for spurious edges by reducing small correlations to zero increasing the visibility of important connections.
Visualization
Gaussian regularized partial correlation network graphs for the ordered and random samples are depicted in Fig. 1. The Fruchterman & Reingold algorithm (Fruchterman & Reingold, Reference Fruchterman and Reingold1991) is used by the R package qgraph to construct network depictions. Green lines represent positive partial correlations between nodes and red lines indicate negative partial correlations. The thickness of the line represents the strength of the given edge. Nodes with the most connections fall in the center of the network and nodes with few connections are depicted on the periphery of the network. Strongly connected nodes are depicted close together. The EBIC hyper parameter was set to γ = 1.0 as values between 0.5 and 1.0 have been reported to lead to an accurate network estimation at moderately high n (Foygel & Drton, Reference Foygel and Drton2010) and higher values favor parsimony and reduce the depiction of spurious negative edges that can result from polychoric-based networks n (Epskamp & Fried, Reference Epskamp and Fried2018).
Fig. 1. Partial correlation networks.
Centrality and stability
Beyond the estimation and interpretation of network edges, centrality indices were calculated to provide information on the importance of specific nodes in the network. The most commonly reported centrality indices are strength, closeness, and betweenness. Centrality estimates are depicted in Fig. 2. Node strength is an index of the connectedness of a specific node to the rest of the network and is a sum of all inward and outward connections. Betweenness is a measure of the number of times a node is included in the shortest path between any two symptoms and measures the degree to which a specific node controls the flow of the network. The final commonly reported centrality index is closeness and is the inverse of the sum of all the shortest distances between a given node and all other nodes. These nodes serve as junction points in the network.
Fig. 2. Centrality estimates.
Before confidently interpreting centrality indices, the stability of the network must be calculated. Stability was estimated using the R package bootnet. This package conducts case-drop bootstrapping to calculate the stability of the strength, betweenness, and closeness indices (see online supplementary Fig. S3). The resulting correlation stability coefficient (CS) is the percentage of cases that can be dropped that still results in a 95% probability of the correlation of the individual centrality indices being ⩾0.7. The recommendation is that the CS should be between 0.25 and 0.5 although the authors of the bootnet package recommend that these cutoffs be used as general guidelines rather than definitive cut points (Epskamp, Borsboom, & Fried, Reference Epskamp, Borsboom and Fried2018).
Accuracy
The R bootnet package was also used to derive the accuracy of the network. A total of 2500 nonparametric bootstrapped samples were used to calculate 95% confidence intervals (CIs) of the edge weights. These bootstrapped CIs along with the sample values are depicted in online supplementary Fig. S4.
Results
Demographics
The two samples did not differ in demographics or their level of combat experiences during deployment (see Table.1). In the ordered sample, 480 (19.1%) individuals met DSM-IV criteria for PTSD. In the random sample, 418 (16.5%) individuals met these criteria.
Ordered network
In the ordered network (see Fig. 1) the strongest connections were observed between intrusive recollections (B1) and distressing dreams (B2), hyper vigilance (D4) and exaggerated startle response (D5), anhedonia (C4) and detachment (C5), and internal avoidance (C1) and external avoidance (C2). The strongest connection was between intrusive recollections (B1) and distressing dreams (B2), and the weakest was between trouble remembering (C3) and anger/irritability (D2). Nonparametric bootstrapping indicated that these edges were significantly stronger than other edges in the network except for distressing dreams (B2) and flashbacks (B3) that were not shown to be significantly different from other edges in the network (see online supplementary Fig. S5). The stability of the ordered network centrality indices was assessed using case-dropping bootstrapping and indicated that while strength [CS (cor = 0.7) = 0.44] is stable and interpretable, closeness [CS (cor = 0.7) = 0.21] and betweenness [CS (cor = 0.7) = 0.13] are not. Therefore, only the strength index is reported here (for the centrality plot, see Fig. 2). According to centrality estimates, the strongest nodes were intrusive recollections (B1) and distressing dreams (B2), whereas the weakest nodes were traumatic amnesia (C3) and sense of foreshortened future (C7). However, nonparametric bootstrapping indicated that except for sense of foreshortened future (C7), most nodes were not significantly different from one another in terms of strength (see online supplementary Fig. S6). Sense of foreshortened future (C7) was found to be significantly different from all nodes except internal avoidance (C1), traumatic amnesia (C3), numbness (C6), difficulty sleeping (D1), and difficulty concentrating (D3).
Random network
In the random network, the strongest connections were observed between intrusive recollections (B1) and distressing dreams (B2), internal avoidance (C5) and external avoidance (C6), intrusive recollections (B1) and external avoidance (C2), anhedonia (C4) and trouble sleeping (D1), and flashbacks (B3) and difficulty concentrating (D3). There were two small negative correlations between intrusive recollections (B1) and anger/irritability (D2), and distressing dreams (B2) and difficulty concentrating (D3). The strongest connection was observed between intrusive recollections (B1) and distressing dreams (B2) and the weakest positive correlation was between internal avoidance (C1) and sense of foreshortened future (D2). Only the edge between intrusive recollections (B1) and distressing dreams (B2) was found to be significantly different from all other edges in the network (see online supplementary Fig. S7).
For the random survey sample, the strength [CS (cor = 0.7) = 0.44] and closeness [CS (cor = 0.7) = 0.28] indices were stable and interpretable, whereas, betweenness [CS (cor = 0.7) = 0.21] was not. According to centrality estimates, the strongest nodes were intrusive recollections (B1), distressing dreams (B2), and external avoidance (C2), whereas the weakest nodes were traumatic amnesia (C3) and sense of foreshortened future (C7). In terms of strength, most nodes were not significantly different from one another except for traumatic amnesia (C3) and sense of foreshortened future (C7) that were significantly different from all other nodes (see online supplementary Fig. S8).
According to the centrality estimates, the node with the highest closeness was internal avoidance (C2). The node with the lowest closeness was sense of foreshortened future (C7). Nonparametric bootstrapping indicated that most nodes were not significantly different from one another (see online supplementary Fig. S8) except for sense of foreshortened future (C7).
Discussion
The current study is the first to demonstrate the influence of order effects using the DSM-IV definition of PTSD. Among the combat veterans who participated in this study, item order strongly influenced both the PTSD prevalence rate and symptoms identified as central to PTSD in network analysis. Of particular interest, the majority of the strongest connections in the ordered network appeared to be driven by item order, whereas connections observed in the random ordered network represented non-proximal, but clinically relevant mechanisms supported by other domains of PTSD research. More specifically, in the ordered-survey sample, the top three strongest connections in the network were observed between symptoms that fell within the same symptom cluster and were sequentially positioned. Interestingly, this is true regardless of the CFA factor structure used to subdivide the DSM-IV symptoms. These findings potentially call into question previous research with network analysis. Although the conclusions of previous research are still relevant, the current study provides some evidence that at least part of the variance previously attributed to symptom relationships may be due to order effects rather than inherent characteristics of PTSD.
The highlighted findings are consistent with prior research in which the strongest connections are often observed within sequentially ordered symptom clusters (Armour et al., Reference Armour, Fried, Deserno, Tsai and Pietrzak2017; Price et al., Reference Price, Legrand, Brier and Hébert-Dufresne2019). Our findings also extend to those reported by Segal et al. (Reference Segal, Wald, Lubin, Fruchter, Ginat, Yehuda and Bar-Haim2019), who previously demonstrated a strengthening of within-cluster connections in the re-experiencing and avoidance symptom clusters following deployment. Most findings reported in the literature fail to mention potential order effects.
In contrast to our findings using the ordered surveys, the strongest connections in the random-survey sample were observed across symptom clusters except for one (intrusive recollections and distressing dreams). Placing the current study's findings in the context of the literature is crucial as it illustrates the potential impact of order effects on diagnostic conceptualization and treatment targets. Symptoms within the same cluster are conceptually related and often worded similarly. For example, the conceptualization of internal and external avoidance might reflect theoretically distinct clinical responses obscured by how intimately the two questions are linked to the survey instrument. It stands to reason that at least some of the partial correlation coefficients are attributable to the reinforcing effects of item order rather than a genuine strength of the connections between potentially distinct symptoms. This added weight may obscure other important connections between symptoms.
Our findings are consistent with the interpretation that in the absence of item order, new cross-cluster connections emerge, for example, the connection between external avoidance and intrusive recollections and the connection between anhedonia and difficulty sleeping. Due to the order effects, these relationships were obscured and not well identified in previous research. The network approach also reinforces the understanding of connections that have been identified in previous research across items. For example, the strongest connections in the random-survey samples were observed between symptoms in the re-experiencing and avoidance clusters, and re-experiencing and hyper arousal clusters. Notably, external avoidance was not identified as one of the most central symptoms in the ordered survey sample but was in the random survey network.
Our network findings also potentially help reinforce and refine fear-based conceptualizations and specific extinction-based interventions for PTSD. These theories highlight the key role of external avoidance in negatively reinforcing fear of trauma cues and the importance of clinically addressing avoidance as a central focus of clinical intervention. The fact that external avoidance had a cascading effect across the random-sample network lends emphasis to interventions that target this symptom. The link between re-experiencing and avoidance is also consistent with previous factor analytic work (Elhai et al., Reference Elhai, Contractor, Tamburrino, Fine, Cohen, Shirley and Galea2015; Price et al., Reference Price, Legrand, Brier and Hébert-Dufresne2019). Although treatment recommendations cannot be made from one analysis, if replicated, our findings support targeting these connections with extinction-based approaches; in-vivo exposure is likely to be particularly influential in the clinical treatment approach, a technique that is routinely used in some, but not all, evidence-based trauma-focused therapies (Beidel et al., Reference Beidel, Frueh, Neer, Bowers, Trachik, Uhde and Grubaugh2019; Foa, Hembree, & Rothbaum, Reference Foa, Hembree and Rothbaum2007). In the random network, two common symptoms of depression, anhedonia and difficulty sleeping, were also among the strongest connections. The connection between two non-proximal depressive symptoms highlights the importance of targeting these symptoms, despite not being within an extinction learning paradigm, and may also support researchers asserting that PTSD is defined too broadly (Hoge et al., Reference Hoge, Yehuda, Castro, McFarlane, Vermetten, Jetly and Rothbaum2016; O'Donnell et al., Reference O'Donnell, Alkemade, Nickerson, Creamer, McFarlane, Silove and Forbes2014) or perhaps that multiple subtypes exist within the PTSD construct (Campbell, Trachik, Goldberg, & Simpson, Reference Campbell, Trachik, Goldberg and Simpson2020; Pietrzak et al., Reference Pietrzak, el-Gabalawy, Tsai, Sareen, Neumeister and Southwick2014).
Although the above-highlighted differences are important, it is equally important to highlight the similarities between the two networks. In both the ordered and random-survey samples, the strongest connection was observed between intrusive recollections and distressing dreams. The fact that this connection was observed across both networks likely speaks to the strength of this connection. This finding is also particularly noteworthy as these two symptoms were also observed to be the most central symptoms across both networks. Also consistent across the random and ordered-survey samples were the weakest symptoms in the network. Traumatic amnesia and a sense of foreshortened future were found to be the least central to both the ordered and random-survey networks. This finding is also consistent with the extant literature highlighting the limited utility of these symptoms and support their removal and/or elaboration in the DSM-5 revision (Rubin, Berntsen, & Bohni, Reference Rubin, Berntsen and Bohni2008). It is important to consider that our study supports some of the changes made to the DSM-5, but also calls into question the utility of a broad approach to diagnose that may over-estimate PTSD prevalence and equally weight symptoms that are not central to the PTSD construct.
Despite the novel design of this study, there are some limitations. Perhaps the most important limitation to consider is that this study was based on the DSM-IV definition of PTSD using an older dataset from early in the Iraq and Afghanistan wars. However, the bulk of the factor analytic and network analysis studies have been based on DSM-IV, and these studies have been used to inform the changes to the definition introduced in DSM-5. The debate remains as to whether the changes in DSM-5 resulted in improved diagnostic or clinical utility, and further research that questions underlying assumptions is important (O'Donnell et al., Reference O'Donnell, Alkemade, Nickerson, Creamer, McFarlane, Silove and Forbes2014). Methodologically, this study is highly unique. A key strength was that the presentation of PTSD items in the random survey was interspersed with other psychopathology and health items. While this may have also resulted in the introduction of different priming effects in the survey completion process, one would assume that a strong PTSD construct would be resistant to the influence of changes in the location or order of clinical symptom items. Clearly, the random ordered survey not only had an important effect on the relationship of items in network analysis but it also produced a lower prevalence rate than the ordered survey, suggesting that individuals have different perceptions of the importance of their symptoms when items are presented in different ways. Another limitation is that soldiers completed either an ordered or random survey. Ideally, the same individual would fill out both surveys with enough time elapsing to allow for diminished recall or learning biases, but not enough time to allow for large fluctuations in symptom presentation. This was not practical considering the survey length and anonymous participation design, and might also have introduced additional psychometric problems stemming from item replication. Methodologically, the randomization worked very well with highly comparable groups. Finally, order effects may have contributed to some individuals meeting criteria in the ordered sample which in turn affected the network structure. Although there is no way to determine which sample is a more accurate representation of individuals with PTSD, the fact that there were significant differences both in prevalence and network structure suggest that the expression of clinical symptoms of PTSD are influenced to a large degree by how they are presented on survey instruments.
It is also important to note that the current study targeted a specific group of soldiers who had returned from deployment six months previously. The resulting networks should therefore not be taken to represent the defining pattern for PTSD, as the centrality of different symptoms may vary across different populations, types of trauma (Benfer et al., Reference Benfer, Bardeen, Cero, Kramer, Whiteman, Rogers and Weathers2018), as a function of time since trauma (Segal et al., Reference Segal, Wald, Lubin, Fruchter, Ginat, Yehuda and Bar-Haim2019), and from pre-treatment to post-treatment (Hoffart et al., Reference Hoffart, Langkaas, Øktedalen and Johnson2019). Indeed, such differences might guide more targeted therapies to specific patient populations.
As our understanding of PTSD continues to grow, it is essential that as a field we continually reassess previously held assumptions and challenge both new and existing analytic approaches. Although network analysis is a robust new approach with a high potential to broaden our understanding of PTSD, it appears that it is vulnerable to the influence of measurement errors such as order effects. The results call into question conclusions reached by many studies that have relied solely on survey instruments ordered exactly as per the current definition. The evidence that the strength of previously identified PTSD relationships may be partially based on order effects has important implications for understanding the diagnostic construct of PTSD and targeting treatments to symptoms that have the greatest potential influence on functioning. Researchers will need to consider order effects in any future analyses, and such analyses also have the potential to identify novel relationships between symptoms both within accepted definitions of PTSD, as well as across disorders. Given the foundational importance of CFA approaches, the influence of order effects should also continue to be studied in CFA. While this study focused on the newer network analysis, there are also gaps in knowledge concerning the potential influence of order effects on factor analytic studies. The findings of this study need replication in other samples, and particularly replication in samples that have used instruments based on DSM-5 and ICD-11 definitions.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291720004377
Conflict of interest
None.
Disclaimer
Material has been reviewed by the WRAIR. There is no objection to its presentation and/or publication. The opinions or assertions contained herein are the private views of the author and are not to be construed as official, or as reflecting true views of the Department of the Army or the Department of Defense. The investigators have adhered to the policies for the protection of human subjects as prescribed in AR 70–25.
