Establishment of the CPHC

The committee was charged with reviewing a large number of detailed proposals within a relatively modest timeframe and established its membership and procedures accordingly. APA President Dr John Oldham appointed Jack McIntyre, M.D. as Chair and Joel Yager, M.D. as Co-Chair of CPHC. Six other members, all co-authors of this paper, were appointed, and all were vetted to assure no industry-related conflicts of interest concerning any diagnostic issues. In accord with the wide range of clinical and public health issues to be considered, the committee was broadly composed and included two members in full-time private practice with clinical appointments at a medical school (JM and JN), and six with full-time academic appointments. Three committee members were trained in Geriatric Psychiatry (VR, JL, MV); one in Child and Adolescent Psychiatry (CG), and one member had extensive involvement in Community Psychiatry and Public Health (AE). The Co-Chair of the Committee (JY) also served on the SRC and two committee members were members of the APA Board of Trustees (AE and JN.) All CPHC members were chosen on the basis of wide clinical interests, to cover broad swaths of psychiatric diagnostic areas of experience, and, anticipating how contentious some of the group discussions might become, for having reputations for being able to ‘play well with others’ – that is, to be constructive small group members. All potential CPHC members were carefully vetted for conflicts of interest using the same criteria established for members of the DSM-5 Task Force.

Material available for CPHC members

For each submitted proposal, the Work Group serving under the Task Force, prepared a Memo Outlining the Evidence for Change (MOEC). The MOECs ranged from 28 to 420 pages. An initial MOEC was submitted to the SRC and, if the proposal was subsequently referred to the CPHC, the original MOEC and revisions and additions the Workgroup thought might benefit the CPHC's review process accompanied the referral. The CPHC's request for information differed slightly from that of the SRC, in line with the committee's increased attention to clinical, public health, and logical consistency issues. The outline for the MOEC requested by the CPHC (which is in practice was variably adhered to by DSM Work Groups in preparing their arguments to support changes) modeled that of the SRC and also requested information about clinical and public health impact. In addition to the MOEC, CPHC received the completed review by the SRC and information from the Field Trials (when available.) For proposals that generated significant controversy in the professional and/or lay literature, articles concerning this controversy were also included. These materials were sent to committee members soon after the proposal was referred to the Committee in order to expedite reviews.

External expert reviewers

The Committee knew in advance that given the nature of the task, the time-frames, and the information available that its determinations would be based largely on collective knowledge and experience filtered through group-process influenced decision-making, inevitably resting on collective knowledge, experiences, and opinions. With the given constraints, no other feasible and reasonable methods seemed possible. In such processes, measurements and rating scales are inescapably inexact, and although personal opinions are unavoidable the process attempts to solicit sufficiently varied input to evoke the ‘wisdom of crowds’ and avoid small-subgroup hegemony. To assure that input was not limited to voices of CPHC members, the Committee developed a review process modeled on procedures utilized by peer-reviewed journals to include the use of external reviewers.

The Chair and Co-Chair recruited the expert reviewers, but, in order to encourage frankness by the external reviewers, the reviewers' identities were blind to all other committee members. Some of the reviewers were well-known experts with extensive publications in the areas covered by the proposals and others, many internationally, were identified as active in those areas by literature reviews using PubMed. Members and advisors of existing DSM-5Work Groups could serve as experts, but not on proposals they were worked on or voted on. Each potential expert reviewer was alerted in the initial invitation that we would not use individuals who had potential industry-related conflicts of interest concerning the diagnoses they were to review; those who agreed to serve were further vetted by DSM administrators for potential conflicts of interest using the same guidelines as for members of the CPHC. Reviewers were assured that their reviews would be confidential and were told that they would be acknowledged and listed in DSM-5 as reviewers, but not identified as to which specific proposal they reviewed. In total, 482 experts were initially approached and 130 individuals accepted, were ultimately vetted, and conducted 138 reviews (a small number reviewed two proposals.) Generally, three–five external experts reviewed each proposal, but on a few occasions, there were more (eight for one proposal); for one proposal only two external reviews were received.

Each external reviewer received the MOEC, field-trial information if available, and in cases where significant public controversy (Narrow et al., Reference Narrow, Clarke, Kuramoto, Kraemer, Kupfer, Greomer and Regier2013) had been generated in the professional and lay media, articles concerning this controversy were included in the packet. The CPHC review template, approved by the Task Force, modeled on the information requested in the MOECs, is shown in Table 1. In addition to completing the template, reviewers were invited to offer additional observations; many reviewers provided additional comments, some quite extensive, often very helpful and illuminating. CPHC primary and secondary reviewers also completed the review templates in advance of discussions.

Table 1. CPHC Reviewer recommendation report

Operations of the CPHC

The DSM-5 Task Force decided on which of the hundreds of diagnostic changes considered for DSM-5 to refer to CPHC for review. Submissions to CPHC were prioritized based on the amount of controversy proposals had stirred up in professional and lay media (high visibility issues were inevitably referred), their prevalence and public health importance, and when certain Work Groups and/or the SRC specifically requested CPHC review. Ultimately, a total of 40 proposals were reviewed, some of which consisted of large topic areas that included several individual diagnoses. Proposals reviewed by CPHC are shown in Table 2.

Table 2. Diagnoses considered by CPHC and their scores

Mechanics of the CPHC review process

Briefly, completed proposals were assigned to a designated primary and secondary CPHC reviewer, and were also distributed to all CPHC members and 4–5 assigned vetted external reviewers who were known only to the CPHC chair and co-chair. Prior to group discussions, CPHC members received blinded reviews from external reviewers. At the start of the discussion for each proposal, similar to procedures in many NIH study sections CPHC members were queried for their preliminary votes of a global rating of the proposal on a 4 point scale (similar but not identical to the scale used by the SRC): (1) Excellent (strong support) (2) Good (moderate support, acceptable) (3) Fair (limited support) and (4) Poor (probably not-justified; do not include). Although all committee members were responsible for reviewing all proposals and individually rating each proposal (or section), each proposal's discussion was led by the primary and secondary reviewers, each of whom independently prepared a review following the CPHC's template. After group discussion, members provided their final numeric votes orally and in writing. For the final reports, primary and secondary receivers prepared summaries of the CPHC discussions and recommendations to accompany the final average tallied scores.

To conduct its business, the CPHC held 24 conference calls throughout 2012, each ranging from 1 to 3 h. On each conference call, up to three proposals were discussed, but some proposals were discussed a second and in one instance a third time. Occasionally, a second or third review resulted following additional input from the Work Group and/or Task Force based on concerns expressed by the SRC and/or CPHC. Also, for a small number of proposals, the CPHC Chair and/or Vice-Chair clarified some issues ‘off line’, between the committee's conference calls, with the Work Group Chair.

Final reports were sent to the President of APA, Dr Dilip Jeste and were used to provide input into the DSM-5 Summit Group's discussions, and subsequently to the BOT.

Controversial issues in CPHC reviews

Proposals that had clear support from SRC were generally not referred to CPHC for consideration, although there were a few notable exceptions; one of which is noted below. Although the fundamental principle behind creating CPHC for DSM-5 was that any proposed revision must be feasible for routine clinical practice, we were fully aware early in the process our work was limited and challenged by the fact that little or no robust evidence generally existed to guide the committee in making determinations as to clinical utility or feasibility. As a rule of thumb, although CPHC did not independently rate the strength of scientific evidence supporting each proposal, CPHC members gave considerable attention and respect to how the SRC rated scientific merit. Unless clear overriding clinical utility, public health issues or logical inconsistencies were identified, CPHC was swayed by the principle that revisions should be guided by research evidence, and proposals lacking such supporting evidence generally received low scores from CPHC.

Although the Committee is bound to honor agreements concerning the confidentiality of the discussions, and space does not permit review of how each proposal was deliberated, the following examples will illustrate some of the major challenges the Committee faced in attempting to determine the clinical utility and public health merit in what was often and largely a ‘data-free’ zone of operations.

One proposal that received strong support from the SRC, the elimination of the bereavement exclusion for major depressive disorder, was nevertheless referred to the CPHC for review as well because considerable controversy existed in the medical and lay press. A prevalent argument was that by eliminating this exclusion the American Psychiatric Association would be pathologizing normal bereavement. While appreciating these concerns, CPHC (and the external reviewers) agreed that the supporting evidence for the proposal was quite strong and that this change would be clinically beneficial without generating untoward public health consequences. Accordingly, approval was recommended (although CPHC also recommended clarifying language in the accompanying text, which was, in fact, adopted in the final publication.)

A complex set of proposals were submitted to SRC and CPHC regarding changes for Autism Spectrum Disorder. A proposal for collapsing Asperger's Disorder as well as Pervasive Development Disorder NOS, Childhood Disintegrative Disorder, and Autism into one category to be called Autism Spectrum Disorder was assessed as having good scientific merit, and accordingly, this aspect of the ASD proposal received good support from the SRC. However, considerable concerns had been raised in the lay community that eliminating Asperger's Disorder would result in deleterious public health consequences regarding benefits and services for individuals previously diagnosed with Asperger's Disorder, and that elimination of this diagnosis would also disadvantage them socially, removing what for some, had become a meaningful identifying label. The proposal was therefore referred to CPHC for review. Although we generally concurred with SRCs assessments based on scientific evidence, nevertheless, considerations of clinical and public health issues led CPHC to recommend the addition of explicit specifiers that would retain the uniqueness of each subgroup, e.g. Autism Spectrum Disorder ‘…predominantly XX type’. Ultimately, this recommendation was not incorporated into the published DSM-5.

More difficult were the considerations of proposals that received ‘borderline’ support or frankly unfavorable ratings from the SRC because that committee's review concluded that scientific evidence was lacking to support the magnitude of proposed changes in diagnostic criteria.

One example of where CPHC differed from SRC was in evaluating a proposal for Catatonia NEC. The SRC voted unfavorably on this proposal due to lack of scientific validators. In contrast, CPHC saw little in the way of the clinical downside and thought that this category could be clinically useful as a residual or working diagnosis, and rated this proposal as ‘good’. Similarly, in consideration of adding ‘craving’ as a criterion for the diagnosis of Substance Use Disorders, in contrast to the SRC rating of limited-poor support, CPHC found this addition to be clinically meaningful and voted ‘good to excellent’ support.

Perhaps the most difficult assignment for CPHC concerned proposals for a complete restructuring in the way that personality disorders might be diagnosed. To briefly put this story in context, before the CPHC was ever involved, the Personality Disorder Work Group had set out a complex and constantly evolving series of proposals for overhauling the diagnostic approach to personality disorders. Initially based predominantly on replacing traditional categorical DSM criteria with a predominantly dimensional system based on temperament-associated traits and excluding the typological thinking usually practiced by clinicians, the Work Group's initial presentations were not well received by the DSM Task Force. In addition, a number of outspoken and well-reputed psychiatric and psychologist personality disorders experts objected to the new model as well as to the fact that their views were not represented in the composition or negotiations of the Work Group. After the Work Group leadership compromised by devising a ‘hybrid’ model that fused typological and trait-based approaches, two Work Group members who advocated for trait-based methods were so displeased that apparently, they resigned. The Work Group's initial MOEC proposals to the SRC focused primarily on deficiencies in the DSM-IV diagnostic approaches; it also offered some evidence-based justification for their new models. However, given the magnitude of the proposed changes, the SRCs judged the MOEC to provide insufficient evidence for scientific validation for the new diagnostic schemes. Even after the Work Group revised the MOEC, the SRC again judged the Work Group's revision to inadequately support claims that the new diagnostic scheme was sufficiently superior to merit discarding the DSM IV criteria in favor of the new proposal. At that point, the Work Group requested CPHC review(Zachar, Krueger, & Kendler, Reference Zachar, Krueger and Kendler2016)

Aware of these controversies and of the significant impact on clinical practice that would follow the adoption of the proposed hybrid criteria; the CPHC planned a more elaborate than usual review process for the personality disorders proposal. Instead of the usual 4–5 external reviewers,

CPHC recruited eight outside reviewers. Given the CPHC's charge to focus on clinical utility, the external reviewers included a mix of academic personality disorder researchers, academic researchers whose work focused on clinical problems noted for substantial comorbidities involving personality disorders, and several highly experienced practitioner-scholars whose clinical areas abutted personality disorder areas. None of the well-known psychiatrists or psychologists who had most publically and prominently opposed the changes was among the external reviewers. In addition to completing the review checklist, most of the external reviewers wrote detailed comments. Within the CPHC, in contrast to a single primary and secondary reviewer, four primary reviewers were assigned, each of whom was obliged to provide a written analysis prior to discussion. All of the reviewers and CPHC members appreciated how much in the way of time, energy, talent, and good intentions the Personality Disorders Work Group had devoted to these efforts. Nevertheless, only one of the external reviewers was favorable about the proposal (an academic whose career focus was on personality trait research); one had positive reactions to parts of the proposal and all the other reviewers were not at all favorable. CPHC members judged the proposal unfavorably, concluding that although elements of the proposed new scheme were probably clinically useful, the package as a whole was not and that imposing these changes would unduly burden clinicians without convincing evidence of substantial benefit over the DSM IV system. An 11th-hour revision of the proposal by the Work Group was received and reviewed by CPHC – bending its own procedural rules in so doing – and was still judged to be too clinically difficult to implement to justify replacing DSM IV criteria.

Accordingly, after final SRC, CPHC, Summit Group and Board of Trustees review, the Work Group was invited to write a section on ‘Alternative DSM-5 Model for Personality Disorders’ in Section III, Emerging Measures and Models. Since a major concern of the CPHC was clinician burden with the proposed new model, members of CPHC felt that placing the model in Section III of DSM-5 was a good solution, giving clinicians the option of using the new model if they were comfortable in doing so. Also, as experience with the new model grows it may become clear that its use is indeed feasible and it will be considered for a future DSM.

Comments and recommendations for future DSM efforts

Our experiences conducting the CPHC lead us to several reflections and recommendations for future DSM-related efforts by the American Psychiatric Association:

• • Overall, we believe that having external review processes to evaluate and respond to Work Group proposals is essential and that such processes should be built into future efforts from their start.

• • We also see value in having different groups assess proposals for scientific merit and separately for clinical and public health utility and impact. We believe that the mindsets, experience, and expertise of individuals focusing on these different facets of proposals differ sufficiently to merit distinct committees.

• • Given the amount of work entailed in the reviews, as per grant proposals in the NIH, we suggest that future proposals be written in as succinct a manner as possible, directly addressing the parameters requested by review committees. The burden on committee members and external reviewers of being expected to thoroughly review hundreds of pages in some single proposals is unreasonable. Although appendices and supplementary material might be offered as optional reading for reviewers, basic proposals should be concisely written.

• • Similar to the SRC, CPHC did not do ‘blinded’ voting. Generally, although not always, the range of votes among CPHC members was narrow, most coming within a point of one another (e.g. 2s and 3s). This degree of convergence undoubtedly resulted from group discussion, group attunement, and group process. In contrast, votes from external reviewers who saw only the written material and who did not have opportunities to interact with one another or with the committee showed a wider range (e.g. on some proposals votes ranged from 1 to 4). Given how personal beliefs, values, and opinions form these ratings and the imprecision of our rating scales, no method will approach perfection. However, experimenting with the use of anonymous Delphi surveys for voting on proposals might prove worthwhile.

We were privileged to participate in the development of DSM-5 and trust that our input and that of the many external reviewers who contributed to this process added to the quality of the final document. We hope that our experience-based recommendations will assist in future DSM-related planning efforts.