While there have been many approaches to classifying religious traditions in the social sciences (see Hackett and Lindsay Reference Hackett and Lindsay2008), the most popular approach is the religious tradition classification scheme, which was most carefully systematized by Steensland et al. (Reference Steensland, Park, Regnerus, Robinson, Bradford Wilcox and Woodberry2000). Their widely-embraced article argued that the most accurate typology of religiosity was to sort individuals into seven distinct groups: evangelical Protestant, mainline Protestant, black Protestant, Jewish, Catholic, other religious groups, and no religion. This approach has become popularly known as “reltrad” and its usage in academic writing is voluminous. A brief search of Google Scholar indicates that over 900 published articles and books utilized the reltrad framework. However, the implementation of this typology has never been fully and accurately operationalized.
While the original article by Steensland et al. (Reference Steensland, Park, Regnerus, Robinson, Bradford Wilcox and Woodberry2000) provides a thorough coding appendix using multiple variables from the General Social Survey (GSS), the actual syntax to generate a consistent reltrad variable was not included with the manuscript. Instead, researchers have been left with two choices: try to write the code from scratch or find a coding file posted online and trust that author had accurately recreated the reltrad typology.
For a significant period of time, the most easily accessible document was available online at Indiana University's website that included reltrad coding for three different software programs: SAS, SPSS, and Stata. Regrettably, this code contained an omission from the coding appendix included with Steensland et al. (Reference Steensland, Park, Regnerus, Robinson, Bradford Wilcox and Woodberry2000). The syntax failed to sort two small, but significant groups (those who claim “Christian” generically and those who describe their religious affiliation as “interdenominational” under the “relig” variable) into the evangelical Protestant category. The end result was that in a number of years, the percentages of evangelicals contained in the sample were underestimated. This difference is illustrated in Figure 1.
Figure 1. Difference between correct reltrad and incorrect reltrad coding of evangelical protestants. Data: General Social Survey, Cumulative File (1972–2014).
The divergence between the incorrect and correct coding does not appear until 1998, and in many years there is a 1% underestimation of evangelicals. However, this difference is magnified in more recent samples of the GSS, with 2011 indicating a 2% underestimation and 2012 showing a 4% underestimation. LifeWay Research often focuses on researching evangelicals, so this variable is particularly important. Our LifeWay Research team (researcher Daniel Price) discovered this coding error and LifeWay Research subsequently informed several colleagues, who have since disseminated corrections. In September 2015, Indiana University removed the coding from their website without comment.
It is difficult to ascertain how much scholarship has been published using this incorrect code, as just a small number of authors include a direct reference to the syntax they employed (Burge Reference Burge2013; Johnson, Scheitle, and Ecklund Reference Johnson, Scheitle and Ecklund2015; Putnam and Campbell Reference Putnam and Campbell2012), however it is a near certainty that many other pieces of published research have reported incorrect findings as a result of this coding error. While no one can claim with certainty that using the proper code would have significantly changed the outcome of published scholarship, a desire to create a repository of scrutinized reltrad coding would help to alleviate this concern in the future. This would allow researchers to more easily recreate the reltrad coding, with the additional benefit of creating a simpler pathway for replication.
As a way to create a long-term, stable web presence LifeWay Research will be hosting a series of coding files to create the correct reltrad classification scheme using the GSS data. The web address for this will be www.lifewayresearch.com/reltrad. These files will be posted for the most widely used statistical software and will be updated on a bi-annual basis as new denominations are added to the GSS. With the exponential rise in the number of non-denominational evangelicals in the United States (Stetzer Reference Stetzer2015b), and the relative stability of evangelical Christianity (Stetzer Reference Stetzer2015a), it becomes even more important to accurately assess their numbers.
We believe this effort fulfills a need expressed by some of the creators of reltrad who wrote, “A centralized database that employs RELTRAD across surveys and for subsequent years of the GSS, and is regularly updated, would benefit the scholarly community” (Woodberry et al. Reference Woodberry, Park, Kellstedt, Regnerus and Steensland2012, 70). It is our hope that other individuals and institutions will begin to create coding repositories of other widely used social science measures and that this will be a catalyst for increased transparency and replicability in the scientific community.
