An influential literature has linked the emergence and consolidation of democratic regimes to a wide variety of factors such as income per capita, inequality, financial openness, revolutions and migration. Robert D. Woodberry (Reference Woodberry2012) makes an important contribution to this research agenda, as he studies how religious actors and motivations affect democracy, a topic that has received little attention in the literature. In his broad historical analysis, Woodberry focuses on ‘regions with histories and class structures radically different from those of Europe’ (245) and argues that conversionary Protestants (CPs) fostered democracy by spreading religious liberty, mass education and communication, printing and colonial reforms, and strengthening civil institutions. These processes created conditions that made stable representative democracies more likely, regardless of the number of converts to Protestantism.
Woodberry claims that the association between CPs and democracy is causal based on a historical and theoretical examination. He also uses regression analyses (both ordinary least squares (OLS) and instrumental variables (IV) approaches that employ over fifty controls) to demonstrate that the historical prevalence of Protestant missionaries and levels of democracy (averaged from 1950 to 1994) are positively correlated in 142 non-European societies.
The impact of Woodberry's article is reflected in more than 400 citations, as well as six awards since its publication in the American Political Science Review. His substantive contribution is important, because it turns modernization theory on its head by challenging its key tenet – that democracy developed as a result of secularization and processes associated with economic development, such as education and urbanization. Scholars are still debating whether modernization theory is right or wrong, but what is unique about Woodberry's argument is that it posits that these developments were in fact spurred by CPs. In order to understand the emergence and consolidation of democratic states, the literature must therefore not sweep religion under the carpet.
Woodberry's historical discussion of how Protestantism affected religious liberty, mass education and communication, printing and colonial reforms, and strengthened civil institutions is well founded in sociological and theological theory.Footnote 1 However, this letter shows that Woodberry's empirical analysis linking CPs and democracy suffers from three severe shortcomings: (1) operationalization of the democracy variable, (2) time period under investigation and (3) an inability to deal with omitted variable bias and establish causality in the IV estimations due to the implausibility of the exclusion restrictions. These problems give rise to concerns about the validity and replicability of his results. While we agree with Woodberry that the democracy literature should not ignore the role of religion, our results are crucial for further work on this topic as they raise questions about the exact causal chain – if any – linking religion and democracy. At the end of the letter we discuss whether Woodberry's theoretical mechanisms linking Protestantism to democracy (religious liberty, mass education and communication, printing, colonial reforms and the strengthening of civil institutions) may be applicable to (Western) Christianity more broadly. We believe that further research, possibly using longer-term and more granular religious data, is needed to answer this question.
This letter is structured as follows. The first section presents a detailed critique of the empirical analysis, while the second section extends the scope of Woodberry's research to include alternative measures of democracy and longer data. The last section offers concluding remarks.
Critique of the Empirical Analysis
The first point of critique is Woodberry's (Reference Woodberry2012) choice of the Bollen-Paxton (BP) data as a measure of democracy. The BP democracy score is constructed using four measures of democratic rule (political rights, competitiveness of nomination process, elected chief executive and effectiveness of elected legislature body) and four measures of political liberties (freedom of broadcast media, freedom of print media, civil liberties and freedom of group opposition). The equally weighted mean of the indices of political rights and legislative effectiveness was then averaged with the index of political opposition. In addition, the score accounts for restrictions on the franchise by capping the political rights variable (and its average with legislative effectiveness) at the franchise percent in a given country (Paxton et al. Reference Paxton2003). The resulting democracy indicator assumes values between 0 and 100.Footnote 2 Woodberry uses Bollen's (Reference Bollen2009) definition of a liberal democracy: ‘the degree to which political liberties and democratic rule exist in a country’ (369).
While conventional democracy indices may lead to some rater bias, there is no apparent reason to favor the BP data over other, more comprehensive dichotomous or continuous indicators of democracy that also cover recent years. In particular, Woodberry's analysis limits the timeframe over which the dependent variable is averaged to only 44 years between 1950–1994.
Therefore, this letter extends Woodberry's data by first using the Boix-Miller-Rosato (BMR) (Reference Boix, Miller and Rosato2013) and Polity IV (PIV) indices.Footnote 3 The broader historical scope of the new variables is relevant, as Woodberry does not present any theoretical predictions regarding the duration of the effect of Protestantism on democracy. Therefore, Woodberry's choice to limit the analysis to the 1950–94 period is rather arbitrary given that more recent data is available.
The BMR index is dichotomousFootnote 4 and defines a country as democratic if it satisfies a minimum set of conditions for democratic contestation and participation. The contestation requirement states that elections need to be free and fair, and the participation requirement requires that the majority of adult men are able to vote. Inspired by Dahl's (Reference Dahl1971) classification, it is the most comprehensive dichotomous measure of democracy available and unlike other indices, it takes a minimal suffrage requirement into account. We also employ the PIV index,Footnote 5 which reduces the sample size by 15 per cent (that is, 121 instead of 142 countries) but provides a comprehensive continuous assessment of democracy (ranging from −10 to +10) based on the presence of electoral institutions, constraints on the exercise of power as well as civil liberties. To further ensure robustness, we replicate the analysis using twenty-four other democracy measures, of which we predominantly report regressions using the Varieties of Democracy (V-Dem) and Freedom House data. From the V-Dem index we use the Electoral Democracy,Footnote 6 Liberal DemocracyFootnote 7 and SuffrageFootnote 8 scores since they most closely capture liberal democracy in the sense of the BP measure. From the Freedom House index we employ the Political RightsFootnote 9 and Civil LibertiesFootnote 10 measures, as well as a combined averaged score of both measures. The relationship between the original BP variable and the alternative measures of democracy is reported in Table A17 and illustrated graphically in Appendix Figure A1. As the table shows, seventeen of the twenty-six measures are correlated with a correlation coefficient larger than 0.80, and only three measures are correlated with a coefficient smaller than 0.70 with the BP measure.
Evidence from New Data
Results with Alternative Democracy Measures (1950–94)
We first replicate Woodberry's analysis using the same time period over which the democracy measure is averaged (1950–94) but instead employ the BMR and PIV datasets (all regressions in this letter follow Woodberry's replication code and use robust regression). Woodberry uses three proxies for CPs: (1) Protestant Missionaries per 10,000 Population in 1923, (2) Years Exposure to Protestant Missions and (3) Percent Evangelized by 1900.Footnote 11 Catholic versions of the first two variables are included as well. Descriptive statistics are displayed in Table A1 and the complete list of countries can be found in Appendix Table A2.
Table 1 (using BMR, 1950–94) merges the results presented in Columns 3 and 4 of Woodberry's Table 2 and Columns 3–5 in Woodberry's Table 3. Appendix Table A3 conducts the same exercise but uses the PIV data.Footnote 12 Table 1 shows that the relationship between CPs and democracy is not robust: the Protestant Missionaries in 1923 variable is only significant in specifications without controls (Columns 2 and 5), which raises concerns about omitted variable bias. Columns 3–5 control for a set of additional variables that relate to the historical influence of colonization processes, such as the date of ‘discovery’, literacy rates and the number of years between ‘discovery’, colonization and missionary contact. Years Exposure to Protestant Missions is insignificant. Percent Evangelized is significantly and robustly associated with democracy, yet, as we explain below and as Woodberry acknowledges, this variable is not a direct measure of Protestantism since it is a ‘retrospective estimate and includes both Europeans and Catholics—not only indigenous people exposed to Protestant missions’ (257, emphasis added).
Table 1. Predicting democracy in non-Western societies (Columns 1–2) and controlling for the process of colonization (Columns 3–5), 1950–94, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013
Note: standard errors reported under coefficients in parentheses (robust regression). The table replicates Table 2, Columns 3–4 (equivalent to Columns 1–2 in current table) and Table 3, Columns 3–5 (equivalent to Columns 3–5 in current table) from Woodberry (Reference Woodberry2012). *** p < 0.01, ** p < 0.05, * p < 0.1
Table 2. Predicting democracy in non-Western societies (Columns 1–2) and controlling for the process of colonization (Columns 3–5), 1950–2015, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013
Notes: standard errors reported under coefficients in parentheses (robust regression). See notes to Table 1. *** p < 0.01, ** p < 0.05, * p < 0.1
Table 3. Instrumental variable regression: mean level of democracy from 1950–2015, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013
Note: robust standard errors reported under coefficients in parentheses. Table 3 replicates Table 6 from Woodberry (Reference Woodberry2012). *** p < 0.01, ** p < 0.05, * p < 0.1
Results with Alternative Democracy Measures and Extended Time Period
Next, we extend the period over which our two democracy measures (BMR and PIV) are averaged (to 1950–2015/16) and show that the results below are at odds with Woodberry's conclusions. Tables 2–3, A4 and A5 employ the democracy score from the BMR dataset, and Tables A6–A9 use the democracy score from the PIV dataset.
Table 2 shows that once more, Protestant missionaries do not predict democracy robustly once the BMR democracy measure covering 1950–2015 is used. In three out of five specifications, the coefficient is neither significant nor positive. Only in Models 2.2 and 2.5 (specifications without controls) does Protestant Missionaries have a significant effect on democracy. Once again, Percent Evangelized has a positive and significant effect on democracy. However, the effect is ambiguous since the measure does not distinguish between Protestant and Catholic believers.
Table 3 replicates Woodberry's IV estimations which are intended to reduce omitted variable bias and to establish causal relationships, which is particularly important given the cross-sectional nature of his data and the fact that missionary activity is non-random. As Jedwab, Meier Zu Selhausen, and Moradi (Reference Jedwab, Meier Zu Selhausen and Moradi2018) demonstrate, Christian missionary expansion in sub-Saharan Africa occurred in relatively more developed and accessible locations, thus questioning findings that causally link missionary activity and contemporary development.
As in Woodberry's analysis, the limited information maximum-likelihood estimator is employed in the IV regressions, which cover two sets of excluded instruments for each Protestant variable. Specifications 3.1–3.3 use the excluded instrument Latitude, while Landlocked Nation and Mean Min. Temperature are used in specifications 3.4–3.6. The low F-statistics from the first-stage regression show that the instruments in specifications 3.1, 3.2 and 3.5 are weak. Therefore, only the analysis of Models 3.3, 3.4 and 3.6 allows for statistically significant inferences. Model 3.3 shows no associations of the key independent variables with democracy. However, in Model 3.4, Percent Evangelized by 1900 seems to positively affect democracy. Years Exposure to Protestant Missions in Model 3.6 has a positive and significant effect on democracy, yet it is unclear why only this particular combination of instruments renders it significant and why the effect is so much higher compared to the OLS specification.
From a substantive point of view, it is also puzzling why some of the instruments do not belong in the main equation.Footnote 13 In fact, in some specifications, Woodberry includes some of the excluded instruments in the main equation (in direct violation of the exclusion restriction), which raises doubts about their appropriateness. More importantly, given the significance of Percent Evangelized, Woodberry does not provide a convincing explanation as to why the effect of the instruments works exclusively through Protestantism rather than through another Christian denomination, such as Catholicism. In fact, his identification strategy is selective, as the excluded variables – favorable geographic and climatic conditions – may be used as predictors for any human settlement (including Protestant and Catholic missions) and thus the development of institutions that are a requisite for democracy. Therefore, the exclusion restrictions in Woodberry's analysis are implausible, meaning that the IV regressions are unable to establish causality.
Tables A4 and A5 control for a number of other variables that may have influenced the development of Christian missions (demographic or public health characteristics, Table A4), or mechanisms by which Protestant missions may have influenced democracy (log GDP per capita and secondary education enrollment rates, Table A5).Footnote 14 The results in Table A4 confirm the significance of Percent Evangelized by 1900 but as stated earlier, the interpretation of the effect remains ambiguous as it conflates Protestantism and Catholicism. The direct measure of Protestant Missionaries is inconclusive, as the coefficients switch signs across different specifications. There is some evidence of a positive association with the Years of Exposure to Protestant Missions. These results are echoed in Table A5, which shows a significant negative relationship between Protestant missionaries and democracy, along with a positive (but only sometimes significant) effect of Percent Evangelized.
The results from the BMR sample are largely confirmed by the evidence from the PIV sample. Woodberry notes that ‘switching democracy measures does not influence Protestant missions’ association with democracy’ (266), but even in his original model (see Table 7 in his Appendix, which uses the Polity2 score), Protestant Missionaries are negatively if insignificantly associated with democracy. Table A6, which replicates Table 2 above using the PIV measure, shows yet again the insignificance of Protestant Missionaries. The only consistently significant measure of Protestantism is Years Exposure to Protestant Missions; however, the post-estimation Wald test indicates that the coefficients are equal to the effect of Years Exposure to Catholic Missions (0.25≤F≤1.65, p < 0.01).Footnote 15
Tables A7 and A8 replicate Tables A4 and A5 using the PIV data. The coefficients on Protestant Missionaries switch signs across specifications and are statistically insignificant. There is an association between democracy and Years of Exposure to Protestant Missions in both tables. However, in Table A8 (which is the more credible table) this effect is equal to Years Exposure to Catholic Missions, as suggested by a Wald test (0.81≤F≤3.23, p < 0.01).
Table A9 reports the results of the replication of the IV estimation using PIV data. However, given the weak instruments in three out of six specifications and the fact that the exclusion restriction is unlikely to be satisfied, the results are not credible. Protestant Missionaries does not have a significant effect.
Robustness Checks
An important concern about Woodberry's results is that the inclusion of post-treatment covariates may lead to significant bias. To account for this concern, we ran models that excluded all variables for which at least some of the coverage occurs after the first year of the democracy average (1950) or the year of the Protestant Missionaries variable (1923), such as Percent Muslim in 1970, Major Oil Producer, Years of Protestant Missions until 1960, Years of Catholic Missions until 1960, various religious composition variables measured in 1970, log GDP per capita (1960–94), education (1960–85) and newspaper circulation (1975–95). There are no changes to the previous findings (results available upon request).
To make sure that our results are not driven by the idiosyncrasies of a particular democracy measure, Appendix Tables A10–A15 replicate Table 2 for six additional democracy measures: Freedom House – Average of Political Rights and Civil Liberties, as well as each index separately, and V-Dem – Electoral Democracy, Liberal Democracy and Suffrage. The results are consistent with the specifications with the BMR and Polity measures. While Percent Evangelized is positive and significant across specifications, Protestant Missionaries and Years Exposure to Protestant Missions are only significant in the specifications with no or few controls. In fact, in the V-Dem Suffrage specification, the effect of Protestant Missionaries is negative and significant. Once again, this raises the issue of omitted variable bias in the cross-sectional regressions, and means that Woodberry's results are unlikely to be causal.
In Table A16, we run regressions with a plethora of additional democracy variables, focusing on the period 1950–94 (as in Woodberry's original specifications), as well as from 1950 until the latest year of coverage of the democracy measure. The only measures for which we obtain consistent and significant positive results for the impact of Protestant Missionaries (across both periods) are the Vanhanen Contestation and Democratization scores. While the former measure is based on the level of the opposition's share of votes in parliamentary or presidential elections, the latter also takes into account the percentage of the population that actually voted. However, neither measure accounts for the state of liberal democracy based on civil liberties, political rights, or the existence of effective and fair legislative and executive bodies – a distinction that is critical for Woodberry's theory.
In Table A18, we present the results using Bayesian model averaging in order to account for uncertainty in the choice of explanatory variables. This approach distinguishes between focus regressors (which are always present in the model) and auxiliary regressors (about which the researcher is not certain). We thus specify Percent Evangelized as the focus regressor, and all other variables as the auxiliary regressors. We highlight independent variables with a posterior inclusion probability (pip)>0.5 or t-ratio>1 (in absolute value) to indicate that a regressor is robustly correlated with the dependent variable and thus should be included in the model.
The results show that only in Woodberry's (Reference Woodberry2012) original specification using the BP measure are all three Protestant variables robustly correlated with democracy. Using eleven alternative measures of democracy from the Freedom House, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013, Polity IV and V-Dem datasets, we estimate fifty-five BMA models in total and up to 17 million models per specification. The variable Protestant Missionaries is significant only in specifications without controls in select Freedom House and V-Dem models. Years Exposure to Protestant Missions is correlated more robustly with democracy measures, but overall there is no consistent pattern using the BMA method. On average, control variables, such as dummies for colonial history or the state of being landlocked, are identified as predictors with the highest posterior inclusion probabilities. This is further proof of the fragility of Woodberry's argument that Protestantism, rather than colonial history or exogenous characteristics, contributes to the development of democracy.
In Table A19, instead of considering the entire period 1950–2016, we present the results by decade, for a total of twenty-six democracy measures. Across all these specifications and democracy measures, we find no consistent pattern linking Protestantism and democracy over time (measured as Protestant missionaries per 1,000). Indeed, for some measures signs and significance alternate from decade to decade.
Questioning the Uniqueness of Protestantism
Our extension of Woodberry's results showed that Percent Evangelized is significant and robust across specifications. This suggests that further research and better data are needed on the role of different strands of Christianity in democratic emergence and consolidation. The analysis in this letter questions the uniqueness of Protestantism, and raises the possibility that it is (Western) Christianity overall that matters. For instance, Djankov and Nikolova (Reference Djankov and Nikolova2018) show that Catholics and Protestants (unlike those of Eastern Orthodox faith) have similar attitudes and values, including perceptions of government and democracy. Building on the previous literature,Footnote 16 Caicedo (Reference Caicedo2019) shows that Protestant as well as Catholic missions had a positive effect on educational attainment and income levels by increasing literacy and teaching various crafts to indigenous populations. This evidence undermines Woodberry's case for the exceptionalism of CP. However, more granular data on different Christian missions is needed to examine this question rigorously.
Conclusion
Woodberry (Reference Woodberry2012) argues that the emergence of a liberal and democratic modernity is the result of CPs. Despite his thorough theoretical analysis and use of both historical and econometric evidence, the quantitative analysis suffers from significant deficiencies. This letter shows that the results are not robust to using alternative measures of democracy or to extending the analyzed time frame. We find that CPs are, at best, a weak predictor of the development of democracy: after using twenty-six alternative democracy indices and extending the timeframe from 1994 to 2015/16, their effect becomes insignificant.
Woodberry's argument reminds us that the democracy literature has often ignored religious factors in favor of other ‘material’ ones such as inequality, economic development or revolutions. Although this is an important omission, rectifying it is not an easy task. In the cross-country setting, even slow-moving factors such as religion or culture rarely evolve in isolation. Woodberry's work shows not only that cross-country correlations – however intuitive – can be fragile, but also that establishing causality is extremely difficult. Our results are essential for further work on this topic, as they question Woodberry's proposed causal chain linking Protestantism and democracy. This does not mean that religion is unimportant, but that there is need for further debate, sub-national data collection and research on the topic in order to understand the (co)-evolution of religion and democracy.
Supplementary material
Data replication sets are available in Harvard Dataverse at: at: https://doi.org/10.7910/DVN/TRA0B6 and online appendices are available at https://doi.org/10.1017/S0007123420000174.
Acknowledgements
The authors thank the UCL Centre for Comparative Studies of Emerging Economies (and in particular Elodie Douarin) for financial support, Robert D. Woodberry for providing data and replication code, and Carles Boix, Miriam Manchin, Ken Scheve, David Stasavage, the Editor (René Lindstädt) and three referees for insightful comments. We also thank Xavier Marquez from Victoria University Wellington for aggregating and hosting a lot of democracy data.
