Citizen deliberation

At the core of the deliberative theory is the deliberative process, which consists of argumentation and fact-based discussion between free and equal citizens, or their representatives (Fung, Reference Fung2003: 344). This is considered more likely to result in ‘good’ policies than an aggregation of individual preferences, as deliberators must justify their views in ways acceptable to others (e.g. Habermas, Reference Habermas1996: 110). Ideally, scholars have proposed that a deliberative process should display several attributes as follows: inclusion, plurality of views, equality of discussion, reciprocity, reasoned justification, reflection, and sincerity (c.f. Kies, Reference Kies2010: 40–53). Although some criteria – such as sincerity and respect – are hard to operationalise (e.g. Monnoyer-Smith and Wojcik, Reference Monnoyer-Smith and Wojcik2012: 43–44), deliberative discussions are, nonetheless, normally designed so that these ideals of a deliberative process can be realised, for instance, by enforcing discussion rules and using moderation in the discussions (Ryfe, Reference Ryfe2005: 63).

Beyond process-based attributes, it is argued that deliberation has potential outcome-related effects. This entails that citizens taking part in deliberation are perceived to be potentially affected by it in several ways. Essentially, the common denominator is the thought of deliberation ‘empowering citizens’ (Barber, Reference Barber1984: 232; Fishkin, Reference Fishkin2009: 23–26) and augmenting their competence for democratic participation (Fung, Reference Fung2003). First, a deliberative process is perceived as a method to change individual preferences and values towards a more collective understanding, as decisions are met through discussions where views need to be justified, and the arguments of others are to be taken into consideration (e.g. Fishkin, Reference Fishkin1995: 142–143). Barber (Reference Barber1984: 232), for instance, regards deliberation as equipping citizens ‘with a capacity for common vision’. This could be labelled as the main ‘democratic outcome’ of deliberative discussion. However, Grönlund et al. (Reference Grönlund, Setälä and Herne2010) remark that deliberation also may result in other ‘side-effects’. Essentially, such effects entail civic virtues or, as Fung 2003: 349–350 calls them, civic skills, which are not the intended effects – that is, the aforementioned collective understanding and vision for the common good – of deliberation, but which, nonetheless, often occur as a result. Thus, an increase in interpersonal trust, political knowledge, feelings of internal and external political efficacy, trust for political institutions, and a readiness for political action have all been discussed in the literature as being potential individual-level outcomes of deliberation (see also Fung, Reference Fung2003: 349–352).

Interpersonal trust might be augmented through the deliberative setting, because, ideally, people with different social backgrounds are ‘forced’ to listen and take other peoples’ opinions into consideration in a setting that enforces rules of sincerity and mutual respect (Dryzek and List, Reference Dryzek and List2003; Grönlund et al., Reference Grönlund, Setälä and Herne2010: 99). Other studies (e.g. Fishkin et al., Reference Fishkin, Luskin and Jowell2000) have indicated that participation in deliberation increases knowledge on the issue being deliberated. Although that is the logically expected outcome, Esterling et al. (Reference Esterling, Neblo and Lazer2011: 485) add that deliberators also seem to have the capacity to ‘encode policy information from outside the context of the deliberation’, and thus achieve an increased general political learning (c.f. Fung, Reference Fung2003: 349–350).

Moreover, it has been argued that participants’ feelings of political efficacy – that is, their confidence in their own political competence – have likely been augmented through deliberation (Gastil, Reference Gastil2000: 358). However, the opposite might also hold true. Being challenged and confronted with opposing views may cause doubts in one’s position and beliefs (Mutz, Reference Mutz2006: 119) or doubt in the effectiveness of group deliberation itself (Gastil, Reference Gastil2000: 358). As Grönlund et al. (Reference Grönlund, Setälä and Herne2010: 98) point out, the scholarly evidence on the effects of deliberation on internal efficacy is inconclusive to date. Turning to external political efficacy, however, Grönlund et al. (Reference Grönlund, Setälä and Herne2010: 98) state that some scholarly evidence exists in support of the notion that deliberation will increase citizens’ feelings that their views and preferences have an influence on the political process – that is, external efficacy. However, it is certainly plausible that being confronted with differing views, and having one’s own views challenged by others, may actually decrease the feeling of external efficacy as well. In addition, one would expect that the impact of deliberation on external political efficacy might be partially contingent on whether the deliberation has an impact on actual policy or not. This thought is also indicated by Fung (Reference Fung2003: 350), who states that increased civic skills are likelier when the deliberations have tangible consequences for the participants – what he calls an ‘empowered deliberation’.

The notion of deliberation having an impact on readiness for political action – an idea that indeed may sound farfetched at first glance – is stringently depicted by Gastil (Reference Gastil2000: 359; c.f. Fung, Reference Fung2003: 352): ‘Past research has shown clear links between […] political efficacy and political action […] therefore, if deliberation can boost participants’ sense of political efficacy, it may also spur greater political engagement’. Thus, an interconnectivity between the various democratic outcomes – in this example between efficacy and proneness for action – of deliberation is indicated. Similarly, it is logical to expect that trust in political institutions might be positively affected by participation in deliberation, partly because of the expected increase in external political efficacy among deliberators. In particular, trust for legislative political institutions – such as parliaments and governments – would be the most likely to receive an increase among deliberators, as deliberators learn and understand the complexities related to political issues and may, thus, become more understanding and trusting of the procedures and actors of representative democracy (e.g. Grönlund et al., Reference Grönlund, Setälä and Herne2010: 99).

However, it should be noted that deliberative theory has received a significant share of criticism. Doubts have been raised as to whether people actually want to take part in decision making, let alone time-consuming deliberation (Hibbing and Theiss-Morse, Reference Hibbing and Theiss-Morse2002). Other concerns pertain directly to the actual deliberative process; Mutz (Reference Mutz2006), for instance, questions whether people are comfortable with situations in which disagreement frequently takes place. In addition, fears have been raised of so-called group polarisation taking place in deliberation among like-minded citizens (enclave deliberation), whereby the group’s average opinion, although becoming more coherent, moves towards a more extreme value as a result of deliberation (Sunstein, Reference Sunstein2009).

Online deliberation: potential and initial findings

As was briefly mentioned above, scholars have looked upon the internet as a potentially suitable venue in which to realise deliberation (e.g. Dahl, Reference Dahl1989: 519; Luskin et al., Reference Luskin, Fishkin and Iyengar2006). Often, the practical organisation of deliberative opportunities has been a significant obstacle when making large-scale deliberation work offline (Coleman and Goetze, Reference Coleman and Goetze2001). Although mini-publics including citizen juries, consensus conferences, and deliberative polls have made some progress, the internet stands in a comparably favourable light as a venue for conveying and organising large-scale deliberative discussions (Luskin et al., Reference Luskin, Fishkin and Iyengar2006: 6–8; Price Reference Price2009). Specifically, the online environment removes many physical constrains – such as time, access, geographical distances – which are often problematic for citizen deliberations offline (see Strandberg, Reference Strandberg2008; Davies et al., Reference Davies, O’Connor, Cochran, Effrat, Parker, Newman and Tam2009: 278–279; Manosevitch, Reference Manosevitch2010).

Beyond practical advantages, some scholars have argued that the online environment, in being more anonymous and non-verbal in nature than discussions offline, might free people from psychological barriers, which could restrict their engagement in deliberation. Stromer-Galley (Reference Stromer-Galley2002: 35–38) found that, for instance, the feeling of being private while typing on your computer at home while, nonetheless, engaging in public political conversations online provides a relief for citizens who might feel uncomfortable in face-to-face settings. Likewise, scholars have optimistically recognised that the online setting could reduce the, often negative, influence of status and social position on citizens’ discussions (e.g. Dahlberg, Reference Dahlberg2001: 14; Witschge, Reference Witschge2004: 116; Price, Reference Price2009: 43). Gastil (Reference Gastil2000: 359), stresses that ‘…if computer-mediated interaction can consistently reduce the independent influence of social-status, it will have a powerful advantage over face-to-face deliberation’. Thus, if focus shifts from who is saying something to what is actually said, the ‘force of the better argument’, central to deliberative theory (e.g. Habermas, Reference Habermas1996), would be more likely to prevail. However, conditions do not always mean that the potential is fulfilled. Concerning anonymity, for instance, several scholars (e.g. Wilhelm, Reference Wilhelm1999; Papacharissi, Reference Papacharissi2004) argue that it actually contributes to a lack of civility and respect, often found in online discussions, as well as reducing peoples’ feelings of responsibility for their inputs (Coleman and Moss, Reference Coleman and Moss2012: 8). In addition, although some studies (e.g. Stromer-Galley, Reference Stromer-Galley2002; Jensen, Reference Jensen2003) suggest that discussions that fulfil deliberative criteria can be found online, many studies conducted hitherto have not readily found support regarding online discussions living up to the rigid qualitative standards of deliberation (e.g. Wilhelm, Reference Wilhelm1999; Jankowski and Van Os, Reference Jankowski and Van Os2004).

Nonetheless, although the online reality has, for various reasons, often been discouraging from a deliberative perspective, some scholars have shed light on an interesting observation (Hamlett and Cobb, Reference Hamlett and Cobb2006; Wright and Street, Reference Wright and Street2007: 850; Coleman and Moss, Reference Coleman and Moss2012: 6): citizen discussion venues designed to be democratic appear to have a better chance of achieving deliberative-type conversations. Therefore, they are more likely to bring about outcomes potentially beneficial to democracy (c.f. Monnoyer-Smith and Wojcik, Reference Monnoyer-Smith and Wojcik2012). After all, the online environment is not per se democratic, and it might be unreasonable to expect that it will bring about democratic citizens’ discussions by itself. Thus, as Barber (Reference Barber1984: 586) states: ‘Technology then can help democracy but only if programmed to do so […]’.

Indeed, a brief overview of the findings from existing experiments in online deliberation (see Reference to author’s own work 2014 for in-depth summary) shows some signs of such discussions partially living up to deliberative standards. Opinion changes, and increased coherence of opinions are, with some exceptions (e.g. Min, Reference Min2007), regularly reported as the main findings in studies (e.g. Price and Capella, Reference Price and Capella2002; Muhlberger, Reference Muhlberger2005: 8; Luskin et al., Reference Luskin, Fishkin and Iyengar2006; Strandberg, Reference Strandberg2008; Price, Reference Price2009; Smith et al., Reference Smith, John, Sturgis and Nomura2009: 21; Bächtiger, Reference Bächtiger2011; Esterling, et al., Reference Esterling, Neblo and Lazer2011; Strandberg and Grönlund, Reference Strandberg and Grönlund2012). Knowledge gains are also often reported, albeit scholars have been uncertain whether these occur because of deliberation or because of the reading of information packages within the deliberation process (see Muhlberger, Reference Muhlberger2005; Strandberg and Grönlund, Reference Strandberg and Grönlund2012). As for other so-called ‘side-effects’ of deliberation, Min (Reference Min2007) found that taking part in an online deliberation increased both participants’ efficacy and willingness to participate in politics significantly. Likewise, Price and Capella (Reference Price and Capella2002: 319–321) found that participants had become significantly more willing to engage in politics and other civic activities, and that their generalised trust also increased after engaging in several waves of online deliberation. Strandberg and Grönlund (Reference Strandberg and Grönlund2012), however, only found a few weak side-effects of deliberating online. Surprisingly, they even found that participants had become slightly less eager to participate in politics after deliberating. Rhee and Kim (Reference Rhee and Kim2009: 230) did not find any significant findings concerning civility, trust, and tolerance in their online deliberation experiment.

Design

A 2×2 factorial design with a separate control group was applied to the experiment.Footnote ³ The benefits of combining a factorial design with a control group are that they enable assessments of both relative effects – differences between treatment groups – and absolute effects – that is, the differences between treatment groups and the control group (see Research Methods Knowledge Base, 2006). Besides variation in the discussion setting – being designed either according to deliberative or non-deliberative principles – this design also includes a variation in the anonymity of participating citizens. Several scholars maintain that whether the identities of the citizens who are discussing are known to each other or not (e.g. Jensen, Reference Jensen2003; Janssen and Kies, Reference Janssen and Kies2005; Davies and Chandler, Reference Davies and Chandler2011: 3) can affect the quality of a discussion as well as citizen deliberations specifically (c.f. Bächtiger, Reference Bächtiger2011). A general critique towards the research in online deliberation has, in fact, been that it has not tested the boundaries of this deliberation, relying mainly on creating favourable conditions for achieving the desired outcomes (Coleman and Moss, Reference Coleman and Moss2012: 6; Strandberg and Grönlund, Reference Strandberg and Grönlund2014). Thus, by including the anonymity factor and also by selecting a controversial discussion topic (same-sex adoption rights),Footnote ⁴ which also tends to influence discursive quality (e.g. Smith et al., Reference Smith, John and Sturgis2013: 711), the conditions under which deliberative design has positive effects on democratic outcomes, or not, can also be explored.

In sum, the design of the experiment consists of four cells in a 2 (discussion setting)×2 (anonymity) factorial design with a separate control group (Figure 1):

Figure 1 Experimental design: 2×2 factorial design with control group.

For the purpose of this study, examining only the main effect of deliberative design across two settings of anonymity and not the full factorial interaction effects of anonymity and deliberation, the relevant comparisons in the experimental design (Figure 1) are between cell 1 and cell 2 and between cell 3 and cell 4, respectively, concerning the assessment of the relative effects of deliberative design on the democratic outcomes. Concerning the absolute effects, each treatment cell is compared, in turn, with the control group.

Participants

A total of 70 adults aged 17 to 68 (mean age 28) participated in the experiment. In all, 52%percent were women and 48% were men. In line with the recommendation by Druckman and Kam (Reference Druckman and Kam2011: 54), the recruitment strategy was to have a dual sample of students and non-students.Footnote ⁵ Although a representative sample is often preferred in deliberation, for instance, to reduce the risk of enclave discussions where only like-minded people discuss with each other (Fishkin, Reference Fishkin2009: 33), the use of students was of necessity, as the participation in this experiment required extensive online skills such as use of webcams and headset equipment (c.f. Kies, Reference Kies2010: 67) – skills that are common among young people such as students. In the final sample, 64% of the participants were students. According to Druckman and Kam (Reference Druckman and Kam2011: 41), the common practice of using convenience samples of students in experiments is often heavily criticised. However, they continue: ‘in contrast to common claims, we argue that student subjects do not intrinsically pose a problem for a study’s external validity’ (see Druckman and Kam, Reference Druckman and Kam2011 for empirical evidence in support of this claim). Nonetheless, it has to be acknowledged that there is a risk of so-called crystallisation effects, due to the fact that graduate students may be relatively strong in those skills required for deliberating, which could mean that the experiment fails to pick up differences in the democratic outcomes as there was too little variation in deliberative capacity (Druckman and Kam, Reference Druckman and Kam2011: 45–46). In other words, regardless of having treatment stimuli, which vary the discursive settings, the characteristics of the participants may ‘triumph’ over the discussion setting, and thus blur the findings.

All participants were promised a gift voucher worth 50 euros and to be entered into a lottery for a weekend boat cruise. Participants who were interested in taking part signed up for the experiment on the study’s website. At this stage, basic demographic information on the participants was collected. Of course, such a non-probability-based recruitment method is vulnerable to volunteering, which could negatively affect the representativeness of the sample (Strandberg and Grönlund, Reference Strandberg and Grönlund2014: 4). However, it is necessary to bear in mind that the purpose of this experiment is not to achieve generalisation about how citizens discuss online, in which case the demographic representativeness of the sample would have been crucial. Instead, the focus here is explicitly on studying the effects of the variation in discursive settings: what happens to the democratic outcomes if we vary the deliberative design across two settings of anonymity? Nonetheless, as discussed earlier, the characteristics of the sample could pose some limitations on the ability to generalise concerning the impact of a discursive setting on the outcomes of discussion. On the other hand, one could argue that the experiment’s population constitutes a rather critical test (c.f. Druckman and Kam, Reference Druckman and Kam2011: 53): if between-treatment differences in post-test outcomes are found, even though the characteristics of the population make these unlikelier than among the general population logic would state that the same manipulation is likely to also have an effect on a more representative population, where, for instance, the risks of crystallisation (Druckman and Kam, Reference Druckman and Kam2011: 45–46) or enclave-deliberation (Fishkin, Reference Fishkin2009: 33) are smaller.

Stimulus

The stimulus consisted of taking part in online discussions that were set up according to the typology depicted in Figure 1 (i.e. each cell is represented by one discussion group). Each discussion setting was either an anonymous or non-anonymous deliberative setting, or an anonymous or non-anonymous non-deliberative discussion. Each participant only participated in one type of discussion. Similarly to some earlier online deliberation studies (e.g. Price and Capella, Reference Price and Capella2002; Noveck, Reference Noveck2004; Muhlberger, Reference Muhlberger2005; Luskin et al., Reference Luskin, Fishkin and Iyengar2006; Strandberg and Grönlund, Reference Strandberg and Grönlund2012, Reference Strandberg and Grönlund2014), the deliberative settings (cells 2 and 4 in Figure 1) were modelled after offline mini-publics. Thus, these were live (synchronous) discussion sessions, which adhered to deliberative guidelines – that is, being honest and sincere, treating each other with respect, accepting the presence and merits of differing views, encouragement of arguing ones’ viewpoints, etc. – which were visible to the participants on a whiteboard in the discussion room during the whole session. A trained facilitator, who was tasked with enforcing the deliberative guidelines, ensuring that the flow of discussion was good, that everyone participated, and helped with technical problems, was also used in both the treatments, which were designed to be deliberative. The anonymous session in the deliberative setting was text-based using pseudonyms for each participant. The non-anonymous session in the deliberative setting used webcams and microphones, and each participant’s identity (i.e. their full name) was known to the other participants. The non-deliberative treatment groups (cells 1 and 3 in Figure 1) took part in a three-week-long text-based asynchronous discussion on a regular discussion board. Thus, participants merely received a topic of discussion but no further assistance nor guidelines on how to conduct themselves (only a legal disclaimer was present). All discussions were held on the study’s website (see more in procedures section). Admittedly, it has to be stressed that the ways in which the discussions were set up introduced additional factors – that is, text vs. audio/video and synchronous/asynchronous – which varied between the groups, thus reducing the internal validity of the experiment. Although it is yet unclear how these affect deliberations and outcomes thereof (see summary in Price, Reference Price2009: 42–43), some scholars suggest that asynchronous text-based discussion is to be preferred for deliberations (e.g. Davies et al., Reference Davies, O’Connor, Cochran, Effrat, Parker, Newman and Tam2009: 283; Smith et al., Reference Smith, John and Sturgis2013: 710), whereas others suggest that synchronous voice-based discussions are better suited to host deliberations (Noveck, Reference Noveck2004: 27; Fishkin, Reference Fishkin2009). The presence of these factors will, nonetheless, be taken into account when interpreting the findings of the experiment.

Dependent variables

The dependent variables were all measured in the post-test questionnaire, which was disseminated electronically via the platform’s website immediately after the respective discussions had ended. The items in the post-test survey were standard survey batteries used previously (e.g. Grönlund et al., Reference Grönlund, Setälä and Herne2010; Strandberg and Grönlund, Reference Strandberg and Grönlund2012) to measure the outcomes – that is, opinions and values, opinion coherence, trust, efficacy, and propensity for political action – of citizen discussions. The items concerning opinions and values were chosen to include not only items directly related to same-sex adoption rights specifically but also closely related policy areas such as family policy in general (see Appendix for full questionnaire). Between-group differences – both relative and absolute effects – were analysed post-test on a group aggregate level.

Procedure

The experiment was conducted as an online Lab-in-the-Field experiment (Morton and Williams, Reference Morton and Williams2010: 296–297), from October to November 2013, in a recently developed online platform for experimental research. The use of an online platform for the experiment can be seen as a means with which to increase the external validity of the experiment. Thus, the setting retains some of the benefits of a controlled lab experiment, such as a relatively high internal validity, and some of the higher external relevance – beyond the lab – of a field experiment.

The experiment consisted of the following three steps: a recruitment stage, the random allocation to treatment groups and the control group, and an experimental stage. Initially, the true purpose of the experiment was withheld from the participants in order to avoid goal-directed behaviour. Thus, they were not aware of the different treatments applied in the experiment. Only that it was a study of online discussion was made aware to the participants. After taking part in their respective discussions, participants were asked to answer the post-test questionnaire. When all participants had submitted their post-test questionnaire, they were debriefed on the actual purpose of the experiment. The setup and stages of the experiment are summarised in Table 1.

Table 1 The stages of the experiment according to treatments

Anonymous setting

The exploration of the main effects of deliberative design starts by examining the relative effects within the anonymous setting. Thus, in Table 2, the post-test mean ranks (used instead of means due to the non-parametric data) of the deliberative design treatment are compared with those of the non-deliberative design treatment.

Table 2 Anonymous setting: relative effects of the deliberative design

*P<0.10.

Each mean rank value is for the deliberative design treatment.

Looking at the findings in Table 2, all the statistically significant differences indicate that the discussion in the forum, designed to be deliberative, had a more positive impact on participants than the regular online discussion. It is noteworthy, though, that there were many items for which no significant between-group differences were found, and the non-parametric effect sizes were also rather small, ranging between 0.11 and 0.40. Nonetheless, in light of the findings that did turn out as significant, deliberative design affected one opinion, the degree of coherence of opinions (although only when measured in range), one item on efficacy, and two items concerning trust for institutions. There were no significant effects indicating more positive outcomes for the non-deliberative design treatment. The presentation of the findings within the anonymous setting now continues by looking at the absolute effects of the two discussions – that is, non-deliberative design vs. deliberative design – when compared with the post-test levels of the control group’s participants (Table 3).

Table 3 Anonymous setting: comparison of absolute effects of the deliberative design

*Significance >0.10; **significance <0.05; ***significance <0.01.

Interestingly, most of the statistically significant differences in Table 3 indicate absolute negative effects compared with the control group for both the non-deliberative and deliberative design treatment groups (effect sizes ranging between 0.01 and 0.55). How to interpret this will be discussed in the conclusion. However, comparing the two treatments’ absolute effects side-by-side, the general pattern is that the deliberative design treatment has much less statistically significant absolute negative effects than the non-deliberative design treatment. Thus, when looking at absolute treatment effects on the whole (Table 3), the findings point in the same direction as the findings for the relative effects (Table 2): deliberative design brought about more positive effects than the non-deliberative design when the discussions took place in an anonymous setting (or at least fewer negative absolute effects). Whether the same patterns are found for the non-anonymous setting will be examined next.

Non-anonymous setting

The examination of the effects of deliberative design in the non-anonymous setting starts by looking at the relative effects (Table 4).

Table 4 Non-anonymous setting: relative effects of the deliberative design

*P<0.10; **P<0.05; ***P<0.01.

Each mean rank value is for the deliberative design treatment.

Generally, although being less evident than in the anonymous setting, the statistically significant findings, which are presented in Table 4, tend to indicate more positive effects for deliberative design than for non-deliberative design. Only concerning ‘readiness for political action’ do both the statistically significant findings show relatively positive effects for the non-deliberative design. For the other aspects – that is, one opinion item, both coherence measures, one efficacy item, and one concerning institutional trust – the deliberative design brought about more positive outcomes among participants. Conversely, one item for efficacy (albeit with the smallest effects size of all significant effects) and ‘trust for the president’ displayed a more positive outcome for the non-deliberative design treatment. Bringing the presentation of the findings of the experiment to a close, Table 5 depicts a comparison of the absolute effects of deliberation in the non-anonymous setting.

Table 5 Non-anonymous setting: comparison of absolute effects of the deliberative design

*Significance >0.10; **significance <0.05; ***significance <0.01.

For the non-deliberative design treatment, as seen in Table 5, most significant absolute effects were negative; only in one opinion item did a positive absolute effect occur. However, the significant absolute effects for the deliberative design treatment are positive in four cases (one opinion, coherence, and two efficacy items) and negative only in two cases (both regarding readiness for collective action). Thus, although the difference in modality and temporality (asynchronous vs. synchronous) used in the two treatment groups could blur the findings, the side-by-side comparison of absolute effects, in the non-anonymous setting, indicates more positive effects for the deliberative design than for the non-deliberative design. In addition, it is again interesting to note that the absolute effects tend to be either negative or not significant, indicating that the control group’s participants were either more positive than the participants in the treatment groups or mostly equally as positive.