Hostname: page-component-6bf8c574d5-mggfc Total loading time: 0 Render date: 2025-02-15T14:27:41.602Z Has data issue: false hasContentIssue false
  • English
  • Français

Short of Suspension: How Suspension Warnings Can Reduce Hate Speech on Twitter

Published online by Cambridge University Press:  22 November 2021

Mustafa Mikdat Yildirim Open the ORCID record for Mustafa Mikdat Yildirim [Opens in a new window] ,
Jonathan Nagler Open the ORCID record for Jonathan Nagler [Opens in a new window] ,
Richard Bonneau Open the ORCID record for Richard Bonneau [Opens in a new window]  and
Joshua A. Tucker Open the ORCID record for Joshua A. Tucker [Opens in a new window]
Get access Rights & Permissions [Opens in a new window]

Abstract

Debates around the effectiveness of high-profile Twitter account suspensions and similar bans on abusive users across social media platforms abound. Yet we know little about the effectiveness of warning a user about the possibility of suspending their account as opposed to outright suspensions in reducing hate speech. With a pre-registered experiment, we provide causal evidence that a warning message can reduce the use of hateful language on Twitter, at least in the short term. We design our messages based on the literature on deterrence, and test versions that emphasize the legitimacy of the sender, the credibility of the message, and the costliness of being suspended. We find that the act of warning a user of the potential consequences of their behavior can significantly reduce their hateful language for one week. We also find that warning messages that aim to appear legitimate in the eyes of the target user seem to be the most effective. In light of these findings, we consider the policy implications of platforms adopting a more aggressive approach to warning users that their accounts may be suspended as a tool for reducing hateful speech online.

Type
Reflection
Information
Perspectives on Politics , Volume 21 , Issue 2: Special Section: Green Political Science , June 2023 , pp. 651 - 663
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

A list of permanent links to Supplemental Materials provided by the authors precedes the References section.

Data replication sets are available in Harvard Dataverse at: https://doi.org/10.7910/DVN/6FTRZZ

References

Beccaria, Cesare. 1963 [1764]. “On Crimes and Punishments.” Trans. H. Paolucci. Indianapolis, IN: Bobbs-Merrill.Google Scholar
Bodrunova, Svetlana S., Blekanov, Ivan, Smoliarova, Anna, and Litvinenko, Anna. 2019. “Beyond Left and Right: Real-World Political Polarization in Twitter Discussions on Inter-Ethnic Conflicts.” Media and Communication 7(3): 119–32. https://doi.org/10.17645/mac.v7i3.1934CrossRefGoogle Scholar
Broockman, David, and Kalla, Joshua. 2016. “Durably Reducing Transphobia: A Field Experiment on Door-to-Door Canvassing.” Science 352(6282): 220–24.CrossRefGoogle ScholarPubMed
Chandrasekharan, Eshwar, Pavalanathan, Umashanthi, Srinivasan, Anirudh, Glynn, Adam, Eisenstein, Jacob, and Gilbert, Eric. 2017. “You Can't Stay Here: The Efficacy of Reddit's 2015 Ban Examined through Hate Speech.” Proceedings of the ACM on Human-Computer Interaction 1(CSCW): 1–22.CrossRefGoogle Scholar
Charnysh, Volha, Lucas, Christopher, and Singh, Prerna. 2015. “The Ties that Bind: National Identity Salience and Pro-Social Behavior toward the Ethnic Other.” Comparative Political Studies 48(3): 267–300.CrossRefGoogle Scholar
Chowdhury, Farhan Asif, Allen, Lawrence, Yousuf, Mohammad, and Mueen, Abdullah. 2020. “On Twitter Purge: A Retrospective Analysis of Suspended Users.” In Companion Proceedings of the Web Conference 2020, 371-378. https://doi.org/10.1145/3366424.3383298CrossRefGoogle Scholar
Conzola, Vincent C., and Wogalter, Michael S.. 2001. “A Communication–Human Information Processing (C–HIP) Approach to Warning Effectiveness in the Workplace.” Journal of Risk Research 4(4): 309–22.CrossRefGoogle Scholar
Cusson, Maurice. 1993. “Situational Deterrence: Fear during the Criminal Event.” Crime Prevention Studies 1(3): 55–68.Google Scholar
Franco, Annie, Malhotra, Neil, and Simonovits, Gabor. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345(6203): 1502–505.CrossRefGoogle ScholarPubMed
Gagliardone, Iginio, Pohjonen, Matti, Beyene, Zenebe, Zerai, Abdissa, Aynekulu, Gerawork, Bekalu, Mesfin, Bright, Jonathan et al. 2016. “Mechachal: Online Debates and Elections in Ethiopia—From Hate Speech to Engagement in Social Media.” Available at SSRN 2831369 (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2831369).CrossRefGoogle Scholar
Geerken, Michael R., and Gove, Walter R.. 1974. “Deterrence: Some Theoretical Considerations.” Law and Society Review 9(3): 497–513.CrossRefGoogle Scholar
Gibbs, Jack P. 1968. “Crime, Punishment, and Deterrence.” Southwestern Social Science Quarterly 48(4): 515–30.Google Scholar
Guynn, Jessica. 2020. “Facebook Ranks Deleting Anti-Black and ‘Most Harmful’ Hate Speech over Comments about White People and Men.” USA Today, December 3. Retrieved March 1, 2021 (https://www.usatoday.com/story/tech/2020/12/03/facebook-ranks-hate-speech-black-over-attacks-white-people-men/3813931001/).Google Scholar
Guynn, Jessica. 2021.“Donald Trump Ruled Facebook, Twitter before He Was Banned. Will @realdonaldtrump Log into Gab or Somewhere Else?” USA Today, February 8. Retrieved March 1, 2021 (https://www.usatoday.com/story/tech/2021/02/08/trump-facebook-twitter-youtube-ban-where-next-gab-parler/4440645001/).Google Scholar
Jacobs, Bruce A. 2010. “Deterrence and Deterrability.” Criminology 48(2): 417–41.CrossRefGoogle Scholar
Kalla, Joshua L., and Broockman, David E.. 2020. “Reducing Exclusionary Attitudes through Interpersonal Conversation: Evidence from Three Field Experiments.” American Political Science Review 114(2): 410–25.CrossRefGoogle Scholar
Kiesler, Sara, Kraut, Robert, Resnick, Paul, and Kittur, Aniket. 2012. “Regulating Behavior in Online Communities.” In Building Successful Online Communities: Evidence-Based Social Design, Kraut, Robert E. and Resnick, Paul, 125–178. Cambridge, MA: MIT Press. https://doi.org/10.7551/mitpress/8472.001.0001Google Scholar
Kumar, Sumit, and Pranesh, Raj Ratn. 2021. “TweetBLM: A Hate Speech Dataset and Analysis of Black Lives Matter-related Microblogs on Twitter.” (https://arxiv.org/abs/2108.12521)Google Scholar
Livni, Ephrat. 2019. “Twitter, Facebook, and Insta Bans Send the Alt-Right to Gab and Telegram.” qz, May 12. Retrieved March 1, 2021 (https://qz.com/1617824/twitter-facebook-bans-send-alt-right-to-gab-and-telegram/).Google Scholar
Montanaro, Domenico. 2021. “Trump Teases Starting His Own Social Media Platform. Here's Why It'd Be Tough.” NPR, March 24. Retrieved June 1, 2021 (https://www.npr.org/2021/03/24/980436658/trump-teases-starting-his-own-social-media-platform-heres -why-itd-be-tough).Google Scholar
Munger, Kevin. 2017. “Tweetment Effects on the Tweeted: Experimentally Reducing Racist Harassment.” Political Behavior 39(3): 629–49.CrossRefGoogle Scholar
Munger, Kevin. 2020. “Don’t @ Me: Experimentally Reducing Partisan Incivility on Twitter.” Journal of Experimental Political Science 8(2): 102–16. doi:10.1017/XPS.2020.14CrossRefGoogle Scholar
Müller, Karsten, and Schwarz, Carlo. 2018. “Fanning the Flames of Hate: Social Media and Hate Crime.” Journal of the European Economic Association 19(4): 2131–67.CrossRefGoogle Scholar
Müller, Karsten, and Schwarz, Carlo. 2020. “From Hashtag to Hate Crime: Twitter and Anti-Minority Sentiment.” Available at SSRN 3149103 (https://ssrn.com/abstract=3149103).Google Scholar
Nagin, Daniel S. 1998. “Criminal Deterrence Research at the Outset of the Twenty-First Century.” Crime and Justice 23: 1–42.CrossRefGoogle Scholar
Paluck, Elizabeth Levy, and Green, Donald P.. 2009a. “Deference, Dissent, and Dispute Resolution: An Experimental Intervention Using Mass Media to Change Norms and Behavior in Rwanda.” American Political Science Review 103(4): 622–44.CrossRefGoogle Scholar
Paluck, Elizabeth Levy, and Green, Donald P.. 2009b. “Prejudice Reduction: What Works? A Review and Assessment of Research and Practice.” Annual Review of Psychology 60(1): 339–67.CrossRefGoogle ScholarPubMed
Paternoster, Raymond. 1987. “The Deterrent Effect of the Perceived Certainty and Severity of Punishment: A Review of the Evidence and Issues.” Justice Quarterly 4(2): 173–217.CrossRefGoogle Scholar
Pennycook, Gordon, Epstein, Ziv, Mosleh, Mohsen, Arechar, Antonio A., Eckles, Dean, and Rand, David G.. 2021. “Shifting Attention to Accuracy Can Reduce Misinformation Online.” Nature 592(7855): 590–95.CrossRefGoogle ScholarPubMed
Peters, Jay. 2020. “Twitter Now Bans Dehumanizing Remarks Based on Age, Disability, and Disease.” theverge, March 5. Retrieved March 1, 2021 (https://www.theverge.com/2020/3/5/21166940/twitter-hate-speech-ban-age-disability-disease-dehumanize).Google Scholar
Pettigrew, Thomas F. 1998. “Intergroup Contact Ttheory.” Annual Review of Psychology 49(1): 65–85.CrossRefGoogle Scholar
Rogers, Ronald W. 1975. “A Protection Motivation Theory of Fear Appeals and Attitude Change.” Journal of Psychology 91(1): 93–114.CrossRefGoogle Scholar
Samii, Cyrus. 2013. “Perils or Promise of Ethnic Integration? Evidence from a Hard Case in Burundi.” American Political Science Review 107(3): 558–73.CrossRefGoogle Scholar
Sherman, Lawrence W. 1993. “Defiance, Deterrence, and Irrelevance: A Theory of the Criminal Sanction.” Journal of Research in Crime and Delinquency 30(4): 445–73.CrossRefGoogle Scholar
Siegel, Alexandra A., and Badaan, Vivienne. 2020. “# No2Sectarianism: Experimental Approaches to Reducing Sectarian Hate Speech Online.” American Political Science Review 114(3): 837–55.CrossRefGoogle Scholar
Silic, Mario, Silic, Dario, and Oblakovic, Goran. 2016. “Restrictive Deterrence: Impact of Warning Banner Messages on Repeated Low-Trust Software Use.” Presented at the 18th International Conference on Enterprise Information Systems (ICEIS 2016), April 25-28. http://doi.org/10.5220/0005831904350442CrossRefGoogle Scholar
Simonovits, Gábor, Kezdi, Gabor, and Kardos, Peter. 2018. “Seeing the World through the Other's Eye: An Online Intervention Reducing Ethnic Prejudice.” American Political Science Review 112(1): 186–93.CrossRefGoogle Scholar
Spangler, Todd. 2020. “Reddit Finally Bans Hate Speech, Removes 2,000 Racist and Violent Forums Including The_Donald.” variety, June 29. Retrieved March 1, 2021 (https://variety.com/2020/digital/news/reddit-bans-hate-speech-groups-removes-2000-subreddits-donald-trump-1234692898/).Google Scholar
Stafford, Mark C., and Warr, Mark. 1993. “A Reconceptualization of General and Specific Deterrence.” Journal of Research in Crime and Delinquency 30(2): 123–35.CrossRefGoogle Scholar
Stockman, Mark, Heile, Robert, and Rein, Anthony. 2015. “An Open-Source Honeynet System to Study System Banner Message Effects on Hackers.” In Proceedings of the 4th annual ACM Conference on Research in Information Technology, 19-2.Google Scholar
Takikawa, Hiroki, and Nagayoshi, Kikuko. 2017. “Political Polarization in Social Media: Analysis of the “Twitter Political Field” in Japan.” 2017 IEEE International Conference on Big Data (Big Data). https://doi.org/10.1109/BigData41644.2017CrossRefGoogle Scholar
Testa, Alexander, Maimon, David, Sobesto, Bertrand, and Cukier, Michel. 2017. “Illegal Roaming and File Manipulation on Target Computers: Assessing the Effect of Sanction Threats on System Trespassers’ Online Behaviors.” Criminology & Public Policy 16(3): 689–726.CrossRefGoogle Scholar
Wilson, Theodore, Maimon, David, Sobesto, Bertrand, and Cukier, Michel. 2015. “The Effect of a Surveillance Banner in an Attacked Computer System: Additional Evidence for the Relevance of Restrictive Deterrence in Cyberspace.” Journal of Research in Crime and Delinquency 52(6): 829–55.CrossRefGoogle Scholar
Wogalter, Michael S. 2006. “Communication-Human Information Processing (C-HIP) Model.” Handbook of Warnings: Case Studies and Analyses, 51–61. Boca Raton: CRC Press.CrossRefGoogle Scholar
Ziems, Caleb, He, Bing, Soni, Sandeep, and Kumar, Srijan. 2020. “Racism Is a Virus: Anti-Asian Hate and Counterhate in Social Media during the COVID-19 Crisis.” arXiv preprint. (arXiv:2005.12423).Google Scholar
Supplementary material: File

Yildirim et al. supplementary material

Appendices A-J

Download Yildirim et al. supplementary material(File)
File 6 MB
Supplementary material: Link

Yildirim et al. Dataset

Dataset

https://doi.org/10.7910/DVN/6FTRZZ
Link