Abstract
How can we induce social media users to be discerning when sharing information during a pandemic? An experiment on Facebook Messenger with users from Kenya (n = 7,498) and Nigeria (n = 7,794) tested interventions designed to decrease intentions to share COVID-19 misinformation without decreasing intentions to share factual posts. The initial stage of the study incorporated: (1) a factorial design with 40 intervention combinations; and (2) a contextual adaptive design, increasing the probability of assignment to treatments that worked better for previous subjects with similar characteristics. The second stage evaluated the best-performing treatments and a targeted treatment assignment policy estimated from the data. We precisely estimate null effects from warning flags and related article suggestions, tactics used by social media platforms. However, nudges to consider the accuracy of information reduced misinformation sharing relative to control by 4.9% (estimate = −2.3 percentage points, 95% CI = [−4.2, −0.35]). Such low-cost scalable interventions may improve the quality of information circulating online.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The data that support the findings of this study are available at https://github.com/gsbDBI/infodemic-replication. Source data are provided with this paper.
Code availability
The analysis code that generate the figures, tables and results presented in this study are available at https://github.com/gsbDBI/infodemic-replication.
References
World Population Review (Facebook, 2022); worldpopulationreview.com/country-rankings/facebook-users-by-country
Swire-Thompson, B., DeGutis, J. & Lazer, D. Searching for the backfire effect: measurement and design considerations. J. Appl. Res. Mem. Cogn. 9, 286–299 (2020).
Even-Dar, E., Mannor, S., Mansour, Y. & Mahadevan, S. Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. J. Mach. Learn. Res. 7, 1079–1105 (2006).
Caria, S. et al. An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan (CESifo, 2020).
Kasy, M. & Sautmann, A. Adaptive treatment assignment in experiments for policy choice. Econometrica 89, 113–132 (2021).
Athey, S. et al. Contextual bandits in a survey experiment on charitable giving: within-experiment outcomes versus policy learning. Preprint at https://doi.org/10.48550/arXiv.2211.12004 (2022).
Pennycook, G. et al. Shifting attention to accuracy can reduce misinformation online. Nature 592, 590–595 (2021).
Guess, A. M. et al. A digital media literacy intervention increases discernment between mainstream and false news in the United States and India. Proc. Natl Acad. Sci. USA 117, 15536–15545 (2020).
Ceylan, G.anderson, I. A. & Wood, W. Sharing of misinformation is habitual, not just lazy or biased. Proc. Natl Acad. Sci. USA 120, e2216614120 (2023).
Pennycook, G. & Rand, D. G. Who falls for fake news? the roles of bullshit receptivity, overclaiming, familiarity and analytic thinking. J. Personal. 88, 185–200 (2020).
Haghdoost, Y. Alcohol poisoning kills 100 Iranians seeking virus protection. Bloomberg Markets (18 March 2020); www.bloomberg.com/news/articles/2020-03-18/alcohol-poisoning-kills-100-iranians-seeking-virus-protection
Gallotti, R., Valle, F., Castaldo, N., Sacco, P. & De Domenico, M. Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics. Nat. Hum. Behav. 4, 1285–1293 (2020).
Bursztyn, L., Rao, A., Roth, C. & Yanagizawa-Drott, D. Opinions as Facts (ECONtribute, 2022).
Pummerer, L. et al. Conspiracy theories and their societal effects during the COVID-19 pandemic. Soc. Psychol. Personal. Sci. 13, 49–59 (2022).
Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K. & Larson, H. J. Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nat. Hum. Behav. 5, 337–348 (2021).
Ecker, U. K. et al. The psychological drivers of misinformation belief and its resistance to correction. Nat. Rev. Psychol. 1, 13–29 (2022).
Clayton, K. et al. Real solutions for fake news? Measuring the effectiveness of general warnings and fact-check tags in reducing belief in false stories on social media. Polit. Behav. 42, 1073–1095 (2020).
Mena, P. Cleaning up social media: the effect of warning labels on likelihood of sharing false news on Facebook. Policy Internet 12, 165–183 (2020).
Kreps, S. E. & Kriner, D. Medical Misinformation in the COVID-19 Pandemic (SSRN, 2020).
Porter, E. & Wood, T. J. The global effectiveness of fact-checking: evidence from simultaneous experiments in Argentina, Nigeria, South Africa and the United Kingdom. Proc. Natl Acad. Sci. USA 118, e2104235118 (2021).
Brashier, N. M., Pennycook, G., Berinsky, A. J. & Rand, D. G. Timing matters when correcting fake news. Proc. Natl Acad. Sci. USA 118, e2020043118 (2021).
Bowles, J., Larreguy, H. & Liu, S. Countering misinformation via Whatsapp: preliminary evidence from the COVID-19 pandemic in Zimbabwe. PLoS ONE 15, e0240005 (2020).
Goldstein, J. A., Grossman, S. & Startz, M. Belief in COVID-19 misinformation in Nigeria. J. Polit. https://doi.org/10.1086/727605
Badrinathan, S. Educative interventions to combat misinformation: evidence from a field experiment in India. Am. Polit. Sci. Rev. 115, 1325–1341 (2021).
Ali, A. & Qazi, I. A. Countering misinformation on social media through educational interventions: evidence from a randomized experiment in Pakistan. J. Dev. Econ. 163, 103108 (2023).
Roozenbeek, J., Suiter, J. & Culloty, E. Countering misinformation: evidence, knowledge gaps and implications of current interventions. Eur. Psychol. https://doi.org/10.1027/1016-9040/a000492 (2023).
Rosenzweig, L. R., Bergquist, P., Hoffmann Pham, K., Rampazzo, F. & Mildenberger, M. Survey sampling in the global south using Facebook advertisements. Preprint at SocArXiv https://doi.org/10.31235/osf.io/dka8f (2020).
Broockman, D. E., Kalla, J. L. & Sekhon, J. S. The design of field experiments with survey outcomes: a framework for selecting more efficient, robust and ethical designs. Polit. Anal. 25, 435–464 (2017).
Dimakopoulou, M., Athey, S. & Imbens, G. Estimation considerations in contextual bandits. Preprint at https://doi.org/10.48550/arXiv.1711.07077 (2017).
Dimakopoulou, M., Zhou, Z., Athey, S. & Imbens, G. Balanced linear contextual bandits. Proc. AAAI Conf. Artif. Intell. 33, 3445–3453 (2019).
Zhan, R., Ren, Z., Athey, S. & Zhou, Z. Policy learning with adaptively collected data. Manage. Sci. https://doi.org/10.1287/mnsc.2023.4921 (2023).
Zhan, R., Hadad, V., Hirshberg, D. A. & Athey, S. Off-policy evaluation via adaptive weighting with data from contextual bandits. in Proc. 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21) 2125–2135 (2021).
Athey, S., Tibshirani, J. & Wager, S. Generalized random forests. Ann. Stat. 47, 1148–1178 (2019).
Robins, J. M. & Rotnitzky, A. Semiparametric efficiency in multivariate regression models with missing data. J. Am. Stat. Assoc. 90, 122–129 (1995).
Meixler, E. Facebook is dropping its fake news red flag warning after finding it had the opposite effect. TIME (22 December 2017); time.com/5077002/facebook-fake-news-articles/
Bode, L. & Vraga, E. K. In related news, that was wrong: the correction of misinformation through related stories functionality in social media. J. Commun. 65, 619–638 (2015).
Nyhan, B. & Reifler, J. When corrections fail: the persistence of political misperceptions. Polit. Behav. 32, 303–330 (2010).
Epstein, Z., Sirlin, N., Arechar, A., Pennycook, G. & Rand, D. The social media context interferes with truth discernment. Sci. Adv. 9, eabo6169 (2023).
Arechar, A. A. et al. Understanding and combatting misinformation across 16 countries on six continents. Nat. Hum. Behav. 7, 1502–1513 (2023).
Pennycook, G. & Rand, D. G. Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation. Nat. Commun. 13, 2333 (2022).
Altay, S., Hacquin, A.-S. & Mercier, H. Why do so few people share fake news? It hurts their reputation. N. Media Soc. 24, 1303–1324 (2022).
Mosleh, M., Pennycook, G. & Rand, D. G. Self-reported willingness to share political news articles in online surveys correlates with actual sharing on Twitter. PLoS ONE 15, e0228882 (2020).
Top 10 African countries with the most Facebook users. ITNews Africa (2016); www.howwe.ug/news/lifestyle/14791/top-10-countries-with-the-most-facebook-users-in-africa
Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G. & Rand, D. G. Fighting COVID-19 misinformation on social media: experimental evidence for a scalable accuracy-nudge intervention. Psychol. Sci. 31, 770–780 (2020).
Ghosh, S. Facebook will show people anti-fake news articles when they post false stories. Insider.com (3 August 2017); www.insider.com/facebook-related-articles-feature-will-show-you-anti-fake-news-2017-8
Robins, J. M., Rotnitzky, A. & Zhao, L. P. Estimation of regression coefficients when some regressors are not always observed. J. Am. Stat. Assoc. 89, 846–866 (1994).
Tibshirani, J., Athey, S. & Wager, S. grf: Generalized Random Forests. R package version 2.3.0 (2020).
Gilens, M. Political ignorance and collective policy preferences. Am. Polit. Sci. Rev. 95, 379–396 (2001).
Martel, C., Pennycook, G. & Rand, D. G. Reliance on emotion promotes belief in fake news. Cogn. Res. Princ. Implic. 5, 47 (2020).
Rosenzweig, L. R., Bago, B., Berinsky, A. J. & Rand, D. G. Happiness and surprise are associated with worse truth discernment of COVID-19 headlines among social media users in Nigeria. Harvard Kennedy School Misinformation Review (10 August 2021); misinforeview.hks.harvard.edu/article/happiness-and-surprise-are-associated-with-worse-truth-discernment-of-covid-19-headlines-among-social-media-users-in-nigeria/
Bago, B., Rosenzweig, L. R., Berinsky, A. J. & Rand, D. G. Emotion may predict susceptibility to fake news but emotion regulation does not seem to help. Cogn. Emot. 36, 1166–1180 (2022).
Bago, B., Rand, D. G. & Pennycook, G. Fake news, fast and slow: deliberation reduces belief in false (but not true) news headlines. J. Exp. Psychol. 149, 1608–1613 (2020).
Costa, M., Schaffner, B. F. & Prevost, A. Walking the walk? Experiments on the effect of pledging to vote on youth turnout. PLoS ONE 13, e0197066 (2018).
Cotterill, S., John, P. & Richardson, L. The impact of a pledge request and the promise of publicity: a randomized controlled trial of charitable donations. Soc. Sci. Q. 94, 200–216 (2013).
Gross, J. J. The emerging field of emotion regulation: an integrative review. Rev. Gen. Psychol. 2, 271–299 (1998).
Yadlowsky, S., Fleming, S., Shah, N., Brunskill, E. & Wager, S. Evaluating treatment prioritization rules via rank-weighted average treatment effects. Preprint at arxiv.org/abs/2111.07966 (2021).
Acknowledgements
We received advertising credits for this study from Facebook Health and funding from the Golub Capital Social Impact Lab and Office of Naval Research grant no. N00014-19-1-246 (S.A.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. For exceptional research assistance, we thank Z. (J.) Li, R. Ruiz, U. Byambadalai and H. (T.) Zong. We thank J. Davis, S. Grossman, L. Jakli, E. Jee, T. Kumar, E. Palikot and A. Siegel for feedback and comments, as well as the participants of the seminar series of the Development Innovation Lab at the Becker Friedman Institute. We thank J. Kiselik for editorial assistance.
Author information
Authors and Affiliations
Contributions
M.O.W., L.R. and S.A. designed the research and wrote the paper. M.O.W. and L.R. performed the experimental studies. M.O.W. analysed the data, with input from S.A. and L.R.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Human Behaviour thanks Gordon Pennycook, Xiao Zhang and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Figs. 1–5 and Tables 1–13. Supplementary methods materials are presented in Section S1. Supplementary results are presented in Section S2.
Source data
Source Data Fig. 1
Discernment measure mean response estimates, standard errors and probabilities for learning stage.
Source Data Fig. 2
Discernment and other sharing measure mean response estimates, standard errors, for evaluation stage.
Source Data Fig. 3
Covariate means, standard errors by assignment group in evaluation stage.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Offer-Westort, M., Rosenzweig, L.R. & Athey, S. Battling the coronavirus ‘infodemic’ among social media users in Kenya and Nigeria. Nat Hum Behav (2024). https://doi.org/10.1038/s41562-023-01810-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41562-023-01810-7