PDF herunterladen
Appl Clin Inform 2023; 14(01): 01-10
DOI: 10.1055/a-1975-4061
Research Article

Australasian Institute of Digital Health Summit 2022–Automated Social Media Surveillance for Detection of Vaccine Safety Signals: A Validation Study

Sedigheh Khademi Habibabadi
1   Department of Paediatrics, Centre for Health Analytics, Murdoch Children's Research Institute Melbourne, Australia
2   Department of Paediatrics, Centre for Health Analytics, Health Informatics Group, Murdoch Children's Research Institute, Melbourne, Australia
3   Department of General Practice, University of Melbourne, Melbourne, Australia
,
Christopher Palmer
2   Department of Paediatrics, Centre for Health Analytics, Health Informatics Group, Murdoch Children's Research Institute, Melbourne, Australia
,
Gerardo L. Dimaguila
1   Department of Paediatrics, Centre for Health Analytics, Murdoch Children's Research Institute Melbourne, Australia
,
Muhammad Javed
2   Department of Paediatrics, Centre for Health Analytics, Health Informatics Group, Murdoch Children's Research Institute, Melbourne, Australia
,
Hazel J. Clothier
1   Department of Paediatrics, Centre for Health Analytics, Murdoch Children's Research Institute Melbourne, Australia
4   Department of Paediatrics, Infectious Diseases Group, SAEFVIC, Murdoch Children's Research Institute, Melbourne, Australia
5   Faculty of Medicine, Dentistry, and Health Sciences, Melbourne School of Population and Global Health, University of Melbourne, Melbourne, Australia
,
Jim Buttery
1   Department of Paediatrics, Centre for Health Analytics, Murdoch Children's Research Institute Melbourne, Australia
4   Department of Paediatrics, Infectious Diseases Group, SAEFVIC, Murdoch Children's Research Institute, Melbourne, Australia
6   Department of Paediatrics, University of Melbourne, Melbourne, Australia
› Institutsangaben Funding None.
› Weitere Informationen
    Lizenzen und Reprints

Abstract

Background Social media platforms have emerged as a valuable data source for public health research and surveillance. Monitoring of social media and user-generated data on the Web enables timely and inexpensive collection of information, overcoming time lag and cost of traditional health reporting systems.

Objectives This article identifies personally experienced coronavirus disease 2019 (COVID-19) vaccine reactions expressed on Twitter and validate the findings against an established vaccine reactions reporting system.

Methods We collected around 3 million tweets from 1.4 million users between February 1, 2021, to January 31, 2022, using COVID-19 vaccines and vaccine reactions keyword lists. We performed topic modeling on a sample of the data and applied a modified F1 scoring technique to identify a topic that best differentiated vaccine-related personal health mentions. We then manually annotated 4,000 of the records from this topic, which were used to train a transformer-based classifier to identify likely personally experienced vaccine reactions. Applying the trained classifier to the entire data set allowed us to select records we could use to quantify potential vaccine side effects. Adverse events following immunization (AEFI) referred to in these records were compared with those reported to the state of Victoria's spontaneous vaccine safety surveillance system, SAEFVIC (Surveillance of Adverse Events Following Vaccination In the Community).

Results The most frequently mentioned potential vaccine reactions generally aligned with SAEFVIC data. Notable exceptions were increased Twitter reporting of bleeding-related AEFI and allergic reactions, and more frequent SAEFVIC reporting of cardiac AEFI.

Conclusion Social media conversations are a potentially valuable supplementary data source for detecting vaccine adverse event mentions. Monitoring of online observations about new vaccine-related personal health experiences has the capacity to provide early warnings about emerging vaccine safety issues.

Keywords

COVID-19 vaccines - social media - Internet - artificial intelligence - patient-generated health data - pharmacovigilance - immunization - vaccination - adverse events

Protection of Human and Animal Subjects

Ethics approval for this study was granted by The Royal Children's Hospital Melbourne Human Research Ethics Committee (HREC) (project ID 85026).


Supplementary Material



Publikationsverlauf

Eingereicht: 11. Juli 2022

Angenommen: 25. Oktober 2022

Accepted Manuscript online:
09. November 2022

Artikel online veröffentlicht:
04. Januar 2023

© 2023. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Black S. Recognizing the importance of vaccine confidence. EBioMedicine 2016; 12: 28-29
    CrossrefPubMedGoogle Scholar
  • 2 Jacobson RM, Adegbenro A, Pankratz VS, Poland GA. Adverse events and vaccination-the lack of power and predictability of infrequent events in pre-licensure study. Vaccine 2001; 19 (17-19): 2428-2433
    CrossrefPubMedGoogle Scholar
  • 3 Hervé C, Laupèze B, Del Giudice G, Didierlaurent AM, Tavares Da Silva F. The how's and what's of vaccine reactogenicity. NPJ Vaccines 2019; 4 (01) 39
    CrossrefPubMedGoogle Scholar
  • 4 Oliveira PMN, Lignani LK, Conceição DAD. et al. Surveillance of adverse events following immunization in the late 2010s: an overview of the importance, tools, and challenges [in Portuguese]. Cad Saude Publica 2020; 36 (Suppl. 02) e00182019
    PubMedGoogle Scholar
  • 5 Pal SN, Duncombe C, Falzon D, Olsson S. WHO strategy for collecting safety data in public health programmes: complementing spontaneous reporting systems. Drug Saf 2013; 36 (02) 75-81
    CrossrefPubMedGoogle Scholar
  • 6 Paul MJ, Dredze M. Social monitoring for public health. Synth Lect Inf Concepts Retr Serv 2017; 9 (05) 1-183
    PubMedGoogle Scholar
  • 7 Wilson AE, Lehmann CU, Saleh SN, Hanna J, Medford RJ. Social media: a new tool for outbreak surveillance. Antimicrob Steward Healthc Epidemiol 2021; 1 (01) e50
    CrossrefPubMedGoogle Scholar
  • 8 Qin L, Sun Q, Wang Y. et al. Prediction of number of cases of 2019 novel coronavirus (COVID-19) using social media search index. Int J Environ Res Public Health 2020; 17 (07) E2365
    CrossrefPubMedGoogle Scholar
  • 9 Medford RJ, Saleh SN, Sumarsono A, Perl TM, Lehmann CU. An “Infodemic”: leveraging high-volume twitter data to understand early public sentiment for the coronavirus disease 2019 outbreak. Open Forum Infect Dis 2020; 7 (07) ofaa258
    CrossrefPubMedGoogle Scholar
  • 10 Tsao SF, Chen H, Tisseverasinghe T, Yang Y, Li L, Butt ZA. What social media told us in the time of COVID-19: a scoping review. Lancet Digit Health 2021; 3 (03) e175-e194
    CrossrefPubMedGoogle Scholar
  • 11 Lanier HD, Diaz MI, Saleh SN, Lehmann CU, Medford RJ. Analyzing COVID-19 disinformation on Twitter using the hashtags #scamdemic and #plandemic: retrospective study. PLoS One 2022; 17 (06) e0268409
    CrossrefPubMedGoogle Scholar
  • 12 Cinelli M, Quattrociocchi W, Galeazzi A. et al. The COVID-19 social media infodemic. Sci Rep 2020; 10 (01) 16598
    CrossrefPubMedGoogle Scholar
  • 13 Sarker A, Ginn R, Nikfarjam A. et al. Utilizing social media data for pharmacovigilance: a review. J Biomed Inform 2015; 54: 202-212
    CrossrefPubMedGoogle Scholar
  • 14 Lardon J, Abdellaoui R, Bellet F. et al. Adverse drug reaction identification and extraction in social media: a scoping review. J Med Internet Res 2015; 17 (07) e171
    CrossrefPubMedGoogle Scholar
  • 15 Karafillakis E, Martin S, Simas C. et al. Methods for social media monitoring related to vaccination: systematic scoping review. JMIR Public Health Surveill 2021; 7 (02) e17149
    CrossrefPubMedGoogle Scholar
  • 16 Yin Z, Fabbri D, Rosenbloom ST, Malin B. A scalable framework to detect personal health mentions on Twitter. J Med Internet Res 2015; 17 (06) e138
    CrossrefPubMedGoogle Scholar
  • 17 Yu H, Ho C, Juan Y, Lin C. Libshorttext: a library for short-text classification and analysis. Rapport interne, Department of Computer Science, National Taiwan University Software. Published online 2013. Accessed November 25, 2022, at: http://www csie ntu edu tw/cjlin/libshorttext
    PubMedGoogle Scholar
  • 18 Wang J, Zhao L. Multi-instance domain adaptation for vaccine adverse event detection. 2018: 97-106
    PubMedGoogle Scholar
  • 19 Lian AT, Du J, Tang L. Using a machine learning approach to monitor COVID-19 vaccine adverse events (VAE) from Twitter data. Vaccines (Basel) 2022; 10 (01) 103
    CrossrefPubMedGoogle Scholar
  • 20 Twitter. Twitter API Academic Research. Published 2022. Accessed September 2, 2022, at: https://developer.twitter.com/en/products/twitter-api/academic-research
    PubMed
  • 21 Pigott F, Kolb J, Montague J, Gonzales A, Moffitt J. Searchtweets, Wrapper for Twitter API v2 search endpoints. Published 2021. Accessed September 2, 2022, at: https://pypi.org/project/searchtweets-v2/
    PubMedGoogle Scholar
  • 22 Clothier HJ, Crawford NW, Kempe A, Buttery JP. Surveillance of adverse events following immunisation: the model of SAEFVIC, Victoria. Commun Dis Intell Q Rep 2011; 35 (04) 294-298
    PubMedGoogle Scholar
  • 23 SAEFVIC. Surveillance of adverse events following vaccination in the community. Published 2022. Accessed November 25, 2022, at: https://www.mcri.edu.au/research/research-areas/infection-and-immunity/saefvic
    PubMedGoogle Scholar
  • 24 McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012; 22 (03) 276-282
    CrossrefPubMedGoogle Scholar
  • 25 Khademi Habibabadi S, Haghighi PD, Habibabadi SK, Haghighi PD, Khademi S, Haghighi PD. Topic modelling for identification of vaccine reactions in Twitter. ACM International Conference Proceeding Series. Published online 2019:31. Accessed November 25, 2022 at: https://dl.acm.org/doi/abs/10.1145/3290688.3290735
    PubMedGoogle Scholar
  • 26 Řehůřek R, Sojka P. Software Framework for Topic Modelling with Large Corpora. Proceedings of LREC 2010 Workshop New Challenges. Published online 2010. Accessed November 25, 2022, at: http://www.muni.cz/research/publications/884893
    PubMedGoogle Scholar
  • 27 Nguyen DQ. jLDADMM: A Java package for the LDA and DMM topic models. 2018 (Dmm):1–5. Accessed November 25, 2022, at: http://arxiv.org/abs/1808.03835
    PubMedGoogle Scholar
  • 28 Mining of Twitter Conversations: 2-Phase Classification Study. JMIR Med Inform 2022; 10 (06) E34305
    CrossrefPubMedGoogle Scholar
  • 29 Nguyen DQ, Vu T, Nguyen AT. BERTweet: a pre-trained language model for English Tweets. arXiv preprint arXiv:200510200. Published online 2020
    PubMedGoogle Scholar
  • 30 WHO. . COVID-19 Vaccines: Safety Surveillance Manual; Published online 2021. Accessed December 3, 2022 at: https://www.who.int/publications/i/item/9789240032781
    PubMed
  • 31 Agency EM. COVID-19 Vaccine AstraZeneca: PRAC preliminary view suggests no specific issue with batch used in Austria | European Medicines Agency. Published 2021. Accessed April 29, 2022, at: https://www.ema.europa.eu/en/news/covid-19-vaccine-astrazeneca-prac-preliminary-view-suggests-no-specific-issue-batch-used-austria
    PubMedGoogle Scholar
  • 32 Crawford NW, Clothier H, Hodgson K, Selvaraj G, Easton ML, Buttery JP. Active surveillance for adverse events following immunization. Expert Rev Vaccines 2014; 13 (02) 265-276
    CrossrefPubMedGoogle Scholar
  • 33 Danchin M, Buttery J. COVID-19 vaccine hesitancy: a unique set of challenges. Intern Med J 2021; 51 (12) 1987-1989
    CrossrefPubMedGoogle Scholar
  • 34 Niemiec E. COVID-19 and misinformation: is censorship of social media a remedy to the spread of medical misinformation?. EMBO Rep 2020; 21 (11) e51420
    CrossrefPubMedGoogle Scholar