Simulating adverse event spontaneous reporting systems as preferential attachment networks

Summary

Background: Spontaneous Reporting Systems [SRS] are critical tools in the post-licensure evaluation of medical product safety. Regulatory authorities use a variety of data mining techniques to detect potential safety signals in SRS databases. Assessing the performance of such signal detection procedures requires simulated SRS databases, but simulation strategies proposed to date each have limitations.

Objective: We sought to develop a novel SRS simulation strategy based on plausible mechanisms for the growth of databases over time.

Methods: We developed a simulation strategy based on the network principle of preferential attachment. We demonstrated how this strategy can be used to create simulations based on specific databases of interest, and provided an example of using such simulations to compare signal detection thresholds for a popular data mining algorithm.

Results: The preferential attachment simulations were generally structurally similar to our targeted SRS database, although they had fewer nodes of very high degree. The approach was able to generate signal-free SRS simulations, as well as mimicking specific known true signals. Explorations of different reporting thresholds for the FDA Vaccine Adverse Event Reporting System suggested that using proportional reporting ratio [PRR] > 3.0 may yield better signal detection operating characteristics than the more commonly used PRR > 2.0 threshold.

Discussion: The network analytic approach to SRS simulation based on the principle of preferential attachment provides an attractive framework for exploring the performance of safety signal detection algorithms. This approach is potentially more principled and versatile than existing simulation approaches.

Conclusion: The utility of network-based SRS simulations needs to be further explored by evaluating other types of simulated signals with a broader range of data mining approaches, and comparing network-based simulations with other simulation strategies where applicable.

Citation: Scott J, Botsis T, Ball R. Simulating adverse event spontaneous reporting systems as preferential attachment networks: Application to the Vaccine Adverse Event Reporting System. Appl Clin Inf 2014; 5: 206–218 http://dx.doi.org/10.4338/ACI-2013-11-RA-0097

Keywords

Simulation - data mining - vaccines - safety - network analysis

• References

• 1 Varricchio F. et al. Understanding vaccine safety information from the Vaccine Adverse Event Reporting System. Pediatr Infect Dis J 2004; 23 (04) 287-294.
• 2 Niu MT, Erwin DE, Braun MM. Data mining in the US Vaccine Adverse Event Reporting System (VAERS): early detection of intussusception and other events after rotavirus vaccination. Vaccine 2001; 19 (32) 4627-4634.
• 3 Roux E. et al. Evaluation of statistical association measures for the automatic signal generation in pharmacovigilance. IEEE Trans Inf Technol Biomed 2005; 9 (04) 518-527.
• 4 Rolka H. et al. Using simulation to assess the sensitivity and specificity of a signal detection tool for multidimensional public health surveillance data. Stat Med 2005; 24 (04) 551-562.
• 5 DuMouchel W, Pregibon D. Empirical Bayes screening for multi-item associations. In: Proceedings of the KDD, ACM; New York: 2001: 67-76.
• 6 Ahmed I. et al. Bayesian pharmacovigilance signal detection methods revisited in a multiple comparison setting. Stat Med 2009; 28 (13) 1774-1792.
• 7 Ahmed I. et al. Pharmacovigilance data mining with methods based on false discovery rates: a comparative simulation study. Clin Pharmacol Ther 2010; 88 (04) 492-498.
• 8 Tubert P. et al. Power and weakness of spontaneous reporting: a probabilistic approach. J Clin Epidemiol 1992; 45 (03) 283-286.
• 9 Ball R, Botsis T. Can network analysis improve pattern recognition among adverse events following immunization reported to VAERS?. Clin Pharmacol Ther 2011; 90 (02) 271-278.
• 10 Botsis T, Ball R. Network analysis of possible anaphylaxis cases reported to the US vaccine adverse event reporting system after H1N1 influenza vaccine. Stud Health Technol Inform 2011; 169: 564-568.
• 11 Barabási A-L, Albert R. Emergence of scaling in random networks. Science 1999; 286 5439 509-512.
• 12 Bianconi G, Barabási A-L.. Bose-Einstein condensation in complex networks. Phys Rev Lett 2001; 86 (24) 5632-5635.
• 13 Haber P. et al. An analysis of rotavirus vaccine reports to the vaccine adverse event reporting system: more than intussusception alone?. Pediatrics 2004; 113 (04) e353-e359.
• 14 Evans SJW, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001; 10 (06) 483-486.
• 15 R Core Team.. R: A language and environment for statistical computing. (R Foundation for Statistical Computing, Vienna, Austria, 2012).
• 16 Jeong H. et al. The large-scale organization of metabolic networks. Nature 2000; 407 6804 651-654.