Association between Search Behaviors and Disease Prevalence Rates at 18 U.S. Children's Hospitals

Abstract

Background Usage of online resources by clinicians in training and practice can provide insight into knowledge gaps and inform development of decision support tools. Although online information seeking is often driven by encountered patient problems, the relationship between disease prevalence and search rate has not been previously characterized.

Objective This article aimed to (1) identify topics frequently searched by pediatric clinicians using UpToDate (http://www.uptodate.com) and (2) explore the association between disease prevalence rate and search rate using data from the Pediatric Health Information System.

Methods We identified the most common search queries and resources most frequently accessed on UpToDate for a cohort of 18 children's hospitals during calendar year 2012. We selected 64 of the most frequently searched diseases and matched ICD-9 data from the PHIS database during the same time period. Using linear regression, we explored the relationship between clinician query rate and disease prevalence rate.

Results The hospital cohort submitted 1,228,138 search queries across 592,454 sessions. The majority of search sessions focused on a single search topic. We identified no consistent overall association between disease prevalence and search rates. Diseases where search rate was substantially higher than prevalence rate were often infectious or immune/rheumatologic conditions, involved potentially complex diagnosis or management, and carried risk of significant morbidity or mortality. None of the examined diseases showed a decrease in search rate associated with increased disease prevalence rates.

Conclusion This is one of the first medical learning needs assessments to use large-scale, multisite data to identify topics of interest to pediatric clinicians, and to examine the relationship between disease prevalence and search rate for a set of pediatric diseases. Overall, disease search rate did not appear to be associated with hospital disease prevalence rates based on ICD-9 codes. However, some diseases were consistently searched at a higher rate than their prevalence rate; many of these diseases shared common features.

Keywords

information retrieval - search query log analysis - educational needs assessment - data mining

Authors' Contributions

D.D. conceptualized and designed the study, performed database generation, drafted the initial manuscript and revised the manuscript, and approved the final manuscript as submitted. T. W. reviewed and revised the manuscript and approved the final manuscript as submitted. T.L. performed the initial statistical analysis, reviewed and revised the manuscript, and approved the final manuscript as submitted. M.H. assisted with conceptualization of the study and database generation, reviewed and revised the manuscript, and approved the final manuscript as submitted. J.B. refined study design, reviewed and revised the manuscript, and approved the final manuscript as submitted. All authors approved the final manuscript as submitted and agreed to be accountable for all aspects of the work.

Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects. The study and its component analyses were approved for exemption by the Institutional Review Board of Boston Children's Hospital.

• References

• 1 Densen P. Challenges and opportunities facing medical education. Trans Am Clin Climatol Assoc 2011; 122: 48-58
• 2 Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?. PLoS Med 2010; 7 (09) e1000326
• 3 Smith R. Strategies for coping with information overload. BMJ 2010; 341: c7126
• 4 Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA 2008; 300 (10) 1181-1196
• 5 Younger P. Internet-based information-seeking behaviour amongst doctors and nurses: a short review of the literature. Health Info Libr J 2010; 27 (01) 2-10
• 6 McKnight M. The information seeking of on-duty critical care nurses: evidence from participant observation and in-context interviews. J Med Libr Assoc 2006; 94 (02) 145-151
• 7 Bennett NL, Casebeer LL, Zheng S, Kristofco R. Information-seeking behaviors and reflective practice. J Contin Educ Health Prof 2006; 26 (02) 120-127
• 8 Del Fiol G, Workman TE, Gorman PN. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med 2014; 174 (05) 710-718
• 9 Kern DE, Thomas PA, Hughes MT. Curriculum Development for Medical Education: A Six-Step Approach. 2nd ed. Baltimore, MD: JHU Press; 2009
• 10 Grant J. Learning needs assessment: assessing the need. BMJ 2002; 324 (7330): 156-159
• 11 Norman GR, Shannon SI, Marrin ML. The need for needs assessment in continuing medical education. BMJ 2004; 328 (7446): 999-1001
• 12 Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical education: a review. Med Educ Online 2002; 7 (01) 4542
• 13 Davies K, Harrison J. The information-seeking behaviour of doctors: a review of the evidence. Health Info Libr J 2007; 24 (02) 78-94
• 14 Santillana M, Nsoesie EO, Mekaru SR, Scales D, Brownstein JS. Using clinicians' search query data to monitor influenza epidemics. Clin Infect Dis 2014; 59 (10) 1446-1450
• 15 McAfee A, Brynjolfsson E. Big data: the management revolution. Harv Bus Rev 2012; 90 (10) 60-66 , 68, 128
• 16 Henke N, Bughin J, Chui M. , et al. The age of analytics: competing in a data-driven world. McKinsey Global Institute. Available at: http://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/the-age-of-analytics-competing-in-a-data-driven-world . December 2016. Accessed December 29, 2016
• 17 UpToDate. About Us. Available at: http://www.uptodate.com/home/about-us . Accessed December 29, 2016
• 18 Silvestri F. Mining query logs: Turning search usage data into knowledge. Found Trends Inform Retriev 2010; 4 (1–2): 12-17
• 19 Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature 2009; 457 (7232): 1012-1014
• 20 Carneiro HA, Mylonakis E. Google trends: a web-based tool for real-time surveillance of disease outbreaks. Clin Infect Dis 2009; 49 (10) 1557-1564
• 21 Yang L, Mei Q, Zheng K, Hanauer DA. Query log analysis of an electronic health record search engine. AMIA Annu Symp Proc 2011; 2011: 915-924
• 22 Blechner M, Kish J, Chadaga V, Dighe AS. Analysis of search in an online clinical laboratory manual. Am J Clin Pathol 2006; 126 (02) 208-214
• 23 Spink A, Yang Y, Jansen J. , et al. A study of medical and health queries to web search engines. Health Info Libr J 2004; 21 (01) 44-51
• 24 Callahan A, Pernek I, Stiglic G, Leskovec J, Strasberg HR, Shah NH. Analyzing information seeking and drug-safety alert response by health care professionals as new methods for surveillance. J Med Internet Res 2015; 17 (08) e204
• 25 Merriam SB. Andragogy and self-directed learning: pillars of adult learning theory. New Dir Adult Contin Educ 2001; 2001 (89) 3-14
• 26 Gottfredson C, Mosher B. Are you meeting all five moments of learning need. Learning Solutions Magazine 2012;18
• 27 Marshall JG, Sollenberger J, Easterby-Gannett S. , et al. The value of library and information services in patient care: results of a multisite study. J Med Libr Assoc 2013; 101 (01) 38-46
• 28 Bennett NL, Casebeer LL, Kristofco RE, Strasser SM. Physicians' Internet information-seeking behaviors. J Contin Educ Health Prof 2004; 24 (01) 31-38
• 29 Wachter R. The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine's Computer Age. New York, NY: McGraw-Hill Education; 2015
• 30 Steiner CM, Kickmeier-Rust MD, Albert D. LEA in private: a privacy and data protection framework for a learning analytics toolbox. J Learn Anal 2016; 3 (01) 66-90
• 31 Boyd D, Crawford K. Critical questions for big data: provocations for a cultural, technological, and scholarly phenomenon. Inf Commun Soc 2012; 15 (05) 662-679
• 32 Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 2006; 296 (09) 1094-1102