Automated Identification of Immunocompromised Status in Critically Ill Children

Abstract

Background Easy identification of immunocompromised hosts (ICHs) would allow for stratification of culture results based on host type.

Methods We utilized antimicrobial stewardship program (ASP) team notes written during handshake stewardship rounds in the pediatric intensive care unit (PICU) as the gold standard for host status; clinical notes from the primary team, medication orders during the encounter, problem list, and billing diagnoses documented prior to the ASP documentation were extracted to develop models that predict host status. We calculated performance for three models based on diagnoses/medications, with and without natural language processing from clinical notes. The susceptibility of pathogens causing bacteremia to commonly used empiric antibiotic regimens was then stratified by host status.

Results We identified 844 antimicrobial episodes from 666 unique patients; 160 (18.9%) were identified as ICHs. We randomly selected 675 initiations (80%) for model training and 169 initiations (20%) for testing. A rule-based model using diagnoses and medications alone yielded a sensitivity of 0.87 (08.6–0.88), specificity of 0.93 (0.92–0.93), and positive predictive value (PPV) of 0.74 (0.73–0.75). Adding clinical notes into XGBoost model led to improved specificity of 0.98 (0.98–0.98) and PPV of 0.9 (0.88–0.91), but with decreased sensitivity 0.77 (0.76–0.79). There were 77 bacteremia episodes during the study period identified and a host-specific visualization was created.

Conclusions An electronic health record–based phenotype based on notes, diagnoses, and medications identifies ICH in the PICU with high specificity.

Keywords

immunosuppression - empiric treatment - EHR phenotype - NLP

Note

This study was approved by the Emory University Institutional Review Board.

Publication History

Received: 07 December 2021

Accepted: 02 April 2022

Accepted Manuscript online:
05 April 2022

Article published online:
19 August 2022

• References

• 1 Hebert C, Hade E, Rahman P. et al. 1515. Patient-based stratification of weighted-incidence syndromic antibiogram (WISCA) for empiric antibiotic prescribing. Open Forum Infect Dis 2018; 5 (Suppl. 01) S469
• 2 DeNegre AA, Ndeffo Mbah ML, Myers K, Fefferman NH. Emergence of antibiotic resistance in immunocompromised host populations: a case study of emerging antibiotic resistant tuberculosis in AIDS patients. PLoS One 2019; 14 (02) e0212969
• 3 Giannella M, Bartoletti M, Conti M, Righi E. Carbapenemase-producing Enterobacteriaceae in transplant patients. J Antimicrob Chemother 2021; 76 (Suppl. 01) i27-i39
• 4 Simpao AF, Ahumada LM, Larru Martinez B. et al. Design and implementation of a visual analytics electronic antibiogram within an electronic health record system at a tertiary pediatric hospital. Appl Clin Inform 2018; 9 (01) 37-45
• 5 Alzoubi H, Alzubi R, Ramzan N, West D, Al-Hadhrami T, Alazab M. A review of automatic phenotyping approaches using electronic health records. Electronics 2019; 8: 1235
• 6 Byrd RJ, Steinhubl SR, Sun J, Ebadollahi S, Stewart WF. Automatic identification of heart failure diagnostic criteria, using text analysis of clinical notes from electronic health records. Int J Med Inform 2014; 83 (12) 983-992
• 7 Afzal Z, Engelkes M, Verhamme KMC. et al. Automatic generation of case-detection algorithms to identify children with asthma from large electronic health record databases. Pharmacoepidemiol Drug Saf 2013; 22 (08) 826-833
• 8 Xia Z, Secor E, Chibnik LB. et al. Modeling disease severity in multiple sclerosis using electronic health records. PLoS One 2013; 8 (11) e78927
• 9 Cohen KB, Glass B, Greiner HM. et al. Methodological issues in predicting pediatric epilepsy surgery candidates through natural language processing and machine learning. Biomed Inform Insights 2016; 8: 11-18
• 10 Lin C, Karlson EW, Canhao H. et al. Automatic prediction of rheumatoid arthritis disease activity from the electronic medical records. PLoS One 2013; 8 (08) e69932
• 11 Liao KP, Cai T, Savova GK. et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ 2015; 350: h1885
• 12 Hurst AL, Child J, Pearce K, Palmer C, Todd JK, Parker SK. Handshake stewardship: a highly effective rounding-based antimicrobial optimization service. Pediatr Infect Dis J 2016; 35 (10) 1104-1110
• 13 Greenberg JA, Hohmann SF, Hall JB, Kress JP, David MZ. Validation of a method to identify immunocompromised patients with severe sepsis in administrative databases. Ann Am Thorac Soc 2016; 13 (02) 253-258
• 14 TF-IDF. In: Sammut C, Webb GI, eds. Encyclopedia of Machine Learning. Boston, MA: Springer. Doi: https://doi.org/10.1007/978-0-387-30164-8_832
• 15 Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. Accessed March 14, 2022 at http://ronan.collobert.com/senna/
• 16 Pedregosa F, Varoquaux G, Gramfort A. et al. Scikit-learn: machine learning in Python. J Mach Learn Res 2011; 12: 2825-2830
• 17 Řehůřek R, Sojka P. Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. Valletta, Malta: ELRA; 2010: 45-50
• 18 Joachims T. Machine learning. . Paper presented at: 10th European Conference on Machine Learning; Chemnitz, Germany; April 21–23, 1998
• 19 Chen D, Müller HM, Sternberg PW. Automatic document classification of biological literature. BMC Bioinformatics 2006; 7: 370
• 20 Savova GK, Ogren PV, Duffy PH, Buntrock JD, Chute CG. Mayo clinic NLP system for patient smoking status identification. J Am Med Inform Assoc 2008; 15 (01) 25-28
• 21 Chen T, Guestrin C. XGBoost: A scalable tree boosting system. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [Internet]. New York, NY: Association for Computing Machinery; 2016: 785-794
• 22 Lundberg S, Lee S-I.. In: Guyon I, Von Luxburg U, Bengio S Wallach H, Fergus R, Vishwanathan S, and Garnett R, eds. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 30 (NIPS 2017); 2017. Curran Associates, Inc.
• 23 Truong WR, Hidayat L, Bolaris MA, Nguyen L, Yamaki J. The antibiogram: key considerations for its development and utilization. JAC Antimicrob Resist 2021; 3 (02) dlab060
• 24 Hurst AL, Child J, Parker SK. Intervention and acceptance rates support handshake-stewardship strategy. J Pediatric Infect Dis Soc 2019; 8 (02) 162-165
• 25 Stang CRT, Jaggi P, Tansmore J. et al. Implementation of a pharmacist-led antimicrobial time-out for medical-surgery services in an academic pediatric hospital. J Pediatr Pharmacol Ther 2021; 26 (03) 284-290