Download PDF
Appl Clin Inform 2025; 16(02): 357-361
DOI: 10.1055/a-2499-4207
Research Article

Validation of an Algorithm to Classify Urine Cultures in Family Medicine

Jack Zhang
1   Section of Anesthesia, Northern Ontario School of Medicine University, Sudbury, Canada
,
Rachael Morkem
2   Department of Family Medicine, Queen's University, Kingston, Canada
,
Akshay Rajaram
2   Department of Family Medicine, Queen's University, Kingston, Canada
3   Department of Emergency Medicine, Queen's University, Kingston, Canada
› Author Affiliations Funding None.
› Further Information
Also available at
    Permissions and Reprints

Abstract

Objectives Automation of test follow-up offers potential reductions in workload for clinicians. The primary objective of the study was to evaluate the performance of MicrobEx, a regular expression-based algorithm in classifying urine culture reports in primary care.

Methods A retrospective validation of MicrobEx was performed using urine culture reports abstracted from a single academic family health team. MicrobEx classifications were compared with labels assigned manually by a human reviewer. Measures of diagnostic performance were calculated.

Results MicrobEx achieved 95.3% accuracy, 88.6% sensitivity, and 100% specificity in classifying 1,999 urine culture reports.

Conclusion The accuracy of MicrobEx was comparable to its performance in the original development and validation study by Eickelberg. Additional work is required to explore and improve the accuracy of MicrobEx and assess its performance across primary care settings and with more complex urine culture reports.

Keywords

urinary tract infections - natural language processing - electronic medical records - family medicine

Protection of Human and Animal Subjects

This study received research ethics board approval.


Supplementary Material



Publication History

Received: 15 July 2024

Accepted: 09 December 2024

Article published online:
23 April 2025

© 2025. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Mikhaeil JS, Jalali H, Orchanian-Cheff A, Chartier LB. Quality assurance processes ensuring appropriate follow-up of test results pending at discharge in emergency departments: a systematic review. Ann Emerg Med 2020; 76 (05) 659-674
    PubMedSearch in Google Scholar
  • 2 Rajaram A, Clapham G, Nemnom M, Thiruganasambandamoorthy V. CAEP/ACMU 2022 Scientific Abstracts. Can J Emerg Med 2022; 24 (01) 1-100
    PubMedSearch in Google Scholar
  • 3 Luz CF, Vollmer M, Decruyenaere J, Nijsten MW, Glasner C, Sinha B. Machine learning in infection management using routine electronic health records: tools, techniques, and reporting of future technologies. Clin Microbiol Infect 2020; 26 (10) 1291-1299
    PubMedSearch in Google Scholar
  • 4 Rockenschaub P, Gill MJ, McNulty D, Carroll O, Freemantle N, Shallcross L. Can the application of machine learning to electronic health records guide antibiotic prescribing decisions for suspected urinary tract infection in the Emergency Department?. PLoS Digit Health 2023; 2 (06) e0000261
    PubMedSearch in Google Scholar
  • 5 Menon S, Murphy DR, Singh H, Meyer AN, Sittig DF. Workarounds and test results follow-up in electronic health record-based primary care. Appl Clin Inform 2016; 7 (02) 543-559
    Thieme ConnectPubMedSearch in Google Scholar
  • 6 de Vries S, Ten Doesschate T, Totté JEE. et al. A semi-supervised decision support system to facilitate antibiotic stewardship for urinary tract infections. Comput Biol Med 2022; 146: 105621
    PubMedSearch in Google Scholar
  • 7 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
    CrossrefPubMedSearch in Google Scholar
  • 8 Flood CM, Thomas B, McGibbon E. Canada's primary care crisis: Federal government response. Healthc Manage Forum 2023; 36 (05) 327-332
    PubMedSearch in Google Scholar
  • 9 Soong C, Shojania KG. Education as a low-value improvement intervention: often necessary but rarely sufficient. BMJ Qual Saf 2020; 29: 353-357
    PubMedSearch in Google Scholar
  • 10 Berlyand Y, Raja AS, Dorner SC. et al. How artificial intelligence could transform emergency department operations. Am J Emerg Med 2018; 36 (08) 1515-1517
    CrossrefPubMedSearch in Google Scholar
  • 11 Matheny ME, Fitzhenry F, Speroff T. et al. Detection of blood culture bacterial contamination using natural language processing. AMIA Annu Symp Proc 2009; 2009: 411-415
    PubMedSearch in Google Scholar
  • 12 Eickelberg G, Luo Y, Sanchez-Pinto LN. Development and validation of MicrobEx: an open-source package for microbiology culture concept extraction. JAMIA Open 2022; 5 (02) ooac026
    PubMedSearch in Google Scholar
  • 13 WELL Health Technologies Corp. . OSCAR Pro. 2024 . Accessed December 16, 2024 at: https://well.company/oscar-pro
    PubMed
  • 14 Jones M, DuVall SL, Spuhl J, Samore MH, Nielson C, Rubin M. Identification of methicillin-resistant Staphylococcus aureus within the nation's Veterans Affairs medical centers using natural language processing. BMC Med Inform Decis Mak 2012; 12 (01) 34
    PubMedSearch in Google Scholar
  • 15 Naik N, Talyshinskii A, Shetty DK, Hameed BMZ, Zhankina R, Somani BK. Smart diagnosis of urinary tract infections: is artificial intelligence the fast-lane solution?. Curr Urol Rep 2024; 25 (01) 37-47
    PubMedSearch in Google Scholar