Methods Inf Med 2022; 61(03/04): 084-089
DOI: 10.1055/s-0042-1749358
Original Article

Automated Identification of Clinical Procedures in Free-Text Electronic Clinical Records with a Low-Code Named Entity Recognition Workflow

Carmelo Macri
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
3   Discipline of Ophthalmology and Visual Sciences, The University of Adelaide, Adelaide, South Australia, Australia
,
Ian Teoh
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
,
Stephen Bacchi
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
Michelle Sun
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
Dinesh Selva
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
Robert Casson
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
WengOnn Chan
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
› Institutsangaben
Funding None.

Abstract

Background Clinical procedures are often performed in outpatient clinics without prior scheduling at the administrative level, and documentation of the procedure often occurs solely in free-text clinical electronic notes. Natural language processing (NLP), particularly named entity recognition (NER), may provide a solution to extracting procedure data from free-text electronic notes.

Methods Free-text notes from outpatient ophthalmology visits were collected from the electronic clinical records at a single institution over 3 months. The Prodigy low-code annotation tool was used to create an annotation dataset and train a custom NER model for clinical procedures. Clinical procedures were extracted from the entire set of clinical notes.

Results There were a total of 5,098 clinic notes extracted for the study period; 1,923 clinic notes were used to build the NER model, which included a total of 231 manual annotations. The NER model achieved an F-score of 0.767, a precision of 0.810, and a recall of 0.729. The most common procedures performed included intravitreal injections of therapeutic substances, removal of corneal foreign bodies, and epithelial debridement of corneal ulcers.

Conclusion The use of a low-code annotation software tool allows the rapid creation of a custom annotation dataset to train a NER model to identify clinical procedures stored in free-text electronic clinical notes. This enables clinicians to rapidly gather previously unidentified procedural data for quality improvement and auditing purposes. Low-code annotation tools may reduce time and coding barriers to clinician participation in NLP research.



Publikationsverlauf

Eingereicht: 02. Dezember 2021

Angenommen: 13. April 2022

Artikel online veröffentlicht:
12. September 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 
  • References

  • 1 Aronson AR, Lang FM. An overview of MetaMap: historical perspective and recent advances. J Am Med Inform Assoc 2010; 17 (03) 229-236
  • 2 Gorrell G, Song X, Roberts A. Bio-YODIE: A Named Entity Linking System for Biomedical Text. ArXiv 2018;abs/1811.04860
  • 3 Richardson C, Rymer J. New development platforms emerge for customer-facing applications. 2014 For Application and Development Professionals. Accessed February 19, 2022 at: https://www.forrester.com/report/New-Development-Platforms-Emerge-For-CustomerFacing-Applications/RES113411
  • 4 Neves M, Ševa J. An extensive review of tools for manual annotation of documents. Brief Bioinform 2021; 22 (01) 146-163
  • 5 Ting DSW, Pasquale LR, Peng L. et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol 2019; 103 (02) 167-175
  • 6 Lin WC, Chen JS, Chiang MF, Hribar MR. Applications of artificial intelligence to electronic health record data in ophthalmology. Transl Vis Sci Technol 2020; 9 (02) 13
  • 7 Gaskin GL, Pershing S, Cole TS, Shah NH. Predictive modeling of risk factors and complications of cataract surgery. Eur J Ophthalmol 2016; 26 (04) 328-337
  • 8 Wang SY, Pershing S, Tran E, Hernandez-Boussard T. Automated extraction of ophthalmic surgery outcomes from the electronic health record. Int J Med Inform 2020; 133: 104007
  • 9 Baughman DM, Su GL, Tsui I, Lee CS, Lee AY. Validation of the Total Visual Acuity Extraction Algorithm (TOVA) for automated extraction of visual acuity data from free text, unstructured clinical records. Transl Vis Sci Technol 2017; 6 (02) 2
  • 10 Liu L, Shorstein NH, Amsden LB, Herrinton LJ. Natural language processing to ascertain two key variables from operative reports in ophthalmology. Pharmacoepidemiol Drug Saf 2017; 26 (04) 378-385
  • 11 Vaci N, Liu Q, Kormilitzin A. et al. Natural language processing for structuring clinical text data on depression using UK-CRIS. Evid Based Ment Health 2020; 23 (01) 21-26
  • 12 Senior M, Burghart M, Yu R. et al. Identifying predictors of suicide in severe mental illness: a feasibility study of a clinical prediction rule (Oxford Mental Illness and Suicide Tool or OxMIS). Front Psychiatry 2020; 11: 268