Download PDF
Appl Clin Inform 2023; 14(03): 400-407
DOI: 10.1055/a-2051-9764
Adolescent Privacy and the Electronic Health Record

A Natural Language Processing Model to Identify Confidential Content in Adolescent Clinical Notes

Naveed Rabbani
1   Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
,
Michael Bedgood
2   California Department of Public Health, Richmond, California, United States
,
Conner Brown
3   Information Services Department, Lucile Packard Children's Hospital, Palo Alto, California, United States
,
Ethan Steinberg
4   Center for Biomedical Informatics Research, Stanford University School of Medicine, Stanford, California, United States
5   Department of Computer Science, Stanford University, Stanford, California, United States
,
Rachel L. Goldstein
6   Division of Adolescent Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
,
Jennifer L. Carlson
6   Division of Adolescent Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
,
Natalie Pageler
1   Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
,
Keith E. Morse
1   Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
› Author Affiliations Funding None.
› Further Information
Also available at
    Permissions and Reprints

Abstract

Background The 21st Century Cures Act mandates the immediate, electronic release of health information to patients. However, in the case of adolescents, special consideration is required to ensure that confidentiality is maintained. The detection of confidential content in clinical notes may support operational efforts to preserve adolescent confidentiality while implementing information sharing.

Objectives This study aimed to determine if a natural language processing (NLP) algorithm can identify confidential content in adolescent clinical progress notes.

Methods A total of 1,200 outpatient adolescent progress notes written between 2016 and 2019 were manually annotated to identify confidential content. Labeled sentences from this corpus were featurized and used to train a two-part logistic regression model, which provides both sentence-level and note-level probability estimates that a given text contains confidential content. This model was prospectively validated on a set of 240 progress notes written in May 2022. It was subsequently deployed in a pilot intervention to augment an ongoing operational effort to identify confidential content in progress notes. Note-level probability estimates were used to triage notes for review and sentence-level probability estimates were used to highlight high-risk portions of those notes to aid the manual reviewer.

Results The prevalence of notes containing confidential content was 21% (255/1,200) and 22% (53/240) in the train/test and validation cohorts, respectively. The ensemble logistic regression model achieved an area under the receiver operating characteristic of 90 and 88% in the test and validation cohorts, respectively. Its use in a pilot intervention identified outlier documentation practices and demonstrated efficiency gains over completely manual note review.

Conclusion An NLP algorithm can identify confidential content in progress notes with high accuracy. Its human-in-the-loop deployment in clinical operations augmented an ongoing operational effort to identify confidential content in adolescent progress notes. These findings suggest NLP may be used to support efforts to preserve adolescent confidentiality in the wake of the information blocking mandate.

Keywords

confidentiality - patient portals - natural language processing - machine learning - health information exchange

Protection of Human and Animal Subjects

The presented work was performed as part of a quality improvement effort at our institution and does not qualify as human subjects research.




Publication History

Received: 12 October 2022

Accepted: 01 March 2023

Accepted Manuscript online:
10 March 2023

Article published online:
24 May 2023

© 2023. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Office of the National Coordinator for Health Information Technology. 21st Century Cures Act: interoperability, information blocking, and the ONC health IT certification program [Internet]. 2020. Accessed July 22, 2022 at: https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification
    PubMed
  • 2 Holmgren AJ, Patel V, Charles D, Adler-Milstein J. US hospital engagement in core domains of interoperability. Am J Manag Care 2016; 22 (12) e395-e402
    PubMedGoogle Scholar
  • 3 Nazi KM, Turvey CL, Klein DM, Hogan TP, Woods SSVA. VA OpenNotes: exploring the experiences of early patient adopters with access to clinical notes. J Am Med Inform Assoc 2015; 22 (02) 380-389
    CrossrefPubMedGoogle Scholar
  • 4 Mishra VK, Hoyt RE, Wolver SE, Yoshihashi A, Banas C. Qualitative and quantitative analysis of patients' perceptions of the patient portal experience with OpenNotes. Appl Clin Inform 2019; 10 (01) 10-18
    Article in Thieme ConnectPubMedGoogle Scholar
  • 5 Delbanco T, Walker J, Bell SK. et al. Inviting patients to read their doctors' notes: a quasi-experimental study and a look ahead. Ann Intern Med 2012; 157 (07) 461-470
    Google Scholar
  • 6 Walker J, Leveille S, Bell S. et al. OpenNotes after 7 years: patient experiences with ongoing access to their clinicians' outpatient visit notes. J Med Internet Res 2019; 21 (05) e13876
    CrossrefPubMedGoogle Scholar
  • 7 Wright E, Darer J, Tang X. et al. Sharing physician notes through an electronic portal is associated with improved medication adherence: quasi-experimental study. J Med Internet Res 2015; 17 (10) e226
    CrossrefPubMedGoogle Scholar
  • 8 Pageler NM, Webber EC, Lund DP. Implications of the 21st Century Cures Act in pediatrics. Pediatrics 2021; 147 (03) e2020034199
    CrossrefPubMedGoogle Scholar
  • 9 Carlson J, Goldstein R, Hoover K, Tyson N. NASPAG/SAHM Statement: the 21st Century Cures Act and adolescent confidentiality. J Adolesc Health 2021; 68 (02) 426-428
    CrossrefPubMedGoogle Scholar
  • 10 Schapiro NA, Mihaly LK. The 21st Century Cures Act and challenges to adolescent confidentiality. J Pediatr Health Care 2021; 35 (04) 439-442
    CrossrefPubMedGoogle Scholar
  • 11 Reddy DM, Fleming R, Swain C. Effect of mandatory parental notification on adolescent girls' use of sexual health care services. JAMA 2002; 288 (06) 710-714
    CrossrefPubMedGoogle Scholar
  • 12 Vukadinovich DM. Minors' rights to consent to treatment: navigating the complexity of State laws. J Health Law 2004; 37 (04) 667-691
    PubMedGoogle Scholar
  • 13 Pathak PR, Chou A. Confidential care for adolescents in the U.S. health care system. J Patient Cent Res Rev 2019; 6 (01) 46-50
    CrossrefPubMedGoogle Scholar
  • 14 Pampati S, Liddon N, Dittus PJ, Adkins SH, Steiner RJ. Confidentiality matters but how do we improve implementation in adolescent sexual and reproductive health care?. J Adolesc Health 2019; 65 (03) 315-322
    CrossrefPubMedGoogle Scholar
  • 15 Sharko M, Jameson R, Ancker JS, Krams L, Webber EC, Rosenbloom ST. State-by-state variability in adolescent privacy laws. Pediatrics 2022; 149 (06) e2021053458
    CrossrefPubMedGoogle Scholar
  • 16 Ginsburg KR, Slap GB, Cnaan A, Forke CM, Balsley CM, Rouselle DM. Adolescents' perceptions of factors affecting their decisions to seek health care. JAMA 1995; 273 (24) 1913-1918
    CrossrefPubMedGoogle Scholar
  • 17 Ford CA, Millstein SG, Halpern-Felsher BL, Irwin Jr CE. Influence of physician confidentiality assurances on adolescents' willingness to disclose information and seek future health care. A randomized controlled trial. JAMA 1997; 278 (12) 1029-1034
    CrossrefPubMedGoogle Scholar
  • 18 Lothen-Kline C, Howard DE, Hamburger EK, Worrell KD, Boekeloo BO. Truth and consequences: ethics, confidentiality, and disclosure in adolescent longitudinal prevention research. J Adolesc Health 2003; 33 (05) 385-394
    CrossrefPubMedGoogle Scholar
  • 19 Arvisais-Anhalt S, Lau M, Lehmann CU. et al. The 21st Century Cures Act and multiuser electronic health record access: potential pitfalls of information release. J Med Internet Res 2022; 24 (02) e34085
    CrossrefPubMedGoogle Scholar
  • 20 Parsons CR, Hron JD, Bourgeois FC. Preserving privacy for pediatric patients and families: use of confidential note types in pediatric ambulatory care. J Am Med Inform Assoc 2020; 27 (11) 1705-1710
    CrossrefPubMedGoogle Scholar
  • 21 Bedgood M, Kuelbs CL, Jones VG, Pageler N. Organizational perspectives on technical capabilities and barriers related to pediatric data sharing and confidentiality. JAMA Netw Open 2022; 5 (07) e2219692
    CrossrefPubMedGoogle Scholar
  • 22 Bedgood M, Rabbani N, Brown C. et al. The prevalence of confidential content in adolescent progress notes prior to the 21st Century Cures Act information blocking mandate. Appl Clin Inform 2023
    PubMedGoogle Scholar
  • 23 Ramos J. Using TF-IDF to determine word relevance in document queries. In: Proceedings of the First Instructional Conference on Machine Learning. International Conference on Machine Learning; 2003
    PubMed
  • 24 Ng AY. Feature selection, L1 vs. L2 regularization, and rotational invariance. In: Proceedings of the Twenty-First International Conference on Machine Learning; 2004 78.
    PubMed
  • 25 Ip W, Yang S, Parker J. et al. Assessment of Prevalence of Adolescent Patient Portal Account Access by Guardians. JAMA Netw Open 2021; 4 (09) e2124733
    CrossrefPubMedGoogle Scholar
  • 26 Xie J, McPherson T, Powell A. et al. Ensuring adolescent patient portal confidentiality in the Age of the Cures Act final rule. J Adolesc Health 2021; 69 (06) 933-939
    CrossrefPubMedGoogle Scholar
  • 27 Lee J, Yang S, Holland-Hall C. et al. Prevalence of sensitive terms in clinical notes using natural language processing techniques: observational study. JMIR Med Inform 2022; 10 (06) e38482
    CrossrefPubMedGoogle Scholar
  • 28 Ni Y, Bachtel A, Nause K, Beal S. Automated detection of substance use information from electronic health records for a pediatric population. J Am Med Inform Assoc 2021; 28 (10) 2116-2127
    CrossrefPubMedGoogle Scholar
  • 29 Murugan A, Gooding H, Greenbaum J. et al. Lessons learned from OpenNotes learning mode and subsequent implementation across a pediatric health system. Appl Clin Inform 2022; 13 (01) 113-122
    Article in Thieme ConnectPubMedGoogle Scholar
  • 30 Office of the National Coordinator for Health Information Technology. Cures Act final rule: information blocking exceptions [Internet]. 2022. Accessed September 12, 2022 at: https://www.healthit.gov/sites/default/files/2022-07/InformationBlockingExceptions.pdf
    PubMed
  • 31 Campbell S, Giadresco K. Computer-assisted clinical coding: a narrative review of the literature on its benefits, limitations, implementation and impact on clinical coding professionals. HIM J 2020; 49 (01) 5-18
    CrossrefPubMedGoogle Scholar
  • 32 Woo M. An AI boost for clinical trials. Nature 2019; 573 (7775): S100-S102
    CrossrefPubMedGoogle Scholar
  • 33 Maguire H, Elana M, Roy R, Vivian L, Washington V, Volpp KG. Asked and answered: building a chatbot to address Covid-19-related concerns. Catalyst non issue content 2020. Accessed April 14, 2023 at: https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0230/
    PubMedGoogle Scholar
  • 34 3M CodeAssist System [Internet]. Accessed September 12, 2022 at: https://www.3m.com/3M/en_US/health-information-systems-us/improve-revenue-cycle/coding/professional/code-assist/
    PubMed
  • 35 Health Language. NLP for unstructured data [Internet]. Accessed September 12, 2022 at: https://www.wolterskluwer.com/en/solutions/health-language/clinical-natural-language-processing
    PubMed
  • 36 Scibilia JP. How to protect maternal health information in newborn's medical record. AAP News 2014; 35 (12) 4-4
    PubMedGoogle Scholar
  • 37 Spector-Bagdady K, Mello MM. Protecting the privacy of reproductive health information after the fall of Roe v Wade. JAMA Health Forum 2022; 3 (06) e222656-e222656
    CrossrefPubMedGoogle Scholar