Download PDF
Appl Clin Inform 2017; 08(02): 369-380
DOI: 10.4338/ACI-2016-09-RA-0149
Research Article
Schattauer GmbH

Automating Clinical Score Calculation within the Electronic Health Record

A Feasibility Assessment
Christopher Aakre
1   Division of General Internal Medicine, Department of Internal Medicine, Mayo Clinic, Rochester, MN, USA
,
Mikhail Dziadzko
2   Department of Anesthesiology, Division of Critical Care, Mayo Clinic, Rochester, MN, USA
,
Mark T. Keegan
2   Department of Anesthesiology, Division of Critical Care, Mayo Clinic, Rochester, MN, USA
,
Vitaly Herasevich
2   Department of Anesthesiology, Division of Critical Care, Mayo Clinic, Rochester, MN, USA
3   Multidisciplinary Epidemiology and Translation Research in Intensive Care (METRIC), Mayo Clinic, Rochester, MN, USA
› Author Affiliations Funding This publication was made possible by CTSA Grant Number UL1 TR000135 from the National Center for Advancing Translational Sciences (NCATS), a component of the National Institutes of Health (NIH). Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NIH.
Further Information

Publication History

received: 01 September 2016

accepted: 07 February 2017

Publication Date:
21 December 2017 (online)

   Permissions and Reprints

Summary

Objectives: Evidence-based clinical scores are used frequently in clinical practice, but data collection and data entry can be time consuming and hinder their use. We investigated the programmability of 168 common clinical calculators for automation within electronic health records.

Methods: We manually reviewed and categorized variables from 168 clinical calculators as being extractable from structured data, unstructured data, or both. Advanced data retrieval methods from unstructured data sources were tabulated for diagnoses, non-laboratory test results, clinical history, and examination findings.

Results: We identified 534 unique variables, of which 203/534 (37.8%) were extractable from structured data and 269/534 (50.4.7%) were potentially extractable using advanced techniques. Nearly half (265/534, 49.6%) of all variables were not retrievable. Only 26/168 (15.5%) of scores were completely programmable using only structured data and 43/168 (25.6%) could potentially be programmable using widely available advanced information retrieval techniques. Scores relying on clinical examination findings or clinical judgments were most often not completely programmable.

Conclusion: Complete automation is not possible for most clinical scores because of the high prevalence of clinical examination findings or clinical judgments – partial automation is the most that can be achieved. The effect of fully or partially automated score calculation on clinical efficiency and clinical guideline adherence requires further study.

Citation: Aakre C, Dziadzko M, Keegan MT, Herasevich V. Automating clinical score calculation within the electronic health record: A feasibility assessment. Appl Clin Inform 2017; 8: 369–380 https://doi.org/10.4338/ACI-2016-09-RA-0149

Keywords

Automation - decision support algorithm - clinical score - knowledge translation - workflow - clinical practice guideline

Clinical Relevance Statement

Automated calculation of commonly used clinical scores within the EHR could reduce the cognitive-workload, improve practice efficiency, and facilitate clinical guideline adherence.


Human Subjects Protects

Humans subjects were not included in the project.


 

  • References

  • 1 Wyatt JC, Altman DG. Commentary: Prognostic models: clinically useful or quickly forgotten?. BMJ 1995; 311 7019 1539-1541.
    CrossrefPubMedGoogle Scholar
  • 2 January CT, Wann LS, Alpert JS, Calkins H, Cigarroa JE, Cleveland JC, Conti JB, Ellinor PT, Ezekowitz MD, Field ME, Murray KT, Sacco RL, Stevenson WG, Tchou PJ, Tracy CM, Yancy CW. American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2014 AHA/ACC/HRS guideline for the management of patients with atrial fibrillation: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and the Heart Rhythm Society. J Am Coll Cardiol 2014; 64 (21) e1-e76.
    CrossrefPubMedGoogle Scholar
  • 3 Kahn SR, Lim W, Dunn AS, Cushman M, Dentali F, Akl EA, Cook DJ, Balekian AA, Klein RC, Le H, Schulman S, Murad MH. Prevention of VTE in Nonsurgical Patients. Chest 2012; 141 (02) e195S-e226S.
    CrossrefPubMedGoogle Scholar
  • 4 Konstantinides S V, Torbicki A, Agnelli G, Danchin N, Fitzmaurice D, Galiè N, Gibbs JSR, Huisman M V, Humbert M, Kucher N, Lang I, Lankeit M, Lekakis J, Maack C, Mayer E, Meneveau N, Perrier A, Pruszczyk P, Rasmussen LH, Schindler TH, Svitil P, Vonk Noordegraaf A, Zamorano JL, Zompatori M. Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). 2014 ESC guidelines on the diagnosis and management of acute pulmonary embolism. Eur Heart J 2014; 35 (Suppl. 43) 3033-3069 3069a–3069k.
    CrossrefPubMedGoogle Scholar
  • 5 Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, Dowell SF, File TM, Musher DM, Niederman MS, Torres A, Whitney CG. Infectious Diseases Society of America/American Thoracic Society Consensus Guidelines on the Management of Community-Acquired Pneumonia in Adults. Clin Infect Dis 2007; 44 (02) S27-S72.
    CrossrefPubMedGoogle Scholar
  • 6 Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A, Harrell FE. The APACHE III Prognostic System. Chest 1991; 100 (06) 1619-1636.
    CrossrefPubMedGoogle Scholar
  • 7 Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II. Crit Care Med 1985; 13 (10) 818-829.
    CrossrefPubMedGoogle Scholar
  • 8 Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. APACHE—acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981; 9 (08) 591-597.
    CrossrefPubMedGoogle Scholar
  • 9 Gage BF, Waterman AD, Shannon W, Boechler M, Rich MW, Radford MJ. Validation of Clinical Classification Schemes for Predicting Stroke. JAMA 2001; 285 (22) 2864.
    CrossrefPubMedGoogle Scholar
  • 10 Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJGM, Lip GYH. A Novel User-Friendly Score (HAS-BLED) To Assess 1-Year Risk of Major Bleeding in Patients With Atrial Fibrillation. Chest 2010; 138 (05) 1093-1100.
    CrossrefPubMedGoogle Scholar
  • 11 Walker G. editor. MDCalc. New York, NY: MD Aware, LLC; 2016 [cited 2016 Apr 21]. Available from: http://www.mdcalc.com
    PubMed
  • 12 Schwartz D, Kruse C. QxMD. Vancouver, BC: QxMD Medical Software Inc.; 2016. [cited 2016 Apr 21]. Available from: http://www.qxmd.com
    Google Scholar
  • 13 Topol EJ. editor. Medscape. New York, NY: Medscape, LLC; 2016 [cited 2016 Apr 21]. Available from: http://www.medscape.com
    PubMedGoogle Scholar
  • 14 Fleischmann R, Duhm J, Hupperts H, Brandt SA. Tablet computers with mobile electronic medical records enhance clinical routine and promote bedside time: a controlled prospective crossover study. J Neurol 2015; 262 (03) 532-540.
    CrossrefPubMedGoogle Scholar
  • 15 Charles D, Gabriel M, Searcy T. Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008–2014. Vol. 23, ONC Data Brief. Office of the National Coordinator for Health Information Technology. 2015
    PubMedGoogle Scholar
  • 16 Tipping MD, Forth VE, O’Leary KJ, Malkenson DM, Magill DB, Englert K, Williams M V. Where did the day go?-A time-motion study of hospitalists. J Hosp Med 2010; 5 (06) 323-328.
    CrossrefPubMedGoogle Scholar
  • 17 Hill RG, Sears LM, Melanson SW. 4000 Clicks: a productivity analysis of electronic medical records in a community hospital ED. Am J Emerg Med 2013; 31 (11) 1591-1594.
    CrossrefPubMedGoogle Scholar
  • 18 Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ 2000; 320 7237 788-791.
    CrossrefPubMedGoogle Scholar
  • 19 Dziadzko MA, Gajic O, Pickering BW, Herasevich V. Clinical calculators in hospital medicine: Availability, classification, and needs. Comput Methods Programs Biomed 2016; 133: 1-6.
    CrossrefPubMedGoogle Scholar
  • 20 Post TW. editor. UpToDate. Waltham, MA: UpToDate, Inc.; 2016. Available from: http://www.uptodate.com
    Google Scholar
  • 21 Nadkarni PM, Ohno-Machado L, Chapman WW. Natural language processing: an introduction. J Am Med Inform Assoc 2011; 18 (05) 544-551.
    CrossrefPubMedGoogle Scholar
  • 22 R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: 2016
    Google Scholar
  • 23 Sintchenko V, Coiera EW. Which clinical decisions benefit from automation? A task complexity approach. Int J Med Inform 2003; 70 2–3 309-316.
    CrossrefPubMedGoogle Scholar
  • 24 Harrison AM, Yadav H, Pickering BW, Cartin-Ceba R, Herasevich V. Validation of Computerized Automatic Calculation of the Sequential Organ Failure Assessment Score. Crit Care Res Pract 2013; 2013: 1-8.
    PubMedGoogle Scholar
  • 25 Thomas M, Bourdeaux C, Evans Z, Bryant D, Greenwood R, Gould T. Validation of a computerised system to calculate the sequential organ failure assessment score. Intensive Care Med 2011; 37 (03) 557-557.
    CrossrefPubMedGoogle Scholar
  • 26 Mitchell WAL. An automated APACHE II scoring system. Intensive Care Nurs 1987; 3 (01) 14-18.
    CrossrefPubMedGoogle Scholar
  • 27 Gooder VJ, Farr BR, Young MP. Accuracy and efficiency of an automated system for calculating APACHE II scores in an intensive care unit. Proc AMIA Annu Fall Symp 1997; 131-135.
    PubMedGoogle Scholar
  • 28 Vinson DR, Morley JE, Huang J, Liu V, Anderson ML, Drenten CE, Radecki RP, Nishijima DK, Reed ME. The Accuracy of an Electronic Pulmonary Embolism Severity Index Auto-Populated from the Electronic Health Record: Setting the stage for computerized clinical decision support. Appl Clin Inform 2015; 6 (02) 318-333.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 29 Li L, Chase HS, Patel CO, Friedman C, Weng C. Comparing ICD9-encoded diagnoses and NLP-processed discharge summaries for clinical trials pre-screening: a case study. AMIA Annu Symp Proc 2008; 404-408.
    PubMedGoogle Scholar
  • 30 Jiang M, Chen Y, Liu M, Rosenbloom ST, Mani S, Denny JC, Xu H. A study of machine-learning-based approaches to extract clinical entities and their assertions from discharge summaries. J Am Med Inform Assoc 2011; 18 (05) 601-606.
    CrossrefPubMedGoogle Scholar
  • 31 Deo RC. Machine Learning in Medicine. Circulation 2015; 132 (20) 1920-1930.
    CrossrefPubMedGoogle Scholar
  • 32 Austin PC, Tu J V, Ho JE, Levy D, Lee DS. Using methods from the data-mining and machine-learning literature for disease classification and prediction: a case study examining classification of heart failure sub-types. J Clin Epidemiol 2013; 66 (04) 398-407.
    CrossrefPubMedGoogle Scholar
  • 33 de Bruijn B, Cherry C, Kiritchenko S, Martin J, Zhu X. Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010. J Am Med Inform Assoc 2011; 18 (Suppl. 05) 557-562.
    CrossrefPubMedGoogle Scholar
  • 34 Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 2016; 316 (22) 2402.
    CrossrefPubMedGoogle Scholar