Download PDF
CC BY 4.0 · ACI open 2021; 05(02): e67-e79
DOI: 10.1055/s-0041-1735470
Review Article

A Scoping Review of Artificial Intelligence Algorithms in Clinical Decision Support Systems for Internal Medicine Subspecialties

Ploypun Narindrarangkura
1   Institute for Data Science and Informatics, University of Missouri, Columbia, United States
,
Min Soon Kim
2   Department of Health Management and Informatics, University of Missouri Institute for Data Science and Informatics, University of Missouri, Columbia, United States
,
Suzanne A. Boren
2   Department of Health Management and Informatics, University of Missouri Institute for Data Science and Informatics, University of Missouri, Columbia, United States
› Author Affiliations Funding None.
› Further Information
   Permissions and Reprints

Abstract

Objectives Artificial intelligence (AI)-based clinical decision support systems (CDSS) have been developed to solve medical problems and enhance health care management. We aimed to review the literature to identify trends and applications of AI algorithms in CDSS for internal medicine subspecialties.

Methods A scoping review was conducted in PubMed, IEEE Xplore, and Scopus to determine articles related to CDSS using AI algorithms that use deep learning, machine learning, and pattern recognition. This review synthesized the main purposes of CDSS, types of AI algorithms, and overall accuracy of algorithms. We searched the original research published in English between 2009 and 2019.

Results Given the volume of articles meeting inclusion criteria, the results of 218 of the 3,467 articles were analyzed and presented in this review. These 218 articles were related to AI-based CDSS for internal medicine subspecialties: neurocritical care (n = 89), cardiovascular disease (n = 79), and medical oncology (n = 50). We found that the main purposes of CDSS were prediction (48.4%) and diagnosis (47.1%). The five most common algorithms include: support vector machine (20.9%), neural network (14.6%), random forest (10.5%), deep learning (9.2%), and decision tree (8.8%). The accuracy ranges of algorithms were 61.8 to 100% in neurocritical care, 61.6 to 100% in cardiovascular disease, and 54 to 100% in medical oncology. Only 20.1% of those algorithms had an explainability of AI, which provides the results of the solution that humans can understand.

Conclusion More AI algorithms are applied in CDSS and are important in improving clinical practice. Supervised learning still accounts for a majority of AI applications in internal medicine. This study identified four potential gaps: the need for AI explainability, the lack of ubiquity of CDSS, the narrow scope of target users of CDSS, and the need for AI in health care report standards.

Keywords

clinical decision support systems - artificial intelligence - deep learning - Machine learning - medicine

Author Contributions

S.A.B. and M.S.K contributed the study design, critical revision of this article, and final approval of the version to be published. P.N. searched the literature, synthesized included studies, and drafted the article.


Supplementary Material



Publication History

Received: 28 October 2020

Accepted: 13 July 2021

Article published online:
14 September 2021

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 HealthIT.gov. Clinical decision support. Accessed August 23, 2020 at: https://www.healthit.gov/topic/safety/clinical-decision-support
    Google Scholar
  • 2 Agency for Healthcare Research and Quality. Clinical decision support. Published June. 2019 . Accessed February 11, 2021 at: http://www.ahrq.gov/cpi/about/otherwebsites/clinical-decision-support/index.html
    Google Scholar
  • 3 Alther M, Reddy CK. Clinical Decision Support Systems. Boca Raton, FL: CRC Press; 2015
    Google Scholar
  • 4 Tate KE, Gardner RM, Weaver LK. A computerized laboratory alerting system. MD Comput 1990; 7 (05) 296-301
    PubMedGoogle Scholar
  • 5 Alanazi A, Al Rabiah F, Gadi H, Househ M, Al Dosari B. Factors influencing pharmacists' intentions to use Pharmacy Information Systems. Inform Med Unlocked. 2018; 11: 1-8
    CrossrefPubMedGoogle Scholar
  • 6 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330 (7494): 765
    CrossrefPubMedGoogle Scholar
  • 7 Panch T, Szolovits P, Atun R. Artificial intelligence, machine learning and health systems. J Glob Health 2018; 8 (02) 020303
    CrossrefPubMedGoogle Scholar
  • 8 Kim E, Rubinstein SM, Nead KT, Wojcieszynski AP, Gabriel PE, Warner JL. evolving use of electronic health records (EHR) for research. Semin Radiat Oncol 2019; 29 (04) 354-361
    CrossrefPubMedGoogle Scholar
  • 9 Mahadevaiah G, Rv P, Bermejo I, Jaffray D, Dekker A, Wee L. Artificial intelligence-based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance. Med Phys 2020; 47 (05) e228-e235
    CrossrefPubMedGoogle Scholar
  • 10 Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 2017; 69S: S36-S40
    CrossrefPubMedGoogle Scholar
  • 11 Kulikowski CA. An opening chapter of the first generation of artificial intelligence in medicine: the First Rutgers AIM Workshop, June 1975. Yearb Med Inform 2015; 10 (01) 227-233
    PubMedGoogle Scholar
  • 12 Shortliffe EH, Axline SG, Buchanan BG, Merigan TC, Cohen SN. An artificial intelligence program to advise physicians regarding antimicrobial therapy. Comput Biomed Res 1973; 6 (06) 544-560
    CrossrefPubMedGoogle Scholar
  • 13 Samuel AL. Some studies in machine learning using the game of checkers. IBM J Res Develop 1959; 3 (03) 210-229
    CrossrefPubMedGoogle Scholar
  • 14 Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med 2019; 380 (14) 1347-1358
    CrossrefPubMedGoogle Scholar
  • 15 Kaelbling LP, Littman ML, Moore AW. Reinforcement learning: a survey. J Artif Intell Res 1996; 4: 237-285
    CrossrefPubMedGoogle Scholar
  • 16 Artetxe A, Beristain A, Graña M. Predictive models for hospital readmission risk: A systematic review of methods. Comput Methods Programs Biomed 2018; 164: 49-64
    CrossrefPubMedGoogle Scholar
  • 17 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521 (7553): 436-444
    Google Scholar
  • 18 Han SS, Kim MS, Lim W, Park GH, Park I, Chang SE. Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J Invest Dermatol 2018; 138 (07) 1529-1538
    CrossrefPubMedGoogle Scholar
  • 19 Rajpurkar P, Irvin J, Ball RL. et al. Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med 2018; 15 (11) e1002686
    CrossrefPubMedGoogle Scholar
  • 20 Jaiswal AK, Tiwari P, Kumar S, Gupta D, Khanna A, Rodrigues JJPC. Identifying pneumonia in chest X-rays: a deep learning approach. Measurement 2019; 145: 511-518
    CrossrefPubMedGoogle Scholar
  • 21 Gulshan V, Peng L, Coram M. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016; 316 (22) 2402-2410
    CrossrefPubMedGoogle Scholar
  • 22 Castaneda C, Nalley K, Mannion C. et al. Clinical decision support systems for improving diagnostic accuracy and achieving precision medicine. J Clin Bioinforma 2015; 5: 4
    CrossrefPubMedGoogle Scholar
  • 23 Moja L, Kwag KH, Lytras T. et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health 2014; 104 (12) e12-e22
    CrossrefPubMedGoogle Scholar
  • 24 Nuckols TK, Smith-Spangler C, Morton SC. et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56
    CrossrefPubMedGoogle Scholar
  • 25 Turek M. Explainable artificial intelligence. Accessed October 30, 2019 at: https://www.darpa.mil/program/explainable-artificial-intelligence
    Google Scholar
  • 26 Edwards L, Veale M. Slave to the algorithm? Why a ‘Right to an Explanation’ is probably not the remedy you are looking for. Social Science Research Network. 2017 . Accessed February 27, 2020 at: https://papers.ssrn.com/abstract=2972855
    Google Scholar
  • 27 Holzinger A, Langs G, Denk H, Zatloukal K, Müller H. Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip Rev Data Min Knowl Discov 2019; 9 (04) e1312
    CrossrefPubMedGoogle Scholar
  • 28 Holzinger A, Biemann C, Pattichis CS, Kell DB. What do we need to build explainable AI systems for the medical domain? ArXiv171209923 Cs Stat. Published online December 28, 2017. Accessed March 2, 2021 at: http://arxiv.org/abs/1712.09923
    Google Scholar
  • 29 Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Precise4Q consortium. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak 2020; 20 (01) 310
    CrossrefPubMedGoogle Scholar
  • 30 FDA. Artificial intelligence and machine learning in software as a medical device. FDA. Published online January 11, 2021. Accessed February 16, 2021 at: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device
    Google Scholar
  • 31 Politi MC, Dizon DS, Frosch DL, Kuzemchak MD, Stiggelbout AM. Importance of clarifying patients' desired role in shared decision making to match their level of engagement with their preferences. BMJ 2013; 347 (dec02 1): f7066-f7066
    CrossrefPubMedGoogle Scholar
  • 32 Stacey D, Légaré F, Lewis K. et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2017; 4: CD001431
    Google Scholar
  • 33 Stoeklé HC, Charlier P, Hervé C, Deleuze JF, Vogt G. Artificial intelligence in internal medicine: between science and pseudoscience. Eur J Intern Med 2018; 51: e33-e34
    CrossrefPubMedGoogle Scholar
  • 34 IBM Watson Health | AI healthcare solutions. IBM Watson Health. Published February 19, 2021. Accessed March 2, 2021 at: https://www.ibm.com/watson-health
    Google Scholar
  • 35 MeVis medical solutions AG. Accessed March 2, 2021 at: https://www.mevis.de/en
    Google Scholar
  • 36 ACP. What is a doctor of internal medicine, or internist?. Accessed March 9, 2021 at: https://www.acponline.org/acp-newsroom/what-is-a-doctor-of-internal-medicine-or-internist-0
    Google Scholar
  • 37 ACP. Internal medicine subspecialties career information. Accessed October 21, 2020 at: https://www.acponline.org/about-acp/about-internal-medicine/subspecialties
    Google Scholar
  • 38 AAMC. Physician specialty data report, 2019. Accessed March 9, 2021 at: https://www.aamc.org/data-reports/workforce/interactive-data/active-physicians-largest-specialties-2019
    Google Scholar
  • 39 Murray NM, Unberath M, Hager GD, Hui FK. Artificial intelligence to diagnose ischemic stroke and identify large vessel occlusions: a systematic review. J Neurointerv Surg 2020; 12 (02) 156-164
    CrossrefPubMedGoogle Scholar
  • 40 Kilic A. Artificial intelligence and machine learning in cardiovascular health care. Ann Thorac Surg 2020; 109 (05) 1323-1329
    CrossrefPubMedGoogle Scholar
  • 41 Jin P, Ji X, Kang W. et al. Artificial intelligence in gastric cancer: a systematic review. J Cancer Res Clin Oncol 2020; 146 (09) 2339-2350
    CrossrefPubMedGoogle Scholar
  • 42 Booth CM, Tannock IF. Randomised controlled trials and population-based observational research: partners in the evolution of medical evidence. Br J Cancer 2014; 110 (03) 551-555
    CrossrefPubMedGoogle Scholar
  • 43 Sanson-Fisher RW, Bonevski B, Green LW, D'Este C. Limitations of the randomized controlled trial in evaluating population-based health interventions. Am J Prev Med 2007; 33 (02) 155-161
    CrossrefPubMedGoogle Scholar
  • 44 Mc Cord KA, Hemkens LG. Using electronic health records for clinical trials: Where do we stand and where can we go?. CMAJ 2019; 191 (05) E128-E133
    CrossrefPubMedGoogle Scholar
  • 45 Bishop CM. Pattern Recognition and Machine Learning. New York, NY: Springer; 2006
    Google Scholar
  • 46 CDC. Introduction | Meaningful use. Published September 17, 2020. Accessed February 16, 2021 at: https://www.cdc.gov/ehrmeaningfuluse/introduction.html
    Google Scholar
  • 47 Murphy EV. Clinical decision support: effectiveness in improving quality processes and clinical outcomes and factors that may influence success. Yale J Biol Med 2014; 87 (02) 187-197
    Google Scholar
  • 48 Wang F, Sun M, Min T. et al. Analysis for early seizure detection system based on deep learning algorithm. In: Schmidt H, Griol DZL, Wang H, Baumbach J, Zheng H, Callejas Z, Hu X, Dickerson J. eds. Proceedings - 2018 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2018. Piscataway, NJ: Institute of Electrical and Electronics Engineers Inc.; 2019: 2382-2389
    Google Scholar
  • 49 Ahmedt-Aristizabal D, Fookes C, Nguyen K, Denman S, Sridharan S, Dionisio S. Deep facial analysis: a new phase I epilepsy evaluation using computer vision. Epilepsy Behav 2018; 82: 17-24
    CrossrefPubMedGoogle Scholar
  • 50 Abibullaev B, Kim MS, Seo HD. Seizure detection in temporal lobe epileptic EEGs using the best basis wavelet functions. J Med Syst 2010; 34 (04) 755-765
    CrossrefPubMedGoogle Scholar
  • 51 Kharat PA, Dudul SV. Clinical decision support system based on Jordan/Elman neural networks. In: 2011 IEEE Recent Advances in Intelligent Computational Systems, RAICS 2011. Piscataway, NJ: Institute of Electrical and Electronics Engineers Inc.; 2011: 255-259
    Google Scholar
  • 52 Kharat PA, Dudul SV. Epilepsy diagnosis based on generalized feed forward neural network. Interdiscip Sci 2012; 4 (03) 209-214
    CrossrefPubMedGoogle Scholar
  • 53 Trambaiolli LR, Spolaôr N, Lorena AC, Anghinah R, Sato JR. Feature selection before EEG classification supports the diagnosis of Alzheimer's disease. Clin Neurophysiol 2017; 128 (10) 2058-2067
    CrossrefPubMedGoogle Scholar
  • 54 Sun M, Wang F, Min T, Zang T, Wang Y. Prediction for high risk clinical symptoms of epilepsy based on deep learning algorithm. IEEE Access 2018; 6: 77596-77605
    CrossrefPubMedGoogle Scholar
  • 55 Golmohammadi M, Harati Nejad Torbati AH, Lopez de Diego S, Obeid I, Picone J. Automatic analysis of eegs using big data and hybrid deep learning architectures. Front Hum Neurosci 2019; 13: 76
    CrossrefPubMedGoogle Scholar
  • 56 Subasi A, Ahmed A, Aličković E, Rashik Hassan A. Effect of photic stimulation for migraine detection using random forest and discrete wavelet transform. Biomed Signal Process Control 2019; 49: 231-239
    CrossrefPubMedGoogle Scholar
  • 57 Rodríguez Aldana Y, Marañón Reyes EJ, Macias FS. et al. Nonconvulsive epileptic seizure monitoring with incremental learning. Comput Biol Med 2019; 114: 103434
    CrossrefPubMedGoogle Scholar
  • 58 Chiang HS, Chen MY, Huang YJ. Wavelet-based EEG processing for epilepsy detection using fuzzy entropy and associative petri net. IEEE Access 2019; 7: 103255-103262
    CrossrefPubMedGoogle Scholar
  • 59 Abdulkadir A, Mortamet B, Vemuri P, Jack Jr CR, Krueger G, Klöppel S. Alzheimer's Disease Neuroimaging Initiative. Effects of hardware heterogeneity on the performance of SVM Alzheimer's disease classifier. Neuroimage 2011; 58 (03) 785-792
    CrossrefPubMedGoogle Scholar
  • 60 Zhang X, Hu B, Ma X, Moore P, Chen J. Ontology driven decision support for the diagnosis of mild cognitive impairment. Comput Methods Programs Biomed 2014; 113 (03) 781-791
    CrossrefPubMedGoogle Scholar
  • 61 Munsell BC, Wee CY, Keller SS. et al. Evaluation of machine learning algorithms for treatment outcome prediction in patients with epilepsy based on structural connectome data. Neuroimage 2015; 118: 219-230
    CrossrefPubMedGoogle Scholar
  • 62 Pillai PS, Leong TY. Alzheimer's Disease Neuroimaging Initiative. Fusing heterogeneous data for Alzheimer's disease classification. Stud Health Technol Inform 2015; 216: 731-735
    PubMedGoogle Scholar
  • 63 Wu X, Zou Q, Hu J. et al. Intrinsic functional connectivity patterns predict consciousness level and recovery outcome in acquired brain injury. J Neurosci 2015; 35 (37) 12932-12946
    CrossrefPubMedGoogle Scholar
  • 64 Hu C, Ju R, Shen Y, Zhou P, Li Q. Clinical decision support for Alzheimer's disease based on deep learning and brain network. In: 2016 IEEE International Conference on Communications, ICC 2016. Piscataway, NJ: Institute of Electrical and Electronics Engineers Inc.; 2016
    Google Scholar
  • 65 Zhang J, Fan Y, Li Q, Thompson PM, Ye J, Wang Y. Empowering cortical thickness measures in clinical diagnosis of Alzheimer's disease with spherical sparse coding. Proc IEEE Int Symp Biomed Imaging 2017; 2017: 446-450
    PubMedGoogle Scholar
  • 66 Yang H, Zhang J, Liu Q, Wang Y. Multimodal MRI-based classification of migraine: using deep learning convolutional neural network. Biomed Eng Online 2018; 17 (01) 138
    CrossrefPubMedGoogle Scholar
  • 67 Nielsen A, Hansen MB, Tietze A, Mouridsen K. Prediction of tissue outcome and assessment of treatment effect in acute ischemic stroke using deep learning. Stroke 2018; 49 (06) 1394-1401
    CrossrefPubMedGoogle Scholar
  • 68 Pinto A, Mckinley R, Alves V, Wiest R, Silva CA, Reyes M. Stroke lesion outcome prediction based on MRI imaging combined with clinical information. Front Neurol 2018; 9: 1060
    CrossrefPubMedGoogle Scholar
  • 69 Ju R, Hu C, Zhou P, Li Q. Early diagnosis of Alzheimer's disease based on resting-state brain networks and deep learning. IEEE/ACM Trans Comput Biol Bioinformatics 2019; 16 (01) 244-257
    CrossrefPubMedGoogle Scholar
  • 70 Law MT, Traboulsee AL, Li DK. et al. Machine learning in secondary progressive multiple sclerosis: an improved predictive model for short-term disability progression. Mult Scler J Exp Transl Clin 2019; 5 (04) 2055217319885983
    Google Scholar
  • 71 Li Y, Charalampaki P, Liu Y, Yang GZ, Giannarou S. Context aware decision support in neurosurgical oncology based on an efficient classification of endomicroscopic data. Int J CARS 2018; 13 (08) 1187-1199
    CrossrefPubMedGoogle Scholar
  • 72 Cha YJ, Jang WI, Kim MS. et al. Prediction of response to stereotactic radiosurgery for brain metastases using convolutional neural networks. Anticancer Res 2018; 38 (09) 5437-5445
    CrossrefPubMedGoogle Scholar
  • 73 Mitchell TM. Machine Learning. New York, NY: McGraw-Hill; 1997
    Google Scholar
  • 74 Hernandez-Boussard T, Bozkurt S, Ioannidis JPA, Shah NH. MINIMAR (MINimum Information for Medical AI Reporting): developing reporting standards for artificial intelligence in health care. J Am Med Inform Assoc 2020; 27 (12) 2011-2015
    CrossrefPubMedGoogle Scholar
  • 75 GBD 2016 Neurology Collaborators. Global, regional, and national burden of neurological disorders, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol 2019; 18 (05) 459-480
    CrossrefPubMedGoogle Scholar
  • 76 WHO. Cardiovascular diseases (CVDs). Accessed March 7, 2021 at: https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds)
    Google Scholar
  • 77 WHO. The top 10 causes of death. Accessed March 7, 2021 at: https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death
    Google Scholar
  • 78 Kochanek KD, Xu JQ, Arias E. Mortality in the United States, 2019. National Center for Health Statistics. 2020 . Accessed March 7, 2021 at: https://www.cdc.gov/nchs/products/databriefs/db395.htm
    Google Scholar
  • 79 Khamparia A, Singh KM. A systematic review on deep learning architectures and applications. Expert Syst 2019; 36 (03) e12400
    CrossrefPubMedGoogle Scholar
  • 80 Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias and clinical safety. BMJ Qual Saf 2019; 28 (03) 231-237
    CrossrefPubMedGoogle Scholar
  • 81 Arroyo-Gallego T, Ledesma-Carbayo MJ, Butterworth I. et al. Detecting motor impairment in early Parkinson's disease via natural typing interaction with keyboards: validation of the neuroQWERTY approach in an uncontrolled at-home setting. J Med Internet Res 2018; 20 (03) e89
    CrossrefPubMedGoogle Scholar
  • 82 Guidi G, Pettenati MC, Melillo P, Iadanza E. A machine learning system to improve heart failure patient assistance. IEEE J Biomed Health Inform 2014; 18 (06) 1750-1756
    CrossrefPubMedGoogle Scholar
  • 83 Isma'eel HA, Cremer PC, Khalaf S. et al. Artificial neural network modeling enhances risk stratification and can reduce downstream testing for patients with suspected acute coronary syndromes, negative cardiac biomarkers, and normal ECGs. Int J Cardiovasc Imaging 2016; 32 (04) 687-696
    CrossrefPubMedGoogle Scholar
  • 84 Costanzo D. Biomedical data acquisition and processing in the decision support services of HEARTFAID platform. In: Proceedings of the 5th IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, IDAACS'2009. Piscataway, NJ: Institute of Electrical and Electronics Engineers Inc.; 2009: 291-296
    Google Scholar
  • 85 Ng T, Chew L, Yap CW. A clinical decision support tool to predict survival in cancer patients beyond 120 days after palliative chemotherapy. J Palliat Med 2012; 15 (08) 863-869
    CrossrefPubMedGoogle Scholar
  • 86 Bertsimas D, Dunn J, Pawlowski C. et al. Applied informatics decision support tool for mortality predictions in patients with cancer. JCO Clin Cancer Inform 2018; 2: 1-11
    Google Scholar
  • 87 De Bari B, Vallati M, Gatta R. et al. Development and validation of a machine learning-based predictive model to improve the prediction of inguinal status of anal cancer patients: a preliminary report. Oncotarget 2016; 8 (65) 108509-108521
    CrossrefPubMedGoogle Scholar
  • 88 Fu MR, Axelrod D, Guth AA. et al. mHealth self-care interventions: managing symptoms following breast cancer treatment. mHealth 2016; 2: 28-28
    CrossrefPubMedGoogle Scholar
  • 89 Miller K, Mosby D, Capan M. et al. Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support. J Am Med Inform Assoc 2018; 25 (05) 585-592
    CrossrefPubMedGoogle Scholar
  • 90 Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform 2005; 38 (01) 75-87
    CrossrefPubMedGoogle Scholar
  • 91 Brunner J, Chuang E, Goldzweig C, Cain CL, Sugar C, Yano EM. User-centered design to improve clinical decision support in primary care. Int J Med Inform 2017; 104: 56-64
    CrossrefPubMedGoogle Scholar
  • 92 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
    CrossrefPubMedGoogle Scholar
  • 93 Bright TJ, Wong A, Dhurjati R. et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
    Google Scholar
  • 94 Jones SS, Rudin RS, Perry T, Shekelle PG. Health information technology: an updated systematic review with a focus on meaningful use. Ann Intern Med 2014; 160 (01) 48-54
    CrossrefPubMedGoogle Scholar
  • 95 Peters MDJ, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid-Based Healthc 2015; 13 (03) 141-146
    CrossrefPubMedGoogle Scholar
  • 96 Tcheng JE, Bakken S, Bates DW. et al. (eds) Optimizing Strategies for Clinical Decision Support: Summary of a Meeting Series. Washington, DC: National Academy of Medicine; 2017
    Google Scholar
  • 97 Sittig DF, Wright A, Osheroff JA. et al. Grand challenges in clinical decision support. J Biomed Inform 2008; 41 (02) 387-392
    CrossrefPubMedGoogle Scholar
  • 98 Hersh WR, Weiner MG, Embi PJ. et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care 2013; 51 (08, Suppl 3): S30-S37
    CrossrefPubMedGoogle Scholar
  • 99 Bhise V, Rajan SS, Sittig DF. et al. Electronic health record reviews to measure diagnostic uncertainty in primary care. J Eval Clin Pract 2018; 24 (03) 545-551
    CrossrefPubMedGoogle Scholar
  • 100 Dexheimer JW, Taylor RG, Kachelmeyer AM, Reed JL. The reliability of computerized physician order entry data for research studies. Pediatr Emerg Care 2019; 35 (03) e61-e64
    CrossrefPubMedGoogle Scholar
  • 101 Murphy DR, Satterly T, Rogith D, Sittig DF, Singh H. Barriers and facilitators impacting reliability of the electronic health record-facilitated total testing process. Int J Med Inform 2019; 127: 102-108
    CrossrefPubMedGoogle Scholar
  • 102 Weston J, Mukherjee S, Chapelle O, Pontil M, Poggio T, Vapnik V. Feature selection for SVMs. Adv Neural Inform Process Syst 13 (NIPS 2000) 2000; 13: 668-674
    PubMedGoogle Scholar
  • 103 Lorena AC, de Carvalho ACPLF. Evolutionary tuning of SVM parameter values in multiclass problems. Neurocomputing 2008; 71 (16–18): 3326-3334
    CrossrefPubMedGoogle Scholar
  • 104 Yu H, Yang J, Han J. Classifying large data sets using SVMs with hierarchical clusters. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '03. New York, NY: ACM Press; 2003: 306
    Google Scholar
  • 105 Gilpin LH, Bau D, Yuan BZ, Bajwa A, Specter MA, Kagal L. Explaining Explanations: An Overview of Interpretability of Machine Learning. 2018 IEEE 5th International Conference on Data Science and Advanced Analytics DSAA 2018; 80-89
    PubMedGoogle Scholar
  • 106 Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995; 20 (03) 273-297
    CrossrefPubMedGoogle Scholar
  • 107 Hsu CW, Lin CJ. A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 2002; 13 (02) 415-425
    CrossrefPubMedGoogle Scholar
  • 108 Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 1996; 49 (11) 1225-1231
    CrossrefPubMedGoogle Scholar
  • 109 Livingstone DJ, Manallack DT, Tetko IV. Data modelling with neural networks: advantages and limitations. J Comput Aided Mol Des 1997; 11 (02) 135-142
    CrossrefPubMedGoogle Scholar
  • 110 Specht DF. A general regression neural network. IEEE Trans Neural Netw 1991; 2 (06) 568-576
    CrossrefPubMedGoogle Scholar
  • 111 Couronné R, Probst P, Boulesteix AL. Random forest versus logistic regression: a large-scale benchmark experiment. BMC Bioinformatics 2018; 19 (01) 270
    CrossrefPubMedGoogle Scholar
  • 112 Li B, Chen X, Li MJ, Huang JZ, Feng S. Scalable random forests for massive data. In: Tan P-N, Chawla S, Ho CK, Bailey J. eds. Advances in Knowledge Discovery and Data Mining. Vol 7301. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 2012: 135-146
    Google Scholar
  • 113 Tin Kam Ho. The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 1998; 20 (08) 832-844
    CrossrefPubMedGoogle Scholar
  • 114 Breiman L. Random forests. Mach Learn 2001; 45 (01) 5-32
    CrossrefPubMedGoogle Scholar
  • 115 Kamiński B, Jakubczyk M, Szufel P. A framework for sensitivity analysis of decision trees. Cent Eur J Oper Res 2018; 26 (01) 135-159
    CrossrefPubMedGoogle Scholar
  • 116 Kotsiantis SB. Decision trees: a recent overview. Artif Intell Rev 2013; 39 (04) 261-283
    CrossrefPubMedGoogle Scholar
  • 117 Quinlan JR. Induction of decision trees. Mach Learn 1986; 1 (01) 81-106
    Google Scholar
  • 118 Samek W, Binder A, Montavon G, Lapuschkin S, Müller K-R. Evaluating the visualization of what a deep neural network has learned. IEEE Trans Neural Netw Learn Syst 2017; 28 (11) 2660-2673
    CrossrefPubMedGoogle Scholar
  • 119 Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. ArXiv14090473 Cs Stat. Published online May 19, 2016. Accessed October 31, 2019 at: https://arxiv.org/abs/1409.0473
    Google Scholar
  • 120 Donahue J, Hendricks LA, Rohrbach M. et al. Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans Pattern Anal Mach Intell 2017; 39 (04) 677-691
    CrossrefPubMedGoogle Scholar