Contributions on Clinical Decision Support from the 2018 Literature

Summary

Objectives: To summarize recent research and select the best papers published in 2018 in the field of computerized clinical decision support for the Decision Support section of the International Medical Informatics Association (IMIA) yearbook.

Methods: A literature review was performed by searching two bibliographic databases for papers referring to clinical decision support systems (CDSSs). The aim was to identify a list of candidate best papers from the retrieved bibliographic records, which were then peer-reviewed by external reviewers. A consensus meeting of the IMIA editorial team finally selected the best papers on the basis of all reviews and the section editors' evaluation.

Results: Among 1,148 retrieved articles, 15 best paper candidates were selected, the review of which resulted in the selection of four best papers. The first paper introduces a deep learning model for estimating short-term life expectancy (>3 months) of metastatic cancer patients by analyzing free-text clinical notes in electronic medical records, while maintaining the temporal visit sequence. The second paper takes note that CDSSs become routinely integrated in health information systems and compares statistical anomaly detection models to identify CDSS malfunctions which, if remain unnoticed, may have a negative impact on care delivery. The third paper fairly reports on lessons learnt from the development of an oncology CDSS using artificial intelligence techniques and from its assessment in a large US cancer center. The fourth paper implements a preference learning methodology for detecting inconsistencies in clinical practice guidelines and illustrates the applicability of the proposed methodology to antibiotherapy.

Conclusions: Three of the four best papers rely on data-driven methods, and one builds on a knowledge-based approach. While there is currently a trend for data-driven decision support, the promising results of such approaches still need to be confirmed by the adoption of these systems and their routine use.

Keywords

Medical informatics - International Medical Informatics Association - Yearbook - Decision Support Systems

Introduction

Clinical decision support has always been a central theme of medical informatics and, thus, naturally one of the sections of the International Medical Informatics Association (IMIA)Yearbook. The goal of this synopsis is to summarize recent research in the domain of decision support and to select the best papers published in this field during 2018. Our literature review targeted research works related to clinical decision support systems (CDSSs) and computerized provider order entry (CPOE) systems. The review supplements this year’s survey paper of the decision support section by Montani and Striani [1], which focuses on the different artificial intelligence (AI) approaches adopted for decision support, with a special focus on data-driven, knowledge-based, or hybrid approaches.

The synopsis is organized as follows: the next section briefly describes the whole review protocol for selecting the best papers on the topic; the following section presents the results of this year’s selection process, and the last section comments the main contributions of the four best papers as well as noticeable research works in the domain of decision support, which were identified during the selection process.

Paper Selection Method

A comprehensive literature search on topics related to CDSSs and CPOE systems was performed to identify candidate best papers following the protocol previously described by Lamy et al. [2]. Two bibliographic databases were used: first, the PubMed/Medline database (from NCBI, National Center for Biotechnology Information) which is dedicated to the biomedical domain, and the Web of Science^® (WoS, Clarivate Analytics) which has a broader scope. Both databases were searched with similar queries targeting journal papers published in 2018, written in English language, and on the aforementioned topics. The adopted strategy, which was implemented for the last 2 years [3] [4] and replicated this year, was based on four exclusive queries yielding four disjoint citation subsets: Q_{Pub plain}, based on a plain-text search in PubMed titles and abstracts using keywords; Q_{Pub indexed}, based on the PubMed indexing scheme using MeSH terms and exclusive of the previous set; Q_{WoS_restricted}, based on a WoS search on non PubMed-indexed papers restricted to the two subject areas “Medical Informatics” and “Health Care Sciences & Services” and, finally, Q_{WoS_filtered}, based on other non-PubMed-indexed papers filtered by non-relevant subject areas.

A first review of the four subsets of retrieved citations was performed by the two section editors to select 15 candidate best papers. Then, following the IMIA Yearbook protocol, these candidate best papers were individually reviewed and rated by external reviewers from the international Medical Informatics community. Based on the reviewers’ ratings and comments, the Yearbook editorial committee had to select the best papers of the year in the decision support domain.

Review Results

Our literature search was conducted on January 14, 2019. A total of 1,148 references were obtained, distributed as follows according to the four queries: 869 for Q_{Pub plain}, 122 for Q_Pub indexed,¹⁶ for Q_{WoS_restricted}, ami 141 for Q_{WoS_filtered}, yielding sub-totals of 991 references from PubMed and 157 from WoS. Compared to the previous year, we retrieved 46 papers less in total. After the first individual review by each of the two section editors, a total of 56 papers (which were not rejected by both section editors) were reviewed again by the two editors to achieve a selection of 15 candidate best papers. Then, following the IMIA Yearbook best paper selection process, these 15 papers were rated by external reviewers and the Yearbook editors. Four papers were finally selected as best papers for 2018 [5] [6] [7] [8] ([Table 1]). These are discussed in the next section, while summaries of their contents are available in the Appendix ofthis synopsis.

Table 1 Best paper selection of articles for the IMIA Yearbook of Medical Informatics 2019 in the section ‘Decision Support’. The articles are listed in alphabetical order of the first author’s surname.

Section Decision Support
▪ Banerjee I, Gensheimer MF, Wood DJ, Henry S, Aggarwal S, Chang DT, Rubin DL. Probabilistic prognostic estimates of survival in metastatic cancer patients (PPES-Met) utilizing free-text clinical narratives. Sci Rep 2018 Jul 3;8(1):10037.
▪ Ray S, McEvoy DS, Aaron S, Hickman TT, Wright A. Using statistical anomaly detection models to find clinical decision support malfunctions. J Am Med Inform Assoc 2018 Jul 1 ;25(7):862-71.
▪ Simon G, DiNardo CD, Takahashi K, Cascone T, Powers C, Stevens R, Allen J, Antonoff MB, Gomez D, Keane P, Suarez Saiz F, Nguyen Q, Roarty E, Pierce S, Zhang J, Hardeman Barnhill E, Lakhani K, Shaw K, Smith B, Swisher S, High R, Futreal PA, Heymach, Chin L. Applying Artificial Intelligence to address the knowledge gaps in cancer care. Oncologist 2018 Nov 16.
▪ Tsopra R, Lamy JB, Sedki K. Using preference learning for detecting inconsistencies in clinical practice guidelines: methods and application to antibiotherapy. Artif Intell Med 2018 Jul;89:24-33.

Discussion and Outlook

In the first paper, Banerjee et al. [5] elaborate on an AI-based approach for predicting complex disease trajectories, focusing particularly on a probabilistic-based, prognostic estimation of survival for metastatic cancer patients. For this purpose, the study analyzed free-text clinical notes contained in electronic medical records (EMRs), while maintaining the temporal visit sequence; a critical aspect for the targeted problem, given that events are irregular in metastatic cancer patients and should be weighted based on persisting temporal context. The main elements of the respective predictive model included: (a) a hybrid pipeline that combined semantic data mining with neural embedding for creating context-aware dense vector representation of the multiple types of free-text clinical notes considered in the study; (b) an efficient deep prognostic model that took as input the context-aware vectorized representation of sequential clinical notes and generated the probability of short-term life expectancy estimate (>3 months), and (c) an interactive visualization method aiming to improve physician understanding of the provided predictions. Especially the last part constitutes an important contribution in the domain with an explicit focus on explainability, which constitutes currently a central issue for the adoption of AI in medicine. Data from relevant databases from the Stanford Cancer Center were employed in the study that demonstrated both high accuracy and explainability, and showcased a promising decision support tool to personalize metastatic cancer treatment.

In the second paper, Ray et al. [6] take note that automated CDSSs have become integral components of EMRs and that any unnoticed malfunction of them might impair the quality or safety of care. The goal of this research was to use and compare anomaly detection models to identify potential CDS malfunctions. Authors focused on three types of anomalies that occur in time and which can be detected by statistical models applied on time series. Retrospective data, originating from a large US hospital where CDS alert/rule triggering is logged, were used to compare six anomaly detection models on time series of four rules with known malfunctions. In these examples, malfunctions led to a rule firing stop or increase and came from changes in the rule premises or errors in drug codes. Models performed differently according to the rules. Further work would focus on online (real-time) anomaly detection. As they are routinely implemented, automated CDSSs should be monitored to avoid poor performance outcomes.

The third paper by Simon et al. [7] is a case study report on the experience the authors gained in developing the Oncology Expert Advisor (OEA), an AI-based CDSS, and through its clinical assessment while introducing it to clinic team members. OEA has been built on top of the widely advertised IBM Watson technologies [9] and has been developed for the MD Anderson cancer center (Houston, TX, USA). In the recent years, public attention had been focused on this important AI-based development along with extensive discussions about the feasibility of its promise. However, this paper is of note, since it scientifically provides insights on the motivations, the results, and the issues encountered. OEA was assigned three core functionalities to support healthcare professionals: patient history summarization from the EMR, treatment options recommendation, and disease management advisory. For each function, the authors discussed the obtained results and the limitations of the approaches, among which the difficulties in establishing ground truth for learning algorithms and in summarizing historical data. When the system was introduced with a clinic team, errors were collected and analyzed. This led the authors to conclude that the development of such systems should not be technology-driven, but that clinical expertise should be brought into the development process from the very early phases.

In the fourth paper, Tsopra et al. [8] elaborate in addressing contradictions and inconsistencies in clinical practice guidelines. In particular, the study proposes a method for the semi-automatic detection of inconsistencies in guidelines using preference learning, which has been successfully applied in antibiotherapy for primary care. Interestingly, before presenting the proposed method, the article provides comprehensive background information concerning preference learning, optimization approaches for metaheuristics, and a knowledge base for antibiotherapy. Learning of the preference model relied both on recommendations and on a knowledge base describing the domain. The authors argue that they successfully built a generic model suitable for all infectious diseases and patient profiles, including both references and necessary features. This model could be the basis for CDSSs targeting antibiotics prescription.

Besides the four best papers selected for the Decision Support section of the 2019 edition of the IMIA Yearbook, several works retrieved from our literature review deserve to be cited. In particular, in terms of methodological contributions, Peleg et al. [10] elaborated on mobile health behavioral support for patients targeting their compliance to therapy, illustrating the proposed approach through a case study on atrial fibrillation management. Kamisalic et al. [11] studied the formalization and acquisition of temporal knowledge for decision support in medical processes, while Yan et al. [12] presented a novel, dialogue-based approach for dealing with uncertain and conflicting information in medical diagnosis. Interestingly, Goodwin and Harabagiu [13] presented a comprehensive study on knowledge representation and inference techniques for medical question answering. In terms of technical implementations, Wulff et al. [14] presented an interoperable CDSS for early detection of Systemic Inflammatory Response Syndrome in pediatric intensive care using open EHR, while El-Sap- pagh et al. [15] combined ontologies and fuzzy logic to develop a quite advanced system for diagnosing diabetes. An interesting study was also presented by Gombolay et al. [16], elaborating on robotic assistance in the coordination of patient care. As regards CDSS interaction design, Chattopadhyay et al. [17] presented a novel study focusing on the design and evaluation of trust-eliciting cues in drug-drug interaction alerts. Regarding CDSS evaluation and impact assessment, Kheterpal et al. [18] assessed the impact of a novel multiparameter DSS targeting intraoperative processes of care and postoperative outcomes. Wang et al. [19] evaluated clinical order patterns which were obtained by applying machine learning methods in clinician cohorts stratified by patient mortality outcomes. Finally, Parshuram et al. [20] presented the EPOCH randomized clinical trial which coped with assessing the effect of an early warning system on all-cause mortality in hospitalized pediatric patients.

Among the four best papers, three implemented data-driven methods [5] [6] [7], whereas one builds on a knowledge-based approach [8]. This is not inconsistent with the survey paper by Montani and Striani [1], which remarked on the prevalence of data-driven AI over knowledge-based AI in the recent scientific literature related to decision support. However, while data-driven approaches offer promising results, they need to bridge the gap with routine use and adoption. This still leaves space for various other approaches as illustrated by the selection of the candidate best papers on decision support introduced above.

Acknowledgement

We would like to thank Martina Hutter and Adrien Ugon for their support, as well as the external reviewers for their participation to the selection process ofthe Decision Support section of the IMIA Yearbook.

• References

• 1 Montani S, Striani M. Artificial Intelligence in clinical decision support: a focused literature survey. Yearb Med Inform 2019; 120-7
• 2 Lamy JB, Seroussi B, Griffon N, Kerdelhue G, Jaulent MC, Bouaud J. Toward a formalization of the process to select IMIA Yearbook best papers. Methods Inf Med. 2015; 54 (02) 135-44
• 3 Koutkias V, Bouaud J. Contributions from the 2016 Literature on Clinical Decision Support. Yearb Med Inform 2017; 26 (01) 133-8
• 4 Koutkias V, Bouaud J. Contributions from the 2017 Literature on Clinical Decision Support. Yearb Med Inform 2018; 27 (01) 122-8
• 5 Banerjee I, Gensheimer MF, Wood DJ, Henry S, Aggarwal S, Chang DT. , et al. Probabilistic prognostic estimates of survival in metastatic cancer patients (PPES-Met) utilizing free-text clinical narratives. Sci Rep. 2018; 8 (01) 10037
• 6 Ray S, McEvoy DS, Aaron S, Hickman TT, Wright A. Using statistical anomaly detection models to find clinical decision support malfunctions. J Am Med Inform Assoc. 2018; 25 (07) 862-71
• 7 Simon G, DiNardo CD, Takahashi K, Cascone T, Powers C, Stevens R. , et al. Applying Artificial Intelligence to address the knowledge gaps in cancer care. Oncologist. 2018 Nov 16. pii: theoncologist.2018-0257
• 8 Tsopra R, Lamy JB, Sedki K. Using preference learning for detecting inconsistencies in clinical practice guidelines: methods and application to antibiotherapy. Artif Intell Med. 2018; 89: 24-33
• 9 Kohn MS, Sun J, Knoop S, Shabo A, Carmeli B, Sow D. , et al. IBM’s Health Analytics and Clinical Decision Support. Yearb Med Inform. 2014; 9: 154-62
• 10 Peleg M, Michalowski W, Wilk S, Parimbelli E, Bonaccio S, O’sullivan D. , et al. Ideating mobile health behavioral support for compliance to therapy for patients with chronic disease: a case study ofatrial fibrillation management. J Med Syst. 2018; 42 (11) 234
• 11 Kamisalic A, Riano D, Welzer T. Formalization and acquisition of temporal knowledge for decision support in medical processes. Comput Methods Programs Biomed. 2018; 158: 207-28
• 12 Yan C, Lindgren H, Nieves JC. A dialogue-based approach for dealing with uncertain and conflicting information in medical diagnosis. Auton Agent Multi-Ag 2018; 32 (06) 861-85
• 13 Goodwin TR, Harabagiu SM. Knowledge representations and inference techniques for medical question answering. ACM Transactions on Intelligent Systems and Technology 2018; 9 (02) 14
• 14 Wulff A, Haarbrandt B, Tute E, Marschollek M, Beerbaum P, Jack T. An interoperable clinical decision-support system for early detection of SIRS in pediatric intensive care using open EHR. Artif Intell Med. 2018; 89: 10-23
• 15 El-Sappagh S, Alonso JM, Ali F, Ali A, Jang JH, Kwak KS. An ontology-based interpretable fuzzy decision support system for diabetes diagnosis. IEEE Access 2018; 6: 37371-94
• 16 Gombolay M, Jessie Yang X, Hayes B, Seo N, Liu Z, Wadhwania S. , et al. Robotic assistance in the coordination of patient care. The International Journal of Robotics Research 2018; 37 (10) 1300-16
• 17 Chattopadhyay D, Verma N, Duke J, Bolchini D. Design and evaluation of trust-eliciting cues in drug-drug interaction alerts. Interacting with Computers 2018; 30 (02) 85-98
• 18 Kheterpal S, Shanks A, Tremper KK. Impact of a novel multiparameter decision support system on intraoperative processes of care and postoperative outcomes. Anesthesiology. 2018; 128 (02) 272-82
• 19 Wang JK, Hom J, Balasubramanian S, Schuler A, Shah NH, Goldstein MK. , et al. An evaluation of clinical order patterns machine-learned from clinician cohorts stratified by patient mortality outcomes. J Biomed Inform. 2018; 86: 109-19
• 20 Parshuram CS, Dryden-Palmer K, Farrell C, Gottesman R, Gray M, Hutchison JS. , et al. Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: the EPOCH randomized clinical trial. JAMA 2018; 319 (10) 1002-12