Appl Clin Inform 2017; 08(03): 945-948
DOI: 10.4338/ACI2017050081
Schattauer GmbH

Response to Lapkoff and Sittig

Who Watches the Watchers: Working Towards Safety for Clinical Decision Support Knowledge Resources
John Poikonen
1  Avhana Health, Baltimore, MD
Edward Fotsch
2  Gemini Health, Sausalito, CA
Christoph U. Lehmann
3  Vanderbilt University Medical Center, Nashville, TN
› Author Affiliations
Further Information

Correspondence to:

Christoph U. Lehmann, MD, FAAP, FACMI
Professor, Biomedical Informatics and Pediatrics
Vanderbilt University Medical Center

Publication History

received: 19 May 2017

accepted in revised form: 15 July 2017

Publication Date:
20 December 2017 (online)


We thank Drs. Labkoff and Sittig for their provocative editorial and proposal [[1]]. We agree that there can be serious usability, workflow, and content issues in EHRs. Based on some structural and practical observations, we present counter arguments in response to the proposed quasi-governmental oversight.

Publishing Clinical References

Labkoff and Sittig advocate that CDS oversight should extend to the content of the linked reference sources. Clinical references have survived peer review under well-established publishing oversight and law. Consumers of clinical references are healthcare professionals considered by the courts to be “learned intermediaries”. That means they are expected to be more than blind followers and must utilize their knowledge, training, and skepticism when using reference materials. Critical review of the literature is a skill that is taught to medical students and residents early in their careers [[2]].


Lack of Harm

The proposed definition of clinical decision support (CDS) to include “online medical compendia” and/or reference links creates in our opinion excessive and unnecessary work. First, broken links probably will not cause end-user errors. Broken, outdated links or links to inappropriate references constitute errors most likely picked up in vendor or user quality assurance testing. In addition, they would be apparent errors to health care professional end users. Actual patient harm resulting from broken links and reference materials would certainly trigger press, litigation, and would involve liability carriers. We are not aware of broken link events and Labkoff and Sittig did not provide examples of actual patient harm. Creating a bureaucracy to deal with these labor-intensive maintenance issues (which have not been proven to constitute safety hazards) would create a ‘cure’ worse than the disease.


Exaggerating the Problem

Labkoff and Sittig exaggerate the problem by offering two (of the three) references as justification where incorrect knowledge was not even linked in EHRs. These references were on-line references that did not meet their extended definition of CDS [[3], [4]]. The third reference discusses an error in calculating risks scores incorrectly, whose risk on “a limited number’ of patients” was considered ‘low’ [[5]]. The evidence that the problem described by Labkoff and Sittig is causing patient harm through EHR use remains unproven.


Creating more Bureaucracy

Labkoff and Sittig do not provide answers on how to fund or staff their proposal, nor do they discuss the limits of scope of authority. The effort required would be immense. Absent legislation, what power would a quasi-governmental entity wield? Who would pay for it? Who would oversee its work? Would the entity have legal authority and ability to enforce by leveraging fines? How would appeals to decisions work? In addition, would its enforcement have a chilling impact on the investment in and development of important new clinical resources and new decision support? Assuring the accuracy of reference information would be a never-ending process since new clinical information is created continually.

A reasonable simple test to determine the need for government intervention is to ask if the “problem” is significant with tangible impact AND if it is not reasonably addressed by the marketplace. The proposal by Labkoff and Sittig appears to fail both aspects of the test, making yet another governmental oversight body superfluous.


Responsible monitoring and regulation

There are others outlining best practices and oversight of CDS. Recommendations for responsible monitoring and regulation of clinical software systems was addressed by a consortium of stakeholders 20 years ago. Their recommendations are worth reviewing [[6]]. The FDA is rolling out pilot programs in their digital health device strategy that are a risk-based approach to regulation. This evolving initiative is expected in the Fall according to a recent blog post [[7]]. While the authors of this letter are not associated with the Clinical Decision Support Coalition, we generally support their position and guidelines for clinical decision support [[8]]. The construct of transparency, competent human intervention, and time to reflect enabling independent review is in our opinion the correct solution. We do not need another quasi-governmental bureaucracy.

While we agree that a Health Information Safety Center would be a worthwhile endeavor to advance the research into EHR safety, we believe that there are many safety issues that have higher priority and more impact on patient care than regulating medical references and compendia.


Multiple Choices Questions

With online medical compendia that is linked to an Electronic Medical record, how is the quality best maintained:

  • Through submission of the content for FDA approval

  • Reporting of errors to a quasi-governmental reporting agency

  • Though the current established publishing oversight and law

  • Review of learned intermediaries

Answer: C The editorial outlines that there are current and established publishing oversight principles and law that can and does support online medical compendia quality.


Conflict of Interest

Dr. Poikonen is employed by Avhana Health, a clinical decision support company. Dr. Fotsch is employed by Gemini Health. Dr. Lehmann is Editor in Chief of Applied Clinical Informatics. The authors have no other conflicts to report.

Human Subject Research Approval

This editorial contains no patient data, therefore, it is not subject to Human Subject Research Approval

Correspondence to:

Christoph U. Lehmann, MD, FAAP, FACMI
Professor, Biomedical Informatics and Pediatrics
Vanderbilt University Medical Center