Abstract
Introduction Drug–drug interaction (DDI) alerts are often implemented in the hospital computerized
provider order entry (CPOE) systems with limited evaluation. This increases the risk
of prescribers experiencing too many irrelevant alerts, resulting in alert fatigue.
In this study, we aimed to evaluate clinical relevance of alerts prior to implementation
in CPOE using two common approaches: compendia and expert panel review.
Methods After generating a list of hypothetical DDI alerts, that is, alerts that would have
been triggered if DDI alerts were operational in the CPOE, we calculated the agreement
between multiple drug interaction compendia with regards to the severity of these
alerts. A subset of DDI alerts (n = 13), with associated patient information, were presented to an expert panel to
reach a consensus on whether each alert should be included in the CPOE.
Results There was poor agreement between compendia in their classifications of DDI severity
(Krippendorff's α: 0.03; 95% confidence interval: –0.07 to 0.14). Only 10% of DDI
alerts were classed as severe by all compendia. On the other hand, the panel reached
consensus on 12 of the 13 alerts that were presented to them regarding whether they
should be included in the CPOE.
Conclusion Using an expert panel and allowing them to discuss their views openly likely resulted
in high agreement on what alerts should be included in a CPOE system. Presenting alerts
in the context of patient cases allowed panelists to identify the conditions under
which alerts were clinically relevant. The poor agreement between compendia suggests
that this methodology may not be ideal for the evaluation of DDI alerts. Performing
preimplementation review of DDI alerts before they are enabled provides an opportunity
to minimize the risk of alert fatigue before prescribers are exposed to false-positive
alerts.
Keywords
medical order entry systems - drug interactions - alert fatigue - alert systems -
clinical decision support
