Keywords
computerized provider order entry - clinical decision support - electronic health
            record - renal dose adjustment - medication alert systems
Background and Significance
         Background and Significance
            Clinical decision support (CDS) tools are decision aides within clinical information
               systems that provide information and knowledge to optimize the delivery of patient
               care.[1]
               [2]
               [3]
               [4] Current clinical information systems apply CDS in a variety of formats. CDS can
               be delivered passively through informational text or a limited selection list. CDS
               can also be delivered actively, such as through noninterruptive user prompts to be
               resolved at a time of the user's choice, or through interruptive alerts requiring
               immediate resolution.[1]
               [2]
               
            The “five rights” of CDS states that the right information should be delivered to
               the right person through the right channel in the right format at the right time.[5] If optimally designed, CDS can augment patient care by providing technical safeguards
               to clinician-end users as they enter medication orders electronically.[2]
               [3] This promise, however, is largely unrealized, as many users respond to poorly designed
               CDS with high override rates and alert fatigue.[4]
               [5]
               [6]
               [7]
               
            Several individual strategies have been successful in improving user adherence to
               alerts. A noninterruptive CDS strategy that provided users with preselected defaults
               and a filtered selection list of context-appropriate order parameters was shown to
               be an effective means of providing renal dose guidance.[8] A passive CDS strategy of providing in-line support text was shown to be an effective
               means of increasing adherence to drug–drug interaction alerts.[9] We leveraged both strategies and designed a multistage CDS system to support successive
               parts of ordering workflow, beginning with order entry, proceeding through order signing,
               and concluding with medication warning resolution. We targeted antimicrobial medications
               requiring renal dose adjustment prescribed to pediatric patients. Renal dose adjustments
               are common clinical decisions that require the user to account for the patient's degree
               of renal impairment using laboratory markers and clinical calculations (e.g., Schwartz's
               equation), and then adjust the medication regimen accordingly.[10]
               [11] If renal dose adjustments are overlooked, patients may experience drug toxicity
               due to their reduced capacity to eliminate these compounds and/or their metabolites.[12]
               [13]
               [14] Antimicrobial medications are commonly administered within inpatient settings, are
               routinely monitored by clinicians, and have well-defined dosing schedules for varying
               degrees of renal impairment.[15]
               [16] This class of medications is therefore well-suited for trialing a CDS system designed
               to be transparent, flexible, minimally intrusive, and integrated closely with medication
               ordering and prescribing.
         Objectives
            The objective of this study was to investigate the impact of the passive dose-guidance
               and successive CDS, which employs varying renal dosing assistance strategies at multiple
               points of the order entry workflow, on adherence of providers to institutional antimicrobial
               dosing guidelines in hospitalized pediatric patients.
         Methods
            Study Setting
            
            The study was conducted at a 1,000-bed academic health center with a pediatric hospital
               containing 276 beds. The electronic health record (EHR) at the time of the study was
               based on the Epic platform (Epic version 2014, Epic Systems, Verona, WI), and medication
               warning data were supplied through a third-party data vendor (First Data Bank, San
               Francisco, CA) through monthly loads.
            
            Antimicrobial Dosing Guidelines
            
            An interdisciplinary committee published pediatric antimicrobial dosing guidelines
               that provided evidence-based dosing parameters for antibiotic, antifungal, and antiviral
               drugs for a variety of indications and disease severities. The guidelines also provided
               dose adjustment recommendations for five discrete categories of renal function: continuous
               renal replacement therapy (CRRT); intermittent hemodialysis (iHD); creatinine clearance
               (CRCL) less than 10 mL/min; CRCL between 10 mL/min and less than 30 mL/min; CRCL between
               30 mL/min and 50 mL/min; or CRCL greater than 50 mL/min.
            
            CDS Intervention and Implementation
            
            The implementation of our successive CDS system was separated into two phases due
               to logistical issues arising from the EHR version upgrade occurring concurrently with
               the project. The phased implementation presented an opportunity to investigate the
               effects of the different CDS elements individually ([Fig. 1]). End users were informed via a brief system-wide e-mail communication following
               a regular system enhancement. Each phase is described in the following subsections:
            
             Fig. 1 Implementation timeline and study design. The solid line denotes the implementation
                  timeline, and the dotted line denotes the study design's comparisons.
                  Fig. 1 Implementation timeline and study design. The solid line denotes the implementation
                  timeline, and the dotted line denotes the study design's comparisons.
            
            
            
            Noninterruptive Dose Warnings and Interruptive Dose Alerts without Renal Dose Adjustment
               Recommendations (Historical Control)
            
            Prior to March 2015, passive dose guidance was absent and dose-checking functionality
               was based on dosing rules provided by the data vendor, which did not provide renal
               adjustment recommendations. If a provider entered order parameters (dose and frequency)
               that exceeded preconfigured thresholds, an in-line dose warning appeared directly
               below order composer's dose field in a noninterruptive manner ([Fig. 2]). If the provider continued through the prescribing workflow and signed the order,
               the system generated an interruptive dose alert, prompting the provider to either
               change the dose or acknowledge/override the alert ([Fig. 3]). Orders placed during this time period served as the historical controls.
            
             Fig. 2 Noninterruptive in-line dose warning. The warning shown contains an actionable dose
                  button that redirects the user to the maximum dose allowed for the patient (not pictured).
                  Fig. 2 Noninterruptive in-line dose warning. The warning shown contains an actionable dose
                  button that redirects the user to the maximum dose allowed for the patient (not pictured).
            
            
            
             Fig. 3 Interruptive dose alert.
                  Fig. 3 Interruptive dose alert.
            
            
            
            
            Addition of Passive Dose-Guidance CDS (Phase I)
            
            During phase I, we implemented passive dose guidance to display within the ordering
               instructions of the order composer ([Fig. 4]). If a provider selected an antimicrobial for order entry, the system displayed
               appropriate dosing recommendations using embedded rules and logic based on the calculated
               CRCL and selected drug. [Table 1] provides the detailed order instructions displayed for cefepime as an illustrative
               example.
            
            
               
                  Table 1 
                     Criteria for passive dose-guidance CDS recommendations. Details for cefepime shown
                     
                  
                     
                     
                        
                        | Criterion | Passive CDS displayed text | 
                     
                  
                     
                     
                        
                        | No HD or CRRT order, CRCL incalculable | Creatinine clearance is not available because the patient may not have height or weight
                              documented or a creatinine level is not available from the last 14 days. Please consult
                              the antibiotic guidelines or pharmacy for assistance. | 
                     
                     
                        
                        | CRRT order | Patient has an active CRRT order. Estimated creatinine clearance: ___ mL/min/1.73
                              m2 (by Schwartz formula based on Cr of __) Recommended doses are: Mild/mod infection: 50 mg/kg/dose every 12 hours Severe infection/febrile neutropenia: 50 mg/kg/dose every 12 hours CF: 50 mg/kg/dose every 12 hours | 
                     
                     
                        
                        | HD order | Patient has an active hemodialysis order. Estimated creatinine clearance: ___ mL/min/1.73
                              m2 (by Schwartz formula based on Cr of __) Recommended doses are: Mild/mod infection: 25 mg/kg/dose every 48 hours Severe infection/febrile neutropenia: 25–50 mg/kg/dose every 48 hours CF: 25–50 mg/kg/dose every 48 hours Schedule dose after hemodialysis | 
                     
                     
                        
                        | CRCL < 10 mL/min | Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __) Recommended doses are: Mild/mod infection: 25 mg/kg/dose every 48 hours Severe infection/febrile neutropenia: 25–50 mg/kg/dose every 48 hours CF: 25–50 mg/kg/dose every 48 hours | 
                     
                     
                        
                        | CRCL 10–29 mL/min | Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __) Recommended doses are: Mild/mod infection: 50 mg/kg/dose every 24 hours Severe infection/febrile neutropenia: 50 mg/kg/dose every 24 hours CF: 50 mg/kg/dose every 24 hours | 
                     
                     
                        
                        | CRCL 30–50 mL/min | Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __) Recommended doses are: Mild/mod infection: 50 mg/kg/dose every 24 hours Severe infection/febrile neutropenia: 50 mg/kg/dose every 12–24 hours CF: 50 mg/kg/dose every 12–24 hours | 
                     
                     
                        
                        | CRCL > 50 mL/min | Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __) Recommended doses are: Mild/mod infection: 50 mg/kg/dose every 12 hours Severe infection/febrile neutropenia: 50 mg/kg/dose every 8 hours CF: 50 mg/kg/dose every 8 hours | 
                     
               
               
               
               Abbreviations: CF, cefepime; CRCL, creatinine clearance; CRRT, continuous renal replacement
                  therapy; HD, hemodialysis.
               
                
            
            
            
             Fig. 4 Passive dose-guidance CDS.
                  Fig. 4 Passive dose-guidance CDS.
            
            
            
            
            Noninterruptive Dose Warnings and Interruptive Dose Alerts with Renal Dose Adjustment
               Recommendations (Phase II)
            
            In phase II, the data vendor updated the dose checking rules with renal dose adjustment
               recommendations. We reconfigured the rules to match dosing recommendations and renal
               adjustment parameters specified by the institution's guidelines. Upon fully implementing
               the successive CDS system in phase II, providers may receive CDS at multiple points
               during ordering workflow: (1) passive dose guidance, matching the institutional guidelines
               and adjusted for renal function, immediately after selecting an antimicrobial medication
               to order, (2) noninterruptive dose checking, matching the institutional guidelines
               and adjusted for renal function, during order parameter selection, and (3) interruptive
               dose alerting, matching the institutional guidelines and adjusted for renal function,
               after order signing.
            
            
            Renal Function and Creatinine Clearance Calculation Assumptions
            
            The system calculated renal function for pediatric patients using a modified Schwartz
               equation,[10] where K = 0.413 for all patients meeting the system's pediatric definition (age
               greater than 1 year but less than 14 years, or age 14–18 years and weighing less than
               50 kg). CRCL was calculated using the most recent serum creatinine result within a
               19-day lookback period. HD- or CRRT-specific recommendations overrode CRCL-based recommendations
               when discrete orders for HD or CRRT existed. If no CRCL was calculable, the system
               stated that renal function was incalculable and instructed the user to refer to the
               institutional guidelines for further assistance.
            
            
            Study Design
            
            The study was a single-center, observational, retrospective cohort study. The study
               compared provider adherence to institutional guidelines across three time points:
               a historical control period prior to implementation, phase I of CDS implementation
               (March 2015 through May 2015), and phase II of CDS implementation (May 2015 through
               October 2015).
            
            We assessed all antimicrobial orders within this time frame. We included antimicrobial
               orders for admitted patients who fit the system's definition of a pediatric patient.
               We excluded orders for patients with a principal diagnosis or problem list indication
               of cystic fibrosis; without a documented creatinine or height; with “once,” “user-specified,”
               “as needed,” or “preoperative” frequencies; with a free-text antimicrobial selected;
               or with antimicrobials without CDS configured. These antimicrobials either lacked
               renal dose adjustment recommendations or were subject to therapeutic drug monitoring
               by clinical pharmacists.
            
            Outcomes and Analysis
            
            The primary outcome of the study was defined as the proportion of orders adherent
               to institutional guidelines with regard to both dose and frequency. The secondary
               outcomes of the study were defined as the proportion of orders adherent to institutional
               guidelines with regard to dose alone, frequency alone, or total daily dose alone.
               We also assessed the proportion of adherent orders stratified by user roles and by
               presence of renal dysfunction. User roles were defined as physicians, pharmacists,
               registered nurses, and advanced practice practitioners (e.g., nurse practitioners
               and physician assistants).
            
            To conduct the analysis, we compared the order parameters between the provider's initial
               order and the parameters recommended by the CDS system. Because the CDS system suggested
               a range of appropriate doses, we defined the reference range as the range of values
               between the lowest possible value and the highest possible value as suggested by the
               institutional dosing guidelines. A 10% allowance was incorporated to account for the
               EHR's threshold for triggering a maximum dose-checking alert.
            
            Data and Statistical Analysis
            
            Order parameters were collected from EHR reports. For each order placed, the system
               logged the initial order parameters selected by the ordering user (e.g., drug, dose,
               frequency) in addition to final order parameters filed after the pharmacist verified
               the order. Our analysis focused on the initial order parameters, rather than the final
               order parameters. These data were then joined to patient-specific information extracted
               from other EHR reports. Mathematical calculations and initial comparisons/analysis
               were conducted in Microsoft Excel (Microsoft, Redmond, WA) spreadsheet software.
            
            Student's t-test was used to compare baseline age and CRCL. χ
               2 test was used to compare baseline gender frequency. Primary and secondary outcomes
               were recorded as categorical variables and reported as frequencies and percentages,
               with odds ratios (ORs) and p-values describing the magnitude of effect. We used a logistical regression with a
               mixed model analysis (PROC GLIMMIX) to assess statistical significance. We used the
               mixed model to account for the interrelatedness of groups of orders clustering within
               individual providers. Statistical tests were performed in SAS (SAS Institute, Cary,
               NC).
            Results
            After excluding all ineligible orders, a total of 1,588 orders remained for further
               analysis, ordered by 488 different unique providers, 618 orders in the control period,
               234 orders in the phase I, and 736 orders in phase II ([Table 2]). The mean patient age was 7.2, 6.7, and 6.8 years in the control period, phase
               I, and phase II, respectively. Although the mean CRCL was different between the time
               periods, the mean CRCL was well above the institutional guideline's minimum threshold
               for normal renal function (CRCL of 50 mL/min) for all time periods.
            
               
                  Table 2 
                     Baseline information
                     
                  
                     
                     
                        
                        |  | Control | Phase I | Phase II | 
                     
                  
                     
                     
                        
                        | Orders written, n (%) | 618 (39) | 234 (15) | 736 (46) | 
                     
                     
                        
                        | Age (y), mean ± SD (p)[a]
                               | 7.3 ± 4.5 | 6.7 ± 4.8 (0.07) | 6.8 ± 4.5 (0.04)[*]
                               | 
                     
                     
                        
                        | Female, n (%, p)[a]
                               | 269 (44) | 114 (49,0.17) | 
                              371 (50,0.01)
                              [*]
                               | 
                     
                     
                        
                        | CRCL, mean ± SD (p)[a]
                               | 182.5 ± 92.9 | 159.8 ± 79.3 (0.001)[*]
                               | 
                              152.1 ± 75 (<0.001)
                              [*]
                               | 
                     
                     
                        
                        | 
                              Orders written by CRCL category
                               | 
                     
                     
                        
                        | CRRT, n (%) | 40 (6) | 27 (12) | 36 (4.9) | 
                     
                     
                        
                        | <10 or HD, n (%) | 30 (5) | 5 (0.3) | 4 (0.5) | 
                     
                     
                        
                        | 10–29, n (%) | 6 (1) | 7 (3) | 9 (1.2) | 
                     
                     
                        
                        | 30–50, n (%) | 17 (3) | 4 (1.7) | 10 (1.4) | 
                     
                     
                        
                        | >50, n (%) | 525 (85) | 191 (82) | 677 (92) | 
                     
                     
                        
                        | 
                              Orders written by provider type
                               | 
                     
                     
                        
                        | Physician, n (%) | 413 (67) | 151 (65) | 472 (64) | 
                     
                     
                        
                        | APP, n (%) | 82 (13) | 43 (18) | 143 (19) | 
                     
                     
                        
                        | Registered nurse, n (%) | 60 (10) | 28 (12) | 86 (12) | 
                     
                     
                        
                        | Pharmacist, n (%) | 63 (10) | 12 (5) | 35 (5) | 
                     
               
               
               
               Abbreviations: APP, advanced practice practitioner (nurse practitioner and physician
                  assistant); CRCL, creatinine clearance; CRRT, continuous renal replacement therapy;
                  HD, hemodialysis.
               
               
               a Student's t-test to compare mean age and creatinine clearance, χ
                  2 test for gender comparison between phase I vs. control period, and between phase
                  II vs. control period.
               
               
               * 
                  p-value < 0.05.
               
                
            
            
            Primary Outcome
            
            The proportion of adherent orders with respect to both dose and frequency increased
               from 74% in the control period to 81% in phase II of the study (OR = 1.54; 95% confidence
               interval = 1.16–2.03; p = 0.003). There was no statistically significant difference in the primary outcome
               in phase I of the study compared with the control period ([Table 3], [Fig. 5], [Fig. 6]).
            
             Fig. 5 Percentages of antimicrobial orders within recommended range specified by institutional
                  guidelines. Control: noninterruptive dose warnings and interruptive dose alerts without
                  renal dose adjustment recommendations. Phase I: addition of passive dose-guidance
                  CDS. Phase II: noninterruptive dose warnings and interruptive dose alerts with renal
                  dose adjustment recommendations.
                  Fig. 5 Percentages of antimicrobial orders within recommended range specified by institutional
                  guidelines. Control: noninterruptive dose warnings and interruptive dose alerts without
                  renal dose adjustment recommendations. Phase I: addition of passive dose-guidance
                  CDS. Phase II: noninterruptive dose warnings and interruptive dose alerts with renal
                  dose adjustment recommendations.
            
            
            
             Fig. 6 Odds ratio of adherent orders compared versus historical control. Control: noninterruptive
                  dose warnings and interruptive dose alerts without renal dose adjustment recommendations.
                  Phase I: addition of passive dose-guidance CDS. Phase II: noninterruptive dose warnings
                  and interruptive dose alerts with renal dose adjustment recommendations.
                  Fig. 6 Odds ratio of adherent orders compared versus historical control. Control: noninterruptive
                  dose warnings and interruptive dose alerts without renal dose adjustment recommendations.
                  Phase I: addition of passive dose-guidance CDS. Phase II: noninterruptive dose warnings
                  and interruptive dose alerts with renal dose adjustment recommendations.
            
            
            
            
               
                  Table 3 
                     Results of the successive CDS system across the phased implementation
                     
                  
                     
                     
                        
                        | Adherence | Control, n (%) | Phase I | Phase II | 
                     
                     
                        
                        | 
                              n (%) | OR (95% CI, p) | 
                              n (%) | OR (95% CI, p) | 
                     
                  
                     
                     
                        
                        | Dose and frequency | 
                     
                     
                        
                        |  Overall | 456 (74) | 178 (76) | 1.11 (0.76–1.16, 0.6) | 596 (81) | 
                              1.54 (1.16–2.03, 0.003)[*]
                                 
                               | 
                     
                     
                        
                        |  Physician | 297 (72) | 117 (77) | 1.31 (0.82–2.1, 0.26) | 380 (81) | 
                              1.66 (1.18–2.35, 0.004)
                              [*]
                               | 
                     
                     
                        
                        |  APP | 63 (77) | 31 (72) | 0.78 (0.3–2.03, 0.61) | 118 (83) | 1.47 (0.7–3.09, 0.31) | 
                     
                     
                        
                        |  Registered nurse | 55 (92) | 23 (82) | 0.42 (<0.001 to >999, 0.42) | 72 (84) | 0.47 (<0.001–512.68, 0.4) | 
                     
                     
                        
                        |  Pharmacist | 41 (65) | 7 (58) | 0.8 (0.2–3.15, 0.74) | 26 (74) | 1.69 (0.58–4.9, 0.33) | 
                     
                     
                        
                        |  CRCL > 50 mL/min | 410 (78) | 152 (80) | 1.01 (0.65–1.58,0.96) | 560 (83) | 1.28 (0.93–1.76, 0.13) | 
                     
                     
                        
                        |  Abnormal renal function | 46 (49) | 26 (60) | 1.69 (0.72–4) | 36 (61) | 1.92 (0.92–4, 0.08) | 
                     
                     
                        
                        | Dose | 517 (84) | 212 (91) | 
                              1.74 (1.04–2.93, 0.04)
                              [*]
                               | 662 (90) | 
                              1.69 (1.19–2.4, 0.004)
                              [*]
                               | 
                     
                     
                        
                        | Frequency | 527 (85) | 192 (82) | 0.78 (0.51–1.21, 0.27) | 644 (88) | 1.2 (0.85–1.68, 0.3) | 
                     
                     
                        
                        | Total daily dose | 491 (79) | 197 (84) | 1.31 (0.86–2.01, 0.21) | 629 (85) | 
                              1.52 (1.12–2.06, 0.01)[*]
                                 
                               | 
                     
               
               
               
               Abbreviations: APP, advanced practice practitioner (nurse practitioner and physician
                  assistant); CI, confidence interval; CRCL, creatinine clearance; OR, odds ratio.
               
               
               Note: n is the number of orders within guidelines-recommended range; % is the percentage
                  of orders per total number of orders for the study phase.
               
               
               * 
                  p-value < 0.05.
               
                
            
            
            
            Secondary Outcomes
            
            The CDS system showed a statistically significant improvement in the proportion of
               orders with a dose within the recommended range in both phase I and phase II compared
               with the control period, and in the proportion of orders with a total daily dose within
               the recommended range in phase II of the study compared with the control period. There
               was no statistically significant difference in the proportion of orders with a frequency
               within the recommended range in any of the study phases. A subgroup analysis of adherent
               orders with regard to dose and frequency stratified by provider type and by renal
               function category is shown in [Table 3]. We observed a statistically significant improvement in the proportion of adherent
               orders among physicians from the control period to phase II of the study. The increase
               in the proportion of adherent orders with respect to both dose and frequency was more
               apparent in patients with reduced renal function when compared with patients with
               normal renal function (CRCL > 50 mL/min) but this difference was not statistically
               significant ([Table 3], [Fig. 5], [Fig. 6]).
            Discussion
            This study found that provider adherence to guideline-recommended pediatric antimicrobial
               regimens improved following the implementation of a successive CDS system that incorporated
               passive and noninterruptive/interruptive CDS elements during phase II of the study.
               This study also investigated the impact of a passive dose-guidance CDS during phase
               I of the study, but no statistical significance difference was found during this phase
               when compared with the control period.
            Physicians placed majority of the orders (65%). In total, 95% of orders placed by
               physicians were by resident physicians in training. The subgroup analysis based on
               provider type showed even a higher improvement in adherence among physicians in relation
               to other provider types. The difference in adherence for orders placed by nonphysician
               providers between study phases did not reach statistical significance because the
               number of orders placed by them was relatively small. The subgroup analysis based
               on presence of renal impairment showed a trend for larger improvement in adherence
               in orders for patients with impaired renal function compared with those with normal
               renal function. The odds of adherence within the renal impairment subgroup were 40
               and 33% higher than the normal renal function subgroup in phases I and II of the study,
               respectively. This trend was not statistically significant and will require a study
               with larger sample size to confirm.
            Our CDS system provided a novel approach for guiding renal dose adjustments that adhered
               to CDS best practices.[3]
               [4] The CDS system provided real-time guidance at multiple stages of ordering workflow:
               immediately after medication selection, then during order parameter selection, and,
               lastly, following order signing. The CDS system incorporated successive safeguards
               that prevented the user from ordering an inappropriate regimen. Additionally, the
               CDS system was designed to minimize intrusiveness and utilized noninterruptive, in-line
               warnings to reduce downstream interruptive alerting, thereby fitting closely with
               a user's natural ordering workflow. Also, the CDS system provided multiple acceptable,
               patient-specific recommendations, thereby giving users' flexibility when choosing
               a clinically appropriate regimen. Lastly, the CDS system was transparent to its users.
               The passive CDS displayed informational text directly replicated from the institutional
               guidelines, and users could view criterion triggering dose warnings through a “view
               details …” hyperlink embedded within the order composer.
            Our CDS system was able to provide transparent CDS. The passive CDS informational
               text began with a statement describing the renal function criterion (e.g., “Estimated
               CRCL: 24.8 mL/min/1.73 m2 [by Schwartz's formula based on Cr of 1.5]”) followed by the corresponding dosing
               recommendations. By using this strategy, the system was effectively delivering drug
               and patient-specific snippets of the guidelines directly to the user at the time of
               ordering. When in-line dose warnings were triggered, users could also reference dose-checking
               criteria through a “view details …” hyperlink, which displayed the systems dose-checking
               logic based on age, weight, and renal function. The desirability of transparency of
               CDS rules has been discussed in the literature. An evaluation study of a computer-assisted
               management program for antibiotics reported significant reduction of inappropriate
               antibiotic regimens, length of stay, and drug costs. In that study, ordering users
               could view the “Explain Logic ...” for a detailed description of the source of recommendations.
               The study authors discussed that an important strength of their system was the transparency
               by which the system displayed its recommended antimicrobial regimen.[17]
               
            Our successive CDS system has several advantages to other published strategies. We
               attributed these advantages to our strategy of using passive dose guidance instead
               of other noninterruptive strategies such as preselecting order defaults. Chertow et
               al's CDS system improved appropriate medication ordering in a general hospitalized
               patient population functioned by preselecting dose and frequency fields and filtering
               inappropriate choices from selection lists based on calculated renal function.[8] However, their system was limited in that dose and frequency selection logic depended
               on calculated CRCL. Therefore, the system had the potential to populate orders with
               clinically inappropriate doses and/or frequencies for patients undergoing CRRT in
               which a calculated CRCL based on serum creatinine may falsely represent renal function.
               Our CDS system contained logic that recognized the presence of HD or CRRT and therefore
               provided the appropriate CDS regardless of the calculated CRCL. Killelea et al's CDS
               system in a pediatric population preselected default dose and frequency values based
               on medication form, patient age, patient weight, and most common indication. Recommendations
               for alternative regimens (e.g., renal impairment or other indications) were displayed
               as text in a separate window. Provider adherence to the preselected order defaults
               was low (30%) and users of this CDS system frequently deviated from order defaults
               because of the system's limited logic, which could only account for a single indication
               at a time.[18] A distinct advantage in our CDS system was its capacity to recommend multiple dosing
               regimens for different indications and disease severities at one time. Our findings
               in phase I demonstrate that passive CDS improved adherence to the recommended regimens
               in regard to dose selection, although its effect on both dose and frequency showed
               a positive trend that did not reach statistical significance.
            Other institutions have utilized novel dose-checking CDS strategies by generating
               interruptive alerts.[19]
               [20]
               [21] Literature also exists showing that provider adherence to interruptive alerts improves
               if they are exposed to fewer interruptive alerts overall.[9] We did not implement a purely interruptive strategy as that would have added to
               the number of interruptive alerts in our EHR system. The goal of our strategy was
               to reduce the number of interruptive alerts by incorporating upstream passive and
               noninterruptive CDS, and improve the quality of interruptive alerts by making them
               specific to patient's renal function.
            Limitations
            
            The study was a retrospective, observational study design at a single academic medical
               center. Most orders in this setting are placed by practitioners in training, and results
               may not be generalizable to other nonacademic settings where most orders are placed
               by more experienced providers.
            
            The CDS system's design, implementation, and investigation were constrained to antimicrobial
               medications within a specific patient demographic. However, the CDS system's design
               is easily replicable and can be readily expanded to include other types of medications
               and broader patient populations.
            
            The study design assumes that institutional clinical guidelines are optimal for every
               patient, and that increased provider adherence to these guidelines is a desired behavior.
               However, these recommendations may be subject to change as the medical community discovers
               more knowledge about antimicrobial medications, infectious disease states, or the
               affected patient populations. Given this fluidity, 100% provider adherence to dose
               suggestions provided by our CDS system may not be a realistic target.
            
            Our study assessed provider adherence to the institutional dosing guidelines but was
               not powered to assess clinical and economic patient outcomes such as length of stay,
               quality of life, and avoidance of adverse drug events. A much larger study is needed
               to understand the impact of novel CDS tools on patient outcomes.
            
            We were constrained by the implementation timeline of our study. This timeline limited
               the number of orders available for study, especially in phase I. We were also unable
               to conduct a direct comparison between phase I and phase II of the study, and were
               therefore unable to separate the cumulative effect of the CDS system in phase I compared
               with phase II. Future work further investigating the difference in effect between
               noninterruptive and interruptive CDS strategies as they pertain to dose guidance and
               renal function would be beneficial.
            Conclusion
            Provider adherence to the institutional antimicrobial dosing guidelines significantly
               improved following the implementation of a successive CDS system combining passive
               and noninterruptive/interruptive strategies. We describe the value of a passive dose-guidance
               CDS system that functions in real time and provides multiple acceptable, patient-specific
               recommendations concurrently. We also describe the value of designing a successive
               CDS system that supports users through multiple stages of ordering workflow and minimizes
               intrusiveness. The CDS system improved provider adherence to institutional dosing
               guidelines using a blend of CDS strategies and can be considered for implementation
               by other institutions using similar EHR systems.