Examining Perceptions of Computerized Physician Order Entry in a Neonatal Intensive Care Unit

Summary

Background: Computerized provider order entry (CPOE) is a technology with potential to transform care delivery. While CPOE systems have been studied in adult populations, less is known about the implementation of CPOE in the neonatal intensive care unit (NICU) and perceptions of nurses and physicians using the system.

Objective: To examine perceptions of clinicians before and after CPOE implementation in the NICU of a pediatric hospital.

Methods: A cross-sectional survey of clinicians working in a Level III NICU was conducted. The survey was distributed before and after CPOE implementation. Participants were asked about their perception of CPOE on patient care delivery, implementation of the system, and effect on job satisfaction. A qualitative section inquired about additional concerns surrounding implementation. Responses were tabulated and analyzed using the Chi-square test.

Results: The survey was distributed to 158 clinicians with a 47% response rate for pre-implementation and 45% for post-implementation. Clinicians understood why CPOE was implemented, but felt there was incomplete technical training. The expectation for increased job satisfaction and ability to recruit high-quality staff was high. However, there was concern about the ability to deliver appropriate treatments before and after implementation. Physicians were more optimistic about CPOE implementation than nurses who remained concerned that workflow may be altered.

Conclusions: Introducing CPOE is a potentially risky endeavor and must be done carefully to mitigate harm. Although high expectations of the system can be met, it is important to attend to differing expectations among clinicians with varied levels of comfort with technology. Interdisciplinary collaboration is critical in planning a functioning CPOE to ensure that efficient workflow is maintained and appropriate supports for individuals with a lower degree of technical literacy is available.

Citation: Beam KS, Cardoso M, Sweeney M, Binney G, Weingart SN. Examining perceptions of computerized physician order entry in a neonatal intensive care unit. Appl Clin Inform 2017; 8: 337–347 https://doi.org/10.4338/ACI-2016-09-RA-0153

Keywords

Implementation and deployment - electronic health records and systems - clinical decision support - inpatient - inpatient CPOE - neonatology

1. Background and Significance

Computerized physician/provider order entry (CPOE) is a technology that has the potential to transform care delivery. Several prior studies have examined the effects of CPOE on medication safety and the delivery of safe and effective patient care [[1]–[3]] with varying results. Studies in pediatric and neonatology settings have generally shown no decrement in medication safety with the implementation of CPOE, but that the addition of clinical pharmacists has a greater effect on preventing medication errors [[4], [5]]. In addition, the introduction of CPOE has not been found to adversely affect the time clinicians spend with patients [[6]]. One study examining workflow with CPOE implementation within a neonatal intensive care unit (NICU) found that time to antibiotic administration did not improve, but time for pharmacy verification did improve [[3]].

While the introduction of CPOE systems has been studied extensively in adult populations, less is known about the implementation of CPOE in the NICU and its effects on the individuals using the system. Given the vulnerability of this patient population, and the limited ability to extrapolate from adult-focused CPOE systems, implementation of CPOE in this setting merits study. One prior study specifically looked at physician and nurse perceptions of CPOE implementation on hospital workflow, however, the effects on providers caring for neonates were not fully examined [[7]]. In our institution there was an opportunity to examine clinician perceptions and nursing workflow surrounding CPOE implementation before and after the system development and roll out within a Level III NICU.

The objective of this report is to examine the perceptions of clinicians before and after CPOE implementation in the NICU of a pediatric hospital. We hypothesize that perceptions of CPOE will be more positive after implementation of the system particularly related to medication safety and efficiency. We also hypothesize that nursing staff will have different perceptions of CPOE implementation than physicians given the different roles in delivering care.

2. Methods

2.1 Setting

This study was performed in a 40-bed, Level III NICU within a 415-bed urban teaching hospital in Boston, MA. The NICU is located within an adult hospital with an attached pediatric hospital rather than a free-standing children’s hospital. Prior to CPOE implementation in the NICU, all physician/ provider orders in the unit, including nursing, medication, diet, laboratory, radiology, respiratory, and ancillary testing orders, were written on paper at the bedside of each patient. Nurses then would acknowledge the order on paper and distribute the order to the appropriate area for further processing. For example, nursing orders were transcribed onto nursing flow sheets or assessment forms, medication orders were faxed to pharmacy, diet orders were faxed daily to formula preparation room, laboratory and radiology orders were entered into computer system by nurses or unit coordinators, and respiratory orders were verbally transmitted to respiratory technicians. CPOE had previously been implemented in the hospital to varying degrees with adult inpatient units having full CPOE including medication orders and other pediatric units having partial CPOE including only non-medication orders (admission and discharge orders). The NICU was the first full CPOE initiative within the pediatric hospital. Implementation of CPOE in the NICU had been delayed to ensure that CPOE implementation would be sure to address complexities and safety concerns that are unique to this vulnerable population, especially the need to be able to address weight-based and ges-tational age dosing of medications.

2.2 Intervention

CPOE was implemented in the NICU in March 2015 after a 2-year development period using the Siemens (now Cerner after February 2, 2015) Soarian CPOE platform. From March 2013 – March 2015, interprofessional teams consisting of neonatologists, pediatric subspecialists, nurses, clinical and informatics pharmacists, and information technologists systematically worked together to customize the Soarian CPOE product for use in the NICU. These teams reviewed the existing CPOE product in use and determined which orders needed to be included, revised, and added for use in the NICU. The IT implementation team had worked on the adult go-live previously and was able to provide guidance on the limitations of the system for pediatrics and things that had previously been successful. The vast majority of the work involved reviewing over 200 medications that had been previously prescribed or had the potential to be prescribed within the NICU. Each medication was reviewed for neonatal dosing guidelines and decision support prior to building a medication-specific order set. In the NICU pre-implementation, all orders for medications based upon weight had to include weight, gestational age and post-menstrual age, and nursing and pharmacy would use that information to check each order manually. The CPOE product had to be customized to allow inclusion of this information. Since weight change in NICU patients can have profound effects on dosing, pre-implementation all NICU medication orders had to be re-written weekly based upon the patient’s new weight to ensure that medication dosing would remain appropriate as weight varied over time. This “re-write” policy was one of the major issues that the NICU CPOE build team addressed during the customization of the CPOE product that did not need to be addressed by other units in the hospital. The team created a “medication dosing weight” order, which is updated weekly and creates a dosing weight that flows into all subsequently placed medication orders. Concurrently, medication administration processes were reviewed and adapted in order to ensure safe and efficient medication preparation and administration. Post-implementation, the weekly revision of the “medication dosing weight” and all weight-based medications has replaced the pre-implementation weekly “re-write” policy.

In addition to reviewing, revising or creating individual orders in the CPOE build, the teams then created NICU-specific order sets designed to ensure smoother work flow in CPOE and to guide clinical practice by including decision support in those order sets. Following completion of the CPOE build, all physicians (including residents, fellows, and attendings), mid-level providers, nurses and pharmacists completed two- to four-hour training sessions given in the 5 months prior to implementation. Physicians and nurses received CPOE training that was developed by IT and the Pediatric CPOE MD lead. They were held on a rolling basis, for specific disciplines starting from 4 months prior to implementation to a few weeks after (for residents unable to make previous sessions). During the implementation the IT team set up a station within the unit for the first week to address any issues on-site. Physicians, pharmacists, and nurses were on pager call and rounded with the teams to evaluate progress. After 2 weeks when no additional issues were identified, the unit-based team was relocated to the IT department.

A hospital-based pediatrician was designated the physician lead for CPOE. When the NICU implementation started, two neonatologists were part of the review group. The implementation team and other clinicians who took part in the CPOE planning and judgment process were excluded from the survey to avoid bias.

2.3 Evaluation

We conducted a survey to assess clinicians’ attitudes, expectations, and experience with CPOE. We used a before-and-after cross-sectional study design with two rounds of written survey data collection: round 1 occurred six months pre-CPOE implementation and round 2 occurred 1 year post-CPOE implementation. Time periods for survey distribution were selected for logistical reasons and also to allow time for clinicians to adjust to the new system. Potential respondents included registered nurses, nurse practitioners, physicians (attending physicians, fellows and residents), and unit coordinators working in the NICU. Though other clinicians had generally worked in the NICU for several years, residents’ perceptions of CPOE implementation were still highly valued, as most residents have significant experience with electronic documentation and ordering during medical school and in non-NICU clinical rotations. All individuals were asked to fill out the same survey for both rounds of data collection. Both physicians and nurses use the CPOE application technology to enter orders, and verify order status, document notes, and update patient status.

The survey was adapted from two prior surveys developed by Wakefield and Careyon [[8]]. The instrument elicited demographic information including job title, years of experience, and typical shift worked in the NICU. The survey also included three sections evaluating perceptions (better, worse, and no change) of CPOE implementation on ability to deliver patient care, overall implementation of CPOE, and effect on job satisfaction (►Supplementary Online Material). Finally, there was an open-ended response section inquiring about any additional thoughts or concerns surrounding the implementation of CPOE. This survey was determined to have face validity based on reviews by a health services researcher as well as unit-based clinicians.

Prior to implementation, paper questionnaires were distributed to nurses and physicians at work in the NICU. Paper surveys were chosen in order to ensure have rapid turn around and confirmation of survey completion. Respondents returned completed surveys to a research assistant. The questionnaires were filled out anonymously. Post-implementation surveys were distributed electronically with a link sent to the institutional email of each potential respondent for ease of distribution. Reminder emails were sent to increase response rate. The evaluation was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects. Participation was voluntary and the study was deemed exempt from review as a quality improvement project by the Institutional Review Board at Tufts Medical Center.

2.4 Data Analysis

We tabulated respondents’ assessments of the impact of implementing CPOE, attitudes toward implementation, and the effect of CPOE implementation on patient care delivery. We compared physician and nurse assessments, attitudes, and effects on patient care delivery based on pre-CPOE implementation and post-CPOE implementation, using the Chi-square test for greater quantities of responses. Analyses were performed using Stata 9.0 (StataCorp, College Station, TX).

3. Results

3.1 Respondent characteristics

A total of 77 eligible respondents completed the questionnaire pre-implementation (47% response rate) and 73 eligible respondents completed the questionnaire post-implementation (45% rate). Pre-implementation, registered nurses represented 59% of the respondents and physicians including nurse practitioners, residents, fellows and attending physicians represented 41%. In post-implementation registered nurses represented 53% and physicians represented 47% (►[Table 1]).

Table 1

Respondent Characteristics

Professional Group	Pre-Implementation N (%)	Post-Implementation N (%)
Nurse	46 (60)	40 (55)
Resident/Fellow	19 (25)	26 (37)
Attending Physician	12 (16)	7 (10)
Total Surveys	77	73
Professional Group	Hours Worked per Week in NICU Mean ± SD
Resident/Fellow	66.1 ± 2.6
Attending Physician	24 ± 9.3

3.2 Comparison of Responses

Prior to implementation, most physicians and nurses agreed that they understood reasons why CPOE was implemented (86%). However, almost one quarter of all respondents felt that they did not receive sufficient training (27%) and that learning CPOE would be difficult (22%). Pre-implementation, one respondent noted, “(There was) too much time between when I learned how to use (CPOE) and when I will actually start. (I) need more training.” Another respondent stated, “This is going to be difficult to learn.” After implementation, respondents continued to indicate that they understood why CPOE was implemented (97%). There was a statistically significant difference between pre-implementation and post-implementation responses regarding perceptions of information technology support; 35% of individuals anticipated that support would be available, but 55% experienced adequate support after implementation (p < 0.001). Individuals also indicated that training was sufficient post-implementation, although there was concern pre-implementation (67%, p < 0.001). Respondents reported that learning to operate CPOE was less difficult in the post-implementation assessment than they had reported at baseline (83%, p < 0.001) ([Table 2]). One individual stated post-implementation that CPOE was “…very necessary in all patient care areas. Streamlines some of the paperwork, reduces human error, allows quick access to needed information.”

Table 2

Selected Pre- and Post-Implementation Responses, Stratified by Physicians and Nurses

Survey Item	Physicians’			Nurses’
	Pre (N = 31)	Post (N = 33)	P-value [*]	Pre (N = 46)	Post (N =40)	P-value [*]
Job satisfaction
Improved	14 (45)	23 (70)	0.06	2 (4)	13 (33)	0.002
No change	17 (55)	9 (27)		26 (57)	19 (48)
Worsened	0 (0)	1 (3)		18 (39)	8 (20)
The hospital’s ability to recruit and retain high-quality staff
Improved	18 (58)	11 (34)	0.06	10 (22)	3 (8)	0.01
No change	13 (42)	21 (66)		36 (18)	32 (80)
Worsened	0 (0)	0 (0)		0 (0)	5 (13)
Overall patient safety
Improved	28 (90)	26 (81)	0.22	25 (54)	24 (60)	0.87
No change	2 (7)	6 (19)		16 (35)	12 (30)
Worsened	1 (3)	0 (0)		5 (11)	4 (10)
Ability to input correct medication orders without needing to call pharmacy for corrections
Improved	23 (74)	30 (91)	0.15	24 (52)	22 (55)	0.34
No change	6 (19)	3 (9)		18 (39)	11 (28)
Worsened	2 (7)	0 (0)		4 (9)	7 (18)
Since CPOE will be/ was implemented, please indicate your level of agreement:
Learning to operate CPOE was not very difficult
Agree	18 (58)	29 (88)	0.03	21 (46)	31 (80)	0.006
Neutral	7 (23)	2 (6)		14 (30)	5 (13)
Disagree	6 (19)	2 (6)		11 (24)	3 (8)
I understand why CPOE was implemented
Agree	30 (97)	32 (97)	0.96	36 (78)	39 (98)	0.03
Neutral	1 (3)	1 (3)		9 (20)	1 (3)
Disagree	0 (0)	0 (0)		1 (2)	0 (0)
I received sufficient training to learn CPOE
Agree	10 (32)	22 (67)	0.02	11 (24)	27 (68)	<0.001
Neutral	14 (45)	7 (21)		22 (48)	10 (25)
Disagree	7 (23)	4 (12)		13 (28)	3 (8)
IT support has always been available when I need help with CPOE
Agree	11 (36)	22 (67)	0.04	16 (35)	18 (45)	0.05
Neutral	18 (58)	10 (30)		17 (37)	19 (48)
Disagree	2 (7)	1 (3)		13 (28)	3 (8)
CPOE will improve/ has improved my ability to:
Give correct treatments
Agree	24 (77)	25 (73)	0.61	28 (61)	25 (63)	0.01
Neutral	7 (23)	8 (24)		18 (39)	9 (23)
Disagree	0 (0)	1 (3)		0 (0)	6 (15)
Treat the correct patient
Agree	22 (71)	20 (61)	0.21	30 (65)	29 (73)	0.03
Neutral	9 (29)	10 (30)		16 (35)	7 (18)
Disagree	0 (0)	3 (9)		0 (0)	4 (10)
Treat at the correct time
Agree	23 (74)	21 (64)	0.21	27 (59)	24 (62)	0.001
Neutral	8 (26)	9 (27)		19 (41)	7 (18)
Disagree	0 (0)	3 (9)		0 (0)	8 (21)
Give the correct amount, dose, or intensity
Agree	26 (84)	28 (85)	0.57	29 (63)	30 (75)	0.004
Neutral	5 (16)	4 (12)		17 (37)	5 (13)
Disagree	0 (0)	1 (3)		0 (0)	5 (13)

*Chi-square statistic CPOE: Computerized Physician Order Entry System NICU: Neonatal Intensive Care Unit

Individuals reported that NICU CPOE increased job satisfaction (21% to 49%), but there was a decrease in the ability to recruit and retain high quality staff (36% to 19%). Further investigation into these results revealed conflicting outcomes. The majority of pre-implementation respondents did not expect CPOE to impact job satisfaction (56% expected no change and 23% expected job satisfaction to worsen), but post-implementation respondents were much more likely to report that CPOE improved job satisfaction (49% reported improvement and 38% reported no change, 12% reported worsening satisfaction). Similarly, a majority of pre-implementation respondents did not expect CPOE to improve retention and attraction of high-quality staff; all respondents expected CPOE to have no change or a positive impact (64% expected no change and 36% expected improvement). Post-implementation, however, respondents were less likely to report beneficial impact (74% reported no change, 19% reported improvement, and 7% reported decline). When these results were further stratified by provider role (►[Table 2]), the increase in job satisfaction remained for both nurses (4% to 35%) and physicians (45% to 70%). While physicians were split on whether CPOE would lessen the time they had for patient care, the majority of nurses (77%) felt it would have a negative impact. Nurses and physicians both reported a decrease in ability to recruit and maintain high quality staff.

Examining respondents’ perceptions of the likely and actual impact of CPOE on clinical care, expectations were high and generally positive for items including the appropriateness of patient care orders, the ability to alert staff to order entry errors before they occur, and overall patient safety. Before and after implementation, most respondents agreed that CPOE would improve the ability to deliver correct treatments (68% vs 67%) to the correct patient (68% vs 67%) at the correct time (65% vs 63%). Pre-implementation, one respondent stated, “There will still be (a lot) of room for human error.” Another respondent added, “…There are some aspects I love about CPOE: more legible and easily accessible. Once everything is computerized it will be great.”

3.3 Differences by Professional Group

At baseline, more physicians than nurses expected that CPOE would improve job satisfaction (45% vs 5%, p < 0.001) (►[Table 2]). This difference persisted in post-implementation results. However, both groups reported a dramatic increase in job satisfaction (70% vs 35%, p = 0.01). In post-implementation surveys, more physicians than nurses perceived that there would be improvement in the need to call the pharmacy to input correct medication orders (91% vs 55%, p = 0.002). Finally, a small group of clinicians disagreed with the ability of CPOE to help deliver safe treatments. A breakdown of these individuals revealed that a higher percentage were nurses rather than physicians.

4. Discussion

4.1. Main findings

In evaluating the implementation of a CPOE system in a NICU, we found that expectations regarding CPOE implementation were high and were generally satisfied. Physicians had higher expectations than nurses for improving job satisfaction, recruiting high quality staff members, and improving overall patient safety. However, after implementation the impact of CPOE on job satisfaction exceeded expectations for both nurses and physicians. Nevertheless, a small group of clinicians expressed persistent concerns about the potential adverse impact of CPOE on their ability to deliver the correct treatments.

4.2 Perceptions among nurses and physicians

Nurses and physicians were noted to have different attitudes before and after CPOE implementation. Nurses appeared to be more cautious at baseline, but appreciated the benefits of CPOE post-implementation. Physicians were more optimistic from the outset and this did not change between pre-implementation and post-implementation responses. We identified a small group of respondents who remained concerned about the implementation of CPOE, which included more nurses than physicians. The differences in these perceptions are likely multifactorial. Many comments from nurses and physicians referenced technical computer challenges rather than workflow issues. We speculate that physician respondents, who included residents and fellow trainees, were in practice for a shorter amount of time on average, have more experience with electronic health record systems, and may be more familiar with incorporating new technologies into practice. Experienced nurses, in comparison, were astute observers of potential patient care risks.

It is possible that electronic order entry systems have a more disruptive effect on nursing workflow than on physician practice and that this may influence their enthusiasm for CPOE. For example, prior to CPOE implementation physicians typically wrote orders at the bedside and sent paper orders to pharmacy using a tube system. The pharmacist would enter the medication order and the nurse would then administer the medication. After CPOE implementation, the physician enters the medication order on a computer, which can occur remotely and not necessarily at the bedside of the patient, and the nurse must verify the order before being sent to pharmacy. The pharmacist then confirms the order and will typically talk with the nurse regarding any discrepancies or timing issues. The physician is usually contacted as well regarding dosing discrepancies. This process generally improves workflow for physicians, allowing for more timely order entry and the ability to enter orders while at the bedside of another patient, however, requires nurses to have a computer at each bedside to verify and carry out orders. Whereas prior to implementation the majority of the order workload was on physicians, after implementation the workload shifted to nursing staff and likely contributed to the difference in attitudes between the two groups surrounding CPOE implementation.

The NICU is a challenging environment to implement CPOE given the vulnerability of its patients, the small doses of medications ordered and administered, the need to titrate treatment to daily and weekly changes in body weight. Neonatal weight changes on a daily basis both with weight gain and loss and medications are dosed according to weight. Some frequently used medications can come in very small doses meaning that a 30 gram weight gain in one day (appropriate weight gain for a neonate) can change the dose by as much as 10%. Typically when an infant loses weight, the medication continues to be dosed according to the highest recorded weight of the infant. However, as the infant gains weight, they effectively “grow out of “ their medication dose and the medication must be adjusted in order to remain effective. The frequent weight changes create a new challenge in CPOE that is not encountered in adult populations and creates potential for medication errors that may not exist in adult populations [[9], [10]]. In our unit, this had been addressed by updating weights weekly to ensure adequate and appropriate dosing.

Like our respondents, Ayatollahi et al. found that physicians and nurses anticipated that CPOE implementation would have a positive impact on patient safety, but there were significant differences among nurse and physician assessments of the likely impact on NICU workflow: nurses were more optimistic than physicians [[7], [11]]. There continues to be discrepant results among studies regarding the ease of use and efficiency associated with CPOE implementation [[12]].

Our study found a small group of individuals who continued to express concern about the effect of CPOE on patient safety, particularly related to treating the right patient at the right time. This is concerning given that one goal of CPOE is to reduce the incidence of medication errors and to improve overall patient safety. However, several studies have examined increased incidence of medication errors related to CPOE implementation [[13]–[15]] and one study has even found an increase in mortality [[16]]. When looking at the implementation of a single CPOE system across three different institutions there are very different effects on mortality with increased, no change, and decreased mortality [[15]–[17]]. Implementation presents certain risks that may be mitigated once end-users have experience with the system. In our institution, an IT group was present to assist the end-users in gaining adequate experience with the system. A large majority of the nursing staff expressed concern that they did not receive enough training pre-implementation to be able to use the system effectively. After implementation, most individuals including nursing staff felt that they were able to use the system effectively for patient care. There is a possible baseline comfort among medical professionals with computer technology in that younger physicians are more likely to use technology seamlessly as it has been part of the daily life during education and clinical practice from the beginning.

Regarding patient safety, the likely effects of the CPOE system could be related to the ability to enter orders remotely. There is room for mistakes when not looking at the patient for which orders are being entered. Additionally, there is an increased need for vigilance when entering orders to ensure that the medication or nursing orders being entered are for the correct patient. Nursing staff checking over orders typically identifies when orders do not seem appropriate for their particular patient, however this may not always be the case. If increased vigilance is not implemented on both the physician and nursing aspect of patient care, there is increased potential for mistakes and decreased patient safety. In most studies there are less medication errors if a clinical pharmacist is also present during rounds to answer questions and double check orders at the time of entry. This would be an interesting aspect to investigate in a future study.

We should understand if the perceived risks on patient safety are valid and if they can be attenuated by improved CPOE functionality or training. While it is possible that there are risks inherent to the introduction of new technology risks may have less to do with inherent attributes of the technology than its precise and thoughtful implementation. Several studies in Pediatrics described implementation of the same electronic medical record and CPOE, but showed different outcomes on morbidity and mortality in each setting; both increased and decreased risk of mortality [[17], [18]]. This potentially represents new vulnerabilities with electronic medical records representing the need for more cohesive implementation decisions. We need to explore how to assess and improve IT-literacy among clinicians in anticipation of introduction of CPOE in novel settings.

4.3 Limitations

There are several limitations of this study, including its relatively small sample size, customized CPOE application, and use of a single institution. The results of this study may not be generalizable to another neonatal intensive care unit, although the lessons learned here and implications for further electronic order entry systems can be considered in future CPOE development. In order to determine whether caregivers’ perceptions after CPOE implementation are influenced by systematic malfunctions, future studies should record and report successes and failures associated with CPOE systems. It was also difficult to compare before-and-after assessments at the individual clinician level, as staff turnover precluded effective matching. Finally, there may be a response bias as questionnaires were distributed during working hours and only about half of all individuals who received the questionnaire during each study period returned the questionnaires. Therefore, not all clinicians working in the NICU filled out the survey if they were not present at any of the times that surveys were distributed.

5. Conclusions

Introducing CPOE is a potentially risky endeavor and must be done carefully to mitigate harm. Although high expectations for the impact of the system can be met, it is important to attend to differing expectations among clinicians and different levels of comfort with technology and change. The perceptions of the effect on workflow depending on job position (nurse versus physician) can vary widely and should be accounted for in the introduction of a new electronic order entry system. Adequate IT support and appropriately equipped computers should also be available. Interprofessional collaboration is necessary in planning an effective CPOE launch. In addition, there are particularly vulnerable populations, particularly the neonatal and pediatric populations, which require special attention to detail throughout the implementation process in order to ensure maximal patient safety.

Question

Implementation of electronic medical record systems including computerized provider order entry (CPOE) requires interdisciplinary communication and agreement for effective and safe implementation. Nurses and physicians use electronic medical record systems in different ways during their daily workflow. Which areas do physicians find are most affected by implementation of CPOE?

• Overall Patient Safety

• Treating at the correct time

• Treating the correct patient

• Job Satisfaction

– Answer: D, Job Satisfaction

Based on our survey of physicians and nurses during the time period surrounding the implementation of CPOE in a single institution, overall physicians did not feel that the CPOE system affected patient safety or treatment methods and times. However, physician job satisfaction appeared to be improved post-implementation. Comparatively, nurses found more of a change in patient safety and treatment times after implementation of CPOE. This is likely due to workflow and how both physicians and nurses use the CPOE in workflow. Typically nursing is at the bedside of an individual patient and often requires close interaction with CPOE during times of patient care. These are situations a physician may not directly see.

Clinical Relevance Statement

Computerized order entry may produce safe and effective patient care. While many studies have examined the effects on patient care and medication administration, the perceptions of individuals using the system every day have not been fully understood. Specific considerations for neonatal order entry and the individuals utilizing the system should be considered to ensure continued effective use of the system.

Conflicts of Interest

The authors declare that they have no conflicts of interest in the research

Acknowledgements

The authors would like to thank the Tufts Medical Center Clinical Informatics Team for developing and implementing CPOE throughout the medical center.

Protection of Human and Animal Subjects

The evaluation was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects. Participation was voluntary and the study was deemed exempt from review as a quality improvement project by the Institutional Review Board at Tufts Medical Center.

• References

• 1 Manias E, Kinney S, Cranswick N, Williams A, Borrott N. Interventions to reduce medication errors in pediatric intensive care. Ann Pharmacother 2014; 48 (10) 1313-1331. doi: 10.1177/1060028014543795. PubMed PMID: 25059205.
• 2 Cartmill RS, Walker JM, Blosky MA, Brown RL, Djurkovic S, Dunham DB, Gardill D, Haupt MT, Parry D, Wetterneck TB, Wood KE, Carayon P. Impact of electronic order management on the timeliness of antibiotic administration in critical care patients. Int J Med Inform 2012; 81 (11) 782-791. doi: 10.1016/j.ijmedinf.2012.07.011. PubMed PMID: 22947701.
• 3 Chapman AK, Lehmann CU, Donohue PK, Aucott SW. Implementation of computerized provider order entry in a neonatal intensive care unit: Impact on admission workflow. Int J Med Inform 2012; 81 (05) 291-295. doi: 10.1016/j.ijmedinf.2011.12.006. PubMed PMID: 22226728.
• 4 Chedoe I, Molendijk HA, Dittrich ST, Jansman FG, Harting JW, Brouwers JR, Taxis K. Incidence and nature of medication errors in neonatal intensive care with strategies to improve safety: a review of the current literature. Drug Saf 2007; 30 (06) 503-513. PubMed PMID: 17536876.
• 5 Wang JK, Herzog NS, Kaushal R, Park C, Mochizuki C, Weingarten SR. Prevention of pediatric medication errors by hospital pharmacists and the potential benefit of computerized physician order entry. Pediatrics 2007; 119 (01) e77-e85. doi: 10.1542/peds.2006–0034. PubMed PMID: 17200262.
• 6 Westbrook JI, Li L, Georgiou A, Paoloni R, Cullen J. Impact of an electronic medication management system on hospital doctors’ and nurses’ work: a controlled pre-post, time and motion study. J Am Med Inform Assoc 2013; 20 (06) 1150-1158. doi: 10.1136/amiajnl-2012–001414. PubMed PMID: 23715803; PubMed Central PMCID: PMCPMC3822109.
• 7 Ayatollahi H, Roozbehi M, Haghani H. Physicians’ and Nurses’ Opinions about the Impact of a Computerized Provider Order Entry System on Their Workflow. Perspect Health Inf Manag. 2015 12: 1g. PubMed PMID: 26604876; PubMed Central PMCID: PMCPMC4632876.
• 8 Wakefield DS, Halbesleben JR, Ward MM, Qiu Q, Brokel J, Crandall D. Development of a measure of clinical information systems expectations and experiences. Med Care 2007; 45 (09) 884-890. doi: 10.1097/MLR.0b013e3180653625. PubMed PMID: 17712259.
• 9 Chuo J, Hicks RW. Computer-related medication errors in neonatal intensive care units. Clin Perinatol 2008; 35 (Suppl. 01) 119-139 ix. doi: 10.1016/j.clp.2007.11.005. PubMed PMID: 18280879.
• 10 Kunac DL, Reith DM. Identification of priorities for medication safety in neonatal intensive care. Drug Saf 2005; 28 (03) 251-261. PubMed PMID: 15733029.
• 11 Cordero L, Kuehn L, Kumar RR, Mekhjian HS. Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. J Perinatol 2004; 24 (02) 88-93. doi: 10.1038/sj.jp.7211000. PubMed PMID: 14872207.
• 12 Khajouei R, Wierenga PC, Hasman A, Jaspers MW. Clinicians satisfaction with CPOE ease of use and effect on clinicians’ workflow, efficiency and medication safety. Int J Med Inform 2011; 80 (05) 297-309. doi: 10.1016/j.ijmedinf.2011.02.009. PubMed PMID: 21419695.
• 13 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163 (12) 1409-1416. doi: 10.1001/archinte.163.12.1409. PubMed PMID: 12824090.
• 14 Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, Strom BL. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-1203. doi: 10.1001/jama.293.10.1197. PubMed PMID: 15755942.
• 15 Kim GR, Miller MR, Ardolino MA, Smith JE, Lee DC, Lehmann CU. Capture and classification of problems during CPOE deployment in an academic pediatric center. AMIA Annu Symp Proc. 2007: 414-417. PubMed PMID: 18693869; PubMed Central PMCID: PMCPMC2655829.
• 16 Keene A, Ashton L, Shure D, Napoleone D, Katyal C, Bellin E. Mortality before and after initiation of a computerized physician order entry system in a critically ill pediatric population. Pediatr Crit Care Med 2007; 8 (03) 268-271. doi: 10.1097/01.PCC.0000260781.78277.D9. PubMed PMID: 17417119.
• 17 Longhurst CA, Parast L, Sandborg CI, Widen E, Sullivan J, Hahn JS, Dawes CG, Sharek PJ. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics 2010; 126 (01) 14-21. doi: 10.1542/peds.2009–3271. PubMed PMID: 20439590.
• 18 Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, Bayir H, Orr RA. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; 116 (06) 1506-1512. Erratum in: Pediatrics 2006; 117(2)594. PubMed PMID: 16322178.