Going Mobile: Resident Physicians' Assessment of the Impact of Tablet Computers on Clinical Tasks, Job Satisfaction, and Quality of Care

Abstract

Background There are few published studies of the use of portable or handheld computers in health care, but these devices have the potential to transform multiple aspects of clinical teaching and practice.

Objective This article assesses resident physicians' perceptions and experiences with tablet computers before and after the introduction of these devices.

Methods We surveyed 49 resident physicians from 8 neurology, surgery, and internal medicine clinical services before and after the introduction of tablet computers at a 415-bed Boston teaching hospital. The surveys queried respondents about their assessment of tablet computers, including the perceived impact of tablets on clinical tasks, job satisfaction, time spent at work, and quality of patient care.

Results Respondents reported that it was easier (73%) and faster (70%) to use a tablet computer than to search for an available desktop. Tablets were useful for reviewing data, writing notes, and entering orders. Respondents indicated that tablet computers increased their job satisfaction (84%), reduced the amount of time spent in the hospital (51%), and improved the quality of care (65%).

Conclusion The introduction of tablet computers enhanced resident physicians' perceptions of efficiency, effectiveness, and job satisfaction. Investments in this technology are warranted.

Keywords

tablet computers - interns and residents - information technology - health care informatics

Background and Significance

While the introduction of hospital information technology (IT) relied on desktop devices and traveling workstations, there is increasing interest in the use of Health Insurance Portability and Accountability Act of 1996-compliant portable and mobile computers to deliver care in a diverse array of practice settings. There are few published studies of the use of portable or handheld computers in health care, but these devices have the potential to transform multiple aspects of clinical teaching and practice.[1] [2] [3] [4] [5] [6] For example, tablet computers have been shown to be an effective learning tool for medical students and residents.[7] [8] [9] [10] [11] [12] [13] [14] [15] [16] Mobile computers may offer practical advantages in alleviating congestion in clinical care areas where physicians, other health care providers, students, and administrative staff compete for vacant devices.[17] [18] Mobile technology could potentially improve the convenience of order-writing, laboratory monitoring, and access to reference materials and decision aids, with downstream benefits in terms of quality of care, efficiency, and clinician satisfaction.

Objective

Few studies to date have assessed resident physicians' perceptions of the benefits of mobile tablet technology for clinical care across inpatient services.[19] Understanding residents' perceptions about the value of tablet computers and their experiences using these devices to support clinical care could inform hospitals' decisions to purchase and deploy tablet computers, and the devices' likely acceptance and use. Therefore, we conducted a pilot project to examine residents' perceptions and experiences of using mobile tablet computers at a teaching hospital before and after the introduction of these devices. We hypothesized that residents would readily adopt and use this technology, and that their expectations would match their experience in improving resident job satisfaction, reducing time spent in the hospital, increasing quality of care, and expediting clinical tasks.

Methods

Setting and Participants

This project was conducted on seven adult general medical-surgical wards and the medical intensive care unit (MICU) of a 415-bed academic medical center in downtown Boston serving a diverse patient population. Each of the participating clinical services was staffed by 1 to 2 postgraduate year (PGY)-1 interns, a PGY-2 or -3 resident, a PGY-4 resident or medical student in some cases, and an attending physician. Residents rotated among subspecialty services, generally at 2- to 3-week intervals, and were assigned to specific wards during daytime and overnight shifts. Resident physicians were responsible for conducting morning work rounds, tracking test results, entering patient orders, ordering consultations, communicating updates to the attending physicians, and educating less-experienced trainees and students. Adult medical-surgical and intensive care unit teams used the Soarian (Cerner, Kansas City, Missouri, United States) electronic health record for inpatient care.

Device Selection and Preparation

With financial support from the office of graduate medical education and the hospital's information services department, we purchased and deployed 13 tablet computers to 8 clinical services from November 13, 2015 to July 1, 2016. In anticipation of the deployment, we identified technical requirements to support Soarian clinical applications, Microsoft Office tools, and medical references in discussions with hospital IT staff. Soarian compatibility requirements included Intel 2.0 GHz processor or equivalent, 512 MB RAM, Microsoft Windows, Internet Explorer 6.0, Adobe Acrobat Reader v8, and Sun Java Virtual Machine Version 1.5. A secure, encrypted wireless network available throughout the hospital connected to the devices automatically. Resident physicians piloted several candidate devices including the Dell Venue Pro 8, Microsoft Surface Pro 2, ASUS MeMo Pad 7, and Lenovo Yoga Pro.

After pilot-testing alternative tablet devices, we surveyed 29 users about their perceptions and experiences. Residents considered the size of the device as an important selection criterion, and were particularly interested in tablets that could fit in a white coat pocket. However, they ultimately preferred larger devices that were lightweight, had a large enough screen to view full electronic medical record (EMR) “pages” and radiographic images, and offered multitouch screen capability, extended battery life, and virtual keypad functionality. However, processing speed was the feature consistently rated most important to users. We conferred with the hospital's infection control practitioners to confirm that the external surface, keyboard, and touch screen of the preferred device could be disinfected satisfactorily using the hospital's existing inventory of antiseptic wipes. After further review of pricing and mobility, we purchased 13 Lenovo Yoga Pro ThinkPads (Morrisville, North Carolina, United States). This device featured high processing speed, touch screen capability, use as either a tablet or a notebook, 4GB memory, midrange cost, and 2.4 pound weight. Although we used the term “tablet computer” consistently throughout the project, this mobile device is more accurately characterized as a light laptop computer with a touch screen and full operating system.

All 13 tablets connected to the hospital's secure wireless network and included Internet Explorer, Soarian (Cerner) EMR and computerized order-entry system software, links to the hospital's Intranet and electronic paging directory, Microsoft Outlook Webmail, and the institutional subscription to UpToDate. Authenticated software ensured that only hospital staff could access these programs and unlock the tablets. Device tracking was enabled from the IT department's central console if necessary. All devices were password-protected and encrypted for security.

Allocation of Tablets

The medicine department is organized into eight subspecialty inpatient services, and admitted patients are assigned to the service that most closely matches their admission diagnosis. One tablet was allocated to each of the two PGY-1 residents on the nephrology and general medicine services, two tablets to the PGY-2 residents who served as night float (covering several medicine subspecialties), and another tablet was provided to a PGY-3 resident in the MICU. We allocated 4 tablets to surgical residents, including 1 surgical oncology intern (PGY-1), 1 colorectal or thoracic surgery intern (PGY-1), 1 vascular surgery intern (PGY-1), and 1 consult resident (PGY-3) on general surgery. We also provided 2 tablets to the neurology team. One tablet was shared among neurology junior residents (PGY-3) and the other was shared among neurology senior residents (PGY-4). The distribution plan was intended to evaluate tablet computer use and utility during a variety of shifts (full day, night float, on-call) and among teams with geographically dispersed patients.

Survey Development

We searched the medical literature for a survey instrument that could be used to assess the impact of tablet computers on clinical care and resident efficiency. Because we found no suitable instrument in the literature, we developed surveys in an iterative fashion based on input from IT staff, medical educators, attending physicians, and medical and surgical residents.

The baseline survey asked respondents to identify tasks for which they believed that tablet computers would be most useful (i.e., entering orders, writing notes, reviewing data, signing out, accessing medical references, or other tasks). It also elicited respondents' assessments of whether tablets would affect their residency program ranking (improved, no change, decreased), improved their job satisfaction (yes, no), total time spent in the hospital (decrease, no change, increase), and the quality of care they delivered (improve, no change, worsen). The survey elicited limited personal information to ensure respondent confidentiality (resident PGY-year, hospital service), a self-assessment of their IT aptitude, and whether respondents owned or used a personal tablet or smartphone. The follow-up survey asked respondents to rate their experience using the tablets by assessing whether the tablets were helpful for clinical care (yes, no), to assess where the residents found the devices most useful (on the wards, in conference, out of the hospital, in-hospital on call, out of hospital on call), and to identify the most valuable applications (entering orders, writing notes, reviewing data, signing out, accessing medical references, other tasks). Respondents were asked also to assess the ease and speed of the tablet computer to a standard desktop workstation for tasks such as searching for an available computer, viewing medical information, writing notes, and entering orders using a 5-point Likert scale. Copies of the instruments are provided in the Appendix ([Supplementary Material], available in the online version).

Project Protocol and Data Collection

Each eligible resident received an email from a research assistant 1 week prior to the clinical rotation in which the tablet computers were available to them. The email message explained the purpose of the project and offered the resident the opportunity to use the tablet during the course of the 3- to 4-week rotation. Participation was optional, but participants agreed to complete a confidential electronic survey before and after the rotation. Participants also agreed to take personal responsibility for the security of the device. Participants were permitted to take tablets home or to store them in a hospital-issued locker. No other incentives or compensation were offered.

Surveys were created and distributed electronically using SurveyMonkey (San Mateo, California, United States) ([Supplementary Material], available in the online version). The baseline survey was distributed 7 days prior to the start of the rotation. A follow-up survey was distributed within 1 week after completing the rotation. A secure, encrypted database was used to track responses. Nonresponders received up to three reminder emails. If a resident used a tablet on multiple rotations, he or she was asked to complete surveys for each rotation. We tracked whether the resident's response was for an initial or repeat trial.

Analyses

We tabulated respondents' characteristics such as gender, clinical service, PGY, and self-reported technological aptitude. We also tabulated residents' assessments of the ease and speed of use of tablet computers compared with traditional desktop workstations for various clinical tasks, such as locating an available desktop computer, viewing medical information, writing notes, and entering orders. We then compared residents' assessments at baseline and follow-up of the effects of tablet computers on clinical tasks, job satisfaction, time spent working in the hospital, and quality of care, using the chi-square statistic. We repeated this analysis, stratifying residents by their self-reported “tech” aptitude. A two-tailed p-value of ≤ 0.05 was considered statistically significant. Analyses were performed using Stata 9.0 (StataCorp, College Station, Texas, United States). Free-text comments from tablet-user surveys were examined. This project was reviewed in advance by the hospital's institutional review board (IRB) and determined to be a quality improvement and educational project exempt from IRB review.

Results

Participant Characteristics

Of 63 eligible residents, 49 (78%) agreed to participate in the project and completed a baseline survey. Of the 14 residents who declined to participate (13 in medicine, 1 in surgery), several expressed reluctance to take personal responsibility for the tablet. Thirty-seven residents completed a follow-up survey. [Table 1] displays participants' demographic characteristics identified via baseline surveys. The majority were male interns who considered themselves above-average in IT literacy. All respondents owned a smart phone and 59% had a personal tablet computer.

Table 1 Respondent characteristics (N = 49)

Characteristics	N (%)
Male	28 (57.1)
Service
Surgery (surgical oncology, colorectal, thoracic, vascular, and general surgery consult)	5 (10.2)
Neurology	8 (16.3)
Internal medicine (general medicine and nephrology)	22 (44.9)
Internal medicine night float	10 (20.4)
Intensive care unit	4 (8.2)
Postgraduate year (PGY)
PGY-1	25 (51.0)
PGY-2	16 (32.7)
PGY-3	5 (10.2)
PGY-4	3 (6.1)
How tech-savvy do you consider yourself?
Very tech savvy	12 (24.5)
Above average	24 (49.0)
Average	10 (20.4)
Below average	3 (6.1)
Not at all	0 (0)
Do you own or use a smartphone?
Yes, Apple	39 (79.6)
Yes, Android	10 (20.4)
No	0 (0)
Do you own or use a tablet?
Yes, iPad	29 (59.2)
Yes, Windows device	2 (4.1)
Yes, other tablet type	3 (6.1)
No	15 (30.6)

Ease and Speed of Use

The tablet computers were well received by house staff on all services and units ([Table 2]). In the follow-up survey, 62% of the 37 respondents reported using the device on a daily basis, and another 27% used the tablet several days per week. Residents found the tablet most useful while working on the inpatient wards (51%) and in the hospital on call (41%). The majority also agreed that it was easier (73%) and faster (70%) to use the tablet than traditional desktop computers, which often have limited availability. “The portability of the tablet really made work much easier,” said a respondent. “Tablets enabled me to work on notes, orders, and collect data when no computers were available.”

Table 2 Respondent assessment of the ease and speed of use of tablet computers (N = 37)

Questions	N (%)
How often did you use the tablet?
Always (daily)	23 (62.2)
Often (few d per wk)	10 (27.0)
Occasionally (once a wk or less)	2 (5.4)
Never	2 (5.4)
Where did you find the tablet most useful?
On inpatient wards	19 (51.4)
Out of hospital (not on call)	1 (2.7)
Out of hospital (on call)	1 (2.7)
In hospital (on call)	15 (40.5)
Other	1 (2.7)
Using the tablet was …	Easier	Faster
…than searching for and locating an available desktop
Agree	27 (73.0)	26 (70.3)
Neutral	4 (10.8)	4 (10.8)
Disagree	2 (5.4)	4 (10.8)
Missing	4 (10.8)	3 (8.1)
…for viewing medical information than on a standard desktop
Agree	14 (37.8)	19 (51.4)
Neutral	12 (32.4)	10 (27.0)
Disagree	7 (18.9)	3 (8.1)
Missing	6 (16.2)	5 (13.5)
…for writing notes than on a standard desktop
Agree	14 (37.8)	21 (56.8)
Neutral	9 (24.3)	7 (18.9)
Disagree	9 (24.3)	7 (18.9)
Missing	5 (13.5)	2 (5.4)
…for entering orders than on a standard desktop
Agree	15 (40.5)	18 (48.6)
Neutral	11 (29.7)	7 (18.9)
Disagree	6 (16.2)	7 (18.9)
Missing	4 (10.8)	4 (10.8)

Respondents indicated that tablets improved the speed at which they could complete work-related tasks such as viewing medical information (51%), writing notes (57%), and entering orders (49%), compared with a standard desktop computer workstation. According to one resident, “[Tablets] improved efficiency so I could spend more time talking to the patient.” Another resident stated that “The ability to quickly put in orders at anytime and anywhere without needing immediate access to a desktop is invaluable.”

Impact on Job Satisfaction and Quality of Care

[Table 3] compares respondents' assessments of the impact of tablet computers on clinical tasks, job satisfaction, time spent in the hospital, and perceived quality of care at baseline and follow-up. Expectations were high at baseline: 98% of respondents anticipated that tablets would be helpful in delivering clinical care, 86% predicted improved job satisfaction, 57% predicted that the device would decrease overall time spent in the hospital, and 69% believed that tablet computers would help to ensure high-quality care. According to one respondent, “Portability and consistent access to EMR will help.” Another respondent wrote that “We have renal rounds in the dialysis unit, but have patients on our service geographically spread throughout the hospital, so it will be convenient to have a portable tablet to work on.”

Table 3 Respondent assessment of tablet impact at baseline and follow-up

Survey item	Baseline survey N = 49 N (%)	Follow-up survey N =37 N (%)	p-Value[a]
Tablets helpful for clinical care			0.400
Yes	48 (98.0)	35 (94.6)
No	1 (2.0)	2 (5.4)
Tablets most useful for			0.320
Orders	19 (38.8)	8 (21.6)
Notes	12 (24.5)	10 (27.0)
Reviewing data	15 (30.6)	14 (37.8)
Accessing medical references	2 (4.1)	1 (2.7)
Other	1 (2.0)	4 (10.8)
Tablets improve your job satisfaction			0.959
Yes	42 (85.7)	31 (83.8)
No	7 (14.3)	5 (13.5)
Missing	0 (0)	1 (2.7)
Tablets change the amount of time spent in the hospital			0.689
Decrease time	28 (57.1)	19 (51.4)
No change	21 (42.9)	17 (45.9)
Increase time	0 (0)	1 (2.7)
Tablets affect the quality of care delivered			0.790
Improve care	34 (69.4)	24 (64.9)
No change in care	15 (30.6)	12 (32.4)
Worsen care	0 (0)	1 (2.7)

^a Chi-square statistic.

On follow-up, there were no statistically significant differences in respondents' perceptions compared with their expectations at baseline. Residents reported that tablet computers facilitated clinical care (95%), improved job satisfaction (84%), reduced time spent in the hospital (51%), and enhanced the quality of care (65%). A greater percent of residents at baseline than at follow-up believed that tablets would be (or were) “most useful” for order-writing (39% vs. 22%), a nonsignificant difference (p = 0.215). In contrast, a greater percent of residents found tablets most useful for writing notes and reviewing data at follow-up than predicted at baseline. Favorable assessments of the benefit of tablets with respect to time spent in the hospital and improved quality of care were lower at follow-up compared with baseline, but these differences were not statistically significant.

Respondents at baseline identified order-writing as the most likely useful function of tablets. At follow-up, “reviewing data” was the most frequently cited use. Differences between expectations and experience may have been influenced by clinical rotation and shift. According to one resident, “I had the tablet on night float and did not use it much since there were plenty of available computers in the ED [emergency department] at that time. However, I do think they will be helpful during long-call hours.” Several residents noted unforeseen challenges with wireless Internet access in certain areas of the hospital. No significant differences were identified in subgroup analyses comparing residents who identified themselves as above average in IT literacy compared with those who self-identified as below average, the factor we hypothesized to have the greatest impact on resident perceptions.

Discussion

This pilot project assessed resident physicians' perceptions and experience of using tablet computers on the inpatient services of a teaching hospital. The majority of residents had high expectations at baseline that tablets would offer improved efficiency, quality of care delivery, and work satisfaction. At follow-up, respondents indicated that these initial expectations were generally satisfied. A majority of respondents reported that tablets were faster than desktops for viewing medical information and writing notes, improved job satisfaction, and improved the quality of care. Mobility and access both offered substantial advantages for residents on busy inpatient services, and residents felt that tablets were faster and easier to locate compared with desktop computers. Interruptions in wireless connectivity were cited as an occasional disappointment.

Our project provides insight into the acceptability of mobile technology for house staff. Several previous studies have evaluated mobile technology's influence on educational outcomes.[16] [20] [21] [22] [23] In a study by Moodley et al, infectious disease fellows identified 24 iPad applications to facilitate their clinical tasks, decision-making, and education.[24] In neurosurgical training, easily accessible information and portability guaranteed by tablets provided more opportunities and time for 92% of residents polled, and was associated with improved performance on 16 of 18 domains on the Congress of Neurological Surgeons Self-Assessment Examination.[25] [26] A study of 9 neurology residents found that tablet computers resulted in time savings in preparing and postprocessing of ward rounds.[27] In contrast, no prior studies have examined the impact of mobile technology on perceptions of clinical care and resident work life.

Mobile technology may be particularly valuable to resident physicians, as they are responsible for providing around-the-clock patient care, often in geographically dispersed locations. Residents frequently move among hospital wards and clinics with the need to access clinical information and reference materials from a variety of sources. In our project, tablet computers allowed residents to find and view pertinent information—including laboratory test results, vital signs, and notes—during teaching conferences or multidisciplinary rounds. Patient care interventions, such as placing orders or preparing discharge paperwork, could be enacted expediently. Tablets also reduced congestion and wait times associated with accessing stationary desktop computers and wheeled workstations.

While the majority of residents used the mobile devices daily, about one-third of residents used the tablets intermittently or not at all. Intermittent use suggests a tentative or stuttering process of technology adoption or the perception that tablets are better for certain tasks or settings. Some individuals who chose not to use the tablets expressed concern about their responsibility for securing the device or about their ability to integrate the technology into their usual workflow. We did not elicit adverse effects related to tablet use for patients, attending physicians, or other members of the care team. Previous research identified an association between computer use by clinicians in safety-net clinics with lower patient satisfaction.[28] Although physician computer order-entry systems reduce adverse drug events in certain settings, the technology's potential negative effects on patient safety and clinician efficiency have also been reported.[29] [30] [31] [32] [33] [34] [35] Use of handheld computers for clinical care has been limited by interoperability, security, and poor user interfaces.[36] Multiple reports suggest that electronic health records increase the time clinicians spend in indirect care,[37] reducing the amount of face-to-face patient care to only 27% of the workday.[38]

Our project is subject to several limitations. As a single institution project, its results cannot be readily generalized. The project was limited to a single tablet device and the number of respondents was relatively small. Clearly, many different types of tablet computers are used for various purposes in health care. Users' experiences reflect the utility of those devices for a specific set of applications. Although the participation rate was high (78% of eligible residents agreed to participate), participation was voluntary and therefore vulnerable to selection bias. Although we believe the surveys have face validity, they have not been subjected to rigorous validity and reliability testing. Future research should ensure that survey tools are carefully tested in advance. Residents who declined to participate could have rated the devices less favorably than those who participated. Because we did not survey tablet users about their use of the devices for nonclinical tasks, it is possible that the tablets conferred unmeasured benefits that influenced respondents' perception of the devices' value.

Conclusion

In conclusion, resident physicians readily adopted tablet computers and found these devices improved the speed and ease of performing clinical care functions, job satisfaction, and residents' perceptions of the quality of care they delivered. Additional experience is needed to understand how best to implement mobile technology in teaching hospitals and whether the technology might introduce unanticipated risks and vulnerabilities. Future research should measure the efficiency and quality of care of teams equipped with mobile computers, and the impact of these devices on patient assessment of patient–doctor communication. We need to know more about the use and value of different types of devices and of best practices for ensuring the security of these potentially important tools.

Clinical Relevance Statement

This project surveyed resident physicians about the impact of tablet computers before and after their introduction on the adult inpatient services of a teaching hospital. Residents physicians found tablets to be easier to use and access than desktop workstations. The majority of residents indicated that tablet computers increased their job satisfaction, reduced the amount of time spent in the hospital, and improved the quality of care. The introduction of mobile information technology may provide significant benefits to frontline clinicians.

Multiple Choice Question

Resident physicians perceived that the introduction of tablet computers affected which of the following?

• Job satisfaction only

• Time spent in the hospital only

• Quality of patient care only

• All of the above

Correct Answer: The correct answer is option d. In this study performed at a teaching hospital, the authors surveyed resident physicians before and after the introduction of tablet computers on each of eight inpatient clinical services. Expectations were high at baseline and generally satisfied. Respondents at follow-up indicated that tablet computers increased their job satisfaction (84%), reduced the amount of time spent in the hospital (51%), and improved the quality of care (65%).

Conflict of Interest

None.

Protection of Human and Animal Subjects

The hospital institutional review board (IRB) reviewed this project in advance and determined that it was a quality improvement and educational project exempt from IRB review and from applicable federal regulations governing human research.

• References

• 1 Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res 2012; 14 (05) 128
• 2 Ventola CL, Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P&T 2014; 39 (05) 356-364
• 3 Landreani F, Caiani EG. Smartphone accelerometers for the detection of heart rate. Expert Rev Med Devices 2017; 14 (12) 935
• 4 Chou D. Health IT and patient safety: building safer systems for better care. JAMA 2012; 308 (21) 2282
• 5 Staggers N, Weir C, Phansalkar S. Patient safety and health information technology: role of the electronic health record. In Hughes RG. , ed. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Chapter 47. Rockville, MD: Agency for Healthcare Research and Quality (US); 2008
• 6 Chu LF, Erlendson MJ, Sun JS, Alva HL, Clemenson AM. Mobile computing in medical education: opportunities and challenges. Curr Opin Anaesthesiol 2012; 25 (06) 699-718
• 7 White T. Will iPad transform med school? Stanford Medicine News Center; September 13, 2010. Available at: https://med.stanford.edu/news/all-news/2010/09/will-ipad-transform-med-school.html . Accessed January 11, 2018
• 8 Lobo MJ, Crandley EF, Rumph JS. , et al. Pilot project of iPad incorporation into graduate medical education. J Grad Med Educ 2013; 5 (01) 142-144
• 9 Charani E, Gharbi M, Moore LSP. , et al. Effect of adding a mobile health intervention to a multimodal antimicrobial stewardship programme across three teaching hospitals: an interrupted time series study. J Antimicrob Chemother 2017; 72 (06) 1825-1831
• 10 Korbage AC, Bedi HS. Mobile technology in radiology resident education. J Am Coll Radiol 2012; 9 (06) 426-429
• 11 Sharpe III EE, Kendrick M, Strickland C, Dodd III GD. The Radiology Resident iPad Toolbox: an educational and clinical tool for radiology residents. J Am Coll Radiol 2013; 10 (07) 527-532
• 12 Berkowitz SJ, Kung JW, Eisenberg RL, Donohoe K, Tsai LL, Slanetz PJ. Resident iPad use: has it really changed the game?. J Am Coll Radiol 2014; 11 (02) 180-184
• 13 Gonzalez NR, Dusick JR, Martin NA. Effects of mobile and digital support for a structured, competency-based curriculum in neurosurgery residency education. Neurosurgery 2012; 71 (01) 164-172
• 14 Dimond R, Bullock A, Lovatt J, Stacey M. Mobile learning devices in the workplace: as much a part of the junior doctors' kit as a stethoscope?. BMC Med Educ 2016; 16 (01) 207
• 15 Sung YT, Chang KE, Liu TC. The effects of integrating mobile devices with teaching and learning on students' learning performance: a meta-analysis and research synthesis. Comput Educ 2016; 94: 252-275
• 16 Bullock A, Webb K. Technology in postgraduate medical education: a dynamic influence on learning. Postgrad Med J 2015; 91 (1081): 646-650
• 17 Niehaus W. Informatics and technology in resident education. PM R 2017; 9 (05) S118-S126
• 18 Vetter SY, Schüller S, Hackbusch M. , et al. Tablets for image review and communication in daily routine of orthopedic surgeons – an evaluation study. J Digit Imaging 2018; 31 (01) 74-83
• 19 George P, Newey CR, Bhimraj A. The tablet device in hospital neurology and in neurology graduate medical education: a preliminary study. Neurohospitalist 2015; 5 (01) 15-21
• 20 Bullock A, De Jong PGM. Technology-enhanced learning. In Swanwick T. , ed. Understanding Medical Education: Evidence, Theory and Practice. 2nd ed. Chichester: Wiley Blackwell; 2014: 149-160
• 21 Khasawneh R, Simonsen K, Snowden J, Higgins J, Beck G. The effectiveness of e-learning in pediatric medical student education. Med Educ Online 2016; 21 (01) 29516
• 22 Cook DA, Hatala R, Brydges R. , et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA 2011; 306 (09) 978-988
• 23 Baumgart DC, Wende I, Grittner U. Tablet computer enhanced training improves internal medicine exam performance. PLoS One 2017; 12 (04) e0172827
• 24 Moodley A, Mangino J, Goff D. Review of infectious diseases applications for iPhone/iPad and Android: from pocket to patient. Clin Infect Dis 2013; 57: 1145-1154
• 25 Dupaix J, Chen JJ, Chun MB, Belcher GF, Cheng Y, Atkinson R. The effect of mobile tablet computer (iPad) implementation on graduate medical education at a multi-specialty residency institution. Hawaii J Med Public Health 2016; 75 (07) 190-195
• 26 Mickan S, Atherton H, Roberts NW, Heneghan C, Tilson JK. Use of handheld computers in clinical practice: a systematic review. BMC Med Inform Decis Mak 2014; 14: 56 . Doi: 10.1186/1472-6947-14-56
• 27 Robinson R. Spectrum of tablet computer use by medical students and residents at an academic medical center. PeerJ 2015; 3 (1133): 3
• 28 Ratanawongsa N, Barton JL, Lyles CR. , et al. Association between clinician computer use and communication with patients in safety-net clinics. JAMA Intern Med 2016; 176 (01) 125-128
• 29 Asan O, Carayon P, Beasley JW, Montague E. Work system factors influencing physicians' screen sharing behaviors in primary care encounters. Int J Med Inform 2015; 84 (10) 791-798
• 30 Bates DW, Leape LL, Cullen DJ. , et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998; 280 (15) 1311-1316
• 31 Radley DC, Wasserman MR, Olsho LE, Shoemaker SJ, Spranca MD, Bradshaw B. Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc 2013; 20 (03) 470-476
• 32 Nuckols TK, Smith-Spangler C, Morton SC. , et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56
• 33 Schiff GD, Amato MG, Eguale T. , et al. Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf 2015; 24 (04) 264-271
• 34 Slight SP, Eguale T, Amato MG. , et al. The vulnerabilities of computerized physician order entry systems: a qualitative study. J Am Med Inform Assoc 2016; 23 (02) 311-316
• 35 Brown CL, Mulcaster HL, Triffitt KL. , et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc 2017; 24 (02) 432-440
• 36 McAlearney AS, Schweikhart SB, Medow MA. Doctors' experience with handheld computers in clinical practice: qualitative project. BMJ 2004; 328 (7449): 1162
• 37 Patel BK, Chapman CG, Luo N, Woodruff JN, Arora VM. Impact of mobile tablet computers on internal medicine resident efficiency. Arch Intern Med 2012; 172 (05) 436-438
• 38 Sinsky C, Colligan L, Li L. , et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760