Interaction Time with Electronic Health Records: A Systematic Review

Abstract

Background The amount of time that health care clinicians (physicians and nurses) spend interacting with the electronic health record is not well understood.

Objective This study aimed to evaluate the time that health care providers spend interacting with electronic health records (EHR).

Methods Data are retrieved from Ovid MEDLINE(R) and Epub Ahead of Print, In-Process and Other Non-Indexed Citations and Daily, (Ovid) Embase, CINAHL, and SCOPUS.

Study Eligibility Criteria Peer-reviewed studies that describe the use of EHR and include measurement of time either in hours, minutes, or in the percentage of a clinician's workday. Papers were written in English and published between 1990 and 2021.

Participants All physicians and nurses involved in inpatient and outpatient settings.

Study Appraisal and Synthesis Methods A narrative synthesis of the results, providing summaries of interaction time with EHR. The studies were rated according to Quality Assessment Tool for Studies with Diverse Designs.

Results Out of 5,133 de-duplicated references identified through database searching, 18 met inclusion criteria. Most were time-motion studies (50%) that followed by logged-based analysis (44%). Most were conducted in the United States (94%) and examined a clinician workflow in the inpatient settings (83%). The average time was nearly 37% of time of their workday by physicians in both inpatient and outpatient settings and 22% of the workday by nurses in inpatient settings. The studies showed methodological heterogeneity.

Conclusion This systematic review evaluates the time that health care providers spend interacting with EHR. Interaction time with EHR varies depending on clinicians' roles and clinical settings, computer systems, and users' experience. The average time spent by physicians on EHR exceeded one-third of their workday. The finding is a possible indicator that the EHR has room for usability, functionality improvement, and workflow optimization.

Keywords

electronic health records and systems - inpatient (inpatient CPOE) - workflow - time and motion - task performance and analysis

Background and Significance

The use of electronic health record systems (EHRs) has become increasingly widespread in recent decades. This widespread adoption is particularly relevant in the United States and have become essentially universal in the United States due to the federal mandate for adoption and meaningful use of EHRs by January 1, 2014. While the adoption of the EHR appears to be inevitable, incorporating the basic requirements of a modern, efficient system remains challenging. The considerable barriers reported are related to costs, physician resistance, and a low level of interoperability.[1] [2] [3]

Numerous studies have examined the advantages and disadvantages of the EHR. Documented benefits include improvements in patient safety, effectiveness, and efficiency.[4] However, drawbacks, besides costs, include risk of privacy violations and information overload.[4] Several studies indicate the negative effect of the use of the EHR on physician burnout.[5] [6] Time spent documenting in the EHR and the time pressure clinicians experience to provide high-quality patient care could be determining factors in the level of physician reported dissatisfaction and intent to leave.[7] [8] Similarly, frontline nurse clinicians have reported frustration with the amount of time spent interacting with the EHR and insufficient time for documentation.[9] Those that use remote EHR systems also reported higher levels of stress, burnout, and isolation.[10]

It is a reality that EHR adoption has drastically changed the health care workflow, regardless of setting. The influence of EHR both on health care providers and patients is widely discussed in the media.[11] The Office of the National Coordinator for Health Information Technology (ONC) in partnership with Centers for Medicare & Medicaid Services has expressed great concern on this matter and suggested a strategy for reducing EHR burden on clinicians.[12] The goal of this initiative is to reduce the effort and time required to record health information in EHRs for clinicians. Time spent on interaction with EHR can serve as a baseline indicator of EHR burden prior to implementation of any strategies for its reduction.

Exploring the amount of time clinical providers spend interacting with the EHR (typing, viewing, and navigating) during their shift is important to reach a better understanding of the place of the EHR in the clinician workflow. However, the amount of time health care providers allocate to the EHR has not been systematically explored.

Objective

The aim of this systematic review is to evaluate the amount of time that clinicians, both physicians, and nurses, spend interacting with EHR during their workday in both outpatient and inpatient settings. This systematic review is intended to aid clinicians and institutions in understanding the current clinician workflow. The primary goal is to show how the EHR mediates the workflow.

Methods

This systematic review was performed at Mayo Clinic Rochester. The review protocol was registered on PROSPERO (ID number CRD42019120270) on February 15, 2019 and can be accessed at http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42019120270. The study was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.[13]

Eligibility Criteria

This study focused on peer-reviewed publications that report the time of interaction with the EHR. To be included in the analysis, papers had to describe the use of the EHR or EHR combined with computerized provider order entry (CPOE) and include measurement of time either in hours, minutes, or in the percentage of a day of primary EHR user. The primary user of technology was considered as any physician, physician assistant, or nurse. It excludes publications, in which the primary user of the EHR was not a clinician, for example administrative, laboratory, research staff, students, or patients. Papers that did not study clinical documentation and CPOE (e.g., clinical decision support tools) were also excluded.

We included all study designs. Secondary literature, conference abstracts, commentaries, opinion editorials, and letters to the editor were excluded. We also excluded studies that performed in a simulated environment.

Studies were eligible for inclusion if they were written in English and published between 1990 and 2021. As computer and internet access since in the early 1990s contributed to more widespread dissemination of the EHR in clinical settings,[14] this timeframe was considered appropriate to our objective.

Information Sources and Search

An experienced librarian with input from the principal investigator developed database selection and search strategies ([Supplementary Material] [available in the online version]). A comprehensive search of the following databases was conducted: Ovid MEDLINE(R) and Epub Ahead of Print, In-Process & Other Non-Indexed Citations and Daily, (Ovid) Embase, CINAHL, and SCOPUS. English titles and abstracts published between January 1990 and December 2018 were initially searched on December 5, 2018. To include recent studies, a final systematic database query was performed on March 1, 2021. Search terms were limited by documented time of interaction with any EHR components or CPOE in outpatient and inpatient settings. Controlled vocabulary supplemented with keywords was used to search for EHR. The Medline search strategy included the following headings and keywords: “Medical records systems, Computerized”, “Electronic medical records”, “CPOE”, “Documentation”, “Workflow”, “Task performance and analysis”, “Time and motion,” “Work redesign,” “Efficiency,” and “Data entry,” “Error.” The terms and filters' syntax for each database are provided in the [Supplementary Material] [available in the online version].

Table 1 Characteristics of studies that describe a physician's time on electronic health record

n	Study (Year)	Type of study	Setting and country	Type of staff	Computer system used	Tool for measurement	EHR tasks evaluated	Narrative summary	Major limitations
1	Arndt et al (2017)[51]	Retrospective observational	Academic health care center that consists of family medicine clinics; U.S.	Physicians	Epic system (Epic Systems Corporation)	Logged time	Documentation, chart review order entry, problem list, other clerical task, inbox tasks	Family medicine physicians spent around 52% of their day on EHR. Tasks included EHR documentation (23.7%), chart review (28.7%), order entry (12.1%), problem list (3.4%); other clerical tasks (8.4%), and inbox tasks (23.7%).	Time spent on some activities could be underestimated as associated tasks (chart review and telephone call) were attributed to one category (chart review). Time spent on EHR could be overestimated, as the event logs show the time of an ongoing task opened in EHR rather than active clinician interaction.
2	Carayon et al (2015)[43]	Time motion	Tertiary care medical center, three ICUs; U.S.	Residents, attending physicians	EpicCare Inpatient Clinical System, spring 2006 version	Direct observations[a] (144 h in total)	Order entry, clinical review and documentation, administrative review and documentation	Residents spent 8.0% (SD = 5.5) of time on order entry, 30.5% (SD = 15.4) on clinical review and documentation, 4.3% (SD = 3.9) on administrative review and documentation. Physicians spent 3.1% (SD = 3.0) of time on order entry, 26.7% (SD = 12.0) on clinical review and documentation, 4.7% (SD = 2.9) on administrative review and documentation.	Multitasking data were excluded from sequential analyses. The data refers to 2006–2008 when EHR were less advanced.
3	Cox et al (2018)[52]	Retrospective observational	Academic institution, department of general surgery; U.S.	Residents	Epic system (Epic Systems Corporation)	Logged time	Total EHR time	The median time spent on EHR by residents was 2.4 hour per day (IQR 0.0–6.13). On average general surgery residents dedicated 38.2% (23.7 hour) of the actual work week on EHR.	Logged time was not categorized according to specific settings (inpatient, outpatient, remote), residents' rotations and shifts. The sensitivity analysis showed an underestimation of EHR interaction time (by 1.2 h) due to inclusion of logged time of one particular rotation (use of another electronic system in parallel with Epic).
4	Cox et al (2021)[53]	Retrospective observational	Academic institution department of general surgery; U.S.	Physicians	Epic system (Epic Systems Corporation)	Logged time	Total EHR time	The mean time spent on EHR by general surgery attendings was 1.96 hour (SD = 2.50) per day. The mean EHR use was 1.70 h (SD = 2.28) between 6 a.m. and 6 p.m. and 0.26 hour (SD = 0.69) between 6 p.m. and 6 a.m. The average EHR use was 2.73 hour (SD = 2.80) per designated clinic days. The average EHR use was 2.19 hour (SD = 2.50) on operative days and 0.91 hour (SD = 1.66) on open days.	There is a possible underestimation of EHR use: logged times from portable devices and another electronic system were not included. Clinical hours may be inaccurate due to schedule changes. There is a possible overestimation of EHR use as the time accounted for both active interaction and passive use of electronic system.
5	Giliberto et al (2020)[54]	Retrospective observational	10 de-identified institutions - ambulatory; otolaryngology; U.S.	Physicians	Epic system, (Epic Systems Corporation)	Logged time	Total EHR time; writing notes, clinical chart review, in basket messages, placing orders, other, schedule, visit navigator	Otolaryngologists and advanced practice provider spent 74.0 minute (SD = 38.8) on EHR interaction daily. Writing notes accounted for 31.7 minute (SD = 20.8) or 43%. Chat review required 10.4 minute (SD = 6.9) or 14%, in basket messages - 7.8 minute (SD = 6.3) or 10%, placing orders - 7.1 minute (SD = 5.5) or 10%, other tasks - 6.2 minute (SD = 7.7) or 8%, schedule - 4.0 minute (SD = 4.1) or 8%, and visit navigator - 5.1 minute (SD = 3.9) or 7%.	The time spent on EHR was underestimated: (1) The EHR interaction time was limited to active EHR use with a short 5-s timeout. (2) Clinically active days were overestimated as any day when EHR was opened (weekends, half-days) was included to the denominator. Due to methodology the proportion of total time on EHR per day in % was not calculated. The data were not aggregated in the [Fig. 1].
6	Hanauer et al (2013)[42]	Time motion	Academic medical center, hematology/oncology; U.S.	Physician assistants	EHR and CPOE (Sunrise Clinical Manager, V. 4.5; Allscripts, Chicago, IL)	Direct observations[a] (70.5 h in total)	Computer reading activities, computer writing (orders and forms)	Computer reading activities required 14% of total time, computer writing activities required 26% of total time, of which 7.2% was accounted for order writing.	The sample size was small (n = 3).
7	Heaton et al (2018)[45]	Time-motion	Academic tertiary care center, ED; U.S.	ED attending physicians	EHR (system is not mentioned)	Direct observations[a] (192 h in total)	Reviewing records, order entry, and shift and postshift documentation	ED attending physicians spent on reviewing records and order entry on average 87.1 minute (SD = 23.2); on shift documentation 55.3 minute (SD = 18.8); on postshift documentation 42.5 minute (SD = 32.5). Mean time spent on EHR interaction was 174.8 minute (SD = 46.9) or 38.5% of shift time.	The type of EHR was not described, the sample size of participants was not mentioned. Time spent on EHR after working hours was collected as self-reported data.
8	Hill et al (2013)[46]	Time motion	Community teaching hospital, ED; U.S.	Physician, residents middle-level providers	McKesson Horizon Emergency Care V.10.3 EHR system	Mouse click count (observation for 30 h)	Data entry Chart review	Emergency physicians spent 43% of shift time on data entry, 12% on case review. The total time of interaction with EHR during the shift was 55%.	The study has a high risk of bias: small sample size (n = 16); short observation time. High variability in outcomes (large standard deviation) might be associated with high variability in training of included participants.
9	Krawiec et al (2020)[55]	Retrospective observational study	University-affiliated tertiary care facility, PICU; U.S.	Physicians	Cerner Powerchart (Cerner Corp, Kansas City, MO)	Logged time	Total EHR time, chart review time, documentation time, and order entry	Physicians spent on EHR interaction a median of 2:10 (IQR 1:31–3:08) h: min or ∼20% of their 10-hour shift. Chart review accounted for 35 minute (IQR 00:14–1:10), documentation time - for 1 hour (IQR 00:45–1:22).	The sample size was small (n = 7). Shift hours were chosen based on a regular PICU workflow. Fluctuations in a physician schedule could present.
10	Mamykina et al (2016)[47]	Time motion	Urban teaching hospital, general medicine service; U.S.	Residents	EHR system (Allscripts Sunrise, Allscripts, Chicago, IL)	Direct observation[a] for 7–14 h	Computer-based activities: writing, viewing, and reading notes	Residents dedicated 364.5 minute or 50.6% of their shift time to computer tasks. Documentation accounted for 189.9 minute or 52.1% of all computer tasks, including writing notes (128.7 minute or 35.3%), viewing the list of available notes (33.9 minute or 9.3%), and reading notes composed by others (27.3 minute or 7.5%).	Major limitations: small sample size (n = 7), the total time of observation was not mentioned.
11	Maloney et al (2020)[56]	Retrospective observational	An academic tertiary care teaching hospital; U.S.	Residents	Canopy (Cerner Corporation, Kansas City, MO)	Logged time	Total EHR time per day Specific EHR tasks were reported per patient chart	The general surgery and OB/GYN residents spent on EHR interaction nearly 2 hour per day. The orthopedic residents spent on EHR tasks 80 minute per day. The total EHR time was significantly different (p < 0.001) between interns (133.3 minute), juniors (115.2 minute), and seniors (95.1 minute). General surgery residents spent on EHR interaction 17% of an average 12-h shift.	Orthopedics residents might act as consultants rather than primary writers of notes. The authors noted discrepancies between the reported number of patient charts per resident per day and the actual number of patients that residents are taking care for daily. The total time is likely to be underestimated.
12	Sinsky et al (2015)[48]	Time motion	Ambulatory care in 4 specialties in 16 practices; U.S.	Physicians	Various EHR systems	Direct observations[a] (430 hour in total)	EHR and desk work	Family physicians spent 49.2% of their day on EHR and desk work.	Time spent on EHR after working hours was collected as self-reported data.
13	Tipping et al (2010)[49]	Time motion	Tertiary care teaching hospital; U.S.	Physicians	EHR (system is not mentioned)	Direct observation[a] (494 h in total)	Mean time (%) spent on EHR, including time on writing, orders, reading/reviewing and others	Hospitalists spent on average on EHR 34.1% (SD = 0.9) of shift time. Writing required 58.4% of all EHR activities, orders (20.2%), reading/reviewin (19.4%), and others (2%).	The sample size of hospitalists was small.
14	Verma et al (2020)[57]	Retrospective observational	Tertiary care center; U.S.	Physicians, residents	CERNER1 (North Kansas City, MO)	Logged time	Total EHR time per day; time on chart review, on orders, on documentation, and on other activities.	In the first academic quarter, mean daily EHR time was 3.8 hour (SD = 2), or 37% (SD = 17), 41% (SD = 14), and 45% (SD = 12) of a day for all clinicians, residents, and interns, respectively. During the fourth academic quarter, the daily time on EHR decreased up to 30%. Clinicians dedicated 38% (SD = 8) of time on chart review, 17% (SD = 7) on orders, 28% (SD = 11) on documentation, and 17% (SD = 7) on other activities.	The work hours could be inaccurate.
15	Wang et al (2019)[58]	Retrospective observational	Academic tertiary care hospital, internal medicine; U.S.	Residents	Epic system, (Epic Systems Corporation)	Logged time	Total EHR time	Internal medicine residents interacted with EHR for average of 5.38 hour (SD = 2.7) or 44% of their shifts. Chart review required more than 40% of EHR time. Depending on training level and rotations, time distribution of other activities was the following: note review 8.1–21.8%, note entry 4–18.4%, order entry 6.2–11.4%, navigator 4.2–8.8%, and results review 4.1–7.7%.	The EHR time could be overestimated as idle intervals of 5 minute and less were attributed to active EHR interaction time.

Abbreviations: ED, emergency department; EHR, electronic health records; IQR, interquartile range; ICU, intensive care unit; OBGYN, obstetrics and gynecology; PICU, pediatric intensive care unit; SD, standard deviation.

^a Time of direct observation refers to postimplementation periods of pre-post design studies.

Study Selection

Duplicate articles were excluded. Study selection was performed by using the online systematic review software Covidence (Alfred Health, Monash University, Melbourne, Australia). The titles and abstracts of the references retrieved in searches were assessed for eligibility twice by two independent reviewers (Y.P. and K.C.). Eligible publications were obtained through the full-text reading of selected articles and independent evaluation. Any discrepancies were resolved through discussion with the third reviewer (V.H.).

Data Collection and Analysis

Data abstraction from the articles selected for inclusion was performed by one reviewer (Y.P.). Extracted information included country, study setting; type and sample size of hospital staff; year and duration of the study; type and characteristic of the computer system; time allocated to interaction with the EHR; and which tasks were performed.

Outcomes of interest were quantitative. The primary outcome was the time of clinician interaction with the EHR, measured in hours of a workday or percentage of a workday. The appropriate portion of each study was analyzed, excluding data beyond the scope of the current review (e.g., time spent using paper records, team communication, patient-physician interaction, and scribes). Meta-analysis was not performed due to high heterogeneity of selected studies.

For assessing risk of bias the 16-item Quality Assessment Tool for Studies with Diverse Designs (QATSDD) was used.[15] The tool showed good reliability and validity for heterogeneous methodologies, including observational time-motion studied.[16] Fourteen criteria were applied for both qualitative and quantitative studies. Each item was scored from 0 to 3 with a maximum quality score of 42. The following characteristics of included studies were additionally assessed: observer training, acknowledgment of the Hawthorne effect, total observation hours, and observation after working hours and on weekends.

Results

Search Results

A total of 6,485 abstracts were identified through the database search ([Fig. 1]). After the removal of duplicate records (n = 1352), a total of 5,133 abstracts remained for title and abstract screening. A total of 4,899 abstracts were excluded, leaving 234 full-text articles eligible for full review. After adding one record identified through manual search, 235 studies were reviewed by two researchers in duplicate. A total of 217 of these full-text articles were excluded due to lack of any outcome of interest (n = 128), nonoriginal studies, including reviews, comments, opinions, editorial letters (n = 46); abstracts (n = 35), or other reasons (n = 8). Most papers that lacked outcome of interest (time spent on EHR per day) reported time on direct or indirect patient care,[17] [18] time on mixed paper and computer tasks,[19] [20] [21] [22] [23] [24] [25] [26] [27] [28] time per note or record,[29] [30] [31] [32] [33] [34] time on EHR per patient visit,[35] [36] [37] [38] [39] and time calculated in a simulated environment.[40] [41] After completion of full-text screening, 18 articles were left for data extraction and analysis.

Fig. 1 Flow chart of study selection. Flow chart describing the process of study selection.

Characteristics of Included Studies

Half of all included studies (50%, n = 9) were time motion or work sampling studies with direct observation techniques.[42] [43] [44] [45] [46] [47] [48] [49] [50] Other study designs were retrospective cohort and cross-sectional studies (44%, n = 8)[51] [52] [53] [54] [55] [56] [57] [58] and one descriptive qualitative study (6%, n = 1).[59] Of conducted studies, 17 were done in the United States,[42] [43] [44] [45] [46] [47] [48] [49] [51] [52] [53] [54] [55] [56] [57] [58] [59] and one in Argentina.[50] A total of 14 studies observed physicians and/or residents ([Table 1]),[43] [45] [46] [47] [48] [49] [51] [52] [53] [54] [55] [56] [57] [58] three studies observed nurses only ([Table 2]),[44] [50] [59] and one study observed physician assistants.[42] The outcome was measured by direct observation using a tablet-based time recorder or mobile devices,[42] [43] [44] [45] [47] [48] [49] direct observation using a worksheet,[50] and counting mouse clicks.[46] Other measurement tools included event logging[51] [52] [53] [54] [55] [56] [57] [58] and survey.[59] The approach that uses log tracking brought a significant advance to workflow analysis. All studies published since late 2018 and included in this review used EHR log data.

Table 2 Characteristics of included studies that describe a nurse's time on electronic health records

n	Study (Year)	Type of study	Setting and country	Type of staff	Computer system used	Tool for measurement	EHR tasks evaluated	Narrative summary	Major limitations
1	Cornell et al (2010)[44]	Time motion	Two hospitals (A and B) in the same health care system, medical/ surgical units; U.S.	Nurses	EHR with medication charting system	Direct observations[a] ranged from 1 to 4 h (240 h in total)	Electronic charting, electronic information retrieval and navigation, other computer use	At hospital A nurses spent 17.8% of time on electronic charting, 10.9% on electronic information retrieval and navigation, 1.1% on other computer use. At hospital B nurses spent 11.4% of time on electronic charting, 4.4% on electronic information retrieval and navigation, and 2.8% on other computer use. Overall, 29.8% of time was distributed to computer-related tasks at hospital A and 18.6% of time was allocated to computer-related tasks at hospital B.	EHR was not well described. The study lacks validity and reliability of measurement tool. Nearly half of all activities were observed less than 2% of the time.
2	Kossman et al (2008)[59]	Descriptive qualitative study	Two community hospitals within one healthcare system, medical-surgical floor and ICU; U.S.	Nurses	EHR (system is not mentioned)	Self-reported	Time on EHR	According to survey responses, nurses spent from 25 to 98% of their shift using the EHR, with a median of 50% and a mean of 56.6%.	The study has high risk of bias due to self-reported outcomes.
3	Schachner et al (2015)[50]	Work-sampling	An academic tertiary level hospital, ICU, intermediate care unit, general care unit; Argentina	Nurses	EHR, ITALICA	Direct observation	EHR use	Nurses spent 23.79% (SD = 3.65) of time on EHR use in ICU, 18.26% (SD = 2.93) in intermediate care unit, and 19.97% (SD = 2.17) in general care unit.	That was a pilot study that used a work sampling technique to assess time on EHR after redesigning electronic charts in the preimplementation phase. Time-motion is more reliable methodology than work sampling to assess specific task durations.[79]

Abbreviations: EHR, electronic health records; ICU, intensive care unit; SD, standard deviation.

^a Time of direct observation refers to postimplementation periods of pre-post design studies.

The majority of studies (n = 15, 83.3%) took place in inpatient settings, of which 10 studies were performed in academic and tertiary centers and university hospitals,[42] [43] [45] [49] [52] [53] [55] [56] [57] [58] and five studies were conducted in community and community teaching hospitals.[44] [46] [47] [50] [59] Three studies were performed in outpatient settings.[48] [51] [54] The setting studied varied between studies. Study settings focused on physicians including family medicine and ambulatory care (n = 3),[48] [51] [54] emergency department (n = 2),[45] [46] intensive care unit (ICU; n = 2),[43] [55] general surgery (n = 3),[52] [53] [56] and general/internal medicine service (n = 4).[47] [49] [57] [58] Physician assistants were studied in hematology and oncology settings (n = 1).[42] Nurse clinician study settings included medical/surgical/general care unit (n = 3),[44] [50] [59] progressive care unit (n = 1),[50] and adult ICU (n = 1).[59] Attending physicians and residents spent from 14% to 52% of their workday interacting with the EHR ([Fig. 2]). The average time allocated to EHR tasks by physicians was 36.6% of their workday. Nurses spent from 18 to 57% of time of their workday on EHR interaction ([Fig. 2]). The latter mean time was reported in a descriptive qualitative study performed at a community hospital to evaluate nurses' perception of EHR.[59] When aggregating data from time-motion studies[44] [50] that are less prone to bias, the average time allocated to EHR tasks by nurses was 22% of their shift.

Fig. 2 Time spent on interaction with electronic health record by clinicians. Blue bars represent nurses and orange bars represent physicians.

Quality Assessment

There was variation in the methodological quality of the included studies. The QATSDD score ranged from 10 to 32 (mean = 21.2; standard deviation [SD] = 6.49). Studies commonly received higher scores on items that referred to the description of research settings, data collection methods, and procedures. A lack of considerations of sample size and representativeness of a sample was given by authors of included studies. Observer training was explicitly described in several studies,[42] [43] [45] [47] [48] [49] of which three studies reported practicing observations with a clinician.[43] [45] [49] Reliability and validity of measurement tool were addressed in five studies[43] [45] [48] [49] [51]; and even fewer studies reported interrater reliability.[43] [48] [49] The observation time varied from 30 to 494 hours with a mean of 207 (SD = 161) hours.[42] [43] [44] [45] [46] [48] [49] [51] Time of interaction with the EHR after working hours or on weekends was assessed in a few studies by tracking logs,[51] [52] [53] by direct observation,[43] or by self-report.[45] [48] Out of nine studies that used direct observation techniques, five studies acknowledged the Hawthorne effect.[42] [43] [45] [48] [49]

Discussion

This review synthesized the results of published studies that estimated the amount of time physicians and nurses dedicate to EHR during their workday. The impact of EHR implementation on physicians' workflow may widely vary depending on their role and clinical settings, as well as type and usability of a computer system in the institution. Acknowledging these factors, we report on the overall high proportion of time a clinician spent interacting with the EHR.

This is particularly the case in outpatient settings. Family medicine physicians spent nearly half of their weekday using the EHR, as similarly reported in the studies that perform log-based analysis[51] and direct observation.[48] Apart from nondirect patient activities, physicians interact with the EHR during face-to-face patient communication. In ambulatory practice, at least one-third of the patient-dedicated time was spent interacting with the EHR.[28] [48] Similarly, Street et al showed that primary care providers spend 39.4% gazing at the EHR during a visit.[35]

In hospital settings, mean time of EHR interaction varied across medical and surgical specialties. Physicians and residents in surgical specialties spent on average 20% less time using the EHR than clinicians in other specialties, including internal medicine, critical care, and emergency medicine (17 vs. 37%). The workflow of interns or residents can be a good reflection of EHR-mediated workflow, as they started their professional career in the era of computerized medicine. In inpatient settings interns, as primary writers of clinical notes, dedicated more than 10 hours of a 24-hour period to EHR interaction.[60] Time-motion studies reported that two-thirds of intern time was allocated to indirect patient care, including 40 to 43% of the time accounted for computer tasks; while only 12 to 13% of time was spent on direct patient contact.[18] [60] Internal medicine residents are EHR top users. The time allocated to EHR was over 50% of resident's shift measured by time-motion method[47] and 44% of a shift calculated by logs data.[58] Unlike internal medicine residents, surgical residents spent less time interacting with EHR daily. However, weekly usage was 38% of actual duty hours.[52]

The mean time of EHR interaction by nurses might be underestimated due to the lack of included studies and short time period after EHR implementation. However, studies that reported documentation time on both electronic and paper charts showed a similar time range from 19 to 35% of total nurse activities.[20] [23] [24] [25] [61] There are lack of reports quantifying log-based time spent on EHR by nurses, compared with physicians. A recently published paper described the first study of this kind to analyze active EHR time spent by nurse practitioners (N.P.) and physician assistants (P.A.). Total daily time spent interacting with the EHR was significantly higher for NP/PA providers, compared with residents (176.5 vs. 152.3 minutes).[62] This demonstrates a greater burden for N.P./P.A. providers compared with resident physicians, as stated by the authors.

Clinical staff is becoming overwhelmed with the EHR burden not only during clinical hours, but also after work hours. After-work time varies from 42 minutes to 1.9 hours for physicians and residents depending on their specialty and setting. However, after-hours activities were generally observed in both outpatient- and inpatient settings regardless of specialty and EHR proficiency.[8] [51] [52] [63] [64] Survey studies showed that after-hours charting was significantly associated with physician burnout.[65]

Electronic health records may save time for information gathering and reviewing simply because clinicians spend less time looking for charts or patient information from disparate systems.[66] In particular, this reduction in time was significant in pre-rounding activities across the studies.[66] [67] However, the EHR was designed to capture and store data and lacks the ability to synthesize and communicate patient information in a user-friendly manner. High-intensity clinical work and time constrain predispose to using multiple encounters at once, resulting in a cognitive switch between patients. With EHR implementation, the number of activities and task switching have increased by 31% for residents and decreased by 23% for attending physicians.[43] The workflow of otolaryngology residents was more fragmented while using the EHR than while using the paper system.[17] Interaction time with the EHR is not a sufficient measure for workflow assessment as disruptions to clinical workflow associated with EHR are not captured.[68]

The average daily total logged time spent on “desktop medicine” was more than 50% of the physician day.[69] Apart from common EHR tasks such as authoring and viewing notes, desktop medicine includes other activities such as communicating with patients through patient portals, communicating with colleagues, searching for the literature, billing, and other clerical activities. However, of all computer activities, the most time consuming was the process of data entry.[46] [47] [49] [51]

The usability of clinical data interfaces is another major issue. Emergency medicine physicians spend more time on order entry and charting than direct contact with patients.[46] In particular, one study has shown that the ordinary electronic order of a 325-mg aspirin requires six mouse clicks.[46] The increase in electronic data availability does not necessarily mean an improvement in the way information is handled and used. International publications have also reported a poor state of EHR usability without any significant changes over time.[70] The log-based analysis has been more commonly used for assessing the time spent on the EHR within the last 3 years. These methods shed light on specific patterns of EHR interactions by various clinicians in health care. However, the time of EHR interaction is not the only indicator of usability. Many other characteristics such as failures of EHR implementation, level of EHR adoption of the different workflow that an electronic system requires, failures to utilize existing technology because of financial constraints, availability of mobile devices, implementation of health information technology tools, and organization-level changes of workflow were beyond the scope of this review.

There are several potential solutions to improve user interaction with the EHR. The voice technology and implementation of medical scribes may be useful in improving workflow productivity.[71] [72] [73] When medical scribes were implemented in the emergency department, the total time physicians spent interacting with the EHR decreased by approximately 30%.[45] User-friendly electronic note design may reduce clinician time spent reviewing and transferring information.[74] The use of visualization techniques such as dashboards are proposed to improve the summary of information and support the decision-making process.[75] User-centered designs are likely not the sole solution. Adapting the EHR to the current clinical context will likely contribute to usability and operability.

To the best of our knowledge, this is the first systematic review exploring time of interaction with the EHR by clinicians. The main focus of previous systematic reviews was to synthesize time on clinical documentation and to measure staff efficiency by comparing pre- and post-EHR periods.[76] [77] [78] These systematic reviews generally showed an increase of time spent on nurses and physicians documentation with the implementation of EHR and CPOE.[77] [78] Unlike our review, the reported time was allocated to both paper and electronic records. In our review, we aggregated data on EHR interaction time that can enhance understanding of the current clinician workflow in health care.

This study has several limitations. We did not examine the quality of documentation and did not quantify different modes of data entry such as typing, dictation, or scribes. There are gaps in our understanding, such as what proportion of time spent on the EHR was necessitated by clinical context or regulatory demands versus time that is simply wasteful. Other administrative tasks performed at the desktop were not explored.

Conclusion

This systematic review evaluates the time health care clinicians spend interacting with the EHR. Physicians spent 37% of their workday and nurses spent 22% of their workday interacting with the EHR. This finding is a possible indicator that the EHR has room for usability, functionality improvement, and workflow optimization.

Clinical Relevant Statement

This systematic review is the first of its kind to provide a narrative synthesis of the systematically selected studies reporting the amount of time spent on interaction with the EHR by clinicians. The systematic review summarizes the existing data on EHR interaction time, available within the past 30 years, and highlights a shift in the preferred methodology of workflow analysis from time-motion studies to EHR event log studies. The findings confirmed evidence that clinicians spend a significant time of their day interacting with EHR technology, regardless of settings and roles. The findings will guide future research directions aimed at improving clinician workflow organization.

Multiple Choice Questions

1. What approach has been most used in recent studies exploring the amount of time spent on interaction with EHR?

   • Time and motion analysis

   • EHR log-based analysis

   • Work-sampling technique

   • Survey

   Correct Answer: The correct answer is option d. The approach that uses log tracking brought a significant advance to workflow analysis. All studies, which had been published since late 2018 and were included in this review, used EHR log data.

2. How much time did physicians spend interacting with EHR per their working day in both inpatient and outpatient settings?

   • The average daily time on EHR was nearly 22% of their workday.

   • The average daily time on EHR was 15% of their workday.

   • The average daily time on EHR was nearly 50% of their workday.

   • The average daily time on EHR was nearly 37% of their workday.

   Correct Answer: The correct answer is option b. The average daily time spent on EHR interaction by physicians was nearly 37% of their workday in both inpatient and outpatient settings. Family medicine physicians spent approximately half of their weekday on EHR (answer c). Nurse clinicians spent interacting with EHR nearly 22% of their workday (answer a).

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects.

• References

• 1 Kruse CS, Kristof C, Jones B, Mitchell E, Martinez A. Barriers to electronic health record adoption: a systematic literature review. J Med Syst 2016; 40 (12) 252
• 2 Wilson K, Khansa L. Migrating to electronic health record systems: a comparative study between the United States and the United Kingdom. Health Policy 2018; 122 (11) 1232-1239
• 3 Bush RA, Kuelbs C, Ryu J, Jiang W, Chiang G. Structured data entry in the electronic medical record: perspectives of pediatric specialty physicians and surgeons. J Med Syst 2017; 41 (05) 75
• 4 Menachemi N, Collum TH. Benefits and drawbacks of electronic health record systems. Risk Manag Healthc Policy 2011; 4: 47-55
• 5 Shanafelt TD, Dyrbye LN, Sinsky C. et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clinic Proceedings. Accessed 2016 at: https://pubmed.ncbi.nlm.nih.gov/27313121/
• 6 Ehrenfeld JM, Wanderer JP. Technology as friend or foe? Do electronic health records increase burnout?. Curr Opin Anaesthesiol 2018; 31 (03) 357-360
• 7 Babbott S, Manwell LB, Brown R. et al. Electronic medical records and physician stress in primary care: results from the MEMO Study. J Am Med Inform Assoc 2014; 21 (e1): e100-e106
• 8 Kroth PJ, Morioka-Douglas N, Veres S. et al. The electronic elephant in the room: physicians and the electronic health record. JAMIA Open 2018; 1 (01) 49-56
• 9 Khairat S, Xi L, Liu S, Shrestha S, Austin C. Understanding the Association Between Electronic Health Record Satisfaction and the Well-Being of Nurses: survey study. JMIR Nursing. 2020; 3 (01) e13996
• 10 Harris DA, Haskell J, Cooper E, Crouse N, Gardner R. Estimating the association between burnout and electronic health record-related stress among advanced practice registered nurses. Appl Nurs Res 2018; 43: 36-41
• 11 Gawande A. Why doctors hate their computers. Published 2018. Accessed December 7, 2018 at: https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers
• 12 Technology OWoTOotNCfHI. Strategy on Reducing Burden Relating to the Use of Health IT and EHRs. Published 2018. Accessed December, 2018 at: https://www.healthit.gov/topic/usability-and-provider-burden/strategy-reducing-burden-relating-use-health-it-and-ehrs 2018
• 13 Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009; 62 (10) 1006-1012
• 14 Evans RS. Electronic health records: then, now, and in the future. Yearb Med Inform 2016; (Suppl. 01) S48-S61
• 15 Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract 2012; 18 (04) 746-752
• 16 Lambe KA, Lydon S, Madden C. et al. Hand hygiene compliance in the ICU: a systematic review. Crit Care Med 2019; 47 (09) 1251-1257
• 17 Victores AJ, Coggins K, Takashima M. Electronic health records and resident workflow: a time-motion study of otolaryngology residents. Laryngoscope 2015; 125 (03) 594-598
• 18 Block L, Habicht R, Wu AW. et al. In the wake of the 2003 and 2011 duty hours regulations, how do internal medicine interns spend their time?. J Gen Intern Med 2013; 28 (08) 1042-1047
• 19 Asaro PV, Boxerman SB. Effects of computerized provider order entry and nursing documentation on workflow. Acad Emerg Med 2008; 15 (10) 908-915
• 20 Wong DH, Gallegos Y, Weinger MB, Clack S, Slagle J, Anderson CT. Changes in intensive care unit nurse task activity after installation of a third-generation intensive care unit information system. Crit Care Med 2003; 31 (10) 2488-2494
• 21 Qian S, Yu P, Hailey DM. The impact of electronic medication administration records in a residential aged care home. Int J Med Inform 2015; 84 (11) 966-973
• 22 Munyisia EN, Yu P, Hailey D. Does the introduction of an electronic nursing documentation system in a nursing home reduce time on documentation for the nursing staff?. Int J Med Inform 2011; 80 (11) 782-792
• 23 Korst LM, Eusebio-Angeja AC, Chamorro T, Aydin CE, Gregory KD. Nursing documentation time during implementation of an electronic medical record. J Nurs Adm 2003; 33 (01) 24-30
• 24 Hakes B, Whittington J. Assessing the impact of an electronic medical record on nurse documentation time. Comput Inform Nurs 2008; 26 (04) 234-241
• 25 Banner L, Olney CM. Automated clinical documentation: does it allow nurses more time for patient care?. Comput Inform Nurs 2009; 27 (02) 75-81
• 26 Carayon P, Smith P, Hundt AS, Kuruchittham V, Li Q. Implementation of an electronic health records system in a small clinic: the viewpoint of clinic staff. Behav Inf Technol 2009; 28 (01) 5-20
• 27 Hripcsak G, Vawdrey DK, Fred MR, Bostwick SB. Use of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc 2011; 18 (02) 112-117
• 28 Joukes E, Abu-Hanna A, Cornet R, de Keizer NF. Time spent on dedicated patient care and documentation tasks before and after the introduction of a structured and standardized electronic health record. Appl Clin Inform 2018; 9 (01) 46-53
• 29 Edsall DW, Deshane P, Giles C, Dick D, Sloan B, Farrow J. Computerized patient anesthesia records: less time and better quality than manually produced anesthesia records. J Clin Anesth 1993; 5 (04) 275-283
• 30 Wormer BA, Colavita PD, Yokeley WT. et al. Impact of implementing an electronic health record on surgical resident work flow, duty hours, and operative experience. Am Surg 2015; 81 (02) 172-177
• 31 Wang Y, Tian Y, Tian LL, Qian YM, Li JS. An electronic medical record system with treatment recommendations based on patient similarity. J Med Syst 2015; 39 (05) 55
• 32 Keshavjee K, Troyan S, Holbrook AM, VanderMolen D. COMPLETE Investigators. Measuring the success of electronic medical record implementation using electronic and survey data. Proc AMIA Symp 2001; 309-313
• 33 Kannampallil TG, Denton CA, Shapiro JS, Patel VL. Efficiency of emergency physicians: insights from an observational study using EHR log files. Appl Clin Inform 2018; 9 (01) 99-104
• 34 Chen L, Guo U, Illipparambil LC. et al. Racing against the clock: internal medicine residents' time spent on electronic health records. J Grad Med Educ 2016; 8 (01) 39-44
• 35 Street Jr RL, Liu L, Farber NJ. et al. Provider interaction with the electronic health record: the effects on patient-centered communication in medical encounters. Patient Educ Couns 2014; 96 (03) 315-319
• 36 Street Jr RL, Liu L, Farber NJ. et al. Keystrokes, mouse clicks, and gazing at the computer: how physician interaction with the EHR affects patient participation. J Gen Intern Med 2018; 33 (04) 423-428
• 37 Shabbir SA, Ahmed LA, Sudhir RR, Scholl J, Li YC, Liou DM. Comparison of documentation time between an electronic and a paper-based record system by optometrists at an eye hospital in south India: a time-motion study. Comput Methods Programs Biomed 2010; 100 (03) 283-288
• 38 Read-Brown S, Sanders DS, Brown AS. et al. Time-motion analysis of clinical nursing documentation during implementation of an electronic operating room management system for ophthalmic surgery. AMIA Annu Symp Proc 2013; 2013: 1195-1204
• 39 Read-Brown S, Hribar MR, Reznick LG. et al. Time requirements for electronic health record use in an academic ophthalmology center. JAMA Ophthalmol 2017; 135 (11) 1250-1257
• 40 Zoghbi V, Caskey RC, Dumon KR. et al. “How to” videos improve residents performance of essential perioperative electronic medical records and clinical tasks. J Surg Educ 2018; 75 (02) 489-496
• 41 Neri PM, Redden L, Poole S. et al. Emergency medicine resident physicians' perceptions of electronic documentation and workflow: a mixed methods study. Appl Clin Inform 2015; 6 (01) 27-41
• 42 Hanauer DA, Zheng K, Commiskey EL, Duck MG, Choi SW, Blayney DW. Computerized prescriber order entry implementation in a physician assistant-managed hematology and oncology inpatient service: effects on workflow and task switching. J Oncol Pract 2013; 9 (04) e103-e114
• 43 Carayon P, Wetterneck TB, Alyousef B. et al. Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit. Int J Med Inform 2015; 84 (08) 578-594
• 44 Cornell P, Riordan M, Herrin-Griffith D. Transforming nursing workflow, part 2: the impact of technology on nurse activities. J Nurs Adm 2010; 40 (10) 432-439
• 45 Heaton HA, Wang R, Farrell KJ. et al. Time motion analysis: impact of scribes on provider time management. J Emerg Med 2018; 55 (01) 135-140
• 46 Hill Jr RG, Sears LM, Melanson SW. 4000 clicks: a productivity analysis of electronic medical records in a community hospital ED. Am J Emerg Med 2013; 31 (11) 1591-1594
• 47 Mamykina L, Vawdrey DK, Hripcsak G. How do residents spend their shift time? A time and motion study with a particular focus on the use of computers. Acad Med 2016; 91 (06) 827-832
• 48 Sinsky C, Colligan L, Li L. et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
• 49 Tipping MD, Forth VE, O'Leary KJ. et al. Where did the day go? A time-motion study of hospitalists. J Hosp Med 2010; 5 (06) 323-328
• 50 Schachner MB, Recondo FJ, Sommer JA. et al. Preimplementation study of a nursing e-chart: how nurses use their time. Stud Health Technol Inform 2015; 216: 255-258
• 51 Arndt BG, Beasley JW, Watkinson MD. et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (05) 419-426
• 52 Cox ML, Farjat AE, Risoli TJ. et al. Documenting or operating: where is time spent in general surgery residency?. J Surg Educ 2018; 75 (06) e97-e106
• 53 Cox ML, Risoli Jr T, Peskoe SB, Turner DA, Migaly J. Quantified electronic health record (EHR) use by academic surgeons. Surgery 2021; 169 (06) 1386-1392
• 54 Giliberto JP, Ator G, Carroll TL, Chan T, Vahabzadeh-Hagh A. National trends in daily ambulatory electronic health record use by otolaryngologists. Laryngoscope 2021; 131 (05) 975-981
• 55 Krawiec C, Haidet P. The impact of patient census on PICU attending electronic health record utilization. Critical Care Medicine Conference: 48th Critical Care Congress of the Society of Critical Care Medicine, SCCM; 2019;47(1 Supplement 1). Accessed 2019 at: https://pubmed.ncbi.nlm.nih.gov/32215894/
• 56 Maloney SR, Peterson S, Kao AM, Sherrill WC, Green JM, Sachdev G. Surgery resident time consumed by the electronic health record. J Surg Educ 2020; 77 (05) 1056-1062
• 57 Verma G, Ivanov A, Benn F. et al. Analyses of electronic health records utilization in a large community hospital. PLoS One 2020; 15 (07) e0233004 [Electronic Resource]
• 58 Wang JK, Ouyang D, Hom J, Chi J, Chen JH. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS One 2019; 14 (02) e0205379
• 59 Kossman SP, Scheidenhelm SL. Nurses' perceptions of the impact of electronic health records on work and patient outcomes. Comput Inform Nurs 2008; 26 (02) 69-77
• 60 Chaiyachati KH, Shea JA, Asch DA. et al. Assessment of inpatient time allocation among first-year internal medicine residents using time-motion observations. JAMA Intern Med 2019; 179 (06) 760-767
• 61 Walker RM, Burmeister E, Jeffrey C. et al. The impact of an integrated electronic health record on nurse time at the bedside: a pre-post continuous time and motion study. Collegian 2020; 27 (01) 63-74
• 62 Watson MD, Elhage SA, Scully C. et al. Electronic health record usage among nurse practitioners, physician assistants, and junior residents. J Am Assoc Nurse Pract 2020; 33 (03) 200-204
• 63 Haidar YM, Moshtaghi O, Mahboubi H. et al. Association between electronic medical record implementation and otolaryngologist productivity in the ambulatory setting. JAMA Otolaryngol Head Neck Surg 2017; 143 (01) 20-24
• 64 Wenger N, Méan M, Castioni J, Marques-Vidal P, Waeber G, Garnier A. Allocation of internal medicine resident time in a Swiss hospital: a time and motion study of day and evening shifts. Ann Intern Med 2017; 166 (08) 579-586
• 65 Eschenroeder Jr HC, Manzione LC, Adler-Milstein J. et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc 2021; 28 (05) 960-966
• 66 Amusan AA, Tongen S, Speedie SM, Mellin A. A time-motion study to evaluate the impact of EMR and CPOE implementation on physician efficiency. J Healthc Inf Manag 2008; 22 (04) 31-37
• 67 Kochendorfer KM, Morris LE, Kruse RL, Ge BG, Mehr DR. Attending and resident physician perceptions of an EMR-generated rounding report for adult inpatient services. Fam Med 2010; 42 (05) 343-349
• 68 Patel VL, Denton CA, Soni HC, Kannampallil TG, Traub SJ, Shapiro JS. Physician workflow in two distinctive emergency departments: an observational study. Appl Clin Inform 2021; 12 (01) 141-152
• 69 Tai-Seale M, Olson CW, Li J. et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017; 36 (04) 655-662
• 70 Kamil RJ, Giddings N, Hoffer M. et al. Electronic health record use among American Neurotology Society members. Otol Neurotol 2018; 39 (09) e876-e882
• 71 Kumah-Crystal YA, Pirtle CJ, Whyte HM, Goode ES, Anders SH, Lehmann CU. Electronic health record interactions through voice: a review. Appl Clin Inform 2018; 9 (03) 541-552
• 72 McCormick BJ, Deal A, Borawski KM. et al. Implementation of medical scribes in an academic urology practice: an analysis of productivity, revenue, and satisfaction. World J Urol 2018; 36 (10) 1691-1697
• 73 Tran BD, Chen Y, Liu S, Zheng K. How does medical scribes' work inform development of speech-based clinical documentation technologies? A systematic review. J Am Med Inform Assoc 2020; 27 (05) 808-817
• 74 Belden JL, Koopman RJ, Patil SJ, Lowrance NJ, Petroski GF, Smith JB. Dynamic electronic health record note prototype: seeing more by showing less. J Am Board Fam Med 2017; 30 (06) 691-700
• 75 Dowding D, Merrill JA, Barrón Y, Onorato N, Jonas K, Russell D. Usability evaluation of a dashboard for home care nurses. Comput Inform Nurs 2019; 37 (01) 11-19
• 76 Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc 2005; 12 (05) 505-516
• 77 Baumann LA, Baker J, Elshaug AG. The impact of electronic health record systems on clinical documentation times: a systematic review. Health Policy 2018; 122 (08) 827-836
• 78 Moore EC, Tolley CL, Bates DW, Slight SP. A systematic review of the impact of health information technology on nurses' time. J Am Med Inform Assoc 2020; 27 (05) 798-807
• 79 Lopetegui M, Yen P-Y, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about?. J Biomed Inform 2014; 49: 292-299

Publikationsverlauf

Eingereicht: 04. Mai 2021

Angenommen: 03. Juli 2021

Artikel online veröffentlicht:
25. August 2021