CC BY-NC-ND 4.0 · Methods Inf Med 2017; 56(S 01): e92-e104
DOI: 10.3414/ME17-05-0002
Original Articles
Schattauer GmbH

Quality Requirements for Electronic Health Record Systems[*]

A Japanese-German Information Management Perspective
Alfred Winter
1   Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
,
Katsuhiko Takabayashi
2   Department of Medical Information and Management, Chiba University Hospital, Chiba, Japan
,
Franziska Jahn
1   Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
,
Eizen Kimura
3   Department of Medical Informatics, Ehime Daigaku Daigakuin Igakukei Kenkyuka Igakubu, Toon, Ehime, Japan
,
Rolf Engelbrecht
4   National ProRec Centre, Ismaning, Germany
,
Reinhold Haux
5   Peter L. Reichertz Institute for Medical Informatics, University of Braunschweig and Hannover Medical School, Braunschweig, Germany
,
Masayuki Honda
6   Division of Medical Informatics, Nagasaki Daigaku Byoin, Nagasaki, Japan
,
Ursula H. Hübner
7   Health Informatics Research Group, Hochschule Osnabrück, Department of Business Management and Social Sciences, Osnabrück, Germany
,
Sozo Inoue
8   Kyushu Kogyo Daigaku, Kitakyushu, Fukuoka, Japan
,
Christian D. Kohl
9   Medical Informatics, Institue of Medical Biometry and Informatics, Heidelberg, Germany
,
Takehiro Matsumoto
10   Nagasaki Daigaku, Nagasaki, Japan
,
Yasushi Matsumura
11   Medical Informatics, Osaka University Graduate School of Medicine, Osaka, Japan
,
Kengo Miyo
12   Kokuritsu Kokusai Iryo Center Kenkyujo, Shinjuku-ku, Tokyo, Japan
,
Naoki Nakashima
13   Medical Information Center, Kyushu University Hospital, Higashi-ku, Fukuoka, Japan
,
Hans-Ulrich Prokosch
14   Department of Medical Informatics, Biometry and Epidemiology, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
,
Martin Staemmler
15   Fachhochschule Stralsund, Stralsund, Germany
› Author Affiliations
Funding: This work has been supported by Deutsche Forschungsgemeinschaft (DFG) grant no. WI 1605 08-1 and by Leipzig University within the DFG funded program of Open Access Publishing.
Further Information

Correspondence to

Prof. Alfred Winter
University of Leipzig
Institute for Medical Informatics, Statistics
and Epidemiology
Haertelstr. 16-18
04107 Leipzig
Germany

Publication History

received: 23 January 2017

accepted: 13 June 2017

Publication Date:
31 January 2018 (online)

 

Summary

Background: For more than 30 years, there has been close cooperation between Japanese and German scientists with regard to information systems in health care. Collaboration has been formalized by an agreement between the respective scientific associations. Following this agreement, two joint workshops took place to explore the similarities and differences of electronic health record systems (EHRS) against the background of the two national healthcare systems that share many commonalities.

Objectives: To establish a framework and requirements for the quality of EHRS that may also serve as a basis for comparing different EHRS.

Methods: Donabedian’s three dimensions of quality of medical care were adapted to the outcome, process, and structural quality of EHRS and their management. These quality dimensions were proposed before the first workshop of EHRS experts and enriched during the discussions.

Results: The Quality Requirements Framework of EHRS (QRF-EHRS) was defined and complemented by requirements for high quality EHRS. The framework integrates three quality dimensions (outcome, process, and structural quality), three layers of information systems (processes and data, applications, and physical tools) and three dimensions of information management (strategic, tactical, and operational information management).

Conclusions: Describing and comparing the quality of EHRS is in fact a multidimensional problem as given by the QRF-EHRS framework. This framework will be utilized to compare Japanese and German EHRS, notably those that were presented at the second workshop.


#

1. Introduction and Objectives

Medical informatics has a considerable responsibility “for improving the health of people, through its contributions to high-quality, efficient health care and to innovative research in biomedicine and related health and computer sciences.” [[1]]. High-quality health care is thereby moving from isolated procedures in a single care delivery organization (CDO, e.g. a hospital or a general practice) to patient-oriented care processes spreading over institutional boundaries. Therefore, not only isolated patient records in CDOs are required. Moreover, health records covering the entire patient history encompassing all of the visited CDOs are needed. Electronic versions, i.e. electronic health record systems (EHRS), are under development at least since 1970 [[2]].

Therefore, it is obvious that the EHRS is a major cornerstone for “improving the health of people”. Due to this importance, high quality EHRS are needed. Unfortunately, describing the quality of EHRS is difficult, especially due to the following problems:

  • An EHRS is a complex system.

  • Describing quality of different EHRS in a comparable way presumes a uniform terminology for describing EHRS and their components and presumes a clear notion of quality.

Describing the quality of EHRS and thereby the effectivity of the resources engaged and comparing different EHRS would be beneficial for various reasons [[3]]:

  • Pragmatic reasons: Professionals such as information system architects, consultants, or scientists could identify more or less effective architectural styles and ways of information management for EHRS.

  • Promotional reasons: Quality assessments confirming the use of certain EHRS to be safe, time and cost efficient, or effective may improve the acceptance among the users.

  • Financial reasons: CFOs of health care institutions and networks try to minimize costs while ensuring the effectiveness of EHRS. Therefore, a clear and comparable description of EHRS’ quality is needed.

  • Scholarly reasons: Medical informatics students shall learn how to construct and manage effective and efficient EHRS.

  • Scientific reasons: From a scientific point of view, the question is what type of architecture and management of EHRS is most efficient for a certain institutional, regional, national or cultural background.

Consequently, having a clear notion of EHRS quality helps to construct more effective and efficient EHRS and may, therefore, contribute to better support of EHRS users and, above all, to better patient care.

Therefore, the paper’s objective is to compile the quality requirements for EHRS by providing

  1. a multidimensional framework for systematizing the requirements for the quality of EHRS;

  2. a set of requirements for quality of EHRS, which are structured by the multidimensional framework.

This paper summarizes the joint efforts of German and Japanese scientists to address these objectives. The endeavor is embedded in a 30 year collaboration of the German Association for Medical Informatics, Biometry and Epidemiology (GMDS) and the Japan Association for Medical Informatics (JAMI), which has been formally confirmed at the MEDINFO 2013 conference (The Memorandum of Understanding between JAMI and GMDS is attached as an ►Online Appendix).

The paper will first provide definitions for the most important concepts for describing EHRS and information management and apply the concepts of quality to EHRS. After that, the commonalities of the German and Japanese healthcare systems as well as differences in the way of using specific information technologies for health are discussed as part of the introduction of German/Japanese workshops. They were held to collect requirements for the quality of EHRS. Initiated by these workshops, a multidimensional framework for describing quality of EHRS was developed and used to systematize the requirements. The framework and requirements will be presented in the results section.

The present paper will close with some conclusions and recommendations and a discussion of its methodological limitations.

The authors explicitly exploit the German/Japanese workshops and, therefore, focus on the papers published by the participants. They consider the results as typical for a German and Japanese point of view to EHRS.


#

2. Methods

2.1 Concepts Used

2.1.1 Electronic Health Record System

The patient’s electronic medical record (EMR) contains the data that documents health care delivery within a particular inpatient or outpatient CDO [[4]]. The electronic health record (EHR), however, is the record containing all of the individual’s information regarding “wellness, health and healthcare […]. Its primary purpose is the support of life-long, effective, high quality and safe integrated health care” [[5]]. Therefore, it is not only “relevant to a subject’s medical treatment but also to a subject’s health in general” [[7]]. Therefore, the EHR comprises health data from various CDOs and may be “expanded to a national or global scale” [[6]]. It is a valuable source not only for clinical research [[8]] but also for health service and epidemiological research, e.g. on intractable diseases [[9]].

The electronic health record system (EHRS) is “a system for recording, retrieving and manipulating information in electronic health records” [[10]]. It comprises different databases and application systems, data repositories, directory services listing human and other resource entities, knowledge services containing terminological systems, care pathways and workflows, end user applications, reporting modules, security and privacy services [[5]], and respective hardware and network components.

In this sense, an EHRS is not one single software product that can be purchased in a shop. Moreover, it is a unique on-site configuration of different integrated application systems, computers, and network components spanning various CDOs.


#

2.1.2 Information System

Given a CDO, its information system can be defined as that part of the CDO that processes and stores data, information, and knowledge. It consists of different application systems, computers, and network components. In case of networked CDOs instead of a single CDO, the network’s information system is called a “transinstitutional Health Information System” (tHIS) that enables information processing and exchange in this network [[11]].

Consequently, an EHRS can be considered a sub information system of a tHIS. Using the three layer graph based meta model 3LGM2 for modeling health information systems [[12]], especially transinstitutional information systems and, therefore, an EHRS can be described by concepts on three layers [[13]]:

  • The domain layer describes an EHRS independently of its implementation by the tasks and processes it supports (e.g. “medical and nursing care planning”). Processes need certain data and provide data for other processes.

  • The logical tool layer concentrates on application components supporting processes. Application components are responsible for the processing, storage, and transfer of data. Computer-based application components are installed software. Interfaces ensure the communication among application components and, therefore, enable their integration.

  • The physical tool layer consists of physical data processing systems (e.g. personal computers, servers, switches, routers, etc.), which are connected in a network.

This approach for structuring and describing information systems turned out to be an appropriate basis for not only describing but also comparing hospital information systems in Japan and Germany [[14]].


#

2.1.3 Information Management

Information management is responsible for planning information systems in general and, therefore, for planning an EHRS and its architecture [[11]], directing its establishment and its operation, and monitoring its development and operation.

Information management is a complex task. In order to reduce complexity, the perspectives

  • strategic information management,

  • tactical information management, and

  • operational information management can be distinguished.

Information management needs an organizational structure that is described by

  • IT-governance (e.g. [[15]]).

Providing a good information system by information management is an intangible service to the stakeholders in a CDO and a health care network. This kind of service is called IT service; therefore,

  • IT-service-management

is also an important concept for describing information management [[16]].


#

2.1.4 Quality of Electronic Health Record System (EHRS)

As a (sub-) information system, an EHRS is a product resulting from information management, which can be considered the manufacturing process of EHRS. The quality of a product is defined as the degree to which it fulfills its consumer’s needs and expectations (ISO 9000 from [[11]]). Quality in this regard is also called “primary quality” [[17]]. Consumers of EHRS are end-users like the doctors, nurses, technicians, administrators, managers, students, researchers and, last but not least, the patients themselves. Since the EHRS is an information system it has to fulfill the following expectations ([[11]] p.34):

  1. Data, information and knowledge logistics: The EHRS has to capture and make available data and information, primarily about patients, [[18]] and knowledge, for example, about diseases, side effects, and interactions of medications to support diagnostics and therapy.

  2. Process support: The EHRS has to sufficiently enable the adequate execution of processes for patient care, supply and disposal management, scheduling, and resource allocation, administration, leadership and governance and research and education.

However, thoroughly describing the quality of a complex system requires considering more dimensions. Donabedian did this when describing the quality of medical care by introducing outcome, process, and structural quality [[19], [20]]. This methodological approach can be adapted for describing the quality of information systems, in general, and EHRS, in particular. Since an EHRS is the outcome of information management, we can, therefore, define the quality of an EHRS from the information management perspective. Thus, the primary quality of EHRS is an outcome quality of information management, quality of information management corresponds to Donabedian’s process quality, and structural quality is about resources for information management:

  • Primary Quality of EHRS or Outcome Quality of information management:

    According to the aforementioned ISO definition of quality, primary quality of EHRS is the degree to which it is able to provide information and knowledge logistics and process support. For measuring this ability, the components of the EHRS on the domain layer, logical tool layer, and physical tool layer need to be considered.

    In Medical Informatics, we have the hypothesis that good primary quality of EHRS will contribute to high outcome quality in medical care. However, the effect of EHRS components on medical outcome has to be evaluated and proven systematically [[21]].

  • Process Quality of information management:

    In order to achieve high quality of a product, the processes for manufacturing the product also must be considered. As stated before, the EHRS is not a single software product but rather a unique and site dependent information system. Therefore, it is the result of information management on site. The quality of information management is the degree to which information management processes in an institution are able to manufacture a high quality EHRS. This aspect of quality is comparable to “process quality” [[19], [20]]. Usually, good primary quality of EHRS is not achievable without good quality of information management processes.

  • Structural Quality of information management: Donabedian’s third dimension of quality is “structural quality” [[19]]. Structural quality of information processing is the quality of the tools, resources, and organizational structures available for information management, i.e. the degree to which these tools and resources are able to provide high quality information management. Tools and resources may be well-trained information management staff and information management tools for project management, enterprise architecture modeling, etc. Usually, good process quality of information management is not achievable without good structural quality, i.e. good quality of resources for information management.

This concept of EHRS quality focusses on the socio-technical system consisting of the EHRS and its information management. Therefore, proving the effect of EHRS components on medical outcome is out of the scope of this paper. However, we dock to an approach of Ancker et al. [[22]], focusing more on the socio-technical system consisting of EHRS and medical care processes. Based on Donabedian’s model, they explore potential variables in the socio-technical environment of EHRS being able to influence positive or negative medical outcome of IT use. Focusing on medical outcome, they identify “technology”, “organization”, “provider”, and “patient” as structural variables. The variables “technology”, “organization”, and “provider” compose a triangle covering “patient” in its center. The process dimension deals with the processes between these variables. EHRS and its primary quality as defined here is covered by the structural variable “technology”, whereas ‘information management’ with its process and structural quality are covered by the structural variable “organization” and the processes between “organization” and “technology”.


#
#

2.2 Japanese-German Experts Workshops on the Quality of EHRS

The first milestone in developing quality requirements for EHRS was JGEHRS 2014, a four days Japanese-German expert workshop. The workshop took place at Kloster Banz in Bad Staffelstein (Bavaria, Germany) from June 30 to July 3, 2014. Eight Japanese and nine German scientists joined the workshop (►[Figure 1]). Based on the aforementioned concepts, the participants started

  1. developing requirements usable for assessing quality of an EHRS and

  2. agreed on a framework for systematizing the requirements.

Zoom Image
Figure 1 Workshop participants at Kloster Banz.

The particular added value of such a joint endeavor of German and Japanese experts in the domain of EHRS has already been proven in a Japanese-German comparison of hospital information systems that was possible because of comparable domain layers in Japanese and German hospitals [[14], [23]]. In addition, both countries are facing a similar challenge of demographic change leading to nearly the same requirements for information systems in the health domain. However, there are differences in the way of using specific information technologies for health. Furthermore, the development history of EHRS was different between the two countries. Germany as well as other European countries started EHRS development before having established electronic medical records (EMR) completely in the participating CDOs, while Japanese focused on the completion of paperless EMR systems in the CDOs before starting EHRS to connect these EMRs [[24]]. These differences provide a sound reason to reassess the current procedures in the own country, to become familiar with other approaches, and to research new ways of implementation and usage. In particular, this is of special interest for EHRS, since these follow different concepts and deployment worldwide.

Well prepared by the long-standing unique tradition of German-Japanese collaboration, we took the opportunity to bring together not only EHRS experts from different cultural backgrounds, but also with different professional training and practice.

Before the workshop, the participants were asked to prepare presentations dealing with criteria for the quality of EHRS. There were no further restrictions on what particular aspects of EHRS should be stressed and what kind of quality should be taken into account. Using this open approach, we intended to collect a broad as well as thorough collection of requirements for ‘good’ EHRS. In order to be able to arrange this collection in a systematic and comprehensible way, the workshop followed three steps:

  1. Collecting examples of EHRS in Germany and Japan:

    EHRS from Germany and Japan were presented and analyzed with respect to architecture, use of standards, EHRS role in a region, functionality, usage, and level of patient centricity. This revealed national particularities and cross-national communalities.

  2. Identification of quality requirements:

    Based on these examples relevant concepts and general requirements for quality of EHRS were identified. In moderated discussion panels, agreement on importance of these quality requirements has been reached by different consensus techniques. Section 3 summarizes the results and complements them with references to publications of the participants and others.

  3. Organizing the requirements in a framework:

    Participants examined the definitions of EHRS, information systems, information management and quality as introduced before and agreed in using them as a basis for a framework for organizing the requirements. This framework, which is used for structuring section 3, is described in section 3.1.

The workshop made use of various interactive methods, in particular presentations of EHRS examples and concepts followed by discussions, creativity techniques (meta-plan, brainstorming and –writing) (►[Figure 2]) and a summary of the workshop in a final statement by all workshop participants.

Zoom Image
Figure 2 Results of the creativity techniques on the quality issues of EHRS.

The second milestone for the quality requirements development was the workshop JGEHRS 2016 attached to the HEC2016 conference in Munich. Further practical examples of EHRS in Japan and German were presented and discussed reflecting the Quality Requirements Framework of EHRS (QRF-EHRS) (c.f. section 3.1).


#
#

3. Results

As a result of applying the concepts of section 2.1 in JGEHRS 2014, we present (1) the hierarchical framework for systematizing the quality requirements of EHRS and (2) the requirements for the quality of EHRS as they were carved out during the workshop.

3.1 QRF-EHRS: The Quality Requirements Framework of EHRS

According to section 2.1.4, the quality of EHRS has three major dimensions: primary quality of the product (outcome of information management), quality of information management processes, and structural quality of information management.

  • Since primary quality is the quality of the EHRS itself it can further be refined using the three layers of EHRS as introduced in section 2.1.2.

  • Information management has three perspectives as introduced in section 2.1.3. These perspectives can be used to further refine quality of information management.

  • Structural quality deals with available tools and resources as has been mentioned in section 2.1.4. We consider humans and money as the most important resources for information management.

Consequently, we use the hierarchical Quality Requirements Framework QRF-EHRS defined in ►[Box 1] for ordering the requirements for quality of EHRS in chapter 3.2.

Box 1

Quality requirements framework of EHRS (QRF-EHRS)

PrimQ: Primary quality

  • PrimQ.0: General requirements

  • PrimQ.1: Requirements for quality of the EHRS domain layer: tasks, processes and data

  • PrimQ.2: … of the EHRS logical tool layer: application components and their integration

  • PrimQ.3: … of the EHRS physical tool layer: computers, devices and their integration

IMQ: Quality of information management

  • IMQ.0: General requirements

  • IMQ.1: Requirements for quality of strategic information management

  • IMQ.2: … of tactical information management

  • IMQ.3: … of operational information management

  • IMQ.4: … of IT service management

  • IMQ.5: … of IT governance

StrQ: Structural quality

  • StrQ.0 General requirements

  • StrQ.1 Requirements for quality of Human resources

  • StrQ.2 … of financial resources

  • StrQ.3 … of tools for information management


#

3.2 Requirements for Quality of EHRS

In this section, we present a selection of requirements that should be considered when assessing the quality of EHRS in the respective dimension. The requirements are based on the experience of experts in the workshop.

3.2.1 (PrimQ) Primary Quality of an EHRS

Primary quality of EHRS deals with the very properties of the EHRS. At the domain layer, the value of an EHRS for the stakeholders is described. At the tool layers, the quality of the architectural foundations is described.

(PrimQ.1) Requirements for the quality of the EHRS domain layer: tasks, processes, and data

The EHRS has to sufficiently enable the adequate execution of tasks and processes for executing and supporting patient care, but also for research and education.

While discussing the primary quality of EHRS, the experts workshops focused on the EHRS domain layer. Japanese and German experts especially wanted to emphasize that an EHRS shall support different stakeholders and users at different sites and shall help integrating medical care and research.

Especially the following requirements must be fulfilled (see ►[Box 2]):

Box 2

(PrimQ.1) Requirements for quality of the EHRS domain layer: tasks, processes and data

  • Continuity of Care from home to hospital and resident doctors, even in disaster

  • Patient empowerment

  • Decision support

  • Medical research and education

  • Efficiency and quality of medical processes

  • Data privacy management and trustee services

  • Data quality management and curation

  • Semantic annotation

Continuity of care from home to hospital and resident doctors, even in disaster

Since lifestyles are changing and the need for mobility in career terms in Japan as well as in Germany is increasing, the EHRS must facilitate the more complex care processes dealing with many doctors and different pharmacies related to one patient [[25]]. EHRS must support data sharing and access rights management processes for patient related medical data such that all health care professionals within a community or region can use the data if they are entitled to use it [[26]]. Especially Japan has painful experiences with disasters like tsunamis and earthquakes; EHRS have to enable the continuity of care in these cases as well. However, the very basis for continuity of care is a unique national patient identification number [[6]].


#

Patient empowerment

According to recent developments in ambient assisted living and health-enabling technologies [[27]] the citizen’s home will be the new diagnostic and therapeutic entity. Therefore, EHRS must become patient centered systems supporting patient’s processes at home and in daily life including home care, consecutive care at rehabilitation and welfare facilities, care for elderly, community health campaigns, and compliance check of medicines. Moreover, EHRS must support patient empowerment in general [[28]] and especially in having secure access to their data which must be presented in an appropriate form, i.e. such that patients can understand the data.


#

Decision support

The EHRS should provide efficient means to support clinicians in their daily diagnostic and therapeutic decisions. In order to avoid medical errors, in general, and especially in the medication processes, the EHRS shall combine patient related information of the EHR with medical knowledge out of a knowledge base [[29], [30]].


#

Medical research and education

The EHRS must enable integrated medical research and care. Therefore, the secondary use of clinical data is as crucial as the integration of research facilities like the management of biobanks (31). The EHRS shall help to find appropriate cohorts for epidemiological research, e.g. on intractable diseases [[9]] or diabetes mellitus [[28]]. Preconditions are minimal data item sets being documented for a sufficient number of individuals. All data collected and provided this way is a valuable source for the medical education of medical professionals after graduation.


#

Efficiency and quality of the medical processes

Good EHRS must support clinical information and knowledge logistics and clinical workflows. The quality of clinical processes and workflows is, therefore, a core requirement for the quality of EHRS. Sophisticated methods for the measurement of workflow quality like process benchmarking [[32]], the workflow composite score [[18]] as well as the degree of adoption of EHRS features in real processes [[33]] are crucial. In practice, good EHRS must provide data and means for optimizing clinical and business processes/workflows. For example, nurses should be disburdened from time-consuming work outside of care.

EHRS must support data, information, and knowledge logistics. Related data, therefore, must be provided on time, at the right location, and to the authorized staff in an appropriate and usable form.


#

Data privacy management and trustee services

EHRS data is extremely sensitive personal data and disappointing patients and citizens in their trust to care systems and EHRS would finally make patient care inefficient if not impossible. Moreover, complying with the national and regional legal and regulatory requirements on data privacy is a matter of course. If the clinical data of the EHR shall be used for research or education, the EHRS must support a trustee by providing means for pseudonymization and anonymization following recommendations as described in [[34]], for example. The management of informed patient consent has to be included. Nevertheless, EHRS data is an asset not only for researchers and teachers but also for commercial companies. However, medical data must not be a commercial good or subject of selling without the explicit informed consent of the patients (c.f. section 3.2.3, StrQ.2).


#

Data quality management and curation

The formal completeness of EHRS data can be supported by using the CEN/ ISO EN13606 standard, a reference information model for the EHR [[35]]. A reference installation of this standard is open EHR [[36], [37], [38]]. With regard to completeness of content, EHRS shall integrate doctors’ documentation and nursing documentation (reasons and interventions). This is the very prerequisite for the true continuity of care since nurses and doctors must cooperate as a highly efficient team. Precision medicine calls for the integration of genotype data with clinical, phenotype data [[39]], including images [[40]].

In order to ensure the high quality content of the EHR, care managers (e.g. nurses, health information managers) are needed to collect and integrate and curate the different pieces of patient’s information. Data correctness and integrity in the EHR shall be traced and audited compliant to regulations for software as a medical device according the European Medical Device Directive (MDD) or FDA regulations [[41]].


#

Semantic annotation

Data and Documents of the EHR should be linked to medical and pharmaceutical knowledge and thereby be semantically annotated in order to provide for mutual understanding of different providers in transition care, i.e. semantic interoperability [[37]]. Several standards are available: ICD10/11, ICNP, ICPM, LOINC, SNOMED, TNM, etc.). Semantic integration calls for well-structured data. A data model should be provided to describe the structure of the data of a given EHR [[42]].


#
#

(PrimQ.2) Requirements for the quality of the EHRS logical tool layer: application components and their integration

According to section 2.1.1, an EHRS is not one single software product that can be purchased in a shop. Moreover, it is a unique configuration of different components, e.g. application systems, computers, and network components in a healthcare network. Therefore, the following requirements apply:

Applications/features needed

At least since 1998 it has been known that an EMR is not simply the set of scanned sheets of a paper based patient record but a means for “managing a patient’s trajectories” [[43]]. This holds for EHR as well. As a consequence, the EHRS must integrate the various applications used in hospitals (e.g. described in [[11]]) and must offer features supporting the management of care in a health care network. Moreover, the following requirements must be met (c.f. ►[Box 3]):

Box 3

(PrimQ.2) Requirements for the quality of the EHRS logical tool layer: application components and their integration

Applications/features:

  • Usability

  • Object identity

  • Access integration

  • Integration of decision support systems

  • Integration of clinical documentation systems and research documentation systems

  • NLP features in EHRS

  • Integration of personal health data

  • Integration of application systems outside of CDOs and health care


#

Usability

All applications of the EHRS shall be user friendly and follow software engineering principles as in ISO 9241 and ISO 9126 [[7]]. Since the EHRS shall support doctors and other medical staff in their very busy and bustling work, good visualization is required i.e. there shall be an application in the EHRS providing the most essential data of a patient at a glance.


#

Object identity

The EHR contains the EMRs of all CDOs in the respective healthcare network. In every hospital of the network, the EMR has to collect data from all hospital departments. Hence, the CDO’s information system should integrate a master patient index for uniquely identifying patients, which is usually provided by the Patient Administration System. An integrated human resource management system has to provide unique identification and authentication of the health professionals and their assignment to one or more CDOs as well as the CDOs themselves.


#

Access integration

EHRS must be accessible not only by doctors but also by pharmacists, researchers, and medical teachers [[26]]. Since secure public health telematics infrastructures for transinstitutional health information systems are rare, pragmatic hybrid solutions integrating the usage of paper forms must also be considered, since they are still used in Germany and Japan [[44], [45]].


#

Integration of decision support systems

Knowledge based CPOE systems must be integrated in the EHRS [[30]]. They must act as a therapy safety system offering medication checks based on medication plans as well as on laboratory findings. Automatic detection of adverse events and automatic reporting of those events to regulatory agencies must be added [[46]]. Automatic packing systems and robots may offer additional safety by reducing errors in the medication process [[29]].

Patient safety in intensive care also depends on the integration of EHR and medical knowledge in the respective Patient Data Management System (PDMS) providing reliable decision support in critical situations. Decision support systems as part of the EHRS shall be certified to ensure correct functioning. This includes the reliability and trustworthiness of the application as well as robust functionality, i.e. the system does what it has to do. As a certification body in Europe EUROREC is available for example [[47]]. EUROREC provides a broad range of criteria from those related to good software engineering to those dealing with special requirements in health care. In addition, the EHRS must obey the legal and regulatory requirements (e.g. European Regulation on Data Protection, European Medical Device Directive (MDD) or FDA).

The EHR must be integrated with workflow management systems supporting clinical pathway execution based on evidence based medical knowledge [[48]]. Decision support systems must be able to use the EHR but should also be ‘big-data-enabled’ to allow a better implementation of the precision medicine agenda [[49]]. Data Warehouses for research (see below) can thus be used for clinical decision support [[9]] and thereby implement one aspect of the bench to bedside translation [[50]].


#

Integration of clinical documentation systems and research documentation systems

An EHRS can only be considered to be of high quality, if it improves research efficiency:

  • The administrative Patient Management System or the Medical Documentation System should provide features for recruitment for clinical trials and support identification of cohorts. An application system for trial administration is required, which is integrated with the financial application systems.

  • To support the direct reuse of EHR data in clinical trials, electronic data capture (EDC) systems should be connected to the clinical application systems in order to transcribe automatically from EHR to electronic case report forms (eCRF) [[51]]. The design and implementation of new data entry forms should be flexible and easy to use and should follow standards like CDISC ODM in order to allow for secondary use [[42], [51]]. Providing structured data for medical research out of the care context may be facilitated by using openEHR archetypes [[37]]. Additional value for medical research can be reached by combining EHR data with social media, knowledge bases, and literature analysis (record linkage) [[37]].

  • The EHRS must provide a research data infrastructure including a central Research Data Warehouse [[9], [52], [53]] as, for example, based on i2b2 offering features for data mining to medical researchers [[54]]. The Data Warehouse can be connected to the clinical application systems by tools for extraction, transport, and loading (ETL) that support converting the proprietary data of different applications to one standard format and load it to a unified Data Warehouse [[55]]. For multi-centered clinical research as well as for epidemiological research the Research Data Warehouses must be usable for many institutions in a certain research network [[9]]. Even in research oriented hospitals, “medical doctors cannot use the tools for their own data analysis individually. User-centered universal tools should be applied for medical researchers to analyze their own data.” [[56]]. Therefore, EHRS should provide easy and quick ‘controlled and regulated’ access to the EHR data. This would leverage the treasure of clinical data and support discovery of new medical knowledge. Therefore, EHRS would contribute to the hospital’s task of being an active part of a learning health system [[57]].


#

NLP features in EHRS

Although structured collections of clinical data are necessary for clinical research and especially clinical trials [[42]], usually much data in EHR is free text, i.e. unstructured and not standardized. Such data can only be processed by computerized application systems if natural language processing (NLP) methods are used [[58]], which is considerably difficult not only for documents in the Japanese language [[59]] but also for those in the German language [[60]].


#

Integration of personal health data

An EHRS must interact with patients and citizens in two ways. First, it must be able to integrate personal data from personal health monitoring devices provided by smartphones, smartwatches in combina tion with various wearable sensors, etc. Secondly, the EHRS shall be able to support patients directly by providing them with a longitudinal care plan, for example, supporting the management of their trajectories from one care provider to another [[28], [61], [62]].


#

Integration of application systems outside of CDOs and health care

Besides integrating ‘traditional’ application systems in the CDO as has been mentioned before, an EHRS should be connected to those applications that are already used by citizens. Hence, the Integration of welfare facilities and of the local TV-program may be necessary [[63]]. Integration with emergency medical services (e.g. helicopters, ambulances) is self-evident.

Architecture (c.f. ►[Box 4])


#

Architecture styles

Architectural styles supporting flexibility, modularity and, therefore, the maintainability of EHRS shall be used as service oriented architectures SOA) and enterprise service bus (ESB) [[64]], for example. EHRS will usually be realized as distributed systems with a variety of data repositories and databases. For data sharing there are two subtypes: The ‘show-type’ is based on a central registry instead of a central database (e.g. AjisaiNet in Japan), whereas the ‘send-type’ offers a central database even for different institutions (e.g. GDBank in Germany) [[44]].

Box 4

(PrimQ.2) Requirements for the quality of the EHRS logical tool layer: application components and their integration

Architecture:

  • Architecture styles

  • Secure communication

  • Communication standards and best practices


#

Secure communication

In case of communication of EHR content the EHRS must provide a means for the end-to-end encryption of messages according to recommendations as for example those of the German Federal Office for Information Security (BSI) [[65]]. Moreover, a secure nationwide health telematics infrastructure is required preventing from inventing the wheel of safe and secure communication again and again for each EHRS usage in related projects. Especially in Europe such infrastructure and standard compliant solutions for cross-border integration are necessary [[66]] and have been addressed in projects like epSOS [[66]] and its follow-up projects.


#

Communication standards to ensure interoperability and best practices:

EHRS shall use standards for storing, communicating, and sharing its data whether they are documents or structured data. The Integrating the Healthcare Enterprise (IHE) initiative provides the relevant best practices for usage of standards in EHRS in the form of profiles [[67]]. IHE profiles, among others, advocate using the Clinical Document Architecture (CDA) for the structuring of clinical documents, using HL7 for text-oriented messages, DICOM for medical images. The upcoming standard FHIR is suggested for defining, storing, and retrieving so-called resources as structured data in SOA environments. Especially for transinstitutional settings and fostering the continuity of care usage of ASTM CCR [[68]] or HL7/ASTM CCD [[69]] must be considered. CEN/ISO EN13606 standard provides the concept of the EHR extract to exchange the entire or part of the EHRs as mentioned before.


#
#

(PrimQ.3) Requirements for the quality of the EHRS physical tool layer: computers, devices, and their integration

(c.f. ►[Box 5])

Box 5

(PrimQ.3) Requirements for the quality of the EHRS physical tool layer: computers, devices, and their integration

  • Servers and networks

  • Clients

  • Integration of devices and biobanks

Servers and Networks:

EHRS must rely on high performant servers and networks. This holds not only for their use in patient care but for clinical research usage as well. Especially the integration of big data as a basis for decision support requires – large RAM, fast disk resources, and super-efficient search and compression algorithms [[70]] as well as high speed servers and networks [[71]]. In order to provide secure operation on a 24/7 basis, redundant computing facilities and virtualization of servers is required [[11]]. Continuous services even in wide area disasters can only be guaranteed by regional or national centralized computing facilities and remote backup systems [[72]]. Since we claimed to integrate TV programs as well, using TV at home and respective networks as media is crucial [[63]].


#

Clients

Mobile clients like laptops, tablet computers, and smartphones shall be used for both health care professionals and patients/citizens. Especially patients and citizens usually will have their own mobile devices. The concept of “bring your own device” (BYOD) results in severe challenges for data security and privacy in EHRS, which call for up to date solutions.


#

Integration of devices and biobanks

Plenty of devices can be used for either collecting patient related data for the EHRS or must be controlled by the EHRS in order to deliver certain services. Those devices must be connected. For example, RFID and other tokens or tags can be used to identify patients and the nurses at their ward round. Bar Coded Medication Administration System (BCMA) [[73]] may reduce medication errors by identifying patients as well as drugs. Automatic packing systems and robots can complement this. The Internet of Things [[74]] offers even smarter connections, e.g. vital sign monitoring by devices using Bluetooth for intermittent or continuous communication; automatic compliance checks in medication by smart blisters. Medical research as a main function of EHRS and precision medicine are demanding for the integration of Biobanks [[75]].


#
#
#

3.2.2 (IMQ) Quality of Information Management Processes

An EHRS only delivers value for patients, citizens, and health care professionals if it is actually under operation and thus provides real services as described as the primary quality of the domain layer (PrimQ.1). This takes for granted well-organized operational information management of EHRS based on strategic and tactical information management. Note that risks for patients are not only the result of weak technology but may also result from bad training of staff in using new systems and thus from bad information management.

Well-organized information management in general not only requires highly motivated and well educated human experts but also optimal communication between the experts. The human factor for the management of high quality EHRS cannot be underestimated. Information management in this socio-technical field, therefore, must carefully care for good teamwork by giving enough chances for personal communication.

(IMQ.1) Requirements for the quality of strategic information management

Strategic information management deals with the entire information system and EHRS of a CDO or a health care network as a whole and establishes strategies and principles for its evolution.

There is a high risk not to meet the requirements for the primary quality of EHRS, if EHRS development is not performed systematically, i.e. by systematic information management and especially strategic information management. Information management should be organized and performed according to accepted textbooks on information management or to a framework such as COBIT [[76]]. COBIT defines IT management and IT governance processes and provides a process capability model for their assessment.

A corner stone of strategic information management is a clear definition of strategic goals in a strategic information management plan [[77]]. In order to meet the requirements of section 3.2.1, the following strategic goals for EHRS should be covered as far as applicable:

EHRS shall

  • enable use case based reasoning for decision support in patient care by secondary use of data;

  • improve patients’ safety;

  • support care for elderlies in their homes;

  • improve cooperation with other healthcare facilities even in remote areas and under disaster conditions;

  • improve patient care in underserved regions (e.g. islands, rural areas);

  • support patients’ self-management;

  • reduce patient turnaround time;

  • improve CDO’s management and administration and efficiency;

  • support high quality research outcomes by high data quality and easy to use features for secondary use of data;

  • contribute to education.

Since EHRS depend on widespread implementation and use of EMR in hospitals, clinics, and doctors’ practices, there is a need for clear national strategies to introduce EHR and for the execution of such strategies [[6], [78], [79]]. EHRS should be independent from governmental or economic stakeholders in order to be trustworthy for both care providers and patients [[80], [81]].


#

(IMQ.2) Requirements for the quality of tactical information management

Based on a strategic information management plan, projects for installing or updating the components of the EHRS must also be executed systematically. A mature procedure model for projects (e.g. according to [[82], [83]] is needed. Those models especially ensure to have all of the relevant stakeholders involved and to have sufficient face-to-face meetings and working groups.

Success of projects can be measured, for example, by average time between defining and starting projects [[84]], percentage of milestones missed, and of killed projects [[84]]. Nevertheless, projects must be assessed especially with respect to their outcomes. Therefore, the evaluation of resulting EHRS features must be performed on a regular basis [[26]]. This is especially important in case of features for medication and shall guarantee patient safety [[29]]. Certified components for EHRS offer a better chance to achieve the addressed benefits [[85]]. In summary, the principles of evidence based medical informatics must be applied [[86]].


#

(IMQ.3) Requirements for the quality of operational information management

An EHRS must be available on a 24/7 basis even in case of disaster. Besides the requirement of virtualized and redundant computing centers and networks (cf. section 3.2.1, PrimQ.3), appropriate staff and other resources must be made available. Therefore, a Business Continuity Plan (BCP) is necessary and the management of EHRS operation shall be based on established standards or best practices, e.g. COBIT and ITIL [[16], [76]].


#

(IMQ.4) Requirements for the quality of IT service management

IT services provided by a local or regional information management department running the EHRS shall have defined and measurable quality [[72]]. This can be achieved by using service level agreements (SLA) [[11]]. IT service management should be implemented following accepted best practices, e.g. ITIL [[16]].


#

(IMQ.5) Requirements for the quality of IT governance

EHRS are transinstitutional information systems and, therefore, the responsibility for its management must be organized in close consultation between all of the parties and stakeholders involved. The intensity and centrality of information management in such a setting must be negotiated with all of the partners [[87]]. In order to ensure the independence as well as acceptance of the EHRS, the establishment of a strategic advisory board (members: Care Providers, IT specialist, academic, governmental, etc.) makes sense.


#
#

3.2.3 (StrQ) Structural Quality of an EHRS: Quality of Resources for Information Management

It may sound self-evident, but it must be emphasized: good EHRS require high-quality resources for information management. Of course, enough money and appropriate tools are needed. However, having enough personnel who are well educated and committed is crucial.

(StrQ.1) Human resources, human capacity management

EHRS require not only the best human experts but also human capacity management. A skills-matrix may be used to document and develop staff’s competences and skills. It must be systematically decided what competences shall be bought from the outside. Usually technical expertise can be ‘bought’ outside easier than e.g. clinical process and project management expertise. It is crucial to have an adequate human capacity management in order to develop the skills and knowledge needed especially for project management.

The term “eHealth literacy” is often used with regard to the ability of users to use EHRS. However, it should also be used with regard to the knowledge of information management staff about standards, profiles and tools in the eHealth domain. With that respect, eHealth literacy must be developed continuously in the information management department. International recommendations on education in bio-medical and health informatics provide thorough overview on knowledge and skills needed for different tasks in information management [[88]].


#

(StrQ.2) Financial resources

EHRS are important and crucial for patient care and for medical research. Therefore, EHRS need to be sustainable [[63], [72], [89]]. Sustainability cannot only be reached by using appropriate information technology, standards, information management, and governance regulations, but must also be founded by ongoing funding [[44]]. Moreover, appropriate business models are needed in order to provide an ongoing financial basis for EHRS. It has to be clearly defined as to what added value an EHRS will bring to what stakeholders and how they would pay for the related services. Some may take the tremendous value of the data in EHRS into account and consider it as an asset. However, selling data for funding the EHRS without clear and informed consent of the patients will undermine and finally ruin patients’ trust. The model of health data cooperative in which personal health data is solely owned and controlled by their members may offer opportunities for trustworthy business models [[90]]. This will also call for effective means for controlling usage of their data by patients and citizens themselves [[91], [92]].


#

(StrQ.3) Tools for information management

Information management for EHRS requires service desk provision, modern ticket, project management, network, or computing center management systems. The integration of these management tools is as much a challenge as the integration of the components of EHRS. IT4IT provides a well-structured framework for solving this integration challenge [[93]].


#
#
#
#

4. Conclusion

Based on the framework QRF-EHRS and the various requirements for EHRS quality, we derive two conclusions:

  1. Assessing the quality of EHRS is a multidimensional problem. Of course CEOs, CIOs, and other stakeholders of healthcare networks are strongly interested in having simple key performance indicators (KPI) that are able to measure the quality of their EHRS, e.g. by a score from 0 (low quality) to 7 (high quality). However, the existence of such scalar key performance indicators (KPI) appears unlikely, although there are popular approaches of this kind that have a considerable impact [[94]].

  2. Quality of EHRS is not only a property of a software product in the vendor’s shop.

Moreover, quality assessment must be applied to the socio-technical system consisting of the unique EHRS as it is installed and the information management taking care of its planning, directing, and monitoring. Therefore, this kind of assessment should encompass all of the dimensions of our framework and it must be done on site.

However, more research is needed in order to make both the framework QRF-EHRS and the quality requirements applicable for practical use. Topics for further research may include:

  • refining the framework QRF-EHRS and measuring its impact;

  • developing key performance indicators (KPI) to actually measure the presented requirements in quantitative terms;

  • exploring the dependencies between the different dimensions of the framework by empirical studies in different hospitals;

  • revealing the strengths and weaknesses in existing EHRS by the usage of the framework;

  • defining best practices as showcases.

Furthermore, the requirements and the KPIs to be developed must be applied for evaluating and comparing existing real-life EHRS and their environment. This approach can be pursued on a national as well as on a binational or multinational level. Best practice examples could be identified and further analyzed to better understand why a particular system is of such high quality.

Apart from quality benchmarking, these requirements can be used to stimulate research on information system success. The approach of DeLone & McLean [[95]] to describe information system success divides quality into system, information, and service quality and mainly covers the users’ perspective on these quality dimensions. The Quality Requirements Framework of EHRS pursues the information management perspective. It thus embeds the success of the EHRS, i.e. outcome quality, into the two success factors process quality and structural quality of information management. The relationship between “information management process quality” and “service quality” as well as other similarities need to be investigated systematically in future work.

Obviously, this paper has methodological limitations. The results, i.e. especially the requirements for the quality of EHRS, are based on the personal experiences and views of expert participants of two workshops (JGEHRS 2014, JGEHRS 2016). The requirements presented in this paper are well grounded by many peer-reviewed papers of the authors themselves and other authors. However, there is no systematic evaluation and, therefore, no evidence of validity. The framework QRF-EHRS proved to be suitable to systematize the requirements, but its completeness is still unclear.

This paper is a collection of aspects readers should – from the authors’ perspective – remember and reflect when planning, directing, and monitoring EHRS. It is open for discussion.


#
#

* Supplementary material published on our website https://doi.org/10.3414/ME17-05-0002


  • References

  • 1 Haux R. Medical informatics: past, present, future. Int J Med Inform 2010; 79 (Suppl. 09) 599-610.
  • 2 Abrahamsson S, Bergstrom S, Larsson K, Tillman S. Danderyd Hospital computer system. II. Total regional system for medical care. Comput Biomed Res 1970; 3 (Suppl. 01) 30-46.
  • 3 Friedman CP, Owens DK, Wyatt JC. Evaluation and Technology Assessment. In: Shortliffe EH, Cimino JJ. editors. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. New York: Springer; 2006. p. 403-446.
  • 4 Garets D, Davis M. Electronic Medical Records vs. Electronic Health Records: Yes, There Is a Difference. Chicago, IL: HIMSS Analytics; 2006. Available from: http://s3.amazonaws.com/rdcms-himss/files/production/public/HIMSSorg/Con-tent/files/WP_EMR_EHR.pdf.
  • 5 International Organization for Standardization (ISO).. ISO 18308: 2011 Requirements for an electronic health record architecture. 2011 [cited 2014 Jan 23].
  • 6 Takabayashi K, Doi S, Suzuki T. Japanese EMRs and IT in Medicine: Expansion, Integration, and Reuse of Data. Healthc Inform Res 2011; 17 (Suppl. 03) 178-183.
  • 7 Hoerbst A, Ammenwerth E. Electronic health records. A systematic review on quality requirements. Methods Inf Med 2010; 49 (Suppl. 04) 320-336.
  • 8 Prokosch HU, Ganslandt T. Perspectives for medical informatics.: Reusing the electronic medical record for clinical research. Methods Inf Med 2009; 48 (Suppl. 01) 38-44.
  • 9 Kimura E, Kobayashi S, Kanatani Y, Ishihara K, Mimori T, Takahashi R. et al. Developing an electronic health record for intractable diseases in Japan. In: User Centred Networked Health Care: Proceedings of MIE 2011. Amsterdam: IOS Press; 2011. p. 255-259 (Stud Health Technol Inform; vol. 169).
  • 10 Health Level Seven International.. HL7 EHR System Functional Model: Health Level Seven International. 2007 Jan 1.
  • 11 Winter A, Haux R, Ammenwerth E, Brigl B, Hellrung N, Jahn F. Health Information Systems-Architectures and Strategies. London: Springer; 2011
  • 12 Winter A, Brigl B, Wendt T. Modeling Hospital Information Systems (Part 1): The Revised Three-Layer Graph-Based Meta Model 3LGM2 . Methods Inf Med 2003; 42 (Suppl. 05) 544-551.
  • 13 Staemmler M. Towards sustainable e-health networks: does modeling support efficient management and operation?. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 1). Amsterdam: IOS Press; 2007. p. 53-57 (Stud Health Technol Inform; vol. 129).
  • 14 Jahn F, Ißler L, Winter A, Takabayashi K. Comparing a Japanese and a German Hospital Information System - Assessing Structural Quality by 3LGM2 Models. Methods Inf Med 2009; 48 (Suppl. 06) 531-539.
  • 15 Weill P, Ross JW. IT Governance - How top performers manage IT decisions right for superior results. Boston, Massachusetts: Harvard Business School Press;; 2004
  • 16 Kabachinski J. Have you heard of ITIL?: It’s time you did. Biomed Instrum Technol 2011; 45 (Suppl. 01) 59-62.
  • 17 Dögl R. Strategisches Qualitätsmanagement im Industriebetrieb: pragmatischer Ansatz zur Erklärung und methodischen Handhabung des Qualitätsphenomens. Göttingen: Vandenhoeck & Ruprecht; 1986. German.
  • 18 Liebe J-D, Hübner U, Straede M-C, Thye J. A top-down approach for developing a workflow composite score to measure clinical information logistics. Methods Inf Med 2015; 54 (Suppl. 05) 424-433.
  • 19 Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966; 44 (Suppl. 03) 166.
  • 20 Donabedian A. Criteria and Standards for Quality Assessment and Monitoring. Quality Review Bulletin 1986; 12 (Suppl. 03) 99-108.
  • 21 Ammenwerth E. Evidence-based Health Informatics: How Do We Know What We Know?. Methods Inf Med 2015; 54 (Suppl. 04) 298-307.
  • 22 Ancker JS, Kern LM, Abramson E, Kaushal R. The Triangle Model for evaluating the effect of health information technology on healthcare quality and safety. J Am Med Inform Assoc 2012; 19 (Suppl. 01) 61-65.
  • 23 Jahn F, Takabayashi K, Suzuki T, Fujita S, Winter A. Comparison between German and Japanese Hospital Information Systems by 3LGM2 analysis. In: Proceedings of the 27th Joint Conference of Medical Informatics (JCMI). Kobe: November 23-25 2007. p. 840-843.
  • 24 Takabayashi K. From Marco Polo Islands. Methods Inf Med 2007; 46 (Suppl. 06) 669-670.
  • 25 Takabayashi K, Hasegawa K, Yamazaki S, Oshima H, Okamura H, Uchida T. [EMI net - electronic medical record sharing in local community for home care medicine]. Gan To Kagaku Ryoho 2003; 30 Suppl 1: 169-172. Japanese.
  • 26 Matsumoto T, Honda M. The evaluation of the need to share medical data on the community medical ICT network service in Nagasaki, Japan. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 1017 (Stud Health Technol Inform; vol. 192).
  • 27 Haux R, Hein A, Kolb G, Kunemund H, Eichelberg M. Five years of interdisciplinary research on ageing and technology: Outcomes of the Lower Saxony Research Network Design of Environments for Ageing (GAL) - an Introduction to this Special Issue on Ageing and Technology. Inform Health Soc Care 2014; 39 3-4 161-165.
  • 28 Kobayashi K, Nakashima N, Inoguchi T, Takaya-nagi R. [Disease management for diabetes mellitus to prevent the onset and progression of complications]. Rinsho Byori 2012; 60 (Suppl. 11) 1101-1106. Japanese.
  • 29 Miyo K, Nittami YS, Kitagawa Y, Ohe K. Development of case-based medication alerting and rec-ommender system: a new approach to prevention for medication error. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 2). Amsterdam: IOS Press; 2007. p. 871-874 (Stud Health Technol Inform; vol. 129).
  • 30 Ammenwerth E, Aly AF, Burkle T, Christ P, Dormann H, Friesdorf W. et al. Memorandum on the Use of Information Technology to Improve Medication Safety. Methods Inf Med 2014; 2014 53(5) 336-343.
  • 31 Prokosch HU, Beck A, Ganslandt T, Hummel M, Kiehntopf M, Sax U. et al. IT Infrastructure Components for Biobanking. Appl Clin Inform 2010; 1 (Suppl. 04) 419-429.
  • 32 Jahn F, Winter A. A KPI Framework for Process-based Benchmarking of Hospital Information Systems. In: User Centred Networked Health Care: Proceedings of MIE 2011. Amsterdam: IOS Press; 2011. p. 542-6 (Stud Health Technol Inform; vol. 169).
  • 33 Hübner U, Ammenwerth E, Flemming D, Schaubmayr C, Sellemann B. IT adoption of clinical information systems in Austrian and German hospitals: results of a comparative survey with a focus on nursing. BMC Med Inform Decis Mak 2010; 10: 8.
  • 34 Aamot H, Kohl CD, Richter D, Knaup-Gregori P. Pseudonymization of patient identifiers for translational research. BMC Med Inform Decis Mak 2013; 13: 75.
  • 35 Farfán FJSedano, Terrón Cuadrado M, Castellanos Clemente Y, Serrano Balazote P, Moner Cano D, Robles Viejo M. Patient Summary and medicines reconciliation: application of the ISO/CEN EN 13606 standard in clinical practice. Stud Health Technol Inform 2011; 166: 189-196.
  • 36 Garde S, Chen R, Leslie H, Beale T, McNicoll I, Heard S. Archetype-based knowledge management for semantic interoperability of electronic health records. In: Medical informatics in a united and healthy Europe: Proceedings of MIE 2009, the XXII International Congress of the European Federation for Medical Informatics. Amsterdam: IOS Press; 2009. p. 1007-1011 (Stud Health Technol Inform; vol. 150).
  • 37 Kohl CD, Garde S, Knaup P. Facilitating secondary use of medical data by using openEHR archetypes. In: Safran C, Marin H, Reti S. editors. MEDINFO 2010 Partnerships for Effective eHealth Solutions. Amsterdam: IOS Press; 2010. p. 1117-1121 (Stud Health Technol Inform; vol. 160).
  • 38 Kobayashi S, Kimura E, Ishihara K. Archetype Model-Driven Development Framework for EHR Web System. Healthc Inform Res 2013; 19 (Suppl. 04) 271-277.
  • 39 Gottesman O, Kuivaniemi H, Tromp G, Faucett WA, Li R, Manolio TA. et al. The Electronic Medical Records and Genomics (eMERGE) Network: past, present, and future. Genet Med 2013; 15 (Suppl. 10) 761-771.
  • 40 van Horn JD, Toga AW. Human neuroimaging as a “Big Data” science. Brain Imaging Behav 2014; 8 (Suppl. 02) 323-331.
  • 41 Gupta P, Janodia MD, Jagadish PC, Udupa N. Medical device vigilance systems: India, US, UK, and Australia. Med Devices (Auckl) 2010; 3: 67-79.
  • 42 Karmen C, Ganzinger M, Kohl CD, Firnkorn D, Knaup-Gregori P. A framework for integrating heterogeneous clinical data for a disease area into a central data warehouse. In: EHealth - for continuity of care: Proceedings of MIE2014. Amsterdam: IOS Press; 2014. p. 1060-1064 (Stud Health Technol Inform; vol. 205).
  • 43 Berg M. Medical Work and the Computer-Based Patient Record: A Sociological Perspective. Methods Inf Med 1998; 37: 294-301.
  • 44 Juhr M, Haux R, Suzuki T, Takabayashi K. Overview of recent trans-institutional health network projects in Japan and Germany. J Med Syst 2015; 39 (Suppl. 05) 234.
  • 45 Schwartze J, Haarbrandt B, Rochon M, Wagner M, Haux R, Kleinschmidt T. et al. Design and implementation of an informed consent process for a standardized health information exchange solution on the example of the lower saxony bank of health. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 318-322 (Stud Health Technol Inform; vol. 192).
  • 46 Neubert A, Dormann H, Prokosch HU, Burkle T, Rascher W, Sojer R. et al. E-pharmacovigilance: development and implementation of a computable knowledge base to identify adverse drug reactions. Br J Clin Pharmacol 2013; 76 Suppl 1: 69-77.
  • 47 EUROREC Institute.. The EuroRec EHR Quality Seal. 2015 [cited 2015 Jan 15].
  • 48 Burkle T, Baur T, Hoss N. Clinical pathways development and computer support in the EPR: lessons learned. In: Ubiquity: Technologies for better health in aging societies: Proceedings of MIE2006. Amsterdam: IOS Press; 2006. p. 1025-1030 (Stud Health Technol Inform; vol. 124).
  • 49 Malin JL. Envisioning Watson as a rapid-learning system for oncology. J Oncol Pract 2013; 9 (Suppl. 03) 155-157.
  • 50 Marincola FM. Translational Medicine: A two-way road. J Transl Med 2003; 1 (Suppl. 01) 1.
  • 51 Matsumura Y, Hattori A, Manabe S, Takeda T, Takahashi D, Yamamoto Y. et al. Interconnection of electronic medical record with clinical data management system by CDISC ODM. In: EHealth - for continuity of care: Proceedings of MIE2014. Amsterdam: IOS Press; 2014. p. 868-872 (Stud Health Technol Inform; vol. 205).
  • 52 Matsumura Y, Kuwata S, Yamamoto Y, Izumi K, Okada Y, Hazumi M. et al. Template-based data entry for general description in medical records and data transfer to data warehouse for analysis. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 1). Amsterdam: IOS Press; 2007. p. 412-416 (Stud Health Technol Inform; vol. 129).
  • 53 Hruby GW, McKiernan J, Bakken S, Weng C. A centralized research data repository enhances retrospective outcomes research capacity: a case report. J Am Med Inform Assoc 2013; 20 (Suppl. 03) 563-567.
  • 54 Ganslandt T, Mate S, Helbing K, Sax U, Prokosch HU. Unlocking Data for Clinical Research - The German i2b2 Experience. Appl Clin Inform 2011; 2 (Suppl. 01) 116-127.
  • 55 Stäubert S, Winter A, Speer R, Loffler M. Designing a Concept for an IT-Infrastructure for an Integrated Research and Treatment Center. In: Safran C, Marin H, Reti S. editors. MEDINFO 2010 Partnerships for Effective eHealth Solutions. Amsterdam: IOS Press; 2010. p. 1319-1323 (Stud Health Technol Inform; vol. 160).
  • 56 Takabayashi K, Ho TB, Yokoi H, Nguyen TD, Kawasaki S, Le SQ. et al. Temporal abstraction and data mining with visualization of laboratory data. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 2). Amsterdam: IOS Press; 2007. p. 1304-1308 (Stud Health Technol Inform; vol. 129).
  • 57 McGlynn EA, Lieu TA, Durham ML, Bauck A, Laws R, Go AS. et al. Developing a data infrastructure for a learning health system: the PORTAL network. J Am Med Inform Assoc 2014; 21 (Suppl. 04) 596-601.
  • 58 Danciu I, Cowan JD, Basford M, Wang X, Saip A, Osgood S. et al. Secondary use of clinical data: the Vanderbilt approach. J Biomed Inform 2014; 52: 28-35.
  • 59 Suzuki T, Doi S, Fujita S, Hatakeyama Y, Honda M, Matsumura Y. et al. Construction of the integrated multicentre discharge summary database. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 1064 (Stud Health Technol Inform; vol. 192).
  • 60 Löpprich M, Krauss F, Ganzinger M, Senghas K, Riezler S, Knaup P. Automated classification of selected data elements from free-text diagnostic reports for clinical research. Methods Inf Med 2016; 55 (Suppl. 04) 373-380.
  • 61 Dykes PC, Samal L, Donahue M, Greenberg JO, Hurley AC, Hasan O. et al. A patient-centered longitudinal care plan: vision versus reality. J Am Med Inform Assoc 2014; 21 (Suppl. 06) 1082-1090.
  • 62 S&I Framework.. Longitudinal Coordination of Care - Interoperable Care Plan Exchange - Use Case v2.0: Framework, S&I; 2014 Jan 1 [cited 2016 Jan 31]. Available from: http://wiki.siframe-work.org/file/detail/LCC%20Care%20Plan%20Ex-change%20Use%20Case%20Finalv2.pdf.
  • 63 Honda M, Matsumoto T, Hirose M. A system for health promotion using tele-homecare technology in community. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 940 (Stud Health Technol Inform; vol. 192).
  • 64 Loya SR, Kawamoto K, Chatwin C, Huser V. Service oriented architecture for clinical decision support: a systematic review and future directions. J Med Syst 2014; 38 (Suppl. 12) 140.
  • 65 Staemmler M, Walz M, Weisser G, Engelmann U, Weininger R, Ernstberger A. et al. Establishing end-to-end security in a nationwide network for telecooperation. In: Quality of life through quality of information: Proceedings of MIE2012. Amsterdam: IOS Press; 2012. p. 512-516 (Stud Health Technol Inform; vol. 180).
  • 66 Fonseca M, Karkaletsis K, Cruz IA, Berler A, Oliveira IC. OpenNCP: a novel framework to foster cross-border e-Health services. In: Digital Healthcare Empowering Europeans: Proceedings of MIE2015. Amsterdam: IOS Press; 2015. p. 617-621 (Stud Health Technol Inform; vol. 210).
  • 67 Stäubert S, Schaaf M, Jahn F, Brandner R, Winter A. Modeling Interoperable Information Systems with 3LGM2 and IHE. Methods Inf Med 2015; 54 (Suppl. 05) 398-405.
  • 68 ASTM E2369-12. Standard Specification for Continuity of Care Record (CCR). 2012 Available from: www.astm.org.
  • 69 HL7.. HL7/ASTM Implementation Guide for CDA® R2 -Continuity of Care Document (CCD*) Release 1. 2014 [cited 2014 Dec 04].
  • 70 Eizen K, Masato S, Kazukuni K, Yoshihito N, Ta-kuji K, Ken I. Securing SSL-VPN with LR-AKE to access personal health record. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 930 (Stud Health Technol Inform; vol. 192).
  • 71 Bellazzi R. Big data and biomedical informatics: a challenging opportunity. Yearb Med Inform 2014; 9 (Suppl. 01) 8-13.
  • 72 Staemmler M. Service delivery for e-Health applications. In: User Centred Networked Health Care: Proceedings of MIE 2011. Amsterdam: IOS Press; 2011. p. 537-541 (Stud Health Technol Inform; vol. 169).
  • 73 Holden RJ, Brown RL, Alper SJ, Scanlon MC, Patel NR, Karsh B-T. That’s nice, but what does IT do? Evaluating the impact of bar coded medication administration by measuring changes in the process of care. Int J Ind Ergon 2011; 41 (Suppl. 04) 370-379.
  • 74 Hofdijk J, Séroussi B, Lovis C, Ehrler F, Sieverink F, Ugon A. et al., editors. Transforming Healthcare with the Internet of Things: Proceedings of the EFMI Special Topic Conference 2016. Amsterdam: IOS Press; 2016. (vol 221).
  • 75 Oberlander M, Linnebacher M, Konig A, Bogoevska V, Brodersen C, Kaatz R. et al. The “North German Tumor Bank of Colorectal Cancer”: status report after the first 2 years of support by the German Cancer Aid Foundation. Langenbecks Arch Surg 2013; 398 (Suppl. 02) 251-258.
  • 76 ISACA.. COBIT 5: A Business Framework for the Governance and Management of Enterprise IT; 2013 [cited 2013 Nov 14]. Available from: http://www.isaca.org/cobit/pages/default.aspx.
  • 77 Brigl B, Ammenwerth E, Dujat C, Graber S, Grosse A, Haber A. et al. Preparing strategic information management plans for hospitals: a practical guideline: SIM plans for hospitals: a guideline. Int J Med Inform 2005; 74 (Suppl. 01) 51-65.
  • 78 Haux R. Health information systems - past, present, future. Int J Med Inform 2006; 75 3-4 268-281.
  • 79 World Health Organization, International Telecommunication Union.. National eHealth Strategy Toolkit. Geneva: World Health Organization; 2012. Jan 1. Available from: http://www.itu.int/pub/D-STR-E_HEALTH.05-2012.
  • 80 Shabo A. It’s time for health record banking!. Methods Inf Med 2014; 53 (Suppl. 02) 63-65.
  • 81 Plischke M, Wagner M, Haarbrandt B, Rochon M, Schwartze J, Tute E. et al. The lower saxony bank of health. Rationale, principles, services, organization and architectural framework. Methods Inf Med 2014; 53 (Suppl. 02) 73-81.
  • 82 AXELOS Ltd.. PRINCE2; 2014 [cited 2014 Mar 5]. Available from: http://www.prince-officialsite.com/.
  • 83 Ammenwerth E, Haux R, Bess A, Blomer R, Bott O, Häber A. et al. IT-Projektmanagement in Krankenhaus und Gesundheitswesen - Einführendes Lehrbuch und Projektleitfaden für das taktische Management von Informationssystemen. Stuttgart: Schattauer;; 2005
  • 84 ServiceNow.. KPI LIBRARY; 2014 [cited 2014 Mar 5]. Available from: http://kpilibrary.com/.
  • 85 Hoerbst A, Ammenwerth E. Quality and Certification of Electronic Health Records: An overview of current approaches from the US and Europe. Appl Clin Inform 2010; 1 (Suppl. 02) 149-164.
  • 86 Ammenwerth E, Rigby M. editors. Evidence-based health informatics: Promoting safety and efficiency through scientific methods and ethical policy. Amsterdam, Netherlands: IOS Press; 2016. (Stud Health Technol Inform; vol 222).
  • 87 Hellrung N, Gusew N, Willkomm M, Haux R. IT-based information management in health care networks: the MedoCom approach. In: EHealth Beyond the Horizon-Get IT There: Proceedings of MIE2008. Amsterdam: IOS Press; 2008. p. 623-628 (Stud Health Technol Inform; vol. 136).
  • 88 Mantas J, Ammenwerth E, Demiris G, Hasman A, Haux R, Hersh W. et al. Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics. First Revision. Methods Inf Med 2010; 49 (Suppl. 02) 105-120.
  • 89 Nakashima N, Nohara Y, Ahamed A, Kuroda M, Inoue S, Ghosh PP. et al. An affordable, usable and sustainable preventive healthcare system for unreached people in Bangladesh. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 1051 (Stud Health Technol Inform; vol. 192).
  • 90 Hafen E, Kossmann D, Brand A. Health data cooperatives - citizen empowerment. Methods Inf Med 2014; 53 (Suppl. 02) 82-86.
  • 91 Kelbert F, Pretschner A. Data usage control enforcement in distributed systems. In: Proceedings of the third ACM conference on Data and application security and privacy. New York, NY, USA: ACM; 2013. p. 71-82.
  • 92 Calvillo J, Roman I, Roa LM. Empowering citizens with access control mechanisms to their personal health resources. Int J Med Inform 2013; 82: 58-72.
  • 93 The Open Group.. The Open Group IT4IT Reference Architecture v1.3; 2014. Available from: https://collaboration.opengroup.org/data/IT4IT/RA_1.3/index.htm?goto=2:1:1:205.
  • 94 Pettit L. Understanding EMRAM and how it can be used by policy-makers, hospital CIOs and their IT teams. World Hosp Health Serv 2013; 49 (Suppl. 03) 7-9.
  • 95 DeLone WH, McLean ER. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems 2003; 19 (Suppl. 04) 9-30.

Correspondence to

Prof. Alfred Winter
University of Leipzig
Institute for Medical Informatics, Statistics
and Epidemiology
Haertelstr. 16-18
04107 Leipzig
Germany

  • References

  • 1 Haux R. Medical informatics: past, present, future. Int J Med Inform 2010; 79 (Suppl. 09) 599-610.
  • 2 Abrahamsson S, Bergstrom S, Larsson K, Tillman S. Danderyd Hospital computer system. II. Total regional system for medical care. Comput Biomed Res 1970; 3 (Suppl. 01) 30-46.
  • 3 Friedman CP, Owens DK, Wyatt JC. Evaluation and Technology Assessment. In: Shortliffe EH, Cimino JJ. editors. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. New York: Springer; 2006. p. 403-446.
  • 4 Garets D, Davis M. Electronic Medical Records vs. Electronic Health Records: Yes, There Is a Difference. Chicago, IL: HIMSS Analytics; 2006. Available from: http://s3.amazonaws.com/rdcms-himss/files/production/public/HIMSSorg/Con-tent/files/WP_EMR_EHR.pdf.
  • 5 International Organization for Standardization (ISO).. ISO 18308: 2011 Requirements for an electronic health record architecture. 2011 [cited 2014 Jan 23].
  • 6 Takabayashi K, Doi S, Suzuki T. Japanese EMRs and IT in Medicine: Expansion, Integration, and Reuse of Data. Healthc Inform Res 2011; 17 (Suppl. 03) 178-183.
  • 7 Hoerbst A, Ammenwerth E. Electronic health records. A systematic review on quality requirements. Methods Inf Med 2010; 49 (Suppl. 04) 320-336.
  • 8 Prokosch HU, Ganslandt T. Perspectives for medical informatics.: Reusing the electronic medical record for clinical research. Methods Inf Med 2009; 48 (Suppl. 01) 38-44.
  • 9 Kimura E, Kobayashi S, Kanatani Y, Ishihara K, Mimori T, Takahashi R. et al. Developing an electronic health record for intractable diseases in Japan. In: User Centred Networked Health Care: Proceedings of MIE 2011. Amsterdam: IOS Press; 2011. p. 255-259 (Stud Health Technol Inform; vol. 169).
  • 10 Health Level Seven International.. HL7 EHR System Functional Model: Health Level Seven International. 2007 Jan 1.
  • 11 Winter A, Haux R, Ammenwerth E, Brigl B, Hellrung N, Jahn F. Health Information Systems-Architectures and Strategies. London: Springer; 2011
  • 12 Winter A, Brigl B, Wendt T. Modeling Hospital Information Systems (Part 1): The Revised Three-Layer Graph-Based Meta Model 3LGM2 . Methods Inf Med 2003; 42 (Suppl. 05) 544-551.
  • 13 Staemmler M. Towards sustainable e-health networks: does modeling support efficient management and operation?. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 1). Amsterdam: IOS Press; 2007. p. 53-57 (Stud Health Technol Inform; vol. 129).
  • 14 Jahn F, Ißler L, Winter A, Takabayashi K. Comparing a Japanese and a German Hospital Information System - Assessing Structural Quality by 3LGM2 Models. Methods Inf Med 2009; 48 (Suppl. 06) 531-539.
  • 15 Weill P, Ross JW. IT Governance - How top performers manage IT decisions right for superior results. Boston, Massachusetts: Harvard Business School Press;; 2004
  • 16 Kabachinski J. Have you heard of ITIL?: It’s time you did. Biomed Instrum Technol 2011; 45 (Suppl. 01) 59-62.
  • 17 Dögl R. Strategisches Qualitätsmanagement im Industriebetrieb: pragmatischer Ansatz zur Erklärung und methodischen Handhabung des Qualitätsphenomens. Göttingen: Vandenhoeck & Ruprecht; 1986. German.
  • 18 Liebe J-D, Hübner U, Straede M-C, Thye J. A top-down approach for developing a workflow composite score to measure clinical information logistics. Methods Inf Med 2015; 54 (Suppl. 05) 424-433.
  • 19 Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966; 44 (Suppl. 03) 166.
  • 20 Donabedian A. Criteria and Standards for Quality Assessment and Monitoring. Quality Review Bulletin 1986; 12 (Suppl. 03) 99-108.
  • 21 Ammenwerth E. Evidence-based Health Informatics: How Do We Know What We Know?. Methods Inf Med 2015; 54 (Suppl. 04) 298-307.
  • 22 Ancker JS, Kern LM, Abramson E, Kaushal R. The Triangle Model for evaluating the effect of health information technology on healthcare quality and safety. J Am Med Inform Assoc 2012; 19 (Suppl. 01) 61-65.
  • 23 Jahn F, Takabayashi K, Suzuki T, Fujita S, Winter A. Comparison between German and Japanese Hospital Information Systems by 3LGM2 analysis. In: Proceedings of the 27th Joint Conference of Medical Informatics (JCMI). Kobe: November 23-25 2007. p. 840-843.
  • 24 Takabayashi K. From Marco Polo Islands. Methods Inf Med 2007; 46 (Suppl. 06) 669-670.
  • 25 Takabayashi K, Hasegawa K, Yamazaki S, Oshima H, Okamura H, Uchida T. [EMI net - electronic medical record sharing in local community for home care medicine]. Gan To Kagaku Ryoho 2003; 30 Suppl 1: 169-172. Japanese.
  • 26 Matsumoto T, Honda M. The evaluation of the need to share medical data on the community medical ICT network service in Nagasaki, Japan. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 1017 (Stud Health Technol Inform; vol. 192).
  • 27 Haux R, Hein A, Kolb G, Kunemund H, Eichelberg M. Five years of interdisciplinary research on ageing and technology: Outcomes of the Lower Saxony Research Network Design of Environments for Ageing (GAL) - an Introduction to this Special Issue on Ageing and Technology. Inform Health Soc Care 2014; 39 3-4 161-165.
  • 28 Kobayashi K, Nakashima N, Inoguchi T, Takaya-nagi R. [Disease management for diabetes mellitus to prevent the onset and progression of complications]. Rinsho Byori 2012; 60 (Suppl. 11) 1101-1106. Japanese.
  • 29 Miyo K, Nittami YS, Kitagawa Y, Ohe K. Development of case-based medication alerting and rec-ommender system: a new approach to prevention for medication error. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 2). Amsterdam: IOS Press; 2007. p. 871-874 (Stud Health Technol Inform; vol. 129).
  • 30 Ammenwerth E, Aly AF, Burkle T, Christ P, Dormann H, Friesdorf W. et al. Memorandum on the Use of Information Technology to Improve Medication Safety. Methods Inf Med 2014; 2014 53(5) 336-343.
  • 31 Prokosch HU, Beck A, Ganslandt T, Hummel M, Kiehntopf M, Sax U. et al. IT Infrastructure Components for Biobanking. Appl Clin Inform 2010; 1 (Suppl. 04) 419-429.
  • 32 Jahn F, Winter A. A KPI Framework for Process-based Benchmarking of Hospital Information Systems. In: User Centred Networked Health Care: Proceedings of MIE 2011. Amsterdam: IOS Press; 2011. p. 542-6 (Stud Health Technol Inform; vol. 169).
  • 33 Hübner U, Ammenwerth E, Flemming D, Schaubmayr C, Sellemann B. IT adoption of clinical information systems in Austrian and German hospitals: results of a comparative survey with a focus on nursing. BMC Med Inform Decis Mak 2010; 10: 8.
  • 34 Aamot H, Kohl CD, Richter D, Knaup-Gregori P. Pseudonymization of patient identifiers for translational research. BMC Med Inform Decis Mak 2013; 13: 75.
  • 35 Farfán FJSedano, Terrón Cuadrado M, Castellanos Clemente Y, Serrano Balazote P, Moner Cano D, Robles Viejo M. Patient Summary and medicines reconciliation: application of the ISO/CEN EN 13606 standard in clinical practice. Stud Health Technol Inform 2011; 166: 189-196.
  • 36 Garde S, Chen R, Leslie H, Beale T, McNicoll I, Heard S. Archetype-based knowledge management for semantic interoperability of electronic health records. In: Medical informatics in a united and healthy Europe: Proceedings of MIE 2009, the XXII International Congress of the European Federation for Medical Informatics. Amsterdam: IOS Press; 2009. p. 1007-1011 (Stud Health Technol Inform; vol. 150).
  • 37 Kohl CD, Garde S, Knaup P. Facilitating secondary use of medical data by using openEHR archetypes. In: Safran C, Marin H, Reti S. editors. MEDINFO 2010 Partnerships for Effective eHealth Solutions. Amsterdam: IOS Press; 2010. p. 1117-1121 (Stud Health Technol Inform; vol. 160).
  • 38 Kobayashi S, Kimura E, Ishihara K. Archetype Model-Driven Development Framework for EHR Web System. Healthc Inform Res 2013; 19 (Suppl. 04) 271-277.
  • 39 Gottesman O, Kuivaniemi H, Tromp G, Faucett WA, Li R, Manolio TA. et al. The Electronic Medical Records and Genomics (eMERGE) Network: past, present, and future. Genet Med 2013; 15 (Suppl. 10) 761-771.
  • 40 van Horn JD, Toga AW. Human neuroimaging as a “Big Data” science. Brain Imaging Behav 2014; 8 (Suppl. 02) 323-331.
  • 41 Gupta P, Janodia MD, Jagadish PC, Udupa N. Medical device vigilance systems: India, US, UK, and Australia. Med Devices (Auckl) 2010; 3: 67-79.
  • 42 Karmen C, Ganzinger M, Kohl CD, Firnkorn D, Knaup-Gregori P. A framework for integrating heterogeneous clinical data for a disease area into a central data warehouse. In: EHealth - for continuity of care: Proceedings of MIE2014. Amsterdam: IOS Press; 2014. p. 1060-1064 (Stud Health Technol Inform; vol. 205).
  • 43 Berg M. Medical Work and the Computer-Based Patient Record: A Sociological Perspective. Methods Inf Med 1998; 37: 294-301.
  • 44 Juhr M, Haux R, Suzuki T, Takabayashi K. Overview of recent trans-institutional health network projects in Japan and Germany. J Med Syst 2015; 39 (Suppl. 05) 234.
  • 45 Schwartze J, Haarbrandt B, Rochon M, Wagner M, Haux R, Kleinschmidt T. et al. Design and implementation of an informed consent process for a standardized health information exchange solution on the example of the lower saxony bank of health. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 318-322 (Stud Health Technol Inform; vol. 192).
  • 46 Neubert A, Dormann H, Prokosch HU, Burkle T, Rascher W, Sojer R. et al. E-pharmacovigilance: development and implementation of a computable knowledge base to identify adverse drug reactions. Br J Clin Pharmacol 2013; 76 Suppl 1: 69-77.
  • 47 EUROREC Institute.. The EuroRec EHR Quality Seal. 2015 [cited 2015 Jan 15].
  • 48 Burkle T, Baur T, Hoss N. Clinical pathways development and computer support in the EPR: lessons learned. In: Ubiquity: Technologies for better health in aging societies: Proceedings of MIE2006. Amsterdam: IOS Press; 2006. p. 1025-1030 (Stud Health Technol Inform; vol. 124).
  • 49 Malin JL. Envisioning Watson as a rapid-learning system for oncology. J Oncol Pract 2013; 9 (Suppl. 03) 155-157.
  • 50 Marincola FM. Translational Medicine: A two-way road. J Transl Med 2003; 1 (Suppl. 01) 1.
  • 51 Matsumura Y, Hattori A, Manabe S, Takeda T, Takahashi D, Yamamoto Y. et al. Interconnection of electronic medical record with clinical data management system by CDISC ODM. In: EHealth - for continuity of care: Proceedings of MIE2014. Amsterdam: IOS Press; 2014. p. 868-872 (Stud Health Technol Inform; vol. 205).
  • 52 Matsumura Y, Kuwata S, Yamamoto Y, Izumi K, Okada Y, Hazumi M. et al. Template-based data entry for general description in medical records and data transfer to data warehouse for analysis. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 1). Amsterdam: IOS Press; 2007. p. 412-416 (Stud Health Technol Inform; vol. 129).
  • 53 Hruby GW, McKiernan J, Bakken S, Weng C. A centralized research data repository enhances retrospective outcomes research capacity: a case report. J Am Med Inform Assoc 2013; 20 (Suppl. 03) 563-567.
  • 54 Ganslandt T, Mate S, Helbing K, Sax U, Prokosch HU. Unlocking Data for Clinical Research - The German i2b2 Experience. Appl Clin Inform 2011; 2 (Suppl. 01) 116-127.
  • 55 Stäubert S, Winter A, Speer R, Loffler M. Designing a Concept for an IT-Infrastructure for an Integrated Research and Treatment Center. In: Safran C, Marin H, Reti S. editors. MEDINFO 2010 Partnerships for Effective eHealth Solutions. Amsterdam: IOS Press; 2010. p. 1319-1323 (Stud Health Technol Inform; vol. 160).
  • 56 Takabayashi K, Ho TB, Yokoi H, Nguyen TD, Kawasaki S, Le SQ. et al. Temporal abstraction and data mining with visualization of laboratory data. In: Kuhn KA, Warren JR, Leong T-Y. editors. Proceedings of Medinfo 2007 (Part 2). Amsterdam: IOS Press; 2007. p. 1304-1308 (Stud Health Technol Inform; vol. 129).
  • 57 McGlynn EA, Lieu TA, Durham ML, Bauck A, Laws R, Go AS. et al. Developing a data infrastructure for a learning health system: the PORTAL network. J Am Med Inform Assoc 2014; 21 (Suppl. 04) 596-601.
  • 58 Danciu I, Cowan JD, Basford M, Wang X, Saip A, Osgood S. et al. Secondary use of clinical data: the Vanderbilt approach. J Biomed Inform 2014; 52: 28-35.
  • 59 Suzuki T, Doi S, Fujita S, Hatakeyama Y, Honda M, Matsumura Y. et al. Construction of the integrated multicentre discharge summary database. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 1064 (Stud Health Technol Inform; vol. 192).
  • 60 Löpprich M, Krauss F, Ganzinger M, Senghas K, Riezler S, Knaup P. Automated classification of selected data elements from free-text diagnostic reports for clinical research. Methods Inf Med 2016; 55 (Suppl. 04) 373-380.
  • 61 Dykes PC, Samal L, Donahue M, Greenberg JO, Hurley AC, Hasan O. et al. A patient-centered longitudinal care plan: vision versus reality. J Am Med Inform Assoc 2014; 21 (Suppl. 06) 1082-1090.
  • 62 S&I Framework.. Longitudinal Coordination of Care - Interoperable Care Plan Exchange - Use Case v2.0: Framework, S&I; 2014 Jan 1 [cited 2016 Jan 31]. Available from: http://wiki.siframe-work.org/file/detail/LCC%20Care%20Plan%20Ex-change%20Use%20Case%20Finalv2.pdf.
  • 63 Honda M, Matsumoto T, Hirose M. A system for health promotion using tele-homecare technology in community. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 940 (Stud Health Technol Inform; vol. 192).
  • 64 Loya SR, Kawamoto K, Chatwin C, Huser V. Service oriented architecture for clinical decision support: a systematic review and future directions. J Med Syst 2014; 38 (Suppl. 12) 140.
  • 65 Staemmler M, Walz M, Weisser G, Engelmann U, Weininger R, Ernstberger A. et al. Establishing end-to-end security in a nationwide network for telecooperation. In: Quality of life through quality of information: Proceedings of MIE2012. Amsterdam: IOS Press; 2012. p. 512-516 (Stud Health Technol Inform; vol. 180).
  • 66 Fonseca M, Karkaletsis K, Cruz IA, Berler A, Oliveira IC. OpenNCP: a novel framework to foster cross-border e-Health services. In: Digital Healthcare Empowering Europeans: Proceedings of MIE2015. Amsterdam: IOS Press; 2015. p. 617-621 (Stud Health Technol Inform; vol. 210).
  • 67 Stäubert S, Schaaf M, Jahn F, Brandner R, Winter A. Modeling Interoperable Information Systems with 3LGM2 and IHE. Methods Inf Med 2015; 54 (Suppl. 05) 398-405.
  • 68 ASTM E2369-12. Standard Specification for Continuity of Care Record (CCR). 2012 Available from: www.astm.org.
  • 69 HL7.. HL7/ASTM Implementation Guide for CDA® R2 -Continuity of Care Document (CCD*) Release 1. 2014 [cited 2014 Dec 04].
  • 70 Eizen K, Masato S, Kazukuni K, Yoshihito N, Ta-kuji K, Ken I. Securing SSL-VPN with LR-AKE to access personal health record. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 930 (Stud Health Technol Inform; vol. 192).
  • 71 Bellazzi R. Big data and biomedical informatics: a challenging opportunity. Yearb Med Inform 2014; 9 (Suppl. 01) 8-13.
  • 72 Staemmler M. Service delivery for e-Health applications. In: User Centred Networked Health Care: Proceedings of MIE 2011. Amsterdam: IOS Press; 2011. p. 537-541 (Stud Health Technol Inform; vol. 169).
  • 73 Holden RJ, Brown RL, Alper SJ, Scanlon MC, Patel NR, Karsh B-T. That’s nice, but what does IT do? Evaluating the impact of bar coded medication administration by measuring changes in the process of care. Int J Ind Ergon 2011; 41 (Suppl. 04) 370-379.
  • 74 Hofdijk J, Séroussi B, Lovis C, Ehrler F, Sieverink F, Ugon A. et al., editors. Transforming Healthcare with the Internet of Things: Proceedings of the EFMI Special Topic Conference 2016. Amsterdam: IOS Press; 2016. (vol 221).
  • 75 Oberlander M, Linnebacher M, Konig A, Bogoevska V, Brodersen C, Kaatz R. et al. The “North German Tumor Bank of Colorectal Cancer”: status report after the first 2 years of support by the German Cancer Aid Foundation. Langenbecks Arch Surg 2013; 398 (Suppl. 02) 251-258.
  • 76 ISACA.. COBIT 5: A Business Framework for the Governance and Management of Enterprise IT; 2013 [cited 2013 Nov 14]. Available from: http://www.isaca.org/cobit/pages/default.aspx.
  • 77 Brigl B, Ammenwerth E, Dujat C, Graber S, Grosse A, Haber A. et al. Preparing strategic information management plans for hospitals: a practical guideline: SIM plans for hospitals: a guideline. Int J Med Inform 2005; 74 (Suppl. 01) 51-65.
  • 78 Haux R. Health information systems - past, present, future. Int J Med Inform 2006; 75 3-4 268-281.
  • 79 World Health Organization, International Telecommunication Union.. National eHealth Strategy Toolkit. Geneva: World Health Organization; 2012. Jan 1. Available from: http://www.itu.int/pub/D-STR-E_HEALTH.05-2012.
  • 80 Shabo A. It’s time for health record banking!. Methods Inf Med 2014; 53 (Suppl. 02) 63-65.
  • 81 Plischke M, Wagner M, Haarbrandt B, Rochon M, Schwartze J, Tute E. et al. The lower saxony bank of health. Rationale, principles, services, organization and architectural framework. Methods Inf Med 2014; 53 (Suppl. 02) 73-81.
  • 82 AXELOS Ltd.. PRINCE2; 2014 [cited 2014 Mar 5]. Available from: http://www.prince-officialsite.com/.
  • 83 Ammenwerth E, Haux R, Bess A, Blomer R, Bott O, Häber A. et al. IT-Projektmanagement in Krankenhaus und Gesundheitswesen - Einführendes Lehrbuch und Projektleitfaden für das taktische Management von Informationssystemen. Stuttgart: Schattauer;; 2005
  • 84 ServiceNow.. KPI LIBRARY; 2014 [cited 2014 Mar 5]. Available from: http://kpilibrary.com/.
  • 85 Hoerbst A, Ammenwerth E. Quality and Certification of Electronic Health Records: An overview of current approaches from the US and Europe. Appl Clin Inform 2010; 1 (Suppl. 02) 149-164.
  • 86 Ammenwerth E, Rigby M. editors. Evidence-based health informatics: Promoting safety and efficiency through scientific methods and ethical policy. Amsterdam, Netherlands: IOS Press; 2016. (Stud Health Technol Inform; vol 222).
  • 87 Hellrung N, Gusew N, Willkomm M, Haux R. IT-based information management in health care networks: the MedoCom approach. In: EHealth Beyond the Horizon-Get IT There: Proceedings of MIE2008. Amsterdam: IOS Press; 2008. p. 623-628 (Stud Health Technol Inform; vol. 136).
  • 88 Mantas J, Ammenwerth E, Demiris G, Hasman A, Haux R, Hersh W. et al. Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics. First Revision. Methods Inf Med 2010; 49 (Suppl. 02) 105-120.
  • 89 Nakashima N, Nohara Y, Ahamed A, Kuroda M, Inoue S, Ghosh PP. et al. An affordable, usable and sustainable preventive healthcare system for unreached people in Bangladesh. In: Lehmann CU, Ammenwerth E, Nohr C. editors. Medinfo 2013. Amsterdam: IOS Press; 2013. p. 1051 (Stud Health Technol Inform; vol. 192).
  • 90 Hafen E, Kossmann D, Brand A. Health data cooperatives - citizen empowerment. Methods Inf Med 2014; 53 (Suppl. 02) 82-86.
  • 91 Kelbert F, Pretschner A. Data usage control enforcement in distributed systems. In: Proceedings of the third ACM conference on Data and application security and privacy. New York, NY, USA: ACM; 2013. p. 71-82.
  • 92 Calvillo J, Roman I, Roa LM. Empowering citizens with access control mechanisms to their personal health resources. Int J Med Inform 2013; 82: 58-72.
  • 93 The Open Group.. The Open Group IT4IT Reference Architecture v1.3; 2014. Available from: https://collaboration.opengroup.org/data/IT4IT/RA_1.3/index.htm?goto=2:1:1:205.
  • 94 Pettit L. Understanding EMRAM and how it can be used by policy-makers, hospital CIOs and their IT teams. World Hosp Health Serv 2013; 49 (Suppl. 03) 7-9.
  • 95 DeLone WH, McLean ER. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems 2003; 19 (Suppl. 04) 9-30.

Zoom Image
Figure 1 Workshop participants at Kloster Banz.
Zoom Image
Figure 2 Results of the creativity techniques on the quality issues of EHRS.