Appl Clin Inform 2017; 08(03): 893-909
DOI: 10.4338/ACI-2016-02-R-0023
Research Article
Schattauer GmbH

Enhancing Patient Safety Event Reporting

A Systematic Review of System Design Features
Yang Gong
1  The University of Texas Health Science Center at Houston, School of Biomedical Informatics, Houston, Texas, United States
,
Hong Kang
1  The University of Texas Health Science Center at Houston, School of Biomedical Informatics, Houston, Texas, United States
,
Xinshuo Wu
1  The University of Texas Health Science Center at Houston, School of Biomedical Informatics, Houston, Texas, United States
,
Lei Hua
1  The University of Texas Health Science Center at Houston, School of Biomedical Informatics, Houston, Texas, United States
› Author Affiliations
Further Information

Correspondence to:

Yang Gong, MD, PhD
The University of Texas Health Science Center at Houston
School of Biomedical Informatics, Houston, Texas, US
Phone: 7135003547   

Publication History

17 March 2017

25 June 2017

Publication Date:
20 December 2017 (online)

 

Summary

Objectives: Electronic patient safety event reporting (e-reporting) is an effective mechanism to learn from errors and enhance patient safety. Unfortunately, the value of e-reporting system (a software or web server based platform) in patient safety research is greatly overshadowed by low quality reporting. This paper aims at revealing the current status of system features, detecting potential gaps in system design, and accordingly proposing suggestions for future design and implementation of the system.

Methods: Three literature databases were searched for publications that contain informative descriptions of e-reporting systems. In addition, both online publicly accessible reporting forms and systems were investigated.

Results: 48 systems were identified and reviewed. 11 system design features and their frequencies of occurrence (Top 5: widgets (41), anonymity or confidentiality (29), hierarchy (20), validator (17), review notification (15)) were identified and summarized into a system hierarchical model.

Conclusions: The model indicated the current e-reporting systems are at an immature stage in their development, and discussed their future development direction toward efficient and effective systems to improve patient safety.

Citation: Gong Y, Kang H, Wu X, Hua L. Enhancing Patient Safety Event Reporting. Appl Clin Inform 2017; 8: 893–909 https://doi.org/10.4338/ACI-2016-02-R-0023


#

Background and Significance

A patient safety event is defined as any process, act of omission, or commission that results in hazardous healthcare conditions and/or unintended harm to the patient [[1]]. Reporting patient safety events is a useful approach for improving patient safety [[2]]. The mechanism of event reporting was first introduced in the high-risk industries such as aviation, nuclear, rail industry, etc. to improve safety and enhance organizational learning from errors. The mechanism was then extended to healthcare systems with additional features such as anonymous reporting, meaningful feedback, ease of reporting, etc [[3], [4]]. Through collecting adverse events and near misses in healthcare, the reporting systems enable safety specialists to analyze events, identify underlying factors, and generate actionable knowledge to mitigate risks [[5]]. Since the emergence of electronic patient safety reporting (e-reporting) systems, the collection and analysis of events tend to become a more efficient manner than traditional paper-based systems [[6], [7]]. However, the promising benefits of such systems in healthcare are yet to be fully seen. Currently, low quality reports have been found as a roadblock hampering the data utility for quality improvement and patient safety research. It is reported that a large amount of inaccurate and incomplete information, such as inconsistent records and mis-classifications, is typically contained in the systems [[8], [9]].

Similar to Electronic Health Record (EHR) systems, data quality in general regarding accuracy, completeness, and timeliness has been a main concern [[10]]. Substantial studies have linked the poor data quality and lack of integration to the design flaws in functionality and usability. A well-designed system tends to generate data of high quality [[11]]. Likewise, a well-designed e-reporting system could serve as a facilitator to enhance data quality for patient safety.

Earlier studies show that quality and rate of event reporting can be greatly affected by user interface associated with human factors [[8], [12]]. It was argued that an effective design of e-reporting system should support social-cognitive process of potential reporters [[13]]. An effective and efficient e-reporting system should guide a reporter to go through the reporting details step-by-step without costing additional time and efforts of the reporter. Unfortunately, by far the design features of e-reporting systems have been addressed in a fragmented way across studies. Although there are reviews and comparative studies regarding extant reporting system design [[13]–[15]], the discussions were limited to organizational culture and never extended to the nuances of interface design. For that reason, the objective of the study was to explore the current status of e-reporting in terms of design features which could improve data quality, to identify the feature hierarchy in terms of the stage of development, and to detect the potential technical gaps and challenges of developing an e-reporting system. To our best knowledge, our study is the first of its kind posing a significant step forward in identifying essential features of e-reporting associated with data quality.

Data quality in an e-reporting system is defined as a multidimensional concept, including accuracy, completeness, and timeliness [[16]]. We define the three data quality dimensions as follows: (1) Accuracy: the degree of proximity of a given patient safety event report to corresponding real world occurrences. The reporting accuracy is subject to user errors and cognitive limitations in memory and reasoning, including but not limited to typographical errors, memory decay, causal attribution and hindsight biases. The accuracy of e-reporting could be improved if these contributing factors are incorporated into design considerations with good usability and functionality. (2) Completeness: the degree to which a given patient safety event report includes necessary information describing the corresponding real world event so as to be sufficiently valid for the purpose of analysis and generation of intervention. The completeness could be enhanced if its criteria are explicitly delineated and properly represented to the reporters with the help of interface features. (3) Timeliness: the degree to which a patient safety event is reported in a timely manner for root cause analysis and the generation of real time intervention. The timeliness could be enhanced by improving the efficiency of the reporting process and offering a smooth review process to generate actionable knowledge.


#

2. Objectives

The purpose of this study was to reveal the current status of design features of electronic patient safety event reporting (e-reporting) systems, detect potential gaps in system design, and accordingly propose suggestions for future design and implementation.


#

3. Methods

3.1. Search strategy

Search resources for peer reviewed publications included three databases: Ovid MEDLIINE (1946 to December week 3 2016), PsycINFO (1806 to December week 4 2016), and Health and Psychological Instruments (1985 to December 2016). Search terms including “medical error/incident/event”, “report/reporting system”, “electronic report”, “healthcare”, and “information system” were applied with different combinations to all field search (title, abstract, keywords, etc.). As this strategy may include articles with high sensitivity and low specificity, we set restrictions on the MeSH Subject Heading to match such term clusters as “Risk/Safety Management”/ “Quality of Health Care”/ “Quality Assurance” and “Patients or Medical Records Systems”/ “Computerized or Hospital Information Systems”. The searching process involved multiple steps with step-specific rules (details shown in ►[Table 1]). Final searches were run on Feb 1, 2017.

Table 1

Search Strategy for Peer Reviewed Publications

Steps

Search Type

Searches

Results

1

All field search

(medical error$ or medical incident$ or medical event$ or near miss$ or medication incident$ or medication error$ or patient safety) and report$ system

808

2

All field search

Incident registry

9

3

All field search

(medical error$ or medical incident$ or medical event$ or near miss$ or medication incident$ or medication error$ or patient safety) and electronic report$

41

4

All field search

(electronic incident report$ or electronic error report$ or electronic error report$) and health care

8

5

All field search

patient occurrence and report$

9

6

All field search

health care and information system$ and error report$

53

7

All field search

(incident report$ system or error report$ system or event report$ system or near miss report$ system) and health care

340

8

Combine

1 or 2 or 3 or 4 or 5 or 6 or 7

971

9

Mesh Subject Heading search

Quality of Health Care or Risk Management or Safety Management or Quality Assurance, Health Care or Patient Safety

191351

10

Mesh Subject Heading search

(Academic Medical Center or Nursing Homes or Medical Records Systems, Computerized or Hospital Information Systems or Adverse Drug Reaction Reporting Systems or Medical Errors) and Humans

79179

11

Combine

8 and 9 and 10

287

12

Remove duplicates

Remove duplicate 11

285


#

3.2. Study selection

The inclusion and exclusion criteria were based upon the review objective, which was to reveal design features of e-reporting systems. Studies that fell into one of the five following categories were included:

  1. Evaluation studies of electronic patient safety event reporting systems;

  2. Intervention studies to improve medical error reporting rates by introducing a new electronic patient safety event reporting system;

  3. Cross-sectional or case-control studies involving reported error rates or error reporting rates;

  4. Review studies of electronic patient safety event reporting systems implemented or still in use at healthcare systems or patient safety organizations;

  5. Studies introducing the process of developing an electronic patient safety event reporting system.

Studies were excluded if one of the following applied:

  1. Studies published in a language other than English;

  2. Studies missing descriptions of the system used for collecting error data;

  3. Studies focusing on paper-based reporting.


#

3.3. Data extraction and organization

Following the search strategy described above, we identified 287 articles which were imported into a reference management software Endnote™ X7. After removing two duplicates and reviewing the reference lists of the 287 articles, we identified additional seven articles, defined as grey literature, and appeared in the reference section through manual searches. As a result, 292 articles were included for further screening. Then, one author examined the titles and abstracts against the inclusion and exclusion criteria, 151 articles were excluded, among which 60% were due to lack of system description rather focusing on error data analytics. The rest excluded were due to one of the reasons, i.e. English abstract only, paper-based system or non-major topic in the retrieval articles. The remaining 141 articles with full text were further scrutinized by two researchers to guarantee the consistency in making decisions.

The entire review process is presented as a PRISMA flow diagram [[17]], which is a commonly used technique to ensure the transparency of systematic reviews and meta-analyses (►[Figure 1]). To identify developing levels of e-reporting systems described in the literature, we extracted design attributes and organized them into a hierarchy in an EMR Adoption Model [[18]], which is commonly used to track EMR progress at hospitals and health systems. The model includes eight stages (0–7) for different levels of EMR cumulative capabilities, based on which hospital executives compare their EMR adoption to similar facilities as well as compare to their state’s average level [[19]]. In this study, we matched each attribute with the capability statement of stages and then placed in the hierarchy with the consideration of its prevalence, practical significance, and technical complexity. Specifically, three researchers who are familiar with patient safety event reporting and patient safety data were invited to review the literatures. They also summarized the design features during reviewing. A pre-discussion was conducted to make sure the features in the highest and lowest levels. The rest features were ranked by the experts individually according to the practical significance. A final discussion was conducted to determine the level of each feature. The occurrence of certain feature was counted if a system applies this feature; and the frequencies of identified features were calculated based upon the most recent publication regarding an identical system. Thus, the maximal frequency of certain feature equals the number of identified systems when those systems exhibit such a feature.

Zoom Image
A PRISMA flow diagram for the literature review process

#

3.4. Search for additional e-reporting systems accessible online

In order to go beyond the peer-reviewed literature, we used Google search engine to identify publicly accessible e-reporting systems and other information sources that contain substantial system descriptions (e.g. screen shots or demonstration videos) as supplemental information. Detailed search strategies are shown in Appendix A. The final searches for supplemental materials were run on Feb 1, 2017. This process detected eight web-based reporting systems, four PDF documents containing system screenshots and descriptions, and two videos introducing event-reporting systems. System design features extracted from these additional materials were merged into the results of the literature review.


#

3.5. Review strategy

A 4-point Likert scale in which 1=irrelevant, 2=somewhat irrelevant, 3=relevant, and 4=highly relevant was adopted to identify literature and system features. Three researchers assigned every literature/feature 1–4 points according to the relational degree. After their individual reviews, the researchers as a panel discussed the results to ensure concordance of the reviews. Three rounds of discussion were performed to reach concordance among the panelists. If the panelists could not reach an agreement about a certain case, the majority would make a decision. Any literature/feature that is labeled with 3 or 4 by majority was regarded as an “agreement” and was included; thus, a literature/feature that is labeled with 1 or 2 by majority was considered an “agreement” but was excluded.


#
#

4. Results

After a careful integration by the researchers, 48 unique e-reporting systems were identified (►[Table 2]). Within the 48 systems, we were able to summarize the following features as a trend of the e-reporting systems: widgets, anonymity or confidentiality, hierarchy, validator, review notification, and reference (►[Table 3]).

Table 2

E-reporting Systems of Patient Safety Events; # System information derived from peer reviewed literature, * System information derived from webpages

System

Year of Publication

Country

System feature (see Table III for feature descriptions)

The Quality Assurance (QA) Database Application* [[32]]

2011

United States

Validator; feedback and communication; reference; integrated interface; one pager; widgets; hierarchy; anonymity or confidentiality

Anesthesia Incident Reporting System* [[48]]

2011

United States

Widgets; hierarchy; validator; anonymity or confidentiality

Anesthesia Incident Reporting System (AIRS)# [[25]]

2011

United States

Integrated interface; anonymity or confidentiality

Patient Safety Reporting System (PSRS) Report Form* [[49]]

2010 (updated)

United States

Widgets

Patient Safety Reporting Form* [[50]]

2010

United States

Widgets; validator; anonymity or confidentiality

AHRQ Common Formats Patient Safety Events Reporting System* [[33]]

>2009

United States

Widgets; hierarchy; feedback and communication

Medication Error Quality Initiative– Individual Error (MEQI-IE) Reporting System# [[34]–[36]]

2006 (introduced) 2009 (updated)

United States

Phonetic algorithm; reference; hierarchy; widgets; feedback and communication

Duke University Health System Safety Reporting System (SRS)# [[37], [38]]

2009

United States

Review notification; hierarchy; widgets; one pager; feedback and communication; anonymity or confidentiality

The Family Reporting System (DATIX Software)* [[26]]

2009

Canada

Validator; reference; integrated interface; widgets; hierarchy

A Web-based Medication Report Form* [[51]]

2008

United States

Color coding; widgets; validator; one pager; anonymity or confidentiality

The MEADERS Event Reporting Form* [[52]]

2008

United States

Widgets

Medi-Event System# [[27]]

2006

United Kingdom

Integrated interface; widgets; reference

Incident Registry Form# [[28]]

2006

Netherlands

Widgets; one pager; reference; integrated interface; anonymity or confidentiality

Incident Reporting and Review eForm (version 4)# [[39]]

2006

Canada

Review notification; widgets; feedback and communication

A Microsoft Access Data Collection Tool* [[53]]

2006

United States

Widgets

Non-optimal Care and Safety Event Form* [[54]]

2006

United States

Widgets; anonymity or confidentiality

Critical Incident Reporting System (CIRS)*# [[55], [56]]

2005

Germany

Widgets; one pager; anonymity or confidentiality; validator

DATIX*# [[23], [29], [40], [41]]

2005

United Kingdom

Review notification; hierarchy; widgets; one pager; integrated interface; anonymity or confidentiality; feedback and communication; validator; color coding

Radiation Oncology Incident Reporting System (ROIR)* [[57]]

2005

United States

Widgets; hierarchy; anonymity or confidentiality; color coding; validator; one pager

A CQI Web-page Reporting System* [[58]]

2005

United States

Widgets; one pager; anonymity or confidentiality

Jeder Fehler Zaehlt*# [[59], [60]]

2004

Germany

Hierarchy; widgets; anonymity or confidentiality; reference; validator

Risk Monitor Pro# [[61]]

2004

United States

Review notification; widgets; hierarchy

A Web-based incident reporting system in Taiwan# [[6]]

2004

China

Validator; review notification

Department of Health Patient Safety Reporting System, State of New Jersey* [[62]]

2004

United States

Widgets; hierarchy; validator; anonymity or confidentiality; reference; color coding

UHC Patient Safety Net Event Report* [[63]]

2004

United States

Widgets; validator; color coding; reference

The Medication-Error Reporting System* [[64]]

2004

United States

Validator

National Patient Safety Agency Incident Reporting Form* [[65]]

2003

United Kingdom

Widgets; validator; anonymity or confidentiality; reference; color coding; hierarchy

The University of Texas Close Call Reporting System# [[24]]

2003

United States

Review notification; widgets; feedback and communication

Falls Menu-Driven Incident-Reporting System (MDIRS)# [[66]]

2003

United States

Validator; widgets

The Ohio State University Health System (OSUHS) Event Reporting System# [[30]]

2003

United States

Reference, review notification; hierarchy; widgets; anonymity or confidentiality; feedback and communication; integrated interface

Medication Error Occurrence Report# [[7]]

2003

United States

Validator; review notification; hierarchy; integrated interface; anonymity or confidentiality

An Electronic Medical Error Reporting System* [[67]]

2003

United States

Anonymity or confidentiality

Event Reporting Management System (ERMS)* [[68]]

2003

United States

Review notification; widgets

The Monthly Summary Report (MSR)* [[69]]

2003

United States

Widgets

Pennsylvania Patient Safety Anonymous Report Form* [[70]]

2002

United States

Widgets

Incident Reporting Form# [[31]]

2002

Australia

Widgets; integrated interface

Baylor Health Care Web Forms# [[71], [72]]

2002 (updated)

United States

Reference; review notification; validator; anonymity or confidentiality

Electronic Error Reporting System# [[73], [74]]

2002

United States

Review notification; hierarchy; anonymity or confidentiality

Web-based Intensive Care Unit Safety Reporting System# [[75]]

2002

United States

Reference; hierarchy; widgets; color coding; anonymity or confidentiality

University of Missouri Health Care (MUHC) Patient Safety Network System# [[42]]

2002

United States

Review notification; hierarchy; widgets; feedback and communication

New Anonymous Medical Error Report Form# [[76]]

2002

United States

Widgets; reference

The Occurrence Screen Database* [[77]]

2002

United States

Review notification; widgets; anonymity or confidentiality

Potential Error and Event Reporting System# [[78]]

2001

United States

Review notification; hierarchy; anonymity or confidentiality

Electronic Reporting System (ERS) # [[9], [79]]

2001

United States

Hierarchy; widgets; anonymity or confidentiality

Incident Reporting System* [[80]]

2001

Japan

Widgets; anonymity or confidentiality

The ASIPS Patient Safety Reporting System* [[81]]

2001

United States

Widgets; anonymity or confidentiality

Medical Incident Reporting System (MIRS)* [[82]]

2001

United States

Widgets; review notification

Online Incident Reporting System (OIRS) # [[83], [84]]

2000

Japan

Hierarchy; widgets; anonymity or confidentiality

Table 3

Identified Design Features in the E-reporting Systems

System design feature

Description

Frequency

Percentage

Widgets

Drop-down lists, check boxes, or radio buttons are used to replace plain text input for convenience when appropriate.

41

85.42%

Anonymity or confidentiality

Allow reporters to report anonymously.

29

60.42%

Hierarchy

Relevant information is shown based on previous user choices.

20

41.67%

Validator

A timely validator to prevent leaving mandatory items blank, inconsistent entries, incorrect formats, etc.

17

35.42%

Review notification

Reviewers are notified by the system generated message once the review is completed.

15

31.25%

References

Search box, examples, or instructions are provided.

13

27.08%

Integrated interface

Reporting interface is integrated into other clinical applications, or the data or functionality of other clinical applications can be shared with the reporting system.

9

18.75%

Feedback and communication

Reporters are allowed to track event review or resolution status, see reviewers’ comments on previously reported cases, and get feedback on the aggregate level (e.g. statistics).

9

18.75%

One pager

All items are shown on one page to reduce potential technical delays.

8

16.67%

Color coding

Important numbers or text are highlighted by colors.

6

12.50%

Phonetic algorithm

Words are indexed by pronunciation to reduce spelling errors.

1

2.08%

The systems shown in ►[Table 2] were implemented in healthcare institutions across the world in the United States, Netherlands, Canada, United Kingdom, Germany, Australia, China, and Japan. The use of e-reporting systems was not limited in a particular clinical area. For example, some of them focus on general patient safety events and some others focus on specific areas such as anesthesia events and radiation oncology events. The time spans over ten years with the oldest introduced in 2000 and the latest in 2011.

The design features were derived and summarized in ►[Table 3], which includes detailed description and occurrence frequency of each feature. Among the features, ‘widgets’ (frequency=41) and ‘anonymity or confidentiality’ (frequency=29) are the most popular features. ‘Validator’, ‘Reference’, ‘Review notification’, and ‘Hierarchy’ revealed an intermediate prevalence. The remaining features were considered as non-widely used.

These derived features were then organized into a conceptual hierarchy as shown in ►[Table 4]. Similar to the evolvement of paper chart to EHR, the features show a trend of advancement of user-centered design. The designs at the early stages (stages 0–2) simply transformed paper forms into e-forms where the features ensuring data quality (stages 3–6) were not pervasive. Based upon this model, we found 12 out of all 48 identified e-reporting systems were actually electronic copies of paper-based reporting forms rather than interactive systems.

Table 4

A Feature Hierarchy Identified in the E-reporting Systems

System design feature

Description

Frequency

Percentage

Widgets

Drop-down lists, check boxes, or radio buttons are used to replace plain text input for convenience when appropriate.

41

85.42%

Anonymity or confidentiality

Allow reporters to report anonymously.

29

60.42%

Hierarchy

Relevant information is shown based on previous user choices.

20

41.67%

Validator

A timely validator to prevent leaving mandatory items blank, inconsistent entries, incorrect formats, etc.

17

35.42%

Review notification

Reviewers are notified by the system generated message once the review is completed.

15

31.25%

References

Search box, examples, or instructions are provided.

13

27.08%

Integrated interface

Reporting interface is integrated into other clinical applications, or the data or functionality of other clinical applications can be shared with the reporting system.

9

18.75%

Feedback and communication

Reporters are allowed to track event review or resolution status, see reviewers’ comments on previously reported cases, and get feedback on the aggregate level (e.g. statistics).

9

18.75%

One pager

All items are shown on one page to reduce potential technical delays.

8

16.67%

Color coding

Important numbers or text are highlighted by colors.

6

12.50%

Phonetic algorithm

Words are indexed by pronunciation to reduce spelling errors.

1

2.08%

The development of e-reporting systems over the years in terms of design features that we had identified was summarized in ►[Table 4]. The hierarchy contains three phases and seven stages, which were paper form (phase I, stage 0), e-form (phase II, stages 1 and 2), and e-reporting (Phase III, stages 3–6). The features aiming to improve accuracy, completeness, or timeliness present an increasing trend over time. Accuracy was not adequately considered in the early stage design, i.e. Phase I & II in terms of usability and functionality, which frequently resulted in user errors, and cognitive limitations in memory and reasoning. There has been a clear trend observed in Phase III, where accuracy of e-reporting is greatly facilitated through intelligent features, such as full validator, data entry prediction, etc. Completeness was not guaranteed in Phase I because the paper form-based reporting systems did not have completeness checking features. A systematic consideration of interoperability promotes both completeness and timeliness when a few common fields can be populated by linking patient’s medical record systems. Meanwhile, timeliness is enhanced by instant communication between reporter and reviewer through feedback access or notification.

Similar to the evolution of EHR, e-reporting systems started from an electronic copy of the paper-based reporting form. In this phase, the reporting systems can be viewed as a primitive alteration of paper-based reporting forms. The use of drop-down lists, check boxes, or radio buttons replaces unnecessary free text boxes, which accelerates the electronic entry process and improves data accuracy by reducing data entry errors. To keep abreast of EHR systems, ideal e-reporting systems should be characterized by flexibility, adaptability, reasoning, temporal dynamics and should contain associated functionalities that convert these essentials toward an improved data quality [[20]].

To further enhance e-reporting systems in terms of accuracy, completeness, and timeliness, e-reporting system designers should incorporate the following features we have characterized through this study.

Validator: By setting embedded constrains or alerts in data entry fields, validators prevent the occurrence of future date, leaving mandatory items blank, inconsistent entry, typographical errors, and incorrect data formats, etc. In e-commerce, for example online banking, validators are a widely used technique in electronic form design to enhance accurate and complete data entry [[21]].

Hierarchy: The hierarchical data layout can reduce the appearance of unreasonable answers from the reporters. Such a layout reduces reporters’ memory load for particular task operations and decreases the likelihood of skipping correct answers. Additionally, this format can lead a top-down reporting process starting from the general to the specific in a logical flow, which potentially helps bridge the reporter’s reasoning and memory gap. The efficiency of electronic data entry process, accuracy and completeness of collected data could also be improved through such a hierarchical layout [[22]].

Access to feedback: This design feature enables reporters to detect inaccurate or incomplete content in previous cases based upon reviewers’ feedback, and to improve the quality of their own reporting. This feature can also help detect certain memory-dependent inaccuracy to which a validator is usually not helpful. The comments or feedback from reviewers may also serve as an individualized event reporting instruction and good resource for reporters to learn from. This interactive communication in a long run may form an iterative learning process to facilitate quality reporting in a virtuous circle.

Data sharing: The function of utilizing stored data in other clinical applications (e.g. EHR) may not have an effect on data quality improvement, but may serve as a basis for developing quality-enhanced artifacts. For example, several e-reporting systems can check patient demographic information against pre-stored patient data, or can incorporate search functionality and ID index for auto-populating [[23], [24]]. In addition, sharing event data among e-reporting systems will contribute to a knowledge base which allows implementing technically sophisticated design, such as event prediction or risk analysis based on similar events in the knowledge base.


#

5. Discussion

5.1. Identified potential technical gaps

Comparing the characteristics discussed above and what we identified through the included patient safety reporting systems, there are technical gaps that can be potentially bridged through system re-engineering without radically changing the systems in use so that accurate and complete patient safety data would be collected in a timely manner.

First of all, the validator function of current reporting systems is still at its infancy. Most of the identified systems only incorporated a few simple edit checks, such as misspelling alerts on the interface. The ratios of the number of edit checks to the number of question items are low. No functionality that targets at internal consistency between questions was identified. Hence, errors of oblivious, mutual contradictory answers are always accepted by the system without any warnings or alerts. Additionally, free text description of events, where major event information is collected, lacks assistant functionality to ensure data entry accuracy and completeness. Of all the identified systems, only one involved an alert of insufficient event description [[23]]. The other systems merely set this section as an optional data entry process.

In addition, current systems exhibit a lack of interoperability and communication. For improving accuracy, completeness and timeliness, it is essential that generic information regarding location and patient should be incorporated with other healthcare information systems, such as EHRs.

Nine identified systems incorporated features that tend to integrate the reporting form into other clinical applications [[7], [25]–[32]]. However, few would allow sharing pre-stored information. For example, in the case when event independent information (e.g. patient demographic data) is required, a reporter needs to manually enter the information that already exists elsewhere in the reporting form, which could cause unnecessary time consumption in a clinical setting. Communication is another feature that is rarely addressed across current systems. Most systems do not contain any description of the mechanism through which an effective communication between reporters and reviewers can be achieved, or merely involved periodical feedback at the aggregate level through meetings, newsletters, or education training programs. Several systems have included the feedback feature in their system design [[23], [24], [29], [30], [32]–[42]]. However, the functionality supporting communication was rarely mentioned.

Overall, current e-reporting systems present a pattern of technical simplicity, which attributes to both technical and non-technical challenges. The adoption of such features should be a technical challenge. Some non-technical reasons may stem, at least partially, from organizational issues [[43]]. For instance, the scarcity of field validators probably indicates a concern about additional security problems came from multiple databases access in a clinical setting. Lacking considerations on system interoperability and communication indicates a poor integration of event reporting procedure into clinicians’ workflow, organizational quality control, and risk management process.


#

5.2. Important considerations and future direction

On the basis of our review of published and grey literature on 48 e-reporting systems, we proposed several essential considerations for the effective design of future e-reporting systems.

  1. Provide embedded validators to assist electronic data entry. Specify pre-determined constrains to reject or warn users when illegal or unlikely values are entered. This feature has been proposed as one of the best practices for electronic form design [[21]]. Along with the advancement of software and users’ experience, some automatic checking on entries are actually essential across all systems [[44]].

  2. Structure a hierarchical data entry supporting adaptive interactions. The display of hierarchical information guided by logical rules has already been a pervasive feature in generic user interfaces. More technically sophisticated methods involving adaptive models or Bayesian network are suggested in e-reporting systems in healthcare. These techniques could optimize the sequence of user selection based upon stored answers in knowledge base. Simulation studies regarding these techniques demonstrated an improvement in accuracy, completeness, and efficiency [[21]].

  3. Incorporate assistant functionality to facilitate free text entry. We have been developing predictive artifacts built in a prototype e-reporting system. The artifacts of auto-suggestion and solution recommend free text through a case-based reasoning [[45]]. Our evaluation result on the artifacts is promising in improving reporting accuracy, completeness, and efficiency.

  4. Provide independent interfaces and recommendation strategies on a role basis in e-reporting, because doctors, nurses, and patients may have different perspectives to provide.

  5. Enable user to view reviewer’s feedback and comments on other relevant reports so as to promote learning and sustain quality reporting.

  6. Enhance interoperability with other clinical applications and allow data sharing.

Due to the promising value of e-reporting systems in patient safety research and quality improvement, we envision there will be a widespread implementation and application of the systems in healthcare. An increasing amount of data generated from e-reporting systems will be utilized for pattern recognition and root-cause analysis. However, our exploration of current e-reporting systems reveals that the development of such systems is still at the primary stage in terms of system usability and functionality. Large amounts of low quality data generated by poorly designed systems significantly hampered system’s effectiveness [[8]]. Lessons learned from EHR evolution indicate that the development of an effective health information system requires multidisciplinary knowledge, including medicine, human factors, cognitive sciences, usability engineering, and others [[46]]. In our another pioneering study, we developed a novel schema to improve the data quality of patient safety event reporting systems based on the design features proposed in this study [[47]]. We believe these efforts and considerations serve as a basis and reference for the application of domain knowledge to promote a rapid growth of e-reporting systems.


#

5.3. Limitations

The information about e-reporting systems was obtained from secondary study of published peer-refereed and grey literature instead of empirical research. It is inevitable that certain system features may be overlooked and were not included in the descriptions in the identified articles. This may lead to our underestimation of the prevalence of specific features and overemphasis on their rarity. Additionally, our study reported here merely focuses on system features associated with data quality. Other design elements (e.g., role-based interface design and multiplatform support) may be also necessary for e-reporting systems but not analyzed and discussed in this report. On the other hand, the features included in this study may have limited generalizability due to the reluctance of organizations to discuss these systems publicly owing to the sensitive nature of such systems.


#
#

6. Conclusions

In this study, 11 system design features of patient safety reporting systems, i.e., widgets, anonymity or confidentiality, hierarchy, validator, review notification, references, integrated interface, feedback and communication, one pager, color coding, and phonetic algorithm, were identified and discussed. The findings indicate that current e-reporting systems remain at an early stage of development, thus more efforts are needed to address the technical gaps and challenges. Essential features or functionalities that enhance data entry quality were sparsely identified. Accordingly, these features and the proposed considerations serve as a guidance by which an efficient and effective e-reporting system is promising in future development and implementation.


#

Multiple Choice Questions

Question: As design features of e-reporting systems, which of the following is more advanced than others? Options: A. Feedback and communication, B. Electronic copy of reporting form, C. System interoperability, D. Hierarchy

Answer: C. System interoperability


#
#

Conflict of Interest

The authors declare that they have no conflicts of interest in the research.

Acknowledgement

This project was in part supported by a grant on patient safety from the University of Texas System (grant number 156374) and a grant from AHRQ (grant number 1R01HS022895), and UTHealth Innovation for Cancer Prevention Research Training Program Post-doctoral Fellowship (Cancer Prevention and Research Institute of Texas grant #RP160015).

Clinical Relevance Statement

This study advances the knowledge regarding the design features of the current e-reporting systems. Our results hold promise in facilitating the development of e-reporting systems and improving the patient safety in clinical settings.


Protection of Human and Animal Subjects

Human and animal subjects were not included in the project.



Correspondence to:

Yang Gong, MD, PhD
The University of Texas Health Science Center at Houston
School of Biomedical Informatics, Houston, Texas, US
Phone: 7135003547   


Zoom Image
A PRISMA flow diagram for the literature review process