Text Prediction on Structured Data Entry in HealthcareA Two-group Randomized Usability Study Measuring the Prediction Impact on User Performance
13 November 2013
accepted: 18 January 2014
20 December 2017 (online)
Background: Structured data entry pervades computerized patient safety event reporting systems and serves as a key component in collecting patient-related information in electronic health records. Clinicians would spend more time being with patients and arrive at a high probability of proper diagnosis and treatment, if data entry can be completed efficiently and effectively. Historically it has been proven text prediction holds potential for human performance regarding data entry in a variety of research areas.
Objective: This study aimed at examining a function of text prediction proposed for increasing efficiency and data quality in structured data entry.
Methods: We employed a two-group randomized design with fifty-two nurses in this usability study. Each participant was assigned the task of reporting patient falls by answering multiple choice questions either with or without the text prediction function. t-test statistics and linear regression model were applied to analyzing the results of the two groups.
Results: While both groups of participants exhibited a good capacity of accomplishing the assigned task, the results were an overall 13.0% time reduction and 3.9% increase of response accuracy for the group utilizing the prediction function.
Conclusion: As a primary attempt investigating the effectiveness of text prediction in healthcare, study findings validated the necessity of text prediction to structured date entry, and laid the ground for further research improving the effectiveness of text prediction in clinical settings.
Citation: Hua L, Wang S, Gong Y. Text prediction on structured data entry in healthcare: A two-group randomized usability study measuring the prediction impact on user performance. Appl Clin Inf 2014; 5: 249–263 http://dx.doi.org/10.4338/ACI-2013-11-RA-0095
- 1 McDonald CJ. The barriers to electronic medical record systems and how to overcome them. Journal of the American Medical Informatics Association 1997; 4 (03) 213-221.
- 2 Kaplan B. Reducing barriers to physician data entry for computer-based patient records. Top Health Inf Manage 1994; 15 (01) 24-34. PubMed PMID: 10135720. Epub 1994/07/07.
- 3 Walsh SH. The clinician’s perspective on electronic health records and how they can affect patient care. BMJ 2004; 328 7449 1184-1187.
- 4 Structured data capture initiative 2013. Available from: http://wiki.siframework.org/Structured+Data+Capture+Initiative.
- 5 AHRQ.. Users’ guide AHRQ Common formats for patient safety organizations : Agency for healthcare Research and Quality. 2008
- 6 Beukelman D, Mirenda P. Augmentative and Alternative Communication. ISBN 1–55766–333–5: Brookes; 2005.
- 7 Mackenzie SI, Soukoreff WR.. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction 2002; 17 2 & 3 147-198.
- 8 Eng J, Eisner JM. Informatics in radiology (infoRAD): Radiology report entry with automatic phrase completion driven by language modeling. Radiographics 2004; 24 (05) 1493-1501.
- 9 Koester HH, Levine SP. Learning and performance of able-bodied individuals using scanning systems with and without word prediction. Assistive technology: the official journal of RESNA 1994; 6 (01) 42-53. PubMed PMID: 10147209. Epub 1993/12/09.
- 10 Millar DC, Light JC, Schlosser RW. The Impact of Augmentative and Alternative Communication Intervention on the Speech Production of Individuals With Developmental Disabilities: A Research Review. J Speech Lang Hear Res 2006; 49 (02) 248-264.
- 11 Gevarter C, O’Reilly MF, Rojeski L, Sammarco N, Lang R, Lancioni GE. et al. Comparisons of intervention components within augmentative and alternative communication systems for individuals with developmental disabilities: A review of the literature. Research in Developmental Disabilities 2013; 34 (12) 4404-4414.
- 12 Kim J, Bates DW. Results of a survey on medical error reporting systems in Korean hospitals. Int J Med Inform 2006; 75 (02) 148-155. PubMed PMID: 16095963. Epub 2005/08/13.
- 13 Gong Y. Data consistency in a voluntary medical incident reporting system. J Med Syst 2009; 35 (04) 609-615. PubMed PMID: 20703528. Epub 2010/08/13.
- 14 Gong Y. Terminology in a voluntary medical incident reporting system: a human-centered perspective. Proceedings of the 1st ACM International Health Informatics Symposium. Arlington, Virginia, USA.: 1882996: ACM; 2010 p. 2-7.
- 15 Rosenthal J, Takach M. 2007 guide to state adverse event reporting systems. National Academy for State Health Policy. 2007
- 16 Kohn LT, Corrigan J.. M., Donaldson, M. S. To err is human: building a safer health system. Report of Committee on Quality of Healthcare in America. Institute of Medicine, National Academy of Science. 1999
- 17 Hua L, Gong Y. Developing a user-centered voluntary medical incident reporting system. Studies in health technology and informatics 2010; 160 Pt 1 203-207.
- 18 Hua L, Gong Y. Usability evaluation of a voluntary patient safety reporting system: Understanding the difference between predicted and observed time values by retrospective think-aloud protocols. In: Kurosu M. editor. Human-Computer Interaction Applications and Services. Lecture Notes in Computer Science. 8005: Springer Berlin Heidelberg;; 2013 p. 94-100.
- 19 Hua L, Gong Y. Design of a user-centered voluntary patient safety reporting system: Understanding the time and response variances by retrospective think-aloud protocols. Stud Health Technol Inform 2013; 192: 729-733. PubMed PMID: 23920653. Epub 2013/08/08.
- 20 Hua L, Gong Y. Information gaps in reporting patient falls: the challenges and technical solutions. Stud Health Technol Inform 2013; 194: 113-118. PubMed PMID: 23941941. Epub 2013/08/15.
- 21 AHRQ.. Common Formats 2011. Available from: https://www.psoppc.org/web/patientsafety/version-1.2_documents
- 22 JQuery. Available from: http://jquery.com.
- 23 Slides JS.. Available from: http://www.slidesjs.com.
- 24 Ehlke A, Challand S, Schmidt T, Carneiro L. Tag-it. Available from: http://aehlke.github.io/tag-it.
- 25 Higginbotham DJ, Bisantz AM, Sunm M, Adams K, Yik F. The effect of context priming and task type on augmentative communication performance. Augmentative and Alternative Communication 2009; 25 (01) 19-31.
- 26 Hunnicutt S, Carlberger J. Improving word prediction using markov models and heuristic methods. Augmentative & Alternative Communication 2001; 17 (04) 255-264.
- 27 AHRQ.. Web M&M mobidity and mortality rounds on the web: Cases & commentaries [cited 2011 12/4]. Available from: http://www.webmm.ahrq.gov/home.aspx.
- 28 Matts JP, Lachin JM. Properties of permuted-block randomization in clinical trials. Controlled Clinical Trials 1988; 9 (04) 327-344.
- 29 Allan J, Englebright J. Patient-centered documentation: an effective and efficient use of clinical information systems. Journal of Nursing Administration 2000; 30 (02) 90-95.
- 30 Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: A systematic review. Journal of the American Medical Informatics Association 2005; 12 (05) 505-516.
- 31 Wachter R. Hospital Incident Reporting Systems: Time to Slay the Beast. San Francisco2009 [cited 2011 August 25th ]. Available from: http://community.the-hospitalist.org/2009/09/20/hospital-incident-reporting-systems-time-to-slay-the-monster.
- 32 James JT. A new, evidence-based estimate of patient harms associated with hospital care. Journal of patient safety 2013; 9 (03) 122-128.
- 33 Tuttle MS, Olson NE, Keck KD, Cole WG, Erlbaum MS, Sherertz DD. et al. Metaphrase: an aid to the clinical conceptualization and formalization of patient problems in healthcare enterprises. Methods of information in medicine 1998; 37 4–5 373-383.
- 34 Light J, Lindsay P, Siegel L, Parnes P. The effects of message encoding techniques on recall by literate adults using AAC systems. Augmentative & Alternative Communication 1990; 6 (03) 184-201.
- 35 Koester HH, Levine S. Effect of a word prediction feature on user performance. Augmentative & Alternative Communication 1996; 12 (03) 155-168.
- 36 Goodenough-Trepagnier C, Rosen M. Predictive assessment for communication aid prescription: Motor-determined maximum communication rate. The vocally impaired: Clinical practice and research 1988: 167-185.
- 37 Wand Y, Wang RY. Anchoring data quality dimensions in ontological foundations. Commun ACM 1996; 39 (11) 86-95.
- 38 Holden RJ, Karsh BT. A review of medical error reporting system design considerations and a proposed cross-level systems research framework. Human Factors 2007; 49 (02) 257-276. PubMed PMID: 17447667. Epub 2007/04/24.