A Randomized Trial Comparing Classical Participatory Design to VandAID, an Interactive CrowdSourcing Platform to Facilitate User-centered Design

Kevin R. Dufendach; Sabine Koch; Kim M. Unertl; Christoph U. Lehmann

doi:10.3414/ME16-01-0098

Methods of Information in Medicine, Table of Contents

Methods Inf Med 2017; 56(05): 344-349
DOI: 10.3414/ME16-01-0098

Paper

A Randomized Trial Comparing Classical Participatory Design to VandAID, an Interactive CrowdSourcing Platform to Facilitate User-centered Design

Authors

Kevin R. Dufendach

¹Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA
Sabine Koch

²Health Informatics Centre, Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden
Kim M. Unertl

³Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA
Christoph U. Lehmann

³Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA

Abstract

Summary

Background: Early involvement of stakeholders in the design of medical software is particularly important due to the need to incorporate complex knowledge and actions associated with clinical work. Standard user-centered design methods include focus groups and participatory design sessions with individual stakeholders, which generally limit user involvement to a small number of individuals due to the significant time investments from designers and end users.

Objectives: The goal of this project was to reduce the effort for end users to participate in co-design of a software user interface by developing an interactive web-based crowd- sourcing platform.

Methods: In a randomized trial, we compared a new web-based crowdsourcing platform to standard participatory design sessions. We developed an interactive, modular platform that allows responsive remote customization and design feedback on a visual user interface based on user preferences. The responsive canvas is a dynamic HTML template that responds in real time to user preference selections. Upon completion, the design team can view the user’s interface creations through an administrator portal and download the structured selections through a REDCap interface.

Results: We have created a software platform that allows users to customize a user interface and see the results of that customization in real time, receiving immediate feedback on the impact of their design choices. Neonatal clinicians used the new platform to successfully design and customize a neonatal handoff tool. They received no specific instruction and yet were able to use the software easily and reported high usability.

Conclusions: VandAID, a new web-based crowdsourcing platform, can involve multiple users in user-centered design simultaneously and provides means of obtaining design feedback remotely. The software can provide design feedback at any stage in the design process, but it will be of greatest utility for specifying user requirements and evaluating iterative designs with multiple options.

Keywords

Computers - informatics - human factors and engineering - human-computer interface

Full Text

References

References
1 Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff Proj Hope 2010; 29 (04) 655-663.
2 Weiner JP, Kfuri T, Chan K, Fowles JB. “e-Iatrogen- esis”: the most critical unintended consequence of CPOE and other HIT. JAMIA 2007; 14 (03) 387-388 discussion 389.
3 Graham J, Dizikes C. Baby’s death spotlights safety risks linked to computerized systems [cited 2016 Apr 15]. Los Angeles Times; 2011 Available from: http://www.latimes.com/ct-met-technology-errors-20110627-story.html.
4 Russ AL, Zillich AJ, Melton BL, Russell SA, Chen S, Spina JR, Weiner M, Johnson EG, Daggy JK, McManus MS, Hawsey JM, Puleo AG, Doebbeling BN, Saleem JJ. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. JAMIA 2014; 21 e2 e287-296.
5 Survey of Clinicians: User satisfaction with electronic health records has decreased since 2010 [cited 2016 Apr 15]. 2013. Available from: https://www.acponline.org/acp-newsroom/survey-of-clinicians-user-satisfaction-with-electronic-health-records-has-decreased-since-2010?hp=.
6 Lehmann CU, Unertl KM, Rioth MJ, Lorenzi NM. Change Management for the Successful Adoption of Clinical Information Systems. Finnell JT, Dixon BE. Clinical Informatics Study Guide.. Springer International Publishing; 2016: 435-456.
7 Scandurra I, Hägglund M, Koch S. From user needs to system specifications: multi-disciplinary thematic seminars as a collaborative design method for development of health information systems. J Biomed Inform 2008; 41 (04) 557-569.
8 McCoy AB, Waitman LR, Gadd CS, Danciu I, Smith JP, Lewis JB, Schildcrout JS, Peterson JF. A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am J Kidney Dis Off J Natl Kidney Found 2010; 56 (05) 832-841.
9 HealthcareUsability. ONC Meaningful Use and Usability Testing [cited 2016 Jun 15]. 2013. Available from: www.healthcareusability.com/article/onc-meaningful-use-and-usability-testing.
10 Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. JAMIA 2015; 22 (06) 1179-1182.
11 Ratwani RM, Benda NC, Hettinger AZ, Fairbanks RJ. Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. JAMA 2015; 314 (10) 1070-1071.
12 Beyer HR, Holtzblatt K. Apprenticing with the Customer. Commun ACM 1995; 38 (05) 45-52.
13 Andre K, Christian N. Participatory Design, User Involvement and Health IT Evaluation. Stud Health Technol Inform 2016; 139-151
14 Angular JS – Superheroic JavaScript MVW Framework [cited 2016 Jun 15]. Available from: https://angularjs.org/.
15 Something New and Powerful: SMART on FHIR®. SMART Health IT. 2014 [cited 2016 Jun 15]. Available from: http://smarthealthit.org/smart-on-fhir/.
16 Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. JAMIA 2016; 23 (05) 899-908.
17 Summary – FHIR v1.0.2. 2015 [cited 2017 Jan 5]. Available from: http://hl7.org/fhir/summary.html.
18 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) – a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381.
19 R: The R Project for Statistical Computing. [cited 2016 Jul 19]. Available from: https://www.r-project.org/.
20 Assistant Secretary for Public Affairs. System Usability Scale (SUS) [cited 2016 Jun 15]. 2013 Available from: http://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html.
21 Sauro J. A practical guide to the system usability scale: background, benchmarks & best practices. Denver: Measuring Usability LLC; 2011