Usability Evaluation of a Distributed User Interface Application for Visuomotor Organization AssessmentFunding This research has been partially supported by the Ministry of Science, Innovation and Universities (Spain) under grant RTI2018–099942-B-I00. This work has been supported by the fellowship 2019/451 from the University of Castilla-La Mancha.
Background This article describes the development and evaluation of a distributed user interface (DUI) application to assess visuomotor organization ability. This application enables therapists to evaluate the acquired brain injury (ABI) on patients, and patients, to perform the assessment on a touch screen while therapists can observe the assessment process in real time on a separated monitor without interfering patients during the process as in traditional methodologies employing physical elements.
Objectives The main goal of this research is the evaluation of the quality in use of DUIs in the Pegboard Construction assessment with patients with ABI from the therapist perspective in the area of occupational therapy.
Methods To evaluate our system, we have performed a usability evaluation following the ISO/IEC 25010 and ISO/IEC 25062 standards to evaluate software usability and quality and it was conducted in collaboration with therapists and psychologists that have previously worked with people with ABI in diagnostic and assessment tasks.
Results We show the results of the evaluation collected in a table that shows the completeness rate for each user for both, assisted (i.e., the percentage of tasks where participants performed with test director assistance) and unassisted tasks (i.e., the percentage of tasks where participants completed tasks autonomously), the total time participants required to complete proposed tasks, the number of mistakes participants performed during the session, and the number of assists they required to finish proposed tasks. In addition, we also evaluated the user satisfaction regarding our application using the system usability scale.
Conclusion The use of information technologies in this field enables therapists to perform these evaluations in a simpler, efficient, and automated way. This proposal enables patients to perform the assessment as it is performed traditionally using paper providing them with a touch screen in which they can easily insert a set of pins into the holes. The usability evaluation of the proposal meets the appropriate design standards for applications of this type, and this is demonstrated by the high degree of satisfaction of the participants.
Keywordscognition - assessment - occupational therapy - acquired brain injury - distributed user interface
Received: 05 November 2019
Accepted: 02 March 2020
07 September 2020 (online)
Georg Thieme Verlag KG
Stuttgart · New York
- 1 Navarra ACIA. Available at: https://www.adacen.org/blog/index.php/que-esel-dano-cerebral-y-cuales-son-sus-secuelas/ . Accessed November 9, 2018
- 2 Cerebral Injury HHRM. Available at: http://xn–daocerebral-2db.es/papelde-la-terapia-ocupacional-en-la-rehabilitacion-del-dano-cerebral/ . Accessed November 12, 2018
- 3 Katz N, Itzkovich M, Averbuch S, Elazar B. Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) battery for brain-injured patients: reliability and validity. Am J Occup Ther 1989; 43 (03) 184-192
- 4 McDermott A. Purpose of measure. In: Nicol Korner-Bitensky, ed. Loewenstein Occupational Theraphy Cognitive Assessment. Ottawa: Heart and Stroke Foundation; 2011
- 5 Annes GCS. Comparison of younger and older healthy American adults on the Loewenstein Occupational Therapy Cognitive Assessment. Occup Ther Int 1996; 3: 157-173
- 6 Zwecker M, Levenkrohn S, Fleisig Y, Zeilig G, Ohry A, Adunsky A. Mini-mental state examination, cognitive FIM instrument, and the Loewenstein Occupational Therapy Cognitive Assessment: relation to functional outcome of stroke patients. Arch Phys Med Rehabil 2002; 83 (03) 342-345
- 7 Jang Y, Chern JS, Lin KC. Validity of the Loewenstein occupational therapy cognitive assessment in people with intellectual disabilities. Am J Occup Ther 2009; 63 (04) 414-422
- 8 Josman N, Katz N. Relationships of categorization on tests and daily tasks in patients with schizophrenia, post-stroke patients and healthy controls. Psychiatry Res 2006; 141 (01) 15-28
- 9 Josman N, Abdallah TM, Engel-Yeger B. Cultural factors affecting the differential performance of Israeli and Palestinian children on the Loewenstein Occupational Therapy Cognitive Assessment. Res Dev Disabil 2010; 31 (03) 656-663
- 10 Cermak SA, Katz N, McGuire E, Greenbaum S, Peralta C, Maser-Flanagan V. Performance of Americans and Israelis with cerebrovascular accident on the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA). Am J Occup Ther 1995; 49 (06) 500-506
- 11 Katz N, Champagne D, Cermak S. Comparison of the performance of younger and older adults on three versions of a puzzle reproduction task. Am J Occup Ther 1997; 51 (07) 562-568
- 12 Melchior J, Vanderdonckt J, Van Roy P. A model-based approach for distributed user interfaces. Paper presented at: Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems. Pisa, Italy; 2011: 11-20
- 13 Penãlver A, L'opez-Esp'ın JJ, Gallud JA, Lazcorreta E, Botella F. Distributed User Interfaces: Specification of Essential Properties in Distributed User Interfaces. Designing Interfaces for the Distributed Ecosystem. London: Springer; 2011: 13-21
- 14 Tesoriero R, Altalhi AH. Model-based development of distributable user interfaces. Univ Access Inf Soc 2019; 18: 719-746
- 15 Villanueva PG, Tesoriero R, Gallud JA. Revisiting the concept of distributed user interfaces. 2013: 1-15 . DOI: 10.1007/978-1-4471-5499-0_1
- 16 Elmqvist N. Distributed user interfaces: state of the art. Paper presented at: Workshop on Distributed User Interfaces 2011 (DUI) at the 29th ACM CHI Conference on Human Factors in Computing Systems; May 7, 2011; Vancouver
- 17 Vandervelpen C, Vanderhulst K, Coninx K. Light-weight distributed web interfaces: preparing the web for heterogeneous environments. Paper presented at: Proceeding of 5th Int. Conf. on Web Engineering ICWE'2005. Sydney, Australia; 2005
- 18 Grolaux D, Roy PV, Vanderdonckt J. Migratable user interfaces: beyond migratory interfaces. Paper presented at: Proceedings of the IEEE/ACM Conference on Mobile and Ubiquitous Systems; August, 2004
- 19 Bandelloni F. Flexible interface migration. Paper presented at: the ACM Conference on Intelligent User Interfaces; January, 2004. Boston, Massachusetts, United States; Funchal
- 20 Demeure J, Sottet S, Calvary G, Coutaz V, Vanderdonckt J. The 4C reference model for distributed user interfaces. Paper presented at: Proceedings of the International Conference on Autonomic and Autonomous Systems; 2008; NW Washington, DC
- 21 Melchior J, Grolaux D, Vanderdonckt J, Roy PV. A toolkit for peer-to-peer distributed user interfaces: concepts, implementation, and applications. Paper presented at: Proceedings of the ACM Symposium on Engineering Interactive Computing System; July, 2009; Pittsburgh, PA
- 22 Penichet VMR, Lozano MD, Gallud JA, Tesoriero R. 12+1 Questions in the Design of Distributed User Interfaces. Paper presented at: Proceedings of the 2014 Workshop on Distributed User Interfaces and Multimodal Interaction; July 1, 2014; Toulouse
- 23 Gallud JA, Penichet VMR. Distributed user interfaces: distributing interactions to facilitate universal access. Univ Access Inf Soc 2019; 18 (04) 717-718
- 24 Marco FA, Gallud JA, Penichet VMR. A study on how users perceive distributed interactions on web applications. IEEE Access 2018; 6: 57389-57399
- 25 Mayhew DJ. The Usability Engineering Lifecycle: A Practitioner's Handbook for User Interface Design. San Francisco; CA: Morgan Kaufmann Publishers; 1999
- 26 Collazos CA, Gil R. Using cross-cultural features in web design patterns. Paper presented at: Proceedings of the 2011 Eighth International Conference on Information Technology: New Generations (USA) IEEE Computer Society. Las Vegas, Nevada, United States; 2011
- 27 Llu'cia Masip RG, Granollers T, Collazos CA. Multiculturality and internationalization in Web interfaces. Rev Adv Sistemas Inform 2009; 6: 192-196
- 28 ISO/IEC 25010. 2011 Systems and software engineering—Systems and software Quality Requirements and Evaluation (SQuaRE) —systems and software quality models; 2013
- 29 Software Engineering—Software Product-Quality Requirements and Evaluation (SQuarE)—Common Industry Format (CIF) for Usability Test Reports. International Organization for Standardization; 2006
- 30 Brooke J. SUS: a “quick and dirty” usability scale. In: Usability Evaluation in Industry. CRC Press; 1996: 189-194