Appl Clin Inform 2022; 13(05): 928-934
DOI: 10.1055/s-0042-1756424
Case Report

Accuracy of Physician Electronic Health Record Usage Analytics using Clinical Test Cases

Brian Lo*
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
2   Centre for Complex Interventions (Digital Interventions Unit), Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Canada
3   Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
,
Lydia Sequeira*
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
2   Centre for Complex Interventions (Digital Interventions Unit), Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Canada
3   Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
,
Gillian Strudwick
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
2   Centre for Complex Interventions (Digital Interventions Unit), Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Canada
3   Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
,
Damian Jankowicz
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
,
Khaled Almilaji
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
,
Anjchuca Karunaithas
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
4   Department of Health and Society, University of Toronto Scarborough, Scarborough, Canada
,
Dennis Hang
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
5   Health Information Science, University of Victoria, Victoria, British Columbia, Canada
,
Tania Tajirian
1   Information Management Group, Centre for Addiction and Mental Health, Toronto, Canada
6   Department of Family and Community Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
› Author Affiliations
Funding There was no formal source of funding for this work. In-kind support was provided by the Centre for Addiction and Mental Health.

Abstract

Usage log data are an important data source for characterizing the potential burden related to use of the electronic health record (EHR) system. However, the utility of this data source has been hindered by concerns related to the real-world validity and accuracy of the data. While time–motion studies have historically been used to address this concern, the restrictions caused by the pandemic have made it difficult to carry out these studies in-person. In this regard, we introduce a practical approach for conducting validation studies for usage log data in a controlled environment. By developing test runs based on clinical workflows and conducting them within a test EHR environment, it allows for both comparison of the recorded timings and retrospective investigation of any discrepancies. In this case report, we describe the utility of this approach for validating our physician EHR usage logs at a large academic teaching mental health hospital in Canada. A total of 10 test runs were conducted across 3 days to validate 8 EHR usage log metrics, finding differences between recorded measurements and the usage analytics platform ranging from 9 to 60%.

Data Availability Statement

The validation data from this project can be shared upon reasonable request to the corresponding author.


Protection of Human and Animal Subjects

The study was reviewed by CAMH Quality Project Ethics Review Board.


* These authors contributed equally.


Supplementary Material



Publication History

Received: 06 May 2022

Accepted: 25 July 2022

Article published online:
05 October 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany