Appl Clin Inform 2022; 13(02): 485-494
DOI: 10.1055/s-0042-1748145
Research Article

Generating and Reporting Electronic Clinical Quality Measures from Electronic Health Records: Strategies from EvidenceNOW Cooperatives

Joshua E. Richardson
1   Center for Health Informatics and Evidence Synthesis, RTI International, Chicago, Illinois, United States
Luke V. Rasmussen
2   Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
David A. Dorr
3   Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, United States
Jenna T. Sirkin
4   NORC at the University of Chicago, Cambridge, Massachusetts, United States
Donna Shelley
5   Department of Public Health Policy and Management, New York University School of Global Public Health, New York, New York, United States
Adovich Rivera
6   Institute of Public Health and Management, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
Winfred Wu
7   Bureau of Primary Care Information Project, New York City Department of Health and Mental Hygiene, New York, New York, United States
Samuel Cykert
8   Division of General Medicine and Clinical Epidemiology and the Cecil G. Sheps Center for Health Services Research, the University of North Carolina, Chapel Hill, North Carolina, United States
Deborah J. Cohen
9   Department of Family Medicine, Oregon Health & Science University, Portland, Oregon, United States
Abel N. Kho
10   Center for Health Information Partnerships (CHiP), Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
› Author Affiliations
Funding This research was supported by the Agency for Healthcare Research and Quality (AHRQ; grant number: R18HS023921). The contents of this product are solely the responsibility of the authors and do not necessarily represent the official views of or imply endorsement by AHRQ or the U.S. Department of Health and Human Services.


Background Electronic clinical quality measures (eCQMs) from electronic health records (EHRs) are a key component of quality improvement (QI) initiatives in small-to-medium size primary care practices, but using eCQMs for QI can be challenging. Organizational strategies are needed to effectively operationalize eCQMs for QI in these practice settings.

Objective This study aimed to characterize strategies that seven regional cooperatives participating in the EvidenceNOW initiative developed to generate and report EHR-based eCQMs for QI in small-to-medium size practices.

Methods A qualitative study comprised of 17 interviews with representatives from all seven EvidenceNOW cooperatives was conducted. Interviewees included administrators were with both strategic and cooperative-level operational responsibilities and external practice facilitators were with hands-on experience helping practices use EHRs and eCQMs. A subteam conducted 1-hour semistructured telephone interviews with administrators and practice facilitators, then analyzed interview transcripts using immersion crystallization. The analysis and a conceptual model were vetted and approved by the larger group of coauthors.

Results Cooperative strategies consisted of efforts in four key domains. First, cooperative adaptation shaped overall strategies for calculating eCQMs whether using EHRs, a centralized source, or a “hybrid strategy” of the two. Second, the eCQM generation described how EHR data were extracted, validated, and reported for calculating eCQMs. Third, practice facilitation characterized how facilitators with backgrounds in health information technology (IT) delivered services and solutions for data capture and quality and practice support. Fourth, performance reporting strategies and tools informed QI efforts and how cooperatives could alter their approaches to eCQMs.

Conclusion Cooperatives ultimately generated and reported eCQMs using hybrid strategies because they determined neither EHRs alone nor centralized sources alone could operationalize eCQMs for QI. This required cooperatives to devise solutions and utilize resources that often are unavailable to typical small-to-medium-sized practices. The experiences from EvidenceNOW cooperatives provide insights into how organizations can plan for challenges and operationalize EHR-based eCQMs.

Protection of Human and Animal Subjects

The Northwestern University Institutional Review Board approved this study.

Supplementary Material

Publication History

Publication Date:
04 May 2022 (online)

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany