Uptake of a Cervical Cancer Clinical Decision Support Tool: A Mixed-Methods Study

Abstract

Objectives Clinical decision support (CDS) tools that provide point-of-care reminders of patients' care needs may improve rates of guideline-concordant cervical cancer screening. However, uptake of such electronic health record (EHR)-based tools in primary care practices is often low. This study describes the frequency of factors associated with, and barriers and facilitators to adoption of a cervical cancer screening CDS tool (CC-tool) implemented in a network of community health centers.

Methods This mixed-methods sequential explanatory study reports on CC-tool use among 480 community-based clinics, located across 18 states. Adoption of the CC-tool was measured as any instance of tool use (i.e., entry of cervical cancer screening results or follow-up plan) and as monthly tool use rates from November 1, 2018 (tool release date) to December 31, 2020. Adjusted odds and rates of tool use were evaluated using logistic and negative-binomial regression. Feedback from nine clinic staff representing six clinics during user-centered design sessions and semi-structured interviews with eight clinic staff from two additional clinics were conducted to assess barriers and facilitators to tool adoption.

Results The CC-tool was used ≥1 time in 41% of study clinics during the analysis period. Clinics that ever used the tool and those with greater monthly tool use had, on average, more encounters, more patients from households at >138% federal poverty level, fewer pediatric encounters, higher up-to-date cervical cancer screening rates, and higher rates of abnormal cervical cancer screening results. Qualitative data indicated barriers to tool adoption, including lack of knowledge of the tool's existence, understanding of its functionalities, and training on its use.

Conclusion Without effective systems for informing users about new EHR functions, new or updated EHR tools are unlikely to be widely adopted, reducing their potential to improve health care quality and outcomes.

Keywords

clinical decision support - community health centers - cervical cancer screening - mixed methods

Background and Significance

Widespread implementation of routine Papanicolaou (Pap) testing yielded decreases in cervical cancer (CC) incidence and mortality.[1] [2] Yet despite the proven benefits of CC screening, in 2016 only two-thirds of 30 to 65-year-old women in the United States. were up-to-date on such screening.[3] Furthermore, 13% of CC deaths are attributed to inadequate follow-up on positive screening results,[4] but 47% of patients with a CC diagnosis had a >6 month interval between the test and receipt of indicated follow-up care.[4] Patients served by community health centers, many of whom experience socioeconomic barriers to acting on care plans (e.g., following up on recommended care), are more likely than those in other settings to experience such delays.[5]

The use of clinical decision support (CDS) tools in electronic health records (EHRs) might help improve rates of such follow-up care. Such tools have been shown to support clinical teams' adherence to care guidelines in some settings,[6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] including the provision of CC follow-up care.[9] [13] [18] None of these studies evaluating CDS for CC follow-up assessed tool adoption, but instead estimated or demonstrated the benefit of these tools in improving CC follow-up care.[9] [13] [18]

Despite these potential benefits, CDS adoption in primary care is suboptimal.[19] Prior research found that when users participate in the development and testing of new/updated CDS tools they are more likely to adopt the tool in practice. However, such engagement is rare. Far more common is that CDS tools are simply activated in EHRs without users receiving information about them.[20] Research is needed about how such widely disseminated tools are adopted and the barriers to their use, especially in care settings serving socioeconomically disadvantaged patient populations.[21]

One CDS tool (see [Supplementary Appendix 1], available in the online version) available in the Epic EHR (Epic Systems, Verona, Wisconsin, United States) is designed to support documenting patients' CC screening results and tracking their receipt of appropriate follow-up care, and thus to increase receipt of such care. This CC cytology SmartForm (referred to hereafter as the “CC-tool”) facilitates documenting screening results received via FAX/PDF, and ordered follow-up care subsequent to an abnormal Pap result, and panel management (generating lists of patients due for screening or follow-up care).

Table 1 Top five provider type users

Provider types	Tool uses	Providers[a]	% all tool uses	Monthly use rate[b]
Registered nurse	3,177	28	29%	6.86
Nurse practitioner	2,239	115	21%	1.32
Physician	1,954	138	18%	1.02
Physician assistant	1,326	28	12%	2.98
Medical assistant	628	22	6%	2.37

Note: Total number of tool touches in the study period = 10,896.

^a Number of providers who used the tool by type.

^b Average monthly use over the study period, calculated as the sum of tool uses by provider type divided by the sum of months providers had made use of the tool—from their first use to the end of the study period.

OCHIN, Inc., is a health information technology provider that modifies Epic to meet community health centers' needs. In October 2018, OCHIN (not an acronym) modified the CC-tool described above to reflect 2012 American Society of Colposcopy and Cervical Pathology guidelines. These guidelines provide recommendations on CC screening frequency and follow-up test for abnormal CC results. It was also modified to let users enter and update related data until follow-up care for a given CC screening result was completed, and to ensure that all added information automatically informed related “Health Maintenance” alerts. While Health Maintenance tools are standard in the Epic EHR, they are only as accurate as the data that feed their algorithms; these CC-tool changes were meant to improve the accuracy of the CC results used by the Health Maintenance tools, and are not standard in the Epic EHR. The modified version of the CC-tool was implemented in OCHIN community health centers in November 2018, and information about the tool including links to a web-based self-directed learning system for details about its use was emailed to all EHR users. In 2020, Healthy People 2030 goals for CC screening were released changing the target for CC screening prevalence from 80.5 to 84.3%. Considering that the average prevalence for CC screening in community health centers is 50%,[22] this target will be challenging to reach for these clinics. Moreover, the new American Society of Colposcopy and Cervical Pathology guidelines were released in 2020 changing the recommendations for screening prevalence and abnormal results follow-up. In response, the 2018 CC-tool was further refined both to reflect these updated guidelines and to integrate input on the tool obtained through the user-centered design process described below.

Objective

Little is known about how CDS tools are adopted in community clinics and the barriers to their use in this setting. This study describes the frequency of and factors associated with use of the 2018 CC-tool and barriers and facilitators to its use.

Methods

Setting

This study uses a sequential mixed method design. The sample included community health centers within the OCHIN practice-based research network, which share an Epic EHR hosted by OCHIN, and provide care regardless of patients' ability to pay. OCHIN EHR data were extracted from the Accelerating Data Value Across a National Community Health Center Network (ADVANCE) Clinical Research Network, a member of PCORnet, and supplemented with CC-tool use data. Analyses included 480 clinics, located across 18 states, at which ≥1 Pap test was ordered after February 1, 2020. This criterion ensured that included clinics were active after March 2020, performing cancer screenings, and able to participate in interviews and the tool refinement process.

Qualitative Analysis

Eight community health centers participated in user-centered design[23] [24] sessions and semi-structured interviews in 4 months. Community health centers recruited to ensure variation[25] in CC-tool utilization among clinic staff involved in CC screening. Participants were from community health centers in diverse geographical locations (i.e., rural, urban, suburban), and had varying care team roles (e.g., medical doctor, midwife, medical assistant, nurse practitioner, panel manager). In qualitative data collection activities in April to August 2021, we observed user-centered design sessions with nine clinic staff representing six clinics; observers took detailed field notes at these sessions. Participants with diverse experience using the CC-tool volunteered to participate in the user-centered design process, by providing feedback on how to improve the tool's support of CC screening and follow-up of abnormal results; this was one source of data on EHR users' knowledge and perceptions of this tool. In addition, we conducted semi-structured interviews with 8 staff from 2 community health centers. Qualitative data were analyzed thematically, informed by constructs from the Consolidated Framework for Implementation Research.[26] [27] [28] Interviews were audio-recorded, professionally transcribed, and uploaded with the field notes to NVivo (QSR International Pty Ltd. (2020) NVivo (Release 1.0) for analysis.

Quantitative Analysis

Adoption of the 2018 version of the CC-tool from November 1, 2018 to December 31, 2020 is described. To differentiate those who used the CC-tool rarely versus regularly, we assessed both any instance of CC-tool use as a binary variable (any vs. never used for entry of screening results or follow-up plan) and the number of tool uses per month during the study period, at the clinic level. Patient panel demographics (race/ethnicity, age, federal poverty level [FPL], rural/urban location) and insurance status were obtained from ADVANCE data. CC screening dates and abnormal Pap results were extracted from order and laboratory results and health maintenance fields in the EHR.

First, CC-tool use was summarized using descriptive statistics. Logistic regression was then used to model the binary outcome of any CC-tool use (use vs. never) and negative binomial regression to model the outcome of monthly tool use rates. Each model considered two sets of covariates: (1) models with practice-level characteristics only and (2) models with practice-level characteristics plus rates of CC screening and rates of abnormal Pap results as covariates.

Results

During the study period, 41% of eligible clinics used the CC-tool at least once. It was used most frequently by registered nurses, nurse practitioners, and medical doctors ([Table 1]). There were few differences in the characteristics of clinics where the CC-tool was ever versus never used ([Table 2]). Notably, 47% of clinics with CC-tool use were in the top tertile of CC screening rates versus 22% of clinics where the tool was never used; 77% of clinics with any tool use were in the highest and middle tertiles for abnormal Pap rates compared to 59% among nonusers. Both logistic and negative binomial models showed that clinics where the CC-tool was used more often had significantly more encounters, a greater proportion of patients of with FPL ≥138%, fewer pediatric patients, and higher rates of up-to-date CC screening and of abnormal results ([Table 3]).

Table 2 Clinic characteristics of cervical cancer screening tool users and nonusers

	User (n = 214)	Nonuser (n = 266)
Panel mix
Ambulatory encounters, mean (SD)	32,131 (37,895)	11,065 (14,754)
Percent White, mean (SD)	62.1 (26.9)	61.3 (26.5)
Percent pediatric patient, mean (SD)	12.6 (12.9)	18.0 (26.6)
Percent Hispanic, mean (SD)	31.4 (26.8)	25.0 (25.6)
Percent Medicaid, mean (SD)	51.9 (19.0)	50.2 (20.8)
Percent uninsured, mean (SD)	14.2 (14.0)	17.1 (15.6)
Percent FPL <138%, mean (SD)	58.7 (27.0)	63.2 (27.8)
Percent rural, mean (SD)	7.0 (3.3)	19.0 (7.1)
Implementation impact
Rates of CC screening, n (%)
Highest tertile (>57.3%)	100 (46.7)	60 (22.6)
Middle tertile (39.7–57.3%)	81 (37.9)	79 (29.7)
Lowest tertile (<39.7%)	33 (15.4)	127 (47.7)
Rates of abnormal Pap results, n (%)
Highest tertile (>17.8%)	79 (36.9)	81 (30.5)
Middle tertile (12.6–17.8%)	85 (39.7)	75 (28.2)
Lowest tertile (<12.6)	50 (23.4)	110 (41.4)

Abbreviations: CC, cervical cancer screening; FPL, federal poverty level; SD, standard deviation.

Note: Users are clinics with clinic staff members who used the CC-tool at least once in the study period.

Table 3 Clinic characteristics associated with the odds and rates of the cervical cancer screening tool use (95% confidence intervals)

	Relative odds of tool use		Relative rates of tool use
	Model A	Model B	Model A	Model B
Panel mix
Ambulatory encounters	1.042 (1.030–1.056)	1.037 (1.025–1.050)	1.053 (1.047–1.060)	1.050 (1.041–1.059)
Percent White	0.993 (0.984–1.001)	0.993 (0.984–1.002)	0.991 (0.979–1.002)	0.988 (0.977–1.000)
Percent pediatric patient	0.985 (0.972–0.996)	0.990 (0.975–1.003)	0.959 (0.950–0.967)	0.968 (0.953–0.984)
Percent Hispanic	1.011 (1.002–1.020)	1.005 (0.995–1.015)	1.011 (0.998–1.024)	1.004 (0.990–1.018)
Percent Medicaid	1.001 (0.987–1.017)	1.005 (0.989–1.021)	1.013 (0.994–1.032)	1.020 (0.997–1.042)
Percent uninsured	0.993 (0.976–1.011)	1.002 (0.983–1.020)	0.974 (0.949–1.001)	0.997 (0.973–1.022)
Percent FPL <138%	0.991 (0.982–0.999)	0.988 (0.979–0.997)	0.990 (0.981–0.999)	0.977 (0.963–0.991)
Percent rural	0.893 (0.219–3.142)	0.753 (0.181–2.724)	0.351 (0.053–2.325)	0.471 (0.112–1.989)
Implementation impact
Rates of CC screening
Highest tertile (>57.3%)	–	Reference	–	Reference
Middle tertile (39.7–57.3%)	–	0.723 (0.438–1.192)	–	0.357 (0.194–0.657)
Lowest tertile (<39.7%)	–	0.249 (0.138–0.441)	–	0.042 (0.017–0.108)
Rates of abnormal Pap results
Highest tertile (>17.8%)	–	Reference	–	Reference
Middle tertile (12.6–17.8%)	–	0.692 (0.408–1.165)	–	0.378 (0.205–0.695)
Lowest tertile (<12.6)	–	0.566 (0.327–0.972)	–	0.955 (0.470–1.941)

Abbreviations: CC, cervical cancer screening; FPL, federal poverty level.

Note: Model A: model with panel mix indicators. Model B: model with panel mix indicators and implementation impact variables. 95% confidence intervals are denoted inside parentheses. Bold denotes statistical significance (p < 0.05). Users are clinics with clinic staff members who used the CC-tool at least once in the study period.

Two themes emerged from the qualitative results: lack of awareness of the tool's existence and lack of knowledge about how to use it in an optimal manner.

Many clinical team members involved in the qualitative data collection processes were not aware of the CC-tool. One clinician who uses the CC-tool shared, “I can tell you the majority of my colleagues don't know that exists, let alone do they know they have to use it.”

Those who did know about the tool learned about it by accident or from a colleague.

Few participants who used the CC-tool felt they knew how to use it effectively. Many noted they received no formal training on its use; as one midwife describes, “I would say that right now for our clinic, our training is user error. You just get in, and you figure it out.” Participants reported lacking a standardized system for tracking abnormal Pap results, indicating a gap in understanding the CC-tool's purpose. One clinician described the impact on patient care from such gaps, “We had two patients this year that I unfortunately had to call about abnormal Pap Smear results done over a year ago…and that's worrisome.” Although many CC-tool users indicated that using it was easy once they knew how to access it, several noted difficulties in re-accessing the tool once they closed it. A clinician shared, “I like how the [CC-tool] is right now because it's four clicks and then done. The only thing I hate is that I can never find it again if I forget to put it in before I review the lab.” Others indicated accidentally not completing all fields in the entry form because it was so long that seeing all fields required scrolling down, which added time burden. Some noted that if the results from the last Pap were not entered correctly in the cytology form, the preventive care section of Health Maintenance defaults to a standard, and at times erroneous, date for the next recommended CC screening. One nurse practitioner said, “For us the biggest issue[s] are the modifiers because we're co-testing, but [the Health Maintenance tools] default to 3 years instead of 5 years, and then you've got to go in and change it. Our QI team does all the outreach and patients are being called in for their Pap smears when they weren't due.” Several participants indicated this lack of trust in Health Maintenance required them to conduct a time-intensive clinical review to ensure appropriate screening interval and modality.

Discussion

These results show that a CDS tool targeting CC prevention was used by fewer than half of clinics with access to it. The CC-tool use was higher in larger clinics and those with higher CC screening and abnormal CC screening result rates, suggesting the tool's perceived utility by these users. These results add evidence on CDS adoption in the community health center setting. Prior research in other settings showed that user-centered design participants are more likely to adopt tools; future analyses will not assess those longitudinal outcomes in this setting.

Yet while CC-tool users generally liked it, many were not aware of it. This is concerning but not unusual, as EHRs are regularly updated and revised to provide new tools and functionalities meant to assist providers in care delivery and panel management. Such updates are necessary to ensure that EHRs' CDS functionalities follow current care guidelines. Yet within a complex EHR system, improvement-focused updates can be easily missed, particularly when multiple modifications are made at once; one study estimated that an average of 2.5 EHR updates per day are implemented in a large integrated care system.[29]

In our study setting, EHR changes are communicated to member community health centers in a scheduled monthly cycle and include guidance on what the change is, rationale for change, and potential workflow impacts. These results suggest that this information may not be disseminated effectively to all potential users. Participants lacked knowledge on how to use the CC-tool effectively; this might be addressed through alternate means of disseminating instructions on new tool use, such as targeted training (e.g., live or on-demand demonstrations, printed step-by-step instructions). Research is needed on best practices for informing users about new EHR functions, or such updates may not be used, or only by those who “just get in, and … figure it out.” This is relevant to diverse CDS tools across all care settings and EHR platforms.

Conclusion

For CDS tools to improve health outcomes, users need to know where such tools are located in the EHR, what they can do, and how they should be used. Thus, while continuous EHR updates are needed, they are unlikely to substantially improve care quality without the concurrent provision of effective communication mechanisms.

Clinical Relevance Statement

Improving communication of new or updated clinical decision support tools is critical for such tools to be used in clinical care.

Multiple-Choice Questions

1. What barrier(s) impact(s) adopting a clinical decision tool?

   • Tool is difficult to locate

   • Lack of knowledge on how to use it

   • Not knowing it exists

   • All of the above

   Correct Answer: The correct answer is option d. All barriers listed above impact the adoption of new or revised electronic health record tools.

2. What factors were associated with higher adoption of the CC-tool by clinic staff?

   • Clinic location

   • Lower rate of abnormal Pap

   • Higher rate of abnormal Pap

   • None of the above

   Correct Answer: The correct answer is option c. Clinics that had higher rates of abnormal Pap also had greater tool adoption rates, which reinforces the benefit of clinical decision support for cervical cancer screenings.

Conflict of Interest

None declared.

Acknowledgements

This work was conducted with the Accelerating Data Value Across a National Community Health Center Network (ADVANCE) Clinical Research Network. OCHIN leads the ADVANCE network in partnership with Health Choice Network, Fenway Health, Oregon Health & Science University, and the Robert Graham Center HealthLandscape. ADVANCE is funded through the Patient-Centered Outcomes Research Institute (PCORI), contract number RI-OCHIN-01-MC.

Protection of Human and Animal Subjects

The Institutional Review Board reviewed and approved this study.

• References

• 1 Benard VB, Thomas CC, King J, Massetti GM, Doria-Rose VP, Saraiya M. Centers for Disease Control and Prevention (CDC). Vital signs: cervical cancer incidence, mortality, and screening - United States, 2007-2012. MMWR Morb Mortal Wkly Rep 2014; 63 (44) 1004-1009
• 2 Moyer VA, Force USPST. U.S. Preventive Services Task Force. Screening for cervical cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2012; 156 (12) 880-891 , W312
• 3 MacLaughlin KL, Jacobson RM, Radecki Breitkopf C. et al. Trends over time in pap and Pap-HPV cotesting for cervical cancer screening. J Womens Health (Larchmt) 2019; 28 (02) 244-249
• 4 Chido-Amajuoyi OG, Shete S. Prevalence of abnormal cervical cancer screening outcomes among screening-compliant women in the United States. Am J Obstet Gynecol 2019; 221 (01) 75-77
• 5 Tsui J, Llanos AA, Doose M, Rotter D, Stroup A. Determinants of abnormal cervical cancer screening follow-up and invasive cervical cancer among uninsured and underinsured women in New Jersey. J Health Care Poor Underserved 2019; 30 (02) 680-701
• 6 Bright TJ, Wong A, Dhurjati R. et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
• 7 Lobach DF. The road to effective clinical decision support: are we there yet?. BMJ 2013; 346: f1616
• 8 Magrath M, Yang E, Ahn C. et al. Impact of a clinical decision support system on guideline adherence of surveillance recommendations for colonoscopy after polypectomy. J Natl Compr Canc Netw 2018; 16 (11) 1321-1328
• 9 Mahmoud AS, Alkhenizan A, Shafiq M, Alsoghayer S. The impact of the implementation of a clinical decision support system on the quality of healthcare services in a primary care setting. J Family Med Prim Care 2020; 9 (12) 6078-6084
• 10 Ravikumar KE, MacLaughlin KL, Scheitel MR. et al. Improving the accuracy of a clinical decision support system for cervical cancer screening and surveillance. Appl Clin Inform 2018; 9 (01) 62-71
• 11 Sequist TD, Zaslavsky AM, Marshall R, Fletcher RH, Ayanian JZ. Patient and physician reminders to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med 2009; 169 (04) 364-371
• 12 Powell BJ, Waltz TJ, Chinman MJ. et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015; 10: 21
• 13 Dupuis EA, White HF, Newman D, Sobieraj JE, Gokhale M, Freund KM. Tracking abnormal cervical cancer screening: evaluation of an EMR-based intervention. J Gen Intern Med 2010; 25 (06) 575-580
• 14 Osheroff JA, Teich JA, Levick D. et al. Improving Outcomes with Clinical Decision Support: An Implementer's Guide. 2nd ed. Chicago IL: HIMSS; 2012
• 15 Militello LG, Diiulio JB, Borders MR. et al. Evaluating a modular decision support application for colorectal cancer screening. Appl Clin Inform 2017; 8 (01) 162-179
• 16 Becker M, Böckmann B, Jöckel KH. et al. Mapping patient data to colorectal cancer clinical algorithms for personalized guideline-based treatment. Appl Clin Inform 2020; 11 (02) 200-209
• 17 Tamposis I, Tsougos I, Karatzas A, Vassiou K, Vlychou M, Tzortzis V. PCaGuard: a software platform to support optimal management of prostate cancer. Appl Clin Inform 2022; 13 (01) 91-99
• 18 MacLaughlin KL, Kessler ME, Komandur Elayavilli R. et al. Impact of patient reminders on papanicolaou test completion for high-risk patients identified by a clinical decision support system. J Womens Health (Larchmt) 2018; 27 (05) 569-574
• 19 Kouri A, Yamada J, Lam Shin Cheung J, Van de Velde S, Gupta S. Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake. Implement Sci 2022; 17 (01) 21
• 20 Gold R, Bunce A, Davis JV. et al. “I didn't know you could do that”: a pilot assessment of EHR optimization training. ACI Open 2021; 5 (01) e27-e35
• 21 Owens C, Chen J, Xu R. et al. Implementation of Health Information Technology for Secondary Cancer Prevention in Primary Care: A Scoping Review. 2022
• 22 Huguet N, Hodes T, Holderness H, Bailey SR, DeVoe JE, Marino M. Community health centers' performance in cancer screening and prevention. Am J Prev Med 2022; 62 (02) e97-e106
• 23 Norman D. . The Design of Everyday Things. New York, NY: Basic Books; 1988
• 24 Rex Hartson H. Human–computer interaction: interdisciplinary roots and trends. J Syst Softw 1998; 43 (02) 103-118
• 25 Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res 2016; 26 (13) 1753-1760
• 26 Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4: 50
• 27 Braun V, Clarke V, Hayfield N, Thematic Analysis GT. . Thematic analysis. In: Liamputtong P, ed. Handbook of Research Methods in Health Social Sciences. Singapore: Springer; 2019:843–860
• 28 Cohen D, Crabtree BF, Damschroder L. et al. Qualitative methods in implementation science. 2015. Accessed May 24, 2023 at: https://cancercontrol.cancer.gov/sites/default/files/2020-09/nci-dccps-implementationscience-whitepaper.pdf
• 29 Liu VX, Haq N, Chan IC, Hoberman B. Inpatient electronic health record maintenance from 2010 to 2015. Am J Manag Care 2019; 25 (01) 18-21

Publication History

Received: 22 January 2023

Accepted: 26 April 2023

Article published online:
02 August 2023