A Provider-Facing Decision Support Tool for Prostate Cancer Screening in Primary Care: A Pilot Study

Keywords

prostate cancer screening - clinical decision support - electronic health record - pilot study - informatics - concordant care

Background and Significance

There is now a wealth of evidence regarding who, when, and how often to screen for prostate cancer to retain the benefits while reducing harm.[1] [2] [3] [4] These have been developed into guidelines. However, these guidelines are frequently not followed. This results in a burden of overdiagnosis and overtreatment which largely falls on older men, when screening continues at high rates despite clear evidence of little benefit.[5] [6] [7] Conversely, screening rates are low in men most likely to benefit: a healthy 55-year old man is no more likely to receive a prostate-specific antigen (PSA) test than a 75-year old with multiple comorbidities.[5] This is particularly a problem for high-risk populations, including men of African ancestry.[6]

PSA screening is implemented in the primary care setting. It is therefore critical to evaluate how primary care physicians (PCPs) use PSA to understand the current gap between guidelines and clinical practice. First, PSA screening is complex, and it is understandable that as generalists, PCPs admit to lacking knowledge about this specialist topic.[8] Second, contemporary guidelines on PSA shared decision-making (SDM)[9] [10] are often clinically impractical. For instance, one recommendation is that PCPs inform patients on 16 separate points and ask 12 questions about preferences; a second includes data that might be hard for patients to understand or assign a value to, such as the risk for deep venous thrombosis.[11] The lack of an appropriate, feasible approach may be why PCPs do not engage their patients in SDM about PSA screening.[11] [12] [13] Thus, there is a need for an effective intervention to assist PCPs in adhering to guidelines through leveraging innovative information technology combined with principles from behavioral sciences. Until this need is met, ineffective practice will continue to obscure the benefit of PSA screening.

The premise of this work is the assumption that dissemination of a rational approach to screening, utilizing the electronic health record (EHR), will be an effective way of influencing physician behavior. Clinical decision support tools, developed with clinician input, have been shown to improve adherence to guideline-concordant care in, for example, cervical cancer screening and inflammatory bowel disease.[14] [15] Hard-coding guidelines into the EHR could also reduce downstream variations in care based on factors such as race. In particular, we believe that any such tool has to be specifically designed to meet the needs of PCPs with regard to time requirements and specialist knowledge required. No such tool for risk-stratified PSA screening exists.

Objectives

We initiated a line of research with the long-term goal of improving guideline-concordant practice, for example, physician compliance, as carried out in two phases. In the development phase, reported in detail separately (see companion paper Doi: 10.1055/s-0044-1782619), we first conducted a focus group with 10 PCPs and used their feedback to develop a clinical decision support tool for PSA screening, which included normative narratives and principles from behavioral economics. Next, we programmed the tool into the EHR (Epic, Verona, WI, United States) and tested it in a 6-month pilot study in clinical practice, reported herein.

Methods

The study was approved by the Institutional Review Boards at the Brigham and Women's Hospital (BWH) Primary Care and Memorial Sloan Kettering Cancer Center.

PCPs in the study were recruited from a large primary care network consisting of 15 practices, including two community health centers, and comprising both academic PCPs and community practitioners (approximately 170 clinicians). The primary care practices serve a broad cross-section of the greater community in the geographic area and provide care for approximately 150,000 patients.

Description of the Intervention

The electronic clinical decision support tool consisted of four components: (1) a hyperlink to the National Comprehensive Cancer Network (NCCN) guideline for early detection of prostate cancer with an image of the risk-stratified algorithm for PSA screening (e.g., recommendations regarding age to start, age to stop, and frequency of rescreening)[4]; (2) a provider-facing tool specifically developed for facilitating SDM conversations in the primary care setting (Simple Schema, developed by members of the study team and described in detail elsewhere,[11] an evidence-based tool focusing on 10 facts, designed to facilitate a discrete choice, and does not require more than a few minutes to implement); (3) three best practice advisories (BPAs) for patients eligible (ages 45–75 years), patients too young (<45 years), and patients too old (>75 years) for PSA screening; and (4) a health maintenance module, which allows the physician to set automated reminders for rescreening based on the patient's prior PSA value and age. Narratives were developed based on behavioral economics principles, a novel aspect of this work, (e.g., setting the guideline-recommended practice as the default, leveraging social proof by referencing what other PCPs would do; shifting the reference point by focusing on health promotion efforts for older men; and having PCPs document override reasons in free text, accountable justification).[16] [17] [18] [19] After using the tool, PCPs could document in the EHR that SDM had taken place.

Prior Focus Group Study

We first established relationships with the health professionals in the study team and the study participants: PCPs, urologists, cancer screening and behavioral economics experts, and programmers in the Epic development team. Next, we conducted a focus group study with the physicians, gauging their interest in and attitudes toward PSA screening and decision support tools using qualitative methods (focus group) and quantitative (survey) methods to develop and test the validity of the intervention. The results of the focus group study are reported in detail separately (Doi: 10.1055/s-0044-1782619). In brief, in our focus group study, we found strong support and buy-in for this tool from the physicians, who were very satisfied with it. Feedback was overwhelmingly positive regarding the need for a provider-facing decision support tool to assist with PSA screening decisions in the primary care setting. The opportunity to use the tool to document in the EHR that SDM has taken place was a particularly desirable feature. The possibility to automatically reschedule the next PSA test using the tool was also liked by the physicians. The encouraging results from the focus group therefore led to the initiation of this pilot study.

Incorporating Feedback from the Focus Group Study and Integrating the Tool Into the Electronic Health Record

Incorporating physician feedback from the focus group (described in detail in companion paper Doi: 10.1055/s-0044-1782619), we successfully finalized the tool-building process and subsequent programming into the EHR, working together with the Epic programming team/Partners eCare through multiple iterations and weekly team calls over several months, solving practical aspects of the programming. The tool was implemented as three separate BPAs that would fire based on patient eligibility (male sex and age) and as a health maintenance module for scheduling the next PSA-test interval. The tool for facilitating SDM was embedded as an image in the BPAs, accessible through a hyperlink, and also including a hyperlink to the NCCN guideline. We demonstrated a finalized decision support design and proved that it was usable (i.e., system “up and running”) within 6 months. Managing to make a complex clinical decision support tool run properly was essential. The tool went live on November 5, 2019, and the study observation period closed on April 5, 2020.

Pilot Study

The aim of this pilot study was to test the feasibility and preliminary effectiveness of the intervention on a small scale, with 10 PCPs within the BWH primary care network. The sample size was chosen to ensure representativeness and different viewpoints among providers in the focus group and to determine the feasibility of the approach in the pilot study, before embarking on a large-scale cluster-randomized trial testing the efficacy.

We hypothesized that the PCPs would find the intervention feasible and acceptable, and that the majority would adhere to and act in accordance with the algorithm based on the information provided. Our primary endpoint was physicians' acceptability of the clinical decision support tool. At 6 months from study initiation, a study investigator contacted the PCPs by secure e-mail to partake in an electronic study-specific survey. The survey had quantitative questions as well as free text boxes for participants to provide open-ended feedback about the decision support tool. In addition to study-specific questions, we used an adaptation of a previously validated measurement instrument for evaluating the acceptability of decision aids, which was developed by the Patient Decision Aids Research group within the Ottawa Decision Support Framework.[20] PCPs were asked to rate their degree of agreement with several statements about the intervention, including usefulness, ease of use, amount of information, and their confidence in discussing PSA screening with their patients.

The study included 10 PCPs from BWH who volunteered to participate. This network consists of 15 practices in the greater Boston area, including two community health centers, with a total of 145 attending physicians at the time of the study. The primary care practices serve a broad cross-section of the greater Boston community and provide care for approximately 150,000 patients. All 10 participants received an incentive at the end of the 6-month study period for participating in the study.

Results

Results of the 6-month post-pilot study are shown in [Table 1]. All 10 PCPs found the tools easy to use. One PCP affirmed this with the comment: “Minimal amount of clicks to achieve desired outcome.” The PCPs either agreed (80%), or strongly agreed (20%), that it was clear as to how to use the tools. One PCP commented: “I found the tool to be easy to find, strategically located and facile to use.” The way the information was presented was found to be good (80%) or excellent (20%), with 80% of PCPs who found the length of the text presentation to be just right. One PCP who found the way the information was presented to be good further elaborated: “I am still reluctant to do a PSA on men > 70, unless they have elevated PSA to begin with.” This PCP also reported that the length of the text presentation on the tool was too long and said, “It does take some time to work through the information.” All PCPs agreed that the tools included enough information to help the PCPs and their patients make a decision about PSA screening, with one PCP commenting, “Yes, but still a difficult question to address.”

Table 1 Survey results at 6 months from study initiation (n = 10)

Question	Strongly agree	Agree	Neither agree nor disagree	Disagree	Strongly disagree
It was clear as to how to use the tools	2 (20%)	8 (80%)	0 (0%)	0 (0%)	0 (0%)
The tools were easy to use	0 (0%)	10 (100%)	0 (0%)	0 (0%)	0 (0%)
	Excellent	Good	Average	Fair	Poor
The way the information was presented was:	2 (20%)	8 (80%)	0 (0%)	0 (0%)	0 (0%)
	Just right	Too long	Too short
The length of text presentation on the tools was:	9 (90%)	1 (10%)	0 (0%)
	Yes	No
Do you feel that we included enough information in the tools to help you and your patient make decisions about PSA screening?	10 (100%)	0 (0%)
The tools consisted of several components. To what extent did you find each of the components USEFUL when you discussed PSA screening with your patients?	Extremely useful	Very useful	Somewhat useful	Slightly useful	Not at all useful
Best Practice Advisory: PSA screening guideline for eligible men—advising to do shared decision-making	0 (0%)	8 (80%)	1 (10%)	1 (10%)	0 (0%)
Best Practice Advisory: Advising against screening for younger men	0 (0%)	5 (50%)	4 (40%)	1 (10%)	0 (0%)
Best Practice Advisory: Advising against screening in older men	0 (0%)	7 (70%)	2 (20%)	1 (10%)	0 (0%)
Simple Schema: Educational material for shared decision-making	0 (0%)	6 (60%)	3 (30%)	1 (10%)	0 (0%)
Health Maintenance: Schedule reminders about PSA screening	1 (10%)	4 (40%)	4 (40%)	1 (10%)	0 (0%)
The tools consisted of several components. To what extent did you find each of the components BURDENSOME when you discussed PSA screening with your patients?	Not at all burdensome	Slightly burdensome	Somewhat burdensome	Very burdensome	Extremely burdensome
Best Practice Advisory: PSA screening guideline for eligible men—advising to do shared decision-making	1 (10%)	5 (50%)	3 (30%)	1 (10%)	0 (0%)
Best Practice Advisory: Advising against screening for younger men[a]	2 (20%)	5 (50%)	1 (10%)	1 (10%)	0 (0%)
Best Practice Advisory: Advising against screening in older men	3 (30%)	5 (50%)	1 (10%)	1 (10%)	0 (0%)
Simple Schema: Educational material for shared decision-making	2 (20%)	6 (60%)	2 (20%)	0 (0%)	0 (0%)
Health Maintenance: Schedule reminders about PSA screening	2 (20%)	3 (30%)	4 (40%)	1 (10%)	0 (0%)
	Much easier	Somewhat easier	About the same as before	Somewhat more difficult	Much more difficult
As compared to your experience before using these tools, did you feel that these tools made prostate cancer screening decisions for yourself as a physician:	1 (10%)	7 (70%)	2 (20%)	0 (0%)	0 (0%)
	Very likely	Likely	Unsure	Unlikely	Very unlikely
Would you consider continuing to use these tools in your clinic?	1 (10%)	9 (90%)	0 (0%)	0 (0%)	0 (0%)
How likely are you to recommend these tools to a colleague?	1 (10%)	9 (90%)	0 (0%)	0 (0%)	0 (0%)
	1—Not at all confident	2	3	4	5—Very confident
Before using these tools, how confident did you feel about your ability to discuss prostate cancer screening with your patient?	0 (0%)	2 (20%)	1 (10%)	5 (50%)	2 (20%)
After using these tools in your clinic, how confident do you feel now about your ability to discuss prostate cancer screening with your patient?	0 (0%)	0 (0%)	0 (0%)	6 (60%)	4 (40%)

Abbreviation: PSA, prostate-specific antigen.

^a There is one missing response to this question, because it was unanswered by one of the primary care physicians who completed the survey.

The decision support tools consisted of several components: three BPAs (SDM for eligible men, advising against PSA in younger men and advising against PSA for older men, respectively), a Simple Schema (educational material for SDM), and a health maintenance module (to schedule reminders about PSA rescreening). PCPs found the BPAs for men eligible for PSA screening (45–75 years) and men too old for PSA screening (>75 years) to be more useful than the BPA advising against PSA screening in younger men (<45 years). The three BPAs were not felt to be burdensome, or were only slightly burdensome, for most of the PCPs, with three reporting the tools to be very burdensome. Regarding the Simple Schema, most PCPs found this educational material very (60%) or somewhat (30%) useful, without being too burdensome (0% reported very or extremely burdensome). Most PCPs (90%) found the health maintenance tool to be extremely, very, or somewhat useful, albeit some reported it to be slightly (30%), somewhat (40%), or very (10%) burdensome. One PCP commented: “I always find the health maintenance module to be cumbersome no matter what screening test.”

As compared to the PCPs' experience before using the tools, 80% felt that the tools made PSA screening decisions much or somewhat easier for themselves as physicians. All PCPs reported that they would consider continuing to use the tools in their clinic and were likely (90%) or very likely (10%) to recommend the tools to a colleague. One PCP said: “I would use the tool but in selected cases when the patient wants the test and the MD thinks it is not indicated” and “I do feel the way the prompts are set up in Epic, we will do more screening.” PCPs' confidence in their ability to discuss PSA screening with their patients improved after using these tools in their clinic.

Discussion

In this study, we pilot tested an EHR-based clinical decision support tool for PSA screening in the primary care setting. Participating PCPs found the tool acceptable and feasible to use. These findings are important, as they serve as the groundwork and demonstrate the feasibility of the approach, which is needed to conduct a subsequent large-scale study in the primary care setting to study whether the decision support tool improves guideline-concordant screening, that is, implementation outcomes as well as screening outcomes.

Adherence to risk-stratified guidelines for PSA testing has the potential to balance the benefits and harms of PSA screening. For instance, the major harm of PSA screening is overdiagnosis, but over 40% of overdiagnoses occur in men outside the guideline-recommended ages for screening.[21] Thus, a >40% reduction in harm would be transformative.

Given the current suboptimal use of PSA, an EHR-based decision support tool is an obvious solution that has already been tested in two prior studies. In the study conducted by Shelton et al at the Veterans Affairs Greater Los Angeles Healthcare System, a computerized clinical decision support tool that reminded physicians about the guideline, not to screen older men, when ordering PSA screening was efficient at reducing overuse of PSA screening in men 75 years and older (relative risk reduction: 30–40%).[22] This study had many strengths, including the large number of participants (n = 2,001) and the use of a rigorous interrupted time-series study design, demonstrating a reproducible decline in inappropriate PSA use when the decision support system was turned on. However, a major limitation was that the implemented guideline was limited to only one target (PSA use in the older population) and did not include additional features or subsequent decisions regarding risk-stratified screening and biopsy.[22] Moreover, the wording on the reminder (or pop-up alert) that appeared on the ordering screen was neither pilot-tested nor were comments solicited from providers, and the alert wording was not based on behavioral science theory. Rather, it was described by the authors as a brief, interruptive educational message that was based on a guideline at the time: “The US Preventive Services Task Force and VHA [Veterans Health Administration] recommend AGAINST screening for prostate cancer in men 75 or older because the harms outweigh the benefits. Reconsider if this is a screening PSA.”[22] Additionally, the pop-up alert gave the provider a choice of continuing or canceling the test but did not require justification for either action. Although documenting guideline divergence is disruptive for clinical workflow, we have previously shown that requiring physicians to provide a clinical justification to override alerts can decrease repeat, unwarranted orders.[19]

In the study conducted by Presti et al at Kaiser Permanente Northern California, the group activated a BPA in the EHR with no prior notice or education to PCPs, instructing PCPs to remove PSA testing orders for men over 70 years with the text, “Routine PSA screening is not recommended in asymptomatic men 70 years or older who do not have prostate cancer or a history of prostate cancer,” or to keep the order with an extra click and review the health system's guideline on PSA screening.[23] Comparing two prespecified before-and-after intervention time periods, the PSA-testing rate declined from 36/100 person-years to 14.9/100 person-years, corresponding to a rate ratio of 0.42 (95% confidence interval: 0.41–0.42).[23] However, the design of the intervention did not build on behavioral economics principles aside from setting “cancel PSA-test order” as the default option, and was, similar to the study by Shelton et al,[22] only focused on avoiding the overuse of PSA in elderly men (from age 70 instead of 75 years). This is in contrast to our tool, which builds upon several behavioral economics principles, namely, leveraging social proof, shifting the reference point, and accountable justification, in addition to setting the default.

While earlier studies of EHR interventions to reduce inappropriate care, for example, antibiotic prescribing, have utilized principles from behavioral science,[24] [25] our intervention used narratives based on behavioral economics theory in the decision support tool, as opposed to prior studies, in which, as an example, an alert would briefly summarize: “antibiotics are not indicated for non-specific upper respiratory infections.”[25] Here, we proposed to look at a more complete picture of what could be done. This went beyond just inappropriate screening of older men to also include appropriate screening recommendations for younger men: risk-stratified recommendations for rescreening based on risk. To the best of our knowledge, no group has attempted to implement a comprehensive prostate cancer screening algorithm into a decision support tool embedded in the EHR. In the current study, PCPs and specialists worked together to craft messages appropriate for the primary care setting while following specialists' best practices. We recruited to the study team a Professor of Business Administration at BWH known to be an expert on negotiation, with a particular interest in cancer communication[26] and who has previously worked with our study team to develop a systematic approach to counseling men with prostate cancer regarding treatment options.[27] Here, we applied those same insights to the primary care setting and demonstrated proof-of-principle of using the application of clinical informatics and an EHR-embedded decision support tool algorithm fed by clinical data (gender, age, and PSA-levels), with automated alerts and suggestions for rescreening to deliver screening a health care service for clinical decision-making. Our successful development and feasibility testing of such a decision support tool thus fills a critical gap in optimizing screening and early detection of prostate cancer. We have now scaled-up the study within the primary care network and are conducting a cluster-randomized trial, where PCPs have been randomized to the decision support tool or paper-based guidelines, to study the effectiveness of the intervention in rationalizing screening, measured as: improved physician compliance with guideline-concordant care, which includes engagement in SDM, appropriate PSA-test orders by age, and biopsy referrals, as well as improved cancer detection rates, which includes decreased overdiagnosis rates, particularly among elderly men, and maintained detection of significant cancers, particularly among the young and healthy. Taken together, the EHR-integrated decision support tool that we have built will provide a foundation to optimize the appropriate use of the PSA test, resulting in a balance between the benefits and harms of PSA screening.

Our study is limited by a small sample size and potential selection bias among PCPs willing to participate possibly being more enthusiastic and knowledgeable about screening compared to nonparticipating PCPs; however, participants in the focus group shared mixed views on screening and found that the tool made decision-making about PSA easier and increased their confidence in discussing PSA screening as compared to before the pilot, suggesting we reached a representative sample. Generalizability is another potential issue, as the study was conducted within a large academic medical practice located in one geographic area in the United States. While our tool was integrated into one of the largest EHR software, we acknowledge that implementation into other systems may require reprogramming and adapting to different interfaces. The ease of using our tool was not compared to other EHR-based tools in our current study—an area for future research. Finally, we acknowledge that there is an array of different decision aids available for prostate cancer screening that could be considered for implementation.[28] [29] [30] We chose to use a Simple Schema based on being brief, evidence-based, facilitating a discrete choice and easy to implement.[11]

Conclusion

We found strong support and buy-in for this decision-support tool from the physicians, who found the tool feasible and acceptable to use in clinical practice. The PCPs found some components of the decision support tool more useful than others, for example, the PCPs felt that the BPAs had higher utility than the health maintenance module. Having completed this work, we have now begun the implementation phase, where we are now performing a cluster-randomized study to test the effectiveness of this innovative electronic decision support intervention as compared to usual care, which will determine if using the tool improves guideline-concordant care for men who are most likely to benefit from PSA screening.

Clinical Relevance Statement

We piloted a test an electronic health record-embedded decision support tool to facilitate prostate-specific antigen screening discussions in the primary care setting and found the tool to be feasible and acceptable. Tools like this can facilitate these conversations in primary care.

Multiple Choice Questions

1. The decision support tool, which was pilot tested in this study, comprised all of the following, except…

   • A risk calculator.

   • A tool for facilitating shared decision-making.

   • Three best practice advisories.

   • A health maintenance module.

   Correct Answer: The correct answer is option a. The tool included a risk-stratified algorithm, not a risk calculator.

2. Based on the results of this pilot study, which of the following answers is incorrect about the decision support tool?

   • Feasible

   • Acceptable

   • Clear to use

   • Complex

   Correct Answer: The correct answer is option d. Most primary care physicians in this pilot study found the decision support tool feasible, acceptable, and clear to use.

• References

• 1 Vickers AJ, Eastham JA, Scardino PT, Lilja H. The Memorial Sloan Kettering Cancer Center recommendations for prostate cancer screening. Urology 2016; 91: 12-18
• 2 Mottet N, Bellmunt J, Briers E. et al. Members of the EAU – ESTRO – ESUR –SIOG Prostate Cancer Guidelines Panel. EAU – EANM – ESTRO – ESUR – SIOG Guidelines on Prostate Cancer. Arnhem, The Netherlands: EAU Guidelines Office; 2020. [cited 2020 May 27]. Accessed January 30, 2024 at: https://uroweb.org/guideline/prostate-cancer/
• 3 Carter HB, Albertsen PC, Barry MJ. et al. Early detection of prostate cancer: AUA Guideline. J Urol 2013; 190 (02) 419-426
• 4 National Comprehensive Cancer Center (NCCN). Clinical Practice Guidelines in Oncology: Prostate Cancer Early Detection, Version 2.2018 2018 [updated April 5, 2008]. Accessed January 30, 2024 at: https://www.nccn.org/professionals/physician_gls/pdf/prostate_detection.pdf
• 5 Jemal A, Fedewa SA, Ma J. et al. Prostate cancer incidence and PSA testing patterns in relation to USPSTF screening recommendations. JAMA 2015; 314 (19) 2054-2061
• 6 Fleshner K, Carlsson SV, Roobol MJ. The effect of the USPSTF PSA screening recommendation on prostate cancer incidence patterns in the USA. Nat Rev Urol 2017; 14 (01) 26-37
• 7 Heijnsdijk EA, Wever EM, Auvinen A. et al. Quality-of-life effects of prostate-specific antigen screening. N Engl J Med 2012; 367 (07) 595-605
• 8 Tasian GE, Cooperberg MR, Cowan JE. et al. Prostate specific antigen screening for prostate cancer: knowledge of, attitudes towards, and utilization among primary care physicians. Urol Oncol 2012; 30 (02) 155-160
• 9 Joseph-Williams N, Newcombe R, Politi M. et al. Toward minimum standards for certifying patient decision aids: a modified Delphi consensus process. Med Decis Making 2014; 34 (06) 699-710
• 10 Makarov DV, Fagerlin A, Finkelstein J. et al. Implementation of Shared Decision Making into Urological Practice. Linthicum, MD: American Urological Association; 2022. . Accessed January 30, 2024 at: https://www.auanet.org/guidelines/guidelines/shared-decision-making
• 11 Vickers AJ, Edwards K, Cooperberg MR, Mushlin AI. A simple schema for informed decision making about prostate cancer screening. Ann Intern Med 2014; 161 (06) 441-442
• 12 Jefferson L, Bloor K, Birks Y, Hewitt C, Bland M. Effect of physicians' gender on communication and consultation length: a systematic review and meta-analysis. J Health Serv Res Policy 2013; 18 (04) 242-248
• 13 Volk RJ, Linder SK, Kallen MA. et al. Primary care physicians' use of an informed decision-making process for prostate cancer screening. Ann Fam Med 2013; 11 (01) 67-74
• 14 Huguet N, Ezekiel-Herrera D, Gunn R. et al. Uptake of a cervical cancer clinical decision support tool: a mixed-methods study. Appl Clin Inform 2023; 14 (03) 594-599
• 15 Miller SD, Murphy Z, Gray JH. et al. Human-centered design of a clinical decision support for anemia screening in children with inflammatory bowel disease. Appl Clin Inform 2023; 14 (02) 345-353
• 16 Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science 1981; 211 (4481) 453-458
• 17 Cialdini RB, Goldstein NJ. Social influence: compliance and conformity. Annu Rev Psychol 2004; 55: 591-621
• 18 Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica 1979; 47 (02) 263-292
• 19 O'Connor SD, Sodickson AD, Ip IK. et al. Journal club: requiring clinical justification to override repeat imaging decision support: impact on CT use. AJR Am J Roentgenol 2014; 203 (05) W482-90
• 20 O'Connor AM, Cranney A. Sample Tool: Acceptability (Osteoporosis Therapy). Ottawa, Canada: The Ottawa Hospital Research Institute; 1996. . Accessed January 30, 2024 at: www.ohri.ca/decisionaid
• 21 Vickers AJ, Sjoberg DD, Ulmert D. et al. Empirical estimates of prostate cancer overdiagnosis by age and prostate-specific antigen. BMC Med 2014; 12: 26
• 22 Shelton JB, Ochotorena L, Bennett C. et al. Reducing PSA-based prostate cancer screening in men aged 75 years and older with the use of highly specific computerized clinical decision support. J Gen Intern Med 2015; 30 (08) 1133-1139
• 23 Presti Jr J, Alexeeff S, Horton B, Prausnitz S, Avins AL. Changing provider PSA screening behavior using Best Practice Advisories: interventional study in a multispecialty group practice. J Gen Intern Med 2020; 35 (Suppl. 02) 796-801
• 24 Meeker D, Linder JA, Fox CR. et al. Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA 2016; 315 (06) 562-570
• 25 Persell SD, Doctor JN, Friedberg MW. et al. Behavioral interventions to reduce inappropriate antibiotic prescribing: a randomized pilot trial. BMC Infect Dis 2016; 16: 373
• 26 Malhotra D. Negotiating the Impossible: How to Break Deadlocks and Resolve Ugly Conflicts (without Money Or Muscle). 1st ed.. Oakland, CA: Berrett-Koehler Publishers; 2016
• 27 Ehdaie B, Assel M, Benfante N, Malhotra D, Vickers A. A systematic approach to discussing active surveillance with patients with low-risk prostate cancer. Eur Urol 2017; 71 (06) 866-871
• 28 Riikonen JM, Guyatt GH, Kilpeläinen TP. et al. Decision aids for prostate cancer screening choice: a systematic review and meta-analysis. JAMA Intern Med 2019; 179 (08) 1072-1082
• 29 Warlick CA, Berge JM, Ho YY, Yeazel M. Impact of a prostate specific antigen screening decision aid on clinic function. Urol Pract 2017; 4 (06) 448-453
• 30 Ivlev I, Jerabkova S, Mishra M, Cook LA, Eden KB. Prostate cancer screening patient decision aids: a systematic review and meta-analysis. Am J Prev Med 2018; 55 (06) 896-907