Suicide Risk Screening for Head and Neck Cancer Patients: An Implementation Study

Abstract

Objectives There is limited research on suicide risk screening (SRS) among head and neck cancer (HNC) patients, a population at increased risk for suicide. To address this gap, this single-site mixed methods study assessed oncology professionals' perspectives about the feasibility, acceptability, and appropriateness of an electronic SRS program that was implemented as a part of routine care for HNC patients.

Methods Staff who assisted with SRS implementation completed (e.g., nurses, medical assistants, advanced practice providers, physicians, social workers) a one-time survey (N = 29) and interview (N = 25). Quantitative outcomes were assessed using previously validated feasibility, acceptability, and appropriateness measures. Additional qualitative data were collected to provide context for interpreting the scores.

Results Nurses and medical assistants, who were directly responsible for implementing SRS, reported low feasibility, acceptability, and appropriateness, compared with other team members (e.g., physicians, social workers, advanced practice providers). Team members identified potential improvements needed to optimize SRS, such as hiring additional staff, improving staff training, providing different modalities for screening completion among individuals with disabilities, and revising the patient-reported outcomes to improve suicide risk prediction.

Conclusion Staff perspectives about implementing SRS as a part of routine cancer care for HNC patients varied widely. Before screening can be implemented on a larger scale for HNC and other cancer patients, additional implementation strategies may be needed that optimize workflow and reduce staff burden, such as staff training, multiple modalities for completion, and refined tools for identifying which patients are at greatest risk for suicide.

Keywords

suicide prevention - patient-reported outcomes - EHR - ePROs

Background and Significance

Suicide is a leading cause of death in the United States and disproportionately affects cancer patients.[1] [2] [3] [4] [5] Individuals with cancer are four times more likely to commit suicide than the general population.[5] Compared with other cancer types, individuals with head and neck cancer (HNC) are at elevated risk for suicide.[6] [7] Prior research suggests that HNC cancer patients are twice as likely to die from suicide compared with other cancer patients.[6] HNC is the seventh most common cancer globally and is partially attributed to alcohol and tobacco use, factors associated with suicide risk.[8] [9] [10] Additionally, due to the location of HNC and the aggressive treatment, HNC patients experience high levels of pain, financial hardship, emotional distress, and functional impairment (e.g., difficulty with speech and swallowing—HNC patients, there is limited research on suicide prevention interventions, such as suicide risk screening (SRS).

Health care systems are increasingly using electronic patient-reported outcomes (ePROs) to screen patients for suicide risk as a part of routine care. Prior studies have tested interventions that integrate ePROs into the electronic health record (EHR) to standardize screening for suicide risk in primary care and emergency department settings.[14] [15] [16] [17] [18] These interventions have demonstrated feasibility and improved access to behavioral health services.[14] [17] [18] ePRO systems for SRS are also being tested in oncology. For example, the Princess Margaret Cancer Centre in Canada integrated the nine-item Patient Health Questionnaire (PHQ-9) into the EHR to screen for suicide risk and found that completion of risk screening was associated with reduced suicide mortality among cancer patients.[19] Additional studies are needed to test EHR-based SRS in other oncology settings and to assess implementation outcomes, such as feasibility and acceptability. While there is strong evidence to suggest that integrating ePROs into routine care delivery can improve patient outcomes, the reach of ePROs has been limited due to implementation barriers.[20] [21] [22] [23] [24] For example, routine screening for psychosocial distress among cancer patients is recommended by clinical guidelines, but implementation varies across settings due to barriers, such clinician time and workflow integration.[25] [26] [27] [28] To ensure SRS can be reliably implemented as a part of routine cancer care, additional studies are needed to assess SRS implementation.

To address this gap, this study assessed oncology professionals' perspectives about the feasibility, acceptability, and appropriateness of implementing an EHR-based SRS program as a part of routine care for HNC patients. The study was conducted at Moffitt Cancer Center (Moffitt), a National Cancer Institute (NCI)-designated Comprehensive Cancer Center. Study findings can be used to design future SRS programs for oncology settings.

Methods

Study Design

The study used a sequential explanatory mixed methods design. A survey was administered among HNC oncology professionals who assisted with SRS implementation and followed by qualitative interviews to expand upon the survey findings and develop a qualitative description of participants' experience with SRS implementation.[29] [30] Data were collected during the initial implementation of the SRS intervention (first 3 months) from February to April 2021 so information gathered could be used to refine the intervention early on during implementation.

Suicide Risk Screening Intervention

Moffitt Cancer Center has implemented ePROs as a part of routine care since 2015 in the Departments of Radiation Oncology and Supportive Care Medicine.[31] [32] [33] [34] In 2021, Moffitt's selected the Department of Head and Neck Oncology (HNC) to be the next clinic for ePRO integration. After gathering feedback from key stakeholders, the Patient Reported Information and Outcomes committee decided to include SRS as a part of the ePRO assessment given the high burden of suicide risk among HNC patients.[6] The final tool, named the patient-reported symptom assessment (PRSA), included physical (e.g., pain, nausea) and psychosocial (e.g., distress) symptoms that were already being collected in other clinics and added ePROs to assess suicidal ideation. The PRSA included 16 items from the PHQ-9 (new addition), the National Comprehensive Cancer Network (NCCN) Distress Thermometer, and the revised Edmonton Symptom Assessment System (ESAS).[7] [8] [9] These tools were selected based on input from the PRIO committee, which includes a diverse group of stakeholders (e.g., oncology, supportive care, social work, research). The ESAS has been validated in cancer patients in general and HNC patients specifically.[35] [36] The NCCN Distress Thermometer has been validated in cancer patients in general.[37] The PHQ-9 has not been validated in cancer patients as a screening tool for suicidal ideation but has been paired with validated SRS tools to determine which patients may benefit from additional screening. Similar to the Princess Margaret Cancer Centre, our Cancer Center implemented a two-stage process: (1) patients first completed the PHQ-9 as a brief screener and (2) patients with concerning scores were prompted to complete a more comprehensive SRS screening (described below). The PRSA was made available in English and in Spanish. All patients with in-person visits were eligible to complete the PRSA during the check in process. Patients were given an iPad by a patient access representative (PAR) and provided with technical assistance (e.g., how to use tablet keyboard). A cloud-based application developed by the study site's application development and patient portal team was installed on the iPad to facilitate PRSA administration and EHR data integration. Multiple presentations at various faculty meetings (e.g., surgical team, medical oncology team) were held to make key stakeholders aware of the change and trainings were held for team members that would support implementation (e.g., nurses, medical assistants, PARs).

Once patients completed the PRSA, clinicians could review the responses in close to real time in the patient's medical record. Patients who reported any response other than “not at all” to the PHQ-9 question, “How often have you been bothered by thoughts that you would be better off dead or thoughts of hurting yourself in some way over the past 2 weeks?” were automatically referred to social work to be screened for active suicidal ideation (e.g., thoughts and plans for self-harm). The social work team assessed patients using the Columbia-Suicide Severity Rating Scale, which has been validated in clinical settings for assessing suicidal ideation.[10] [11] [12] [13] If a patient was identified as having active suicidal ideation, the social work team implemented additional interventions (e.g., suicide safety plan). Nurses were responsible for reviewing patients' PRSA scores with the patient and determining if additional intervention was needed (e.g., symptom management support). For example, patients requiring additional symptom management support could be referred to Moffitt's Supportive Care Medicine Department. Patients reporting distress were offered the option to meet with the social work team.

During the pilot, the PRSA was completed by 746 patients. Among the 746 patients who completed the PRSA, 34 patients reported passive suicidal ideation based on PHQ-9 score and were referred to social work for SRS. Of the 34 patients screened for active suicidal ideation by social work, 3/34 (8.8%) patients were identified as having immediate suicide risk and received additional intervention (e.g., suicide safety plan). The remaining patients not at immediate risk for suicide (n = 31) received additional services including supportive care (n = 28), referral to social service agencies (e.g., financial counseling) (n = 22), and facilitation of goals of care conversations (n = 7).

Data Collection

Participating oncology health care professionals completed a one-time survey and one-time interview. The survey was administered electronically via Qualtrics (Qualtrics, Provo, Utah, United States). All staff who assisted with PRSA implementation (e.g., nurses, medical assistants, PARs, advanced practice providers [APPs], social workers, physicians) received an email inviting them to participate in a survey. Informed consent was provided at the time of the survey. The survey response rate was 65.9% (29/44 individuals). The survey assessed perceptions about implementation outcomes including feasibility, acceptability, and appropriateness.[38] [39] Outcomes were assessed using the Feasibility of the Intervention Measure, the Acceptability of the Intervention Measure, and the Intervention Appropriateness Measure.[38] The measures contain four items with response options ranging from 0 (strongly disagree) to 5 (strongly agree). Items for each measure were summed to create a total score (range: 0–20). A total score of 20 represents the best implementation outcome (e.g., highest level of participant satisfaction). A validated cutoff for these measures has not been established; therefore, a score of 16 or higher was defined as a cutoff for establishing feasibility, acceptability, and appropriateness based on a prior study among oncology professionals (a score of 16 denotes an average response of agree or strongly agree for each item within the measure).[40]

All staff who assisted with PRSA implementation or support (e.g., clinical informatics staff) received an email inviting them to participate in an interview. We did not include informatics staff in the survey given they may not be able to answer key questions (e.g., how appropriate the intervention is for this patient population). We chose, however, to include them in the interview process given their key role in supporting the intervention. Informed consent was provided at the time of the interview. The interview participation rate was 51.0% (25/49 individuals). Individuals who declined to participate cited lack of time and competing priorities as the primary reasons. The interviews were conducted by two qualitative specialists from Moffitt's Participation Research, Interventions, and Measurement Core through videoconference, recorded, and transcribed verbatim. The interviews were facilitated using a semi-structured interview guide that assessed perceptions about implementation outcomes, barriers, and facilitators guided by the Consolidated Framework for Implementation Research[41] ([Supplementary Table S1], available in the online version). The interviews were approximately 30 minutes in length (mean time: 31.1 minutes; standard deviation [SD]: 12.3 minutes). Participants did not receive any incentives for study participation.

Data Analysis

For the survey data, we calculated descriptive statistics (e.g., mean, SD, percentages) to summarize staff perceptions about feasibility, acceptability, and appropriateness using Stata version 17.0 (StataCorp, College Station, Texas, United States). For the interview data, we used a hybrid approach by developing a codebook that included codes based on concepts from the interview guide (implementation outcomes, barriers, and facilitators) and themes that emerged from the data.[42] [43] Two qualitative research specialists from Moffitt's Participant, Research, Intervention, and Measurement Core coded all transcripts and discussed and resolved any coding discrepancies using NVivo 12 Plus (Burlington, Massachusetts, United States). The individuals set a threshold for determining when data saturation was achieved, which was reached at 25 interviews.[44] Therefore, no additional interviews were conducted. For study reporting, the study team adhered to the Consolidated Criteria for Reporting Qualitative Research guidelines.[45] Moffitt's Institutional Review Board of Record, Advarra, reviewed the study protocol and determined the study to be exempt.

Results

Sample Characteristics

Survey participants (N = 29) included nurses (20.7%), medical assistants (20.7%), PARs (17.2%), APPs (13.8%), physicians (10.3%), and social workers (17.2%) ([Table 1]). About a quarter (27.6%) of survey participants had experience with implementing PROs as a part of clinical care previously. The average job tenure of survey participants was 8.2 years (SD: 6.7).

Table 1 Survey and interview participant sample characteristics

Characteristic	Survey participants, N = 29	Interview participants, N = 25
Occupation, N (%)
Nurse	6 (20.7)	5 (20.0)
Medical assistant	6 (20.7)	4 (16.0)
Patient access representative	5 (17.2)	3 (12.0)
Advanced practice provider	4 (13.8)	3 (12.0)
Physician	3 (10.3)	2 (8.0)
Social worker	5 (17.2)	4 (16.0)
Informatics staff	NA	4 (16.0)
Job tenure in years, mean (standard deviation)	8.2 (6.7)	7.9 (6.1)
Prior experience with patient reported outcomes, N (%)
Yes	8 (27.6)	10 (40.0)
No	21 (72.4)	15 (60.0)
Sex, N (%)
Female	17 (58.6)	16 (64.0)
Male	12 (41.4)	9 (36.0)
Race/ethnicity, N (%)
Non-Hispanic Black/African American	3 (10.3)	3 (12.0)
Hispanic/Latinx	5 (17.2)	4 (16.0)
Asian	5 (17.2)	5 (20.0)
Non-Hispanic White	16 (55.2)	13 (52.0)

Abbreviation: NA, not applicable.

Interview participants (N = 25) included nurses (20.0%), medical assistants (16.0%), PARs (12.0%), APPs (12.0%), physicians (8.0%), social workers (16.0%), and informatics staff (16.0%; [Table 1]). Almost half (40.0%) of interview participants had experience with implementing PROs in clinical care previously. The average job tenure of interview participants was 7.9 years (SD: 6.1).

Perceptions about Feasibility

Overall, participants rated the SRS assessment as having low feasibility (mean total score: 12.6; SD: 4.4 points; [Table 2]). The ratings varied across staff roles ([Table 3]). Feasibility scores were lower among nurses (mean: 9.6; SD: 4.2) and medical assistants (mean: 10.4; SD: 5.0) compared with APPs (mean: 12.8; SD: 5.9), PARs (mean: 14.6, SD: 3.0), physicians (mean: 16.7; SD: 31), and social workers (mean: 17.5; SD: 0.7).

Table 2 Perceptions about feasibility, acceptability, and appropriateness of patient-reported symptom assessment, N = 29

Item	Strongly disagree N (%)	Disagree N (%)	Neutral N (%)	Agree N (%)	Strongly agree N (%)	Average score mean (SD)
Feasibility
PRSA seems  implementable	4 (13.8)	6 (20.7)	7 (24.1)	10 (34.5)	2 (6.9)	3.1 (1.2)
PRSA seems  possible	4 (13.8)	3 (10.3)	6 (20.7)	13 (44.8)	3 (10.3)	3.3 (1.2)
PRSA seems doable	4 (13.8)	3 (10.3)	6 (20.7)	13 (44.8)	3 (10.3)	3.3 (1.2)
PRSA seems easy  to use	4 (13.8)	7 (24.1)	8 (27.6)	8 (27.6)	2 (6.9)	2.9 (1.2)
Total score	–	–	–	–	–	12.6 (4.4)
Acceptability
PRSA meets my  approval	5 (17.2)	11 (37.9)	8 (27.6)	2 (6.9)	3 (10.3)	2.6 (1.2)
PRSA is appealing  to me	7 (24.1)	8 (27.6)	7 (24.1)	2 (6.9)	5 (17.2)	2.7 (1.4)
I like PRSA	6 (20.7)	7 (24.1)	10 (34.5)	2 (6.9)	4 (13.8)	2.8 (1.3)
I welcome PRSA	5 (17.2)	2 (6.9)	8 (27.6)	9 (31.0)	5 (17.2)	3.3 (1.3)
Total score	–	–	–	–	–	11.5 (4.8)
Appropriateness
PRSA seems fitting	4 (13.8)	7 (24.1)	8 (27.6)	7 (24.1)	3 (10.3)	3.0 (1.3)
PRSA seems  suitable	4 (13.8)	5 (17.2)	9 (31.0)	7 (24.1)	4 (13.8)	3.2 (1.3)
PRSA seems  applicable	5 (17.2)	5 (17.2)	9 (31.0)	7 (24.1)	3 (10.3)	3.0 (1.3)
PRSA seems like a  good match	5 (17.2)	8 (27.6)	9 (31.0)	4 (13.8)	3 (10.3)	2.8 (1.3)
Total score	–	–	–	–	–	12.1 (4.8)

Abbreviations: PRSA, patient-reported symptom assessment; SD, standard deviation.

Note: The Feasibility, Acceptability, and Appropriateness scores range from 0 to 20 with higher scores indicating better implementation.

Table 3 Perceptions about feasibility, acceptability, and appropriateness based on staff role

Characteristic	Feasibility score, mean (SD)	Acceptability, mean (SD)	Appropriateness, mean (SD)
Occupation
Nurse	9.6 (4.2)	8.2 (3.9)	8.0 (4.0)
Medical assistant	10.4 (5.0)	7.8 (4.3)	9.8 (4.2)
Patient access representative	14.6 (3.0)	12.0 (5.7)	12.5 (5.3)
Advanced practice provider	12.8 (5.9)	12.0 (4.8)	13.2 (5.3)
Physician	16.7 (3.1)	18.3 (1.5)	16.7 (3.1)
Social worker	17.5 (0.7)	19.5 (0.7)	20.0 (0.0)

Abbreviation: SD, standard deviation.

During the interviews, participants discussed several factors that may affect feasibility, which varied across staff roles ([Supplementary Table S1], available in the online version). Nurses and medical assistants, for example, described there was insufficient time before and during the visit to screen for suicide risk and review other symptoms (e.g., pain). Prior to the visit, medical assistants did not feel there was enough time to read through and complete the SRS, leading patients to rush when completing the questionnaire and not fully reading the questions. During the visit, nurses described having many priorities to address related to other initiatives (e.g., medicine reconciliation) and not having enough time to address suicide risk.

In addition to lack of time, medical assistants, nurses, and APPs described that there was insufficient staffing to reliably implement SRS. APPs described how nursing staff availability varies across providers and that if an APP does not have nursing staff support for a given patient, review of the patient-reported symptoms may get missed or delayed until the end of the visit. Medical assistants and nurses expressed concerns about social work staff availability. Participants noted that there was only one main social worker assigned for the HNC clinic. When the assigned social worker was out of the office, some team members felt it was challenging to find another social worker to assist with SRS in a timely manner.

Participants also discussed how SRS fits within their current workflow. Certain care team members, including PARs and social workers, felt that the initiative fit well within their workflow. The social work team discussed how they are well prepared for screening patients for suicide risk and are already engaged in this work. PARs also felt that SRS fit within their workflow because they already have a process for assisting patients with new questionnaires. Other staff, however, including nurses, APPs, and physicians described not knowing how to fit the review of patient-reported outcomes into their workflow. Some physicians, for example, described not knowing what to do with the information and ignoring it.

Perceptions about Acceptability

Overall, participants rated SRS as having low acceptability intervention (mean: 11.5, SD: 4.8) ([Table 2]). Acceptability scores were lower among nurses (mean: 8.2; SD: 3.9) and medical assistants (mean: 7.8, SD: 4.3) and higher among physicians (mean: 18.3; SD: 1.5) and social workers (mean: 19.5; SD: 0.7; [Table 3]).

All health care team members noted several factors that affected staff and patient satisfaction with the SRS. Participants were satisfied, for example, with the symptom discussions that resulted from the screening. Staff members noted being able to identify unmet mental health care needs, facilitate advanced care planning and goals of care conversations, and develop a more holistic view of the patient. Participants also appreciated that the screening collects symptoms that can be difficult for patients to discuss, such as mental health concerns and certain physical symptoms (e.g., constipation, diarrhea). A few health care team members noted that screening improves data collection on quality of life, information that could be useful for researchers to identify gaps in care delivery.

Nurses, medical assistants, PARs, and clinical informatics staff noted some components of the screening process that they were dissatisfied with. Clinical informatics and nursing staff described challenges with EHR data integration. For example, if the Wi-Fi connection was lost in the HNC clinic, data could be lost if a patient had not completed their screening. Clinical informatics staff described instances where patient data were lost and the SRS had to be recompleted, creating frustration for staff and patients. Nurses were also concerned patient data that may not display in real time due to Wi-Fi connectivity issues, which could result in missing a patient who may be at-risk for suicide (e.g., in the instance that positive screening results do not display in EHR before the visit ends). Nurses also noted that the screening could be burdensome for patients who had frequent visits (e.g., more than one visit in a week), resulting in patient frustration. Medical assistants and PARs expressed concerns about only providing one modality to complete the screening (e.g., tablet completion). They noted that some patients had difficulty using the tablet (e.g., vision impairment, dexterity issues). Nurses also expressed fears about risk management. Nurses were concerned that they are primarily responsible for the intervention and felt that there should be another level of data review (e.g., by APP or physician) so that if the nurse misses a concerning score, another member of the care team will have the opportunity to catch it. Nurses felt that secondary review of the data by an APP or physician highly varied across providers.

Perceptions about Appropriateness

Participants rated suicide screening as having low appropriateness for the HNC patient population and the oncology clinic setting (mean: 12.1; SD: 4.8; [Table 2]). Appropriateness scores were lower among nurses (mean: 8.0; SD: 4.0) and medical assistants (mean: 9.8; SD: 4.2) and higher among physicians (mean: 16.7; SD: 3.1) and social workers (mean: 20.0; SD: 0.0; [Table 3]).

During the interviews, participants described factors that may improve the appropriateness of the PRSA, such as changing the suicidal ideation items, the target population, and the timeframe for symptom recall. For example, health care team members expressed divergent opinions about the appropriateness of the PHQ-9 as a measure of suicidal ideation among cancer patients. Some members of the clinical team indicated that the item was too broad and should be separated into two questions, one that measures passive wish for death and one item that measures intention for self-harm. Other team members felt that the PHQ-9 was an appropriate tool for identifying patients with complex mental health needs who needed further intervention. For example, one social worker mentioned that the PHQ-9 helps identify two groups—patients who may have suicidal ideation and patients who have other unmet mental health care needs (e.g., high distress). Both groups are likely to benefit from social work intervention. Participants also recommended adding disease-specific symptoms that are relevant to HNC (e.g., swallowing difficulty) and social determinants of health correlated with suicide risk (e.g., financial hardship).

Team members also noted that the PRSA may not be relevant for all patients served by the HNC clinic. For example, one medical assistant mentioned that the PRSA captures symptoms (e.g., diarrhea) that may be more relevant for patients who are undergoing active treatment rather than patients who are in the surveillance phase of their cancer care journey. A member of the clinical informatics staff described how the PRSA is currently targeted to all patients with an in-person visit in the HNC clinic and that it may be more valuable to target the form based on patient characteristics (e.g., type of cancer, phase of cancer care continuum). Participants were also concerned about the relevance of the recall period. For example, the PHQ-9 recall period is 2 weeks, which may be confusing for patients with multiple visits within a 2-week timeframe. Additionally, participants felt that the PRSA may lack relevance for patients who have not had a change in symptoms since the last PRSA completion.

Recommendations

During the interviews, participants provided several recommendations for improving implementation including (1) increasing staffing availability (e.g., nursing, social work) to support SRS, (2) improving SRS staff training (e.g., providing ongoing rather than one-time training, guidance on patient communication about suicide risk), (3) additional modalities for SRS completion to give patients more time and accommodate patients with disabilities (e.g., vision impairment), (4) educating patients and staff on the purpose of SRS and how it fits within broader institutional goals to improve buy-in, (5) reducing the number of times patients are required to complete the SRS (e.g., once a month versus every in-person visit), and (6) ensuring the screening captures PROs important for HNC patients (e.g., swallowing difficulty).

Discussion

In this study, we assessed oncology professionals' perceptions about the feasibility, accessibility, and appropriateness of an electronic SRS for individuals with HNC. Overall, perceptions about feasibility, acceptability, and appropriateness varied widely depending on the team member's role within implementation. Oncology professionals who were responsible for administering the SRS and reviewing the scores, including medical assistants and nurses, rated SRS as having low feasibility, acceptability, and appropriateness. Other members of the care team, including social workers, PARs, APPs, and physicians rated SRS as having higher feasibility, acceptability, and appropriateness. Participants recommended several strategies that could be tested in the future to support successful implementation of SRS, such as strengthening staff training, increasing accessibility for individuals with disabilities, and improving risk stratification.

Numerous studies have documented staff training as a barrier to PRO implementation in general and SRS implementation specifically.[46] [47] [48] [49] Several training programs have been developed that focus on how to identify and monitor symptoms based on PRO data and how to use PRO data as a part of clinical care.[50] [51] Recommended elements of training include having talking points that clinicians can use to facilitate communication about PROs, information about the evidence base for PROs (e.g., how, when used effectively, PRO collection can affect patient outcomes), and information about implementation (e.g., rationale for PRO selection, how to interpret, and document PRO scores). There is a need for additional research to rigorously evaluate clinician training programs to determine which models may be most effective at supporting integration of PROs into routine care in the context of SRS. One model that could be tested is implementation coaching, which has been used to support implementation of other evidence-based interventions (e.g., human papillomavirus vaccination).[52] [53]

In addition to clinician training, study participants recommended improving PRO accessibility for patients with disabilities through multimodal screening. Completing SRS on a tablet proved difficult for patients with visual impairment, cognitive impairment, or loss of dexterity. Previous research by Bennett et al comparing patient-reported data collection by tablet and voice response found that both modalities can be used to effectively collect PROs, an option that may be preferable for adults with vision impairment and older adults compared with tablet screening.[54] [55] Further research is needed on optimizing access to PROs for individuals with cancer and disabilities that may affect PRO completion (e.g., vision impairment, cognitive impairment, dexterity-related conditions).

Participants in our study reported mixed perspectives about whether SRS is feasible to implement given current staffing and workload. Therefore, strategies should be tested that optimize the workflow of SRS and reduce burden on staff, particularly nurses. For example, researchers have tested the validity of using data already available within the EHR to predict which patients may be at risk for suicide in primary care and emergency department settings.[56] [57] [58] [59] [60] One study found that an EHR-data model (area under the curve [AUC] = 0.775) had similar performance at classifying suicide risk among patients receiving care in a pediatric emergency department (N = 13,420) compared with a patient-reported SRS model (AUC = 0.754).[59] Future studies could test a similar approach in additional care settings including oncology. Other strategies may include improving risk stratification. Like prior studies, our study found that 8.8% of patients flagged for additional follow-up based on PHQ-9 scores were classified as having active suicidal ideation.[12] [61] Research has recommended approaches such as combining the use of multiple PROs to improve classification of suicide risk.[62] Additional testing is needed to evaluate what data types and which PROs may be most effective at identifying suicide risk in HNC and other cancer patients.

Participants were divided about whether the included ePROs were appropriate for HNC patients. The PRSA was designed to capture symptoms that are common among all cancer patients (e.g., nausea) and could be scaled across oncology clinics. Participants focused on HNC care delivery (e.g., nurses) rated the tool as lower in appropriateness than staff who work across clinics (e.g., social work), which may account for these differences. The tool did not capture common symptoms for HNC patients (e.g., swallowing difficulty). Further, some staff had concerns that SRS may be less relevant for individuals who are no longer undergoing active treatment and that assessing SRS and other ePROs at every clinic visit was burdensome for patients and staff. Additional research is needed to come to a consensus on what are best measures for SRS, when individuals with cancer should be screened, and how often.

Limitations

Our study has a few limitations. First, our study was conducted at an NCI-designated Comprehensive Cancer Center, and therefore, the findings may not be generalizable to other cancer care settings. Second, our study captured the perspectives of health care professionals about SRS implementation and does not capture patients' perspectives. Future studies are needed to capture the patient perspective. Additionally, while our survey (65.9%) and interview (51.0%) response rates were moderately high; there are likely important perspectives that were missed. This highlights the need for additional studies to assess implementation of SRS in oncology settings. Further, SRS was implemented alongside other ePROs (e.g., distress), and therefore, our study identified implementation issues regarding ePRO implementation broadly and SRS implementation specifically. We also did not measure cost or cost-effectiveness as a part of this pilot study. Future studies should assess the cost of SRS implementation. Finally, our interviews were conducted early-on during implementation so that information from the surveys and information could be used to refine the intervention. Therefore, perspectives regarding implementation may have changed over time, as the intervention was refined, and staff grew more accustomed to SRS. We are unable to evaluate this currently because SRS was removed from the PRSA tool due to the implementation barriers documented in this study. Our team is currently working on refining our approach through usability testing of the tool, development of additional training tools, and selection of additional instruments to assess suicide risk prior to a second pilot.

Conclusion

Overall, cancer care team members had mixed perspectives about the feasibility of implementing SRS as a part of routine care and identified important areas for future research to improve future SRS implementation. Before SRS screening can be implemented on a larger scale for HNC and other cancer patients, additional implementation strategies may be needed, such as staff training, multiple modalities for completion, and refined tools for identifying which patients are at greatest risk for suicide.

Clinical Relevance Statement

Individuals with cancer are at increased risk for suicide. ePRO monitoring programs can identify patients at-risk but are challenging to implement due to limited staffing, insufficient training, and limited information technology capacity among health care systems. Strategies are needed that improve implementation of ePRO systems to reduce burden among already taxed clinic staff.

Multiple Choice Questions

1. Why are patient-reported outcome (PRO) monitoring programs challenging to implement in clinical practice?

   • Patient-level barriers (e.g., patient engagement)

   • Clinician-level barriers (e.g., clinician self-efficacy)

   • System-level barriers (e.g., information technology [IT] capacity)

   • All of the above

   Correct Answer: The correct answer is option d. PRO monitoring programs can be challenging to implement due to patient-level barriers, such as patient engagement with PROs, clinician-level barriers, such as clinician self-efficacy for using PROs in clinical practice, and system-level barriers, such as limited IT capacity to support implementation.

2. How can PRO systems be modified to improve accessibility among individuals with disabilities?

   • Offering multiple modalities for completion

   • Increasing the font size of PRO assessments

   • Providing technical assistance with PRO assessments

   • All of the above

   Correct Answer: The correct answer is option d. Research has recommended several strategies for improving PRO accessibility among individuals with disabilities including offering multiple modalities for completion (e.g., paper, electronic), increasing the font size of assessments, and providing technical assistance with PRO assessments for individuals who may have difficulty using electronic PRO systems.

Conflict of Interest

D.E.R. is on the Board of Directors for NanoString Technologies, Inc.

Availability of the Data

To protect the privacy of the individuals that participated in this study, the individual-level data underlying this article cannot be shared. Summary-level data may be requested.

Author Contributions

B.K.: Conceptualization; Writing—original draft; Writing—review & editing; Methodology; A.B.: Writing—review & editing; M.M.: Writing—review & editing; R.P.: Writing—review & editing; P.R.: Writing—review & editing; J.H.J.: Writing—review & editing; D.E.R.: Writing—review & editing; O.T.N.: Writing—review & editing; S.P.: Writing—review & editing; S.M.G.: Writing—review & editing; K.T.: Conceptualization; Methodology; Project administration.

Protection of Human and Animal Subjects

Moffitt Cancer Center Institutional Review Board of Record, Advarra, reviewed the study protocol and determined the study to be exempt.

• References

• 1 Fang F, Fall K, Mittleman MA. et al. Suicide and cardiovascular death after a cancer diagnosis. N Engl J Med 2012; 366 (14) 1310-1318
• 2 Henson KE, Brock R, Charnock J, Wickramasinghe B, Will O, Pitman A. Risk of suicide after cancer diagnosis in England. JAMA Psychiatry 2019; 76 (01) 51-60
• 3 Misono S, Weiss NS, Fann JR, Redman M, Yueh B. Incidence of suicide in persons with cancer. J Clin Oncol 2008; 26 (29) 4731-4738
• 4 Ravaioli A, Crocetti E, Mancini S. et al. Suicide death among cancer patients: new data from northern Italy, systematic review of the last 22 years and meta-analysis. Eur J Cancer 2020; 125: 104-113
• 5 Zaorsky NG, Zhang Y, Tuanquin L, Bluethmann SM, Park HS, Chinchilli VM. Suicide among cancer patients. Nat Commun 2019; 10 (01) 207
• 6 Osazuwa-Peters N, Simpson MC, Zhao L. et al. Suicide risk among cancer survivors: head and neck versus other cancers. Cancer 2018; 124 (20) 4072-4079
• 7 Kam D, Salib A, Gorgy G. et al. Incidence of suicide in patients with head and neck cancer. JAMA Otolaryngol Head Neck Surg 2015; 141 (12) 1075-1081
• 8 Sung H, Ferlay J, Siegel RL. et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2021; 71 (03) 209-249
• 9 Echeverria I, Cotaina M, Jovani A, Mora R, Haro G, Benito A. Proposal for the inclusion of tobacco use in suicide risk scales: results of a meta-analysis. Int J Environ Res Public Health 2021; 18 (11) 6103
• 10 Isaacs JY, Smith MM, Sherry SB, Seno M, Moore ML, Stewart SH. Alcohol use and death by suicide: a meta-analysis of 33 studies. Suicide Life Threat Behav 2022; 52 (04) 600-614
• 11 Chua KS, Reddy SK, Lee MC, Patt RB. Pain and loss of function in head and neck cancer survivors. J Pain Symptom Manage 1999; 18 (03) 193-202
• 12 Chang DC, Chen AW, Lo YS, Chuang YC, Chen MK. Factors associated with suicidal ideation risk in head and neck cancer: a longitudinal study. Laryngoscope 2019; 129 (11) 2491-2495
• 13 Massa ST, Osazuwa-Peters N, Adjei Boakye E, Walker RJ, Ward GM. Comparison of the financial burden of survivors of head and neck cancer with other cancer survivors. JAMA Otolaryngol Head Neck Surg 2019; 145 (03) 239-249
• 14 Gardner W, Klima J, Chisolm D. et al. Screening, triage, and referral of patients who report suicidal thought during a primary care visit. Pediatrics 2010; 125 (05) 945-952
• 15 Etter DJ, McCord A, Ouyang F. et al. Suicide screening in primary care: use of an electronic screener to assess suicidality and improve provider follow-up for adolescents. J Adolesc Health 2018; 62 (02) 191-197
• 16 Davis M, Rio V, Farley AM, Bush ML, Beidas RS, Young JF. Identifying adolescent suicide risk via depression screening in pediatric primary care: an electronic health record review. Psychiatr Serv 2021; 72 (02) 163-168
• 17 Fein JA, Pailler ME, Barg FK. et al. Feasibility and effects of a web-based adolescent psychiatric assessment administered by clinical staff in the pediatric emergency department. Arch Pediatr Adolesc Med 2010; 164 (12) 1112-1117
• 18 Wintersteen MB. Standardized screening for suicidal adolescents in primary care. Pediatrics 2010; 125 (05) 938-944
• 19 Gascon B, Leung Y, Espin-Garcia O, Rodin G, Chu D, Li M. Suicide risk screening and suicide prevention in patients with cancer. JNCI Cancer Spectr 2021; 5 (04) pkab057
• 20 Stover AM, Kurtman R, Walker Bissram J. et al. Stakeholder perceptions of key aspects of high quality cancer care to assess patient-reported outcome measures: a systematic review. Cancers 2021; 13 (14) 3628
• 21 Stover AM, Tompkins Stricker C, Hammelef K. et al. Using stakeholder engagement to overcome barriers to implementing patient-reported outcomes (PROs) in cancer care delivery: approaches from 3 prospective studies. Med Care 2019; 57 (1, Suppl 5 Suppl 1): S92-S99
• 22 Basch E, Deal AM, Dueck AC. et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA 2017; 318 (02) 197-198
• 23 Basch E, Deal AM, Kris MG. et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016; 34 (06) 557-565
• 24 Denis F, Basch E, Septans AL. et al. Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA 2019; 321 (03) 306-307
• 25 Donovan KA, Jacobsen PB. Progress in the implementation of NCCN guidelines for distress management by member institutions. J Natl Compr Canc Netw 2013; 11 (02) 223-226
• 26 Donovan KA, Deshields TL, Corbett C, Riba MB. Update on the implementation of NCCN guidelines for distress management by NCCN member institutions. J Natl Compr Canc Netw 2019; 17 (10) 1251-1256
• 27 Jacobsen PB, Ransom S. Implementation of NCCN distress management guidelines by member institutions. J Natl Compr Canc Netw 2007; 5 (01) 99-103
• 28 Jacobsen PB, Norton WE. The role of implementation science in improving distress assessment and management in oncology: a commentary on “Screening for psychosocial distress among patients with cancer: implications for clinical practice, healthcare policy, and dissemination to enhance cancer survivorship”. Transl Behav Med 2019; 9 (02) 292-295
• 29 Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health 2011; 38 (01) 44-53
• 30 Kim H, Sefcik JS, Bradway C. Characteristics of qualitative descriptive studies: a systematic review. Res Nurs Health 2017; 40 (01) 23-42
• 31 Johnstone PAS, Lee J, Zhou JM. et al. A modified Edmonton Symptom Assessment Scale for symptom clusters in radiation oncology patients. Cancer Med 2017; 6 (09) 2034-2041
• 32 Johnstone PAS, Bulls HW, Zhou JM. et al. Congruence of multiple patient-related outcomes within a single day. Support Care Cancer 2019; 27 (03) 867-872
• 33 Bulls HW, Chang PH, Brownstein NC. et al. Patient-reported symptom burden in routine oncology care: examining racial and ethnic disparities. Cancer Rep (Hoboken) 2022; 5 (03) e1478
• 34 Turner K, Brownstein NC, Thompson Z. et al. Longitudinal patient-reported outcomes and survival among early-stage non-small cell lung cancer patients receiving stereotactic body radiotherapy. Radiother Oncol 2022; 167: 116-121
• 35 Noel CW, Forner D, Chepeha DB. et al. The Edmonton Symptom Assessment system: a narrative review of a standardized symptom assessment tool in head and neck oncology. Oral Oncol 2021; 123: 105595
• 36 Nekolaichuk C, Watanabe S, Beaumont C. The Edmonton Symptom Assessment system: a 15-year retrospective review of validation studies (1991–2006). Palliat Med 2008; 22 (02) 111-122
• 37 Hoffman BM, Zevon MA, D'Arrigo MC, Cecchini TB. Screening for distress in cancer patients: the NCCN rapid-screening measure. Psychooncology 2004; 13 (11) 792-799
• 38 Weiner BJ, Lewis CC, Stanick C. et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci 2017; 12 (01) 108
• 39 Proctor E, Silmere H, Raghavan R. et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011; 38 (02) 65-76
• 40 Wood KC, Pergolotti M, Marshall T. et al. Usability, acceptability, and implementation strategies for the Exercise in Cancer Evaluation and Decision Support (EXCEEDS) algorithm: a Delphi study. Support Care Cancer 2022; 30 (09) 7407-7418
• 41 Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4: 50
• 42 Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods 2006; 5 (01) 80-92
• 43 Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res 2019; 280: 112516
• 44 Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS One 2020; 15 (05) e0232076
• 45 Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007; 19 (06) 349-357
• 46 Nguyen H, Butow P, Dhillon H, Sundaresan P. A review of the barriers to using patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs) in routine cancer care. J Med Radiat Sci 2021; 68 (02) 186-195
• 47 Foster A, Croot L, Brazier J, Harris J, O'Cathain A. The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews. J Patient Rep Outcomes 2018; 2: 46
• 48 Glenwright BG, Simmich J, Cottrell M. et al. Facilitators and barriers to implementing electronic patient-reported outcome and experience measures in a health care setting: a systematic review. J Patient Rep Outcomes 2023; 7 (01) 13
• 49 Antunes B, Harding R, Higginson IJ. EUROIMPACT. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med 2014; 28 (02) 158-175
• 50 Santana MJ, Haverman L, Absolom K. et al. Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res 2015; 24 (07) 1707-1718
• 51 Skovlund PC, Ravn S, Seibaek L, Thaysen HV, Lomborg K, Nielsen BK. The development of PROmunication: a training-tool for clinicians using patient-reported outcomes to promote patient-centred communication in clinical cancer settings. J Patient Rep Outcomes 2020; 4 (01) 10
• 52 Ballengee LA, Rushton S, Lewinski AA. et al. Effectiveness of quality improvement coaching on process outcomes in health care settings: a systematic review. J Gen Intern Med 2022; 37 (04) 885-899
• 53 Gilkey MB, Heisler-MacKinnon J, Boynton MH, Calo WA, Moss JL, Brewer NT. Impact of brief quality improvement coaching on adolescent HPV vaccination coverage: a pragmatic cluster randomized trial. Cancer Epidemiol Biomarkers Prev 2023; 32 (07) 957-962
• 54 Bennett AV, Dueck AC, Mitchell SA. et al; National Cancer Institute PRO-CTCAE Study Group. Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Health Qual Life Outcomes 2016; 14: 24
• 55 Owen-Smith A, Mayhew M, Leo MC. et al. Automating collection of pain-related patient-reported outcomes to enhance clinical care and research. J Gen Intern Med 2018; 33 (Suppl. 01) 31-37
• 56 Barak-Corren Y, Castro VM, Nock MK. et al. Validation of an electronic health record-based suicide risk prediction modeling approach across multiple health care systems. JAMA Netw Open 2020; 3 (03) e201262
• 57 Walsh CG, Johnson KB, Ripperger M. et al. Prospective validation of an electronic health record-based, real-time suicide risk model. JAMA Netw Open 2021; 4 (03) e211428
• 58 Nock MK, Millner AJ, Ross EL. et al. Prediction of suicide attempts using clinician assessment, patient self-report, and electronic health records. JAMA Netw Open 2022; 5 (01) e2144373
• 59 Haroz EE, Kitchen C, Nestadt PS, Wilcox HC, DeVylder JE, Kharrazi H. Comparing the predictive value of screening to the use of electronic health record data for detecting future suicidal thoughts and behavior in an urban pediatric emergency department: a preliminary analysis. Suicide Life Threat Behav 2021; 51 (06) 1189-1202
• 60 Walker RL, Shortreed SM, Ziebell RA. et al. Evaluation of electronic health record-based suicide risk prediction models on contemporary data. Appl Clin Inform 2021; 12 (04) 778-787
• 61 Henry M, Rosberger Z, Bertrand L. et al. Prevalence and risk factors of suicidal ideation among patients with head and neck cancer: longitudinal study. Otolaryngol Head Neck Surg 2018; 159 (05) 843-852
• 62 Bryan CJ, Allen MH, Thomsen CJ. et al. Improving suicide risk screening to identify the highest risk patients: results from the PRImary Care Screening Methods (PRISM) study. Ann Fam Med 2021; 19 (06) 492-498

Publication History

Received: 12 January 2024

Accepted: 27 March 2024

Article published online:
22 May 2024