Developing Mobile Clinical Decision Support for Nursing Home Staff Assessment of Urinary Tract Infection using Goal-Directed Design

Summary

Background: Unique characteristics of nursing homes (NHs) contribute to high rates of inappropriate antibiotic use for asymptomatic bacteriuria (ASB), a benign condition. A mobile clinical decision support system (CDSS) may support NH staff in differentiating urinary tract infections (UTI) from ASB and reducing antibiotic days.

Objectives: We used Goal-Directed Design to: 1) Characterize information needs for UTI identification and management in NHs; 2) Develop UTI Decide, a mobile CDSS prototype informed by personas and scenarios of use constructed from Aim 1 findings; 3) Evaluate the UTI Decide prototype with NH staff.

Methods: Focus groups were conducted with providers and nurses in NHs in Denver, Colorado (n = 24). Qualitative descriptive analysis was applied to focus group transcripts to identify information needs and themes related to mobile clinical decision support for UTI identification and management. Personas representing typical end users were developed; typical clinical context scenarios were constructed using information needs as goals. Usability testing was performed using cognitive walk-throughs and a think-aloud protocol.

Results: Four information needs were identified including guidance regarding resident assessment; communication with providers; care planning; and urine culture interpretation. Design of a web-based application incorporating a published decision support algorithm for evidence-based UTI diagnoses proceeded with a focus on nursing information needs during resident assessment and communication with providers. Certified nursing assistant (CNA) and registered nurse (RN) personas were constructed in 4 context scenarios with associated key path scenarios. After field testing, a high fidelity prototype of UTI Decide was completed and evaluated by potential end users. Design recommendations and content recommendations were elicited.

Conclusions: Goal-Directed Design informed the development of a mobile CDSS supporting participant-identified information needs for UTI assessment and communication in NHs. Future work will include iterative deployment and evaluation of UTI Decide in NHs to decrease inappropriate use of antibiotics for suspected UTI.

Citation: Jones W, Drake C, Mack D, Reeder B, Trautner B, Wald HL. Developing mobile clinical decision support for nursing home staff assessment of urinary tract infection using goal-directed design.Appl Clin Inform 2017; 8: 632–650 https://doi.org/10.4338/ACI-2016-12-RA-0209

Keywords

Antimicrobial stewardship - urinary tract infections - clinical decision support - goal-directed design - nursing home - communications

1. Background and Significance

Nursing homes (NHs) are facing a new mandate to reduce inappropriate antibiotic use in the face of the rising prevalence of antibiotic-resistant bacteria [[1]]. These antimicrobial stewardship (AMS) efforts are largely adapted from the acute care setting, despite the fact that the NH environment has several unique characteristics that present care challenges [[2]]. First, NH patients are uniquely frail, many with physical and cognitive impairments that interfere with diagnostic evaluation and increase susceptibility to infections. Second, NH providers—physicians, nurse practitioners (NP) and physician assistants (PA)— are infrequently present in the NH at the time of a patient’s presentation with suspected infection. Finally, nurses—registered nurses (RN), licensed practical nurses (LPN), and certified nursing assistants (CNA)—play an enhanced role in resident management and determine the flow of information to providers [[3]].

Among residents of NHs, urinary tract infections (UTIs) are the most commonly reported and treated healthcare-associated infection. A recent study suggests 3.0% to 5.2% of NH residents have a UTI at any point in time [[4]]. Diagnosis of UTIs in the NH setting is complicated by the widespread presence of bacterial colonization of the urinary tract without local signs or symptoms of infection, which is referred to as asymptomatic bacteriuria (ASB) [[5], [6]]. Antimicrobial treatment of ASB is widespread despite evidence that it does not improve outcomes, probably because ASB is both common and challenging to differentiate from symptomatic UTI in frail elders [[7], [8]]. Overtreatment of ASB contributes to antibiotic overuse in the NH setting and the increased incidence of adverse events, including Clostridium difficile infection (CDI) and the acquisition of multi-drug resistant (MDR) organisms [[9]–[11]]. In a recent study, a paper-based algorithm incorporating national guidelines was effective in reducing urine testing and treatment of ASB in a NH setting [[12]].

Health information technology designed to help healthcare professionals with decision making tasks is known as electronic clinical decision support system (eCDSS) [[13]]. eCDSS has been proposed as an important technology to enhance existing AMS programs and the implementation of core AMS strategies [[14]]. With respect to the diagnosis and management of UTI in the NH context, eCDSS has the potential to reduce antibiotic overuse by improving diagnostic accuracy and thereby mitigating overtreatment of ASB. Since the NH sector has been reported to lag in health information technology infrastructure [[15]], smartphone applications may be a promising platform for disseminating an eCDSS intervention directly to individual staff members. The use of any information system in the NH must be considered in the context of the unique workflows in NHs described above.

1.1 Goal-Directed Design

Goal-Directed Design (GDD) is an approach that involves targeted users in the design of systems that will support their goals[16] through integrated use of persona-driven[17]–[20] and scenario-based design [[18], [19], [21], [22]]. GDD represents the progression of the integrated use of these well-known design methods that have been widely used for decades outside the health domain [[23]–[25]]. A persona is a description of a hypothetical individual that typifies a person who will use an information system [[17]]. Personas are used to ground the technology, conceptualizing the idea of an individual stakeholder and envisioning the ways he or she might interact with the information system. Scenario-based design focuses on the activities and information needs of the people who will use an information system rather than the system itself or the capabilities of technology [[21]]. An information need is user perception of a gap in information support or technical resources that enable her to know, learn, or do something better [[26]]. Scenarios of use are narratives that are easily understood by people who perform work in nontechnical domains; they provide a design vocabulary for laypersons involved in the design process [[27]]. Scenarios constitute descriptions of work activities that satisfy the goal for each information need. GDD defines several types of scenarios – two of these being context and key path scenarios. Context scenarios tell the story of a particular persona, its goals, and motivations for using the system under design [[16]]. Key path scenarios describe specific user interactions and steps to navigate the future system [[16]]. In this study, we used GDD to characterize information needs, and develop and test a mobile CDSS prototype (UTIDecide) by engaging NH staff.

2. Objectives

To our knowledge, there is no prior investigation of clinicians’ information needs around AMS activities or the needs of clinicians in long term care when assessing a patient’s change in status. To bridge these identified gaps, this study used GDD to develop an eCDSS, titled UTI Decide, to improve UTI management of suspected UTI in NHs. We sought to: 1) Characterize information needs for management of suspected UTI in the NH context; 2) Develop the UTI Decide prototype informed by personas and scenarios of use constructed from Aim 1 findings; 3) Evaluate the UTI Decide prototype with frontline NH staff using cognitive walk-throughs and a think-aloud protocol.

3. Methods

We conducted a three aim design study that engaged 20 participants from 6 NHs and 2 physician practices in the Denver area. ►[Figure 1] for a broad overview of the steps taken to implement the GDD study in the nursing home context. Inclusion criteria were: frontline staff (RN or CNA), managers (RN) or providers (MD, NP, or PA) at a Denver-area NH currently providing direct care to residents and greater than 21 years of age. All procedures were subject to expedited IRB oversight by the Colorado Multiple Institutional Review Board. Participants provided informed consent prior to enrollment in the study and received an incentive of a $50 gift card for participating in a focus group.

Fig. 1 Study flow diagram

Given the constraints of work responsibilities and schedules, our participants could not engage in a time-intensive design project. Therefore, we chose focus groups to solicit information needs and individual usability sessions to solicit design feedback at a time of participant convenience. Two on-site focus groups allowed participants to demonstrate interactions and identify problems with individual information systems on an ad hoc basis during the sessions. Two telephone focus groups were conducted to allow for flexibility in engaging providers and nurse managers from different sites [[28]]. Based on focus group findings, our team created a conceptual information system design that included scenarios of use and personas.

Our cross-disciplinary design team consisted of a geriatrician, an infectious disease physician, a user-centered design methodologist, a public health PhD student, and two medical students, one of whom has a degree in computer science and served as the programmer. In addition to extensive experience with aging, the health needs of older adults, and antimicrobial stewardship. for UTI, individual team members have expertise in computer science, software engineering and qualitative research methods. In addition, the design team worked with a community stakeholder advisory board and consulted several times with 2 NH providers and a former director of nursing during the design process.

3.1 Aim 1 Methods

Focus Groups: Four focus groups (n=4, 5, 4, 7) were conducted with healthcare professionals who work with older adult residents in NHs. Participants provided verbal consent prior to the start of the focus groups. Participant roles included nurses (RNs), nurse managers, directors of nursing, nurse practitioners (NPs), physicians (MDs), and physician assistants (PAs). Focus group participants were full time staff at a NH or affiliated provider group. Focus group participants were invited to attend by email. Focus groups were conducted from June 2015 to August 2015. An interview guide was developed based on prior work [[29]], field experience, and a literature review. We used the technology acceptance model (TAM) to inform questions about mobile technology use in NHs and to determine information needs associated with UTI diagnosis and management in the NH setting.[30] The interview guide aimed at identifying barriers and facilitators to adoption of eCDSS through exploring thoughts and perspectives on the following themes: knowledge and awareness of guidelines at time of assessment and interpretation of culture results, attitudes about using guidelines, external barriers to and facilitators of eCDSS implementation and acceptance, perceived usefulness of decision support, social norms regarding smartphone use in this setting, and a description of workflow for UTI evaluation, diagnosis, and management. Focus group data collection stopped at theoretical saturation when participant responses became repeated and when data was sufficient to fully develop three domains of the focus group guide.

3.2 Focus Group Analysis

All focus groups were audio-recorded and transcribed verbatim by a research assistant. Any information from which the particular respondent or NH could be identified was removed. Open coding was conducted by two researchers with training in qualitative methods. Focus group transcription was next read for emergent themes. Once emergent themes were consolidated between the two researchers, the text was coded independently for a priori themes from the focus group guide and newly emerged themes derived from the text. The two coders then met to synthesize results; codes were highly overlapped and discordant codes were discussed and agreed upon. Coded transcripts from both researchers were reviewed and compared by the principal investigator to ensure interrater reliability. An iterative analysis was applied to transcripts from focus groups. A member-checking survey to validate identified workflow and information needs was sent to all focus group participants who provided legible e-mails as part of study enrollment materials (n=17, 8 responded). Member checking is a method of re-stating results with original participants to check for accuracy. Survey questions were based on prior design research [[19]] and adapted by the research team for the current study through a series of meetings. Summarized results were triangulated with the expertise of a community stakeholder advisory board.

3.3 Aim 2 Methods

Design Process: The study team conducted the design process collaboratively. To construct the RN and CNA/LPN personas, the study team summarized and aggregated information related to experience, education, and job responsibilities for nursing staff. The team, with input from nursing home professionals, determined that for UTI Decide, an important distinction should be made between nurses licensed to assess patients and nurses not licensed to assess patients. Given that neither CNAs nor LPNs are licensed to assess patients in the state of Colorado, these roles were combined into one persona. This persona will be referred to as a CNA in the remainder of the text. Next, context and key path scenarios were developed using information needs from Aim 1 and participant design recommendations identified from focus groups. The design team chose to focus efforts on information needs of nurse roles (steps 1 and 2 of the UTI diagnostic and management workflow, ►[Figure 2]), deferring a build for the information needs of provider roles for future efforts (steps 3 and 4 of the UTI diagnostic and management,►[Figure 2]). Using the two personas, scenarios were matched to the work of CNAs and RNs.

Mobile Application Development: A UTI Decide prototype was developed by the research team. We first analyzed the data from focus groups to build a clinical algorithm that combined NH UTI workflow with evidence-based guidelines on UTI diagnosis and management in long term care facilities.[31] The algorithm we developed was adapted from a validated paper-based algorithm created to combat common cognitive errors in UTI diagnosis. (See Appendix) Specifically, a previous study showed that use of the algorithm improved inter-rated reliability on distinguishing ASB from UTI, in comparison to providers’ diagnoses made without the algorithm, and demonstrated effectiveness in a clinical trial.[12],[29] The algorithm was then converted into a web-based application developer. For the initial prototype, a web-based platform was chosen to allow access for testing on both mobile and desktop applications. While mobile access of the application is a primary focus of this project, initial design engagements with nursing home participants identified potential concerns, since resolved, related to wireless access network access and use of mobile technologies by nursing home staff. We redesigned the user interface iteratively in a series of design sessions to improve usability and functionality prior to pilot testing.

Fig. 2 UTI Diagnosis and Management Workflow in Nursing Homes

3.4 Aim 3 Methods

Cognitive walk-throughs and think-aloud usability tests

Cognitive walk-throughs and think-aloud protocols are two common usability test methods that are often used in conjunction with each other [[32]]. A cognitive walk-through is a usability test method to identify appropriate goal-action sequences and barriers to completion of specific tasks [[33]]. A think-aloud method is a usability test method where users ‘think aloud’ while using a system’s interface to perform tasks.[34],[35] Landmark studies using mathematical modeling of problem discovery rates in usability studies have presented empirical evidence that most problems are discovered by usability testing with 3 to 5 participants [[36]–[39]]. Experimental tests of think-aloud protocols have resulted in similar findings and recommendations of 3 to 5 participants [[40]].

We conducted usability test sessions with participants (n=4) who held nurse or nurse manager roles at a NH. Two researchers (BR and CD) moderated usability sessions jointly and alternated between primary and secondary interviewer roles. Usability test sessions consisted of cognitive walkthroughs [[33]] using a think-aloud protocol [[34]]. Participants were instructed to use 4 common patient scenarios as they stepped through the UTI Decide interface while “thinking aloud” about their observations during episodes of use. The usability interview guide was based on our prior research [[18], [26]] and the TAM [[30]]. Interview questions were designed to prompt responses about usability of the eCDSS interface. Usability sessions were recorded using a digital audio recorder and transcribed verbatim by a member of the research team (CD).

Usability Transcript Analysis: Usability test session transcripts were analyzed to identify design recommendations and a priori codes (perceived ease of use and perceived usefulness) drawn from the TAM [[30]]. The coding strategy was devised by two coders (CD and BR). Both coders read the transcripts of usability test sessions to familiarize themselves with the raw data and refine the coding approach. One coder (CD) conducted a preliminary analysis of the transcripts and then both coders met to discuss and finalize the coding approach. One coder (CD) subsequently conducted final coding of all transcripts. A second coder (BR) reviewed and verified the final coding results as a test of face validity [[41]]. Coded results were grouped into themes.

Usability test transcripts were transcribed verbatim and read for three primary a priori codes based on interview guide questions: 1) perceived ease of use of the UTI Decide application; 2) perceived usability; and 3) content and design recommendations. Two coders analyzed the usability transcript data and then met to compare coded text and come to a consensus.

4. Results

4.1 Aim 1 Results

We describe the validated workflow and information needs, personas, and scenarios that comprise the conceptual design of UTI Decide. The proposed eCDSS is intended to meet 2 objectives: 1) to support evidence-based RN assessment of NH residents with a change in status suspected to have a UTI, 2) to support interdisciplinary communication of a resident’s change in status to the off-site provider. The current conceptual design is meant to support use that is untethered from other information systems and without archiving assessment results.

4.2 Workflow and Information needs

Analysis of focus group transcripts led to the identification of a UTI diagnostic workflow as well as 4 predominant information needs specific to the work of nurses and providers. Further, we elicited a high-level workflow for patients suspected of having a UTI from focus group participants (►[Figure 2]). The workflow proceeds in 4 steps and is initiated when a resident is noted to have a change in clinical condition. This may be a subtle change such as a change in urine appearance or decreased energy or a major change such as a fever or change in mental status. The episode commences when a change in status is noted in the patient by an RN or CNA. First, a nurse will perform an assessment which may include vital signs, interview, physical examination, and/or visual examination of the urine. Second, the nurse will contact the provider to communicate her clinical impression and obtain orders. Third, the provider and nurse will implement a care plan that may involve monitoring, urine or blood testing, and/or empiric antibiotic therapy. Finally, the resident will be monitored and reassessed over the following 72 hours at which time any laboratory data should be available to support maintaining or changing clinical course. The episode terminates when the patient is stable and any therapy is completed.

4.2.1 Information needs

Four primary information needs were identified. Two of the identified information needs were specific to the work of nurses and two were related to the information needs of providers. Nurses’ information needs related to assessment of the patient and communication with providers. Provider information needs related to communicating with staff and interpreting the results of diagnostic testing. ►[Table 1] shows the information needs identified for each role. Upon member-checking our results with 17 of the focus group participants, agreement with information needs was 100%.

Table 1

Information needs, scenarios, and roles

Information Needs	Scenarios	Roles
		CAN	LPN[*]	RN	Mgr	PA/NP	MD/DO
Access to diagnostic guidance when assessing a patient with a change in status	LPN notes Mr. Y has a change in status. The RN asks her to use the app to remind herself of what data she should collect regarding his vitals and his symptoms.		X	X
Access to communication guidance	RN B is contacting provider Dr. X for Mr. Y’s change in status. She opens the app to guide what to recommend to Dr. X and how to structure the conversation.		X	X	X	X	X
Access to diagnostic and treatment guidance for provider at time of notification	Dr. X is speaking with RN B. He opens the app to determine if the reported symptoms are consistent with UTI. He also looks for guidance regarding empiric treatment recommendations.			X	X	X	X
Access to culture interpretation guidance	RN B calls Dr. X with culture results. He opens the app for advice on culture interpretation and treatment choice and duration.				X	X	X

CNA=certified nursing assistant; LPN=licensed practical nurse, RN=registered nurse, Mgr=Nurse Manager; PA/NP = physician assistant/nurse practitioner, MD/DO=medical doctor/doctor of osteopathy *LPN role differs depending on regulatory environment across states and regions. For instance, in the state of Colorado, LPNs are not licensed for resident assessment.

1. Access to diagnostic guidance when assessing a patient with a change in status

When assessing a patient with a change in status, frontline nursing staff could benefit from improved access to UTI evaluation guidance. At the time of presentation, participants reported that guideline-discordant signs and symptoms, such as behavior changes, falls, confusion, and foulsmelling urine, are often mistakenly thought to indicate a possible UTI. As one focus group participant noted:

“A lot of them will call because this person has had multiple falls, therefore we want to check them out for this, this, and this…I think it’s just the easiest thing to rule out, so first thing they look for.”

Participants expressed interest in technology that could support users on what to assess for and what findings are consistent with a UTI.

2. Access to communication guidance

Communication as a workflow barrier emerged as a key theme in focus groups. Nursing staff described frequent difficulty in communicating information to providers about patients with a suspected UTI due to inadequate documentation or uncertainty about what information needs to be provided. Similarly, providers reported that the majority of their UTI cases are phone consultations and noted that healthcare professionals often have not gathered sufficient information before calling them. One focus group participant observed:

“I think the biggest problem…is knowing what clinical information is being conveyed to you that would really give an indication to treat and having a reliance on it. It’s not usual to get a call where the data you would expect is present.”

Providers reported that this communication barrier is worsened by difficulty obtaining patient information from charts given that many NHs do not have electronic healthcare record systems.

3. Access to diagnostic and treatment guidance for provider at time of notification

The third information need we identified was improved access to diagnostic and treatment guidance for providers at time of notification of a possible UTI. Participants agreed that improved access to diagnostic guidance for UTI is needed. Most participants were familiar with evidence-based guidelines for UTI, and a few participants noted that they routinely reference evidence-based guidelines to inform decision making, including the Infectious Diseases Society of America guidelines,[42] the Sanford Guide,[43] and the revised McGeer Criteria.[31] However, several participants noted that many of their colleagues are largely unaware of guideline recommendations, a perspective shared by both nurses and providers. As a result, testing and treatment for UTI are frequently ordered in spite of clinical findings that do not indicate testing or treatment according to evidence-based guidelines. This quote by a focus group participant illustrates this point:

“…we’ve been doing this for years: ordering unnecessary UAs [urinalyses] and treating things that are not really infection for years and years and years. There’s a roadblock…This is a learning curve and we’re all just still learning..”

Additionally, participants noted that there are significant discrepancies among NH guidelines for diagnosing and managing UTIs and expressed interest in improved access to standardized, evidence-based guidelines.

4. Access to urine culture interpretation guidance

Finally, we identified interpretation of urine cultures as an additional information need in the NH context. Several participants stated that they follow guideline recommendations when reviewing microbiology data and deciding on antibiotic therapy. However, many participants noted that guideline-discordant decision making is prevalent in the NH setting and that antibiotics are frequently prescribed despite limited or absent culture data. To illustrate, one focus group participant stated:

“I find that our medical director quite often treats rather than being systematic about it. Some people come back positive and they may be fine, or appear fine, and she treats them anyway. I have a few providers who do that as well.”

Participants expressed a desire for improved access to guideline recommendations when reviewing urine culture data.

4.3 Personas

We constructed two personas: one for an RN persona – “Denise” and another for a certified nursing assistant (CNA) persona – “Marie”-, both of which are shown in ►[Figure 3]. It is important to note that the CNA is generally not licensed to perform assessment; rather, she can gather data for the nurse (such as vital signs, etc.). Further, the CNA will generally not be directly communicating with the provider. In contrast, the RN will be responsible for the assessment and communicating findings from the assessment to the provider.

Fig. 3 “Denise” RN Persona and “Marie” CNA Persona

4.4 Scenarios of Use

Four context scenarios were constructed that describe user tasks using identified information needs as goals for a patient with or without an indwelling urinary catheter and with or without guideline concordant UTI symptoms. We chose the first because the guidelines differ based on catheterization and the second because the eCDSS is designed to change assessment behavior when patients lack guideline concordant UTI symptoms. ►[Table 1] shows the scenarios, the information need each scenario supports, and the persona that engages in the work activity described by the scenario. Of interest, the LPN role differs depending on regulatory environment across states and regions. For instance, in the state of Colorado, LPNs are not licensed for resident assessment.

In addition, four corresponding key path scenarios that described a successful path for task completion using UTI Decide were constructed (►[Figure 4]). Scenario construction and development of prototype interfaces and features occurred iteratively with the two processes informing each other.

Fig. 4 Key Path Scenarios

4.5 Aim 2 Results: Prototype Design

The objective of aim 2 was to develop a UTI Decide prototype informed by personas and context scenarios constructed from Aim 1 findings. The interface and content of the prototype was designed to satisfy the information needs of nurses involved in the identified UTI management workflow in steps 1 and 2 of their interaction with the eCDSS. Information needs of steps 3 and 4 were not addressed in this prototype. ►[Figure 5] shows screen captures of the prototype that resulted from the design process illustrated in ►[Figure 1].

Fig. 5 Screen Shots of High-fidelity Prototype

The UTI Decide prototype is structured as an electronic decision tree designed to guide the user to an evidence-based assessment of suspected UTI. It was built on a web-based platform to provide flexibility to potential uses in NHs with limited wireless internet or cellular phone access. The user is initially asked whether the patient is catheterized or non-catheterized. The user is then prompted to evaluate and record signs and symptoms of UTI. If the signs and/or symptoms selected are not guideline concordant, the eCDSS suggests not moving forward with the UTI workup and suggests consideration of non-urinary conditions. If the signs and/or symptoms are guideline discordant, the eCDSS suggests additional conditions to consider before moving forward with the UTI workup. Thus, guideline recommendations are incorporated into the eCDSS at various points in the decision tree.

The UTI Decide prototype includes additional features intended to educate users about evidence-based management and diagnosis of UTI. When the user selects signs or symptoms that are not diagnostic of UTI, the eCDSS explains this to the user and suggests that further testing for UTI may not be required at this time. Further, the eCDSS suggests additional steps, such as observation or alternative diagnoses, per the decision support tool. Additionally, the eCDSS includes pop-ups describing common points of confusion in the UTI diagnostic and management workflow.

To mitigate the identified communication barrier between NH staff and providers, UTI Decide includes a feature that structures previously entered data into text in the SBAR (Situation, Background, Assessment, Recommendation) format for the user to use when communicating with a provider.[44] SBAR is a standardized communication approach championed by the Agency for Healthcare Policy and Research as a means to promote patient safety. SBAR is widely taught to nurses.[44],[45] The user is given the option of generating an SBAR script relevant to the case, and if the user selects yes, data previously entered by the user is structured into the SBAR format. This feature is intended to suggest and synthesize all information the provider will likely need to evaluate the patient during the nurse-provider phone call.

4.6 Aim 3 Results: Prototype evaluation

Generally, participants found the application easy to use and useful. Participants were able to easily and quickly navigate the application to evaluate the scenarios provided to them. Each of the four participants reached the desired outcome of the eCDSS: an evidence based assessment of whether the patient in the scenario had symptoms concordant or non-concordant with UTI. Also, each participant reached the desired SBAR output.

Participants found the application easy to use:

“…what I like about this is that it’s accessible. It’s more accessible than you having to write down your assessment and then go to the computer. If I had the phone on me, while I was assessing, I could just jot it down there. I think that’s nice and user-friendly, in that sense.” (Usability Participant 2)

Participants also identified the usefulness of UTI Decide:

“This is data collecting. You’re not saying “per your assessment, make these recommendations”; it’s basically saying “this is the evidence based recommendations”. They can start those. They can collect data. They can start those. [sic] And getting them to start that as quickly as possible—that may be the piece most beneficial in the long run.” (Usability Participant 1)

and:

“Yes. It helps you. Because it’s what information you need before you call the doctor.”(Usability Participant 4)

4.7 Design recommendations

Participants had several design recommendations. The first was providing a zoom feature:

“For me, I like color, font. Clean. It may want to have a zoomable feature;” (Usability Participant 3).

The second was providing a printable or email-ready report feature. This recommendation was made by two of the four participants who wanted to be able to easily print from the application to include the output of the suggestions/results in the residents’ charts.

“I guess saving it as a PDF file and emailing it to yourself. It’d be a good option. I would rather do that than double my work and type it all in again.” (Usability Participant 2)

Both participants felt that printing the results was a key point in using UTI Decide as a communication tool among frontline staff and providers.

Additional design recommendations included removing the highlighting feature from the list of symptoms, making the scroll bar more obvious and prominent, making the “none of the above” option more obvious and prominent, and moving up the “next” button on the first screen,

4.8 Content recommendations

There were several content-related recommendations. An important piece of feedback was that changing the non-concordant suggestions format from “have you considered?” to “have you done?” would increase data collection by the user (nurse) and ensure adequate data is collected before communicating with the decision-maker (provider). For example, one informant noted that “have you considered gas pains?” could be made more active by changing the suggestion to “when was the resident’s last bowel movement?”, a piece of information helpful to those diagnosing UTIs and prescribing antibiotics.

A general piece of feedback we received was to keep the language simple and provide additional “pop up” content in the informational “i” icon for each sign and symptom as a means of providing context:

“Maybe being able to get information on all of them [the symptoms] rather than just some. I think it’s great that for ‘change in mental status’, you can click the information and it shows you ‘acute onset’… and maybe more information on those might be good.” (Usability Participant 2)

One participant wanted to include white blood cell (WBC) count workflow in application; evaluation of WBC count is an aspect of the CDC guidance for UTI surveillance in long term care [[31]].

“It’s good. I still want to see—still want to chase that white count down.” (Usability Participant 3)

5. Discussion

Our study confirmed the need for information technology that supports communication and decision making at multiple points in the UTI workflow in the NH context. Examples of activities that could benefit from eCDSS include assessment of a patient with a change in status suspected due to UTI, communication between frontline staff and providers, diagnosis and treatment for providers at time of notification of possible UTI, and culture interpretation. The findings from this study describe commonalities in information needs across NHs. To our knowledge, our study is the first to incorporate these information needs into the conceptual design of an eCDSS.

The benefit of GDD is that, in characterizing the UTI management and diagnostic workflow in this context, we have identified important points of intervention where decision support could improve diagnostic accuracy and management efficiency. Another benefit of this approach is the development of an informatics tool that incorporates the perspective and preferences of all stakeholders. While scenario-based and person-driven design have been used in prior health-related research in recent years [[18]–[21], [26], [27]], this novel application of GDD for mobile clinical decision support in the nursing home context represents a natural progression in the use of integrated design methodology.

This study is limited by the relatively low number of participants in focus groups, which may limit external validity. Furthermore, selection bias is possible that given that those healthcare professionals interested in technology and improving UTI management were more likely to attend focus groups. To simplify design, we limited the scope to the first two identified information needs of assessing nurses. It is not known if changing nursing behavior alone can change urine testing and reduce antibiotic use. Further development work on the 3^rd and 4^th information needs for providers will be an important next step. Additionally, understanding how RNs and providers use guidelines in daily practice would impact patients and would also be of importance in future work. Finally, nursing home residents and family caregivers can be important partners in prescribing decisions. Future development might engage these important stakeholders.

Attempts to implement the application developed from this work will be limited unless workflow of assessment in NHs is better understood. The current understanding of work in NHs is heavily skewed to medication administration and communication [[46]]. Further, the categorization of nurses licensed to assess residents versus those not licensed to assess residents will require translation across health care regions and regulatory contexts.

6. Conclusions

This study demonstrates the potential of Goal-Directed Design for the development of eCDSS for AMS. The documented information needs from this study contribute to the general knowledge of UTI diagnosis and management in the NH context, as well as to the larger issues of communication barriers between NHs and providers and the optimal design of eCDSS in this context. Resolving information needs in this workflow through eCDSS has the potential to mitigate antibiotic overuse and improve diagnostic accuracy.

Larger studies that include more participants and a greater number of study sites are needed to better characterize information needs and to optimize design of eCDSS. Decision support in nursing homes has great potential to impact the behaviors of a diverse and dynamic workforce. Future eCDSS might be developed to impact prescribing for other HAIs, or assessment for other healthcare associated infections in this setting and others (e.g. emergency departments). Further research will enable meaningful decision support and improved effectiveness of antimicrobial prescribing while maintaining efficiency and provider autonomy.

Multiple Choice Questions

1. When employing Goal-Directed Design to develop a mobile clinical decision support application that implements evidence-based guidelines for assessing a clinical condition, which of the following users are most important to engage:

• Researchers

• Administrators

• Clinicians

• Technology Vendors

The correct answer is C. Clinicians will most directly use the mobile clinical decision support application. Thus, it is most important to identify their information needs and preferences in the context where the deliver care. While researchers, administrators and technology vendors all may have a stake or role to play in development and use of the application, clinicians should be the primary focus of applications that implement evidence-based guidelines at the point of care. In addition, nursing home residents or patients and their family members may benefit from the information captured and recorded by these applications.

2. When designing clinical decision support for nursing home staff, which of the following steps in workflow can be structured around the SBAR (Situation, Background, Assessment, and Recommendation) framework?

• Assessment

• Communication

• Care Planning

• Monitoring

The correct answer is B. The SBAR framework is used to address common errors in communication in workplaces requiring high reliability. It has been widely adopted in healthcare to structure clinical communication among clinicians. In the NH context, there are many barriers to effective communication, making a structured format particularly valuable. In particular, the medical providers are commonly absent from the NH and entirely reliant on the nurses for “situational awareness”, or the understanding of the entirely of a clinical scenario. The SBAR framework is used only after an initial assessment of the resident is completed so that the nurse can share her findings with the provider. The assessment step can be informed by clinical algorithms such as that incorporated in UTI Decide. The care planning step by the provider and nurse occurs after the communication of the relevant clinical facts and can be informed by clinical guidelines such as those outlining appropriate antimicrobial therapy. Finally, a period of monitoring follows, after which a subsequent SBAR communication may occur if the clinical scenario has not been resolved.

Conflict of Interest

The authors declare that they have no conflicts of interest in the research.

Acknowledgments

Greg Gahm MD and the clinicians who participated in our focus groups and usability tests.

Clinical Relevance Statement

This study indicates that healthcare providers in the NH context are eager to be engaged in the design, development, and evaluation of information systems that support their work and decision making. Furthermore, results suggest that eCDSS could be successfully adopted in this context and integrated into the existing UTI diagnostic and management workflow in NHs.

Human Subjects Protections

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the Colorado Multiple Institutional Review Board.

• References

• 1 Centers for Medicare & Medicaid Services (CMS), HHS. ,,Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities. Final rule.“ Federal register 2016; 81 (192) 68688.
• 2 Morrill HJ, Caffrey AR, Jump RL, Dosa D, LaPlante KL. Antimicrobial stewardship in long-term care facilities: A call to action. J Am Med Dir Assoc 2016; 17 (02) 183-e1.
• 3 Crnich CJ, Jump R, Trautner B, Sloane PD, Mody L. Optimizing antibiotic stewardship in nursing homes: a narrative review and recommendations for improvement. Drug & Aging 2015; 32 (09) 699-716.
• 4 Dwyer LL, Harris Kojetin LD, Valverde RH, Frazier JM, Simon AE, Stone ND, Thompson ND. Infections in long term care populations in the United States. J Am Geriatr Soc 2013; 61 (03) 341-349.
• 5 Nicolle LE. Asymptomatic bacteriuria in the elderly. Infect Dis Clin North Am 1997; 11 (03) 647-662.
• 6 Nicolle LE. Urinary tract infections in long-term care facilities. Infect Control Hosp Epidemiol 1993; 14 (04) 220-225.
• 7 Nicolle LE, Mayhew WJ, Bryan L. Prospective randomized comparison of therapy and no therapy for asymptomatic bacteriuria in institutionalized elderly women. Am J Med 1987; 83 (01) 27-33.
• 8 Abrutyn E, Mossey J, Berlin JA, Boscia J, Levison M, Pitsakis P, Kaye D. Does asymptomatic bacteriuria predict mortality and does antimicrobial treatment reduce mortality in elderly ambulatory women?. Ann Intern Med 1994; 120 (10) 827-833.
• 9 Courjon J, Pulcini C, Cua E, Risso K, Guillouet F, Bernard E, Roger PM. Antibiotics-related adverse events in the infectious diseases department of a French teaching hospital: a prospective study. Eur J Clin Micro-biol Infect Dis 2013; 32 (12) 1611-1616.
• 10 Werner NL, Hecker MT, Sethi AK, Donskey CJ. Unnecessary use of fluoroquinolone antibiotics in hospitalized patients. BMC Infect Dis 2011; 11 (01) 1.
• 11 Shlaes DM, Gerding DN, John JF, Craig WA, Bornstein DL, Duncan RA, Eckman MR, Farrer WE, Greene WH, Lorian V, Levy S. Society for Healthcare Epidemiology of America and Infectious Diseases Society of America Joint Committee on the Prevention of Antimicrobial Resistance: Guidelines for the prevention of antimicrobial resistance in hospitals. Clin Infect Dis 1997; 25 (03) 584-599.
• 12 Trautner BW, Grigoryan L, Petersen NJ, Hysong S, Cadena J, Patterson JE, Naik AD. Antimicrobial Stewardship Approach for Urinary Catheter-Associated Asymptomatic Bacteriuria. JAMA Intern Med 2015; 175 (07) 1120-1127.
• 13 Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, Rigon G, Vaona A, Ruggiero F, Mangia M, Iorio A. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health Res 2014; 104 (12) e12-e22.
• 14 Forrest GN, Van Schooneveld TC, Kullar R, Schulz LT, Duong P, Postelnick M. Use of electronic health records and clinical decision support systems for antimicrobial stewardship. Clin Infect Dis 2014; 59 (03) S122-S133.
• 15 Abramson EL, McGinnis S, Moore J, Kaushal R. A statewide assessment of electronic health record adoption and health information exchange among nursing homes. Health Serv Res 2014; 49 1pt2 361-372.
• 16 Cooper A, Reimann R, Cronin D, Noessel C. About face: the essentials of interaction design. John Wiley & Sons; 2014
• 17 Pruitt J, Grudin J. Personas: practice and theory. Paper presented at: Conference on Designing for user experiences; 2003
• 18 Turner AM, Reeder B, Ramey J. Scenarios, personas and user stories: User-centered evidence-based design representations of communicable disease investigations. J Biomed Inform 2013; 46 (04) 575-584.
• 19 Reeder B, Turner AM. Scenario-based design: A method for connecting information system design with public health operations and emergency management. J Biomed Inform 2011; 44 (06) 978-988.
• 20 Reeder B, Zaslavksy O, Wilamowska KM, Demiris G, Thompson HJ. Modeling the oldest old: personas to design technology-based solutions for older adults. Paper presented at: AMIA Annual Symposium Proceedings; 2011
• 21 Carroll JM. Making use: scenario-based design of human-computer interactions. MIT press; 2000
• 22 Reeder B, Demiris G. Building the PHARAOH framework using scenario-based design: a set of pandemic decision-making scenarios for continuity of operations in a large municipal public health agency. J Med Syst 2010; 34 (04) 735-739.
• 23 Carroll JM, Rosson MB. Getting around the task-artifact cycle: How to make claims and design by Scenario. ACM Trans Inf Syst 1992; 10 (02) 181.
• 24 Carroll JM. Scenario-based design: envisioning work and technology in system development. New York: Wiley; 1995
• 25 Cooper A. The inmates are running the asylum. Indianopolis, IN: Sams; 1999
• 26 Reeder B, Hills RA, Turner AM, Demiris G. Participatory design of an integrated information system design to support public health nurses and nurse managers. Public Health Nurs 2014; 31 (02) 183-192.
• 27 Filippidou D. Designing with scenarios: a critical review of current research and practice. REQUIR ENG 1998; 3 (01) 1-22.
• 28 Flynn KE, Hahn CL, Kramer JM, Check DK, Dombeck CB, Bang S, Perlmutter J, Khin-Maung-Gyi FA, Weinfurt KP. Using central IRBs for multicenter clinical trials in the United States. PLoS One 2013; 8 (01) e54999.
• 29 Trautner BW, Bhimani RD, Amspoker AB, Hysong SJ, Garza A, Kelly PA, Payne VL, Naik AD. Development and validation of an algorithm to recalibrate mental models and reduce diagnostic errors associated with catheter-associated bacteriuria. BMC Med Inform Decis Mak 2013; 13: 48.
• 30 Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989: 319-340.
• 31 Stone ND, Ashraf MS, Calder J, Crnich CJ, Crossley K, Drinka PJ, Gould CV, Juthani-Mehta M, Lauten-bach E, Loeb M, MacCannell T. Surveillance definitions of infections in long-term care facilities: revisiting the McGeer criteria. Infect Control Hosp Epidemiol 2012; 33 (10) 965-977.
• 32 Yen P-Y, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19 (03) 413-422.
• 33 Kaufman DR, Patel VL, Hilliman C, Morin PC, Pevzner J, Weinstock RS, Goland R, Shea S, Starren J. Usability in the real world: assessing medical information technologies in patients’ homes. J Biomed Inform 2003; 36 (01) 45-60.
• 34 Jaspers MW, Steen T, van Den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform 2004; 73 (11) 781-795.
• 35 Turner AM, Reeder B, Wallace JC. A resource management tool for public health continuity of operations during disasters. Disaster Med Public Health Prep 2013; 7 (02) 146-152.
• 36 Turner CW, Lewis JR, Nielsen J. Determining usability test sample size. International encyclopedia of ergonomics and human factors 2006; 3 (02) 3084-3088.
• 37 Virzi RA. Refining the test phase of usability evaluation: How many subjects is enough?. Hum Factors 1992; 34 (04) 457-468.
• 38 Lewis JR. Sample sizes for usability studies: Additional considerations. Hum Factors 1994; 36 (02) 368-378.
• 39 Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. In: Proceedings of the INTERACT‘93 and CHI‘93 conference on Human factors in computing systems. 1993 May 1 (pp. 206–213). ACM.
• 40 Nielsen J. Estimating the number of subjects needed for a thinking aloud test. International journal of human-computer studies 1994; 41 (03) 385-397.
• 41 Boyatzis RE. Transforming qualitative information: Thematic analysis and code development. Sage; 1998
• 42 Nicolle LE, Bradley S, Colgan R, Rice JC, Schaeffer A, Hooton TM. Infectious Diseases Society of America guidelines for the diagnosis and treatment of asymptomatic bacteriuria in adults. Clin Infect Dis 2005: 643-54.
• 43 Gilbert DN, Chambers HF, Eliopoulos GM, Saag MS, Pavia AT, Black D. The Sanford Guide to Antimicrobial Therapy 2016. Sperryville, VA: Antimocribal Therapy; 2016
• 44 Haig KM, Sutton S, Whittington J. SBAR: a shared mental model for improving communication between clinicians. Jt Comm J Qual Saf 2006; 32 (03) 167-175.
• 45 Agency for Healthcare Research and Quality. Communicating a change in a resident‘s condition. Rockville, MD.
• 46 Tariq A, Gerogiou A, Westbrook J. Enabling seamless information exchange: Lessons from implementation of electronic medication administration records in residential aged care facilities. Proceedings 19th Triennial Congress of the IEA. Melbourne, 9–14 August, 2015