Core Components for a Clinically Integrated mHealth App for Asthma Symptom Monitoring

Abstract

Background mHealth apps may be useful tools for supporting chronic disease management.

Objective Our aim was to apply user-centered design principles to efficiently identify core components for an mHealth-based asthma symptom–monitoring intervention using patient-reported outcomes (PROs).

Methods We iteratively combined principles of qualitative research, user-centered design, and “gamification” to understand patients' and providers' needs, develop and refine intervention components, develop prototypes, and create a usable mobile app to integrate with clinical workflows. We identified anticipated benefits and burdens for stakeholders.

Results We conducted 19 individual design sessions with nine adult patients and seven clinicians from an academic medical center (some were included multiple times). We identified four core intervention components: (1) Invitation—patients are invited by their physicians. (2) Symptom checks—patients receive weekly five-item questionnaires via the app with 48 hours to respond. Depending on symptoms, patients may be given the option to request a call from a nurse or receive one automatically. (3) Patient review—in the app, patients can view their self-reported data graphically. (4) In-person visit—physicians have access to patient-reported symptoms in the electronic health record (EHR) where they can review them before in-person visits. As there is currently no location in the EHR where physicians would consistently notice these data, recording a recent note was the best option. Benefits to patients may include helping decide when to call their provider and facilitating shared decision making. Benefits to providers may include saving time discussing symptoms. Provider organizations may need to pay nurses extra, but those costs may be offset by reduced visits and hospitalizations.

Conclusion Recent systematic reviews show inconsistent outcomes and little insight into functionalities required for mHealth asthma interventions, highlighting the need for systematic intervention design. We identified specific features for adoption and engagement that meet the stated needs of users for asthma symptom monitoring.

Keywords

mHealth - symptom monitoring - provider workflows - electronic health record

Background and Significance

Mobile health applications (mHealth apps) have the potential to improve chronic condition management and enhance patient–provider interactions between visits. More than three-quarters of Americans currently own smartphones and rates of smartphone ownership are rising among older adults and people with low household incomes.[1] [2] However, most patients' interaction with providers between visits are not affected by mHealth apps, except for patients' ability to view their medical information and send secure messages to their providers through mobile patient portals. Of the more than 165,000 mHealth apps available, many have low usability, do not provide clinical utility, have minimal uptake, or are abandoned soon after first use.[1] [3] Few are designed to be integrated into clinical workflows, even those that are highly rated. Little is known about how to develop mHealth functionality that will not only provide clinical utility for chronic condition management but also will be adopted and used by patients and providers. Identifying such mHealth functionality represents a key challenge for informatics.

To begin to address this challenge, we developed an mHealth intervention for asthma. Asthma is a chronic condition that affects more than 25 million individuals in the United States, 300 million worldwide, and its incidence is increasing.[4] [5] Uncontrolled asthma interferes substantially in patients' daily lives and often results in patients requiring emergency medical services and hospitalizations. It is estimated that 1.75 million asthma-related visits to U.S. emergency departments occur each year (9 visits for every 100 patients with asthma), and they are disproportionately common among minorities and those of lower socioeconomic status.[6] mHealth apps have the potential to help patients conveniently track symptoms and facilitate interaction with providers. Current clinical guidelines recommend clinicians adjust treatment based on frequent monitoring of patients' symptoms.[7] [8] These guidelines are based on considerable evidence showing that asthma control can be largely achieved for patients in controlled trial settings when patients' symptoms are routinely monitored by clinicians.[9] However, in real-life settings, such intensive symptom monitoring does not happen routinely.[10] [11] Compared with other diseases such as diabetes and hypertension, which can be monitored with objective measures (hemoglobin A1c and blood pressure, respectively), measuring symptom frequency and severity for asthma and other chronic diseases has proved more challenging.

Two recent systematic reviews identified 12 randomized controlled trials (RCTs) of mHealth apps[12] [13] and other digital approaches to facilitate asthma intervention such as web-based tools[14] and text messaging.[15] Both reviews came to similar conclusions: interventions were complex and involved multiple components, and clinical effectiveness was inconsistent. Intervention components included symptom monitoring, asthma action plans, educational materials, games and quizzes, team communication tools, electronic diaries, decision support for physicians, peak flow measurements, and medication use.[13] [16] The reviews urged future research to better understand the contribution of intervention components to outcomes and factors that influence adoption and continued usage. Furthermore, in our informal review of the 12 RCTs, we found two additional limitations: none described a development approach using principles of user-centered design, and none explicitly considered sustainable integration into providers' routine workflows in developing the intervention.

To address these limitations, we attempted to develop an intervention “from the ground up” starting with identifying the core components for implementing asthma symptom monitoring via mHealth. Although most previous efforts developed multicomponent interventions without iterative user testing, our development approach focused only on symptom monitoring, engaged users at every step, and attempted to identify only the core components that meet the needs of users. This approach is similar to developing a minimum viable product,[17] a concept that is commonly used to guide software development in industry, but not often used in research.[18] Our intention is that the intervention we design will serve as a building block for further incremental development and feasibility testing of additional components, so that we can develop more complex interventions systematically with an understanding of how each component contributes. These findings may be applicable to other chronic conditions, thereby shortening development cycles and improving understanding of trial results.

Summary of Intervention

We designed a patient-facing mHealth app that runs on iOS and Android, a web-based clinician dashboard for use by care managers and physicians, and the workflows needed for implementation in routine care. Weekly notifications are sent to the patient app to complete questionnaires regarding asthma-related symptoms, and results are then available for review on the dashboard by care managers and physicians. The dashboard can also output summary reports to be added to the electronic health record (EHR) as a note prior to patient visit.

Objective

We attempted to identify the core components for an asthma symptom–monitoring intervention in an efficient manner using user-centered design methods. We defined the following criteria as design goals:

• Patients would use it. Specifically, a substantial portion of asthma patients would complete a questionnaire about their symptoms on a weekly basis.

• Clinicians would find it clinically beneficial and low burden. Specifically, the intervention would require only minimal changes to workflows.

We selected these proximate objectives for the purposes of design because they are necessary conditions for ultimately achieving improvements in health outcomes and utilization, which we plan to test in subsequent work. We attempted to reach saturation in intervention components defined as those that users considered as the minimum requirements to achieve the aforementioned criteria and would be feasible to develop with modest resources.

Methods

Setting

We recruited physicians and licensed practical nurses (LPNs) from clinical practices at two different sites within Brigham and Women's Hospital, an academic medical center in Boston, Massachusetts, from May 2015 through December 2016. We recruited patients for design sessions (which took the form of individual semistructured interviews) identified by one physician researcher (C.H.F.). We selected patients who were diagnosed with asthma and owned smartphones. We attempted to include ethnically diverse patients and a range of ages and income levels. While primary care is the setting in which most asthma patients are treated, for our intervention development, we chose clinics that specialized in pulmonary or allergic diseases to facilitate recruitment of physicians and diverse patients with difficulty controlling their asthma.

Clinical Guidelines and Rationale

Asthma symptom–monitoring requires periodic assessments using validated instruments.[7] [8] Following our goal of a minimum intervention, we selected a five-question asthma assessment instrument: the asthma control measure (ACM).[19] The ACM is a patient-reported outcome measure (PROM), has been shown to be reliable and valid, and has similar characteristics to the more widely used Asthma Control Test,[20] but has the additional advantage of not requiring the purchase of a license thereby improving scalability. Though guidelines suggest asthma control should be assessed every 2 to 4 weeks,[9] we chose weekly assessments because many controlled trials in which patients' asthma were successfully controlled involved weekly assessments, and less frequent monitoring would likely not be reinforcing. (ACM questions are listed in Appendix A).

Theoretical Frameworks and Concepts

Our development work is supported by the Health Belief Model (HBM).[21] [22] The HBM is a cognitive theory for understanding individual health-related behaviors and characterizes how individuals come to engage in preventative care management of their conditions. It provides a theoretical foundation for other asthma interventions.[23] The HBM argues that several factors influence an individual's propensity to engage in beneficial health behavior, including general health motivations, perceived threat of illness, and perceived benefits and barriers to compliant behavior. Our work focuses on the latter category. Through the design process, we attempted to maximize benefits to patients of symptom monitoring while minimizing barriers and burdens.

To help maximize benefits, we also adapted concepts from “gamification.” Gamification is the use of game design elements in non-game contexts.[24] [25] [26] We assessed individual game design elements from the published literature for ideas to enhance and support our core objectives, workflows, and functionality in a way that creates additional benefit for users, and tested these ideas with users during design sessions.

User-Centered Design and Development Process

We applied principles of user-centered design and qualitative research through individual design sessions with a diverse sample of end users.[27] Because the major benefits of technology arise when work processes are altered to take advantage of the technology's potential, we developed technology and workflows concurrently.[28] [29] We included a patient representative on the research team with whom we consulted about overall design before each development stage and on an ad hoc basis when design questions arose (e.g., when we received conflicting feedback from users). We also alternated our design sessions with patients and with clinicians so that we would be able to develop an intervention that incorporated both perspectives. We applied lessons learned from one investigator's (E.Z.) prior work developing a platform for collecting PROMs. We also conducted a heuristic evaluation involving three usability experts using Nielsen's heuristics[30] (see Appendix B).

The development process proceeded in four broad phases ([Table 1]). Although the steps largely occurred in sequence, at times we needed to return to a previous phase (e.g., return to low-fidelity mockups to clarify and improve navigation after app software development had begun). Within each phase, we attempted to reach saturation in desired intervention components before advancing to the next phase.[31] For example, we continued to conduct design sessions in phase 1 (workflow refinement) until we found that the users generally agreed with our proposed workflows and did not require changes in our high-level workflow or functionality specifications. In keeping with the intention of identifying core components, components that were viewed as essential to achieve our objectives were included; components that were desired but not essential were deferred to subsequent versions (see future enhancements later); and components that were strongly thought to support objectives but not clearly essential were included if they were feasible to develop and users believed they had minimal chance of detracting from other essential components (e.g., smiley face for completed questionnaires).

Table 1 Workflow and technology development steps

Development phase	Workflow	Technology
1. Workflow refinement	Defined workflow roles and responsibilities	Described functionality
2. Intervention outline	Outlined workflow diagram (Microsoft Visio)	Developed low-fidelity prototype (Balsamiq)
3. Intervention detail	Detailed workflow diagram (Microsoft Visio)	Developed high-fidelity prototype (Invision)
4. Intervention final	Finalized workflows	Complete app front end and back end developed

In phase 1, we developed our first workflows and refined them through semistructured interviews with the goal of understanding users' needs and motivations. We used an interview guide and asked patient-users questions about general experiences with asthma, motivation for engaging in symptom monitoring, motivation and barriers to answering weekly questions related to asthma symptoms, desired methods and timing for facilitating contact with clinicians as a result of symptom scores, impact of symptom monitoring on relationship with clinician, and the potential role of symptom scores during in-person visits. Physician questions included invitation processes, types of patients most likely to benefit from symptom monitoring, utility of weekly patient-reported symptom scores, score thresholds needed for generating notifications, methods for establishing contact between patient and clinician, and workflow changes that would maximize utility of symptom data. For both patients and physicians, we showed them the five-question ACM for reference.

In phase 2's design sessions, we used updated interview guides, described our proposed intervention, and developed workflow diagrams and low-fidelity mockups (sketches to illustrate rough ideas) of the user interfaces. We asked users to use the “think aloud” protocol when reviewing the mockups.[32] We asked more detailed questions about the topics discussed in phase 1. For example, we asked patients about how they would like to see their own self-reported data, from whom they would like to receive a call if symptoms are worsening, and their reaction to a smiley face as a reward for completing the weekly questionnaire (concept borrowed from gamification). For physicians, we asked about the process for handling notifications and making the symptom data available during in-person visits. For care managers, we asked LPNs about additional time requirements to handle patient-generated notifications and aspects of the notification logic.

As we began to define our functionality, we assessed the technical feasibility of implementing this intervention using our clinical sites' existing EHR. We found that the EHR supported some but not all of our required features; so, we contracted with a third-party app development company (ADK Group, Boston, Massachusetts, United States) to help develop high-fidelity prototypes and to write the software.

In phase 3 design sessions, we user-tested high-fidelity prototypes (close to a final product) with patients and clinicians, with a focus on clarity of wording, look-and-feel, and navigation. To ensure our design followed best practices for smartphone apps, we reviewed the design with two experts in smartphone user interfaces from Apple, Inc. We also continued to enhance the workflows by providing more detail of the steps and options. In phase 4, we finalized the workflows and worked with our contractor as they wrote the software, and conducted a summative heuristic evaluation with three evaluators.

One researcher (R.S.R.) led all design sessions, while another (Z.P. or K.K.) asked additional questions and took notes. Both reviewed the notes afterward for accuracy and resolved differences. The interviews were audio recorded unless the participant declined. At the conclusion of each session, one researcher summarized key findings and another reviewed. Differences were resolved by listening to the recordings. Patients were given a $25 incentive per design session. The institutional review boards of RAND and Partners Healthcare approved this work.

Results

We conducted 19 individual interviews or design sessions with patients (n = 9, ages 21–74 years, 6 females, 2 African Americans, 1 Latino, 2 low-income, 2 section-8 housing residents) and clinicians (n = 7, 5 physicians, 2 nurses) practicing at an academic medical center. We held two design sessions with most patients. Two patients had well-controlled asthma for many years and said they would likely not answer a weekly five-item questionnaire about their symptoms. All clinicians confirmed that the intervention would be most useful for patients who had at least some recent difficulty controlling their asthma. Therefore, we defined our target population for the intervention to include patients with poorly controlled asthma, a recent asthma exacerbation, and newly diagnosed. Our patient representative confirmed this decision.

With this sample, we achieved saturation to define the core components of an intervention that met the stated needs of users and our design objectives. Below, we describe each component ([Table 2]), key design decisions ([Table 3]), and benefits and burdens to each stakeholder ([Table 4]). Our heuristic assessment of usability revealed minimal issues and inconsistencies and is summarized in Appendix B.

Table 2 Core components for asthma symptom–monitoring intervention according to stated needs of users

Intervention component	Component description
1. Invitation	Patients are invited by their own physicians during in-person visits and handed off to another staff member who helps them install the app and understand the functionality
2. Weekly symptom checks and notifications	Patients receive weekly questionnaire prompts on their smartphones. Questionnaires expire after 48 h. If symptoms meet specific conditions of worsening or severity, the patient is given the option to send a notification to a care manager who works with patient's physician (e.g., an LPN, medical assistant). The care manager will then call the patient within 24 h and triage the call as if it were a patient phone call. If symptoms are severe, the care manager is auto-notified (i.e., patient is not given the option to request a call), but the care manager has the option to disable the auto-notify feature for specific patients
3. Patient's review of symptoms	Patients can view their recent 2 mo of symptom history easily and longer history on demand. Patients can review their aggregate symptom score as well as scores for each of the five questions
4. In-person visit	Physicians have easy access to the patient's symptom history in the EHR, and they review it prior to a patient visit

Table 3 Key design decisions

Design question	Decision and rationale
Who receives the notifications of problematic symptoms?	Decision: A care manager (e.g., nurse or medical assistant) who works with the physician and has access to patients' records would receive notifications and call patients Rationale: Originally, we planned for the notifications to go to physicians, but all physicians were strongly in favor of delegating that responsibility to someone else. Furthermore, no patients objected to having a care manager (or any staff member) serve this role. However, it was important to several patients that the care manager has a connection with their physician and has access to patients' health history. The physicians believed a nurse would be most appropriate for this role because nurses typically triage patients' phone calls and the workflows for responding to notifications would be similar. Ultimately, the extra responsibility could be considered part of a nurse's existing work, similar to taking patients' phone calls
Under what conditions should notifications be generated?	Decision: The patient is given the option to generate a notification if symptoms are at least 3 points or more below baseline or at least 3 points worse than their previous weeks' score. The notification is automatically generated if the symptoms are at least 6 points or worse than baseline, at least 6 points or worse than the previous weeks' score, or the most severe response on any individual question. Due to some patients potentially having severe baselines and triggering the auto-notification feature every week, care managers have the option to disable the auto-notify feature so that patients will always be given the option to request a call Rationale: We developed this feature through several iterations to balance the needs of patients (facilitate a conversation with a care manager if symptoms are potentially problematic) and clinical staff (minimize unnecessary notifications) Lacking an empirical basis, we used the minimum clinically significant difference for a comparable questionnaire (Asthma Control Test) as a key threshold and verified the notification logic with all physicians during design sessions
How should symptom history be presented to patients and clinicians?	Decision: A symptom summary is available with details on demand. For patients, the data are accessible from their smart phone, with the most recent 2 months most easily accessible by default. For providers, data should be accessible in the EHR where they would notice it (e.g., as a recent note) and their mobile device (if their EHR is not available on their mobile device) Rationale: Most patients wanted the previous 1–2 months data to be easily available and to be able to see all of their data if they wanted to drill down. For providers, prior to an in-person visit, the most likely place most said they would notice data would be as a recent note. However, there was no consistent location in the EHR where they would definitely look before each visit
How should the app be “gamified?”	Decision: We added three components inspired by game design. First, we added text before each weekly questionnaire explaining its purpose (known as “framing”). Second, we provided time constraints for patients to complete each questionnaire (48 h), which also helped restrict the hours at which care manager would need to be available. Third, if patients responded to four questionnaires in a row, they were shown a smiley face Rationale: Several patients expressed support for each of these game design features, and no users were against them. Patients were particularly supportive of the smiley face. We rejected other gamification concepts as inappropriate for this application (e.g., social comparisons)
How should security and privacy be ensured?	Decision: Patients are required to enter a 5-digit pin to open the app. Physicians and care managers are required to enter a secure password to view patient data Rationale: Institutional policy required the 5-digit pin. Most patients agreed that a 5-digit pin requirement was acceptable and would not inhibit use

Table 4 Benefits and burdens of an asthma mHealth intervention

	Patients	Clinicians	Provider organizations
Benefits	Helps patients make decisions on when to call provider	May improve physician's professional satisfaction due to ability to make better, timelier care decisions, and adhere to clinical guidelines	Potential to reduce costs from decreased office visits, hospitalizations and readmissions, ED visits (especially relevant for accountable care)
	Gives patients a tool to track symptoms over time	May streamline conversations with patients about symptoms during visits	Potential to treat more patients, improve patient retention
	Facilitates discussion with provider about interval history of symptoms (improves recall)		Potential to improve patient satisfaction and experience measures
	May mitigate exacerbations and reduce hospitalizations	May reduce volume of uninteresting visits from patients with well-controlled asthma
	May decrease need for as frequent visits
Burden	Patients must answer 5 questions about their asthma each week	Physicians must invite patients	Care manager's time responding to notifications would need to be covered
	Patients must install app one time and use a 5-digit PIN to open it	Physicians would ideally briefly review recent PRO data (accessible in EHR) before or during patient visits	Management would need to set up technology, provide technical support, oversee process to ensure notifications are addressed, and train clinicians
		May increase volume of phone calls	Potential to increase phone calls
		Care manager in clinic has additional responsibility of calling patients when notification is generated

Abbreviations: ED, emergency department; EHR, electronic health record.

Note: Content of patients and clinicians' columns were identified by users; provider organization column is an analysis from the research team.

Component 1: Invitation

A patient's own physician invites her to participate, ideally during an in-person visit, and then hands her off to another staff member who helps the patient set up and become acquainted with the functionality on their phone. Our design sessions with patients and prior experience with patient-reported outcomes (PROs) support this decision: patients are much more likely to adhere to recommendations made by their physician.

Component 2: Weekly Symptom Checks and Notifications

Each week, patients receive an automatically generated prompt on their smartphone at a designated time, with follow-up reminders if they do not respond. Prompts ask the patient to complete a five-item asthma questionnaire within a 48-hour window after which the questionnaire expires. (Patients preferred the term questionnaire to survey because the latter evoked associations with customer service.) As none of our interviewed patients expressed preferences for timing of the prompt, they are generated when convenient for the care manager (e.g., an LPN or medical assistant) to be available to call them during working hours. (No notifications are issued on Thursday or Friday to try to catch patient's issues during workdays.) After the patient completes the questionnaire, simple logic rules determine if the patient's symptoms are worse than the previous week, worse than baseline, or severe on any given question. If the criteria are met, the patient is given the option of requesting a call from a care manager, who works with the patient's physician ([Fig. 1]). We purposefully designed this workflow to be similar to how a care manager might handle a patient phone call about their symptoms so that workflow changes would be minimal. If symptoms meet more severe criteria, the care manager is auto-notified (i.e., patient is not given the option). However, the care manager has the option to disable the auto-notify feature for a period of time, thereby requiring the patient to request a phone call for severe symptoms. The ability to disable auto-notifications may prevent excess calls from the care manager to patients with severe baselines. The care manager will have access to a view of the patient's complete self-reported symptom history ([Fig. 2]). If a notification is sent, the app sends a brief follow-up survey to ask the patient if the call occurred and assess satisfaction.

Fig. 1 Question 1 of ACM and example questionnaire completion page. ACM, asthma control measure.

Fig. 2 Care manager's view (data are fictional).

We encountered two challenges in designing this component. First, some of the patients in our design sessions had difficulty understanding the concept of a baseline and said there was no typical week in terms of their symptoms. Furthermore, their baselines may change by the seasons. We therefore ask patients to choose as their baseline response symptoms on an average week in which they are not experiencing an asthma ”flare” and would probably not call their physician, and we acknowledge that these may not be exact. We did not attempt to design a way to create a new baseline based on season because patients found it challenging to think about this feature without having used the app. Second, some patients asked for a definition of an asthma attack. So as to avoid interrupting the flow of the questions, we included a definition at the beginning of the survey and gave the patients the option to stop showing it in the future.

Component 3: Patient Review of Symptoms

The app allows patients to review their symptom history at any time ([Fig. 3]). In our design sessions, patients generally agreed to the need to easily review their most recent 1 to 2 months of scores but have more of their previous data available on demand. However, there was a range in patients' views of the importance of this feature, with some finding it more valuable than others. We therefore developed an interactive widget allowing them to review their recent 2- and 6-month symptom scores, as well as the ability to view each question response in graphic form. The data points are color coded per severity according to our notification logic.

Fig. 3 Patient's view of symptom graph (data are fictional, y-axis is modified ACM, blue dots are unchanged or improved symptoms; yellow are somewhat concerning; red are severe symptoms). ACM, asthma control measure.

Component 4: In-Person Visit

All patient-reported data are available to the physician through their EHR as a recent note ([Fig. 4]). Most patients emphasized that it would be important to them that their physicians were aware of their reported data during the in-person visit, and if the physicians were not aware, they might lose motivation to continue responding to the weekly questionnaires. Physicians also wanted the recent scores available. However, physician workflows vary and we found no clear place in this EHR that physicians consistently reviewed before a visit. All physicians we spoke with said that in the existing EHR, one place where they would likely notice the data before a visit would be a recent note, but they could not guarantee they would always notice it.

Fig. 4 Summary of patient symptoms accessible to physicians in medical record (data are fictional).

Benefits and Burdens

As in other studies, we found that users weighed the benefit and burdens in their decisions to use novel functionality. We explicitly weighed those considerations in our design[23] [33] and attempted to maximize benefit while minimizing burden. A summary of the identified benefits and burdens to patients, clinicians, and provider organizations identified in our design sessions is displayed in [Table 4].

Anticipated Implementation Challenges

We have identified several potential challenges to implementing the intervention in routine clinical practice:

1. The clinic must train a staff member to help patients download and use the symptom-monitoring app. This step may be doable without involvement of a staff member, but someone in the clinic would need to be able to answer patients' questions.

2. The symptom-monitoring software would need to be integrated into a clinic's EHR such that it automatically produced a note summarizing the patient's recent symptoms or some other kind of electronic indication showing that new PRO data was available prior to a visit. (We plan to perform a feasibility test of use during the in-person visit using the “wizard of oz” or “fake backend” approach,[34] [35] [36] in which we manually paste a one-page summary of the patient's scores into the EHR as a recent note before each patient visit.)

3. The PRO data would need to be available on the physician's mobile device, ideally through mobile access to their EHR. Several physicians requested this, but we are not certain if this is a critical requirement, at least today.

4. The symptom-monitoring software would ideally also be integrated into a patient portal (a Web site or app that connects patients to their providers) so that the patient would not be required to download a separate app. We are not certain if this is a critical requirement because few of our design patients used a patient portal. However, adding such functionality might motivate more patients to sign up for patient portals and benefit from its other features.

5. A care manager (e.g., LPN or medical assistant) in the clinic would need to respond to notifications and call patients.

Potential Unintended Consequences

We identified two main potential unintended consequences. First, some patients might become overreliant on the app and wait to call their physician until the app suggests it. To mitigate this concern, it is important that when patients are introduced to the intervention, they understand that the app will not make decisions for them and that they should always use their own judgment. We also include text in multiple places in the app that reminds patients they have the option of calling their doctor or going to the emergency room. In future versions, we expect that patient access to an updated asthma action plan may further address this concern. Second, some patients may request many calls from a care manager when the calls have questionable utility. If we find this during testing, we will develop additional strategies to address this concern.

Discussion

We identified core components for implementing asthma symptom monitoring between visits as part of routine care using smartphone-based PROs. We achieved saturation of minimum intervention requirements with a modest number of patients and design sessions. Our findings provide an integrated, functional building block of interrelated features that can be tested, refined, and enhanced in a systematic fashion. Our approach to developing an intervention of starting small and incrementally adding new components may seem cumbersome for researchers eager to demonstrate outcomes. However, our results show that even for what may appear to be a simple intervention, critical insights for the design may arise through user testing, and an iterative user-centered approach can reveal these insights efficiently. This approach addresses the limitations of previous literature in which details on how the interventions contribute to study outcomes are not known and interventions involve many complex components. Interventions that do not take this incremental “bottom-up” approach and do not invest in user testing will more likely encounter barriers that could have been foreseen and addressed.[37]

In addition to the asthma-related mHealth apps that have been formally evaluated, there are many other apps available for download.[38] These apps contain a multitude of features, including medication tracking, symptom tracking, self-recorded asthma action plan, educational videos, guidance for which asthma medication to take, and peak flow measurement tracking. Though many of these features may be useful to patients, most lack detailed descriptions of their development process, evidence of ability to sustain engagement over time, or evidence of clinical utility. Furthermore, these apps are marketed primarily as direct-to-consumer products, with little clinical integration. We found in our user testing that a core requirement for patients to remain motivated was the way the app enhanced the patient–provider relationship, and so we began with the most basic functionality that would address that relationship while being consistent with clinical guidelines. We also specified the workflows that providers would need to adopt for this app to be effective. As we add features, we will build on this foundation and consider adding other features, such as those included in these apps, based on the results of our pilot test and further user testing.

Perhaps the major barrier we identified is the lack of a clear place in the EHR for entering PROs generated between visits where the physician would likely notice them. Our results suggest that a recent note is one possible location, but not all physicians look there before each visit. The need for such a location in the EHR may increase as more health data are captured in digital form between visits and made available in the EHRs. Previous studies that attempt to integrate PROs into clinical workflow for cancer and orthopedic care have centered on in-clinic patient reports or have used paper workarounds.[39] [40] [41] We hope that EHR designers and implementers will consider this need as a future design requirement and create a standardized location for these types of data, perhaps as part of an overview or agenda functionality.

Limitations

The patients and clinicians we included represent a modest convenience sample from a single institution in one region. Patients were recruited through one physician and may not be representative of the greater population. However, our patients were demographically diverse, and we reached saturation of critical components. Second, the intervention may not be appropriate for all patients. Testing will help us understand what kinds of patients are suitable. Third, similar to findings with the asthma action plan, our designed intervention will likely not be successful if physicians are not supportive. Some physicians may require training to become familiar with measures of asthma control and additional support until they can experience the benefits. Finally, we changed our questionnaire items from monthly to weekly, which may affect the psychometric properties somewhat.

Future Enhancements

In our design process, users identified many intervention ideas that we deferred until after our planned feasibility trial because none of the users said they were essential to using the app. Many of these are supported by clinical guidelines and are candidates for future iterative development: diary functionality to track symptom flare-ups, variable timing of symptom questionnaire based on reported severity, support for patient-initiated symptom checks between weekly check-ins, integration of asthma action plans,[23] [42] [43] [44] tailored educational materials, medication regimen and reminders, and use of objective data (e.g., inhaler use, peak flow) to inform monitoring.[45] Other candidates for future features include digital self-management interventions that have proven effective in trials but have minimal uptake.[46] Additionally, if this intervention is widely adopted, the PROMs collected may prove useful for secondary purposes, such as evaluating the effect of other interventions on asthma symptoms, or comparing health systems performance for asthma care.

Conclusion

Through iterative design sessions, we identified four core components for routinely monitoring asthma symptoms that met the stated needs of patients and clinicians, and benefits and barriers of implementation to key stakeholders. The intervention's components form a starting point and can be pilot tested and incrementally expanded. By combining user-centered design and qualitative research methods, we developed a new mHealth intervention in a systematic and highly efficient way.

Clinical Relevance Statement

Recent systematic reviews of mHealth asthma interventions show inconsistent outcomes and little insight into success factors. Through iterative design sessions, we identified four core components for routinely monitoring asthma symptoms between visits via mHealth app installed on the patient's smartphones. This work begins to build a body of knowledge for how to use mHealth to incorporate PROs into routine care.

Multiple Choice Question

Where, in an electronic health record (EHR), is the optimal location to include patient-reported between-visit symptom data so that physicians will notice them before an in-person visit with a patient?

• In a recent clinical note.

• On a summary page.

• In a special PRO module.

• There is no optimal location.

Correct answer: The correct answer is D. Although some physicians may notice PRO in various parts of the EHR, there is no one designated location which physicians will always look prior to a patient visit. There is no industry-wide standard of EHR functionality to ensure certain data are on physicians' agendas for in-person visits.

Conflict of Interest

Dr. Zimlichman is an advisor to Valera Health. Dr. Landman reports he is a senior editor at Ranked Health, a project that evaluates digital health apps, run by the Hacking Medicine Institute, a nonprofit organization. Dr. Bates is a coinventor on Patent No. 6029138 held by Brigham and Women's Hospital on the use of decision support software for medical management, licensed to the Medicalis Corporation. He holds a minority equity position in the privately held company Medicalis which develops web-based decision support for radiology test ordering. He serves on the board for SEA Medical Systems, which makes intravenous pump technology. He consults for EarlySense, which makes patient safety monitoring systems. He receives equity and cash compensation from QPID, Inc, a company focused on intelligence systems for electronic health records. He receives cash compensation from CDI (Negev), Ltd, which is a not-for-profit incubator for health IT startups. He receives equity from Enelgy which makes software to support evidence-based clinical decisions. He receives equity from Valera Health which makes software to help patients with chronic diseases. He receives equity from Intensix which makes software to support clinical decision making in intensive care. He receives equity from MDClone which takes clinical data and produces de-identified versions of it. Dr. Bates' financial interests have been reviewed by Brigham and Women's Hospital and Partners HealthCare in accordance with their institutional policies.

Acknowledgments

The authors thank Josie Elias for helping to facilitate implementation of the software at Brigham and Women's Hospital, and Alyson Youngblood, Chrissy Sovak, and Tiffani Bright for conducting the heuristic evaluation. This work was supported by the Agency for Healthcare Research and Quality grant #1R21HS023960.

Protection of Human and Animal Subjects

This study was reviewed by RAND and Partners Healthcare Institutional Review Board.

• References

• 1 Singh K, Drouin K, Newmark LP. , et al. Many mobile health apps target high-need, high-cost populations, but gaps remain. Health Aff (Millwood) 2016; 35 (12) 2310-2318
• 2 Smith A. Record Shares of Americans Now Own Smartphones, Have Home Broadband. Pew Research Center; 2017
• 3 Laing BY, Mangione CM, Tseng CH. , et al. Effectiveness of a smartphone application for weight loss compared with usual care in overweight primary care patients: a randomized, controlled trial. Ann Intern Med 2014; 161 (10, Suppl): S5-S12
• 4 Centers for Disease Control and Prevention. Asthma in the US. CDC Vital Signs, [cited May 2011]; Available at: http://www.cdc.gov/vitalsigns/asthma
• 5 World Health Organization. Asthma fact sheet No. 307. Updated November 2013. [cited 2008]; Available at: http://www.who.int/topics/asthma/en
• 6 Wang T, Srebotnjak T, Brownell J, Hsia RY. Emergency department charges for asthma-related outpatient visits by insurance status. J Health Care Poor Underserved 2014; 25 (01) 396-405
• 7 Lang DM. New asthma guidelines emphasize control, regular monitoring. Cleve Clin J Med 2008; 75 (09) 641-653
• 8 van der Meer V, van Stel HF, Bakker MJ. , et al; SMASHING (Self-Management of Asthma Supported by Hospitals, ICT, Nurses and General practitioners) Study Group. Weekly self-monitoring and treatment adjustment benefit patients with partly controlled and uncontrolled asthma: an analysis of the SMASHING study. Respir Res 2010; 11 (01) 74
• 9 Braido F. Failure in asthma control: reasons and consequences. Scientifica (Cairo) 2013; 2013: 549252
• 10 Okelo SO, Butz AM, Sharma R. , et al. Interventions to modify health care provider adherence to asthma guidelines: a systematic review. Pediatrics 2013; 132 (03) 517-534
• 11 Wisnivesky JP, Lorenzo J, Lyn-Cook R. , et al. Barriers to adherence to asthma management guidelines among inner-city primary care providers. Ann Allergy Asthma Immunol 2008; 101 (03) 264-270
• 12 Marcano Belisario JS, Huckvale K, Greenfield G, Car J, Gunn LH. Smartphone and tablet self management apps for asthma. Cochrane Database Syst Rev 2013; ;( (11) CD010013
• 13 Hui CY, Walton R, McKinstry B, Jackson T, Parker R, Pinnock H. The use of mobile applications to support self-management for people with asthma: a systematic review of controlled studies to identify features associated with clinical effectiveness and adherence. J Am Med Inform Assoc 2017; 24 (03) 619-632
• 14 Wiecha JM, Adams WG, Rybin D, Rizzodepaoli M, Keller J, Clay JM. Evaluation of a web-based asthma self-management system: a randomised controlled pilot trial. BMC Pulm Med 2015; 15 (01) 17
• 15 Vasbinder EC, Janssens HM, Rutten-van Mölken MP. , et al; e-MATIC Study Group. e-Monitoring of asthma therapy to improve compliance in children using a real-time medication monitoring system (RTMM): the e-MATIC study protocol. BMC Med Inform Decis Mak 2013; 13 (01) 38
• 16 Rasmussen LM, Phanareth K, Nolte H, Backer V. Internet-based monitoring of asthma: a long-term, randomized clinical study of 300 asthmatic subjects. J Allergy Clin Immunol 2005; 115 (06) 1137-1142
• 17 Ries E. The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. New York: Crown Business; 2011
• 18 Risso NA, Neyem A, Benedetto JI. , et al. A cloud-based mobile system to improve respiratory therapy services at home. J Biomed Inform 2016; 63: 45-53
• 19 Lara M, Edelen MO, Eberhart NK, Stucky BD, Sherbourne CD. Development and validation of the RAND Asthma Control Measure. Eur Respir J 2014; 44 (05) 1243-1252
• 20 Schatz M, Sorkness CA, Li JT. , et al. Asthma Control Test: reliability, validity, and responsiveness in patients not previously followed by asthma specialists. J Allergy Clin Immunol 2006; 117 (03) 549-556
• 21 Becker MH, Maiman LA. Sociobehavioral determinants of compliance with health and medical care recommendations. Med Care 1975; 13 (01) 10-24
• 22 Becker MH, Radius SM, Rosenstock IM, Drachman RH, Schuberth KC, Teets KC. Compliance with a medical regimen for asthma: a test of the health belief model. Public Health Rep 1978; 93 (03) 268-277
• 23 Lau AY, Arguel A, Dennis S, Liaw ST, Coiera E. “Why Didn't it Work?” Lessons from a randomized controlled trial of a web-based personally controlled health management system for adults with asthma. J Med Internet Res 2015; 17 (12) e283
• 24 Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining gamification. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments; 2011:9–15
• 25 Mekler ED, Brühlmann F, Opwis K, Tuch AN. Disassembling gamification: the effects of points and meaning on user motivation and performance. CHI'13 Extended Abstracts on Human Factors in Computing Systems; 2013:1137–1142
• 26 Nicholson S. A User-Centered Theoretical Framework for Meaningful Gamification. Madison, WI: Games+Learning+Society 8.0; 2012
• 27 Steele Gray C, Khan AI, Kuluski K. , et al. Improving patient experience and primary care quality for patients with complex chronic disease using the electronic patient-reported outcomes tool: adopting qualitative methods into a user-centered design approach. JMIR Res Protoc 2016; 5 (01) e28
• 28 Jones SS, Heaton PS, Rudin RS, Schneider EC. Unraveling the IT productivity paradox--lessons for health care. N Engl J Med 2012; 366 (24) 2243-2245
• 29 Rudin RS, Bates DW, MacRae C. Accelerating innovation in health IT. N Engl J Med 2016; 375 (09) 815-817
• 30 Nielsen J. 10 Usability Heuristics for User Interface Design. 1995 [cited May 18, 2017]; Available at: http://www.nngroup.com/articles/ten-usability-heuristics/
• 31 Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Transaction publishers; 2009
• 32 Lewis C. Using the “Thinking-Aloud” Method in Cognitive Interface Design. Yorktown Heights, NY: IBM TJ Watson Research Center; 1982
• 33 Harle CA, Listhaus A, Covarrubias CM. , et al. Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report. J Am Med Inform Assoc 2016; 23 (01) 74-79
• 34 Asch DA, Rosin R. Innovation as Discipline, not fad. N Engl J Med 2015; 373 (07) 592-594
• 35 Bella M, Hanington B. Universal Methods of design. America: Rockport; 2012
• 36 Klemmer SR, Sinha AK, Chen J, Landay JA, Aboobaker N, Wang A. Suede: a Wizard of Oz prototyping tool for speech user interfaces. Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology;2000:1–10
• 37 van Gaalen JL, van Bodegom-Vos L, Bakker MJ, Snoeck-Stroband JB, Sont JK. Internet-based self-management support for adults with asthma: a qualitative study among patients, general practitioners and practice nurses on barriers to implementation. BMJ Open 2016; 6 (08) e010809
• 38 Rodriguez D. 8 Apps That Make Asthma Management Easier. 2016 [cited 2016 August 14]; Available at: http://www.everydayhealth.com/hs/adult-asthma/asthma-management-apps/
• 39 Basch E, Deal AM, Kris MG. , et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016; 34 (06) 557-565
• 40 Rotenstein LS, Agarwal A, O'Neil K. , et al. Implementing patient-reported outcome surveys as part of routine care: lessons from an academic radiation oncology department. J Am Med Inform Assoc 2017; 24 (05) 964-968
• 41 Slover JD, Karia RJ, Hauer C, Gelber Z, Band PA, Graham J. Feasibility of integrating standardized patient-reported outcomes in orthopedic care. Am J Manag Care 2015; 21 (08) e494-e500
• 42 Ring N, Jepson R, Hoskins G. , et al. Understanding what helps or hinders asthma action plan use: a systematic review and synthesis of the qualitative literature. Patient Educ Couns 2011; 85 (02) e131-e143
• 43 Ring N, Jepson R, Pinnock H. , et al. Developing novel evidence-based interventions to promote asthma action plan use: a cross-study synthesis of evidence from randomised controlled trials and qualitative studies. Trials 2012; 13 (01) 216
• 44 Ring N, Malcolm C, Wyke S. , et al. Promoting the use of personal asthma action plans: a systematic review. Prim Care Respir J 2007; 16 (05) 271-283
• 45 Do Q, Tran S, Robinson K. Big Data and mHealth Drive Asthma Self-Management. 2015 International Conference on Computational Science and Computational Intelligence (CSCI); IEEE; 2015:806–809
• 46 Morrison D, Wyke S, Agur K. , et al. Digital asthma self-management interventions: a systematic review. J Med Internet Res 2014; 16 (02) e51