Appl Clin Inform 2017; 08(04): 1031-1043
DOI: 10.4338/ACI-2017-06-RA-0096
Research Article
Schattauer GmbH Stuttgart

Core Components for a Clinically Integrated mHealth App for Asthma Symptom Monitoring

Robert S. Rudin
Christopher H. Fanta
Zachary Predmore
Kevin Kron
Maria O. Edelen
Adam B. Landman
Eyal Zimlichman
David W. Bates
Further Information

Address for correspondence

Robert S. Rudin, PhD
RAND Corporation
20 Park Plaza, Suite 920, Boston, MA 02116
United States   

Publication History

14 June 2017

30 August 2017

Publication Date:
14 December 2017 (online)



Background mHealth apps may be useful tools for supporting chronic disease management.

Objective Our aim was to apply user-centered design principles to efficiently identify core components for an mHealth-based asthma symptom–monitoring intervention using patient-reported outcomes (PROs).

Methods We iteratively combined principles of qualitative research, user-centered design, and “gamification” to understand patients' and providers' needs, develop and refine intervention components, develop prototypes, and create a usable mobile app to integrate with clinical workflows. We identified anticipated benefits and burdens for stakeholders.

Results We conducted 19 individual design sessions with nine adult patients and seven clinicians from an academic medical center (some were included multiple times). We identified four core intervention components: (1) Invitation—patients are invited by their physicians. (2) Symptom checks—patients receive weekly five-item questionnaires via the app with 48 hours to respond. Depending on symptoms, patients may be given the option to request a call from a nurse or receive one automatically. (3) Patient review—in the app, patients can view their self-reported data graphically. (4) In-person visit—physicians have access to patient-reported symptoms in the electronic health record (EHR) where they can review them before in-person visits. As there is currently no location in the EHR where physicians would consistently notice these data, recording a recent note was the best option. Benefits to patients may include helping decide when to call their provider and facilitating shared decision making. Benefits to providers may include saving time discussing symptoms. Provider organizations may need to pay nurses extra, but those costs may be offset by reduced visits and hospitalizations.

Conclusion Recent systematic reviews show inconsistent outcomes and little insight into functionalities required for mHealth asthma interventions, highlighting the need for systematic intervention design. We identified specific features for adoption and engagement that meet the stated needs of users for asthma symptom monitoring.


Background and Significance

Mobile health applications (mHealth apps) have the potential to improve chronic condition management and enhance patient–provider interactions between visits. More than three-quarters of Americans currently own smartphones and rates of smartphone ownership are rising among older adults and people with low household incomes.[1] [2] However, most patients' interaction with providers between visits are not affected by mHealth apps, except for patients' ability to view their medical information and send secure messages to their providers through mobile patient portals. Of the more than 165,000 mHealth apps available, many have low usability, do not provide clinical utility, have minimal uptake, or are abandoned soon after first use.[1] [3] Few are designed to be integrated into clinical workflows, even those that are highly rated. Little is known about how to develop mHealth functionality that will not only provide clinical utility for chronic condition management but also will be adopted and used by patients and providers. Identifying such mHealth functionality represents a key challenge for informatics.

To begin to address this challenge, we developed an mHealth intervention for asthma. Asthma is a chronic condition that affects more than 25 million individuals in the United States, 300 million worldwide, and its incidence is increasing.[4] [5] Uncontrolled asthma interferes substantially in patients' daily lives and often results in patients requiring emergency medical services and hospitalizations. It is estimated that 1.75 million asthma-related visits to U.S. emergency departments occur each year (9 visits for every 100 patients with asthma), and they are disproportionately common among minorities and those of lower socioeconomic status.[6] mHealth apps have the potential to help patients conveniently track symptoms and facilitate interaction with providers. Current clinical guidelines recommend clinicians adjust treatment based on frequent monitoring of patients' symptoms.[7] [8] These guidelines are based on considerable evidence showing that asthma control can be largely achieved for patients in controlled trial settings when patients' symptoms are routinely monitored by clinicians.[9] However, in real-life settings, such intensive symptom monitoring does not happen routinely.[10] [11] Compared with other diseases such as diabetes and hypertension, which can be monitored with objective measures (hemoglobin A1c and blood pressure, respectively), measuring symptom frequency and severity for asthma and other chronic diseases has proved more challenging.

Two recent systematic reviews identified 12 randomized controlled trials (RCTs) of mHealth apps[12] [13] and other digital approaches to facilitate asthma intervention such as web-based tools[14] and text messaging.[15] Both reviews came to similar conclusions: interventions were complex and involved multiple components, and clinical effectiveness was inconsistent. Intervention components included symptom monitoring, asthma action plans, educational materials, games and quizzes, team communication tools, electronic diaries, decision support for physicians, peak flow measurements, and medication use.[13] [16] The reviews urged future research to better understand the contribution of intervention components to outcomes and factors that influence adoption and continued usage. Furthermore, in our informal review of the 12 RCTs, we found two additional limitations: none described a development approach using principles of user-centered design, and none explicitly considered sustainable integration into providers' routine workflows in developing the intervention.

To address these limitations, we attempted to develop an intervention “from the ground up” starting with identifying the core components for implementing asthma symptom monitoring via mHealth. Although most previous efforts developed multicomponent interventions without iterative user testing, our development approach focused only on symptom monitoring, engaged users at every step, and attempted to identify only the core components that meet the needs of users. This approach is similar to developing a minimum viable product,[17] a concept that is commonly used to guide software development in industry, but not often used in research.[18] Our intention is that the intervention we design will serve as a building block for further incremental development and feasibility testing of additional components, so that we can develop more complex interventions systematically with an understanding of how each component contributes. These findings may be applicable to other chronic conditions, thereby shortening development cycles and improving understanding of trial results.

Summary of Intervention

We designed a patient-facing mHealth app that runs on iOS and Android, a web-based clinician dashboard for use by care managers and physicians, and the workflows needed for implementation in routine care. Weekly notifications are sent to the patient app to complete questionnaires regarding asthma-related symptoms, and results are then available for review on the dashboard by care managers and physicians. The dashboard can also output summary reports to be added to the electronic health record (EHR) as a note prior to patient visit.



We attempted to identify the core components for an asthma symptom–monitoring intervention in an efficient manner using user-centered design methods. We defined the following criteria as design goals:

  • Patients would use it. Specifically, a substantial portion of asthma patients would complete a questionnaire about their symptoms on a weekly basis.

  • Clinicians would find it clinically beneficial and low burden. Specifically, the intervention would require only minimal changes to workflows.

We selected these proximate objectives for the purposes of design because they are necessary conditions for ultimately achieving improvements in health outcomes and utilization, which we plan to test in subsequent work. We attempted to reach saturation in intervention components defined as those that users considered as the minimum requirements to achieve the aforementioned criteria and would be feasible to develop with modest resources.




We recruited physicians and licensed practical nurses (LPNs) from clinical practices at two different sites within Brigham and Women's Hospital, an academic medical center in Boston, Massachusetts, from May 2015 through December 2016. We recruited patients for design sessions (which took the form of individual semistructured interviews) identified by one physician researcher (C.H.F.). We selected patients who were diagnosed with asthma and owned smartphones. We attempted to include ethnically diverse patients and a range of ages and income levels. While primary care is the setting in which most asthma patients are treated, for our intervention development, we chose clinics that specialized in pulmonary or allergic diseases to facilitate recruitment of physicians and diverse patients with difficulty controlling their asthma.


Clinical Guidelines and Rationale

Asthma symptom–monitoring requires periodic assessments using validated instruments.[7] [8] Following our goal of a minimum intervention, we selected a five-question asthma assessment instrument: the asthma control measure (ACM).[19] The ACM is a patient-reported outcome measure (PROM), has been shown to be reliable and valid, and has similar characteristics to the more widely used Asthma Control Test,[20] but has the additional advantage of not requiring the purchase of a license thereby improving scalability. Though guidelines suggest asthma control should be assessed every 2 to 4 weeks,[9] we chose weekly assessments because many controlled trials in which patients' asthma were successfully controlled involved weekly assessments, and less frequent monitoring would likely not be reinforcing. (ACM questions are listed in Appendix A).


Theoretical Frameworks and Concepts

Our development work is supported by the Health Belief Model (HBM).[21] [22] The HBM is a cognitive theory for understanding individual health-related behaviors and characterizes how individuals come to engage in preventative care management of their conditions. It provides a theoretical foundation for other asthma interventions.[23] The HBM argues that several factors influence an individual's propensity to engage in beneficial health behavior, including general health motivations, perceived threat of illness, and perceived benefits and barriers to compliant behavior. Our work focuses on the latter category. Through the design process, we attempted to maximize benefits to patients of symptom monitoring while minimizing barriers and burdens.

To help maximize benefits, we also adapted concepts from “gamification.” Gamification is the use of game design elements in non-game contexts.[24] [25] [26] We assessed individual game design elements from the published literature for ideas to enhance and support our core objectives, workflows, and functionality in a way that creates additional benefit for users, and tested these ideas with users during design sessions.


User-Centered Design and Development Process

We applied principles of user-centered design and qualitative research through individual design sessions with a diverse sample of end users.[27] Because the major benefits of technology arise when work processes are altered to take advantage of the technology's potential, we developed technology and workflows concurrently.[28] [29] We included a patient representative on the research team with whom we consulted about overall design before each development stage and on an ad hoc basis when design questions arose (e.g., when we received conflicting feedback from users). We also alternated our design sessions with patients and with clinicians so that we would be able to develop an intervention that incorporated both perspectives. We applied lessons learned from one investigator's (E.Z.) prior work developing a platform for collecting PROMs. We also conducted a heuristic evaluation involving three usability experts using Nielsen's heuristics[30] (see Appendix B).

The development process proceeded in four broad phases ([Table 1]). Although the steps largely occurred in sequence, at times we needed to return to a previous phase (e.g., return to low-fidelity mockups to clarify and improve navigation after app software development had begun). Within each phase, we attempted to reach saturation in desired intervention components before advancing to the next phase.[31] For example, we continued to conduct design sessions in phase 1 (workflow refinement) until we found that the users generally agreed with our proposed workflows and did not require changes in our high-level workflow or functionality specifications. In keeping with the intention of identifying core components, components that were viewed as essential to achieve our objectives were included; components that were desired but not essential were deferred to subsequent versions (see future enhancements later); and components that were strongly thought to support objectives but not clearly essential were included if they were feasible to develop and users believed they had minimal chance of detracting from other essential components (e.g., smiley face for completed questionnaires).

Table 1

Workflow and technology development steps

Development phase



1. Workflow refinement

Defined workflow roles and responsibilities

Described functionality

2. Intervention outline

Outlined workflow diagram (Microsoft Visio)

Developed low-fidelity prototype (Balsamiq)

3. Intervention detail

Detailed workflow diagram (Microsoft Visio)

Developed high-fidelity prototype (Invision)

4. Intervention final

Finalized workflows

Complete app front end and back end developed

In phase 1, we developed our first workflows and refined them through semistructured interviews with the goal of understanding users' needs and motivations. We used an interview guide and asked patient-users questions about general experiences with asthma, motivation for engaging in symptom monitoring, motivation and barriers to answering weekly questions related to asthma symptoms, desired methods and timing for facilitating contact with clinicians as a result of symptom scores, impact of symptom monitoring on relationship with clinician, and the potential role of symptom scores during in-person visits. Physician questions included invitation processes, types of patients most likely to benefit from symptom monitoring, utility of weekly patient-reported symptom scores, score thresholds needed for generating notifications, methods for establishing contact between patient and clinician, and workflow changes that would maximize utility of symptom data. For both patients and physicians, we showed them the five-question ACM for reference.

In phase 2's design sessions, we used updated interview guides, described our proposed intervention, and developed workflow diagrams and low-fidelity mockups (sketches to illustrate rough ideas) of the user interfaces. We asked users to use the “think aloud” protocol when reviewing the mockups.[32] We asked more detailed questions about the topics discussed in phase 1. For example, we asked patients about how they would like to see their own self-reported data, from whom they would like to receive a call if symptoms are worsening, and their reaction to a smiley face as a reward for completing the weekly questionnaire (concept borrowed from gamification). For physicians, we asked about the process for handling notifications and making the symptom data available during in-person visits. For care managers, we asked LPNs about additional time requirements to handle patient-generated notifications and aspects of the notification logic.

As we began to define our functionality, we assessed the technical feasibility of implementing this intervention using our clinical sites' existing EHR. We found that the EHR supported some but not all of our required features; so, we contracted with a third-party app development company (ADK Group, Boston, Massachusetts, United States) to help develop high-fidelity prototypes and to write the software.

In phase 3 design sessions, we user-tested high-fidelity prototypes (close to a final product) with patients and clinicians, with a focus on clarity of wording, look-and-feel, and navigation. To ensure our design followed best practices for smartphone apps, we reviewed the design with two experts in smartphone user interfaces from Apple, Inc. We also continued to enhance the workflows by providing more detail of the steps and options. In phase 4, we finalized the workflows and worked with our contractor as they wrote the software, and conducted a summative heuristic evaluation with three evaluators.

One researcher (R.S.R.) led all design sessions, while another (Z.P. or K.K.) asked additional questions and took notes. Both reviewed the notes afterward for accuracy and resolved differences. The interviews were audio recorded unless the participant declined. At the conclusion of each session, one researcher summarized key findings and another reviewed. Differences were resolved by listening to the recordings. Patients were given a $25 incentive per design session. The institutional review boards of RAND and Partners Healthcare approved this work.



We conducted 19 individual interviews or design sessions with patients (n = 9, ages 21–74 years, 6 females, 2 African Americans, 1 Latino, 2 low-income, 2 section-8 housing residents) and clinicians (n = 7, 5 physicians, 2 nurses) practicing at an academic medical center. We held two design sessions with most patients. Two patients had well-controlled asthma for many years and said they would likely not answer a weekly five-item questionnaire about their symptoms. All clinicians confirmed that the intervention would be most useful for patients who had at least some recent difficulty controlling their asthma. Therefore, we defined our target population for the intervention to include patients with poorly controlled asthma, a recent asthma exacerbation, and newly diagnosed. Our patient representative confirmed this decision.

With this sample, we achieved saturation to define the core components of an intervention that met the stated needs of users and our design objectives. Below, we describe each component ([Table 2]), key design decisions ([Table 3]), and benefits and burdens to each stakeholder ([Table 4]). Our heuristic assessment of usability revealed minimal issues and inconsistencies and is summarized in Appendix B.

Table 2

Core components for asthma symptom–monitoring intervention according to stated needs of users

Intervention component

Component description

1. Invitation

Patients are invited by their own physicians during in-person visits and handed off to another staff member who helps them install the app and understand the functionality

2. Weekly symptom checks and notifications

Patients receive weekly questionnaire prompts on their smartphones. Questionnaires expire after 48 h. If symptoms meet specific conditions of worsening or severity, the patient is given the option to send a notification to a care manager who works with patient's physician (e.g., an LPN, medical assistant). The care manager will then call the patient within 24 h and triage the call as if it were a patient phone call. If symptoms are severe, the care manager is auto-notified (i.e., patient is not given the option to request a call), but the care manager has the option to disable the auto-notify feature for specific patients

3. Patient's review of symptoms

Patients can view their recent 2 mo of symptom history easily and longer history on demand. Patients can review their aggregate symptom score as well as scores for each of the five questions

4. In-person visit

Physicians have easy access to the patient's symptom history in the EHR, and they review it prior to a patient visit

Table 3

Key design decisions

Design question

Decision and rationale

Who receives the notifications of problematic symptoms?

Decision: A care manager (e.g., nurse or medical assistant) who works with the physician and has access to patients' records would receive notifications and call patients

Rationale: Originally, we planned for the notifications to go to physicians, but all physicians were strongly in favor of delegating that responsibility to someone else. Furthermore, no patients objected to having a care manager (or any staff member) serve this role. However, it was important to several patients that the care manager has a connection with their physician and has access to patients' health history. The physicians believed a nurse would be most appropriate for this role because nurses typically triage patients' phone calls and the workflows for responding to notifications would be similar. Ultimately, the extra responsibility could be considered part of a nurse's existing work, similar to taking patients' phone calls

Under what conditions should notifications be generated?

Decision: The patient is given the option to generate a notification if symptoms are at least 3 points or more below baseline or at least 3 points worse than their previous weeks' score. The notification is automatically generated if the symptoms are at least 6 points or worse than baseline, at least 6 points or worse than the previous weeks' score, or the most severe response on any individual question. Due to some patients potentially having severe baselines and triggering the auto-notification feature every week, care managers have the option to disable the auto-notify feature so that patients will always be given the option to request a call

Rationale: We developed this feature through several iterations to balance the needs of patients (facilitate a conversation with a care manager if symptoms are potentially problematic) and clinical staff (minimize unnecessary notifications) Lacking an empirical basis, we used the minimum clinically significant difference for a comparable questionnaire (Asthma Control Test) as a key threshold and verified the notification logic with all physicians during design sessions

How should symptom history be presented to patients and clinicians?

Decision: A symptom summary is available with details on demand. For patients, the data are accessible from their smart phone, with the most recent 2 months most easily accessible by default. For providers, data should be accessible in the EHR where they would notice it (e.g., as a recent note) and their mobile device (if their EHR is not available on their mobile device)

Rationale: Most patients wanted the previous 1–2 months data to be easily available and to be able to see all of their data if they wanted to drill down. For providers, prior to an in-person visit, the most likely place most said they would notice data would be as a recent note. However, there was no consistent location in the EHR where they would definitely look before each visit

How should the app be “gamified?”

Decision: We added three components inspired by game design. First, we added text before each weekly questionnaire explaining its purpose (known as “framing”). Second, we provided time constraints for patients to complete each questionnaire (48 h), which also helped restrict the hours at which care manager would need to be available. Third, if patients responded to four questionnaires in a row, they were shown a smiley face

Rationale: Several patients expressed support for each of these game design features, and no users were against them. Patients were particularly supportive of the smiley face. We rejected other gamification concepts as inappropriate for this application (e.g., social comparisons)

How should security and privacy be ensured?

Decision: Patients are required to enter a 5-digit pin to open the app. Physicians and care managers are required to enter a secure password to view patient data

Rationale: Institutional policy required the 5-digit pin. Most patients agreed that a 5-digit pin requirement was acceptable and would not inhibit use

Table 4

Benefits and burdens of an asthma mHealth intervention



Provider organizations


Helps patients make decisions on when to call provider

May improve physician's professional satisfaction due to ability to make better, timelier care decisions, and adhere to clinical guidelines

Potential to reduce costs from decreased office visits, hospitalizations and readmissions, ED visits (especially relevant for accountable care)

Gives patients a tool to track symptoms over time

May streamline conversations with patients about symptoms during visits

Potential to treat more patients, improve patient retention

Facilitates discussion with provider about interval history of symptoms (improves recall)

Potential to improve patient satisfaction and experience measures

May mitigate exacerbations and reduce hospitalizations

May reduce volume of uninteresting visits from patients with well-controlled asthma

May decrease need for as frequent visits


Patients must answer 5 questions about their asthma each week

Physicians must invite patients

Care manager's time responding to notifications would need to be covered

Patients must install app one time and use a 5-digit PIN to open it

Physicians would ideally briefly review recent PRO data (accessible in EHR) before or during patient visits

Management would need to set up technology, provide technical support, oversee process to ensure notifications are addressed, and train clinicians

May increase volume of phone calls

Potential to increase phone calls

Care manager in clinic has additional responsibility of calling patients when notification is generated

Abbreviations: ED, emergency department; EHR, electronic health record.

Note: Content of patients and clinicians' columns were identified by users; provider organization column is an analysis from the research team.

Component 1: Invitation

A patient's own physician invites her to participate, ideally during an in-person visit, and then hands her off to another staff member who helps the patient set up and become acquainted with the functionality on their phone. Our design sessions with patients and prior experience with patient-reported outcomes (PROs) support this decision: patients are much more likely to adhere to recommendations made by their physician.


Component 2: Weekly Symptom Checks and Notifications

Each week, patients receive an automatically generated prompt on their smartphone at a designated time, with follow-up reminders if they do not respond. Prompts ask the patient to complete a five-item asthma questionnaire within a 48-hour window after which the questionnaire expires. (Patients preferred the term questionnaire to survey because the latter evoked associations with customer service.) As none of our interviewed patients expressed preferences for timing of the prompt, they are generated when convenient for the care manager (e.g., an LPN or medical assistant) to be available to call them during working hours. (No notifications are issued on Thursday or Friday to try to catch patient's issues during workdays.) After the patient completes the questionnaire, simple logic rules determine if the patient's symptoms are worse than the previous week, worse than baseline, or severe on any given question. If the criteria are met, the patient is given the option of requesting a call from a care manager, who works with the patient's physician ([Fig. 1]). We purposefully designed this workflow to be similar to how a care manager might handle a patient phone call about their symptoms so that workflow changes would be minimal. If symptoms meet more severe criteria, the care manager is auto-notified (i.e., patient is not given the option). However, the care manager has the option to disable the auto-notify feature for a period of time, thereby requiring the patient to request a phone call for severe symptoms. The ability to disable auto-notifications may prevent excess calls from the care manager to patients with severe baselines. The care manager will have access to a view of the patient's complete self-reported symptom history ([Fig. 2]). If a notification is sent, the app sends a brief follow-up survey to ask the patient if the call occurred and assess satisfaction.

Zoom Image
Fig. 1 Question 1 of ACM and example questionnaire completion page. ACM, asthma control measure.
Zoom Image
Fig. 2 Care manager's view (data are fictional).

We encountered two challenges in designing this component. First, some of the patients in our design sessions had difficulty understanding the concept of a baseline and said there was no typical week in terms of their symptoms. Furthermore, their baselines may change by the seasons. We therefore ask patients to choose as their baseline response symptoms on an average week in which they are not experiencing an asthma ”flare” and would probably not call their physician, and we acknowledge that these may not be exact. We did not attempt to design a way to create a new baseline based on season because patients found it challenging to think about this feature without having used the app. Second, some patients asked for a definition of an asthma attack. So as to avoid interrupting the flow of the questions, we included a definition at the beginning of the survey and gave the patients the option to stop showing it in the future.


Component 3: Patient Review of Symptoms

The app allows patients to review their symptom history at any time ([Fig. 3]). In our design sessions, patients generally agreed to the need to easily review their most recent 1 to 2 months of scores but have more of their previous data available on demand. However, there was a range in patients' views of the importance of this feature, with some finding it more valuable than others. We therefore developed an interactive widget allowing them to review their recent 2- and 6-month symptom scores, as well as the ability to view each question response in graphic form. The data points are color coded per severity according to our notification logic.

Zoom Image
Fig. 3 Patient's view of symptom graph (data are fictional, y-axis is modified ACM, blue dots are unchanged or improved symptoms; yellow are somewhat concerning; red are severe symptoms). ACM, asthma control measure.


Component 4: In-Person Visit

All patient-reported data are available to the physician through their EHR as a recent note ([Fig. 4]). Most patients emphasized that it would be important to them that their physicians were aware of their reported data during the in-person visit, and if the physicians were not aware, they might lose motivation to continue responding to the weekly questionnaires. Physicians also wanted the recent scores available. However, physician workflows vary and we found no clear place in this EHR that physicians consistently reviewed before a visit. All physicians we spoke with said that in the existing EHR, one place where they would likely notice the data before a visit would be a recent note, but they could not guarantee they would always notice it.

Zoom Image
Fig. 4 Summary of patient symptoms accessible to physicians in medical record (data are fictional).


Benefits and Burdens

As in other studies, we found that users weighed the benefit and burdens in their decisions to use novel functionality. We explicitly weighed those considerations in our design[23] [33] and attempted to maximize benefit while minimizing burden. A summary of the identified benefits and burdens to patients, clinicians, and provider organizations identified in our design sessions is displayed in [Table 4].


Anticipated Implementation Challenges

We have identified several potential challenges to implementing the intervention in routine clinical practice:

  1. The clinic must train a staff member to help patients download and use the symptom-monitoring app. This step may be doable without involvement of a staff member, but someone in the clinic would need to be able to answer patients' questions.

  2. The symptom-monitoring software would need to be integrated into a clinic's EHR such that it automatically produced a note summarizing the patient's recent symptoms or some other kind of electronic indication showing that new PRO data was available prior to a visit. (We plan to perform a feasibility test of use during the in-person visit using the “wizard of oz” or “fake backend” approach,[34] [35] [36] in which we manually paste a one-page summary of the patient's scores into the EHR as a recent note before each patient visit.)

  3. The PRO data would need to be available on the physician's mobile device, ideally through mobile access to their EHR. Several physicians requested this, but we are not certain if this is a critical requirement, at least today.

  4. The symptom-monitoring software would ideally also be integrated into a patient portal (a Web site or app that connects patients to their providers) so that the patient would not be required to download a separate app. We are not certain if this is a critical requirement because few of our design patients used a patient portal. However, adding such functionality might motivate more patients to sign up for patient portals and benefit from its other features.

  5. A care manager (e.g., LPN or medical assistant) in the clinic would need to respond to notifications and call patients.


Potential Unintended Consequences

We identified two main potential unintended consequences. First, some patients might become overreliant on the app and wait to call their physician until the app suggests it. To mitigate this concern, it is important that when patients are introduced to the intervention, they understand that the app will not make decisions for them and that they should always use their own judgment. We also include text in multiple places in the app that reminds patients they have the option of calling their doctor or going to the emergency room. In future versions, we expect that patient access to an updated asthma action plan may further address this concern. Second, some patients may request many calls from a care manager when the calls have questionable utility. If we find this during testing, we will develop additional strategies to address this concern.



We identified core components for implementing asthma symptom monitoring between visits as part of routine care using smartphone-based PROs. We achieved saturation of minimum intervention requirements with a modest number of patients and design sessions. Our findings provide an integrated, functional building block of interrelated features that can be tested, refined, and enhanced in a systematic fashion. Our approach to developing an intervention of starting small and incrementally adding new components may seem cumbersome for researchers eager to demonstrate outcomes. However, our results show that even for what may appear to be a simple intervention, critical insights for the design may arise through user testing, and an iterative user-centered approach can reveal these insights efficiently. This approach addresses the limitations of previous literature in which details on how the interventions contribute to study outcomes are not known and interventions involve many complex components. Interventions that do not take this incremental “bottom-up” approach and do not invest in user testing will more likely encounter barriers that could have been foreseen and addressed.[37]

In addition to the asthma-related mHealth apps that have been formally evaluated, there are many other apps available for download.[38] These apps contain a multitude of features, including medication tracking, symptom tracking, self-recorded asthma action plan, educational videos, guidance for which asthma medication to take, and peak flow measurement tracking. Though many of these features may be useful to patients, most lack detailed descriptions of their development process, evidence of ability to sustain engagement over time, or evidence of clinical utility. Furthermore, these apps are marketed primarily as direct-to-consumer products, with little clinical integration. We found in our user testing that a core requirement for patients to remain motivated was the way the app enhanced the patient–provider relationship, and so we began with the most basic functionality that would address that relationship while being consistent with clinical guidelines. We also specified the workflows that providers would need to adopt for this app to be effective. As we add features, we will build on this foundation and consider adding other features, such as those included in these apps, based on the results of our pilot test and further user testing.

Perhaps the major barrier we identified is the lack of a clear place in the EHR for entering PROs generated between visits where the physician would likely notice them. Our results suggest that a recent note is one possible location, but not all physicians look there before each visit. The need for such a location in the EHR may increase as more health data are captured in digital form between visits and made available in the EHRs. Previous studies that attempt to integrate PROs into clinical workflow for cancer and orthopedic care have centered on in-clinic patient reports or have used paper workarounds.[39] [40] [41] We hope that EHR designers and implementers will consider this need as a future design requirement and create a standardized location for these types of data, perhaps as part of an overview or agenda functionality.


The patients and clinicians we included represent a modest convenience sample from a single institution in one region. Patients were recruited through one physician and may not be representative of the greater population. However, our patients were demographically diverse, and we reached saturation of critical components. Second, the intervention may not be appropriate for all patients. Testing will help us understand what kinds of patients are suitable. Third, similar to findings with the asthma action plan, our designed intervention will likely not be successful if physicians are not supportive. Some physicians may require training to become familiar with measures of asthma control and additional support until they can experience the benefits. Finally, we changed our questionnaire items from monthly to weekly, which may affect the psychometric properties somewhat.


Future Enhancements

In our design process, users identified many intervention ideas that we deferred until after our planned feasibility trial because none of the users said they were essential to using the app. Many of these are supported by clinical guidelines and are candidates for future iterative development: diary functionality to track symptom flare-ups, variable timing of symptom questionnaire based on reported severity, support for patient-initiated symptom checks between weekly check-ins, integration of asthma action plans,[23] [42] [43] [44] tailored educational materials, medication regimen and reminders, and use of objective data (e.g., inhaler use, peak flow) to inform monitoring.[45] Other candidates for future features include digital self-management interventions that have proven effective in trials but have minimal uptake.[46] Additionally, if this intervention is widely adopted, the PROMs collected may prove useful for secondary purposes, such as evaluating the effect of other interventions on asthma symptoms, or comparing health systems performance for asthma care.



Through iterative design sessions, we identified four core components for routinely monitoring asthma symptoms that met the stated needs of patients and clinicians, and benefits and barriers of implementation to key stakeholders. The intervention's components form a starting point and can be pilot tested and incrementally expanded. By combining user-centered design and qualitative research methods, we developed a new mHealth intervention in a systematic and highly efficient way.


Clinical Relevance Statement

Recent systematic reviews of mHealth asthma interventions show inconsistent outcomes and little insight into success factors. Through iterative design sessions, we identified four core components for routinely monitoring asthma symptoms between visits via mHealth app installed on the patient's smartphones. This work begins to build a body of knowledge for how to use mHealth to incorporate PROs into routine care.


Multiple Choice Question

Where, in an electronic health record (EHR), is the optimal location to include patient-reported between-visit symptom data so that physicians will notice them before an in-person visit with a patient?

  • In a recent clinical note.

  • On a summary page.

  • In a special PRO module.

  • There is no optimal location.

Correct answer: The correct answer is D. Although some physicians may notice PRO in various parts of the EHR, there is no one designated location which physicians will always look prior to a patient visit. There is no industry-wide standard of EHR functionality to ensure certain data are on physicians' agendas for in-person visits.

Appendix A ACM Questions (Modified from Monthly to Weekly)
  1. How often did you have an asthma attack in the past week?

    • Not at all.

    • Once or twice.

    • Three to six times.

    • Once a day.

    • More than once a day.

  2. How often have you been awakened at night because of your asthma symptoms in the past week?

    • Never.

    • Once.

    • Twice.

    • Three or more times but not every night.

    • Every night.

  3. How much did your asthma interfere with your normal activities in the past week?

    • Not at all.

    • A little.

    • A moderate amount.

    • A lot.

  4. How often have you used a rescue inhaler that gives quick relief from asthma symptoms in the past week?

    • Never.

    • Once.

    • Two or more times but not daily.

    • Daily.

    • Several times a day, most days.

  5. How often did you have shortness of breath in the past week?

    • Not at all.

    • Once or twice a week.

    • Three to six times a week.

    • Once a day.

    • More than once a day.

Appendix B Heuristic Usability Assessment


We recruited three usability experts (two professional designers at RAND and one PhD informatician who has research expertise in usability). Experts were shown how to install the app, shown how to use it, and asked to review Nielsen's 10 usability heuristics ( They were then asked to identify as many usability issues as they could find over the course of 2 weeks and assign each one a severity score:

  • 0 = I don't agree that this is a usability problem at all.

  • 1 = Cosmetic problem only: need not be fixed unless extra time is available on project.

  • 2 = Minor usability problem: fixing this should be given low priority.

  • 3 = Major usability problem: important to fix, so should be given high priority.

  • 4 = Usability catastrophe: imperative to fix this before product can be released.


We received 20 responses with a mean of 1.9. Five responses received a score of 3 or 4:

  • Weekly notification did not appear. We believe this was because the user had not signed in properly. We added clearer instructions to make sure this would not be a problem in the future.

  • User was signed out in the middle of a survey with no visible indication. This occurred because the user started answering a questionnaire and then paused for an hour before returning to it. We added instructions that the user should answer all questions quickly and in one sitting. We plan to have the app automatically sign out in the future.

  • Color contrast was not strong enough in several places. We plan to revisit the color scheme specifically for phones with low brightness.

  • Questionnaire items lacked links to instructions or homepage. We added instructions that the user should answer all questions quickly and in one sitting.

  • Questionnaire items lacked exit link. We will consider adding an exit option pending further user testing.

We addressed the other less severe issues as much as possible with existing resources.


Conflict of Interest

Dr. Zimlichman is an advisor to Valera Health. Dr. Landman reports he is a senior editor at Ranked Health, a project that evaluates digital health apps, run by the Hacking Medicine Institute, a nonprofit organization. Dr. Bates is a coinventor on Patent No. 6029138 held by Brigham and Women's Hospital on the use of decision support software for medical management, licensed to the Medicalis Corporation. He holds a minority equity position in the privately held company Medicalis which develops web-based decision support for radiology test ordering. He serves on the board for SEA Medical Systems, which makes intravenous pump technology. He consults for EarlySense, which makes patient safety monitoring systems. He receives equity and cash compensation from QPID, Inc, a company focused on intelligence systems for electronic health records. He receives cash compensation from CDI (Negev), Ltd, which is a not-for-profit incubator for health IT startups. He receives equity from Enelgy which makes software to support evidence-based clinical decisions. He receives equity from Valera Health which makes software to help patients with chronic diseases. He receives equity from Intensix which makes software to support clinical decision making in intensive care. He receives equity from MDClone which takes clinical data and produces de-identified versions of it. Dr. Bates' financial interests have been reviewed by Brigham and Women's Hospital and Partners HealthCare in accordance with their institutional policies.


The authors thank Josie Elias for helping to facilitate implementation of the software at Brigham and Women's Hospital, and Alyson Youngblood, Chrissy Sovak, and Tiffani Bright for conducting the heuristic evaluation. This work was supported by the Agency for Healthcare Research and Quality grant #1R21HS023960.

Protection of Human and Animal Subjects

This study was reviewed by RAND and Partners Healthcare Institutional Review Board.

Address for correspondence

Robert S. Rudin, PhD
RAND Corporation
20 Park Plaza, Suite 920, Boston, MA 02116
United States   

Zoom Image
Fig. 1 Question 1 of ACM and example questionnaire completion page. ACM, asthma control measure.
Zoom Image
Fig. 2 Care manager's view (data are fictional).
Zoom Image
Fig. 3 Patient's view of symptom graph (data are fictional, y-axis is modified ACM, blue dots are unchanged or improved symptoms; yellow are somewhat concerning; red are severe symptoms). ACM, asthma control measure.
Zoom Image
Fig. 4 Summary of patient symptoms accessible to physicians in medical record (data are fictional).