Focus Section Health IT Usability: Applying a Task-Technology Fit Model to Adapt an Electronic Patient Portal for Patient Work

Abstract

Objectives Although electronic patient portals are offered by most health care organizations, poor usability and poor fit to patient needs may pose barriers to adoption. We collaborated with an academic hospital to conduct iterative user evaluation of a newly deployed portal designed to deliver inpatient data upon hospital discharge.

Methods Three evaluators applied heuristic usability evaluation and conducted 23 individual user testing sessions with patients with chronic disease or managing the care of family members with chronic disease. Evaluation and development/improvement were conducted iteratively. User testing and analysis of qualitative data were both conducted from the perspective of a task-technology fit framework, to assess the degree of fit between the portal and patient work.

Results Ability to complete health information management tasks, perceived usability, and positive comments from users improved over the course of the iterative development. However, patients still encountered significant difficulties accomplishing certain tasks such as setting up proxy accounts. The problems were most severe when patients did not start with a clear understanding of tasks that they could accomplish. In exploring the portal, novice users frequently described anecdotes from their own medical history or constructed fictional narratives about a hypothetical patient.

Conclusion Chronic illness imposes a significant workload on patients, and applying a task-technology framework for evaluation of a patient portal helped improve the portal's fit to patient needs. However, it also revealed that patients often lack a clear understanding of tasks that would help them accomplish personal health information management. Portal developers may need to educate patients about types of patient work involving medical centers, in a way that developers of clinical information systems do not need to do. An approach to doing this might be to provide narratives about hypothetical patients.

Keywords

electronic patient portal - health information management - chronic disease - workload - usability - task-technology fit

Background and Significance

As a result of the HITECH Act of 2009, which incentivized eligible hospitals and providers to make electronic data directly available to patients,[1] electronic patient portals are now offered by most health care organizations and are becoming more popular.[2] [3] Portals can provide individualized information to patients on their medical conditions and medications, laboratory results, their care teams, their hospital stays, and their expected care plans. These digital communication solutions usually also allow for secure, asynchronous patient–provider communication, and other functions such as refilling medications, scheduling appointments, or paying bills.[4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] People who use e-health resources feel better prepared for clinical encounters, ask more relevant questions, know more about their health care, and are more likely to take steps to improve their health.[19] Limited evidence has also linked the use of portal features to improved outcomes for chronic conditions such as diabetes, depression, and hypertension.[20] [21] [22] [23] Over the long term, it is also hoped that portals might engage patients and improve patient activation, which is associated with adoption of healthy behaviors, better disease self-management, increased health information seeking,[24] and better health outcomes.[25]

However, despite these potential positive outcomes, it is well established that racial/ethnic minorities and patients with limited health literacy, income, and education are significantly less likely to use portals.[26] [27] [28] [29] [30] [31] Although the overall use of digital health information management tools has increased, a segment remains disengaged.[32] As described in the DeLone and McLean model, an information system cannot be expected to produce its desired outcomes unless it is perceived as usable and contributes to user satisfaction.[33] Usability (the capacity of the system to allow users to carry out their tasks safely, effectively, efficiently, and enjoyably[34]) has been reported as a barrier to use of patient portals.[35] [36] [37] [38] [39] [40] [41] Usability is important to users with high levels of education and routine Internet use, who want applications that meet usability standards similar to those in retail industries.[32] It may be even more important to users with less education and computer expertise, who have less awareness of technology and limited technology skills.[32] [42] [43] [44] Design features that improve usability and perceived organization and clarity influence patient portal adoption.[45] [46] Specific usability barriers in portals include difficulty navigating through functions, confusing terms, poor display of information, confusing functionality, and long times to perform a task.[41] [47] [48]

Usability is often studied as a function of an individual user interacting with the technology interface. However, as multiple researchers have previously pointed out, patient portals and other information technologies are used by patients in the context of the work that their medical conditions impose.[49] [50] [51] Usability can be improved only with an understanding of the tasks that patients need to perform in the course of their “illness work.”[52] We therefore approached usability for patients through the lens of task-technology fit models,[53] specifically Ammenwerth et al's “fit between individuals, task, and technology” (FITT) framework[54] ([Fig. 1]). According to FITT, technology adoption depends on the fit between the attributes of the users (e.g., skills, motivation, and knowledge), the attributes of the technology (e.g., usability, functionality, performance), and the attributes of the tasks (e.g., their organization and complexity).

Fig. 1 Adapted from Ammenwerth's fit between individuals, task, and technology (FITT) model, our conceptual model links electronic patient portal adoption with fit among personal health information management (PHIM) tasks, patients, and the portal. In this study, we assessed five PHIM tasks and collected information about patients, the portal, patient-task fit, patient-portal fit, and task-portal fit.

In previous work, we had identified types and characteristics of personal health information management (PHIM) tasks that patients with chronic illnesses do as part of their illness work.[49] Some of the tasks described most frequently included checking/tracking personal medical indicators such as laboratory results, sharing medical record information from one doctor to other health care providers, and searching for background medical/health information.[49] In the current project, we evaluated an electronic patient portal to determine how easily it could be used by diverse patient populations to support these sorts of PHIM tasks. We collaborated with a large academic hospital in New York City, New York-Presbyterian, which had recently relaunched an electronic patient portal and was interested in improving its usability. The portal, www.myNYP.org, provides patients with inpatient data such as laboratory results, procedures, and care instructions after their hospital discharge, and is currently available in English, Spanish, and Mandarin versions. (An additional feature that we did not evaluate was an inpatient segment that delivers up-to-date information to patients during their hospital stay.[55])

Objectives

Our objectives were to identify task-technology fit problems and usability challenges in the novel portal, recommend solutions, and to evaluate whether the recommended design changes improved usability. We conducted this study in three stages. First, we applied heuristic usability testing to explore the technology and identify and remediate obvious usability problems. Second, we conducted user testing to assess fit between the PHIM tasks and the electronic patient portal (task-technology fit) as well as fit between patients and PHIM tasks (patient-task fit) and between patients and the technology (patient-technology fit, specifically perceived usability); specific barriers to the PHIM tasks were presented to the development team for remediation. Third, we conducted a final round of user testing to determine how well the changes improved task-technology fit and patient-technology fit (perceived usability).

Methods

Phase 1

We launched this project with a heuristic usability evaluation, because this technique is an efficient way of identifying characteristics of the technology and remediating obvious usability barriers before time-intensive user testing is started.[56] Before the portal went live, three evaluators applied a heuristic usability checklist developed by Zhang et al (built upon work by Nielsen and Schneiderman).[56] Each evaluator individually analyzed the portal, identified heuristics that were violated, and assigned a severity score to each heuristic that was violated. After several rounds of discussion, a consensus usability score was reached. Based on the observations and consensus scores, a report of recommendations on improvements was presented to the portal development team. After the development team selected recommendations to implement on the basis of feasibility and institutional priorities, heuristic evaluation was conducted a second time using the same procedures.

Phase 2

Phase 2 was user testing in which users were invited to perform typical health information management tasks[49] and to determine how well they could use the portal to perform these tasks.

For user testing, inclusion criteria were: 18 to 95 years of age; ability to speak in English; ability to use a computer and navigate through Web sites; and either having a chronic medical condition or being a caregiver for a person with a chronic medical condition. Patients were recruited with multiple methods, including flyers in hospital clinics, an online sign-up form on the existing patient portal, and direct approach in waiting areas. Patients who met the inclusion criteria and were interested in participating were introduced to the study using a standardized script. Informed consent was obtained. At the end of the session, the patient was given a gift card worth US $10. The project was approved by the Weill Cornell Institutional Review Board.

Participants were invited to log in to the portal using a fictitious account that was prepopulated with patient data. The participants were first invited to explore the portal for its various functionalities and affordances, then invited to complete five tasks chosen to be representative health information management tasks[49]:

1. Reviewing personal information from a previous hospitalization.

2. Creating a report to be given to a physician at another organization.

3. Reading up about a specific medical condition.

4. Changing the account password.

5. Enabling a family member's access to the account (proxy access).

Screen activity was recorded through an audio recording and screen-capture software (Morae, TechSmith, Inc., Okemos, Michigan, United States). An additional layer of rich qualitative data was collected by inviting the participants to provide a continuous verbal accounting of their thoughts as they completed the tasks, a procedure known as the concurrent “verbal protocol” or “think-aloud protocol” procedure. These protocols have been demonstrated to capture cognitive processes during problem solving.[13] In addition to performing each task, participants were invited to comment on how well the technology allowed them to perform the task. Audio recordings were transcribed and analyzed.

Sampling was wrapped up after 12 participants, when data analysis (described below) suggested we had achieved saturation (new interviews were not producing substantively new themes).[57] [58] However, after we concluded sampling, we excluded one interview because the participant's demographic questionnaire showed that the participant did not meet all the inclusion criteria. Our final totals throughout the rest of the paper therefore refer to 11 participants in this phase.

At the end of the user testing session, participants were invited to complete a demographic questionnaire, a standard usability questionnaire,[15] the Single Item Literacy Screener (SILS),[59] and the Subjective Numeracy Scale (SNS), a self-report measure of quantitative ability and preferences.[60]

Phase 3

Phase 2 recommendations were presented to the portal development team, and after selective implementation, an additional group of users was recruited to conduct user testing, using the same procedures and interview guide. We targeted 12 users for this phase to match the 12 that had been interviewed in the earlier phase.

Data Analysis

Data from user testing underwent two types of analysis. First, Morae videos were reviewed to assess whether each task was completed by the participant (task-technology fit; [Fig. 1]). Tasks were categorized as completed easily; completed with assistance; not completed; or replaced with different task to accomplish the goal. An example of this substitute task was taking a screenshot of laboratory values to share with a doctor rather than creating a continuity of care document (CCD) report.

In a clinical setting, such substitute tasks are generally classified as “workarounds,” nonstandard procedures used by professionals to accomplish a task in situations when deficiencies in system design create barriers to using the standard procedure.[61] However, in our observations, it was clear that in many cases the patients did not understand the health information task in the first place, and specifically did not know the standard/recommended procedure for accomplishing the task. Given these ambiguities, we opted to call them “substitute tasks” instead of workarounds.

In the second part of the data analysis, audio recordings of the concurrent verbal protocols were transcribed and coded qualitatively to assess task-technology fit, user-technology fit, and user-task fit ([Fig. 1]). The first two transcripts were reviewed by three coinvestigators (J.S.A., S.A., B.H.) to develop the initial codebook. After that, each transcript was coded independently by at least two and in some cases three of the researchers, who met to establish consensus on each. As the Ammenwerth et al FITT framework[54] focuses on the relationship between the technology, the task, and the user, we developed the initial codebook to focus primarily on mismatches between the user and the technology and the user and the task ([Fig. 1]). A third category, mismatches between the technology and the task, was also used to identify occasional Web site bugs, which are not presented here but were presented to the portal development team for remediation. Additional codes, such as the emotional valence expressed by the participants, were developed through inductive thematic analysis.[62]

System usability scores, which did not satisfy the Shapiro–Wilk test for normality, were compared with the Wilcoxon rank-sum test, using R version 3.4.1.

Results

Heuristic Evaluation

In the pre-go-live heuristic evaluation ([Fig. 2]), the most severe usability barrier identified was Failure to use users' language, with a severity score of 4 out of possible 5. An example was that instead of inviting patients to create a report of their medical information, the Web site invited patients to click a button labeled “CCD export.” The most severe potential error was the availability of an option for patients to accidentally delete the entire medical record within the portal.

Fig. 2 Heuristic evaluation, performed by the researchers, showed gains in some domains as well as losses in others. Losses were largely attributable to the introduction of new functionality and reorganization during the development phase.

The portal development team implemented several recommendations that were feasible, could be accommodated with developer workload, and met institutional priorities. However, simultaneously new functionality was implemented, and functions were reorganized. For example, a bill-paying function was updated and replaced. As a result, even after revisions, the second heuristic evaluation found that Failure to use users' language remained the most severe problem with a score of 4. After the first evaluation, the most severe errors were corrected and additional explanatory material was added, creating improvements in Error prevention and Help and documentation. Conversely, the additional content meant that three domains scored worse the second time around. (Additional improvements and updates were made after the second heuristic evaluation, as part of the ongoing development cycle.)

User Testing

In the user testing phases, the 23 participants represented a range of ages, races and ethnicities, and insurance categories ([Table 1]). Participants had chronic conditions including type 1 diabetes, type 2 diabetes, and cancer, or were caring for family members with conditions such as ulcerative colitis and thalassemia. Only two reported needing assistance with medical documents (it is likely that patients with lower levels of literacy were screened out because of inability to use computers). Self-reported numeracy ranged from 17 to 48 on the scale of 8 (low numeracy) to 48 (high numeracy); 38% reported a score of 30 or less.

Table 1 Participant demographics

	Group 1		Group 2
	N	%	N	%
Number of females	10	83	10	83
Age category
18–40 y	7	58	6	50
41–64 y	4	33	6	50
65+ y	1	8	0	0
Education level
High school grad or GED	3	25	3	25
Some college	2	17	3	25
College degree	4	33	4	33
Master's degree or higher	3	25	2	17
Race/ethnicity
White	9	75	3	25
Hispanic/Latino	0	0	5	42
Asian	2	17	1	8
African-American	0	0	1	8
Other	1	8	0	0
No response	0	0	4	33
Insurance coverage
Private insurance	5	42	4	33
Medicare	2	17	1	8
Medicaid	3	25	2	17
Other	2	17	0	0
Marital status
Married	6	50	6	50
Not married	6	50	1	8
No response	0	0	5	42
Single Item Literacy Scale
Any literacy problem	1	8	1	8
No literacy problem	6	50	8	67
No response	5	42	3	25
Subjective Numeracy Scale
Below 30 (lower numeracy)	4	33	8	67
30+ (higher numeracy)	5	42	7	58

Abbreviation: GED, General Educational Development.

Note: Numbers do not sum to 100% because of rounding.

When invited to complete the five tasks, changing the account password was the only task that was completed easily by almost all of the participants ([Fig. 3]). This was probably because for account settings, standards were familiar from Web commerce sites.

Fig. 3 Patients' ability to complete several personal health information management (PHIM) tasks improved after portal redesign.

User performance on most other tasks improved noticeably in phase 3, after design changes suggested in phase 2. At this stage, most participants were able to identify and review data from previous hospitalizations (task 1), and create a report for an external physician (task 2). In phase 2, many patients had difficulty finding the medical encyclopedia resource because it was labeled “Medline Plus” (which is the actual name of the free National Library of Medicine resource that was used, www.medlineplus.org), but in phase 3, most patients were able to find it and navigate to it after it was relabeled “Health Topics.”

Almost none of the patients were able to grant proxy access to a family member. Usually, this was because prior to meeting with us, they did not recognize this could be done. When given a scenario about sharing data with a family member, many responded by offering to share their username and password (which we classified as a “substituted task”). In addition to this conceptual problem, patients also encountered difficulties from the fact that the account enabled both obtaining access to another person's account as well as granting access to another person. Because both functions were unfamiliar to most patients, patients frequently mistook one function for the other. These functions were labeled as “Add Family Member” and “Health Record Sharing,” respectively.

The average System Usability Scale (SUS) score improved from 69.2 (SD: 20.5) among the first group of participants to 81.9 (18.2) among the second group of participants who saw the redesigned Web site (p = 0.049).

Themes

Five overall themes ([Table 2]) arose from the examination of task-technology fit and the analysis of the qualitative data.

Table 2 Theme summary

Theme	Participant quotes	Identifier
Theme 1: Mismatch between user and technology	I don't know what Create a Medical Report means...I'm not familiar with that report making process. I've never done that before	PID-01, 27 y. College educated
	[Reads aloud] “Continuity of care.” Yeah, I'll have to learn about continuity of care from someone, I think. I mean it just hits you foreign. You know, there is no continuity of – to me, continuity is some kind of narrow, real narrow definition	PID-03, 74 y. College educated
	[Reads aloud] “CCD export.” What does CCD mean? I don't understand. What is IPHR? Why is everything so abbreviated over here? That's really annoying.	PID-04, 26 y. College educated
Theme 2: Mismatch between user and tasks	What's “Create a Medical Report”? What would you use that for?	PID-22, 53 y. College educated
	All I would want to know is what are my laboratory results. I don't want to tell them when I did it, where I did it, who ordered it. I wouldn't want to tell them all this information. This looks like it's for a doctor.	PID-06, 31 y. College educated
	Well, most people aren't going to know the name of the test they took. I mean a lot of those tests are medical names. I would know the laboratory I went to, maybe the date and time, but the name – the result name. value – I wouldn't know any of that information	PID-13, 44 y. College educated
Theme 3: Seeking help	I don't know. I'm so bad. Is this a map to scroll down?	PID-11, 47 y. College educated
	Could this be it? ... I'm sorry, folks… I'm not really sure this is it. So let me just look at these categories again...no, I'm going to “Ask the Doctor for Help”	PID-19, 40 y. High school/GED
	I don't know if I'm navigating very seamless on this	PID-22, 53 y. College educated
Theme 4: Using system to learn	Okay, discharge diagnosis. Okay, so at least, like, I can then go back and confirm that, you know, whatever I was admitted for is the same thing I was discharged with so there's no, like, billing error	PID-04, 26 y. College educated
	Hospital Record. You see that? That's smart. So you can share with your dad right there	PID-08, 18 y. High school/GED
	This research help topics, it's great. It's great for the person who is newly diagnosed. I wish I had this early on. That's a great one	PID-16, 41 y. College educated
Theme 5: Importance of storytelling	Request an appointment. Okay...And that would be really nice because the doctor I see actually here, usually I have to call, usually the doctor's nurse is not available when you call. Then they call you back. They do the appointment, but it's not like very easy...so yeah, this would be really helpful, what I'd like to get the appointment for a specific doctor	PID-17, 49 y. College educated
	Well I'm just looking over the disorders, conditions, you know, the health issues. And my son particularly has Crohn's disease, so I was just looking to see if I find it here	PID-14, 46 y. High school/GED
	...so I would like to know, like, you know, my mom's medical history. Like, I can't remember everything about it, so you having that record and I can link into it so whenever a physician then asks me what is your mother's – you know, do you have diabetes in your family, I'm, like, I don't know, I can't think of it right now, I'll just go into the portal and look up my mom. That'd be nice	PID-04, 26 y. College educated

Abbreviation: GED, General Educational Development.

Theme 1: Mismatch between User and Technology

Many of the patients, who had considerable experience with chronic conditions, displayed fairly nuanced mental models of their (or their family member's) condition.[63] [64] However, in navigating the patient portal, some had difficulty forming a clear understanding of how the electronic portal worked. When there were mismatches between user mental models and the technology, they manifested primarily as vocabulary misunderstandings, as portal functionality that did not perform as the patient expected, and as requests for clarification and help.

All 23 participants encountered problems with portal vocabulary on multiple occasions, either misinterpreting terms or encountering terms that they did not understand at all. Especially in the latter case, this led to frustration and self-doubt. A college-educated participant said that she would need someone to help her interpret everything in the portal: “I don't know. I don't know what this all means. You know, unless I'm sitting there Googling every word, I wouldn't know what this means and it's not really fair for me.” Most of these confusing terms were not medical but rather health information management terms, such as “HIM” referring to health information management (and meaning more specifically the hospital office that handles health information management requests), and “CCD,” the HL-7 term that refers to the continuity of care document.

All of the participants also encountered situations in which the functions did not perform as they expected. One example was that when patients clicked a button to create a report for their doctor, many did not recognize that the report appeared as a separate window (a PDF) in front and obscuring the view of the portal. Patients who did not recognize that they were in a different window had great difficulty navigating back to the remainder of the portal.

Theme 2: Mismatch between User and Tasks

Despite the patients' understanding of their medical conditions, most (19 out of 23) did not have a very concrete understanding of health information management tasks. For example, many patients had encountered the need to share their medical data, but few were aware it was possible to export electronic copies of records or share their medical record with family members or proxies. This sometimes led to mixed reactions when the participant discovered portal features that did not align with their expectations and/or beliefs.

Proxy access was a particular challenge. Most participants did not know that this was possible, although, many when they learned about it agreed that it could be very useful. Nevertheless, as described earlier, almost none of them were able to successfully use the portal to grant proxy access to a (fictitious) family member. Furthermore, not all participants welcomed the possibility of proxy access. One man said, semi-jokingly, “Oh, wow. I don't know if I like that…. certainly not full access to my records, because if myself and my brother were in line to get a ten-million-dollar inheritance, he might want to see me dead, or I might want to see him dead.”

Some patients recognized the problems inherent in having medical records scattered across multiple doctors, and were interested in consolidating data and sharing it. Yet, the portal functions designed to help patients handle this problem did not appear to be intuitive. For example, a portal function that allowed patients to input their own data from other doctors was considered strange by almost all of the patients who discovered it. One woman said that patient-entered laboratory data was unlikely to be accurate, and was concerned about the workload involved. “All I would want to know is what are my lab results. I don't want to tell them when I did it, where I did it, who ordered it. I wouldn't want to tell them all this information. This looks like it's for a doctor.” Another example was that many patients who agreed that they would want to share their hospitalization data with another doctor outside the system did not understand the concept of developing a report (using the CCD format). Instead, many substituted a different task to accomplish the goal, such as taking notes or screenshots of laboratory values.

Theme 3: Seeking help

The mismatches described under Theme 1 and Theme 2 meant that patients frequently asked the interviewer for assistance understanding terms or tasks. In some cases, the questions were needed because there was insufficient guidance on the portal. One man said, “I'm not really sure this is it. So let me just look at these categories again...no, I'm going to ‘Ask the Doctor for Help.’” Many participants who became enthusiastic about the portal during the user testing asked to be helped through the process of signing up for their own account.

However, others immediately asked for help without searching for instructions or attempting the task. For example, when one woman was asked to do the proxy access task, she said, “I don't know, because would I have to give them my password and login for them to log in to see the information? Or is there some way I could send information to their e-mail or something like that?”

Although some patients expressed frustration at the Web site (such as the woman cited earlier who said, “it's not really fair for me”), many instead blamed themselves for failing to understand or complete a task. For instance, one man took the responsibility of not completing a task completely upon himself by saying, “I may have a problem. I didn't look at this line.” Another woman said, “I mean, I feel like I'm letting you down…” Others blamed both: “One thing I don't like is having to hit the Back button all the time to get the prompts.... unless there's another way that I'm not seeing to get to it.”

Theme 4: Using the System to Learn

As the participants explored the portal, they discovered and tested new functions. In particular, most spent several minutes at the beginning of each session exploring the menu options and clicking large buttons on the splash page to see what they would do. One participant, looking at a shared record, realized what it was for, and said, “So my son could be in it and I could be in it, too.” Another, viewing a list of previous hospitalizations in the portal, noted, “And it gives you a list of all the hospital records. So if I wanted to print my records, I could just go in here and get a copy of them myself, right?” later adding, “Oh, I see. Okay, and it tells you all the dates that you were admitted.” Generally, these discoveries led to positive reactions. One person, discovering the “Health Topics” resource, said “If I was to type in ‘diabetes,’ I would expect information about it – which it does – and then all these various links to it, which is very helpful.” After initial exploration, one person said approvingly, “It's a good, useful access for a person at home or anywhere now. You're easily able to go on your phone and log in and see what's going on.” There were, however, occasional negative responses. One woman, who was interested to learn that she could add previous laboratory tests to her portal record, said, with disappointment, “So I would have to know the name of the test,” later adding, “not everyone knows the name of the test they took.” Using the system to learn about the system can be considered a way of improving user-technology fit by changing the user's own understanding.

Theme 5: Importance of Storytelling

Almost all of the participants (22 of 23) used personal anecdotes as a way of understanding the portal's functions, particularly new functions that they were previously unfamiliar with. For example, one woman with a child with chronic illness described a terrible recent year in which her son had had to be hospitalized 11 different times, and talked about how the portal account could be extremely useful for her to go back to look at his laboratory data from those hospitalizations. Another, while reviewing the appointment list in the portal, recounted how she had once confirmed multiple medical appointments for her condition, only to receive an automated phone call saying that they were at a different time. Another woman looking at the portal for the first time said that creating a story would help her decide what to look at in the portal: “I'm just trying to make a scenario while I'm at the hospital, what I would want to know.”

Discussion

Chronic illness imposes a significant workload on patients and caregivers, who need to learn to manage physician appointments, disease education, and self-management tasks, and even perform the “emotional work” of support and reassurance.[51] [52] An important component of illness work is health information management, including such tasks as tracking personal medical data and transferring medical information from provider to provider.[49]

Therefore, in evaluating an electronic patient portal, we applied a task-technology fit framework to improve not only usability but also to match the patient work.[54] The FITT framework identifies three important relationships to examine: the user-technology fit, the user-task relationship, and the task-technology fit. In applying this framework, we identified several areas in which a patient portal did not fully match the needs of users, including the use of technical health information management terms and individual functions that did not produce the results that patients expected. Remediating these areas improved patients' ability to perform several common health information management tasks with the portal.

However, our work on the user-task relationship also revealed that patients often lack a clear understanding of the component of illness work that involves health information management. For example, patients told us that their illness work required them to share data from organization to organization or within their family. However, few recognized that their medical record could be transformed into a shareable electronic copy, or that their medical record account could be shared directly with family members. Without this conceptual understanding of the task already in place, it was very difficult for the portal to do a good job of supporting the user in accomplishing the task. Portal developers may need to take responsibility for educating patients—not about patient work, but rather about unfamiliar ways that portals can support patient work. Generally, when a task-technology fit model is applied to a sociotechnical situation (such as the use of an electronic health record [EHR]), there is an assumption that the users agree upon and understand the dimensions of the task. This may not be true in the case of patient work.

Our work suggests that an effective way to demonstrate the link between portal functionality and patient work might be through narratives of fictional or archetypal patients. We found that many patients, when learning about the portal, improved their own understanding by linking portal functionalities to anecdotes from their own illness course or constructing hypothetical narratives about fictitious patients. Developers of patient portals might consider text or video narratives to help patients develop solid mental models of portal functions. “Personas,” profiles of archetypal users accomplishing personally relevant tasks, are used in user-centered design to help developers orient themselves toward the needs of users.[65] Such personas could possibly also be transformed into helpful educational tools for patients themselves.

User-centered product design involves understanding the needs, values, and abilities of users to improve the quality of users' interactions with and perceptions of the technology. In this study, we explored the views and needs of patients in iterative phases toward the development of a patient portal. A significant conclusion of this work is that incorporating the perspectives and needs of potential portal users will allow the system to evolve in a more meaningful way for the target population. Developing context-specific features based on user input led to improved acceptance and more positive reactions from the users, which in turn has implications for mass adoption and use.

Limitations

This study should be interpreted in light of several limitations. First, our study sample was selected through convenience sampling and may not be representative of larger target populations; specifically, patients who responded to our invitation to test the portal already had familiarity with computers and felt comfortable using them. This limits generalizability to less computer-literate populations. Second, we included only individuals who themselves had chronic conditions or cared for family members with chronic conditions; this was because previous work by ourselves and others shows that portals are more frequently used by individuals with medical conditions.[28] [66] However, this may limit applicability of our findings to healthier individuals. Third, although we were able to recruit participants with low levels of numeracy, our sample did not include individuals with very low levels of literacy, probably because the computer literacy requirement ended up screening out very low-literacy individuals. We also conducted all interviews in English, meaning that all participants were comfortable communicating in English. These two elements are likely to limit generalizability to the least literate populations as well as those with limited English proficiency. Finally, although the sample size was determined by saturation in data analysis, the possibility of larger samples leading to different conclusions should not be overlooked.

We conducted this evaluation from the perspective of the FITT model, a validated model showing that as these three dimensions of fit improve, technology adoption increases.[54] In the current study, we were able to show improvements in several of these fit dimensions, but demonstrating a quantitative increase in user adoption was beyond the scope of the study, which is an inherent limitation of the qualitative approach we used.

Conclusion

Chronic illness imposes a significant workload on patients, and applying a task-technology framework to evaluate a patient portal helped improve the portal's fit to patient needs. However, it also revealed that patients often lack a clear understanding of specific tasks that would help them accomplish PHIM work. Portal developers may need to take responsibility for educating patients about patient work, in a way that developers of clinical information systems do not need to do. In addition, patient understanding of portals may be improved by providing narratives about hypothetical patients.

Clinical Relevance Statement

Electronic patient portals may fail to achieve their objectives of empowering patients if they are not designed with patient needs in mind. We demonstrate that evaluating a patient portal from a task-technology fit perspective can lead to substantive improvements in patients' ability to accomplish health information management tasks.

Multiple Choice Question

A task-technology fit model demonstrates that, to improve the odds of technology adoption:

• Improving usability is sufficient

• Improving user training is sufficient

• Improving task-technology fit is sufficient

• Improving fit between users, tasks, and technology is sufficient

  Correct Answer: The correct answer is option d. Task-technology fit models, such as those of Goodhue and Ammenwerth, actually focus on the three-way fit between the person using the technology, the task they are trying to accomplish, and the technology being used to accomplish it. This perspective demonstrates that usability is only part of the solution. Historically, TTF models have generally been used in professional or workplace settings, but in the current project, we applied these concepts to patient work.

Conflict of Interest

None.

Protection of Human and Animal Subjects

This study was approved by the Weill Cornell Institutional Review Board.

• References

• 1 Blumenthal D. Stimulating the adoption of health information technology. N Engl J Med 2009; 360 (15) 1477-1479
• 2 Ancker JS, Hafeez B, Kaushal R. Socioeconomic disparities in adoption of personal health records over time. Am J Manag Care 2016; 22 (08) 539-540
• 3 Ancker JS, Silver M, Kaushal R. Rapid growth in use of personal health records in New York, 2012-2013. J Gen Intern Med 2014; 29 (06) 850-854
• 4 Lin CT, Wittevrongel L, Moore L, Beaty BL, Ross SE. An Internet-based patient-provider communication system: randomized controlled trial. J Med Internet Res 2005; 7 (04) e47
• 5 Ammenwerth E, Schnell-Inderst P, Hoerbst A. The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res 2012; 14 (06) e162
• 6 Wilcox LG, Gatewood J, Morris D, Tan DS, Feiner S, Horvitz E. Physician attitudes about patient-facing information displays at an urban emergency department. AMIA Annu Symp Proc 2010; 2010: 887-891
• 7 Card RF. Critically Thinking about Medical Ethics. Upper Saddle River, NJ: Prentice Hall; 2004
• 8 Cimino JJ, Patel VL, Kushniruk AW. What do patients do with access to their medical records?. Stud Health Technol Inform 2001; 84 (Pt 2): 1440-1444
• 9 Collins SA, Vawdrey DK, Kukafka R, Kuperman GJ. Policies for patient access to clinical data via PHRs: current state and recommendations. J Am Med Inform Assoc 2011; 18 (Suppl. 01) i2-i7
• 10 Greenhalgh T, Wood GW, Bratan T, Stramer K, Hinder S. Patients' attitudes to the summary care record and HealthSpace: qualitative study. BMJ 2008; 336 (7656): 1290-1295
• 11 Hassol A, Walker JM, Kidder D. , et al. Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging. J Am Med Inform Assoc 2004; 11 (06) 505-513
• 12 Johnson KB. Project HealthDesign: advancing the vision of consumer-clinician-computer collaborations. J Biomed Inform 2010; 43 (5, Suppl): S1-S2
• 13 Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A research agenda for personal health records (PHRs). J Am Med Inform Assoc 2008; 15 (06) 729-736
• 14 Nazi KM, Hogan TP, Wagner TH. , et al. Embracing a health services research perspective on personal health records: lessons learned from the VA My HealtheVet system. J Gen Intern Med 2010; 25 (Suppl. 01) 62-67
• 15 Ralston JD, Carrell D, Reid R, Anderson M, Moran M, Hereford J. Patient web services integrated with a shared medical record: patient use and satisfaction. J Am Med Inform Assoc 2007; 14 (06) 798-806
• 16 Reti SR, Feldman HJ, Ross SE, Safran C. Improving personal health records for patient-centered care. J Am Med Inform Assoc 2010; 17 (02) 192-195
• 17 Tang PC, Lee TH. Your doctor's office or the Internet? Two paths to personal health records. N Engl J Med 2009; 360 (13) 1276-1278
• 18 Halamka JD, Mandl KD, Tang PC. Early experiences with personal health records. J Am Med Inform Assoc 2008; 15 (01) 1-7
• 19 Ricciardi L, Mostashari F, Murphy J, Daniel JG, Siminerio EP. A national action plan to support consumer engagement via e-health. Health Aff (Millwood) 2013; 32 (02) 376-384
• 20 Goldzweig CL, Orshansky G, Paige NM. , et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013; 159 (10) 677-687
• 21 Simon GE, Ralston JD, Savarino J, Pabiniak C, Wentzel C, Operskalski BH. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med 2011; 26 (07) 698-704
• 22 Lau M, Campbell H, Tang T, Thompson DJS, Elliott T. Impact of patient use of an online patient portal on diabetes outcomes. Can J Diabetes 2014; 38 (01) 17-21
• 23 McMahon GT, Gomes HE, Hickson Hohne S, Hu TM, Levine BA, Conlin PR. Web-based care management in patients with poorly controlled diabetes. Diabetes Care 2005; 28 (07) 1624-1629
• 24 Fowles JB, Terry P, Xi M, Hibbard J, Bloom CT, Harvey L. Measuring self-management of patients' and employees' health: further validation of the Patient Activation Measure (PAM) based on its relation to employee characteristics. Patient Educ Couns 2009; 77 (01) 116-122
• 25 Remmers C, Hibbard J, Mosen DM, Wagenfield M, Hoye RE, Jones C. Is patient activation associated with future health outcomes and healthcare utilization among patients with diabetes?. J Ambul Care Manage 2009; 32 (04) 320-327
• 26 Sarkar U, Karter AJ, Liu JY. , et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010; 15 (Suppl. 02) 183-196
• 27 Sarkar U, Karter AJ, Liu JY. , et al. Social disparities in internet patient portal use in diabetes: evidence that the digital divide extends beyond access. J Am Med Inform Assoc 2011; 18 (03) 318-321
• 28 Ancker JS, Barrón Y, Rockoff ML. , et al. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med 2011; 26 (10) 1117-1123
• 29 Smith SG, O'Conor R, Aitken W, Curtis LM, Wolf MS, Goel MS. Disparities in registration and use of an online patient portal among older adults: findings from the LitCog cohort. J Am Med Inform Assoc 2015; 22 (04) 888-895
• 30 Goel MS, Brown TL, Williams A, Hasnain-Wynia R, Thompson JA, Baker DW. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med 2011; 26 (10) 1112-1116
• 31 Yamin CK, Emani S, Williams DH. , et al. The digital divide in adoption and use of a personal health record. Arch Intern Med 2011; 171 (06) 568-574
• 32 Greenspun H, Thomas S, Scott G, Betts D. Health Care Consumer Engagement No “one-size-fits-all” Approach. Washington, DC: Deloitte Center for Health Solutions; 2015
• 33 DeLone WH, McLean ER. The Delone and McLean model of information systems success: a 10-year update. J Manage Inf Syst 2003; 19: 9-30
• 34 Preece J, Rogers Y, Sharp H, Benyon D, Holland S, Carey T. Human-Computer Interaction. Essex, United Kingdom: Addison-Wesley Longman Ltd.; 1994: 775
• 35 Turvey C, Klein D, Fix G. , et al. Blue Button use by patients to access and share health record information using the Department of Veterans Affairs' online patient portal. J Am Med Inform Assoc 2014; 21 (04) 657-663
• 36 Haggstrom DA, Saleem JJ, Russ AL, Jones J, Russell SA, Chumbler NR. Lessons learned from usability testing of the VA's personal health record. J Am Med Inform Assoc 2011; 18 (Suppl. 01) i13-i17
• 37 Goel MS, Brown TL, Williams A, Cooper AJ, Hasnain-Wynia R, Baker DW. Patient reported barriers to enrolling in a patient portal. J Am Med Inform Assoc 2011; 18 (Suppl. 01) i8-i12
• 38 Georgsson M, Staggers N. Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. J Am Med Inform Assoc 2016; 23 (01) 5-11
• 39 Hung M, Conrad J, Hon SD, Cheng C, Franklin JD, Tang P. Uncovering patterns of technology use in consumer health informatics. Wiley Interdiscip Rev Comput Stat 2013; 5 (06) 432-447
• 40 Berry DL, Halpenny B, Bosco JLF, Bruyere Jr J, Sanda MG. Usability evaluation and adaptation of the e-health Personal Patient Profile-Prostate decision aid for Spanish-speaking Latino men. BMC Med Inform Decis Mak 2015; 15 (01) 56
• 41 Segall N, Saville JG, L'Engle P. , et al. Usability evaluation of a personal health record. AMIA Annu Symp Proc 2011; 2011: 1233-1242
• 42 Taha J, Sharit J, Czaja SJ. The impact of numeracy ability and technology skills on older adults' performance of health management tasks using a patient portal. J Appl Gerontol 2014; 33 (04) 416-436
• 43 Taha J, Sharit J, Czaja SJ. Usability of an electronic personal health record (PHR) among a diverse group of adults. Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting; 2014:619–623
• 44 Zarcadoolas C, Vaughon WL, Czaja SJ, Levy J, Rockoff ML. Consumers' perceptions of patient-accessible electronic medical records. J Med Internet Res 2013; 15 (08) e168
• 45 Price M, Bellwood P, Davies I. Using usability evaluation to inform Alberta's personal health record design. Stud Health Technol Inform 2015; 208: 314-318
• 46 Lazard AJ, Watkins I, Mackert MS, Xie B, Stephens KK, Shalev H. Design simplicity influences patient portal use: the role of aesthetic evaluations for technology acceptance. J Am Med Inform Assoc 2016; 23 (e1): e157-e161
• 47 Finkelstein J, Knight A, Marinopoulos S. , et al. Enabling patient-centered care through health information technology. Evid Rep Technol Assess (Full Rep) 2012; (206) 1-1531
• 48 Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff (Millwood) 2013; 32 (01) 63-68
• 49 Ancker JS, Witteman HO, Hafeez B, Provencher T, Van de Graaf M, Wei E. The invisible work of personal health information management among people with multiple chronic conditions: qualitative interview study among patients and providers. J Med Internet Res 2015; 17 (06) e137
• 50 Ancker JS, Witteman HO, Hafeez B, Provencher T, Van de Graaf M, Wei E. “You get reminded you're a sick person”: personal data tracking and patients with multiple chronic conditions. J Med Internet Res 2015; 17 (08) e202
• 51 Moen A, Brennan PF. Health@Home: the work of health information management in the household (HIMH): implications for consumer health informatics (CHI) innovations. J Am Med Inform Assoc 2005; 12 (06) 648-656
• 52 Corbin J, Strauss A. Managing chronic illness at home: three lines of work. Qual Sociol 1985; 8 (03) 224-247
• 53 Goodhue D, Thompson R. Task-technology fit and individual performance. Manage Inf Syst Q 1995; 19 (02) 213-236
• 54 Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study. BMC Med Inform Decis Mak 2006; 6: 3
• 55 Woollen J, Prey J, Wilcox L. , et al. Patient experiences using an inpatient personal health record. Appl Clin Inform 2016; 7 (02) 446-460
• 56 Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform 2003; 36 (1-2): 23-30
• 57 Miller WL, Crabtree BF. Qualitative analysis: how to begin making sense. Fam Pract Res J 1994; 14 (03) 289-297
• 58 Morse J. The significance of saturation. Qual Health Res 1995; 5: 147-149
• 59 Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract 2006; 7 (21) 21
• 60 Fagerlin A, Zikmund-Fisher BJ, Ubel PA, Jankovic A, Derry HA, Smith DM. Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Med Decis Making 2007; 27 (05) 672-680
• 61 Koppel R, Wetterneck T, Telles JL, Karsh B-T. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc 2008; 15 (04) 408-423
• 62 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3 (02) 77-101
• 63 Meyer D, Leventhal H, Gutmann M. Common-sense models of illness: the example of hypertension. Health Psychol 1985; 4 (02) 115-135
• 64 Leventhal H, Leventhal EA, Breland JY. Cognitive science speaks to the “common-sense” of chronic illness management. Ann Behav Med 2011; 41 (02) 152-163
• 65 LeRouge C, Ma J, Sneha S, Tolle K. User profiles and personas in the design and development of consumer health technologies. Int J Med Inform 2013; 82 (11) e251-e268
• 66 Ancker JS, Nosal S, Hauser D, Way C, Calman N. Access policy and digital divide in patient access to medical records. Health Policy and Technology 2017; 6 (01) 3-11