Appl Clin Inform 2017; 08(04): 1127-1143
DOI: 10.4338/ACI-2017-06-RA-0111
Research Article
Schattauer GmbH Stuttgart

Health IT Usability Focus Section: Adapting EHR-Based Medication Instructions to Comply with Plain Language Guidance—A Randomized Experiment

Jessica S. Ancker
,
Alexander Send
,
Baria Hafeez
,
Snezana N. Osorio
,
Erika Abramson
Further Information

Address for correspondence

Jessica S. Ancker, MPH, PhD
425 East 61st Street, Suite 301, New York, NY 10065
United States   

Publication History

30 June 2017

12 October 2017

Publication Date:
14 December 2017 (online)

 

Abstract

Objective Patient instructions are generally written by clinicians. However, clinician-centered language is challenging for patients to understand; in the case of pediatric medication instructions, consequences can be serious. Using examples of clinician-written medication instructions from an electronic health record, we conducted an experiment to determine whether parental misinterpretations would be reduced by instructions that followed best practices for plain language.

Methods We selected examples of dosing instructions from after-visit summaries in a commercial electronic health record. A demographically diverse sample of parents and adult caregivers was recruited from an online panel to participate in an English-language experiment, in which they received a comprehension questionnaire with either original after-visit summary instructions or instructions revised to comply with federal and other sources of plain-language guidance.

Results Nine-hundred and fifty-one respondents completed the experiment; 50% were women, the mean age was 36 years, and 38% had less than a 4-year college education. The revisions were associated with an 8 percentage point increase in correct answers overall (from 55% to 63%, p < 0.001), although revisions were not equally effective for all instructions. Health literacy and health numeracy were strong and independent predictors of comprehension. Overall, mistakes on comprehension questions were common, with respondents missing an average of 41% (6.1 of 15) of questions.

Conclusion In this experimental study, a relatively simple intervention of revising text was associated with a modest reduction in frequency of misinterpretations of medication instructions. As a supplement to more intensive high-touch interventions, revising electronic health record output to replace complex language with patient-centered language in an automated fashion is a potentially scalable solution that could reduce medication administration errors by parents.


#

Background and Significance

Individualized medication instructions are typically written by physicians or pharmacists and presented to the patient in the form of an electronic after-visit summary, discharge instruction, or prescription printed or delivered via an electronic patient portal. Unfortunately, it is well-established that patients frequently misunderstand clinician language, even when clinicians try to be simple. For example, one study showed that nearly 40% of a sample of clinic outpatients could not correctly operationalize the seemingly simple instruction “twice a day.”[1] Comprehension problems are more frequent among those with low health literacy and health numeracy.[2] [3] [4] [5] [6] [7] An estimated 14% of U.S. adults[8] and up to 29% of parents[6] have limited health literacy, defined as the skills and knowledge needed to obtain, understand, and apply information to their own health and medical decisions.[9] [10] [11] [12] Low health numeracy is even more prevalent and is common at even the highest literacy levels.[13] [14] Health numeracy is the skill set needed to apply quantitative information to health, including information about risks, times and dates, and quantities.[13] [15] [16]

Particularly in the case of pediatric medications, the consequences can be serious or even fatal, as children are uniquely vulnerable to adverse events from medication errors.[17] [18] More than 70,000 children present to U.S. emergency departments each year with accidental overdoses,[19] most caused by parent or caregiver administration errors.[2] [20] [21] [22] [23] [24] No comparable national-level data are available about underdosing, but this has also been found to be a common problem in small-scale studies.[2] [22] [25] [26] In administering children's medications, parents must operationalize instructions about dose, route, and frequency, as well as additional issues unique to children such as weight-based dosing, age-based thresholds and contraindications, and liquid medications.[2] [22] [25] [26]

One way to prevent medication errors could be to redesign the product that facilitates the errors, in this case, the medication instructions that parents frequently misunderstand. This perspective comes from human factors engineering, the practice of optimizing human well-being and system performance by making products, tasks, and systems compatible with the needs, abilities, and limitations of people, as well as from usability engineering, the subset of human factors engineering that focuses on making electronic systems easy to learn, satisfying to use, and matched to user needs, capabilities, and goals.[27] [28] [29] In fact, a large body of research already shows that patients, even those with low literacy or numeracy, can effectively manage medication administration when supported with evidence-based materials.[30] [31] Examples include plain-language instructions,[1] [3] as well as more complex materials such as visual schedules displaying pills on a calendar,[32] [33] “pictograph” illustrations,[1] [7] [34] [35] and marked pillboxes and preloaded syringes.[31] [35] [36] [37] In other words, the comprehension errors arise not solely from the patient's literacy but rather from a system that fails to match communication modality to patient capabilities.[13] [15] [38] [39]

To date, many of the communication strategies found to be efficacious in controlled research situations have not been implemented widely in practice because they are labor-intensive and difficult to implement consistently.[40] [41] An intervention with modest efficacy that reaches large numbers of patients reproducibly at low cost and effort may have a large public health impact.[42] [43] [44] In fact, the net effect may be larger than the effect of a highly efficacious intervention that is challenging to implement and so reaches only small numbers.[42] [43] [44]

We therefore examined whether electronic health records (EHRs) could be targeted for low-effort, scalable, usability interventions that would improve comprehension of medication instructions.


#

Objective

Our objective was to assess whether easy-to-implement wording changes based on best practices for plain language would improve comprehension of common medication instructions, using a randomized experiment. The intervention involved wording changes with limited visual illustrations, because this approach could be easily implemented and automated in existing commercial EHRs. Our secondary objectives were (1) to assess the effects of literacy, numeracy, and demographics on comprehension, and (2) to assess the effect of the intervention on the subsets of wrong answers associated with overdoses and underdoses.


#

Methods

Experimental Design

In this between-subject experiment, participants were randomly assigned to see the original wording (usual care) or the revisions (intervention; [Table 1]), and then to answer 15 multiple-choice comprehension questions (see Appendix).

Table 1

Original and revised instructions

Original wording (usual care)

Revised wording (intervention)

Primary research question: Electronic health record instructions

2 pills every 12 h

Give 2 pills in the morning, and 2 pills in the evening

Administer 5 drops into both ears 2 times daily

In the morning: 5 drops in right ear and 5 drops in left ear

In the evening: 5 drops in right ear and 5 drops in left ear

10 mL by mouth every 4 h

Give 10 mL at all of these times

 • Morning

 • Noon-time

 • Late afternoon

 • Bedtime

 • Midnight

 • 4 a.m.

1 pill per day for 3 d, switch to 1 pill every other day for 4 d

Monday morning

Tuesday morning

Wednesday morning

Thursday morning

Friday morning

Saturday morning

Sunday morning

1 pill

1 pill

1 pill

No pill

1 pill

No pill

1 pill

Apply topically 4 times daily until rash is gone and for 3 additional days

Until rash is gone: Spread the cream on skin every morning, noon, afternoon, and evening

After rash is gone: Keep using the cream in the same way for 3 more days

Give every 4 h as needed for fever/pain, max 5 doses

Give this medicine if your child has fever or pain. Wait at least 4 h. Does the child still have fever or pain? If so, give medicine again. Don't give it more than 5 times in one day

Give on an empty stomach

Give 30 min before the child eats breakfast

2 drops each nostril ×3–4 per day as needed

If the child's nose is stuffy, give 2 drops in the right nostril and 2 drops in the left nostril

How often per day: up to 4 times

For very high fevers, alternate between tylenol and ibuprofen every 3 h

If your child has a fever of 104 degrees or higher:

 • Start with a dose of tylenol

 • 3 h later, check to see if the fever is still high. If so, give a dose of ibuprofen

 • 3 h later, check to see if the fever is still high. If so, give a dose of tylenol

You can switch between the medicines in this way until the fever comes down

Secondary research question: Over-the-counter medication instructions

1 unpacked level scoop (8.5 g) per 2 fl oz

To make

Use this much water

Mix in this much powder

2 ounces

2 ounces

1 scoop

4 ounces

4 ounces

2 scoops

6 ounces

6 ounces

3 scoops

Weight (lb)

Age (mo)

Dose (mL)

If you know your child's weight, use this information:

 • under 18 pounds, give 1.25 mL

 • 18 pounds or more, give 1.875 mL

If you do not know your child's weight, use this information:

 • less than 12 mo old, give 1.25 mL

 • 12 mo or older, give 1.875 mL

But if your child is under 6 mo old, ask a doctor whether you should use this medicine.

Under 6 mo

Ask a doctor

12–17 lbs

6–11 mo

1.25 mL

18–23 lbs

12–23 mo

1.875 mL


#

Setting and Sample

Survey Sampling International (SSI; www.surveysampling.com) is an international survey panel and market research firm widely used in online surveys. We contracted with SSI to recruit a sample. The inclusion criteria were that participants had to be: (1) U.S. adults with primary caregiving responsibilities for at least one child under the age of 18, and (2) comfortable completing a questionnaire in English. We also specified to SSI that a minimum of 30% of the sample should have less than a college education. SSI has a large existing online panel of registered individuals who have agreed to be contacted for online surveys and questionnaires to be entered into drawings to earn modest incentives of their choice (which might include money, airline miles, etc.). These registered individuals have all completed extensive demographic questionnaires. Using our eligibility criteria, SSI disseminated the recruitment announcement to individuals who met our criteria, while continuously monitoring the education levels of those who agreed to participate. Recruitment was closed early for the higher education category and extended for the lower education category to ensure the 30% representation we had specified.


#

Questionnaire Development

We selected examples of common dosing instructions from the after-visit summary of a commercial EHR (Epic Systems, Verona, Wisconsin, United States). Instructions in our institution's after-visit summary already take a step toward patient-centeredness by replacing acronyms and abbreviations (e.g., a provider instruction of “2× per day” is automatically replaced with “2 times a day”).

For exploratory purposes, we also added two questions from common over-the-counter pediatric pain relievers and formula, plus an instruction to alternate between acetaminophen and ibuprofen for fever reduction. (Although this practice is not encouraged at our institution in light of an American Academy of Pediatrics white paper,[45] we included it because patients elsewhere may encounter it.)

To create the revised versions of the instructions, we selected a package of five plain-language principles that were well supported by evidence (although not always in the context of medication instructions; see references) and that appeared to be easily applicable to informatics practice (e.g., could potentially be accomplished through automatic term substitutions rather than major redesign). We drew from www.plainlanguage.gov and other resources such as Shoemaker et al[46] and the references listed below:

  1. Avoid unfamiliar terms, jargon, and abbreviations. This involved substituting familiar terms for unfamiliar ones where possible,[47] [48] [49] [50] defining medical terms parenthetically when they could not be replaced with more familiar terms,[46] [50] and inserting explanations of selected concepts that require medical knowledge (e.g., for “on an empty stomach,” explaining how long before and after the meal the child would have an empty stomach).[46]

  2. Perform computations for the reader.[13] [46] Instead of instructing the reader to administer medications every X hours, administration times were phrased in terms of recurring events such as morning and evening or mealtimes.[1] [50] [51]

  3. Avoid complex sentences containing multiple instructions or steps.[46] [49] This involved putting one instruction or step per bullet or sentence,[46] separating multiple “if-then” or “when” conditions into a separate instruction,[49] and keeping the “if” or “when” statement short and placing it after the verb.[49]

  4. Use active voice and address the reader.[46]

  5. Use illustrations if they reinforce or explain the text.[46] [49] [52] [53]

We opted not to apply formulas to reduce the “grade level” of the text; revisions to reduce the grade level would have involved replacing long words with short ones and breaking long sentences into shorter ones.[54] [55] This decision was made on basis of research suggesting that word familiarity was more important to comprehension than word length,[56] and that the contextual explanations needed to help novices understand health-related text often make text longer.[57] [58]


#

Covariates

For all participants, health literacy was assessed with the three-item Chew scale.[59] [60] Following Wallace et al,[61] we assigned 1 point to each question for which the participant answered that they had any difficulty or required help, to produce a health literacy score of 0 (adequate), 1 (marginal), or 2 or 3 (inadequate). Health numeracy was assessed with the 8-item Short Numeracy Understanding in Medicine Instrument (S-NUMi).[62] [63] Following Schapira, a health numeracy score of 7 or higher was classified as high numeracy, 4 to 6 as adequate numeracy, and 3 or less as low numeracy.

In addition, participants were asked about their personal experience of administering pediatric medications, their medical training or experience, and personal and family demographics.


#

Analytic Methods

The primary outcome was number of correct answers to the comprehension questions. A secondary outcome was incorrect responses that would have led to overdoses or underdoses. To establish this, each of the incorrect response options for each question was classified a priori as a potential overdose or underdose by a pharmacist collaborator (A.S.), employing professional judgment and reference works as appropriate.

Student's t-tests and analysis of variance for continuous variables and chi-squared tests for categorical ones were used to assess comparability of the two arms as well as bivariate associations with the primary outcome (correct answers to the comprehension questions). Descriptive statistics and tables were computed in IBM SPSS Statistics version 24 (Armonk, New York, United States). Variables significant at 0.05 on bivariate analysis were considered for multivariate regression models to assess the secondary outcomes of the effects of literacy, numeracy, and demographics on comprehension. Multivariate models were constructed in SAS (version 9.4, Cary, North Carolina, United States) using proc GLM applying forward and backward selection to drop nonsignificant variables, and proc GLMselect using stepwise selection on the basis of the Akaike information criterion.


#

Human Subjects Research Approval

The Weill Cornell Institutional Review Board determined that the project was exempt because no personally identifying information was collected from the participants.


#
#

Results

A total of 1,012 individuals started the questionnaire, and 952 (94.1%) completed it (with equal dropout rates in the two arms, p = 0.23). One response was eliminated for an apparent data quality problem (self-reported age over 90 years) leaving a final sample size of 951.

As shown in [Table 2], half the sample were women, the mean age was ∼37 years, 83% were white, 38% had educational attainment of less than a bachelor's degree, and ∼12% had health insurance provided by Medicaid (the U.S. public insurance program for low-income individuals and families). One quarter reported that they had some form of medical experience or training. Inclusion criteria included having primary caregiving responsibilities for a child under the age of 18, but some participants also reported having children older than 18 years. Participant characteristics were well balanced across the two arms with the exception of age: respondents in the control (usual care) arm were an average of ∼1 year older than respondents in the intervention arm.

Table 2

Participant demographic and knowledge/skills characteristics

Characteristics

Whole group

Version 1

(usual care)

Version 2

(intervention)

p-Value

N

951

480

471

Demographic questions

Women, N (%)

475 (49.9)

240 (50.0)

235 (49.9)

0.97

Mean (SD) age

36.5 (9.5)

37.1 (9.7)

35.8 (9.4)

0.03

Race (multiple-choice possible), N (%)

 White or Caucasian

789 (82.8)

394 (81.9)

395 (83.7)

0.47

 Black or African-American

91 (9.5)

45 (9.4)

46 (9.7)

0.84

 Asian

46 (4.8)

28 (5.8)

18 (3.8)

0.15

 American Indian or Alaska Native

20 (2.1)

11 (2.3)

9 (1.9)

0.68

 Native Hawaiian or Pacific Islander

4 (0.4)

1 (0.2)

3 (0.6)

0.37

 Other

27 (2.8)

14 (2.9)

13 (2.8)

0.88

 Prefer not to say

7 (0.7)

2 (0.4)

5 (1.1)

0.25

Hispanic/Latino, N (%)

>0.99

 Hispanic/Latino

112 (11.8)

57 (11.9)

55 (11.7)

 Not Hispanic/Latino

823 (86.4)

415 (86.3)

408 (86.4)

 Prefer not to say

16 (1.7)

8 (1.7)

8 (1.7)

Education, N (%)

0.55

 Did not complete high school

16 (1.7)

8 (1.7)

8 (1.7)

 Completed high school

194 (20.4)

87 (18.1)

107 (22.7)

 Completed a 2-y college degree

151 (15.9)

82 (17.1)

69 (14.6)

 Completed a 4-y college degree

290 (30.5)

146 (30.4)

144 (30.6)

 Completed graduate degree

297 (31.2)

155 (32.3)

142 (30.1)

 Prefer not to say

3 (0.3)

2 (0.4)

1 (0.2)

Household income, N (%)

0.59

 0–$24,999

89 (9.3)

45 (9.4)

44 (9.3)

 $25,000–$49,999

159 (16.7)

75 (15.6)

84 (17.8)

 $50,000–$74,999

214 (22.5)

102 (21.2)

112 (23.7)

 $75,000–$99,999

231 (24.2)

126 (26.2)

105 (22.2)

 $100,000–$124,999

136 (14.3)

66 (13.7)

70 (14.8)

 $125,000 or more

122 (12.8)

66 (13.7)

56 (11.9)

Insurance coverage, N (%)

0.12

 None

38 (4.0)

18 (3.7)

20 (4.2)

 Private

564 (59.2)

292 (60.7)

272 (57.6)

 Medicare managed care

42 (4.4)

28 (5.8)

14 (3.0)

 Medicare

113 (11.9)

50 (10.4)

63 (13.3)

 Medicaid managed care

21 (2.2)

8 (1.7)

13 (2.8)

 Medicaid

90 (9.4)

50 (10.4)

40 (8.5)

 Both Medicare and Medicaid

41 (4.3)

18 (3.7)

23 (4.9)

 Don't know

16 (1.7)

5 (1.0)

11 (2.3)

 Other

26 (2.7)

11 (2.3)

15 (3.2)

Children's insurance, N (%)

0.77

 None

31 (3.3)

14 (2.9)

17 (3.6)

 Private

601 (63.1)

310 (64.4)

291 (61.7)

 Medicaid managed care

80 (8.4)

44 (9.1)

36 (7.8)

 Medicaid

190 (19.9)

89 (18.5)

101 (21.4)

 Don't know

17 (1.8)

8 (1.7)

9 (1.9)

 Other

32 (3.4)

15 (3.1)

17 (3.6)

Number of children

 Overall

1,879

977

902

 Mean (SD)

2.1 (1.1)

2.1 (1.2)

2.0 (1.1)

0.11

 Had at least one child <2 y

172

90

82

0.72

 Had at least one child 2–5 y

401

196

205

0.57

 Had at least one child 6–11 y

579

300

279

0.54

 Had at least one child 12–18 y

520

275

245

0.43

 Had at least one child >18 y

207

116

91

0.49

 Caregiver only; no children

39 (4.1%)

19 (4.0%)

20 (4.2%)

Knowledge and skills questions

How confident are you that you know your youngest child's current weight?

0.83

 Very

501 (52.7)

255 (53.1)

246 (52.2)

 Somewhat

386 (40.6)

191 (39.8)

195 (41.4)

 Not at all

64 (6.7)

34 (7.1)

30 (6.4)

 Has “medical training or experience”

240 (25.2)

124 (25.8)

116 (24.6)

0.67

Has given children medications often/very often in last 3 mo

251 (25.2)

121 (25.1)

120 (25.5)

0.14

Has used kitchen spoons or other nonstandard dosing for children

313 (32.9)

171 (35.6)

142 (30.2)

0.07

Health literacy, N (%)

0.17

 Adequate

542 (57.0)

258 (53.8)

262 (55.6)

 Marginal

203 (21.3)

104 (21.7)

99 (21.0)

 Inadequate

206 (21.7)

96 (20.0)

110 (23.4)

Health numeracy, N (%)

0.38

 High

235 (24.7)

126 (26.3)

109 (23.1)

 Adequate

430 (45.2)

207 (43.1)

223 (47.3)

 Low

286 (30.1)

147 (30.6)

139 (29.5)

Abbreviation: SD, standard deviation.


Note: Medicaid and Medicaid managed care are public insurance programs in the United States available only to low-income individuals and families. Medicare and Medicare managed care are public insurance programs in the United States available to all individuals over age 65 as well as to younger people with certain severe disabilities or chronic kidney failure.


As shown in [Table 3], the revised instructions were associated with a 1.1-point absolute or 13.3% relative improvement in the medication comprehension score (from 8.3 to 9.4 correct out of 15 questions; p < 0.01). In secondary outcomes, the intervention reduced the likelihood of responses that would have led to medication underdoses but not likelihood of selecting responses associated with overdoses ([Table 3], rows 2 and 3). For the individual comprehension questions, the revised instructions significantly improved the likelihood of selecting the correct answer for seven of the questions, significantly reduced the likelihood of selecting the correct answer for two questions, and made no difference for the remaining six questions ([Table 3]).

Table 3

Medication instruction comprehension results

Whole group

Version 1

(usual care)

Version 2

(intervention)

p-Value

Mean (SD) correct answers

8.9 (3.8)

8.3 (3.8)

9.4 (3.7)

<0.01a

Mean (SD) answer with risk of overdose

2.7 (1.8)

2.6 (1.8)

2.7 (1.8)

0.68

Mean (SD) answer with risk of underdose

2.7 (2.5)

3.2 (2.5)

2.1 (2.4)

<0.01a

Correct answers

 Question 1 (how many pills in one day), N (%)

650 (68.2)

276 (57.4)

374 (79.2)

<0.01a

 Question 2 (how many pills in the morning), N (%)

701 (73.6)

303 (63.0)

398 (84.3)

<0.01a

 Question 3 (how many drops in one day), N (%)

496 (52.0)

192 (39.9)

304 (64.4)

<0.01a

 Question 4 (how many drops in the morning), N (%)

569 (59.7)

244 (50.7)

325 (68.9)

<0.01a

 Question 5 (how many doses in one day), N (%)

626 (65.7)

308 (64.0)

318 (67.4)

0.28

 Question 6 (how many pills in one week), N (%)

629 (66.0)

314 (65.3)

315 (66.7)

0.63

 Question 7 (how much powder to make 6 ounces of formula), N (%)

705 (74.0)

328 (68.2)

377 (79.9)

<0.01a

 Question 8 (what is the last day to use the cream), N (%)

126 (13.2)

60 (12.5)

66 (14.0)

0.49

 Question 9 (should you give another dose now), N (%)

497 (52.2)

258 (53.6)

239 (50.6)

0.35

 Question 10 (how much medicine for a 6-mo-old), N (%)

478 (50.2)

219 (45.5)

259 (54.9)

< 0.01a

 Question 11 (when should your child eat breakfast), N (%)

505 (53.0)

291 (60.5)

214 (45.3)

<0.01a,b

 Question 12 (how many drops right now), N (%)

502 (52.7)

253 (52.6)

249 (52.8)

0.96

 Question 13 (how many times can you give the drops in a day), N (%)

638 (66.9)

268 (55.7)

370 (78.4)

<0.01a

 Question 14 (alternating medicines: which medicine now), N (%)

698 (73.2)

344 (71.5)

354 (75.0)

0.22

 Question 15 (alternating medicines: which medicine at 5 p.m.), N (%)

607 (63.7)

349 (72.6)

258 (54.7)

<0.01a,b

Abbreviation: SD, standard deviation.


Note: Findings marked with a are statistically significant at 0.05 in the hypothesized direction. Findings marked with b are statistically significant but in the direction opposite to the hypothesized direction.


Bivariate analyses showed that health numeracy was a very strong predictor of comprehension. Mean comprehension scores for individuals with high, adequate, and low numeracy were 12.0, 9.8, and 5.0, respectively. Health literacy was also a strong predictor, with mean scores for individuals with adequate, marginal, and inadequate literacy of 10.1, 8.0, and 6.4, respectively.

When we controlled for health numeracy and literacy in the multivariable analyses, none of the following demographic variables was statistically significant and therefore were dropped from the final model: parental age, race, ethnicity, education, household income, insurance status, being a self-reported frequent administrator of pediatric medications, and having medical training/experience. The final model (model R 2 = 0.562) demonstrated that being a woman, having higher health numeracy, having higher health literacy, and receiving the revised instructions were all significantly and independently associated with improvements in comprehension score ranging from 0.4 points to ∼1 point ([Table 4]). Interaction terms between version and numeracy and version and literacy were not statistically significant, suggesting the effect of the revision was comparable in all literacy and numeracy levels.

Table 4

Multivariate regression predictors of instruction comprehension

Parameter

Estimate

Standard error

t Value

Pr > |t|

Intercept

2.65

0.27

9.67

<0.0001

Received Version 2 (intervention)

1.07

0.16

6.50

<0.0001

Female

1.06

0.17

6.17

<0.0001

Health numeracy score

1.14

0.04

26.31

<0.0001

Health literacy category

0.41

0.10

4.14

<0.0001

Note: The estimates in this linear model represent estimated changes (improvements) in total comprehension score associated with each variable. Variables not included in the final model after forward/backward stepwise selection were: parental age, race, ethnicity, education, household income, insurance status, being a self-reported frequent administrator of pediatric medications, and having medical training/experience.



#

Discussion

Several plain-language revisions have been individually demonstrated to improve comprehension. In the current experiment, we demonstrated that a package of five of these revisions employed together was associated with improved comprehension of common medication instructions. In addition, the experiment showed that health literacy and health numeracy were both independently associated with comprehension, and that the effect size associated with the plain-language revisions did not differ by literacy level or numeracy level. The package of revisions was associated with fewer wrong answers that would have led to underdoses but not overdoses. Although the total comprehension score was higher with the revisions, there was no effect for a subset of six questions, and comprehension was lower for a subset of two questions.

Our findings were consistent with many other studies showing that health literacy was associated with misunderstanding of instructions[1] [2] [3] [6] [64] and that text revisions can assist in interpretation.[65] [66] In addition, however, we demonstrated that health numeracy was independently related to comprehension, and that numeracy accounted for significantly more of the variability in performance than literacy did. Furthermore, other demographic variables were not associated with comprehension in models that controlled for health literacy and numeracy (with the exception of gender). We conclude that poor health numeracy may be an underrecognized predictor of poor comprehension and may in fact account for many of the previously observed demographic predictors of medication misinterpretations.[13] Numeracy is a particularly important factor given that low numeracy is more prevalent than low literacy and is found even among people with adequate literacy.[10] [67] [68] Of the individuals in our study with adequate health literacy, 15% had low health numeracy.

In addition to these primary and secondary findings, it is noteworthy that medication instructions disseminated to patients via commercial EHR technology were frequently misunderstood by a diverse sample of parents of all literacy and numeracy levels. For example, many parents did not appear to understand that instructions including the word “max” indicated an upper maximum threshold for the medication. No individual instruction was well understood by more than 72% of respondents. Consequently, it is possible that these instructions pose a safety threat to pediatric patients. Furthermore, only 53% of the parents were confident that they knew the current weight of their youngest child, and 33% reported using kitchen utensils or other nonstandard devices for pediatric medications. Such basic information would be needed for self-administered weight-based dosing (such as might be common with over-the-counter medications), and about basic safety precautions about measuring pediatric medications. Our sample reported a very high rate of using nonstandard devices for pediatric medications, a practice that has previously been shown to increase the rate of medication errors.[22] Other studies have found that between 6 and 23% of parents used or described using nonstandard instruments.[5] [22] [61]

The effect size associated with this intervention was modest. However, as a supplement to more intensive high-touch interventions, we propose that revising EHR output to replace complex language with simpler language is a potentially scalable solution that could reduce medication administration errors by parents. Some of these highly effective high-touch interventions include providing patients with customized medication administration tools,[36] [37] consultations with dedicated medication nurses,[69] and illustrated or diagrammed instructions.[30] [32] [33] [52] [70]

Although the revised instructions were associated with comprehension scores ∼8 percentage points higher overall (from 55 to 63%), the effect size was not equal for each instruction. The revisions were associated with a significantly higher rate of correct answers for seven of the component questions and a significantly significant lower rate of correct answers for two of the component questions.

Limitations of the Study

This study should be interpreted in light of several limitations. Testing five plain-language revisions at once means that conclusions can be drawn only about the package of all five, not about the relative efficacy of each type of revision. The online-only format, although it produced a demographically diverse population with a range of education and literacy levels, probably excluded individuals in the very lowest computer literacy and/or literacy categories. It is possible, therefore, that real-world comprehension is actually worse than what we found in our sample. Also, many of the medication instructions used in this study were drawn from an EHR system that had already undergone in-house customization to replace abbreviations (e.g., “2 × ” was already automatically replaced with “2 times”). It is possible that the effect of our intervention would be even greater if the revised instructions were compared with the prescribing provider's original instructions.

The study was administered in English only. Black and Hispanic patients were underrepresented (9.5% of our sample was black compared with ∼13% of the U.S. population, and 11.8% were Hispanic compared with ∼17% of the U.S. population). A large number of respondents reported some medical experience or training. We specifically developed the inclusion criteria to include any adult caregiver of a child, and the demographics suggest the possibility that the sample may have included grandparents. However, we did not collect data that would have allowed us to break down the results by whether the respondent was a parent or another type of a caregiver. Because we were testing a hypothesis about application of plain-language guidelines, we did not pretest the questionnaire for optimization before the survey, which may have contributed to the situations in which revised instructions reduced comprehension. A final limitation is that we used hypothetical questions only; generalizability to actual medication administration is not known. However, it seems possible that parents of sick children in reality might perform worse than they did in this relatively low-stress questionnaire study.


#
#

Conclusion

Misinterpretations of pediatric medication instructions commonly provided to patients are frequent. Simple language revisions, most of which could be implemented in the EHR without the need for additional formatting or graphics, were associated with reduced frequency of misinterpretations overall, although not for every instruction. Revising EHR output through automated substitutions could be a scalable solution that would reduce the number of parents who misinterpret pediatric medication instructions. However, instructions were still subject to misinterpretation even after the revision, so this approach should be considered a low-effort supplement to more intensive high-touch interventions that could reduce parental medication administration errors. Furthermore, additional usability and literacy testing will be required to fully develop such an intervention and test it in practice. Future testing would be most beneficial if it focused specifically on the most vulnerable groups such as individuals with limited health literacy or English proficiency.


#

Clinical Relevance Statement

Patients derive important information from EHR-generated documents. A relatively simple intervention of replacing complex text with patient-friendly text reduced misinterpretations that would be likely to lead to medication mistakes. The intervention tested here could be automated with relatively simple phrase substitution.


#

Multiple Choice Questions

  1. Limited health numeracy, which is more prevalent than limited health literacy, is best defined as poor ability to:

    • Conduct statistical analysis of health data

    • Apply quantitative information to health decisions

    • Interpret peer-reviewed medical journal articles

    • Apply written and oral information to health decisions

    Correct answer: The correct answer is B, apply quantitative information to health decisions. Health numeracy is the set of skills and knowledge that patients need to read, understand, and apply quantitative information to personal health decisions. Health numeracy is not used to describe the more advanced set of quantitative skills used by physicians and scientists to analyze data or stay informed about the peer-reviewed literature. Health literacy similarly describes a broad set of skills and knowledge that patients need to read, understand, and apply written, textual, and oral information to their health; some authors consider health numeracy to be a subset of health literacy.

  2. Patients with limited health literacy have difficulty comprehending medication instructions. Examples of interventions that have been associated with significant improvements in comprehension by low-literacy populations include:

    • Medication instructions revised in plain language

    • Health literacy coursework for patients

    • Training on effective prescription writing for physicians

    • Fully automated consumer-friendly translations of instructions

    Correct answer: The correct answer is A, medication instructions revised in plain language. This study adds new evidence to an already extensive literature on revising medication instructions to improve patient comprehension. Health literacy coursework, and effective prescription writing training, both appear to be useful ideas but have not been demonstrated to improve medication comprehension by patients. Also, no fully automated method has yet been demonstrated to improve patient comprehension of medication instructions.


#

Appendix

Zoom Image
Zoom Image
Zoom Image

#
#

Conflict of Interest

None.

Acknowledgments

The researchers thank Sana Ali for assisting with the questionnaire programming.

Protection of Human and Animal Subjects

The study was reviewed by the Weill Cornell Institutional Review Board and was considered exempt because no personally identifying information was collected from the participants.


Funding

This study was supported by a pilot grant from the Department of Healthcare Policy & Research. Dr. Ancker is supported by K01 HS 021531 from the Agency for Healthcare Research and Quality.



Address for correspondence

Jessica S. Ancker, MPH, PhD
425 East 61st Street, Suite 301, New York, NY 10065
United States   


Zoom Image
Zoom Image
Zoom Image