Appl Clin Inform 2017; 08(04): 1127-1143
DOI: 10.4338/ACI-2017-06-RA-0111
Research Article
Schattauer GmbH Stuttgart

Health IT Usability Focus Section: Adapting EHR-Based Medication Instructions to Comply with Plain Language Guidance—A Randomized Experiment

Jessica S. Ancker
,
Alexander Send
,
Baria Hafeez
,
Snezana N. Osorio
,
Erika Abramson
Weitere Informationen

Address for correspondence

Jessica S. Ancker, MPH, PhD
425 East 61st Street, Suite 301, New York, NY 10065
United States   

Publikationsverlauf

30. Juni 2017

12. Oktober 2017

Publikationsdatum:
14. Dezember 2017 (online)

 

Abstract

Objective Patient instructions are generally written by clinicians. However, clinician-centered language is challenging for patients to understand; in the case of pediatric medication instructions, consequences can be serious. Using examples of clinician-written medication instructions from an electronic health record, we conducted an experiment to determine whether parental misinterpretations would be reduced by instructions that followed best practices for plain language.

Methods We selected examples of dosing instructions from after-visit summaries in a commercial electronic health record. A demographically diverse sample of parents and adult caregivers was recruited from an online panel to participate in an English-language experiment, in which they received a comprehension questionnaire with either original after-visit summary instructions or instructions revised to comply with federal and other sources of plain-language guidance.

Results Nine-hundred and fifty-one respondents completed the experiment; 50% were women, the mean age was 36 years, and 38% had less than a 4-year college education. The revisions were associated with an 8 percentage point increase in correct answers overall (from 55% to 63%, p < 0.001), although revisions were not equally effective for all instructions. Health literacy and health numeracy were strong and independent predictors of comprehension. Overall, mistakes on comprehension questions were common, with respondents missing an average of 41% (6.1 of 15) of questions.

Conclusion In this experimental study, a relatively simple intervention of revising text was associated with a modest reduction in frequency of misinterpretations of medication instructions. As a supplement to more intensive high-touch interventions, revising electronic health record output to replace complex language with patient-centered language in an automated fashion is a potentially scalable solution that could reduce medication administration errors by parents.


#

Background and Significance

Individualized medication instructions are typically written by physicians or pharmacists and presented to the patient in the form of an electronic after-visit summary, discharge instruction, or prescription printed or delivered via an electronic patient portal. Unfortunately, it is well-established that patients frequently misunderstand clinician language, even when clinicians try to be simple. For example, one study showed that nearly 40% of a sample of clinic outpatients could not correctly operationalize the seemingly simple instruction “twice a day.”[1] Comprehension problems are more frequent among those with low health literacy and health numeracy.[2] [3] [4] [5] [6] [7] An estimated 14% of U.S. adults[8] and up to 29% of parents[6] have limited health literacy, defined as the skills and knowledge needed to obtain, understand, and apply information to their own health and medical decisions.[9] [10] [11] [12] Low health numeracy is even more prevalent and is common at even the highest literacy levels.[13] [14] Health numeracy is the skill set needed to apply quantitative information to health, including information about risks, times and dates, and quantities.[13] [15] [16]

Particularly in the case of pediatric medications, the consequences can be serious or even fatal, as children are uniquely vulnerable to adverse events from medication errors.[17] [18] More than 70,000 children present to U.S. emergency departments each year with accidental overdoses,[19] most caused by parent or caregiver administration errors.[2] [20] [21] [22] [23] [24] No comparable national-level data are available about underdosing, but this has also been found to be a common problem in small-scale studies.[2] [22] [25] [26] In administering children's medications, parents must operationalize instructions about dose, route, and frequency, as well as additional issues unique to children such as weight-based dosing, age-based thresholds and contraindications, and liquid medications.[2] [22] [25] [26]

One way to prevent medication errors could be to redesign the product that facilitates the errors, in this case, the medication instructions that parents frequently misunderstand. This perspective comes from human factors engineering, the practice of optimizing human well-being and system performance by making products, tasks, and systems compatible with the needs, abilities, and limitations of people, as well as from usability engineering, the subset of human factors engineering that focuses on making electronic systems easy to learn, satisfying to use, and matched to user needs, capabilities, and goals.[27] [28] [29] In fact, a large body of research already shows that patients, even those with low literacy or numeracy, can effectively manage medication administration when supported with evidence-based materials.[30] [31] Examples include plain-language instructions,[1] [3] as well as more complex materials such as visual schedules displaying pills on a calendar,[32] [33] “pictograph” illustrations,[1] [7] [34] [35] and marked pillboxes and preloaded syringes.[31] [35] [36] [37] In other words, the comprehension errors arise not solely from the patient's literacy but rather from a system that fails to match communication modality to patient capabilities.[13] [15] [38] [39]

To date, many of the communication strategies found to be efficacious in controlled research situations have not been implemented widely in practice because they are labor-intensive and difficult to implement consistently.[40] [41] An intervention with modest efficacy that reaches large numbers of patients reproducibly at low cost and effort may have a large public health impact.[42] [43] [44] In fact, the net effect may be larger than the effect of a highly efficacious intervention that is challenging to implement and so reaches only small numbers.[42] [43] [44]

We therefore examined whether electronic health records (EHRs) could be targeted for low-effort, scalable, usability interventions that would improve comprehension of medication instructions.


#

Objective

Our objective was to assess whether easy-to-implement wording changes based on best practices for plain language would improve comprehension of common medication instructions, using a randomized experiment. The intervention involved wording changes with limited visual illustrations, because this approach could be easily implemented and automated in existing commercial EHRs. Our secondary objectives were (1) to assess the effects of literacy, numeracy, and demographics on comprehension, and (2) to assess the effect of the intervention on the subsets of wrong answers associated with overdoses and underdoses.


#

Methods

Experimental Design

In this between-subject experiment, participants were randomly assigned to see the original wording (usual care) or the revisions (intervention; [Table 1]), and then to answer 15 multiple-choice comprehension questions (see Appendix).

Table 1

Original and revised instructions

Original wording (usual care)

Revised wording (intervention)

Primary research question: Electronic health record instructions

2 pills every 12 h

Give 2 pills in the morning, and 2 pills in the evening

Administer 5 drops into both ears 2 times daily

In the morning: 5 drops in right ear and 5 drops in left ear

In the evening: 5 drops in right ear and 5 drops in left ear

10 mL by mouth every 4 h

Give 10 mL at all of these times

 • Morning

 • Noon-time

 • Late afternoon

 • Bedtime

 • Midnight

 • 4 a.m.

1 pill per day for 3 d, switch to 1 pill every other day for 4 d

Monday morning

Tuesday morning

Wednesday morning

Thursday morning

Friday morning

Saturday morning

Sunday morning

1 pill

1 pill

1 pill

No pill

1 pill

No pill

1 pill

Apply topically 4 times daily until rash is gone and for 3 additional days

Until rash is gone: Spread the cream on skin every morning, noon, afternoon, and evening

After rash is gone: Keep using the cream in the same way for 3 more days

Give every 4 h as needed for fever/pain, max 5 doses

Give this medicine if your child has fever or pain. Wait at least 4 h. Does the child still have fever or pain? If so, give medicine again. Don't give it more than 5 times in one day

Give on an empty stomach

Give 30 min before the child eats breakfast

2 drops each nostril ×3–4 per day as needed

If the child's nose is stuffy, give 2 drops in the right nostril and 2 drops in the left nostril

How often per day: up to 4 times

For very high fevers, alternate between tylenol and ibuprofen every 3 h

If your child has a fever of 104 degrees or higher:

 • Start with a dose of tylenol

 • 3 h later, check to see if the fever is still high. If so, give a dose of ibuprofen

 • 3 h later, check to see if the fever is still high. If so, give a dose of tylenol

You can switch between the medicines in this way until the fever comes down

Secondary research question: Over-the-counter medication instructions

1 unpacked level scoop (8.5 g) per 2 fl oz

To make

Use this much water

Mix in this much powder

2 ounces

2 ounces

1 scoop

4 ounces

4 ounces

2 scoops

6 ounces

6 ounces

3 scoops

Weight (lb)

Age (mo)

Dose (mL)

If you know your child's weight, use this information:

 • under 18 pounds, give 1.25 mL

 • 18 pounds or more, give 1.875 mL

If you do not know your child's weight, use this information:

 • less than 12 mo old, give 1.25 mL

 • 12 mo or older, give 1.875 mL

But if your child is under 6 mo old, ask a doctor whether you should use this medicine.

Under 6 mo

Ask a doctor

12–17 lbs

6–11 mo

1.25 mL

18–23 lbs

12–23 mo

1.875 mL


#

Setting and Sample

Survey Sampling International (SSI; www.surveysampling.com) is an international survey panel and market research firm widely used in online surveys. We contracted with SSI to recruit a sample. The inclusion criteria were that participants had to be: (1) U.S. adults with primary caregiving responsibilities for at least one child under the age of 18, and (2) comfortable completing a questionnaire in English. We also specified to SSI that a minimum of 30% of the sample should have less than a college education. SSI has a large existing online panel of registered individuals who have agreed to be contacted for online surveys and questionnaires to be entered into drawings to earn modest incentives of their choice (which might include money, airline miles, etc.). These registered individuals have all completed extensive demographic questionnaires. Using our eligibility criteria, SSI disseminated the recruitment announcement to individuals who met our criteria, while continuously monitoring the education levels of those who agreed to participate. Recruitment was closed early for the higher education category and extended for the lower education category to ensure the 30% representation we had specified.


#

Questionnaire Development

We selected examples of common dosing instructions from the after-visit summary of a commercial EHR (Epic Systems, Verona, Wisconsin, United States). Instructions in our institution's after-visit summary already take a step toward patient-centeredness by replacing acronyms and abbreviations (e.g., a provider instruction of “2× per day” is automatically replaced with “2 times a day”).

For exploratory purposes, we also added two questions from common over-the-counter pediatric pain relievers and formula, plus an instruction to alternate between acetaminophen and ibuprofen for fever reduction. (Although this practice is not encouraged at our institution in light of an American Academy of Pediatrics white paper,[45] we included it because patients elsewhere may encounter it.)

To create the revised versions of the instructions, we selected a package of five plain-language principles that were well supported by evidence (although not always in the context of medication instructions; see references) and that appeared to be easily applicable to informatics practice (e.g., could potentially be accomplished through automatic term substitutions rather than major redesign). We drew from www.plainlanguage.gov and other resources such as Shoemaker et al[46] and the references listed below:

  1. Avoid unfamiliar terms, jargon, and abbreviations. This involved substituting familiar terms for unfamiliar ones where possible,[47] [48] [49] [50] defining medical terms parenthetically when they could not be replaced with more familiar terms,[46] [50] and inserting explanations of selected concepts that require medical knowledge (e.g., for “on an empty stomach,” explaining how long before and after the meal the child would have an empty stomach).[46]

  2. Perform computations for the reader.[13] [46] Instead of instructing the reader to administer medications every X hours, administration times were phrased in terms of recurring events such as morning and evening or mealtimes.[1] [50] [51]

  3. Avoid complex sentences containing multiple instructions or steps.[46] [49] This involved putting one instruction or step per bullet or sentence,[46] separating multiple “if-then” or “when” conditions into a separate instruction,[49] and keeping the “if” or “when” statement short and placing it after the verb.[49]

  4. Use active voice and address the reader.[46]

  5. Use illustrations if they reinforce or explain the text.[46] [49] [52] [53]

We opted not to apply formulas to reduce the “grade level” of the text; revisions to reduce the grade level would have involved replacing long words with short ones and breaking long sentences into shorter ones.[54] [55] This decision was made on basis of research suggesting that word familiarity was more important to comprehension than word length,[56] and that the contextual explanations needed to help novices understand health-related text often make text longer.[57] [58]


#

Covariates

For all participants, health literacy was assessed with the three-item Chew scale.[59] [60] Following Wallace et al,[61] we assigned 1 point to each question for which the participant answered that they had any difficulty or required help, to produce a health literacy score of 0 (adequate), 1 (marginal), or 2 or 3 (inadequate). Health numeracy was assessed with the 8-item Short Numeracy Understanding in Medicine Instrument (S-NUMi).[62] [63] Following Schapira, a health numeracy score of 7 or higher was classified as high numeracy, 4 to 6 as adequate numeracy, and 3 or less as low numeracy.

In addition, participants were asked about their personal experience of administering pediatric medications, their medical training or experience, and personal and family demographics.


#

Analytic Methods

The primary outcome was number of correct answers to the comprehension questions. A secondary outcome was incorrect responses that would have led to overdoses or underdoses. To establish this, each of the incorrect response options for each question was classified a priori as a potential overdose or underdose by a pharmacist collaborator (A.S.), employing professional judgment and reference works as appropriate.

Student's t-tests and analysis of variance for continuous variables and chi-squared tests for categorical ones were used to assess comparability of the two arms as well as bivariate associations with the primary outcome (correct answers to the comprehension questions). Descriptive statistics and tables were computed in IBM SPSS Statistics version 24 (Armonk, New York, United States). Variables significant at 0.05 on bivariate analysis were considered for multivariate regression models to assess the secondary outcomes of the effects of literacy, numeracy, and demographics on comprehension. Multivariate models were constructed in SAS (version 9.4, Cary, North Carolina, United States) using proc GLM applying forward and backward selection to drop nonsignificant variables, and proc GLMselect using stepwise selection on the basis of the Akaike information criterion.


#

Human Subjects Research Approval

The Weill Cornell Institutional Review Board determined that the project was exempt because no personally identifying information was collected from the participants.


#
#

Results

A total of 1,012 individuals started the questionnaire, and 952 (94.1%) completed it (with equal dropout rates in the two arms, p = 0.23). One response was eliminated for an apparent data quality problem (self-reported age over 90 years) leaving a final sample size of 951.

As shown in [Table 2], half the sample were women, the mean age was ∼37 years, 83% were white, 38% had educational attainment of less than a bachelor's degree, and ∼12% had health insurance provided by Medicaid (the U.S. public insurance program for low-income individuals and families). One quarter reported that they had some form of medical experience or training. Inclusion criteria included having primary caregiving responsibilities for a child under the age of 18, but some participants also reported having children older than 18 years. Participant characteristics were well balanced across the two arms with the exception of age: respondents in the control (usual care) arm were an average of ∼1 year older than respondents in the intervention arm.

Table 2

Participant demographic and knowledge/skills characteristics

Characteristics

Whole group

Version 1

(usual care)

Version 2

(intervention)

p-Value

N

951

480

471

Demographic questions

Women, N (%)

475 (49.9)

240 (50.0)

235 (49.9)

0.97

Mean (SD) age

36.5 (9.5)

37.1 (9.7)

35.8 (9.4)

0.03

Race (multiple-choice possible), N (%)

 White or Caucasian

789 (82.8)

394 (81.9)

395 (83.7)

0.47

 Black or African-American

91 (9.5)

45 (9.4)

46 (9.7)

0.84

 Asian

46 (4.8)

28 (5.8)

18 (3.8)

0.15

 American Indian or Alaska Native

20 (2.1)

11 (2.3)

9 (1.9)

0.68

 Native Hawaiian or Pacific Islander

4 (0.4)

1 (0.2)

3 (0.6)

0.37

 Other

27 (2.8)

14 (2.9)

13 (2.8)

0.88

 Prefer not to say

7 (0.7)

2 (0.4)

5 (1.1)

0.25

Hispanic/Latino, N (%)

>0.99

 Hispanic/Latino

112 (11.8)

57 (11.9)

55 (11.7)

 Not Hispanic/Latino

823 (86.4)

415 (86.3)

408 (86.4)

 Prefer not to say

16 (1.7)

8 (1.7)

8 (1.7)

Education, N (%)

0.55

 Did not complete high school

16 (1.7)

8 (1.7)

8 (1.7)

 Completed high school

194 (20.4)

87 (18.1)

107 (22.7)

 Completed a 2-y college degree

151 (15.9)

82 (17.1)

69 (14.6)

 Completed a 4-y college degree

290 (30.5)

146 (30.4)

144 (30.6)

 Completed graduate degree

297 (31.2)

155 (32.3)

142 (30.1)

 Prefer not to say

3 (0.3)

2 (0.4)

1 (0.2)

Household income, N (%)

0.59

 0–$24,999

89 (9.3)

45 (9.4)

44 (9.3)

 $25,000–$49,999

159 (16.7)

75 (15.6)

84 (17.8)

 $50,000–$74,999

214 (22.5)

102 (21.2)

112 (23.7)

 $75,000–$99,999

231 (24.2)

126 (26.2)

105 (22.2)

 $100,000–$124,999

136 (14.3)

66 (13.7)

70 (14.8)

 $125,000 or more

122 (12.8)

66 (13.7)

56 (11.9)

Insurance coverage, N (%)

0.12

 None

38 (4.0)

18 (3.7)

20 (4.2)

 Private

564 (59.2)

292 (60.7)

272 (57.6)

 Medicare managed care

42 (4.4)

28 (5.8)

14 (3.0)

 Medicare

113 (11.9)

50 (10.4)

63 (13.3)

 Medicaid managed care

21 (2.2)

8 (1.7)

13 (2.8)

 Medicaid

90 (9.4)

50 (10.4)

40 (8.5)

 Both Medicare and Medicaid

41 (4.3)

18 (3.7)

23 (4.9)

 Don't know

16 (1.7)

5 (1.0)

11 (2.3)

 Other

26 (2.7)

11 (2.3)

15 (3.2)

Children's insurance, N (%)

0.77

 None

31 (3.3)

14 (2.9)

17 (3.6)

 Private

601 (63.1)

310 (64.4)

291 (61.7)

 Medicaid managed care

80 (8.4)

44 (9.1)

36 (7.8)

 Medicaid

190 (19.9)

89 (18.5)

101 (21.4)

 Don't know

17 (1.8)

8 (1.7)

9 (1.9)

 Other

32 (3.4)

15 (3.1)

17 (3.6)

Number of children

 Overall

1,879

977

902

 Mean (SD)

2.1 (1.1)

2.1 (1.2)

2.0 (1.1)

0.11

 Had at least one child <2 y

172

90

82

0.72

 Had at least one child 2–5 y

401

196

205

0.57

 Had at least one child 6–11 y

579

300

279

0.54

 Had at least one child 12–18 y

520

275

245

0.43

 Had at least one child >18 y

207

116

91

0.49

 Caregiver only; no children

39 (4.1%)

19 (4.0%)

20 (4.2%)

Knowledge and skills questions

How confident are you that you know your youngest child's current weight?

0.83

 Very

501 (52.7)

255 (53.1)

246 (52.2)

 Somewhat

386 (40.6)

191 (39.8)

195 (41.4)

 Not at all

64 (6.7)

34 (7.1)

30 (6.4)

 Has “medical training or experience”

240 (25.2)

124 (25.8)

116 (24.6)

0.67

Has given children medications often/very often in last 3 mo

251 (25.2)

121 (25.1)

120 (25.5)

0.14

Has used kitchen spoons or other nonstandard dosing for children

313 (32.9)

171 (35.6)

142 (30.2)

0.07

Health literacy, N (%)

0.17

 Adequate

542 (57.0)

258 (53.8)

262 (55.6)

 Marginal

203 (21.3)

104 (21.7)

99 (21.0)

 Inadequate

206 (21.7)

96 (20.0)

110 (23.4)

Health numeracy, N (%)

0.38

 High

235 (24.7)

126 (26.3)

109 (23.1)

 Adequate

430 (45.2)

207 (43.1)

223 (47.3)

 Low

286 (30.1)

147 (30.6)

139 (29.5)

Abbreviation: SD, standard deviation.


Note: Medicaid and Medicaid managed care are public insurance programs in the United States available only to low-income individuals and families. Medicare and Medicare managed care are public insurance programs in the United States available to all individuals over age 65 as well as to younger people with certain severe disabilities or chronic kidney failure.


As shown in [Table 3], the revised instructions were associated with a 1.1-point absolute or 13.3% relative improvement in the medication comprehension score (from 8.3 to 9.4 correct out of 15 questions; p < 0.01). In secondary outcomes, the intervention reduced the likelihood of responses that would have led to medication underdoses but not likelihood of selecting responses associated with overdoses ([Table 3], rows 2 and 3). For the individual comprehension questions, the revised instructions significantly improved the likelihood of selecting the correct answer for seven of the questions, significantly reduced the likelihood of selecting the correct answer for two questions, and made no difference for the remaining six questions ([Table 3]).

Table 3

Medication instruction comprehension results

Whole group

Version 1

(usual care)

Version 2

(intervention)

p-Value

Mean (SD) correct answers

8.9 (3.8)

8.3 (3.8)

9.4 (3.7)

<0.01a

Mean (SD) answer with risk of overdose

2.7 (1.8)

2.6 (1.8)

2.7 (1.8)

0.68

Mean (SD) answer with risk of underdose

2.7 (2.5)

3.2 (2.5)

2.1 (2.4)

<0.01a

Correct answers

 Question 1 (how many pills in one day), N (%)

650 (68.2)

276 (57.4)

374 (79.2)

<0.01a

 Question 2 (how many pills in the morning), N (%)

701 (73.6)

303 (63.0)

398 (84.3)

<0.01a

 Question 3 (how many drops in one day), N (%)

496 (52.0)

192 (39.9)

304 (64.4)

<0.01a

 Question 4 (how many drops in the morning), N (%)

569 (59.7)

244 (50.7)

325 (68.9)

<0.01a

 Question 5 (how many doses in one day), N (%)

626 (65.7)

308 (64.0)

318 (67.4)

0.28

 Question 6 (how many pills in one week), N (%)

629 (66.0)

314 (65.3)

315 (66.7)

0.63

 Question 7 (how much powder to make 6 ounces of formula), N (%)

705 (74.0)

328 (68.2)

377 (79.9)

<0.01a

 Question 8 (what is the last day to use the cream), N (%)

126 (13.2)

60 (12.5)

66 (14.0)

0.49

 Question 9 (should you give another dose now), N (%)

497 (52.2)

258 (53.6)

239 (50.6)

0.35

 Question 10 (how much medicine for a 6-mo-old), N (%)

478 (50.2)

219 (45.5)

259 (54.9)

< 0.01a

 Question 11 (when should your child eat breakfast), N (%)

505 (53.0)

291 (60.5)

214 (45.3)

<0.01a,b

 Question 12 (how many drops right now), N (%)

502 (52.7)

253 (52.6)

249 (52.8)

0.96

 Question 13 (how many times can you give the drops in a day), N (%)

638 (66.9)

268 (55.7)

370 (78.4)

<0.01a

 Question 14 (alternating medicines: which medicine now), N (%)

698 (73.2)

344 (71.5)

354 (75.0)

0.22

 Question 15 (alternating medicines: which medicine at 5 p.m.), N (%)

607 (63.7)

349 (72.6)

258 (54.7)

<0.01a,b

Abbreviation: SD, standard deviation.


Note: Findings marked with a are statistically significant at 0.05 in the hypothesized direction. Findings marked with b are statistically significant but in the direction opposite to the hypothesized direction.


Bivariate analyses showed that health numeracy was a very strong predictor of comprehension. Mean comprehension scores for individuals with high, adequate, and low numeracy were 12.0, 9.8, and 5.0, respectively. Health literacy was also a strong predictor, with mean scores for individuals with adequate, marginal, and inadequate literacy of 10.1, 8.0, and 6.4, respectively.

When we controlled for health numeracy and literacy in the multivariable analyses, none of the following demographic variables was statistically significant and therefore were dropped from the final model: parental age, race, ethnicity, education, household income, insurance status, being a self-reported frequent administrator of pediatric medications, and having medical training/experience. The final model (model R 2 = 0.562) demonstrated that being a woman, having higher health numeracy, having higher health literacy, and receiving the revised instructions were all significantly and independently associated with improvements in comprehension score ranging from 0.4 points to ∼1 point ([Table 4]). Interaction terms between version and numeracy and version and literacy were not statistically significant, suggesting the effect of the revision was comparable in all literacy and numeracy levels.

Table 4

Multivariate regression predictors of instruction comprehension

Parameter

Estimate

Standard error

t Value

Pr > |t|

Intercept

2.65

0.27

9.67

<0.0001

Received Version 2 (intervention)

1.07

0.16

6.50

<0.0001

Female

1.06

0.17

6.17

<0.0001

Health numeracy score

1.14

0.04

26.31

<0.0001

Health literacy category

0.41

0.10

4.14

<0.0001

Note: The estimates in this linear model represent estimated changes (improvements) in total comprehension score associated with each variable. Variables not included in the final model after forward/backward stepwise selection were: parental age, race, ethnicity, education, household income, insurance status, being a self-reported frequent administrator of pediatric medications, and having medical training/experience.



#

Discussion

Several plain-language revisions have been individually demonstrated to improve comprehension. In the current experiment, we demonstrated that a package of five of these revisions employed together was associated with improved comprehension of common medication instructions. In addition, the experiment showed that health literacy and health numeracy were both independently associated with comprehension, and that the effect size associated with the plain-language revisions did not differ by literacy level or numeracy level. The package of revisions was associated with fewer wrong answers that would have led to underdoses but not overdoses. Although the total comprehension score was higher with the revisions, there was no effect for a subset of six questions, and comprehension was lower for a subset of two questions.

Our findings were consistent with many other studies showing that health literacy was associated with misunderstanding of instructions[1] [2] [3] [6] [64] and that text revisions can assist in interpretation.[65] [66] In addition, however, we demonstrated that health numeracy was independently related to comprehension, and that numeracy accounted for significantly more of the variability in performance than literacy did. Furthermore, other demographic variables were not associated with comprehension in models that controlled for health literacy and numeracy (with the exception of gender). We conclude that poor health numeracy may be an underrecognized predictor of poor comprehension and may in fact account for many of the previously observed demographic predictors of medication misinterpretations.[13] Numeracy is a particularly important factor given that low numeracy is more prevalent than low literacy and is found even among people with adequate literacy.[10] [67] [68] Of the individuals in our study with adequate health literacy, 15% had low health numeracy.

In addition to these primary and secondary findings, it is noteworthy that medication instructions disseminated to patients via commercial EHR technology were frequently misunderstood by a diverse sample of parents of all literacy and numeracy levels. For example, many parents did not appear to understand that instructions including the word “max” indicated an upper maximum threshold for the medication. No individual instruction was well understood by more than 72% of respondents. Consequently, it is possible that these instructions pose a safety threat to pediatric patients. Furthermore, only 53% of the parents were confident that they knew the current weight of their youngest child, and 33% reported using kitchen utensils or other nonstandard devices for pediatric medications. Such basic information would be needed for self-administered weight-based dosing (such as might be common with over-the-counter medications), and about basic safety precautions about measuring pediatric medications. Our sample reported a very high rate of using nonstandard devices for pediatric medications, a practice that has previously been shown to increase the rate of medication errors.[22] Other studies have found that between 6 and 23% of parents used or described using nonstandard instruments.[5] [22] [61]

The effect size associated with this intervention was modest. However, as a supplement to more intensive high-touch interventions, we propose that revising EHR output to replace complex language with simpler language is a potentially scalable solution that could reduce medication administration errors by parents. Some of these highly effective high-touch interventions include providing patients with customized medication administration tools,[36] [37] consultations with dedicated medication nurses,[69] and illustrated or diagrammed instructions.[30] [32] [33] [52] [70]

Although the revised instructions were associated with comprehension scores ∼8 percentage points higher overall (from 55 to 63%), the effect size was not equal for each instruction. The revisions were associated with a significantly higher rate of correct answers for seven of the component questions and a significantly significant lower rate of correct answers for two of the component questions.

Limitations of the Study

This study should be interpreted in light of several limitations. Testing five plain-language revisions at once means that conclusions can be drawn only about the package of all five, not about the relative efficacy of each type of revision. The online-only format, although it produced a demographically diverse population with a range of education and literacy levels, probably excluded individuals in the very lowest computer literacy and/or literacy categories. It is possible, therefore, that real-world comprehension is actually worse than what we found in our sample. Also, many of the medication instructions used in this study were drawn from an EHR system that had already undergone in-house customization to replace abbreviations (e.g., “2 × ” was already automatically replaced with “2 times”). It is possible that the effect of our intervention would be even greater if the revised instructions were compared with the prescribing provider's original instructions.

The study was administered in English only. Black and Hispanic patients were underrepresented (9.5% of our sample was black compared with ∼13% of the U.S. population, and 11.8% were Hispanic compared with ∼17% of the U.S. population). A large number of respondents reported some medical experience or training. We specifically developed the inclusion criteria to include any adult caregiver of a child, and the demographics suggest the possibility that the sample may have included grandparents. However, we did not collect data that would have allowed us to break down the results by whether the respondent was a parent or another type of a caregiver. Because we were testing a hypothesis about application of plain-language guidelines, we did not pretest the questionnaire for optimization before the survey, which may have contributed to the situations in which revised instructions reduced comprehension. A final limitation is that we used hypothetical questions only; generalizability to actual medication administration is not known. However, it seems possible that parents of sick children in reality might perform worse than they did in this relatively low-stress questionnaire study.


#
#

Conclusion

Misinterpretations of pediatric medication instructions commonly provided to patients are frequent. Simple language revisions, most of which could be implemented in the EHR without the need for additional formatting or graphics, were associated with reduced frequency of misinterpretations overall, although not for every instruction. Revising EHR output through automated substitutions could be a scalable solution that would reduce the number of parents who misinterpret pediatric medication instructions. However, instructions were still subject to misinterpretation even after the revision, so this approach should be considered a low-effort supplement to more intensive high-touch interventions that could reduce parental medication administration errors. Furthermore, additional usability and literacy testing will be required to fully develop such an intervention and test it in practice. Future testing would be most beneficial if it focused specifically on the most vulnerable groups such as individuals with limited health literacy or English proficiency.


#

Clinical Relevance Statement

Patients derive important information from EHR-generated documents. A relatively simple intervention of replacing complex text with patient-friendly text reduced misinterpretations that would be likely to lead to medication mistakes. The intervention tested here could be automated with relatively simple phrase substitution.


#

Multiple Choice Questions

  1. Limited health numeracy, which is more prevalent than limited health literacy, is best defined as poor ability to:

    • Conduct statistical analysis of health data

    • Apply quantitative information to health decisions

    • Interpret peer-reviewed medical journal articles

    • Apply written and oral information to health decisions

    Correct answer: The correct answer is B, apply quantitative information to health decisions. Health numeracy is the set of skills and knowledge that patients need to read, understand, and apply quantitative information to personal health decisions. Health numeracy is not used to describe the more advanced set of quantitative skills used by physicians and scientists to analyze data or stay informed about the peer-reviewed literature. Health literacy similarly describes a broad set of skills and knowledge that patients need to read, understand, and apply written, textual, and oral information to their health; some authors consider health numeracy to be a subset of health literacy.

  2. Patients with limited health literacy have difficulty comprehending medication instructions. Examples of interventions that have been associated with significant improvements in comprehension by low-literacy populations include:

    • Medication instructions revised in plain language

    • Health literacy coursework for patients

    • Training on effective prescription writing for physicians

    • Fully automated consumer-friendly translations of instructions

    Correct answer: The correct answer is A, medication instructions revised in plain language. This study adds new evidence to an already extensive literature on revising medication instructions to improve patient comprehension. Health literacy coursework, and effective prescription writing training, both appear to be useful ideas but have not been demonstrated to improve medication comprehension by patients. Also, no fully automated method has yet been demonstrated to improve patient comprehension of medication instructions.


#

Appendix

Zoom Image
Zoom Image
Zoom Image

#
#

Conflict of Interest

None.

Acknowledgments

The researchers thank Sana Ali for assisting with the questionnaire programming.

Protection of Human and Animal Subjects

The study was reviewed by the Weill Cornell Institutional Review Board and was considered exempt because no personally identifying information was collected from the participants.


Funding

This study was supported by a pilot grant from the Department of Healthcare Policy & Research. Dr. Ancker is supported by K01 HS 021531 from the Agency for Healthcare Research and Quality.


  • References

  • 1 Davis TC, Federman AD, Bass III PF. , et al. Improving patient understanding of prescription drug label instructions. J Gen Intern Med 2009; 24 (01) 57-62
  • 2 Lokker N, Sanders L, Perrin EM. , et al. Parental misinterpretations of over-the-counter pediatric cough and cold medication labels. Pediatrics 2009; 123 (06) 1464-1471
  • 3 Bailey SC, Pandit AU, Yin S. , et al. Predictors of misunderstanding pediatric liquid medication instructions. Fam Med 2009; 41 (10) 715-721
  • 4 Waldrop-Valverde D, Osborn CY, Rodriguez A, Rothman RL, Kumar M, Jones DL. Numeracy skills explain racial differences in HIV medication management. AIDS Behav 2010; 14 (04) 799-806
  • 5 Yin HS, Dreyer BP, Foltin G, van Schaick L, Mendelsohn AL. Association of low caregiver health literacy with reported use of nonstandardized dosing instruments and lack of knowledge of weight-based dosing. Ambul Pediatr 2007; 7 (04) 292-298
  • 6 Yin HS, Johnson M, Mendelsohn AL, Abrams MA, Sanders LM, Dreyer BP. The health literacy of parents in the United States: a nationally representative study. Pediatrics 2009; 124 (Suppl. 03) S289-S298
  • 7 Yin HS, Mendelsohn AL, Wolf MS. , et al. Parents' medication administration errors: role of dosing instruments and health literacy. Arch Pediatr Adolesc Med 2010; 164 (02) 181-186
  • 8 Kutner M, Greenberg E, Jin Y, Paulsen C, White S. The Health Literacy of America's Adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Health Statistics; 2006. . Contract No.: NCES 2006–483
  • 9 McCray AT. Promoting health literacy. J Am Med Inform Assoc 2005; 12 (02) 152-163
  • 10 Kutner M, Greenberg E, Baer J. A First Look at the Literacy of America's Adults in the 21st Century: National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics; 2005
  • 11 Rudd R, Kirsch I, Yamamoto K. Literacy and Health in America. Princeton, NJ: Center for Global Assessment Policy Information Center Research and Development Educational Testing Service; 2004
  • 12 Nielsen-Bohlman L, Panzer AM, Kindig DA. , eds. Committee on Health Literacy. Health Literacy: A Prescription to End Confusion. Washington, DC: Institute of Medicine, The National Academies Press; 2004
  • 13 Ancker JS, Kaufman D. Rethinking health numeracy: a multidisciplinary literature review. J Am Med Inform Assoc 2007; 14 (06) 713-721
  • 14 Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Med Decis Making 2001; 21 (01) 37-44
  • 15 Ancker JS. Addressing health literacy and numeracy through systems approaches. In: Patel VL, Arocha JF, Ancker JS. , eds. Cognitive Informatics in Health and Biomedicine: Understanding and Modeling Health Behaviors. Cham: Springer International Publishing; 2017: 237-251
  • 16 Golbeck AL, Ahlers-Schmidt CR, Paschal AM, Dismuke SE. A definition and operational framework for health numeracy. Am J Prev Med 2005; 29 (04) 375-376
  • 17 Zandieh SO, Goldmann DA, Keohane CA, Yoon C, Bates DW, Kaushal R. Risk factors in preventable adverse drug events in pediatric outpatients. J Pediatr 2008; 152 (02) 225-231
  • 18 Walsh KE, Kaushal R, Chessare JB. How to avoid paediatric medication errors: a user's guide to the literature. Arch Dis Child 2005; 90 (07) 698-702
  • 19 Schillie SF, Shehab N, Thomas KE, Budnitz DS. Medication overdoses leading to emergency department visits among children. Am J Prev Med 2009; 37 (03) 181-187
  • 20 Kang AM, Brooks DE. US poison control center calls for infants 6 months of age and younger. Pediatrics 2016; 137 (02) e20151865
  • 21 Li SF, Lacher B, Crain EF. Acetaminophen and ibuprofen dosing by parents. Pediatr Emerg Care 2000; 16 (06) 394-397
  • 22 Yin HS, Dreyer BP, Ugboaja DC. , et al. Unit of measurement used and parent medication dosing errors. Pediatrics 2014; 134 (02) e354-e361
  • 23 Mehndiratta S. Strategies to reduce medication errors in pediatric ambulatory settings. J Postgrad Med 2012; 58 (01) 47-53
  • 24 Johnson KB, Butta JK, Donohue PK, Glenn DJ, Holtzman NA. Discharging patients with prescriptions instead of medications: sequelae in a teaching hospital. Pediatrics 1996; 97 (04) 481-485
  • 25 Simon HK, Weinkle DA. Over-the-counter medications. Do parents give what they intend to give?. Arch Pediatr Adolesc Med 1997; 151 (07) 654-656
  • 26 Patel VL, Branch T, Arocha JF. Errors in interpreting quantities as procedures: the case of pharmaceutical labels. Int J Med Inform 2002; 65 (03) 193-211
  • 27 Carayon P, Schoofs Hundt A, Karsh B-T. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
  • 28 Carayon P. , ed. Human Factors and Ergonomics in Healthcare and Patient Safety. 2nd ed. Boca Raton, FL: CRC Press, Taylor & Francis Group; 2012
  • 29 Rosson MB, Carroll JM. Usability Engineering: Scenario-based Development of Human-Computer Interaction. San Francisco: Morgan Kaufmann; 2002
  • 30 Moore JO, Boyer EW, Safren S. , et al. Designing interventions to overcome poor numeracy and improve medication adherence in chronic illness, including HIV/AIDS. J Med Toxicol 2011; 7 (02) 133-138
  • 31 Fogarty L, Roter D, Larson S, Burke J, Gillespie J, Levy R. Patient adherence to HIV medication regimens: a review of published and abstract reports. Patient Educ Couns 2002; 46 (02) 93-108
  • 32 Machtinger EL, Wang F, Chen LL, Rodriguez M, Wu S, Schillinger D. A visual medication schedule to improve anticoagulation control: a randomized, controlled trial. Jt Comm J Qual Patient Saf 2007; 33 (10) 625-635
  • 33 Kripalani S, Schmotzer B, Jacobson TA. Improving Medication Adherence through Graphically Enhanced Interventions in Coronary Heart Disease (IMAGE-CHD): a randomized controlled trial. J Gen Intern Med 2012; 27 (12) 1609-1617
  • 34 Bailey SC, Wolf MS, Lopez A. , et al. Expanding the Universal Medication Schedule: a patient-centred approach. BMJ Open 2014; 4 (01) e003699
  • 35 Yin HS, Dreyer BP, Moreira HA. , et al. Liquid medication dosing errors in children: role of provider counseling strategies. Acad Pediatr 2014; 14 (03) 262-270
  • 36 McMahon SR, Rimsza ME, Bay RC. Parents can dose liquid medication accurately. Pediatrics 1997; 100 (3 Pt 1): 330-333
  • 37 Frush KS, Luo X, Hutchinson P, Higgins JN. Evaluation of a method to reduce over-the-counter medication dosing error. Arch Pediatr Adolesc Med 2004; 158 (07) 620-624
  • 38 Baker DW. The meaning and the measure of health literacy. J Gen Intern Med 2006; 21 (08) 878-883
  • 39 Rudd RE. Improving Americans' health literacy. N Engl J Med 2010; 363 (24) 2283-2285
  • 40 Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005; 353 (05) 487-497
  • 41 Nieuwlaat R, Wilczynski N, Navarro T. , et al. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2014; (11) CD000011
  • 42 Workgroup to Evaluate and Enhance the Reach and Dissemination of Health Promotion Interventions. RE-AIM. Available at: http://re-aimorg . Accessed June 12, 2017
  • 43 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (09) 1322-1327
  • 44 Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: how can the RE-AIM framework help?. J Am Med Inform Assoc 2009; 16 (06) 889-897
  • 45 Sullivan JE, Farrar HC. ; Section on Clinical Pharmacology and Therapeutics; Committee on Drugs. Fever and antipyretic use in children. Pediatrics 2011; 127 (03) 580-587
  • 46 Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 2014; 96 (03) 395-403
  • 47 Leroy G, Endicott JE, Mouradi O, Kauchak D, Just ML. Improving perceived and actual text difficulty for health information consumers using semi-automated methods. AMIA Annu Symp Proc 2012; 2012: 522-531
  • 48 Julian-Reynier C, Welkenhuysen M, Hagoel L, Decruyenaere M, Hopwood P. ; CRISCOM Working Group. Risk communication strategies: state of the art and effectiveness in the context of cancer genetic services. Eur J Hum Genet 2003; 11 (10) 725-736
  • 49 www.plainlanguage.gov
  • 50 Bailey SC, Hasnain-Wynia R, Chen AH. , et al. Developing multilingual prescription instructions for patients with limited English proficiency. J Health Care Poor Underserved 2012; 23 (01) 81-87
  • 51 Bailey SC, Sarkar U, Chen AH, Schillinger D, Wolf MS. Evaluation of language concordant, patient-centered drug label instructions. J Gen Intern Med 2012; 27 (12) 1707-1713
  • 52 Yin HS, Mendelsohn AL, Fierman A, van Schaick L, Bazan IS, Dreyer BP. Use of a pictographic diagram to decrease parent dosing errors with infant acetaminophen: a health literacy perspective. Acad Pediatr 2011; 11 (01) 50-57
  • 53 Ancker JS, Senathirajah Y, Kukafka R, Starren JB. Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc 2006; 13 (06) 608-618
  • 54 Kincaid J, Fishburne R, Rogers R, Chissom B. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report 8–75. Memphis: Naval Air Station; 1975
  • 55 McLaughlin GH. SMOG grading: a new readability formula. J Read 1969; 12 (08) 639-646
  • 56 Leroy G, Kauchak D. The effect of word familiarity on actual and perceived text difficulty. J Am Med Inform Assoc 2014; 21 (e1): e169-e172
  • 57 Zarcadoolas C. The simplicity complex: exploring simplified health messages in a complex world. Health Promot Int 2011; 26 (03) 338-350
  • 58 Zarcadoolas C, Pleasant A, Greer DS. Understanding health literacy: an expanded model. Health Promot Int 2005; 20 (02) 195-203
  • 59 Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004; 36 (08) 588-594
  • 60 Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract 2006; 7 (21) 21
  • 61 Wallace LS, Keenum AJ, DeVoe JE, Bolon SK, Hansen JS. Women's understanding of different dosing instructions for a liquid pediatric medication. J Pediatr Health Care 2012; 26 (06) 443-450
  • 62 Schapira MM, Walker CM, Miller T. , et al. Development and validation of the numeracy understanding in Medicine Instrument short form. J Health Commun 2014; 19 (Suppl. 02) 240-253
  • 63 Schapira MM, Walker CM, Cappaert KJ. , et al. The numeracy understanding in medicine instrument: a measure of health numeracy developed using item response theory. Med Decis Making 2012; 32 (06) 851-865
  • 64 Davis TC, Wolf MS, Bass III PF. , et al. Literacy and misunderstanding prescription drug labels. Ann Intern Med 2006; 145 (12) 887-894
  • 65 Wolf MS, Davis TC, Curtis LM. , et al. A patient-centered prescription drug label to promote appropriate medication use and adherence. J Gen Intern Med 2016; 31 (12) 1482-1489
  • 66 Wolf MS, Bailey SC, Serper M. , et al. Comparative effectiveness of patient-centered strategies to improve FDA medication guides. Med Care 2014; 52 (09) 781-789
  • 67 Rampey BD, Finnegan R, Goodman M. , et al. Skills of US Unemployed, Young, and Older Adults in Sharper Focus: Results from the Program for the International Assessment of Adult Competencies (PIAAC) 2012/2014. Washington, DC: National Center for Education Statistics, U.S. Department of Education; 2016. . Contract No.: NCES 2016–039
  • 68 Ancker JS, Weber EU, Kukafka R. Effects of game-like interactive graphics on risk perceptions and decisions. Med Decis Making 2011; 31 (01) 130-142
  • 69 Greengold NL, Shane R, Schneider P. , et al. The impact of dedicated medication nurses on the medication administration error rate: a randomized controlled trial. Arch Intern Med 2003; 163 (19) 2359-2367
  • 70 Yin HS, Dreyer BP, van Schaick L, Foltin GL, Dinglas C, Mendelsohn AL. Randomized controlled trial of a pictogram-based intervention to reduce liquid medication dosing errors and improve adherence among caregivers of young children. Arch Pediatr Adolesc Med 2008; 162 (09) 814-822

Address for correspondence

Jessica S. Ancker, MPH, PhD
425 East 61st Street, Suite 301, New York, NY 10065
United States   

  • References

  • 1 Davis TC, Federman AD, Bass III PF. , et al. Improving patient understanding of prescription drug label instructions. J Gen Intern Med 2009; 24 (01) 57-62
  • 2 Lokker N, Sanders L, Perrin EM. , et al. Parental misinterpretations of over-the-counter pediatric cough and cold medication labels. Pediatrics 2009; 123 (06) 1464-1471
  • 3 Bailey SC, Pandit AU, Yin S. , et al. Predictors of misunderstanding pediatric liquid medication instructions. Fam Med 2009; 41 (10) 715-721
  • 4 Waldrop-Valverde D, Osborn CY, Rodriguez A, Rothman RL, Kumar M, Jones DL. Numeracy skills explain racial differences in HIV medication management. AIDS Behav 2010; 14 (04) 799-806
  • 5 Yin HS, Dreyer BP, Foltin G, van Schaick L, Mendelsohn AL. Association of low caregiver health literacy with reported use of nonstandardized dosing instruments and lack of knowledge of weight-based dosing. Ambul Pediatr 2007; 7 (04) 292-298
  • 6 Yin HS, Johnson M, Mendelsohn AL, Abrams MA, Sanders LM, Dreyer BP. The health literacy of parents in the United States: a nationally representative study. Pediatrics 2009; 124 (Suppl. 03) S289-S298
  • 7 Yin HS, Mendelsohn AL, Wolf MS. , et al. Parents' medication administration errors: role of dosing instruments and health literacy. Arch Pediatr Adolesc Med 2010; 164 (02) 181-186
  • 8 Kutner M, Greenberg E, Jin Y, Paulsen C, White S. The Health Literacy of America's Adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Health Statistics; 2006. . Contract No.: NCES 2006–483
  • 9 McCray AT. Promoting health literacy. J Am Med Inform Assoc 2005; 12 (02) 152-163
  • 10 Kutner M, Greenberg E, Baer J. A First Look at the Literacy of America's Adults in the 21st Century: National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics; 2005
  • 11 Rudd R, Kirsch I, Yamamoto K. Literacy and Health in America. Princeton, NJ: Center for Global Assessment Policy Information Center Research and Development Educational Testing Service; 2004
  • 12 Nielsen-Bohlman L, Panzer AM, Kindig DA. , eds. Committee on Health Literacy. Health Literacy: A Prescription to End Confusion. Washington, DC: Institute of Medicine, The National Academies Press; 2004
  • 13 Ancker JS, Kaufman D. Rethinking health numeracy: a multidisciplinary literature review. J Am Med Inform Assoc 2007; 14 (06) 713-721
  • 14 Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Med Decis Making 2001; 21 (01) 37-44
  • 15 Ancker JS. Addressing health literacy and numeracy through systems approaches. In: Patel VL, Arocha JF, Ancker JS. , eds. Cognitive Informatics in Health and Biomedicine: Understanding and Modeling Health Behaviors. Cham: Springer International Publishing; 2017: 237-251
  • 16 Golbeck AL, Ahlers-Schmidt CR, Paschal AM, Dismuke SE. A definition and operational framework for health numeracy. Am J Prev Med 2005; 29 (04) 375-376
  • 17 Zandieh SO, Goldmann DA, Keohane CA, Yoon C, Bates DW, Kaushal R. Risk factors in preventable adverse drug events in pediatric outpatients. J Pediatr 2008; 152 (02) 225-231
  • 18 Walsh KE, Kaushal R, Chessare JB. How to avoid paediatric medication errors: a user's guide to the literature. Arch Dis Child 2005; 90 (07) 698-702
  • 19 Schillie SF, Shehab N, Thomas KE, Budnitz DS. Medication overdoses leading to emergency department visits among children. Am J Prev Med 2009; 37 (03) 181-187
  • 20 Kang AM, Brooks DE. US poison control center calls for infants 6 months of age and younger. Pediatrics 2016; 137 (02) e20151865
  • 21 Li SF, Lacher B, Crain EF. Acetaminophen and ibuprofen dosing by parents. Pediatr Emerg Care 2000; 16 (06) 394-397
  • 22 Yin HS, Dreyer BP, Ugboaja DC. , et al. Unit of measurement used and parent medication dosing errors. Pediatrics 2014; 134 (02) e354-e361
  • 23 Mehndiratta S. Strategies to reduce medication errors in pediatric ambulatory settings. J Postgrad Med 2012; 58 (01) 47-53
  • 24 Johnson KB, Butta JK, Donohue PK, Glenn DJ, Holtzman NA. Discharging patients with prescriptions instead of medications: sequelae in a teaching hospital. Pediatrics 1996; 97 (04) 481-485
  • 25 Simon HK, Weinkle DA. Over-the-counter medications. Do parents give what they intend to give?. Arch Pediatr Adolesc Med 1997; 151 (07) 654-656
  • 26 Patel VL, Branch T, Arocha JF. Errors in interpreting quantities as procedures: the case of pharmaceutical labels. Int J Med Inform 2002; 65 (03) 193-211
  • 27 Carayon P, Schoofs Hundt A, Karsh B-T. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
  • 28 Carayon P. , ed. Human Factors and Ergonomics in Healthcare and Patient Safety. 2nd ed. Boca Raton, FL: CRC Press, Taylor & Francis Group; 2012
  • 29 Rosson MB, Carroll JM. Usability Engineering: Scenario-based Development of Human-Computer Interaction. San Francisco: Morgan Kaufmann; 2002
  • 30 Moore JO, Boyer EW, Safren S. , et al. Designing interventions to overcome poor numeracy and improve medication adherence in chronic illness, including HIV/AIDS. J Med Toxicol 2011; 7 (02) 133-138
  • 31 Fogarty L, Roter D, Larson S, Burke J, Gillespie J, Levy R. Patient adherence to HIV medication regimens: a review of published and abstract reports. Patient Educ Couns 2002; 46 (02) 93-108
  • 32 Machtinger EL, Wang F, Chen LL, Rodriguez M, Wu S, Schillinger D. A visual medication schedule to improve anticoagulation control: a randomized, controlled trial. Jt Comm J Qual Patient Saf 2007; 33 (10) 625-635
  • 33 Kripalani S, Schmotzer B, Jacobson TA. Improving Medication Adherence through Graphically Enhanced Interventions in Coronary Heart Disease (IMAGE-CHD): a randomized controlled trial. J Gen Intern Med 2012; 27 (12) 1609-1617
  • 34 Bailey SC, Wolf MS, Lopez A. , et al. Expanding the Universal Medication Schedule: a patient-centred approach. BMJ Open 2014; 4 (01) e003699
  • 35 Yin HS, Dreyer BP, Moreira HA. , et al. Liquid medication dosing errors in children: role of provider counseling strategies. Acad Pediatr 2014; 14 (03) 262-270
  • 36 McMahon SR, Rimsza ME, Bay RC. Parents can dose liquid medication accurately. Pediatrics 1997; 100 (3 Pt 1): 330-333
  • 37 Frush KS, Luo X, Hutchinson P, Higgins JN. Evaluation of a method to reduce over-the-counter medication dosing error. Arch Pediatr Adolesc Med 2004; 158 (07) 620-624
  • 38 Baker DW. The meaning and the measure of health literacy. J Gen Intern Med 2006; 21 (08) 878-883
  • 39 Rudd RE. Improving Americans' health literacy. N Engl J Med 2010; 363 (24) 2283-2285
  • 40 Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005; 353 (05) 487-497
  • 41 Nieuwlaat R, Wilczynski N, Navarro T. , et al. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2014; (11) CD000011
  • 42 Workgroup to Evaluate and Enhance the Reach and Dissemination of Health Promotion Interventions. RE-AIM. Available at: http://re-aimorg . Accessed June 12, 2017
  • 43 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (09) 1322-1327
  • 44 Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: how can the RE-AIM framework help?. J Am Med Inform Assoc 2009; 16 (06) 889-897
  • 45 Sullivan JE, Farrar HC. ; Section on Clinical Pharmacology and Therapeutics; Committee on Drugs. Fever and antipyretic use in children. Pediatrics 2011; 127 (03) 580-587
  • 46 Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 2014; 96 (03) 395-403
  • 47 Leroy G, Endicott JE, Mouradi O, Kauchak D, Just ML. Improving perceived and actual text difficulty for health information consumers using semi-automated methods. AMIA Annu Symp Proc 2012; 2012: 522-531
  • 48 Julian-Reynier C, Welkenhuysen M, Hagoel L, Decruyenaere M, Hopwood P. ; CRISCOM Working Group. Risk communication strategies: state of the art and effectiveness in the context of cancer genetic services. Eur J Hum Genet 2003; 11 (10) 725-736
  • 49 www.plainlanguage.gov
  • 50 Bailey SC, Hasnain-Wynia R, Chen AH. , et al. Developing multilingual prescription instructions for patients with limited English proficiency. J Health Care Poor Underserved 2012; 23 (01) 81-87
  • 51 Bailey SC, Sarkar U, Chen AH, Schillinger D, Wolf MS. Evaluation of language concordant, patient-centered drug label instructions. J Gen Intern Med 2012; 27 (12) 1707-1713
  • 52 Yin HS, Mendelsohn AL, Fierman A, van Schaick L, Bazan IS, Dreyer BP. Use of a pictographic diagram to decrease parent dosing errors with infant acetaminophen: a health literacy perspective. Acad Pediatr 2011; 11 (01) 50-57
  • 53 Ancker JS, Senathirajah Y, Kukafka R, Starren JB. Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc 2006; 13 (06) 608-618
  • 54 Kincaid J, Fishburne R, Rogers R, Chissom B. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report 8–75. Memphis: Naval Air Station; 1975
  • 55 McLaughlin GH. SMOG grading: a new readability formula. J Read 1969; 12 (08) 639-646
  • 56 Leroy G, Kauchak D. The effect of word familiarity on actual and perceived text difficulty. J Am Med Inform Assoc 2014; 21 (e1): e169-e172
  • 57 Zarcadoolas C. The simplicity complex: exploring simplified health messages in a complex world. Health Promot Int 2011; 26 (03) 338-350
  • 58 Zarcadoolas C, Pleasant A, Greer DS. Understanding health literacy: an expanded model. Health Promot Int 2005; 20 (02) 195-203
  • 59 Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004; 36 (08) 588-594
  • 60 Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract 2006; 7 (21) 21
  • 61 Wallace LS, Keenum AJ, DeVoe JE, Bolon SK, Hansen JS. Women's understanding of different dosing instructions for a liquid pediatric medication. J Pediatr Health Care 2012; 26 (06) 443-450
  • 62 Schapira MM, Walker CM, Miller T. , et al. Development and validation of the numeracy understanding in Medicine Instrument short form. J Health Commun 2014; 19 (Suppl. 02) 240-253
  • 63 Schapira MM, Walker CM, Cappaert KJ. , et al. The numeracy understanding in medicine instrument: a measure of health numeracy developed using item response theory. Med Decis Making 2012; 32 (06) 851-865
  • 64 Davis TC, Wolf MS, Bass III PF. , et al. Literacy and misunderstanding prescription drug labels. Ann Intern Med 2006; 145 (12) 887-894
  • 65 Wolf MS, Davis TC, Curtis LM. , et al. A patient-centered prescription drug label to promote appropriate medication use and adherence. J Gen Intern Med 2016; 31 (12) 1482-1489
  • 66 Wolf MS, Bailey SC, Serper M. , et al. Comparative effectiveness of patient-centered strategies to improve FDA medication guides. Med Care 2014; 52 (09) 781-789
  • 67 Rampey BD, Finnegan R, Goodman M. , et al. Skills of US Unemployed, Young, and Older Adults in Sharper Focus: Results from the Program for the International Assessment of Adult Competencies (PIAAC) 2012/2014. Washington, DC: National Center for Education Statistics, U.S. Department of Education; 2016. . Contract No.: NCES 2016–039
  • 68 Ancker JS, Weber EU, Kukafka R. Effects of game-like interactive graphics on risk perceptions and decisions. Med Decis Making 2011; 31 (01) 130-142
  • 69 Greengold NL, Shane R, Schneider P. , et al. The impact of dedicated medication nurses on the medication administration error rate: a randomized controlled trial. Arch Intern Med 2003; 163 (19) 2359-2367
  • 70 Yin HS, Dreyer BP, van Schaick L, Foltin GL, Dinglas C, Mendelsohn AL. Randomized controlled trial of a pictogram-based intervention to reduce liquid medication dosing errors and improve adherence among caregivers of young children. Arch Pediatr Adolesc Med 2008; 162 (09) 814-822

Zoom Image
Zoom Image
Zoom Image