Keywords
health literacy - pediatrics - drug overdose - medication errors - survey and questionnaires
Background and Significance
Background and Significance
Individualized medication instructions are typically written by physicians or pharmacists
and presented to the patient in the form of an electronic after-visit summary, discharge
instruction, or prescription printed or delivered via an electronic patient portal.
Unfortunately, it is well-established that patients frequently misunderstand clinician
language, even when clinicians try to be simple. For example, one study showed that
nearly 40% of a sample of clinic outpatients could not correctly operationalize the
seemingly simple instruction “twice a day.”[1] Comprehension problems are more frequent among those with low health literacy and
health numeracy.[2]
[3]
[4]
[5]
[6]
[7] An estimated 14% of U.S. adults[8] and up to 29% of parents[6] have limited health literacy, defined as the skills and knowledge needed to obtain,
understand, and apply information to their own health and medical decisions.[9]
[10]
[11]
[12] Low health numeracy is even more prevalent and is common at even the highest literacy
levels.[13]
[14] Health numeracy is the skill set needed to apply quantitative information to health,
including information about risks, times and dates, and quantities.[13]
[15]
[16]
Particularly in the case of pediatric medications, the consequences can be serious
or even fatal, as children are uniquely vulnerable to adverse events from medication
errors.[17]
[18] More than 70,000 children present to U.S. emergency departments each year with accidental
overdoses,[19] most caused by parent or caregiver administration errors.[2]
[20]
[21]
[22]
[23]
[24] No comparable national-level data are available about underdosing, but this has
also been found to be a common problem in small-scale studies.[2]
[22]
[25]
[26] In administering children's medications, parents must operationalize instructions
about dose, route, and frequency, as well as additional issues unique to children
such as weight-based dosing, age-based thresholds and contraindications, and liquid
medications.[2]
[22]
[25]
[26]
One way to prevent medication errors could be to redesign the product that facilitates
the errors, in this case, the medication instructions that parents frequently misunderstand.
This perspective comes from human factors engineering, the practice of optimizing human well-being and system performance by making products,
tasks, and systems compatible with the needs, abilities, and limitations of people,
as well as from usability engineering, the subset of human factors engineering that focuses on making electronic systems
easy to learn, satisfying to use, and matched to user needs, capabilities, and goals.[27]
[28]
[29] In fact, a large body of research already shows that patients, even those with low
literacy or numeracy, can effectively manage medication administration when supported
with evidence-based materials.[30]
[31] Examples include plain-language instructions,[1]
[3] as well as more complex materials such as visual schedules displaying pills on a
calendar,[32]
[33] “pictograph” illustrations,[1]
[7]
[34]
[35] and marked pillboxes and preloaded syringes.[31]
[35]
[36]
[37] In other words, the comprehension errors arise not solely from the patient's literacy
but rather from a system that fails to match communication modality to patient capabilities.[13]
[15]
[38]
[39]
To date, many of the communication strategies found to be efficacious in controlled
research situations have not been implemented widely in practice because they are
labor-intensive and difficult to implement consistently.[40]
[41] An intervention with modest efficacy that reaches large numbers of patients reproducibly
at low cost and effort may have a large public health impact.[42]
[43]
[44] In fact, the net effect may be larger than the effect of a highly efficacious intervention
that is challenging to implement and so reaches only small numbers.[42]
[43]
[44]
We therefore examined whether electronic health records (EHRs) could be targeted for
low-effort, scalable, usability interventions that would improve comprehension of
medication instructions.
Objective
Our objective was to assess whether easy-to-implement wording changes based on best
practices for plain language would improve comprehension of common medication instructions,
using a randomized experiment. The intervention involved wording changes with limited
visual illustrations, because this approach could be easily implemented and automated
in existing commercial EHRs. Our secondary objectives were (1) to assess the effects
of literacy, numeracy, and demographics on comprehension, and (2) to assess the effect
of the intervention on the subsets of wrong answers associated with overdoses and
underdoses.
Methods
Experimental Design
In this between-subject experiment, participants were randomly assigned to see the
original wording (usual care) or the revisions (intervention; [Table 1]), and then to answer 15 multiple-choice comprehension questions (see Appendix).
Table 1
Original and revised instructions
|
Original wording (usual care)
|
Revised wording (intervention)
|
|
Primary research question: Electronic health record instructions
|
|
2 pills every 12 h
|
Give 2 pills in the morning, and 2 pills in the evening
|
|
Administer 5 drops into both ears 2 times daily
|
In the morning: 5 drops in right ear and 5 drops in left ear
In the evening: 5 drops in right ear and 5 drops in left ear
|
|
10 mL by mouth every 4 h
|
Give 10 mL at all of these times
• Morning
• Noon-time
• Late afternoon
• Bedtime
• Midnight
• 4 a.m.
|
|
1 pill per day for 3 d, switch to 1 pill every other day for 4 d
|
Monday morning
|
Tuesday morning
|
Wednesday morning
|
Thursday morning
|
Friday morning
|
Saturday morning
|
Sunday morning
|
|
1 pill
|
1 pill
|
1 pill
|
No pill
|
1 pill
|
No pill
|
1 pill
|
|
Apply topically 4 times daily until rash is gone and for 3 additional days
|
Until rash is gone: Spread the cream on skin every morning, noon, afternoon, and evening
After rash is gone: Keep using the cream in the same way for 3 more days
|
|
Give every 4 h as needed for fever/pain, max 5 doses
|
Give this medicine if your child has fever or pain. Wait at least 4 h. Does the child
still have fever or pain? If so, give medicine again. Don't give it more than 5 times
in one day
|
|
Give on an empty stomach
|
Give 30 min before the child eats breakfast
|
|
2 drops each nostril ×3–4 per day as needed
|
If the child's nose is stuffy, give 2 drops in the right nostril and 2 drops in the
left nostril
How often per day: up to 4 times
|
|
For very high fevers, alternate between tylenol and ibuprofen every 3 h
|
If your child has a fever of 104 degrees or higher:
• Start with a dose of tylenol
• 3 h later, check to see if the fever is still high. If so, give a dose of ibuprofen
• 3 h later, check to see if the fever is still high. If so, give a dose of tylenol
You can switch between the medicines in this way until the fever comes down
|
|
Secondary research question: Over-the-counter medication instructions
|
|
1 unpacked level scoop (8.5 g) per 2 fl oz
|
To make
|
Use this much water
|
Mix in this much powder
|
|
2 ounces
|
2 ounces
|
1 scoop
|
|
4 ounces
|
4 ounces
|
2 scoops
|
|
6 ounces
|
6 ounces
|
3 scoops
|
|
Weight (lb)
|
Age (mo)
|
Dose (mL)
|
If you know your child's weight, use this information:
• under 18 pounds, give 1.25 mL
• 18 pounds or more, give 1.875 mL
If you do not know your child's weight, use this information:
• less than 12 mo old, give 1.25 mL
• 12 mo or older, give 1.875 mL
But if your child is under 6 mo old, ask a doctor whether you should use this medicine.
|
|
–
|
Under 6 mo
|
Ask a doctor
|
|
12–17 lbs
|
6–11 mo
|
1.25 mL
|
|
18–23 lbs
|
12–23 mo
|
1.875 mL
|
Setting and Sample
Survey Sampling International (SSI; www.surveysampling.com) is an international survey panel and market research firm widely used in online
surveys. We contracted with SSI to recruit a sample. The inclusion criteria were that
participants had to be: (1) U.S. adults with primary caregiving responsibilities for
at least one child under the age of 18, and (2) comfortable completing a questionnaire
in English. We also specified to SSI that a minimum of 30% of the sample should have
less than a college education. SSI has a large existing online panel of registered
individuals who have agreed to be contacted for online surveys and questionnaires
to be entered into drawings to earn modest incentives of their choice (which might
include money, airline miles, etc.). These registered individuals have all completed
extensive demographic questionnaires. Using our eligibility criteria, SSI disseminated
the recruitment announcement to individuals who met our criteria, while continuously
monitoring the education levels of those who agreed to participate. Recruitment was
closed early for the higher education category and extended for the lower education
category to ensure the 30% representation we had specified.
Questionnaire Development
We selected examples of common dosing instructions from the after-visit summary of
a commercial EHR (Epic Systems, Verona, Wisconsin, United States). Instructions in
our institution's after-visit summary already take a step toward patient-centeredness
by replacing acronyms and abbreviations (e.g., a provider instruction of “2× per day”
is automatically replaced with “2 times a day”).
For exploratory purposes, we also added two questions from common over-the-counter
pediatric pain relievers and formula, plus an instruction to alternate between acetaminophen
and ibuprofen for fever reduction. (Although this practice is not encouraged at our
institution in light of an American Academy of Pediatrics white paper,[45] we included it because patients elsewhere may encounter it.)
To create the revised versions of the instructions, we selected a package of five
plain-language principles that were well supported by evidence (although not always
in the context of medication instructions; see references) and that appeared to be
easily applicable to informatics practice (e.g., could potentially be accomplished
through automatic term substitutions rather than major redesign). We drew from www.plainlanguage.gov and other resources such as Shoemaker et al[46] and the references listed below:
-
Avoid unfamiliar terms, jargon, and abbreviations. This involved substituting familiar
terms for unfamiliar ones where possible,[47]
[48]
[49]
[50] defining medical terms parenthetically when they could not be replaced with more
familiar terms,[46]
[50] and inserting explanations of selected concepts that require medical knowledge (e.g.,
for “on an empty stomach,” explaining how long before and after the meal the child
would have an empty stomach).[46]
-
Perform computations for the reader.[13]
[46] Instead of instructing the reader to administer medications every X hours, administration
times were phrased in terms of recurring events such as morning and evening or mealtimes.[1]
[50]
[51]
-
Avoid complex sentences containing multiple instructions or steps.[46]
[49] This involved putting one instruction or step per bullet or sentence,[46] separating multiple “if-then” or “when” conditions into a separate instruction,[49] and keeping the “if” or “when” statement short and placing it after the verb.[49]
-
Use active voice and address the reader.[46]
-
Use illustrations if they reinforce or explain the text.[46]
[49]
[52]
[53]
We opted not to apply formulas to reduce the “grade level” of the text; revisions
to reduce the grade level would have involved replacing long words with short ones
and breaking long sentences into shorter ones.[54]
[55] This decision was made on basis of research suggesting that word familiarity was
more important to comprehension than word length,[56] and that the contextual explanations needed to help novices understand health-related
text often make text longer.[57]
[58]
Covariates
For all participants, health literacy was assessed with the three-item Chew scale.[59]
[60] Following Wallace et al,[61] we assigned 1 point to each question for which the participant answered that they
had any difficulty or required help, to produce a health literacy score of 0 (adequate),
1 (marginal), or 2 or 3 (inadequate). Health numeracy was assessed with the 8-item
Short Numeracy Understanding in Medicine Instrument (S-NUMi).[62]
[63] Following Schapira, a health numeracy score of 7 or higher was classified as high
numeracy, 4 to 6 as adequate numeracy, and 3 or less as low numeracy.
In addition, participants were asked about their personal experience of administering
pediatric medications, their medical training or experience, and personal and family
demographics.
Analytic Methods
The primary outcome was number of correct answers to the comprehension questions.
A secondary outcome was incorrect responses that would have led to overdoses or underdoses.
To establish this, each of the incorrect response options for each question was classified
a priori as a potential overdose or underdose by a pharmacist collaborator (A.S.),
employing professional judgment and reference works as appropriate.
Student's t-tests and analysis of variance for continuous variables and chi-squared tests for
categorical ones were used to assess comparability of the two arms as well as bivariate
associations with the primary outcome (correct answers to the comprehension questions).
Descriptive statistics and tables were computed in IBM SPSS Statistics version 24
(Armonk, New York, United States). Variables significant at 0.05 on bivariate analysis
were considered for multivariate regression models to assess the secondary outcomes
of the effects of literacy, numeracy, and demographics on comprehension. Multivariate
models were constructed in SAS (version 9.4, Cary, North Carolina, United States)
using proc GLM applying forward and backward selection to drop nonsignificant variables,
and proc GLMselect using stepwise selection on the basis of the Akaike information
criterion.
Human Subjects Research Approval
The Weill Cornell Institutional Review Board determined that the project was exempt
because no personally identifying information was collected from the participants.
Results
A total of 1,012 individuals started the questionnaire, and 952 (94.1%) completed
it (with equal dropout rates in the two arms, p = 0.23). One response was eliminated for an apparent data quality problem (self-reported
age over 90 years) leaving a final sample size of 951.
As shown in [Table 2], half the sample were women, the mean age was ∼37 years, 83% were white, 38% had
educational attainment of less than a bachelor's degree, and ∼12% had health insurance
provided by Medicaid (the U.S. public insurance program for low-income individuals
and families). One quarter reported that they had some form of medical experience
or training. Inclusion criteria included having primary caregiving responsibilities
for a child under the age of 18, but some participants also reported having children
older than 18 years. Participant characteristics were well balanced across the two
arms with the exception of age: respondents in the control (usual care) arm were an
average of ∼1 year older than respondents in the intervention arm.
Table 2
Participant demographic and knowledge/skills characteristics
|
Characteristics
|
Whole group
|
Version 1
(usual care)
|
Version 2
(intervention)
|
p-Value
|
|
N
|
951
|
480
|
471
|
|
|
Demographic questions
|
|
|
|
|
|
Women, N (%)
|
475 (49.9)
|
240 (50.0)
|
235 (49.9)
|
0.97
|
|
Mean (SD) age
|
36.5 (9.5)
|
37.1 (9.7)
|
35.8 (9.4)
|
0.03
|
|
Race (multiple-choice possible), N (%)
|
|
|
|
|
|
White or Caucasian
|
789 (82.8)
|
394 (81.9)
|
395 (83.7)
|
0.47
|
|
Black or African-American
|
91 (9.5)
|
45 (9.4)
|
46 (9.7)
|
0.84
|
|
Asian
|
46 (4.8)
|
28 (5.8)
|
18 (3.8)
|
0.15
|
|
American Indian or Alaska Native
|
20 (2.1)
|
11 (2.3)
|
9 (1.9)
|
0.68
|
|
Native Hawaiian or Pacific Islander
|
4 (0.4)
|
1 (0.2)
|
3 (0.6)
|
0.37
|
|
Other
|
27 (2.8)
|
14 (2.9)
|
13 (2.8)
|
0.88
|
|
Prefer not to say
|
7 (0.7)
|
2 (0.4)
|
5 (1.1)
|
0.25
|
|
Hispanic/Latino, N (%)
|
|
|
|
>0.99
|
|
Hispanic/Latino
|
112 (11.8)
|
57 (11.9)
|
55 (11.7)
|
|
|
Not Hispanic/Latino
|
823 (86.4)
|
415 (86.3)
|
408 (86.4)
|
|
|
Prefer not to say
|
16 (1.7)
|
8 (1.7)
|
8 (1.7)
|
|
|
Education, N (%)
|
|
|
|
0.55
|
|
Did not complete high school
|
16 (1.7)
|
8 (1.7)
|
8 (1.7)
|
|
|
Completed high school
|
194 (20.4)
|
87 (18.1)
|
107 (22.7)
|
|
|
Completed a 2-y college degree
|
151 (15.9)
|
82 (17.1)
|
69 (14.6)
|
|
|
Completed a 4-y college degree
|
290 (30.5)
|
146 (30.4)
|
144 (30.6)
|
|
|
Completed graduate degree
|
297 (31.2)
|
155 (32.3)
|
142 (30.1)
|
|
|
Prefer not to say
|
3 (0.3)
|
2 (0.4)
|
1 (0.2)
|
|
|
Household income, N (%)
|
|
|
|
0.59
|
|
0–$24,999
|
89 (9.3)
|
45 (9.4)
|
44 (9.3)
|
|
|
$25,000–$49,999
|
159 (16.7)
|
75 (15.6)
|
84 (17.8)
|
|
|
$50,000–$74,999
|
214 (22.5)
|
102 (21.2)
|
112 (23.7)
|
|
|
$75,000–$99,999
|
231 (24.2)
|
126 (26.2)
|
105 (22.2)
|
|
|
$100,000–$124,999
|
136 (14.3)
|
66 (13.7)
|
70 (14.8)
|
|
|
$125,000 or more
|
122 (12.8)
|
66 (13.7)
|
56 (11.9)
|
|
|
Insurance coverage, N (%)
|
|
|
|
0.12
|
|
None
|
38 (4.0)
|
18 (3.7)
|
20 (4.2)
|
|
|
Private
|
564 (59.2)
|
292 (60.7)
|
272 (57.6)
|
|
|
Medicare managed care
|
42 (4.4)
|
28 (5.8)
|
14 (3.0)
|
|
|
Medicare
|
113 (11.9)
|
50 (10.4)
|
63 (13.3)
|
|
|
Medicaid managed care
|
21 (2.2)
|
8 (1.7)
|
13 (2.8)
|
|
|
Medicaid
|
90 (9.4)
|
50 (10.4)
|
40 (8.5)
|
|
|
Both Medicare and Medicaid
|
41 (4.3)
|
18 (3.7)
|
23 (4.9)
|
|
|
Don't know
|
16 (1.7)
|
5 (1.0)
|
11 (2.3)
|
|
|
Other
|
26 (2.7)
|
11 (2.3)
|
15 (3.2)
|
|
|
Children's insurance, N (%)
|
|
|
|
0.77
|
|
None
|
31 (3.3)
|
14 (2.9)
|
17 (3.6)
|
|
|
Private
|
601 (63.1)
|
310 (64.4)
|
291 (61.7)
|
|
|
Medicaid managed care
|
80 (8.4)
|
44 (9.1)
|
36 (7.8)
|
|
|
Medicaid
|
190 (19.9)
|
89 (18.5)
|
101 (21.4)
|
|
|
Don't know
|
17 (1.8)
|
8 (1.7)
|
9 (1.9)
|
|
|
Other
|
32 (3.4)
|
15 (3.1)
|
17 (3.6)
|
|
|
Number of children
|
|
|
|
|
|
Overall
|
1,879
|
977
|
902
|
|
|
Mean (SD)
|
2.1 (1.1)
|
2.1 (1.2)
|
2.0 (1.1)
|
0.11
|
|
Had at least one child <2 y
|
172
|
90
|
82
|
0.72
|
|
Had at least one child 2–5 y
|
401
|
196
|
205
|
0.57
|
|
Had at least one child 6–11 y
|
579
|
300
|
279
|
0.54
|
|
Had at least one child 12–18 y
|
520
|
275
|
245
|
0.43
|
|
Had at least one child >18 y
|
207
|
116
|
91
|
0.49
|
|
Caregiver only; no children
|
39 (4.1%)
|
19 (4.0%)
|
20 (4.2%)
|
|
|
Knowledge and skills questions
|
|
|
|
|
|
How confident are you that you know your youngest child's current weight?
|
|
|
|
0.83
|
|
Very
|
501 (52.7)
|
255 (53.1)
|
246 (52.2)
|
|
Somewhat
|
386 (40.6)
|
191 (39.8)
|
195 (41.4)
|
|
Not at all
|
64 (6.7)
|
34 (7.1)
|
30 (6.4)
|
|
Has “medical training or experience”
|
240 (25.2)
|
124 (25.8)
|
116 (24.6)
|
0.67
|
|
Has given children medications often/very often in last 3 mo
|
251 (25.2)
|
121 (25.1)
|
120 (25.5)
|
0.14
|
|
Has used kitchen spoons or other nonstandard dosing for children
|
313 (32.9)
|
171 (35.6)
|
142 (30.2)
|
0.07
|
|
Health literacy, N (%)
|
|
|
|
0.17
|
|
Adequate
|
542 (57.0)
|
258 (53.8)
|
262 (55.6)
|
|
|
Marginal
|
203 (21.3)
|
104 (21.7)
|
99 (21.0)
|
|
|
Inadequate
|
206 (21.7)
|
96 (20.0)
|
110 (23.4)
|
|
|
Health numeracy, N (%)
|
|
|
|
0.38
|
|
High
|
235 (24.7)
|
126 (26.3)
|
109 (23.1)
|
|
|
Adequate
|
430 (45.2)
|
207 (43.1)
|
223 (47.3)
|
|
|
Low
|
286 (30.1)
|
147 (30.6)
|
139 (29.5)
|
|
Abbreviation: SD, standard deviation.
Note: Medicaid and Medicaid managed care are public insurance programs in the United
States available only to low-income individuals and families. Medicare and Medicare
managed care are public insurance programs in the United States available to all individuals
over age 65 as well as to younger people with certain severe disabilities or chronic
kidney failure.
As shown in [Table 3], the revised instructions were associated with a 1.1-point absolute or 13.3% relative
improvement in the medication comprehension score (from 8.3 to 9.4 correct out of
15 questions; p < 0.01). In secondary outcomes, the intervention reduced the likelihood of responses
that would have led to medication underdoses but not likelihood of selecting responses
associated with overdoses ([Table 3], rows 2 and 3). For the individual comprehension questions, the revised instructions
significantly improved the likelihood of selecting the correct answer for seven of
the questions, significantly reduced the likelihood of selecting the correct answer
for two questions, and made no difference for the remaining six questions ([Table 3]).
Table 3
Medication instruction comprehension results
|
Whole group
|
Version 1
(usual care)
|
Version 2
(intervention)
|
p-Value
|
|
Mean (SD) correct answers
|
8.9 (3.8)
|
8.3 (3.8)
|
9.4 (3.7)
|
<0.01a
|
|
Mean (SD) answer with risk of overdose
|
2.7 (1.8)
|
2.6 (1.8)
|
2.7 (1.8)
|
0.68
|
|
Mean (SD) answer with risk of underdose
|
2.7 (2.5)
|
3.2 (2.5)
|
2.1 (2.4)
|
<0.01a
|
|
Correct answers
|
|
|
|
|
|
Question 1 (how many pills in one day), N (%)
|
650 (68.2)
|
276 (57.4)
|
374 (79.2)
|
<0.01a
|
|
Question 2 (how many pills in the morning), N (%)
|
701 (73.6)
|
303 (63.0)
|
398 (84.3)
|
<0.01a
|
|
Question 3 (how many drops in one day), N (%)
|
496 (52.0)
|
192 (39.9)
|
304 (64.4)
|
<0.01a
|
|
Question 4 (how many drops in the morning), N (%)
|
569 (59.7)
|
244 (50.7)
|
325 (68.9)
|
<0.01a
|
|
Question 5 (how many doses in one day), N (%)
|
626 (65.7)
|
308 (64.0)
|
318 (67.4)
|
0.28
|
|
Question 6 (how many pills in one week), N (%)
|
629 (66.0)
|
314 (65.3)
|
315 (66.7)
|
0.63
|
|
Question 7 (how much powder to make 6 ounces of formula), N (%)
|
705 (74.0)
|
328 (68.2)
|
377 (79.9)
|
<0.01a
|
|
Question 8 (what is the last day to use the cream), N (%)
|
126 (13.2)
|
60 (12.5)
|
66 (14.0)
|
0.49
|
|
Question 9 (should you give another dose now), N (%)
|
497 (52.2)
|
258 (53.6)
|
239 (50.6)
|
0.35
|
|
Question 10 (how much medicine for a 6-mo-old), N (%)
|
478 (50.2)
|
219 (45.5)
|
259 (54.9)
|
< 0.01a
|
|
Question 11 (when should your child eat breakfast), N (%)
|
505 (53.0)
|
291 (60.5)
|
214 (45.3)
|
<0.01a,b
|
|
Question 12 (how many drops right now), N (%)
|
502 (52.7)
|
253 (52.6)
|
249 (52.8)
|
0.96
|
|
Question 13 (how many times can you give the drops in a day), N (%)
|
638 (66.9)
|
268 (55.7)
|
370 (78.4)
|
<0.01a
|
|
Question 14 (alternating medicines: which medicine now), N (%)
|
698 (73.2)
|
344 (71.5)
|
354 (75.0)
|
0.22
|
|
Question 15 (alternating medicines: which medicine at 5 p.m.), N (%)
|
607 (63.7)
|
349 (72.6)
|
258 (54.7)
|
<0.01a,b
|
Abbreviation: SD, standard deviation.
Note: Findings marked with a are statistically significant at 0.05 in the hypothesized direction. Findings marked
with b are statistically significant but in the direction opposite to the hypothesized direction.
Bivariate analyses showed that health numeracy was a very strong predictor of comprehension.
Mean comprehension scores for individuals with high, adequate, and low numeracy were
12.0, 9.8, and 5.0, respectively. Health literacy was also a strong predictor, with
mean scores for individuals with adequate, marginal, and inadequate literacy of 10.1,
8.0, and 6.4, respectively.
When we controlled for health numeracy and literacy in the multivariable analyses,
none of the following demographic variables was statistically significant and therefore
were dropped from the final model: parental age, race, ethnicity, education, household
income, insurance status, being a self-reported frequent administrator of pediatric
medications, and having medical training/experience. The final model (model R
2 = 0.562) demonstrated that being a woman, having higher health numeracy, having higher
health literacy, and receiving the revised instructions were all significantly and
independently associated with improvements in comprehension score ranging from 0.4
points to ∼1 point ([Table 4]). Interaction terms between version and numeracy and version and literacy were not
statistically significant, suggesting the effect of the revision was comparable in
all literacy and numeracy levels.
Table 4
Multivariate regression predictors of instruction comprehension
|
Parameter
|
Estimate
|
Standard error
|
t Value
|
Pr > |t|
|
|
Intercept
|
2.65
|
0.27
|
9.67
|
<0.0001
|
|
Received Version 2 (intervention)
|
1.07
|
0.16
|
6.50
|
<0.0001
|
|
Female
|
1.06
|
0.17
|
6.17
|
<0.0001
|
|
Health numeracy score
|
1.14
|
0.04
|
26.31
|
<0.0001
|
|
Health literacy category
|
0.41
|
0.10
|
4.14
|
<0.0001
|
Note: The estimates in this linear model represent estimated changes (improvements)
in total comprehension score associated with each variable. Variables not included
in the final model after forward/backward stepwise selection were: parental age, race,
ethnicity, education, household income, insurance status, being a self-reported frequent
administrator of pediatric medications, and having medical training/experience.
Discussion
Several plain-language revisions have been individually demonstrated to improve comprehension.
In the current experiment, we demonstrated that a package of five of these revisions
employed together was associated with improved comprehension of common medication
instructions. In addition, the experiment showed that health literacy and health numeracy
were both independently associated with comprehension, and that the effect size associated
with the plain-language revisions did not differ by literacy level or numeracy level.
The package of revisions was associated with fewer wrong answers that would have led
to underdoses but not overdoses. Although the total comprehension score was higher
with the revisions, there was no effect for a subset of six questions, and comprehension
was lower for a subset of two questions.
Our findings were consistent with many other studies showing that health literacy
was associated with misunderstanding of instructions[1]
[2]
[3]
[6]
[64] and that text revisions can assist in interpretation.[65]
[66] In addition, however, we demonstrated that health numeracy was independently related
to comprehension, and that numeracy accounted for significantly more of the variability
in performance than literacy did. Furthermore, other demographic variables were not
associated with comprehension in models that controlled for health literacy and numeracy
(with the exception of gender). We conclude that poor health numeracy may be an underrecognized
predictor of poor comprehension and may in fact account for many of the previously
observed demographic predictors of medication misinterpretations.[13] Numeracy is a particularly important factor given that low numeracy is more prevalent
than low literacy and is found even among people with adequate literacy.[10]
[67]
[68] Of the individuals in our study with adequate health literacy, 15% had low health
numeracy.
In addition to these primary and secondary findings, it is noteworthy that medication
instructions disseminated to patients via commercial EHR technology were frequently
misunderstood by a diverse sample of parents of all literacy and numeracy levels.
For example, many parents did not appear to understand that instructions including
the word “max” indicated an upper maximum threshold for the medication. No individual
instruction was well understood by more than 72% of respondents. Consequently, it
is possible that these instructions pose a safety threat to pediatric patients. Furthermore,
only 53% of the parents were confident that they knew the current weight of their
youngest child, and 33% reported using kitchen utensils or other nonstandard devices
for pediatric medications. Such basic information would be needed for self-administered
weight-based dosing (such as might be common with over-the-counter medications), and
about basic safety precautions about measuring pediatric medications. Our sample reported
a very high rate of using nonstandard devices for pediatric medications, a practice
that has previously been shown to increase the rate of medication errors.[22] Other studies have found that between 6 and 23% of parents used or described using
nonstandard instruments.[5]
[22]
[61]
The effect size associated with this intervention was modest. However, as a supplement
to more intensive high-touch interventions, we propose that revising EHR output to
replace complex language with simpler language is a potentially scalable solution
that could reduce medication administration errors by parents. Some of these highly
effective high-touch interventions include providing patients with customized medication
administration tools,[36]
[37] consultations with dedicated medication nurses,[69] and illustrated or diagrammed instructions.[30]
[32]
[33]
[52]
[70]
Although the revised instructions were associated with comprehension scores ∼8 percentage
points higher overall (from 55 to 63%), the effect size was not equal for each instruction.
The revisions were associated with a significantly higher rate of correct answers
for seven of the component questions and a significantly significant lower rate of
correct answers for two of the component questions.
Limitations of the Study
This study should be interpreted in light of several limitations. Testing five plain-language
revisions at once means that conclusions can be drawn only about the package of all
five, not about the relative efficacy of each type of revision. The online-only format,
although it produced a demographically diverse population with a range of education
and literacy levels, probably excluded individuals in the very lowest computer literacy
and/or literacy categories. It is possible, therefore, that real-world comprehension
is actually worse than what we found in our sample. Also, many of the medication instructions
used in this study were drawn from an EHR system that had already undergone in-house
customization to replace abbreviations (e.g., “2 × ” was already automatically replaced
with “2 times”). It is possible that the effect of our intervention would be even
greater if the revised instructions were compared with the prescribing provider's
original instructions.
The study was administered in English only. Black and Hispanic patients were underrepresented
(9.5% of our sample was black compared with ∼13% of the U.S. population, and 11.8%
were Hispanic compared with ∼17% of the U.S. population). A large number of respondents
reported some medical experience or training. We specifically developed the inclusion
criteria to include any adult caregiver of a child, and the demographics suggest the
possibility that the sample may have included grandparents. However, we did not collect
data that would have allowed us to break down the results by whether the respondent
was a parent or another type of a caregiver. Because we were testing a hypothesis
about application of plain-language guidelines, we did not pretest the questionnaire
for optimization before the survey, which may have contributed to the situations in
which revised instructions reduced comprehension. A final limitation is that we used
hypothetical questions only; generalizability to actual medication administration
is not known. However, it seems possible that parents of sick children in reality
might perform worse than they did in this relatively low-stress questionnaire study.
Conclusion
Misinterpretations of pediatric medication instructions commonly provided to patients
are frequent. Simple language revisions, most of which could be implemented in the
EHR without the need for additional formatting or graphics, were associated with reduced
frequency of misinterpretations overall, although not for every instruction. Revising
EHR output through automated substitutions could be a scalable solution that would
reduce the number of parents who misinterpret pediatric medication instructions. However,
instructions were still subject to misinterpretation even after the revision, so this
approach should be considered a low-effort supplement to more intensive high-touch
interventions that could reduce parental medication administration errors. Furthermore,
additional usability and literacy testing will be required to fully develop such an
intervention and test it in practice. Future testing would be most beneficial if it
focused specifically on the most vulnerable groups such as individuals with limited
health literacy or English proficiency.
Clinical Relevance Statement
Clinical Relevance Statement
Patients derive important information from EHR-generated documents. A relatively simple
intervention of replacing complex text with patient-friendly text reduced misinterpretations
that would be likely to lead to medication mistakes. The intervention tested here
could be automated with relatively simple phrase substitution.
Multiple Choice Questions
Multiple Choice Questions
-
Limited health numeracy, which is more prevalent than limited health literacy, is
best defined as poor ability to:
-
Conduct statistical analysis of health data
-
Apply quantitative information to health decisions
-
Interpret peer-reviewed medical journal articles
-
Apply written and oral information to health decisions
Correct answer: The correct answer is B, apply quantitative information to health decisions. Health
numeracy is the set of skills and knowledge that patients need to read, understand,
and apply quantitative information to personal health decisions. Health numeracy is
not used to describe the more advanced set of quantitative skills used by physicians
and scientists to analyze data or stay informed about the peer-reviewed literature.
Health literacy similarly describes a broad set of skills and knowledge that patients
need to read, understand, and apply written, textual, and oral information to their
health; some authors consider health numeracy to be a subset of health literacy.
-
Patients with limited health literacy have difficulty comprehending medication instructions.
Examples of interventions that have been associated with significant improvements
in comprehension by low-literacy populations include:
-
Medication instructions revised in plain language
-
Health literacy coursework for patients
-
Training on effective prescription writing for physicians
-
Fully automated consumer-friendly translations of instructions
Correct answer: The correct answer is A, medication instructions revised in plain language. This
study adds new evidence to an already extensive literature on revising medication
instructions to improve patient comprehension. Health literacy coursework, and effective
prescription writing training, both appear to be useful ideas but have not been demonstrated
to improve medication comprehension by patients. Also, no fully automated method has
yet been demonstrated to improve patient comprehension of medication instructions.
