Keywords tobacco use treatment - clinical decision support - human-centered design
Background and Significance
Background and Significance
By helping parents quit smoking, pediatric clinicians protect children and families
from significant health harms. More than 40% of the U.S. pediatric population is regularly
exposed to secondhand smoke (SHS), most often by a parent.[1 ]
[2 ] SHS exposure among children increases the risk of sudden infant death syndrome,
chronic respiratory diseases, such as asthma, and lung cancer in adulthood.[3 ] Parents who quit smoking eliminate the majority of their children's SHS exposure,[1 ] decrease the risk of their children becoming smokers when they grow up,[4 ] and can increase parents' own life expectancy.[5 ] Pediatric clinicians are well-positioned to protect children from SHS exposure by
promoting tobacco use treatment for parents who smoke,[6 ]
[7 ] but appropriate treatments to parent smokers are not routinely delivered in pediatric
settings.[8 ] Major barriers to wider adoption include a lack of methods to support effective
pediatrician–parent communication regarding tobacco use treatment and mechanisms to
systematize, consistently deliver, and scale effective interventions.[8 ]
[9 ]
[10 ]
Communication regarding smoking cessation likely influences parental decisions to
engage in treatment. In pediatric settings, tobacco use treatment messages that emphasize
the impact of smoking to parents on their child may increase acceptance of cessation
treatment.[11 ]
[12 ] Effective messages and interventions for parental tobacco use treatment can be systematized
through electronic health records' (EHRs') clinical decision support (CDS) systems.
These systems are effective in improving health care process measures across a wide
variety of care settings.[13 ]
[14 ] CDS systems may also improve engagement with clinical interventions for tobacco
use.[15 ] In adult settings, CDS systems that connect patients to Quitlines (telephone-based
counseling programs effective in helping smokers quit[16 ]) led to a 13-fold increase in the proportion of smokers who enroll in treatment[17 ] and can be successfully implemented with minimal promotion.[18 ] In pediatric settings, CDS systems, including those developed by members of this
study team, have shown potential in helping pediatricians screen for SHS exposure,
provide treatment, and similarly increase Quitline enrollment.[19 ]
[20 ]
[21 ]
[22 ]
[23 ]
[24 ]
[25 ] Workflow limitations, however, have prevented their full incorporation into routine
clinical practice. For CDS systems to be effective, the interface needs to be designed
with the needs of parents and clinicians taken into consideration. In addition, it
is critical to consider how other aspects of the CDS system are integrated into clinical
workflows such that clinical staff can continue their work processes. These types
of workflows are often not considered leading to failures during CDS system implementation.
Developing CDS systems using human-centered design (HCD) approaches may help move
beyond simple prompts and achieve a more effective and integrated workflow for tobacco
use treatment in pediatric settings. HCD, guided by usability experts, involves an
analysis of the work environment, active user involvement in the development process,
iterative systems development, and testing systems in real-word settings.[26 ]
[27 ]
[28 ]
[29 ]
[30 ]
[31 ] HCD approaches have been shown to help ensure that technology is easy-to-use and,
thus, more consistently and effectively used.[27 ]
[28 ]
[30 ]
[32 ] While HCD approaches provide a methodological foundation for developing effective
CDS and have been used to develop smoking cessation CDS system in adult care settings,[33 ]
[34 ] they have not been rigorously applied to systems focused on tobacco use treatment
in pediatric settings. This critical gap in the application of HCD to smoking cessation
efforts in pediatric settings has limited the effectiveness of efforts to reduce parent
smoking.
Objective
This study aimed to develop a CDS system by using HCD processes that identifies parents
who smoke, provides motivational messages to quit smoking (informed by behavioral
science), and supports delivery of evidence-based tobacco treatment.
Methods
We applied a rigorous HCD process that engaged participants from three cohorts in
workflow analysis and a series of iterative formative scenario-based usability testing
sessions that captured a range of objective and subjective metrics on CDS system usability,
utility, and intent to use. This work was performed by a multidisciplinary team with
expertise in pediatric primary care, smoking cessation, CDS, HCD methods, and software
engineering.
System Design
The CDS system was designed to support three cohorts of system users as follows: (1)
parents/caregivers who smoke (hereafter referred to as parents), (2) primary care
pediatric clinicians, and (3) clinical staff. The initial design of the CDS system
was based on clinical expertise and research on tobacco use treatment in pediatric
settings,[8 ]
[23 ]
[35 ] prior work developing and evaluating CDS systems for parent tobacco use,[20 ]
[22 ] expertise on EHR programming and CDS, and the application of HCD methods such as
usability heuristics, scenario development, and cognitive walkthrough.[26 ]
[27 ]
[28 ]
[29 ]
[30 ]
[31 ] The final system design was developed through a more detailed HCD process,[29 ]
[30 ]
[31 ] as described below.
Study Setting, Participants, and Recruitment
This study took place with the Children's Hospital of Philadelphia (CHOP) Care Network,
a highly productive primary care practice-based research network.[36 ] The study was conducted in practices identified as having higher rates of parent
smoking. Inclusion criteria were as follows: parents were eligible if they were a
self-identified smoker, aged ≥18 years, present at their child's health care visit,
and able to communicate in English; pediatric clinicians were eligible if they were
an attending pediatrician or nurse practitioner; and clinical staff were eligible
if they were either the office manager or lead patient scheduler. Parent participants
were identified through EHR patient chart reviews, and study procedures were performed
immediately after the clinical encounter and audio recorded. Pediatric clinicians
were recruited via e-mail, and sessions were performed remotely using a video-sharing
platform and audio recorded as well. Pediatric clinicians were recruited via purposeful
sampling to ensure a range of confidence and perceived roles in providing tobacco
use treatment ([Table 1 ]).[37 ] Clinical staff participants were recruited via e-mail, and sessions were performed
remotely using a video-sharing platform. All sessions were audio recorded. The study
was approved by the CHOP Institutional Review Board.
Table 1
System subjective and objective evaluation metrics
Parent CDS system development
Qualitative
Quantitative
Outcome measure
Think-aloud protocol:
Parent review of prototype with interviewer
Parent thinks aloud, communicating understanding of interface, task, and choice selection;
Interviewer probes while not guiding
Questionnaire:
TAM: validated measure for usability and utility of CDS tools
Semi-structured interview:
Questions regarding system design
Outcome measure
Error rate: percentage of errors committed by participant, per scenario
Analysis
Think-aloud protocol:
Sessions recorded, reviewed, and debriefed by study team members
Most serious usability problems identified and discussed
Questionnaire:
Parent rates prototype's usability, functionality, and general satisfaction
Semi-structured interview:
Identify missing components or concerns
Content reviewed to inform the next iteration of prototype
Analysis
Error rate: measure where participant deviates from expected actions in performing
task, recorded as either critical error or noncritical error
Pediatric clinician CDS system development
Qualitative
Outcome measure
Think-aloud protocol:
Clinician review of prototype with interviewer; prototypes addressed multiple scenarios
of parent smoker status, including only one parent smokes, parent and other family
member smokes, only other family members smoke, and scenarios where parent may change
their mind about accepting treatment after a discussion with the clinician
Clinicians think aloud, communicating understanding of interface, task, and choice
selection; Interviewer probes while not guiding
Questionnaire:
TAM: validated measure for usability and utility of CDS tools
Semi-structured interview:
Questions regarding system design
Analysis
Think-aloud protocol:
Sessions recorded, reviewed, and debriefed by study team members
Most serious usability problems identified and discussed
Questionnaire:
Clinician rates prototype's usability, functionality, and general satisfaction
Semi-structured interview:
Identify missing components or concerns
Content reviewed to inform the next iteration of prototype
Clinical staff workflow analysis
Qualitative
Outcome measure
Semistructured interview:
Questions describe pros and cons of each workflow at their practice
Questionnaire:
Formal evaluated mock-ups of proposed workflow
Analysis
Semistructured interview:
Staff asked to describe pros and cons of each workflow at their practice.
Questionnaire:
Staff rate suitability for their clinical site and whether the workflow was efficient
and easy to manage
Abbreviations: CDS, clinical decision support; TAM, Technology acceptance model.
Usability Testing: Parent System Component
Functional interactive prototypes of the system's component intended for use by parents
were developed using REDCap electronic survey building and data capture tools.[38 ]
[39 ] This component was a screening questionnaire with questions about the parent's and/or
other family member's tobacco use, motivational smoking cessation messages, and options
to connect with three evidence-based tobacco use treatments: nicotine replacement
therapy (NRT) patch and/or gum, referral to the Quitline (counseling services via
phone[16 ]), and/or referral to SmokefreeTXT (counseling via text messaging[40 ]). Parents interacted directly with system prototypes on a laptop, allowing the recording
of objective measures including task completion and error rates. In these sessions,
participants were encouraged to talk aloud while performing tasks,[41 ] following a think-aloud protocol,[42 ] communicating their understanding of the user interface and their selections in
performing the tasks. The facilitator encouraged the parent to say whatever comes
into their mind as they complete the task. This approach (1) gathered responses to
open ended questions on the usability and functionality of the system and (2) encouraged
subjects to add opinions, ideas, and suggestions. After the usability testing, participants
completed a system usability, utility, and trust questionnaire based on the Technology
Acceptance Model (TAM),[43 ]
[44 ] followed by a brief semistructured interview. Interview questions focused on desired
features of the system, general process of screening for tobacco use, content of motivational
smoking cessation messages, and tobacco treatment.
Usability Testing: Pediatric Clinician System Component
High fidelity, semi-interactive, and scenario-based mockups of the system components
used by clinicians were created in a rapid prototyping application (Axure, San Diego,
California, United States). Usability testing was conducted remotely to accommodate
participants' availability. While the mockups were high fidelity, they did not fully
support direct interaction. As a result, we applied a pluralistic walkthrough format[45 ] where test facilitators presented each scenario to participants and prompted them
to describe the scenario-based action for each screen. If the participant selected
the intended action of the design, the facilitator would proceed to the next screen.
If a participant selected an unintended action, the facilitator would describe the
result of that action and allow the participant to select another action. If the participant
was unable to identify the correct action, the facilitator would explain the intended
action and proceed. After the prototypes' review, participants responded to a questionnaire
based on the TAM to assess perceived ease of use, perceived usefulness, and intent
to use.[43 ] The questionnaire was followed by a brief semistructured interview with questions
about most- and least-liked features, general satisfaction, suggested changes to the
prototypes, and overall fit of the proposed system within the current visit workflow.
Workflow Analysis: Clinic Staff System Component
While clinical staff are not direct end users of the CDS system, they have both an
important role in supporting parents using the system and valuable experience in how
the system could best fit within the clinic workflow. Participants reviewed graphic
representations of three distinct workflow options for delivering the parent system
including completing the screening questionnaire prior to the appointment (via the
EHR patient portal), in the office on a dedicated tablet, or in the office on the
parent's own smartphone device. After workflow reviews, participants completed a TAM
questionnaire rating the system on perceived ease of use, perceived usefulness, and
intent to use.[43 ] Participants then responded to semistructured questions on the positives and negatives
of each workflow and which one would fit best within their practice workflow.
Outcome Measures
Our outcome measures addressed the following two goals: (1) determine that the CDS
system meets the functional objective of identifying parents who smoke, providing
motivational messages, and connecting parents to treatments; and (2) ensure the system
is usable, useful, and likely to be adopted. For the parent and pediatric clinician
user testing, we captured a mix of objective and subjective data ([Table 1 ]). Objective data included critical errors (defined as tasks that participants were
unable to complete) and noncritical errors (defined as tasks where participants encountered
difficulty but were able to recover without assistance to complete the task). Subjective
data included participant response to a pretest demographic questionnaire, postscenario
Likert's scale questions based on the TAM, comments from the think-aloud protocol,
and semistructured interview questions administered at the end of the usability test
sessions. For clinical staff, outcome measures focused on ensuring the system was
usable, efficient, and fit within the workflow of the clinical staff. The evaluation
approach leveraged the Hierarchical Task Analysis methods[46 ]
[47 ] in which staff reviewed, commented on, and evaluated mockups of proposed workflows.
Analysis
Demographic survey responses for each participant cohort were characterized using
descriptive statistics. For parent and pediatric clinician design review sessions,
subjective analysis involved assessment of think-aloud comments, questionnaires assessing
usability, functionality, general satisfaction, and responses to open-ended questions
that were coded to identify key themes ([Table 1 ]). Recordings were analyzed using NVivo 11 (QSR International) analysis software
to code and identify themes. Objective analysis assessed participant error rate while
completing specific CDS prototype activities. The research team responded to the occurrence
of errors and/or negative subjective results by modifying the system and continuing
to test with new participants. For clinical staff sessions, analysis involved assessment
of responses to open-ended questions and questionnaires assessing usability, functionality,
and general satisfaction. In all, the optimization process ensured that the system
design met users' needs ,a level of usability, functionality, and usefulness that
supports screening parents for smoking, providing motivational messages, and connecting
to tobacco use treatment.
Sample Size
For the parents and pediatric clinicians, we anticipated iteratively testing and refining
the CDS system prototype with a minimum of 30 participants. At least 97% of all problems
with software systems can be identified with 30 participants.[48 ]
[49 ] For the clinical staff, we performed workflow review with a sample of staff until
we reached thematic saturation.
Results
Participants and Recruitment
The HCD process engaged 70 representative participants including 30 parents, 30 pediatric
clinicians, and 10 clinic staff. Parents were recruited from 2 different practices,
clinicians from 10 practices, and staff from 5 practices ([Table 2 ]). The clinicians expressed a range of confidence in providing tobacco use treatment,
including prescribing NRT.
Table 2
Participant demographics
Characteristic
Parent caregivers (n = 30)
% (n )
Pediatric clinicians (n = 30)
% (n )
Clinical staff (n = 10)
% (n )
Age (y)
≤ 24
7 (2)
–
–
25–34
43 (13)
17 (5)
20 (2)
35–44
43 (13)
43 (13)
40 (4)
45–54
7 (2)
17 (5)
20 (2)
55–64
–
20 (6)
20 (2)
65 or older
–
3 (1)
–
Sex
Female
77 (23)
80 (24)
100 (10)
Male
23 (7)
20 (6)
–
Race
White
13 (3)
83 (25)
–
Black/African American
77 (23)
7 (2)
–
Asian
3 (1)
10 (3)
–
American Indian or Alaska Native
3 (1)
–
–
Other
10 (3)
–
–
Multiracial
7 (2)
–
–
Ethnicity
Hispanic
17 (5)
–
–
Non-Hispanic
83 (25)
100 (30)
–
Relationship to patient
Parent
93 (28)
–
–
Grandparent
3 (1)
–
–
Other guardian
3 (1)
–
–
Clinical title/role
Attending physician
–
87 (26)
–
Nurse practitioner
–
13 (4)
–
Office manager/director
–
–
50 (5)
Patient services representative
–
–
50 (5)
Experience in clinical practice (y)
0–10
–
53 (16)
30 (3)
11–20
–
17 (5)
40 (4)
21–30
–
17 (5)
20 (2)
> 30
–
13 (4)
10 (1)
How comfortable do you feel providing smoking cessation counseling to parents or other
caregivers who smoke?
Very uncomfortable
–
0 (0)
–
Somewhat uncomfortable
–
10 (3)
–
Neutral
–
20 (6)
–
Somewhat comfortable
–
50 (15)
–
Very comfortable
–
20 (6)
–
How comfortable do you feel treating parents or other caregivers who smoke with nicotine
replacement therapy?
Very uncomfortable
–
37 (11)
–
Somewhat uncomfortable
–
17 (5)
–
Neutral
–
3 (1)
–
Somewhat comfortable
–
20 (6)
–
Very comfortable
–
23 (7)
–
Parent System Component
Formative testing of the parent system (the screening questionnaire) was performed
over four iterative versions of system prototypes. Modifications were primarily in
the editing of system language but also included adding functionality. The initial
version was tested with five participants and had a critical error rate of 20% and
a noncritical error rate of 60%. All participants reviewing the initial version rated
the system as easy to understand and easy to use. Usability testing continued for
three additional versions. The final version was tested with 10 participants and had
no critical or noncritical errors. Ninety percent of participants rated the questionnaire
as easy to understand and 80% as easy to use ([Table 3 ]).
Table 3
Parent system summary of prototype iterations
Version Information
Statistics
Design changes
Prototype version
Participants (n )
% Critical error
% Noncritical error
Content clear and easy to understand (% strongly agree/agree)
Content easy to use (% strongly agree/agree)
Intro screening questions
Intro summary
Three treatment options
End info screen
1
5
20
60
100
100
NA
NA
NA
NA
2
8
0
38
100
100
None
None
Major edits
None
3
7
0
29
86
100
None
None
Minor wording edits
Minor wording edits
4
10
0
0
90
80
Minor edits
Minor edits
Minor grammar edits
Minor wording and display edits
Abbreviation: NA, not available.
Several important design requirements were identified, including minimizing the likelihood
that the questionnaire would elicit feelings from parents of being judged for their
smoking, keeping the time spent completing the questionnaire as short as possible,
and presenting information relevant to their child's health as a motivator for behavior
change. Most parents did not feel “judged” regarding smoking behaviors by the content
of the questionnaire (93%), and most parents agreed they would be comfortable filling
out the questionnaire (90%) and having their child's pediatrician know about smoking
in their family (77%; [Fig. 1 ]). The nonjudgmental aspect of the language and the screener was also reflected during
the design screener review, “You are not judgmental, so yeah. I appreciate it.”
Fig. 1 Parent technology acceptance model (TAM) questionnaire (n = 30).
Parent participants shared that the parent CDS system was easy to complete, and that
the information presented was helpful and informative. One illustrative comment included:
“It was short, sweet and simple because it was getting straight to the point ….” Many
mentioned their child's health as their motivation to quit. We received overwhelmingly
positive feedback on the motivational messages that lists the benefits of quitting
smoking. Despite its length and level of detail, respondents found it self-explanatory,
helpful and the appropriate length: “I think that it's all good components to helping
to quit smoking. Making it brighter in your brain to think about your kids ….” Participants
found the variety of treatment options helpful, and none of the subjects expressed
concerns about the default opt-in to the treatments.
Clinician System Component
Formative testing of the clinician system was performed over six iterative versions
of system prototypes. Modifications included redesign of interaction components, workflow,
content, and adding functionality. The initial version was tested with only two participants
but identified severe issues requiring a reorganization of user interface elements
and workflow. The subsequent five iterations identified more minor issues that required
modest changes but also resulted in discovering requirements for added functionality
and information. During most sessions, questions would arise from clinician participants
about features, functionality, and workflows. We used data from the usability sessions
to add key features, reorganize and change layouts, optimize workflows, and provide
information about treatment resources ([Fig. 2 ]).
Fig. 2 Pediatric clinician system summary of prototype iterations. Version, participants,
design changes; version: 1; participants: 2; scenarios in testing: caregiver smokes
and accepts treatment; caregiver smokes and declines treatment; design changes: changed
notification about parent options and treatment option presentation. Version: 2; participants:
3; scenarios in testing: same as version 1; design changes: reorganized main decision
support screen and moderate edits to documentation feature. Version: 3; participants:
3; scenarios in testing: same as version 1 plus other family member smokes, no smokers
in family; design changes: added main discussion prompts to main decision support
screen and information to submission feature, minor edits to documentation. Version:
4; participants: 11; scenarios in testing: same as above; design changes: minor edits
to main decision support screen layout and tips for talking with caregiver screen,
simplified documentation. Version: 5; participants: 7; scenarios in testing: same
as above; design changes: minor edits to main decision support screen, treatment options
screen, and submission feature. Version: 6; participants: 4; scenarios in testing:
same as above; design changes: minor edits to main decision support screen to clarify
parent/caregiver respondent information, tips for talking with parents, treatment
options, and submission feature; added billing support, hover tips for quick information,
and contact information for treatment options.
The responses across all iterations to the pediatric clinician TAM questionnaire were
positive. Overall, across all system versions, more than 95% of participants strongly
agreed/agreed that the system would help them provide treatment to parent smokers
and would improve outcomes in parent smoking cessation ([Fig. 3 ]). Further, more than 95% of participants strongly agreed/agreed that they would
use the system and that they were satisfied with the system.
Fig. 3 Pediatric clinician technology acceptance model (TAM) questionnaire (n = 30).
Pediatric clinician participants were supportive of the system and its role in helping
parents quit smoking. One illustrative comment included, “I like that it identifies
and uses what I knew was evidence-based tactics for smoking cessation and it is aligned
with my philosophy as far as what I'm offering.” Some clinician participants expressed
the need for some time to get used to the system before feeling completely confident
but still confirmed that the system was easy to use and well presented. One participant
explained: “It's easy to use. It provides the information needed. It sort of walks
you through it step by step. Resources are right there at your fingertips … I would
think for many like me we're not so aware of the options for nicotine replacement
and, again, just for smoking cessation programs.” Clinicians emphasized the value
of automating tobacco treatments and documentation as much as possible to reduce the
burden on them during visits with competing priorities. They were comfortable with
treatment being provided, regardless of a discussion with the parent.
Workflow Analysis: Clinic Staff System Component
Review and qualitative assessment of three different workflows revealed that staff
perceived the EHR patient portal reminder workflow as the most preferred for staff,
parents, and clinicians at their practice (90% agreed/strongly agreed). They also
rated it as the workflow that would be the most efficient and easy to manage (100%
agreed/strongly agreed). However, during the postreview, semistructured interview
participants unanimously preferred having all three workflows available as part of
the questionnaire completion, as opposed to relying solely on EHR patient portal workflow.
Participants reported that their parent population prefers to complete questionnaires
through a range of modalities. A multifaceted approach with a combination of questionnaire
completion at home via the EHR patient portal, in the office via tablet, or on a parent's
own smartphone device is widely utilized for other questionnaires in these practices,
and therefore would be a preferred strategy for this system.
Final System Development
An overarching theme of the design review sessions across all cohorts was automating
key functions as much as possible. We iteratively developed the final CDS system based
on the subjective and objective data from the design review sessions and workflow
analysis. This system is now live at five primary care practices in our electronic
health record (EHR). The system supports screening for parent smoking via several
modalities (personal computer or smartphone device) prior to the patient's visit to
decrease the burden on in-office completion or in the office on a tablet device. If
the parent indicates that they are a smoker, the parent portion of the system displays
information designed to motivate treatment engagement, followed by presentation of
three treatment options for them to accept (1) NRT, (2) Quitline, and/or (3) SmokefreeTXT.
The system automatically connects the parent with the treatment options they selected,
and the NRT is directly delivered to the parent, if interested. The pediatric clinician
is notified of the parent's selections within several key sections in the EHR, including
the best practice alert section, questionnaire review section, and main note, all
to account for different clinician workflows. The system includes information to help
the clinician in what is often a difficult conversation about smoking, with simple
guidance on key conversation prompts that motivate treatment. The system supports
automated documentation content including the office visit progress note, after visit
summary, and billing. Finally, the system will periodically remind the clinician to
follow-up on the parent's effort to quit. See [Fig. 4 ] for system workflow.
Fig. 4 Automated process for family tobacco control screening and treatment delivery. EHR,
electronic health record.
Discussion
We applied HCD processes to develop a CDS system that supports parent tobacco use
treatment through pediatric settings. We identified key system functions and features
through initial prototyping. These features were prioritized and refined though iterative
end-user feedback. Previous work had shown that a CDS system targeted at clinicians
could improve process measures related to parent tobacco use,[23 ]
[24 ] but further work was needed to better incorporate these systems into office workflows.[20 ]
[22 ] In this study, parents, pediatric clinicians, and clinical staff all identified
directly screening parents for tobacco use and then directly offering treatment through
an electronic questionnaire as critical system features. Parents valued this approach,
as it allowed them to disclose tobacco use in a nonjudgmental way. Pediatric clinicians
and staff felt this approach could most efficiently identify parents and then systematize
treatment. Participant interactions with iterative system prototypes helped identify
how to make these key functions acceptable and usable. Parents valued motivational
messages, informed by behavioral science, that focused on the benefits of smoking
cessation to their child's health[11 ]
[12 ] and simple, easy-to-understand explanations of treatment options. Pediatric clinicians
valued automation of parent screening, treatment, and documentation with simple guidance
on key conversation prompts that support treatment. Similarly, clinical staff valued
methods for parents to complete key system features ahead of the visit to decrease
in-office workflow burdens.
Developing CDS systems using HCD approaches is a growing area of research interest,
as it helps clinical application developers move beyond simple prompts and alerts
to systems that more effectively support clinical workflows.[50 ]
[51 ]
[52 ] Clinicians spend a significant portion of their professional time using the EHR,
with a large percentage of clinicians spending more time interacting with the EHR
than interacting with patients.[53 ]
[54 ] EHR-related stress is prevalent, and there may be a particularly strong relationship
between poor EHR usability and physician burnout.[55 ] Developing CDS systems with the end user of the system at the center of the processes
is critical, considering the well-known time burdens and disproportionate stress on
clinicians through the current processes.[53 ]
[55 ]
[56 ] Further, our HCD approach provides a more holistic approach to system development,
incorporating usability assessment of the CDS from both the interface level (parents
and pediatric clinicians) and the office workflow level (clinical staff).
Despite our success in achieving our objectives, the EHR presented many barriers to
usability and functionality, primarily due to the inability to account for family
relationships. EHR vendors have focused on communication and data privacy for adult
health care settings, with technological services developed for the physician–adult
patient dyad. These systems fail to account for the family aspect of child health
care, including the inability to readily link family members and address health at
the family or household-level.[57 ]
[58 ]
Limitations
Our study has several limitations. First, we do not have data on the CDS system's
impact on parent smoking cessation rates. Nonetheless, by leveraging the team's previous
experience developing effective CDS systems for tobacco use treatment[20 ]
[22 ] and grounding this work in rigorous HCD methods, our design process maximizes the
likelihood of CDS effectiveness. Second, pediatric clinician usability testing was
conducted remotely, using web conferencing software, which did not fully support direct
interaction with the prototypes. Thus, we did not have recorded error metrics for
participant interactions with the system. We felt the improved accessibility to pediatric
clinicians using this approach outweighed the effort of in-person testing during the
pandemic. Third, we did not perform a formal power analysis. We performed formative
iterative testing where the objective was to apply an HCD approach to system design
using high-fidelity mockups prior to development. Literature on number of participants
for summative testing lacks consensus and is less clear on formative testing.[59 ]
[60 ] Fourth, the facilitator-led usability testing and workflow analysis sessions may
be prone social to desirability bias. Fifth, while we sought a representative sample
for usability testing and workflow analysis, participant feedback may not be representative
of all parent, pediatric clinician, or staff perspectives. Nonetheless, for pediatric
clinicians, we did not identify meaningful differences in TAM questionnaire responses
by demographic or professional characteristics, including, for example, experience
in clinical practice. Finally, the CDS system to support parent tobacco use treatment
was limited to a single institution as part of a research effort.
Conclusion
A CDS system to support parental tobacco cessation in pediatric primary care, developed
through a HCD process, proved easy to use and acceptable to parents, clinicians, and
office staff, justifying future efforts to test its impact on parental smoking cessation
outcomes in the clinical setting.
Clinical Relevance Statement
Clinical Relevance Statement
In this study, parents, pediatric clinicians, and clinical staff, all identified directly
screening parents for tobacco use and then directly offering treatment through an
electronic questionnaire as critical system features. Parents valued this approach,
as it allowed them to disclose tobacco use in a nonjudgmental way. Pediatric clinicians
and staff felt this approach could identify parents most efficiently and then systematize
treatment.
Multiple Choice Questions
Multiple Choice Questions
One of the key features of human-centered design (HCD) approaches is:
A stochastic analysis describing a sequence of possible events.
Active use involvement in the development process.
Information present at the right time in the workflow.
A formatting system for displaying material within the electronic health record.
Correct Answer: The correct answer is option b. HCD approaches, guided by usability experts, involve
an analysis of the work environment, active user involvement in the development process,
iterative systems development, and testing systems in real-word settings.
A think-aloud protocol is a data-gathering method in which:
Individuals are observed performing their typical clinical workflows.
Electronic health record data are queried to answer a clinical question.
Participants talk aloud while performing key prototype tasks.
Participants give feedback as a group, describing key desired features for a new system.
Correct Answer: The correct answer is option c. The think-aloud protocol involves participants talking
aloud while performing tasks, communicating their understanding of the user interface,
and their selections in performing the tasks.
