Keywords software design - therapeutic drug monitoring - usability testing - factor VII - factor
VIII - factor IX
Background and Significance
Background and Significance
The large degree variability in dose-exposure relationships between patients contributes
to uncertainty in our ability to predict whether a given patient at a given dose of
a given drug will respond favorably, develop unwanted side effects, or perceive the
requirements for administration cumbersome enough to interfere with their daily quality
of life (QOL). Antihemophilic factors represent one-such class; dose-exposure profiles
vary between and within patients (as they grow and develop) and the consequences of
inadequate dosing are marked. Too little factor increases the risk of serious bleeds
while too much factor results in excessive drug costs and, in some cases, undesirable
administration regimens. For these drugs, clinicians can apply specialized knowledge
of pharmacokinetics (PK) to account for this variability and individualize dosing
recommendations. PK-guided dose individualization affords the opportunity to achieve
clinician-driven therapeutic targets with no more factor than is necessary to prevent
hemorrhagic episodes.[1 ] In fact, PK strategies have been used to guide the replacement of various clotting
factors for more than two decades.[2 ]
[3 ]
[4 ]
[5 ]
Successful application of PK-guided dose individualization to routine patient care
requires (1) comprehensive knowledge of the patient, (2) a thorough understanding
of the pharmacologic principles that drive the relationship between dose-exposure,
and (3) expertise in mathematical and PK modeling and simulation.[6 ] Providers with expert knowledge of the latter often have limited involvement with
the patient or their primary medical team and engaging them outside the normal flow
of care can introduce delays in arriving at the best dosing solution for the patient.[7 ] For decades, countless computer-based tools have been developed in an attempt to
integrate PK-individualization into the clinical care workflow.[8 ]
[9 ]
[10 ]
[11 ] Most exist as standalone applications and relatively few were designed to interface
with the electronic health record (EHR).[12 ]
[13 ] Consequently, very few of these tools experience utilization that extends beyond
the academic center at which they are developed. Moreover, examinations of usability
(when conducted) often engage individuals who already possess expertise in PK rather
than clinician end users for whom they are intended.[12 ]
[13 ] As a result, valid estimates of usefulness remain undefined.
Objective
At our institution, we employ PK individualization for difficult to manage hemophilia
patients who are unresponsive to standard dosing regimens. This is routinely accomplished
using a classical pharmacokinetic approach where multiple (often five or more), patient-specific,
exposure levels are examined to describe a mathematical model with parameter estimates
unique to the individual patient that can be used for dose simulation. As described
above, this typically occurs outside the flow of care via consultation with clinical
pharmacology experts. Our primary goal was to create a simple, tailored, clinical
decision support (CDS) solution that addresses our providers needs and integrates
into the flow of care. In recent years, there has also been increasing dialogue related
to the value of population PK (popPK) for dose optimization where a limited number
of exposure levels (typically 1–2) are used to modify parameter estimates that derive
from a broader population of individuals. Our secondary goal was to provide a solution
that allows clinicians to gain additional experience with the predictive performance
of these models. The objective of this paper is to describe the development and usability
of this CDS tool for antihemophilic factor dose individualization.
Methods
Design
Our approach followed a typical user-centered design process to ensure the successful
development of a product that targets the needs of its users.[14 ]
[15 ] This included the identification of users and an early assessment of workflows and
clinical challenges. User stories were constructed to ensure that the project team
developed a solution which addressed the needs of the providers impacted by the application.
Construction of the wireframe, or high-fidelity mockup, was followed by iterative
prototyping and empirical measurement throughout design and development. The development
strategy was driven by the features associated with successful CDS namely: (1) involving
local users, (2) integrating with the charting/order entry system, (3) avoiding additional
data entry, (4) availability at the time/location of decision making, (5) providing
a recommendation (vs. an assessment), and (6) justifying the decision with evidence.[16 ]
Requirements Analysis
The project began by engaging physicians, nurses, pharmacists, and pharmacologists
at a standalone pediatric hospital that serves pediatric patients and provides consultative
services for adult patients at a neighboring institution. The existing processes associated
with antihemophilic factor PK individualization were documented including the sequence
of tasks and information flows. High-level tasks were deconstructed into subtasks
and operations, and the information sources required in each subtask were recorded.
The output included process charts, task flow diagrams, task decomposition tables,
and use case scenarios.[17 ] Additional discussions centered on functionality currently not embedded in the workflow
but desirable for optimal CDS utility. The major features identified by the users
included: (1) transparent integration with the EHR, (2) application to factors VII,
VIII, and IX, (3) application to standard and extended half-life products, (4) simulation
for both prophylactic and presurgical/acute bleed regimens, and (5) integration of
both classical and popPK models.
User Stories and Iteration Methodology
Features of the application were designed and implemented in an iterative fashion
following agile development principles.[18 ] Four clinical scenarios were used to frame the user stories:
AV had a factor 9 pharmacokinetic profile evaluated at 7 years and 7 months of age.
You would like to identify a prophylactic dose that results in a trough activity level
of 5%.
FL had a factor 9 pharmacokinetic profile evaluated at 1 year and 4 months of age.
He will be coming in for surgery and you would like to identify a loading and maintenance
dose for acute bleeds that maintain peak levels around 80% and trough levels around
30%.
PP had a factor 7 pharmacokinetic profile evaluated at 42 years of age. You would
like to explore whether a more abbreviated sampling strategy could have still reliably
predicted PP's pharmacokinetic profile.
LF had a full factor 8 pharmacokinetic profile evaluated at 3 years of age and had
repeated random sampling 5 months later. You would like to identify how effectively
existing population models predict these levels.
These user stories were integral to the development and quality assurance (QA) process
wherein stakeholders worked with a business and QA analyst to document, justify, and
prioritize feature development and bug fixes. A standardized format for user stories
was implemented, significantly reducing confusion and development/debug cycles when
compared against prior software development.[19 ] User stories helped prioritize work and prompted in-depth discussions with stakeholders
that aided the development team's understanding of the business cases, justifications
behind features, and bug fix requests. DevOps process, the practice of development
and operations participating together in the entire service lifecycle from design
through the development process to production support, was followed as a part of the
product service lifecycle.
Front-End Design
Information gathered from the requirements analysis informed the design of the CDS
prototype which was developed in Adobe XD. Highcharts, a javascript library, was used
to implement visualizations presented by the user interface (UI). The design was guided
by feedback gathered from the development of an earlier CDS tool developed at our
institution and a 296-item heuristic checklist.[19 ]
[20 ] The major domains under consideration were: visibility of system status, match between
system and the real world, user control and freedom, consistency and standards, error
prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic
and minimalist design, and help and documentation.
Responsive web design best practices were used to implement the look and feel of the
web application and achieve a consistent experience across different web browsers.[21 ] Angular bootstrap was used for the various UI widgets which were further customized
based on the required functionality.[22 ]
[23 ] The UI interface was created with a strong focus on reusability. As a result, the
web application was implemented as a single page application using Angular 6. Visual
Studio was used to integrate back-end code with the front-end UI.
Back-End Development
Owing to the historical reliance on, and comfort of our users with, classical PK approaches,
dose individualization in this application occurs with the use of patient-specific
PK parameter estimates. PopPK models were also integrated into the application to
facilitate practice improvement considerations. Though not used to support dose individualization
with the initial release of the application, providers can explore whether publicly
accessible popPK models effectively describe the disposition of antihemophilic factor
in their patient population. When our providers have accumulated sufficient evidence
to support the use of specific popPK models, the provider team can request that one
or more of these models be pulled forward to permit dose optimization using a minimal
sampling approach coupled with popPK.
The software flow was based on an algorithm developed and modified by the clinical
pharmacologist during the course of practice. For application of classical PK, we
formulated our compartmental modeling approach as a nonlinear least square problem,
with parameters estimated by curve-stripping and optimized using the Levenberg–Marquardt's
algorithm.[24 ] A series of nested logic functions were used to determine whether the data should
be fit to a one-, two-, or three-compartment model. For datasets where a prestudy
dose contributed to measurable factor activity levels, the baseline contribution of
the prior dose was removed from the profile using the superposition principle.[25 ] For sparsely sampled datasets, a decision tree aided in model selection or defaulted
to the application of popPK models. Twelve peer-reviewed popPK models, for which all
parameters were clearly specified in the publication or provided to the authors via
personal communication, were coded into the application ([Supplementary Material ], available in the online version). PopPK models were accessible to the user only
if all of the relevant covariates were populated in the EHR.
The simulation engine was coded to individualize dosing for prophylaxis or acute bleeds
using algorithms that require the user to specify two inputs (i.e., dose, dosing interval,
desired peak, and desired trough) with the engine returning the remaining two as outputs.
Additional features were added for simulating nonuniform dosing intervals (e.g., Monday,
Wednesday, and Friday dosing). Quantitative goodness-of-fit (GOF) criteria, traditionally
used to determine the appropriateness of a model (e.g., objective function, weighted
sum of squares, and coefficients of variation), were log-adjusted, weighted, and combined
into a 10-point scale to inform a visual indicator that alerts the user to the strength
of the model.
Unit testing for each executable path was performed with 23,000 simulated patient
profiles. These profiles were created from the range of patient-derived pharmacokinetic
parameter estimates observed in children and adults with the commercially available
factor formulations. Iterative refinement of the algorithm was performed until no
fewer than 98% of results from the simulated profiles were in agreement with the expected
results generated by applying the same calculations in Excel. Recognizing that we
are unable to fully eliminate edge cases, a penalty structure was nested into the
GOF algorithm to inform the user when the reliability of the model may be compromised.
The algorithm was implemented using C# programming language. Jupyter Notebook was
used for visual evaluation.
Integration with the Electronic Health Record
To enable interoperability with any EHR system, we created an Application Programming
Interface (API) for EHR integration. We designed an industry standard REST API, which
delivers EHR data that our tool can consume provided that the output complies with
a predefined signature.[26 ] For example, patient demographics are currently being provided by a REST API which
requires a medical record number (MRN) and in return it sends patient name and DOB
among other demographics. With this architecture, the application can consume demographics
from any REST API that complies with the predefined API signature for demographics
features. If the application owner decides to switch EHR vendors, they can simply
create a REST API which complies with defined signature and point the application
to that API. These REST APIs are generally not provided by EHR vendors. Application
owners need to create REST APIs, or alternatively FHIR APIs, which can communicate
with the EHR and supply the requisite data for the application. Our decision to rely
on a REST API was influenced by the limited availability of FHIR services at the time
of development.
For our system (Cerner Millenium, Cerner Corporation, Kansas City, Missouri, United
States), we also created a PowerForm (i.e., a template for clinical data entry) to
facilitate data requests and eliminate duplicative data capture. Completing the form
indicates that the event is ready to be pulled into our application. To access EHR-based
data, users must be registered in a prespecified authentication group controlled by
our security team. As an added QA measure, the application records user information
and actions when EHR data are requested. The software was also built with a manual
entry feature to accommodate the analysis of data from patients who are not contained
within the EHR.
Usability Testing
Structured cognitive walkthroughs (CW) were conducted with content experts (CE) and
end-users (EU) representing medicine, pharmacy, pharmacology, and nursing to assess
efficiency, ease-of-use, and user satisfaction among a representative user populations.
CE reflected individuals having completed formal postdoctoral training in clinical
pharmacology. EU reflected clinicians that could reasonably be expected to interact
with the software (i.e., physicians, nurses, and pharmacists). Using the application,
participants were asked to complete a series of tasks centered on the four clinical
scenarios described above. All four scenarios were of similar complexity and completed
in one sitting lasting approximately 1 hour. We employed a think-aloud protocol in
which participants were encouraged to verbalize their thoughts while audio and video
of their interaction with the application were captured on logging software (Techsmith
Morae, Okemos, Michigan, United States). We also recorded time to complete tasks,
number of clicks required to complete tasks, task success, and user perceptions of
task ease/difficulty. The Post-Study System Usability Questionnaire (PSSUQ v3) was
used to capture overall user satisfaction and perceived usability.[27 ] Recommendations arising from the CW were incorporated into the application's final
design.
Data Analysis
Standard descriptive statistics were applied to describe the participant populations
and the usability outcomes. To identify whether task times improved with repeated
use of the application, differences in time to completion between comparable, successive
tasks were analyzed using a paired t -test or analysis of variance. Statistical differences in task performance between
user subpopulations were examined by application of a two-tailed, unpaired Student's
t -test. The significance limit accepted for all statistical analyses was α = 0.05. All analyses were performed in SPSS v24 (IBM Corp, Armonk, New York, United
States).
Results
Form and Function
The CDS tool developed systematically walks the EU through a series of five to seven
screens:
Patient search ([Fig. 1 ]): This screen allows users to enter an MRN or select manual entry if the patient
is not in our EHR system (e.g., patients receiving care at the neighboring adult hospital).
Users that are not on the hospital network will encounter an initial authentication
screen prior to this patient search page.
Fig. 1 Patient search screen where users can enter a medical record number (upper) and identify
the pharmacokinetics study of interest (lower) (fictional patient data). Manual data
can also be selected subsequently bypassing the search results screen.
Search results ([Fig. 1 ]): Entering an MRN will pull up all instances of antihemophilic factor PK data available
for that child on the search results page. From this page users can select the PK
study date of interest or return to patient search. Selecting “manual entry” on the
previous page will bypass this screen.
Laboratory results ([Fig. 2 ]): This screen displays the relevant patient information, dosing history, and laboratory
information that have been pulled from the EHR for confirmation and visual inspection.
Users can deselect data points that appear errant using the exclude laboratory results
checkboxes or “switch to manual entry” mode if information is incorrect or missing
so that the data can be entered or corrected. Alternatively, users can proceed directly
to curve fitting via the “view model fit” button. Users with sparse data may encounter
one of two warnings alerting them to issues of reliability or advising them of analysis
types to which they are restricted.
Fig. 2 Laboratory results page where users can confirm the accuracy of the data being imported
(upper) and transition to the model fit page where they can examine modeling results
and explore relevant popPK models (lower) (fictional patient data).
Model fit ([Fig. 2 ]): After executing the modeling algorithm, users observe a graphic of the curve fit
detailing the model-type. They can assess the GOF by looking at the relationship between
the observed data and the curve on the graph, by examining the observed and predicted
concentrations in the table to the left of the graph or via the tachometer which consolidates
GOF criteria into a single indicator (green alerts the user that the model is strong,
yellow signals caution, and red indicates that there may be a problem with the fitting).
If users are dissatisfied with the model, they can go back to laboratory results and
reinspect the raw data to determine whether selected samples need to be excluded from
the fit. Users that are satisfied with the model can proceed to “simulation” or stay
on this page to examine how well-published popPK models fit their patient's data.
Only popPK models for which all of the necessary patient specific information is available
are presented for the user. “Switch to manual entry” mode on the preceding screen
allows the user to update any missing information and access the omitted popPK models.
Simulation ([Fig. 3 ]): Users can explore dosing recommendations with user-defined target exposures or
examine exposures that result from user-defined dosing regimens. Users are not restricted
to regularly scheduled administration and can simulate irregular dosing intervals
that may be more convenient for their patients. The simulations can be repeated ad infinitum until the clinician settles on a dosing regimen that achieves the goals of therapy.
Fig. 3 Simulation page where users can conduct simulations for target dose, interval, peak,
or trough for prophylaxis (upper) and acute bleeds (lower) (fictional patient data).
View report: When users have decided on a final dosing recommendation, they can click
“create report,” review their recommendations, add comments, hide or expose popPK
curve fits, and download the report.
Usability
In total, 12 CE and 12 EU completed the usability testing. Their baseline characteristics
are detailed in ([Table 1 ]). The majority of these individuals were under the age of 40 and had been in practice
less than 10 years. There was a slight preponderance of females over males, which
is consistent with the demographic breakdown of the workforce at our pediatric institution.
With one exception, all of these participants spent more than 15 hours on a computer
each week, most commonly a Windows based-operating system, and accessed the EHR daily.
Table 1
Characteristics of participants in the usability testing
Characteristic
Group
Content experts (n = 12)
End users (n = 12)
Gender
Female
6
9
Male
6
3
Age group (y)
26–39
8
9
40–59
3
2
60–74
1
1
Current role
Physician
8
6
Pharmacist
3
3
Nurse
1
2
Other
0
1
Years in current role
<5
3
5
5–10
6
3
10–15
0
2
>15
3
2
Activities performed on computer
Median (range)
5 (3–8)
5.5 (2–8)
Hours/week on computer
5–15
1
0
15–25
4
4
26±
7
8
Computer platform most often used
Mac
2
3
Windows
10
9
Browsers used when computing
Median (range)
1 (1–3)
2 (2–4)
Frequency with which EHR is accessed
Daily
8
11
Weekly
1
0
Never
3
1
Frequency with which computerized CDS tools are used
Daily
1
0
Once or twice a week
2
1
About once a month
1
3
A couple of times
4
5
Never
4
3
Frequency with which TDM is used in clinical decision making
Daily
0
1
Once or twice a week
3
1
About once a month
2
6
A couple of times
1
1
Never
6
1
Frequency with which PK calculations are applied to patient care
Once or twice a week
1
1
About once a month
2
1
A couple of times
2
4
Never
7
6
Proficiency with PK calculations
Strong
3
0
Moderate
8
3
Weak
1
6
Not proficient at all
0
3
Abbreviations: CDS, clinical decision support; EHR, electronic health record; PK,
pharmacokinetics; TDM, therapeutic drug monitoring.
Usability testing revealed that our CDS tool could be efficiently navigated by our
testers. Median task time across all tasks was 13.2 seconds with a median 1.0 mouse
clicks per task. Greater than 95% of tasks were completed with no difficulty. For
comparable tasks that were encountered more than once during testing, task times dropped
significantly for all but one that asked the user to identify the newly simulated
dose ([Table 2 ]). After experience was gained with the application, users spent most of their time
simulating new dosing regimens and finalizing the report for download. By contrast,
observational or one-click task such as performing curve fitting and identifying the
mathematical model required the least amount of time.
Table 2
Time to complete selected tasks. Data are represented as mean ± standard deviation
Task
Event 1 (min)
Event 2 (min)
Event 3 (min)
Event 4 (min)
p -Value
Import data[a ]
0.72 ± 0.40
0.27 ± 0.18
0.20 ± 0.09
0.31 ± 0.20
<0.01
Inspect the data[a ]
0.66 ± 0.41
0.32 ± 0.21
0.24 ± 0.15
0.19 ± 0.10
<0.01
Perform curve fitting[a ]
0.34 ± 0.28
0.06 ± 0.04
0.05 ± 0.02
0.07 ± 0.09
<0.01
Evaluate the goodness of fit[b ]
0.50 ± 0.33
0.17 ± 0.10
0.18 ± 0.16
<0.01
Identify the model that was fit[b ]
0.43 ± 0.53
0.09 ± 0.22
0.03 ± 0.01
<0.01
Simulate a new dose for specified therapeutic targets[b ]
0.86 ± 0.26
0.45 ± 0.20
0.34 ± 0.12
<0.01
Identify the new dose
0.17 ± 0.34
0.22 ± 0.17
0.20 ± 0.11
0.694
Modify the simulation to adjust the dose
0.85 ± 0.51
0.33 ± 0.14
<0.01
Examine the therapeutic targets with the modified dose
0.36 ± 0.23
0.25 ± 0.13
0.025
Finalize the report[c ]
1.22 ± 0.56
0.98 ± 0.46
0.30 ± 0.18
<0.01
a Events 2, 3, and 4 significantly faster than event 1.
b Event 2 and 3 significantly faster than event 1.
c Event 3 significantly faster than events 1 and 2.
No differences in mean or total task times were observed between EU and CE. Similarly,
performance metrics were largely uniform across our population irrespective of their
background characteristics. No differences were observed by gender, age group, general
computer use patterns, training, or tenure. The number of clicks required to navigate
through the cases did decrease systematically with increased frequency of therapeutic
drug monitoring experience, but this did not attain statistical significance (p = 0.054).
When observation and feedback obtained from usability testing was collated, issues
surrounding five common themes emerged: (1) design: esthetic preferences related to
color use, font size, and positioning of various elements within the application represented
the most abundant comments we received (34%). (2) Feedback: confusion around the feedback
provided by the application represented the next most abundant issues identified by
users (29%). In this category, we include descriptors (e.g., headers, labels, figure
legends, and units) that were inadequate, nonsensical, or missing. (3) Features: issues
specific to selected features of the application represented 21% of the feedback received.
These included pop-up notifications that the user could not close, graphical features
that autoscaled to data, user selections that were reflected on a graph but not in
the companion table, and visibility of the help feature which could not be identified
by eight users. (4) System: issues stemming from the development mode in which we
were working contributed to approximately 12% of issues. Some users experienced a
system that was operating slowly, freezing on selected screens, or jumping screens
unbeknownst to the user. There were also episodes of screens prepopulating with cached
information from the preceding clinical scenario. (5) Navigation: less than 5% of
the issues identified related to navigation within the application and all of these
could be attributed to the user navigating to an earlier screen using the back button
in the browser rather than the back button nested in the application. Irrespective
of these issues, satisfaction with the application was extremely positive. Using a
7-point Likert scale with 1 representing the most favorable response and 7 the least
favorable response, overall satisfaction with the application rated a 1.42. Scores
for system quality rated 1.56, information quality rated 1.73, and interface quality
rated 1.33. The distribution of scores for each element of the PSSUQ is provided in
[Fig. 4 ].
Fig. 4 Histogram of Poststudy System Usability Questionnaire responses provided by our usability
testers.
Discussion
Main Findings
In this manuscript, we describe the development and testing of CDS software that shifts
the flow of PK-based therapeutic decision-making from a consultative service model
to seamless point-of-care execution. Direct access to antihemophilic factor modeling
and simulation, 24/7, offers primary hematology providers enhanced flexibility when
caring for patients. Though clinical pharmacologists remain accessible to the hematology
team, they satisfy an ancillary role reserved for mathematically complex scenarios.
This approach diverges from traditional hospital-based PK services where specialists
are consulted to conduct modeling and simulation. However, we made a business decision
to prioritize the efficiency of care delivered by the patient's primary provider over
the protection of a billable domain for a secondary specialist.
We focused on developing an application that integrates transparently with the EHR,
intelligently filtering, organizing, and delivering the relevant information to the
tool at the appropriate time.[13 ] We also vetted the application with a multidisciplinary group of providers to ensure
that it performs optimally and supports the needs of the end user. To that end, we
appear to have been successful as evidenced by median scores from the usability questionnaires
that lie between 1 and 2 and objective performance metrics which demonstrate that
key tasks are rapidly learned after working through one use case.
Related Dosing Tools
Two other publicly accessible PK-driven dosing tools have been described, though there
are notable differences with the tool we developed. The U.S. Food and Drug Administration
(FDA)-cleared myPKFiT is a web-based dosing tool for optimization of Advate therapy.[28 ] WAPPS-Hemo is a web-based service that can be applied to factor VIII and IX products.[29 ] Both adopt a popPK approach though only myPKFiT references their base model, which
has been peer-reviewed and published. Both must be accessed independently and rely
on the transfer or transcription of patient specific variables. As a service, WAPPS-Hemo
also introduces a delay between the time the data are submitted, and a dosing recommendation
is returned as compared with our tool which delivers a recommendation in real-time
as the clinician performs the simulations. Finally, each has constraints with respect
to the execution of their modeling and/or simulation algorithms.[30 ]
Importantly, side-by-side comparisons of these two tools demonstrates that they do
not always arrive at the same dose and both generate a different dose than is derived
using traditional popPK software (NONMEM).[30 ] Whether this results from the uncertainty nested in their parameter estimates, or
the fact that they may be applying a model developed in a population with characteristics
that differ from the patients that were examined is unclear. Armed with these data,
our providers recognized the need to collect more empiric data on the performance
of various population models before broad implementation which is why our tool relies
on classical PK for dose simulation but integrates popPK at the modeling step to provide
the user with estimates of reliability and uncertainty that can be expected with these
approaches.
Limitations
There exist a several of limitations tied to the development of the application. At
present, the application represents a partially rather than fully EHR-integrated CDS
tool. Single sign-on authentication onto the institutional computing system permits
access to the application via a weblink, as opposed to direct access from within the
EHR, necessitating the user to enter the patients MRN. We are exploring the ability
to embed the weblink directly into the PowerForm, thus bypassing the need for MRN
entry but have yet to test this functionality. We also considered, but elected to
delay, an automated push of the results back into the EHR. As with our previous CDS
tools, we commit a period of time to fully QA the application's output in a real-world
setting before automating delivery back to the medical record. When sufficient experience
has been gained, a routine HL7 protocol can be used to implement this change and push
the output to a dedicated location in the patients' chart.
Additional limitations relate to the findings of our usability testing. All testers
in this study derived from a single institution with a robust clinical pharmacology
program. As a result, they may share a more complete understanding of PK than users
at other institutions. This may also explain why we observed essentially no differences
in performance metrics between our CE and end users. Our inability to discriminate
differences in performance as a function of participant characteristics may have been
influenced by sample size. However, our previous usability study for a different PK-based
CDS application with a similar sample size could still to distinguish differences
between different participant groups.[19 ] Alternatively, this might be explained by the slightly more complex nature of the
modeling and simulation requirements for the medications addressed by this application.
This assertion is corroborated by slightly longer task times for similar actions in
this, compared with the earlier, study.
Next Steps
Immediate next-steps involve prospectively examining the accuracy and predictive performance
of the CDS to expose whether the back-end model can be augmented. We are also architecting
a companion patient-facing application. Despite an established link between adherence,
frequency of bleeds, and QOL,[31 ]
[32 ]
[33 ] adherence rates vary substantially and strategies to promote adherence in hemophilia
patients are not widely addressed.[32 ]
[33 ]
[34 ]
[35 ]
[36 ]
[37 ]
[38 ] Patients rarely receive more than superficial insight into the dose selection strategies
used by their prescriber or an evidence-based explanation detailing the consequences
of altered dosing regimens. Consequently, it is unrealistic to expect that patients
with chronic conditions are self-motivated to comply with regimens that are incompatible
with their lifestyle.[39 ]
[40 ]
[41 ] Our application feature allowing providers to explore dosing recommendations beyond
the idealized (e.g., administration Monday and Thursday vs. every 72 hours) was borne
out of this understanding; however, the simple act of individualizing treatment cannot
guarantee adherence. Providing access to adaptive, individualized, patient-centered
education may have the power to enhance knowledge around prescribed dosing regimens,
promote shared decision making, and ultimately influence adherence behaviors.
Conclusion
We developed a user-friendly CDS tool for hematology provider that affords seamless
transition from patient assessment, to pharmacokinetic modeling and simulation, and
subsequent dose selection.
Clinical Relevance Statement
Clinical Relevance Statement
Coupling early and broad stakeholder engagement with a provider-centered design strategy
can result in point-of-care solutions that are favorably received by clinicians. EHR
integration further increases the utility of these solutions and the likelihood of
uptake in clinical settings.
Multiple Choice Questions
Multiple Choice Questions
Successful clinical decision support tools possess which of the following attributes?
They exist as standalone solutions.
They require additional data entry.
They are available at the time/location of decision-making.
They provide an assessment only without a formal recommendation.
Correct Answer: The correct answer is option c.
Which of the following statements is true about structured CW?
They should only be conducted with CE.
They should only be conducted with EU.
They only generate subjective data on interactions between the user and your software.
They generate data that can be used to iterate your software design.
Correct Answer: The correct answer is option d.
