Keywords
testing and evaluation - data quality - emergency and disaster care - intensive and
critical care - public health
Background and Significance
Background and Significance
Capturing accurate and relevant clinical data in real time is an ongoing challenge
in clinical and public health research. In the context of a national public health
emergency, this issue becomes even more pressing due to the need to provide quality
patient care and to allocate resources rapidly.[1] One way to address these challenges is to identify research priorities and prepare
forms to capture data. Having these forms prespecified ensures that they can be deployed
in a timely manner during a real emergency. Validated scoring tools are one type of
data capture form that is important to have ready for rapid deployment. These tools
facilitate information sharing between clinical sites, comparisons across studies,
and reproducibility of methods and analyses.[2] Similarly, the use of widely available technologies and software to collect such
data is important to promote participation by as many sites as possible.[3]
The United States Critical Illness and Injury Trials Group-Program for Emergency Preparedness
(USCIIT-PREP) is a group of clinicians who are committed to building the capacity
for such research and preparedness efforts.[4] USCIIT-PREP's focus is on developing communication, data collection, and data reporting
networks that would allow sites to rapidly deploy clinical research protocols in the
event of a public health emergency.[5] The specific threats that USCIIT-PREP's work addresses include infectious diseases
that lead to critical illness such as influenza, anthrax, and severe acute respiratory
infection.
REDCap
Research Electronic Data Capture (REDCap) is an electronic data collection tool, created
by Vanderbilt University and a consortium of institutions, which is widely used in
academic medical centers for clinical trials and other research studies.[3] Main advantages of REDCap are that it is user-friendly, intuitive, and allows custom
calculations and logic to be incorporated into instruments. Another advantage is that
REDCap is available at no cost for consortium participants. The literature has demonstrated
REDCap's flexibility and customizability by using it to build data management systems
for a variety of purposes and subject areas, including ARDS/sepsis patient data,[6] genitourinary oncological data,[7] and pacemaker long-term outcomes.[8] REDCap is an effective tool for collecting data across multisite studies, which
is especially important for gathering sufficient data to investigate rare conditions.[9]
Another useful feature of REDCap is the Shared Data Instrument Library, which allows
researchers across the REDCap consortium to share and reuse instruments. This resource
aims to reduce the amount of time spent building data collection instruments by decreasing
duplication of effort in the design phase, thereby allowing instruments to move from
development to production more quickly. The Shared Library also promotes the use of
validated tools and data standards to facilitate data sharing across institutions
and studies.[10] The ability to quickly deploy shared data collection instruments is especially important
due to the time-sensitive nature of many public health emergencies.[11]
APACHE II
The Acute Physiology and Chronic Health Evaluation (APACHE) II score is used in the
critical care setting to evaluate patients' severity of disease and risk of death,
as well as other adverse outcomes.[12] The APACHE II score can also be used to stratify patients by risk level, aiding
in the evaluation of clinical therapies and interventions.[12] APACHE II has been independently validated for use in several acutely ill patient
populations, such as breast cancer patients in the intensive care unit (ICU),[13] neurological ICU patients at risk of nosocomial infection,[14] and hemorrhagic stroke patients.[15]
The APACHE II scoring tool provides a quantitative assessment of a patient's mortality
risk by assigning points in three categories: the patient's age, the patient's Acute
Physiology measurements, and the patient's chronic health status ([Table 1]). In each category, the possible values for each measurement are separated into
ranges and each range is assigned a point value. Normal measurements are assigned
zero points, and increasingly abnormal measurements receive higher points. For example,
age points increase with the patient's age, whereas chronic health status points are
assigned based on a patient's health history and admission status. Nonoperative and
emergency surgery ICU patients receive a higher number of chronic health points than
patients admitted for elective surgery. The addition of all points in an individual
category constitutes the category “subscore.” The sum of the age, acute physiologic
measurements, and chronic health status subscores equals the total APACHE II score,
which can range from zero (best) to 71 (worst).[12]
Table 1
Measurements included in APACHE II score calculation
|
APACHE II measurements
|
Original ranges (assigned points)
|
|
Age subscore
|
|
|
Age
|
≤44 (0); 45–54 (2); 55–64 (3); 65–74 (5); ≥75 (6)
|
|
Acute Physiology subscore
|
|
|
Temperature
|
36–38.4 C (0); 34–35.9 C or 38.5–38.9 C (1); 32–33.9 C (2), 30–31.9 C or 39–40.9 C
(3); ≤29.9 C or ≥41 C (4)
|
|
MAP
|
70–109 (0); 55–69 or 110–129 (2); 130–159 (3); ≤49 or ≥160 (4)
|
|
Heart rate
|
70–109 (0); 55–69 or 110–139 (2); 40–54 or 140–179 (3); ≤39 or ≥180 (4)
|
|
Respiratory rate
|
12–24 (0); 10–11 or 25–34 (1); 6–9 (2); 35–49 (3); ≤5 or ≥50 (4)
|
|
If FiO2 < 0.5, PaO2
|
If FiO2 ≥ 0.5, A-a difference
|
PaO2:
> 70 (0); 61–70 (1); 55–60 (3); < 55 (4)
|
A-a Difference:
< 200 (0); 200–349 (2); 350–499 (3); > 499 (4)
|
|
Arterial pH
|
Serum bicarbonate
|
Arterial pH:
7.33–7.49 (0); 7.5–7.59 (1); 7.25–7.32 (2); 7.15–7.24 or 7.6–7.69 (3); <7.15 or ≥ 7.7
(4)
|
Serum bicarbonate:
22–31.9 (0); 32–40.9 (1); 18–21.9 (2); 15–17.9 or 41–51.9 (3); < 15 or ≥ 52 (4)
|
|
Serum sodium
|
130–149 (0); 150–154 (1); 120–129 or 155–159 (2); 111–119 or 160–179 (3); ≤110 or
≥180 (4)
|
|
Serum potassium
|
3.5–5.4 (0); 3–3.4 or 5.5–5.9 (1); 2.5–2.9 (2); 6–6.9 (3); <2.5 or ≥7 (4)
|
|
Creatinine if no acute renal failure (ARF)
|
Creatinine if YES ARF
|
No ARF:
0.6–1.4 (0); <.6 or 1.5–1.9 (2); 2–3.4 (3); ≥3.5 (4)
|
Yes ARF:
<.6 or 1.5–1.9 (4); 2–3.4 (6); ≥3.5 (8)
|
|
Hematocrit (%)
|
30–45.9 (0); 46–49.9 (1); 20–29.9 or 50–59.9 (2); <20 or ≥60 (4)
|
|
White blood cell count
|
3–14.9 (0); 15–19.9 (1); 1–2.9 or 20–39.9 (2); <1 or ≥40 (4)
|
|
Glasgow Coma Scale (GCS)
|
GCS eyes + GCS speech + GCS motor
|
|
GCS eyes
|
(4) Spontaneous opening; (3) to speech; (2) to pain; (1) absent
|
|
GCS speech
|
(5) Converses/oriented; (4) converses/disoriented; (3) inappropriate; (2) incomprehensible;
(1) absent
|
|
GCS motor
|
(6) Moves to command; (5) localizes to pain; (4) withdraws to pain (flex); (3) decorticate
(flexor posturing); (2) decerebrate (extensor posturing); (1) Absent
|
|
Chronic health status subscore
|
(5) Nonsurgical admission or admission for emergent operation; (2) admission for elective
operation
|
Abbreviations: APCHE II, Acute Physiology and Chronic Health Evaluation II; FiO2, fraction of inspired oxygen; MAP, mean arterial pressure; PaO2, partial pressure of arterial oxygen.
Current APACHE II Implementations
Two APACHE II implementations currently exist in the REDCap Shared Library.[10] However, these forms have several limitations in the context of an emergent situation,
where rapid and accurate data collection is critically important. The first implementation,
Version 1, provides space for point values to be entered manually for age, physiological
measurements, and chronic health, but does not provide guidance on what those point
values should be. The second implementation, Version 2 (with scoring), automatically
calculates subscores and the final APACHE II score, but it requires all data to be
entered twice: once in a free text field for the specific value, and once as a multiple-choice
question to assign a point value to the measurement. Addressing these issues—uncertain
instructions for assigning points and the time burden posed by double data entry—is
important to make the APACHE II instrument appropriate for use in an emergency situation.
Objectives
The objective of this article is to describe the methodology used to create an electronic
APACHE II instrument that (1) leverages REDCap functionality for automated calculations
and logic, (2) may be shared across research sites using REDCap's shared library,
(3) overcomes limitations in existing shared instruments for APACHE II in the REDCap
library, and (4) provides a paradigm for data collection instruments that can be used
during public health emergencies.
Methods
The APACHE II form was implemented in REDCap version 6.5.8. As the data collection
instrument created in this study was intended to be used as an electronic Case Report
Form (eCRF), best practices for designing CRFs and/or eCRFs were considered throughout
the design process. These guidelines indicated that many of the challenges in design,
collection of precise data, and user-friendliness could be addressed during the creation
process with careful planning by a multidisciplinary team.[16] To that end, our team included informatics, critical care, nursing, software development,
and data experts. Additionally, our process included several steps to intentionally
elicit input from additional subject matter experts during the initial implementation
and evaluation. Best practices also guided the design of the instrument itself.[16] Among the recommendations incorporated in our implementation of the USCIIT-PREP
APACHE II instrument were consistent formatting, clear and concise questions, clear
skip patterns, avoidance of “check all that apply,” specification of units, and use
of consistent data formats. These details are described in the following subsections.
To facilitate data entry, we divided the three APACHE II original categories into
four sections: age, Acute Physiology, Glasgow Coma Scale (GCS), and chronic health
status. The original APACHE II instrument included GCS as one of the 12 measurements
in the Acute Physiology category. However, to provide complete information to clinicians
and accurately determine the number of points assigned to a patient's GCS, our APACHE
II instrument placed GCS in a separate section with its own subscore (see [Table 1]).
Age
This section was configured as two data fields: a multiple-choice question where the
user could select the appropriate age from a list, and a calculated field that generated
the age subscore. In the first field, each answer choice contained an age range that
corresponded to 0, 2, 3, 5, or 6 points, with increased age corresponding to a higher
score (as shown in [Table 1]). The choice to report age using a multiple-choice field, rather than free text,
reflected best practices for CRF design by minimizing the chance of a typing error.
The subscore was configured as a calculated field whose value equaled the points assigned
to the age chosen in the preceding question.
Acute Physiology
The Acute Physiology subscore of our APACHE II instrument consisted of 11 measurements,
([Table 1]). Measurements were configured as multiple-choice fields where only one answer choice
could be selected. Each answer choice contained a range of physiological values and
each range was assigned a point value: zero for the range containing the normal measurement,
and increasing point values for increasingly abnormal ranges. To facilitate data entry,
the ranges were reorganized so each numerical range was presented as its own answer
choice, even if more than one range was assigned the same point value, and ranges
were presented in ascending numerical order for clarity. [Table 2] shows an example of the reorganization of the answers made to improve clarity.
Table 2
Reorganization of answer choices to improve clarity
|
Data entry element
|
Original choice list
|
Points
|
Reorganized choice list
|
Points
|
|
Temperature (°C)
|
36–38.4
34–35.9 or 38.5–38.9
32–33.9
30–31.9 or 39–40.9
≤ 29.9 or ≥ 41
|
0
1
2
3
4
|
≤ 29.9
30–31.9
32–33.9
34–35.9
36–38.4
38.5–38.9
39–40.9
≥ 41
|
4
3
2
1
0
1
3
4
|
In the original APACHE II, data must be entered for three pairs of questions. The
first of these pairs is the partial pressure of O2 in arterial blood (PaO2) or the alveolar to arterial (A-a) Difference. Whether the clinician should enter
PaO2 or A-a difference depends on the patient's fraction of inspired oxygen (FiO2) measurement. The second set of paired questions is the arterial pH or serum bicarbonate:
two different laboratory values, of which one is entered at the user's discretion.
The third pair of questions is the creatinine level, with different response choices
depending on whether the patient is in acute renal failure or not. For each of these
paired questions, only one of the two measurements should be included in the calculation
of Acute Physiology subscore. The USCIIT-PREP APACHE II instrument was built to prevent
users from entering incorrect data by hiding each pair from view using branching logic.
An “either/or” question was inserted before each pair to specify which measurement
would be entered. Upon answering the preceding “either/or” question, the corresponding
field would appear for the end-user to answer. For example, in the PaO2 or A-a Difference pair, the end-user was asked first for the FiO2 measurement, and, depending on the answer, the corresponding question would be displayed.
No points are assigned to the “either/or” questions (e.g., FiO2).
Improvements were also implemented in the Acute Physiology subscore, which was designed
as a calculated field. In accordance with best practices for CRF development, the
equation for this field was designed to fulfill several requirements. First, it needed
to assign point values to each of the answer choices for every field in this section
(since many questions had multiple answer choices that received the same number of
points). Second, since REDCap's default functionality is to require all fields included
in a calculation to be filled for the calculation to execute, the calculation needed
to execute even when some fields in the section remained unfilled. Finally, it needed
to add all of the point values for the Acute Physiology section without requiring
the user to perform any calculations by hand or reference another source for scoring
guidelines.
Glasgow Coma Scale
The original APACHE II form included points for a patient's GCS as a component of
the Acute Physiology score. However, it simply asked for the “Glasgow Coma Score points,”
with a note saying to calculate it as “15 minus the actual GCS,” without providing
any guidance on how to directly calculate “actual GCS.” To facilitate data entry,
we chose to include questions to help end-users determine a patient's actual GCS.
The GCS consists of three questions regarding a patient's eye-opening response, motor
response, and verbal response. These three questions were configured in REDCap as
radio button fields, with a point value assigned for each response (see [Table 1] for point values). Two calculated fields, “Glasgow Coma Scale Total” and the Glasgow
Coma Score points (“GCS Subscore”) were included in this section. GCS Total was automatically
calculated as a sum of the point values for eye, motor, and verbal responses, and
the GCS subscore was automatically calculated as 15 minus actual GCS. Calculated fields
and radio buttons, rather than free text, were used to promote data consistency and
reduce error. For clarity, these questions were separated into their own section rather
than being included in the Acute Physiology section, as shown in [Fig. 1].
Fig. 1 USCIIT-PREP APACHE II Instrument: Glasgow Coma Scale (GCS) and GCS subscore.
Chronic Health Status
Points are added to the APACHE II score based on the purpose of a patient's hospital
admission only if the patient had prior chronic health problems. In the original APACHE
II form (and in previous REDCap implementations), the criteria to determine if a patient
has chronic health problems were grouped together in one long paragraph. For clarity,
we separated the individual conditions from this paragraph into a series of 5 “yes”
or “no” screening questions. Answering yes to any one of these questions indicated
the presence of prior chronic illness and would trigger a question to appear regarding
the patient's hospital admission status. Two points were assigned for the option “Admission
for elective operation,” while five points are assigned for the answer “Non-surgical
admission or admission for emergent operation,” as shown in [Table 1].
The chronic health status subscore was configured as a calculated field, where the
value reflected the points for the patient's hospital admission status, or zero points
if all five of the chronic health screening questions were answered “no.”
Final APACHE II Score Calculation
The APACHE II score was given by a calculated field where the equation is the sum
of the age, Acute Physiology, GCS, and chronic health subscores. The choice to use
a calculated field was made to decrease the chance of error arising from manual calculations.
Preliminary Evaluations
Usability testing and preliminary evaluation of the instrument were conducted using
data derived from four published case reports of critically ill influenza and pneumonia
patients.[17]
[18]
[19]
[20] Published case data of critically ill respiratory patients was used because of its
availability, suitability for preliminary data entry evaluation, and applicability
to USCIIT-PREP's mission of conducting research among critically ill patients. The
USCIIT-PREP APACHE II instrument was first validated by three informatics experts,
with special attention to usability and data consistency. Phase one of preliminary
evaluation compared the USCIIT-PREP APACHE II Instrument to the two APACHE II instruments
previously available in the REDCap Shared Library. Several characteristics of the
instruments were compared, including the total number of fields, field types, number
of manual versus automatic calculations, and the minimum number of clicks required
to complete each instrument. Finally, a second phase of preliminary evaluation of
the APACHE II instrument was conducted during a USCIIT-PREP preparedness study of
critically ill patients with influenza. This study included clinical sites across
the country and was intended to test a larger data collection instrument, which included
APACHE II, more thoroughly. The test data from the published case reports were used
during the preparedness simulation, along with additional details provided by clinicians
at the study sites. In both phases of evaluation, the time taken to complete each
instrument was tracked automatically through REDCap logs and calculated as the aggregate
time spent on each form (page), inclusive of all sessions when the same form was saved
and returned to later. The implementation process in REDCap including initial configuration,
testing, revisions, and final adjustments took ∼40 hours total.
Results
The four sections of the final USCIIT-PREP APACHE II instrument consist of a total
of 34 data fields, 21 of which are marked as required. Twenty-seven (79.4%) are radio
button fields where the clinician must select only one answer, and seven (20.6%) are
automatically populating fields, including six calculations and the participant identification
number assigned by REDCap. There are no free text fields in the instrument.
The six calculated fields contain variables that reference previous fields in the
instrument. The number of variables per calculation ranged from 1 to 17, with a median
of 3 ([Table 3]). Although each patient's Acute Physiology subscore is the sum of 11 measurements,
the calculation incorporated 17 variables to account for the different possible combinations
of choices in the three sets of paired questions.
Table 3
Calculations and number of variables
|
Calculation
|
Number of variables
|
|
Age subscore
|
1
|
|
Acute Physiology subscore
|
17
|
|
Actual Glasgow Coma Scale
|
3
|
|
Glasgow Coma Scale subscore
|
3
|
|
Chronic health subscore
|
1
|
|
Total APACHE II score
|
4
|
Abbreviation: APCHE II, Acute Physiology and Chronic Health Evaluation II.
During usability testing by informatics experts, a total of three patient records
were entered into the APACHE II instrument. These three records contained data in
17 (81%), 8 (38%), and 1 (5%) of the 21 required fields. Minor issues were identified
by the informatics experts and subsequently resolved by consensus. For example, in
the chronic health section, the original answer choices “1) Non-surgical or emergent
operation; or 2) Elective operation” were determined to be ambiguous. At first glance,
it was unclear to which category medical patients would belong because both answer
choices appeared to refer only to surgical patients. Therefore, the wording of the
two choices was changed to “1) Non-surgical admission or admission for emergent operation;
or 2) Admission for elective operation” to distinguish nonsurgical patients from other
types of patients. Another change made for data consistency was the addition of the
brief instruction “Use worst values within the first 24 hours of ICU admission” at
the top of the form. This note was included to prevent incorrect data being entered
in cases where more than one value or measurement might be available.
The evaluation comparing the USCIIT-PREP APACHE II instrument to the two existing
REDCap instruments was done by the first author and is summarized in [Table 4]. This comparison confirmed that the final APACHE II score was the same for each
patient across all three instruments. However, while the final score was the same,
the time needed to complete each instrument was substantially different. Versions
1 and 2 contained more free text fields and manual calculations than the USCIIT-PREP
APACHE II instrument, which in turn contained more automatically calculated fields
and more multiple-choice fields ([Table 4]). Each multiple-choice field was completed with one click. While completing Version
1, both APACHE II and GCS references had to be consulted to determine various point
values, contributing to the overall time. While completing Version 2, the GCS reference
was needed. The USCIIT-PREP APACHE II instrument did not require any external resources
and had the lowest average completion time.
Table 4
Comparison of APACHE II instruments
|
Instrument name
|
Fields
|
Clicks
|
Free text fields
|
Manual calculations
|
Automatic calculations
|
Time (seconds)
|
|
APACHE II Version 1
|
23
|
3–4
|
13
|
3
|
1
|
323.5
|
|
APACHE II Version 2 (with scoring)
|
37
|
16–17
|
11
|
1
|
3
|
183.5
|
|
USCIIT-PREP APACHE II
|
35
|
23–24
|
0
|
0
|
6
|
97.5
|
Abbreviation: APCHE II, Acute Physiology and Chronic Health Evaluation II.
During the USCIIT-PREP public health emergency preparedness study, 34 simulated cases
were created by 10 sites across the U.S. End-users who completed these cases were
clinical research staff, including physicians and nurses, with a range of prior REDCap
experience. The average time spent completing the APACHE II instrument was 257.06 seconds,
or ∼4.3 minutes (minimum 22 seconds, maximum 954 seconds, median of 150.5 seconds).
The 21 required fields ranged in completeness from 35.3 to 97.1% in the 34 patient
records. Fields completed in only 35.3% of patient records include serum sodium, serum
potassium, hematocrit, white blood cell count, and GCS eye, verbal, and motor assessments.
Age was complete in 97.1% of records.
Discussion
The ability to conduct research during public health emergencies is vital to enhance
ongoing responses and to prepare for future events.[11] Considering the many challenges faced by clinicians and other responders when treating
patients during emergencies, it is important for research instruments to be designed
in a way that allows data to be collected quickly and accurately without interfering
with treatment.[5] This study demonstrates the ability to leverage REDCap to design a data collection
instrument that can achieve these aims.
The USCIIT-PREP APACHE II instrument represents several improvements over previous
implementations in the context of public health emergencies, when time, supplies,
and medical staff are under stress. For example, the Glasgow Coma Score points (GCS
subscore) form is designed to save time and improve data consistency by minimizing
the effort required and using automatically calculated fields. Providing sufficient
detail and assistance to end-users is especially important because the reliability
of GCS has been found to depend on the experience and training of the scorer, as well
as the consistency of the scoring instrument.[21] Data collection instruments that maximize consistency are especially important during
an emergency, when end-users may include individuals with less training or experience
completing specific scoring tools.
Another change that aims to facilitate optimal medical care during an emergency is
the reorganization of answer choices (as in [Table 2]) for data fields in the Acute Physiology section, providing a more user-friendly
layout. In the previously available Version 2 of the APACHE II instrument, the multiple-choice
questions were organized so that each answer choice included the range or ranges corresponding
to one particular point value. For example, one point was given for a temperature
measurement of “34-35.9 or 38.5-38.9” degrees Celsius. Including nonconsecutive ranges
in one answer choice, separated by the word “or,” could cause confusion and delays
in the data entry process. Instead, the USCIIT-PREP APACHE II instrument lists all
numerical ranges in ascending numerical order. This rearrangement necessitates a more
complex equation for the calculation of the Acute Physiology subscore than the previous
arrangement does. However, considering the context of emergency preparedness, the
choice to use a more complex calculation results in a more user-friendly and efficient
data entry format for clinicians.
Finally, the assignment of point values to measurements in the Acute Physiology section
improves upon previous implementations by completely automating the process. Version
1 of the instrument does not provide any point values or calculations, requiring clinicians
to consult an external APACHE II source to determine point values for each measurement
and add up scores by hand. Version 2 of the instrument does include point values,
but requires each data field to be entered twice (once as text and once as a radio
button) and utilizes the less intuitive multiple-choice answer format described above.
The USCIIT-PREP APACHE II instrument, on the other hand, includes calculated fields
to both assign point values and automatically add up scores with just a single click
for each measurement.
Preliminary evaluation of the USCIIT-PREP APACHE II instrument confirmed that it is
faster to complete than previous implementations, while still providing an accurate
final score. Recommendations from expert reviewers obtained during usability testing
confirmed the importance of carefully considering wording of questions and instructions,
helping end-users understand exactly what data to enter. This is especially important
for data consistency and for the validity of the final APACHE II score.
The comparison of the three APACHE II instruments highlighted the differences in the
design and the effort required to complete the three instruments. In general, having
fewer free text fields and manual calculations provides less opportunity for error,
while having more automatic calculations and multiple-choice questions improves both
efficiency and accuracy. The simulated public health emergency exercise resulted in
a slightly higher median time to complete the instrument than the instrument comparison
(150.5 vs. 97.5 seconds). However, this difference may be due to the fact that many
clinicians involved in the emergency simulation had to supplement test cases with
additional data and were also using the complete USCIIT-PREP data collection instrument
for the first time. In addition, the participation of multiple sites provided the
opportunity for a wider group of users, most of whom were familiar with REDCap, to
use the instrument and provide feedback. Anecdotally, a few users commented that they
found the new USCIIT-PREP APACHE II instrument better and easier to use than either
of the previous APACHE II instruments in the REDCap Shared Library. Although the simulation
also resulted in several patient records with missing data, cases entered using medical
record data from a small number of real patients (not described in this article) produced
complete records, as expected.
Limitations of the USCIIT-PREP APACHE II instrument include that it represented only
one scoring tool among many that are commonly used in clinical settings, and it was
tested using a small number of cases in a nonemergency situation. More data about
the instrument will be available after the upcoming influenza season, when there will
be further opportunity for it to be used prospectively with real patients undergoing
treatment in the ICUs of multiple hospitals across the country. The instrument has
been submitted to the REDCap Shared Library and is pending approval. To continue building
the capability for data collection across multiple sites during emergencies, future
work should include identifying other validated scoring instruments to develop in
a similar manner. Future work should also extend our evaluations by involving a wider
range of third-party end-users and by utilizing medical record data.
Conclusion
This study demonstrates improvements to a data collection instrument that result in
improved timeliness and accuracy, both of which are critical during the response to
a public health emergency. Future work is needed to create a library of readily available
data collection instruments that can be deployed in an emergency situation. The ability
to collect and share data during public health emergencies depends on the development
of this research infrastructure.
Clinical Relevance Statement
Clinical Relevance Statement
This project demonstrates the feasibility of building a data collection instrument
that can capture accurate data in a short amount of time. These methods can be used
to prepare for clinical research efforts during public health emergencies.
