Keywords record linkage - dissemination - patient engagement - stakeholder engagement - patient-centered
outcomes research
Background and Significance
Background and Significance
In the past 15 years, there has been tremendous advancement in data science methods
that leverage data collected in routine care for clinical and translational research.[1 ] Innovations in extraction, transformation, and loading,[2 ] data harmonization,[3 ] data quality assessment,[4 ]
[5 ] and data governance[6 ]
[7 ] have emerged and are available to guide this research, yet there are few experts
qualified to apply these methods. Ensuring innovative data science methods are disseminated
broadly and can be accurately and appropriately applied to clinical research is critical
to ensuring research rigor.[8 ]
Record linkage (RL) methods are among the recent advances in secondary use of clinical
and claims data for research.[9 ]
[10 ]
[11 ] RL refers to the technical and analytic methods for effectively and securely matching
patient records from multiple distinct health systems and data platforms, such as
electronic health records, administrative claims, patient-reported outcomes measures,
and digital health devices.[12 ]
[13 ]
[14 ] RL has several benefits for research, such as enhancing the amount and types of
information available about each patient (e.g., information about all services a patient
has received, what prescriptions have been filled, how much physical activity they
have done, and their quality of life), which are not typically all included in one
data source. This facilitates answering patient-centered research questions about
the extent to which certain treatments and interventions impact objective patient
behavior, self-reported outcomes, and clinical outcomes, and the relationships among
these factors. RL can also enable deduplication to ensure the research sample represents
unique individuals (e.g., not double counting a patient who has been seen at more
than one health care institution if a researcher has received records from multiple
institutions). This helps ensure that the size of the population comprising the research
sample is accurately described.
To realize the promise of RL methods for advancing patient-centered outcomes research
(PCOR), investigators must adopt and apply RL methods to the testing of novel hypotheses
using secondary health data from multiple sources. University of Colorado Record Linkage
(CURL) is an electronic health RL software for linking electronic health data from
disparate systems (e.g., clinical and claims data).[9 ] As with all research products, active dissemination to potential adopters and influencers
is needed to support broad researcher adoption.[15 ]
[16 ] Dissemination of data science and informatics methods may benefit from the design
of materials that communicate value propositions and facilitate the use of novel health
care informatics tools in research and, eventually, clinical care.[17 ]
As RL and other informatics tools are often used to research patient data[12 ]
[18 ] (and results have direct implications for and potential value to patient care and
health outcomes) patient perspectives are a critical component of positioning their
value to promote dissemination and uptake.[19 ]
[20 ] Further, as patient engagement throughout the research lifecycle is increasingly
prioritized,[21 ]
[22 ] communication materials for engaging with patients around the implications of research
methods are required for effective dissemination of novel informatics tools and methodologies.
Thus, engaging patients and other stakeholders in dissemination planning allows for
the development of effective messages and materials for conveying the value of RL
methods, which in turn will promote future adoption of RL methods and ensure transparency
in how patient data are used in PCOR and resulting clinical informatics solutions.
Objectives
Our objective was to identify the needs and perspectives of patients and other stakeholders
as potential adopters and influencers of RL methods and tools and to design messages
and materials to guide the use of RL methods and tools (henceforth, “the RL dissemination
package”).
First, we present the methods used to engage stakeholders in the iterative design
and development of the RL dissemination package, including the application of user
experience (UX) research and design methods. Next, we describe how the results were
applied to iterate design choices. To conclude, we discuss the larger contributions
of this work to patient engagement in informatics and dissemination science.
Methods
Overview of Dissemination Planning Framework
The dissemination planning framework ([Fig. 1 ]) begins with stating dissemination goals and objectives.[17 ] Stakeholders are engaged in addressing identified goals and objectives by delineating
the dissemination plan components[23 ]: (1) Audience segments (potential adopters and influencers of RL methods); (2) Messages
(language and imagery to convey the value and impetus for using RL methods); (3) Packaging
(the format or medium of the messaging, such as print or video); and (4) Distribution
channels (mechanisms to deliver the RL dissemination package to the audience, such
as email or television).
Fig. 1 Dissemination planning framework. Figure depicts the processes and components involved
in research dissemination planning.
Context
The current project describes patient and other stakeholder engagement in a Patient-Centered
Outcomes Research Institute (PCORI)-funded Methods Award focused on the development
of incremental privacy-preserving record linkage (iPPRL) methods.[24 ]
[25 ] The iPPRL method represents one type of process for RL; it is available for use
via the CURL system. Prior to launching the current project, we developed an explainer
video[26 ] that describes how CURL addresses key challenges to RL, such as establishing unique
identifiers, cleaning and normalizing data, and encrypting data to preserve privacy.
This video and a corresponding survey evaluation served as a launching point for conversations
with key stakeholders in the current project.
Design Team
The design team included a patient partner and five researchers with expertise in
informatics, RL, stakeholder engagement, communication, dissemination and implementation
science, user testing, and project management.
Goals and Objectives for Dissemination
Potential adopters of RL methods and tools include investigators conducting PCOR and
making research methods decisions as well as data managers and analysts who apply
those methods. Potential RL methods influencers include regulatory personnel who set
and enforce policies about research data privacy and security. There are also numerous
ways in which people whose health data are used in RL research (i.e., patients) represent
RL influencers. Patient perspectives are directly relevant, given that their health
care records are primary sources for RL research. Many patients are active participants
in PCOR and other types of clinical and translational research (e.g., clinical trials,
biobanks, registries). Dissemination goals and objectives were, therefore, to:
Provide investigators and analysts with the tools and resources needed to plan and
execute research using RL methods and the CURL platform.
Provide patient stakeholders with the communication materials necessary to be comfortable
and accepting of research using health data and RL methods.
Designing Messaging, Packaging, and Distribution Channels
To design the RL dissemination package, we employed patient engagement methods, customer
discovery and value proposition design methods,[27 ] and UX design methods.[28 ]
[29 ]
[Fig. 2 ] provides an overview and timeline of all activities.
Fig. 2 Design timeline. Figure provides an overall timeline with brief descriptions of all
design activities. CURL, The University of Colorado Record Linkage; RL, record linkage.
Patient Engagement Methods
Two methods were used for patient engagement in this project. First, we partnered
with a community member who served as a member of the design team. The purpose was
to integrate the patient perspective into all aspects of planning for future dissemination
and implementation of RL methods developed in the iPPRL project. The patient partner
has no training in research or informatics. She is a person who lives with multiple
chronic health conditions and has served as a patient stakeholder representative on
numerous PCOR projects. The patient partner met twice a month with other members of
the design team and was a coequal partner in decision-making. Her role frequently
involved suggesting plain language[30 ] for patient-oriented messages and materials and drafting scripts and graphic design
concepts with the team's communication expert. Over several months, the design team
conducted multiple colearning sessions[31 ]—a core principle of engagement—and discussed concerns patients may have about RL
research methods and how we might distill the processes and importance of the research
into a format accessible to a general audience.
Second, we engaged patients who serve on research advisory boards involved in conducting
research using patient health data. Patient research advisory boards were identified
via the design team's professional network. At key points during the design process,
patient stakeholder representatives provided input to inform RL dissemination package
revisions and identify distribution channels for the patient-oriented RL dissemination
package materials.
Customer Discovery and Value Proposition Design
Based on established value proposition design methods,[27 ] we conducted customer discovery interviews to inform RL dissemination package features
and resources needed to support the adoption and use of RL methods and tools. Value
proposition design provides a framework whereby a product or service is positioned
around what the customer values and needs. To begin, the design team emailed potential
RL adopters within their professional networks with an invitation to participate in
an interview about their work and RL-related challenges. Potential RL adopters included
those known by or recommended to the design team as (1) people already using RL for
health data in some capacity who were new to CURL; (2) people whose research aligned
with the future use of RL methods (e.g., investigators who work with patient record
data; (3) other stakeholders who informed RL adoption (data security and regulatory
officials). Snowball sampling was used to identify additional people to interview
by asking interviewees to nominate others. This sampling methodology was appropriate,
as potential adopters of RL (and CURL, specifically) are a somewhat niche community
that would be difficult to reach via broader sampling methods.
Customer discovery interview guides (tailored to stakeholder type) assessed the “jobs,
pains, and gains” (i.e., the key elements of value proposition design) of using RL
methods in research.[32 ]
[33 ]
[34 ] Questions focused on describing the work surrounding RL (e.g., “What is your job
related to RL?”) and identifying related pains (e.g., “What are the major challenges
you encounter?”) and gains (e.g., “What are some positive outcomes or improvements
you'd like to achieve at work?”). Customer discovery interviews were conducted in
June and July 2019 with 21 people. Interviews lasted 45 to 60 minutes and were conducted
by two design team members via phone or in person. Interviews were audio recorded;
interviewers took comprehensive notes. A matrix-based coding approach[35 ] was used to synthesize notes and identify themes related to “jobs, pains, and gains.”
Themes were then reviewed by design team members to articulate the current and aspirational
features of CURL that serve as “pain relievers” and “gain creators” for those using
RL methods.
Insights from customer discovery informed value propositions for key audiences. Value
propositions indicate how RL methods and tools align with the desired gains or relieve
pains for audience segments. We then used value proposition statements to inform messaging
about the benefits of using the CURL platform.
User Experience Design
Based on findings from the patient engagement and value proposition design activities,
we developed a RL Planning Workbook for investigators and RL animated explainer videos
for PCOR patient stakeholders. We used UX design principles and methods to iteratively
develop and refine prototypes.[15 ]
[36 ]
[37 ] In UX design, designers apply both what they have learned (e.g., from patient engagement,
customer discovery, and value proposition design) and the identified associated specifications
to create sketches, mockups, and product prototypes.[29 ] The design phase typically requires multiple iterations before producing a high-fidelity
product. We conducted user testing with both stakeholder groups to determine whether
the products adequately met users' needs.
Record Linkage Explainer Videos for Patient Stakeholders
To address patient stakeholder information needs, two animated explainer videos focus
on data security and privacy ([Fig. 3 ]).[38 ]
[39 ] We adapted notes from colearning sessions to create video scripts. A graphic designer
developed videos based on the scripts and visual ideas for representing concepts.
Fig. 3 Patient videos. Figure depicts two screen captures. One from the patient data and
privacy regulations video and one from the privacy-protected record linkage video.
HIPPA, Health Insurance Portability and Accountability Act.
We used two methods to test initial video prototypes with patients and other stakeholders.
First, in July 2020, one member of the design team emailed members of existing patient
stakeholder groups for other research projects asking them to view and evaluate the
videos using a brief online survey. We also sent the survey to investigators who use
stakeholder engagement methods. Survey participants comprised eight investigators
and seven patient stakeholders. The survey consisted of three open-ended questions,
plus one additional question specifically for investigators. Questions included: “What
are your general impressions of the videos? What do you like or dislike?”; “What (if
anything) did you find confusing or unclear?”; and “What questions or concerns do
you have about RL after watching the videos?” Additionally, investigators were prompted
to respond to: “As a researcher, we imagine that these videos would be helpful for
engaging patient stakeholders and educating them about RL research. In thinking about
your past or future/potential work, what else would you need in order to effectively
engage patient stakeholders in research that involves RL?”
Next, we presented the videos to five members of an established patient research advisory
board and asked: “What are your overall thoughts about the content of the videos?”;
“What else (beyond the videos) would you like to know about RL research methods?”;
and “How can we best educate patients about RL research methods?” We used this feedback
to refine the video scripts and graphics.
Record Linkage Planning Workbook
The RL Planning Workbook[40 ] was developed to meet the needs of potential CURL users as identified during the
customer discovery and value proposition design activities. The Workbook includes
worksheets to be used by researchers planning research projects that incorporate RL
methods ([Fig. 4 ]). The Workbook reflects steps in the RL process (as shown in the ideal process maps,
[Fig. 4 ]) and integrates guiding questions for decision-making, such as: “What types of data
do I plan to use?” and “Do my data meet the criteria performing RL?”
Fig. 4 Record Linkage Planning Workbook. Figure includes sample workbook pages. CURL, The
University of Colorado Record Linkage.
Record Linkage Workshop
A workbook prototype was presented during a RL workshop in November 2019.[41 ] Workshop attendees were invited colleagues with knowledge and work related to health
data and RL including data managers, investigators, and institutional leadership and
regulatory officials. Many were stakeholders we also engaged during customer discovery
interviews. Three RL subject matter experts (SMEs; outside of the design team) led
small group sessions where participants used the workbook to talk through application
in sample research contexts. RL SMEs were locally and nationally recognized for their
work using RL, but who were not otherwise involved in CURL. Participants made notes
indicating anything confusing or unclear in the workbook's contents. The full group
then discussed the workbook's usefulness and opportunities for improvement. Overall,
workshop participants expressed that the workbook was useful, had relevant prompts
for RL planning, appropriately described the RL challenges, and offered pathways for
RL solutions. However, participants also felt the layout was hard to follow, instructions
were vague, and that the workbook's applicability to other aspects of the research
planning process (e.g., grant proposals, institutional review board [IRB] protocols)
was unclear. We revised workbook content and improved visual design by adding further
instructions, flowcharts, and decision trees to illustrate RL requirements and guide
RL methods selection.
In spring 2021, we conducted user testing of the revised RL Planning Workbook using
a think aloud protocol[42 ] (a common user testing method) with 12 participants of varying levels of RL expertise.
Prior to completing the think aloud session, participants were asked to consider a
project involving RL. During the session, participants navigated the workbook with
this project in mind while vocalizing cognitive processes and sharing decision-making
considerations aloud. The sessions were facilitated by two researchers with usability
expertise. Insights from usability testing informed final workbook changes.
Distribution of the Record Linkage Dissemination Package
Final Patient Advisory Board Meeting
In June 2021, we met with the patient research advisory board to share the revised
patient stakeholder-oriented RL explainer videos and gather perspectives about distribution.
The discussion was framed around four questions: “In what contexts should videos be
used for patient engagement?”; “What additional information should accompany the videos?”;
“How might this support patient stakeholder engagement in patient-centered research
that uses RL?”; and “Based on these videos, as a patient stakeholder, what would be
your expectations about how security and privacy of patient health data would be managed
in an RL project?”
Record Linkage Town Hall
In July 2021, we presented the final workbook at a RL town hall and gathered suggestions
for dissemination.[43 ] Town hall participants (N = 58) were patient stakeholders, data managers, data analysts and programmers, researcher
investigators, technical professionals, and regulatory compliance representatives.
Several final town hall participants were individuals previously engaged in this project.
Town hall attendees were provided with an overview of using novel RL research methods.
Afterward, they were shown the workbook and the explainer videos and were asked: “As
we finalize the workbook, to what extent should it be agnostic to RL tools such as
CURL or specific to preparing to use CURL?”; “What types of local resources might
be needed to support users of the workbook?”; and “How might you use these videos
in the context of engaging stakeholders in research?” Additionally, we asked questions
specific to distribution planning, such as, “With whom should we share these materials?;
How might we reach them?”; and “When and where do people learn about RL methods?”
Results
Patient Engagement Insights
The patient partner recommended key design features of RL dissemination package components
designed for patient stakeholder representatives. First, the “KIS” principle (keep
it simple) was suggested as a guide for all plain language materials. Second, the
patient partner emphasized that the content should convey two main points: why it
was beneficial to use patient data in research (i.e., the types of research that could
be conducted and the advances in health care that research could lead to) and what
rules researchers must follow to protect patient data used in research (e.g., Health
Insurance Portability and Accountability Act [HIPAA] rules, need for informed consent
under certain circumstances). The broader set of patient stakeholders subsequently
engaged during the design process corroborated the importance of these factors. Thus,
the messages designed for patient stakeholder audiences focused on plain language
messaging about the benefits to society of RL research and the protections in place
to ensure patient data privacy and security.
Animated Videos: Patient Data and Privacy Regulations and Privacy-Protected Record
Linkage
Patient videos included patient-oriented content and language, such as the benefits
of RL methods and opportunities for answering important research questions with richer
datasets enabled through RL. [Table 1 ] provides an overview of the RL dissemination package design strategies—including
key messages, packaging, and distribution choices. Videos were designed to increase
shared understanding between researchers and patients engaged in the planning and
conduct of research studies that employ RL methods. The first video focuses on regulations,
privacy and security, and processes for using patient health data in research.[38 ] The second video provides an overview of RL methods and how RL may be used for research.[39 ]
Table 1
RL dissemination package design elements
Audience segment
Key messages
Packaging (format)
Distribution (communication channels)
Patients engaged in research
• Benefits of research using RL of patient data
• Rules governing use of patient data for research
Animated videos using plain language
Patient engagement research activities (in-person/virtual sessions, email, etc.)
Investigators
How to:
• Establish opportunity and need for RL
• Determine feasibility of RL
• Select and describe linkage approach
• Establish data-sharing agreements and approvals
• Execute RL using CURL
RL Planning Workbook;
findings from iPPRL project
Biostatistics and health data groups on campus, and academic societies (e.g., American
Medical Informatics Association [AMIA], the North American Primary Care Research Group
[NAPCRG], AcademyHealth)
Data analysts
Methods for improving data quality and efficiency
RL Planning Workbook; findings from iPPRL project
Findings published in research journals, conference presentations
Leadership and regulatory officials
Standard language for IRBs and data owners, highlighting how privacy-preserving RL
methods protect data privacy and ownership
Resources accompanying RL Planning Workbook, including animated videos
(see above)
Abbreviations: CURL, The University of Colorado Record Linkage; iPPRL, incremental
privacy-preserving record linkage; IRB, institutional review board; RL, record linkage.
Patient stakeholder engagement efforts suggested that rules governing patient data
use—along with data privacy and security—were among their top concerns. Thus, we incorporated
messaging around the rules and processes that govern patient data use for research
into patient communication materials. Additionally, patient stakeholder engagement
efforts revealed concerns about the initial pacing of the videos—including the use
of up-tempo music, which overwhelmed patients, with one describing it as creating
“a sense of panic.” Thus, in subsequent iterations, we slowed the pace of the videos
to allow viewers more time to digest the information presented. We also made adaptations
to meet the needs of viewers who might have vision or hearing impairments by (1) adding
subtitles and (2) lowering the volume of the music to make the voiceover more easily
heard.
Patient stakeholders found the final drafts of the videos to be clear and well-structured,
noting that complex information was presented systematically so that viewers had the
needed knowledge and familiarity before moving onto the next topic. Stakeholders felt
the videos would be well suited for use as part of the informed consent process, especially
for studies with a relatively small number of participants. They also suggested these
videos could be used to generate more universal support for and confidence in health
care research.
Customer Discovery and Value Proposition Design Insights
Results of the customer discovery interviews revealed jobs to be done, the principal
pains and gains, and metrics for success for each of three customer segments. [Table 2 ] presents insights from the customer discovery process that were used to inform value
proposition design. Value proposition statements focused on CURL's ability to standardize
the process for sharing data and to establish best practices for RL ([Table 3 ]).
Table 2
Jobs, pains, gains, and metrics of success identified in customer discovery
Main themes
Quotes
Jobs
Data managers
Principally perform work surrounding health data engineering, analytics, delivery,
and reporting, as well as software and systems development
“I manage the analytics department that does majority of linkages and do some of the
linkages myself as well. I oversee projects and staff that do the hands-on programming
to complete linkage.”
Researchers
Work involves using health data to identify eligible patients, developing clinical
and institutional informatics, and/or designing interventional studies to improve
patient health
“I am often integrating solutions for how health data are generated, and then I also
collect and use the data to look at outcomes. I am often working with minimum two
sources of data and linking the data in a de-identified way.”
Leadership and regulatory officials
Responsible for managing clinical trial agreements and data requests and delivery;
overseeing and providing infrastructure for HIPAA compliance; data governance and
surveillance; and/or addressing regulatory and legal issues surrounding data privacy
“We enrich datasets using multiple data sources. These datasets are set up primarily
for research. We provide secure analytics infrastructure preconfigured for HIPAA compliance
standards.”
Pains
Data managers
● Current process for RL is inefficient and time-consuming
● Challenges when requesting others to pull data in the format needed
● Taking steps to ensure that they are not giving out bad information (such as false
positives and false negatives)
“When we work with large datasets, we are going to introduce some false positives
or false negatives. We don't have the capacity to review everything. We have to be
open and honest about our assessment of the quality.”
Researchers
● Establishing data-sharing partnerships and contracts (data use agreements [DUA],
business associate agreements [BAA], nondisclosure agreements); navigating legal hurdles
● Communicating privacy and security methods surrounding data sharing (e.g., IRB protocols)
● Determining the most viable RL approach given their research question, resources,
and expertise
“The technical side of data sharing is easy; the governance side is hard.”
Sharing encrypted personally identifiable information (PII) is the same as sharing
PII in the view of a privacy/compliance officer (sentiment of one researcher)
Leadership and regulatory officials
● Understanding the process and how data security and privacy would be protected and
conform to HIPAA and other policies
● Require assurances that data sharing is “not opening any back doors”
“Dealing with the health care culture around the paranoia of use of health data across
institutional boundaries and getting executive support is key. Do not underestimate
the security and compliance efforts; it is much more effort than you think it's going
to be.”
Gains
Data managers
● Taking steps to ensure that they are not giving out bad information (such as false
positives and false negatives) is important
“Advances in probabilistic matching and solid validated metrics of how certain you
are that could be portrayed to clinicians would be helpful. It would be useful to
give them a number or percentage of certainty. Standard rules are needed; it's currently
very ad hoc.”
Researchers
● Establishing data-sharing partnerships and contracts (DUA, BAA, nondisclosure agreements);
navigating legal hurdles
● Communicating privacy and security methods surrounding data sharing (e.g., IRB protocols)
● Determining which RL approach would be most viable given their research question,
resources, and expertise available to them
“The goal is to alleviate concerns one might have about are we actually looking at
data the correct way? Are we actually looking at the thing we're trying to look at
with the data that we have? This would improve the validity of research and the conclusions
we make.”
Leadership and regulatory officials
● Understanding the process and how data security and privacy would be protected and
conform to HIPAA and other relevant policies
● Require assurances data sharing is “not opening any back doors”
“When I know that there are technical safeguards in place, because humans make mistakes.
Computers don't unless we've programmed them wrong. When there are technical safeguards
in place, that makes a privacy officer comfortable.”
Metrics of success
Data managers
● Ability to complete RL requests in a timely manner
● Confidence they are not giving out bad information (such as false positives and
false negatives)
“Success is timeliness in our ability to do linkages, including timeliness of putting
in data-sharing agreements.”
Researchers
● Confidence in navigating the data-sharing process
● Developing “well-trodden paths” for data sharing across universities and outside
organizations
“Success is getting quality data. This means being able to get all the data elements
you want—this is the challenging part—and low levels of missing data.”
Leadership and regulatory officials
● Research protocols that use CURL clearly demonstrate compliance with data privacy
regulations
● Making it easy for leadership to say “yes” to data-sharing/RL research
“Probabilistic linkage is important to us. If CURL helps us keep up with the greatest
and latest tech, that would be beneficial for us. We currently don't have the constant
improvement that comes from a managed service like CURL.”
Abbreviations: CURL, The University of Colorado Record Linkage; IRB, institutional
review board; HIPPA, Health Insurance Portability and Accountability Act; RL, record
linkage.
Table 3
Value propositions
For investigators that need to perform record linkage in their research, CURL provides
an integrated solution that enables them to perform data normalization, customization,
obfuscation, and linkage better than custom, manually coded solutions (which require
developing multiple processes and employing multiple tools) providing the benefit
of a more efficient method for conducting record linkage with high-volume data
For data managers that need to perform record linkage tasks, CURL provides user-friendly,
built-in linkage solutions and standard data specifications to assist the data linkage
process, enabling them to perform data normalization, customization, obfuscation,
and linkage better than custom, manually coded solutions (which require developing
multiple processes and employing multiple tools), providing the benefit of accuracy,
flexibility, efficiency, and data security
For institutional leadership and regulatory officials that need to ensure data privacy
and security standards are met, CURL provides an IRB-approved solution that meets
the federal regulatory requirements for human subjects research and data use that
enables them to protect their institution from fiduciary risks associated with protected
health information (PHI) data breaches better than resource-intensive manual data
and protocol review processes, providing the benefit of state-of-the-art data security
and the option for privacy-preserving data sharing
Abbreviations: CURL, The University of Colorado Record Linkage; IRB, institutional
review board.
Customer discovery revealed several features to incorporate into the RL dissemination
package. First, investigators who will incorporate RL into their protocols need guidance
on how to determine which RL approach would be most viable given their research question,
resources, and the expertise available to them, as well as data access permissions
and constraints. Second, regulatory and security personnel who make decisions about
allowing RL to be conducted need to fully understand the process and how the data
security and privacy will be protected and conform to HIPAA and other relevant policies.
Third, in terms of efficiency and quality of linkage and resulting datasets, analysts
and data managers who ultimately perform RLs need to see the benefits of using novel
RL methods rather than the manual, custom processes they have traditionally used.
Investigators needed communication materials to facilitate the research development
process for those new to conducting research using RL methods and for investigators
more experienced with RL who want greater clarity on how to integrate RL more comprehensively
in their work. Regulatory personnel value standard language for IRBs and data owners,
highlighting how privacy-preserving RL methods protect data privacy and ownership.
Finally, for those performing RL, dissemination of the findings from this project
should emphasize improvements in data quality and efficiency. These insights informed
the development of the RL Planning Workbook.
Record Linkage Planning Workbook
The purpose of the RL Planning Workbook is to inform the integration of RL methods
into research projects using CURL ([Table 1 ]). The workbook guides users through the steps to iteratively plan their RL project
([Fig. 4 ]). Each stage incorporates key questions—along with definitions and explanation of
RL concepts—to help the user ensure they address the decisions critical to conducting
research. The workbook is meant to be used in conjunction with other research team
members, including statisticians or analysts who will conduct RL tasks and related
analyses.
Town hall participants found the RL Planning Workbook useful for identifying solutions
and navigating the numerous parameters and restrictions of datasets. They believe
the workbook could be used to prepare investigators for productive consultations and
collaboration with RL experts—which investigators expect they will need to successfully
use RL in their research.
Discussion
A dissemination planning process involving patient engagement, customer discovery,
and UX design methods yielded an RL dissemination package that met the needs of audience
segments, including patients engaged in RL research and researchers, data managers,
leadership, and regulatory officials. For patient stakeholders, animated videos create
a shared understanding in a format that is accessible and able to be readily distributed
via multiple channels (e.g., email, social media). To adopt RL methods, researchers
need guidance and resources to support RL integration into research design. The RL
dissemination package provides messaging with this guidance and points researchers
to resources where needed. It also provides a common language to help researchers
communicate with data managers, leadership, and regulatory officials. The RL workbook
provides data managers—who need tools and metrics for assuring the validity of RL
results—a pathway for navigating research design and analytic plans with their researcher
counterparts. The packaging choice of a workbook allows users to interact with key
messages and record their own RL plan. It also can be shared and used online (as a
PDF) or printed to enhance distribution. Together the workbook and videos detail the
methods for ensuring data privacy and security to meet the information needs of leadership
and regulatory officials.
This work builds on prior research through application of a dissemination planning
framework[15 ] and multiple stakeholder engagement in informatics research.[21 ]
[43 ] Engaging patients and other stakeholders throughout the design process was valuable
in designing effective messages and dissemination strategies for CURL. We demonstrate
the value of applying patient engagement methods[31 ]; customer discovery and value proposition design methods[27 ]; and UX design[28 ]
[29 ] to dissemination planning by identifying the intersection of stakeholder needs and
the CURL platform's effectiveness at meeting those needs. For example, although our
initial plans for the patient engagement videos focused on explaining how and why
RL is used for health care research, feedback from patient stakeholders demonstrated
a high level of concern about data privacy and security. Therefore, we developed one
video that primarily focused on explaining the rules and regulations surrounding sharing
and using patient data for research.
Limitations and Future Research
Limitations and Future Research
Included in the limitations of this work is that a relatively small set of perspectives
informed RL dissemination package design. Next steps for this work are to evaluate
the impact of the RL dissemination package on adoption and use of RL methods.
Additionally, as informatics tools such as CURL necessarily require iterative development
to keep pace with evolving technology, subsequent efforts should be made to update
dissemination package materials to reflect such changes.
Conclusion
Guided by the dissemination planning framework, the RL dissemination package was developed
to meet the needs of diverse stakeholders. By using an iterative approach that incorporated
user feedback throughout the development process, we were able to produce materials
that fit the context of PCOR that employs RL methods. This work advances the science
of patient engagement in informatics research by demonstrating the utility of engaging
patients in RL methods dissemination design process to establish such methods as acceptable,
appropriate, and useful.
Clinical Relevance Statement
Clinical Relevance Statement
PCOR and other types of clinical and translational research can be advanced through
the use of novel RL methods, but investigators need resources to support the integration
of these methods into their work. RL dissemination provides the resources necessary
to facilitate the use of RL methods to enhance clinical outcomes research—ultimately,
improving the translation of clinical outcomes data to health care practice. Patient
engagement in the design process helps to ensure methods are developed and used with
patient perspectives in mind. Furthermore, patient stakeholders engaged in research
using RL methods may benefit from simple explainer videos to prepare them for this
role.
Multiple-Choice Questions
Multiple-Choice Questions
Value proposition design methods include the identification of potential customers'_____?
Correct Answer : The correct answer is option a. Value proposition design focuses on product alignment
with creating gains and relieving pains for customers.
How did patient stakeholder engagement improve the value of the RL Dissemination Package
in this study?
Patient stakeholders generated ideas for how to best market CURL to researchers
Patient stakeholders submitted their data as test cases for using CURL
Patient stakeholders shared their concerns about the use of patient data and contributed
ideas as to how best to communicate the value of using patient data for research
Patient stakeholders refused to support this work
Correct Answer : The correct answer is option c. As promoting patient engagement in RL research was
a key aim of this work, it was important to identify strategies for communicating
with patient stakeholders about the methods and implications of RL.
