Coverage of Physical Therapy Assessments in the Observational Medical Outcomes Partnership Common Data Model

Abstract

Background High-value care aims to enhance meaningful patient outcomes while reducing costs and is accelerated by curating data across health care systems through common data models (CDMs), such as Observational Medical Outcomes Partnership (OMOP). Meaningful patient outcomes, such as physical function, must be included in these CDMs. However, the extent to which physical therapy assessments are covered in the OMOP CDM is unclear.

Objective This study aimed to examine the extent to which physical therapy assessments used in neurologic and orthopaedic conditions are in the OMOP CDM.

Methods After identifying assessments, two reviewer teams independently mapped the neurologic and orthopaedic assessments into the OMOP CDM. Agreement within the reviewer team was assessed by the number of assessments mapped by both reviewers, one reviewer but not the other, or neither reviewer. The reviewer teams then reconciled disagreements, after which agreement and the average number of concept ID numbers per assessment were assessed.

Results Of the 81 neurologic assessments, 48.1% (39/81) were initially mapped by both reviewers, 9.9% (8/81) were mapped by one reviewer but not the other, and 42% (34/81) were unmapped. After reconciliation, 46.9% (38/81) were mapped by both reviewers and 53.1% (43/81) were unmapped. Of the 79 orthopaedic assessments, 46.8% (37/79) were initially mapped by both reviewers, 12.7% (10/79) were mapped by one reviewer but not the other, and 48.1% (38/79) were unmapped. After reconciliation, 48.1% (38/79) were mapped by both reviewers and 51.9% (41/79) were unmapped. Most assessments that were mapped had more than one concept ID number (2.2 ± 1.3 and 4.3 ± 4.4 concept IDs per neurologic and orthopaedic assessment, respectively).

Conclusion The OMOP CDM includes some physical therapy assessments recommended for use in neurologic and orthopaedic conditions but many have multiple concept IDs. Including more functional assessments in the OMOP CDM and creating guidelines for mapping would improve our ability to include functional data in large datasets.

Keywords

OMOP Common Data Model - physical therapy - function - standards

Background and Significance

The United States spends twice as much annually per person on health care compared to other high-income countries, yet we obtain worse health outcomes, such as shorter life expectancy and lower quality of life.[1] [2] This necessitates a shift towards high-value health care, where meaningful patient outcomes are improved while lowering the costs associated with those outcomes (i.e., better outcomes per dollar spent).[3] [4] [5] Large real-world data about interventions, outcomes, and costs are needed to move towards high-value care because they allow us to identify what interventions result in the best outcomes at the lowest cost. The outcomes needed for value-based care initiatives must be meaningful to patients.[3] [4] [5] [6] While there are a number of important outcomes, physical function is particularly important because it is salient across all medical diagnoses,[3] [4] [5] [6] and therefore, it is essential that databases used for value-based care initiatives include assessments of physical function. Since physical function is influenced by factors other than physical ability (e.g., cognition, depression, self-efficacy), we will refer to physical function simply as “function” to reflect this broader perspective.

The widespread adoption of electronic medical records (EMRs) has facilitated the development of these types of real-world databases. These real-world databases are even more useful for value-based care initiatives when data are aggregated across health care systems. Unfortunately, the same metric, such as a functional assessment, is frequently referred to differently across health care systems ([Table 1], Health care System 1 and 2 columns), presenting a barrier to aggregating data across systems. Further complicating this issue is that important functional assessments are often documented in unstructured data fields, leaving these data with no structured reference in the EMR.[7] To overcome this barrier, a number of common data models (CDMs) have been developed (e.g., Sentinel, the National Patient-Centered Clinical Research Network, Informatics for Integrating Biology in the Bedside, and Observational Medical Outcomes Partnership [OMOP]).[8] CDMs provide a standard set of terms (i.e., vocabulary) and structure so that data from different EMRs can be harmonized ([Table 1], Harmonized Data column) and then aggregated.[9] [10] [11] Of these CDMs, the OMOP CDM[12] [13] has gained popularity due to its broad coverage of medical data elements, flexibility and simplicity, applicability to multiple languages, and robust open-source tools.[10] [14] [15] [16] [17] [18] [19]

Table 1 Demonstration of common data models to harmonize three commonly used assessments of function

Functional assessment	Health care System 1		Health care System 2		Harmonized data[a]
	Terminology	Location	Terminology	Location	Terminology	Location
6-minute walk test	304700083	derived_flowsheet	3042800623	clarity_flowsheet	40766814	Measurement
AM-PAC Basic Mobility 6 Click	30441058001	derived_flowsheet	3042801216	clarity_flowsheet	4165295	Measurement
Disabilities of the Arm, Shoulder And Hand	3047712991	derived_flowsheet	Unstructured data	Clinical notes	601842	Measurement

Abbreviation: AM-PAC, Activity Measure for Post-Acute Care.

^a These are the codes from the Observational Medical Outcomes Partnership Common Data Model; however, the information in this table is intended to demonstrate the importance of data harmonization using common data models. As detailed in the manuscript and shown in [Supplementary Table S1] (available in the online version), these are not the codes that we recommend for these functional measures.

To leverage the OMOP CDM, health care systems must map their data into that CDM through an extract–transform–load (ETL) process. In this process, key data elements are first identified or extracted. These key data elements are then mapped into the OMOP CDM via a code map. Finally, the data from the EMR are loaded into the OMOP CDM and stored for subsequent use. Thus, the utility of the OMOP CDM for creating robust datasets that can be used for high-value care is dependent on how well data elements are represented in the OMOP CDM. As a result, many medical fields have examined the extent to which key metrics in their field are included in the OMOP CDM.[15] [16] [20] [21] However, despite the importance of function to patients and to value-based care, the extent to which key functional assessments are included in the OMOP CDM has not been examined and, therefore, these important assessments are rarely included in large, real-world datasets.

Objectives

Our purpose was to determine the extent to which physical therapy assessments that are recommended for use in neurologic and orthopaedic patient populations are contained within the OMOP CDM. We selected these two patient groups because they represent a large portion of patients seen in rehabilitation settings and of health care spending in the United States.[22] [23] We will specifically examine assessments used by physical therapists because they are experts in physical function, and therefore, routinely measure physical function and the other functional domains that influence it. We included assessments of function that are recommended in clinical practice guidelines (CPGs) from the American Physical Therapy Association. We hypothesized that (1) there would be good agreement between reviewers related to the mapping of the assessments and (2) less than half of the selected assessments would be included in the OMOP CDM. This work serves as a first step to ensure that functional assessments can be harmonized across health care systems and included in databases that facilitate our shift towards high-value health care.

Methods

This work was determined to not involve human subjects by the University of Utah IRB. This manuscript has been posted as a preprint on the medRxiv server.[24]

Assessment and Reviewer Selection

Because there are numerous functional assessments used when treating individuals with neurologic and orthopaedic conditions, we used CPGs to identify assessments with strong psychometric properties and with strong recommendations for use in clinical practice in these patient groups. Thus, the assessments that we examined should be widely used by organizations following best practice guidelines. Two authors with expertise in neurologic rehabilitation selected the neurologic assessments for inclusion. Together these authors have over 30 years of clinical and research experience with neurologic rehabilitation (H.A.H. with 19 years of experience and M.A.F. with 11 years of experience). They used the American Physical Therapy Association Academy of Neurologic Physical Therapy's Evidence Database to Guide Effectiveness (EDGE) documents for individuals with multiple sclerosis,[25] Parkinson's disease,[26] spinal cord injury,[27] stroke,[28] traumatic brain injury,[29] and vestibular disorders[30] to select the functional assessments included. These documents serve as CPGs by providing recommendations about which functional assessments should be used in each specific patient population based on a thorough review of the assessments' psychometric properties (i.e., validity, reliability) by content matter experts. Within each EDGE document, the functional assessments are labeled as “highly recommended,” “recommended,” “unable to recommend,” or “not recommended.” They also make recommendations about which functional assessments entry-level physical therapy practitioners should learn to administer or be exposed to. We included assessments that were either (1) highly recommended or recommended in more than one neurologic patient population or (2) recommended for entry-level physical therapy practitioners to learn to administer.

Two authors with expertise in orthopaedic rehabilitation selected the orthopaedic assessments for inclusion. Combined, these authors have over 49 years of experience in orthopaedic rehabilitation (A.T. with 28 years of experience and J.M. with 21 years of experience). Orthopaedic functional assessments were identified from the American Physical Therapy Association's Academy of Orthopaedic Physical Therapy's CPGs for Achilles tendinopathy,[31] ankle instability,[32] carpal tunnel syndrome,[33] heel pain,[34] anterior cruciate ligament (ACL) injury prevention,[35] knee ligamentous instability,[36] meniscal and cartilage lesions,[37] anterior knee pain,[38] hamstring injury,[39] hip fracture,[40] hip osteoarthritis,[41] non-arthritic hip pain,[42] lateral elbow pain,[43] low back pain,[44] [45] concussion,[46] neck pain,[47] occupational injuries,[48] pelvic girdle pain,[49] and adhesive capsulitis.[50] Recommendations are graded as “A: strong evidence,” “B: moderate evidence,” “C: weak evidence,” “D: conflicting evidence,” “E: theoretical foundation evidence,” or “F: expert opinion.” We included all functional assessments that were graded A, B, or C, except when selecting assessments for individuals with concussions. All functional assessments in this patient group were graded as “F: expert opinion.” We decided to include these, however, as they were the only measures recommended for this patient group.

If an assessment was identified in both neurologic and orthopaedic CPGs, only one team of reviewers completed the mapping process described below. These assessments are identified in [Supplementary Table S1] (available in the online version). For each of the assessments examined, we mapped the total score into the OMOP CDM, rather than each individual item of the assessment. This decision was made because typically only the total score on these assessments is documented during clinical practice; thus, the mapping information for the total score would be most useful to the medical community.

Initial Mapping

After identifying the functional assessments, we followed a similar process to that of previous work to assess their content coverage in the OMOP CDM.[15] [16] [20] Two reviewers independently mapped the neurologic (M.A.F. and H.A.H.) and the orthopaedic (P.H. with 11 years of experience and A.T.) assessments into the OMOP CDM using Usagi.[51] Usagi is one of the many open-source tools developed by Observational Health Data Sciences and Informatics (OHDSI) to support the use of the OMOP CDM. Usagi uses term similarity to provide an automated suggestion for the standard OMOP CDM vocabulary term, called a concept identification (ID) number, for each functional assessment.[51] Two reviewers independently reviewed the suggested concept ID number provided by Usagi to determine if (1) the concept ID number was correct and (2) there were other concept ID numbers onto which the assessment should be mapped. For each functional assessment, reviewers selected all appropriate concept ID numbers that were standard concepts, as opposed to non-standard, in the OMOP CDM.

Metrics of Initial Agreement

After the initial mapping, each reviewer exported their mappings from Usagi for analysis. Two primary metrics were calculated separately for the neurologic assessments and for the orthopaedic assessments using R (version 4.2.1).[52] The first metric was assessment agreement, in which each assessment was labeled as being (1) mapped by both reviewers, (2) mapped by reviewer A but not reviewer B, (3) mapped by reviewer B, but not reviewer A, or (4) unmapped by both reviewers. The second metric was the concept ID number agreement. This metric looked specifically at the assessments that were mapped by both reviewers to determine if the two reviewers mapped each assessment to the same concept ID number. Each concept ID number was categorized into one of three categories: (1) mapped by both reviewers, (2) mapped by reviewer A but not reviewer B, or (3) mapped by reviewer B but not reviewer A. There was no unmapped category for this metric as it only applies to assessments that were mapped to at least one concept ID number.

Reconciliation

After the analysis of the initial mapping, each reviewer team reviewed the following: (1) assessments that were unmapped by both reviewers, (2) assessments that were mapped by one but not the other reviewer, and (3) concept ID numbers that were mapped by one but not the other reviewer. The reviewer teams reconciled any differences in the mappings in these three areas. Following reconciliation, the same metrics described in the “Metrics of Initial Agreement” section were calculated on the reconciled mappings.

Statistical Analysis

For assessment agreement and concept ID number agreement, we report (1) the portion of metrics in each category, (2) Cohen's kappa,[53] (3) the percent overall agreement, and (4) Gwet's agreement coefficient.[54] [55] We calculate Gwet's agreement coefficient because there is a well-documented paradox of having a high percent overall agreement with a low kappa when the observations are imbalanced as expected in the analysis of concept ID number agreement (i.e., we expect no observations in the unmapped category).[54] [56] [57] [58] The Gwet's agreement coefficient is interpreted similarly to Cohen's kappa, with >0.81 being very good agreement, 0.61 to 0.8 being good agreement, 0.41 to 0.6 being moderate agreement, 0.21 to 0.4 being fair agreement, and less than or equal to 0.2 being poor agreement.[54] We also present Sankey plots to visualize changes in the mapping category from the initial mapping to reconciliation for each of these metrics. Lastly, we provide descriptive statistics of the number of concept ID numbers per assessment after reconciliation.

Results

Assessment Agreement

We identified 160 unique functional assessments, 81 neurologic and 79 orthopaedic, to include in our examination. During the initial mapping for the neurologic assessments, 48.1% (39/81) of assessments were mapped by both reviewers, 9.9% (8/81) were mapped by one but not the other reviewer, and 42% (34/81) were unmapped by both reviewers ([Fig. 1A]). This resulted in a 90.1% overall agreement with a Cohen's kappa and Gwet's agreement coefficient of 0.80, indicating good agreement. After reconciliation, both reviewers mapped 46.9% of assessments (38/81), while the remaining 53.1% (43/81) were determined to be unmapped ([Fig. 1A]). This resulted in a 100% overall agreement and a Cohen's kappa and Gwet's agreement coefficient of 1, indicating perfect agreement.

Fig. 1 Sankey plot of (A) neurologic and (B) orthopaedic assessments after the initial mapping and after reconciliation.

During the initial mapping of the orthopaedic assessments, 46.8% (37/79) of the assessments were mapped by both reviewers, 12.7% (10/79) were mapped by one reviewer but not the other, and 48.1% (38/79) were unmapped by both reviewers ([Fig. 1B]). This resulted in 87.3% overall agreement with a Cohen's kappa and Gwet's agreement coefficient of 0.74 and 0.75, respectively, indicating good agreement. After reconciliation, both reviewers mapped 48.1% of assessments (38/79), while the remaining 51.9% (41/79) were determined to be unmapped ([Fig. 1B]). This resulted in a 100% overall agreement with Cohen's kappa and Gwet's agreement coefficient of 1, indicating perfect agreement.

Concept Identification Number Agreement

For the neurologic assessments that were initially mapped by both reviewers, there were 233 unique concept ID numbers identified. Of these, 33.5% (78/233) were the same between both reviewers, while 66.6% (155/233) were mapped by only one reviewer ([Fig. 2A]). This resulted in a 33.5% overall agreement with Cohen's kappa and Gwet's agreement coefficient indicated poor agreement (−0.8 and −0.2, respectively). During reconciliation, it was determined that this poor agreement was primarily due to one reviewer selecting individual items for two large testing batteries in error, as we agreed to focus only on the total score for this work. Based on our goal to map only the total score, we determined these individual item-level concept ID numbers to be incorrect during reconciliation; thus, after completing the reconciliation process, 36.0% (84/233) concept ID numbers were determined to be correct, while 64.0% (149/233) were determined to be incorrect ([Fig. 2A]). After reconciliation, there was a 100% overall agreement with a Cohen's kappa and Gwet's agreement coefficient of 1, indicating perfect agreement. For the 38 neurologic assessments that were mapped, there were 84 unique concept ID numbers. Ten neurologic assessments had a single unique concept ID number with an average number of concept ID numbers per the neurologic assessment of 2.2 (1.3) and a median of 2 ([Fig. 3A]). Importantly, many functional assessments had concept ID numbers in multiple domains of the OMOP CDM, typically in the measurement and observation domain ([Supplementary Table S1] [available in the online version]).

Fig. 2 Sankey plots of (A) neurologic and (B) orthopaedic concept ID numbers after the initial mapping and after reconciliation. ID, identification.

Fig. 3 Number of concept ID numbers per (A) neurologic and (B) orthopaedic assessments that were mapped by both reviewers after reconciliation. ID, identification.

For the orthopaedic assessments that were initially mapped, there were 196 unique concept ID numbers identified. Of these, 77.0% (151/196) were the same between both reviewers, while 23.0% (45/196) were mapped by only one reviewer ([Fig. 2B]). The overall percent agreement was 77.0%. Cohen's kappa and Gwet's agreement coefficients were −0.09 and 0.71, respectively. This demonstrates the well-documented paradox that can occur with Cohen's kappa when observations are imbalanced,[54] [56] [57] [58] necessitating the inclusion of Gwet's agreement.[55] After reconciliation, 83.2% (163/196) concept ID numbers were determined to be correct, while 16.8% (33/196) were determined to be incorrect ([Fig. 2B]). After reconciliation, overall percent agreement was 100% with a Cohen's kappa and Gwet's agreement coefficient of 1, indicating perfect agreement. The final mapping for orthopaedic assessments resulted in 163 unique concept ID numbers for the 38 orthopaedic measures that were mapped. Only four orthopaedic assessments had a single unique concept ID number. The average number of concept ID numbers per orthopaedic assessment was 4.3 (4.4) with a median of 2.5 ([Fig. 3B]). Many of the assessments with a large number of appropriate concept ID numbers were the Patient-Reported Outcomes Measurement Information System measures as there are numerous versions of these assessments in the OMOP CDM ([Supplementary Table S1] [available in the online version]). As with the neurologic assessments, many measures had at least one concept ID number in the measurement and observation domain of the OMOP CDM.

Discussion

In this study, we examined the extent to which functional assessments are included in the OMOP CDM specifically for neurologic and orthopaedic conditions. Our hypotheses were supported such that there was high agreement between reviewers and that less than 50% of assessments were included in the OMOP CDM. We also found that there were multiple standard OMOP CDM concept ID numbers for most functional assessments.

We found that 46.9% and 48.1% of the neurologic and orthopaedic assessments, respectively, were included in the OMOP CDM after reconciliation. The optimistic conclusion from this finding is that almost half of the assessments that are recommended in CPGs to measure function in these two broad patient groups are already in the OMOP CDM; thus, because of the importance of these constructs to all patients, these assessments should be included in datasets that leverage the OMOP CDM. Of note, although we selected these assessments based on CPGs for neurologic and orthopaedic conditions, many of the assessments are valid in other patient populations and, therefore, can be used across an even broader group of patients. For example, the European Quality of Life was included as an orthopaedic assessment. This assessment, however, has been validated in neurologic[59] [60] and general populations[61] as well. Similarly, the 10-meter walk test, which is used to calculate gait speed, was included as a neurologic assessment; however, this metric has been called the “sixth vital sign”[62] and is a critical metric across all patient diagnoses. This highlights that the mapping of functional assessments has significance across medical diagnoses and that the OMOP CDM supports the harmonization of some of these assessments.

The more pessimistic interpretation of our findings is that over half of the assessments we selected based on CPGs from the American Physical Therapy Association are not included in the OMOP CDM. Some of the assessments not included in the OMOP CDM are particularly troublesome because of their importance in guiding treatment and understanding patient outcomes. For example, the Functional Gait Assessment is a predictor of falls in neurologic patient groups,[63] [64] [65] and is a recommended functional assessment in all individuals with neurologic diagnoses.[66] This tool is also reliable, valid, and predictive of falls in geriatric populations.[67] [68] Yet, this assessment is not mapped into the OMOP CDM. Similarly, the Fear Avoidance Beliefs Questionnaire is recommended with strong evidence in three CPGs[45] [48] [69] and with weak evidence in one other CPG.[32] Further, fear avoidance is known to significantly affect the quality of life and function,[70] [71] yet it is not mapped into the OMOP CDM. Our findings and these specific examples demonstrate the need for expanding the OMOP CDM vocabulary to include more assessments of function.

A significant limitation in the coverage of functional assessments in the OMOP CDM that we identified is that many functional assessments had multiple standard concept ID numbers. The lack of a unique concept ID number is problematic as individuals undertaking an ETL at one site may select a different concept ID number than the individual conducting the ETL process at a different site. This challenge limits our ability to harmonize these assessments across health care systems with the OMOP CDM. Agreement and guidelines about which concept ID number should be used from functional assessments are needed. Providing these recommendations is beyond the scope of this work and will require collaboration across the OHDSI community. Based on this work, there are two primary areas that these guidelines should address. First, there should be clarification regarding onto which domain of the OMOP CDM the functional assessment should be mapped. This clarification is needed because many assessments had a single unique concept ID number when constrained to a single domain of the OMOP CDM. For example, we identified two appropriate concept ID numbers for the Short Physical Performance Battery, which is a commonly used functional assessment in orthopaedic and geriatric patient groups; one of the concept ID numbers was in the measurement domain, while the other was in the observation domain ([Supplementary Table S1] [available in the online version]). This situation occurred for many functional assessments. Clarification on which domain is correct would help mitigate the challenge with multiple concept ID numbers. Secondly, guidelines should address assessments that have multiple concept ID numbers within the same domain. For example, the 6-minute walk test is a commonly used assessment of walking endurance and has three vocabulary terminologies in the OMOP CDM ([Supplementary Table S1] [available in the online version]). This could be addressed by providing clear recommendations for which concept ID number to use or by revising the OMOP CDM vocabulary to minimize redundancy within the CDM.

Despite the current limitations of the OMOP CDM for functional assessments, we have several recommendations to move toward integrating functional assessments into value-based care initiatives. First, we do not believe that health systems should wait for improvements to the OMOP CDM before mapping functional data into it. Instead, we recommend that health care systems prioritize functional assessments that map to a single OMOP CDM concept ID. We also recommend that when there are multiple concept IDs, informaticians, rehabilitation professionals, and research teams should partner to select the concept ID that is most appropriate. Critically, this decision must be well-documented to facilitate harmonization with other data sources. Lastly, if a functional assessment is not in the OMOP CDM, there are processes for creating your own standard terminology to ensure the completeness of the mapped data that systems should use for these functional assessments.[72] [73] Our second recommendation is that rehabilitation experts (i.e., the experts in these functional assessments) must take an active role in improving the coverage of functional assessments in the OMOP CDM. Because the OMOP CDM is community-driven, there are avenues to do this. For example, there are OHDSI forums dedicated to discussions about standardized vocabulary.[74] [75] On these forums, users could post concerns about the ambiguity of the OMOP CDM or about omissions from the OMOP CDM. There are also OHDSI workgroups specifically for improving the OMOP CDM and for including rehabilitation in the OHDSI community. To ensure that functional assessments are included accurately and comprehensively within the OMOP CDM, rehabilitation professionals must engage in and lead these workgroups.

We used a manual process that was facilitated by Usagi to map the identified functional assessments. Although the process that we used is similar to that of other work[15] [20] and to that recommended by the Book of OHDSI,[13] there have been other, more automated approaches taken in the literature.[16] [76] For example, Usagi provides a “match score” to reflect how confident the tool is in the match. As a result, some have used the threshold of this match score to determine content mapping.[16] Although this approach decreases the time needed to create these mappings, there are concerns about the accuracy as data elements can have similar names and not be a correct match. Others have developed automatic algorithms and tools for their specific data type that map their data into the OMOP CDM more accurately than Usagi.[76] We did not take these approaches in this current work, resulting in our mapping being tedious. The tediousness of code mapping is common when conducting the ETL process but the time required to review mappings manually is a barrier to many organizations mapping their data into the OMOP CDM. We provide a table of the mappings that we found to minimize the time needed by other groups when transforming their functional assessments into the OMOP CDM ([Supplementary Table S1] [available in the online version]); however, a process that automatically and accurately maps other functional assessments into the OMOP CDM may increase the presence of this important data type in large scale, real-world databases. This potential should be explored while keeping in mind the potential tradeoff between generalizability, accuracy, and automaticity.

While the current work significantly contributes to our ability to use the OMOP CDM, there are several limitations. Our examination focused on functional assessments that are recommended in physical therapy CPGs. This ensured that we examined assessments with strong psychometric properties that are part of best clinical practice. However, it also led to two limitations. First, although physical therapists routinely measure functional domains that influence physical function (e.g., cognition, depression, self-efficacy), they are not experts in those domains. Future work should focus on mapping functional assessments used by other rehabilitation professionals, who are experts in those other domains, into the OMOP CDM. Second, some functional assessments may not be recommended in CPGs but still used commonly in clinical practice. These measures were not included in our examination. Due to the large number of functional assessments, this limitation is inevitable; however, it would be attenuated if there was a widely used standard battery of functional assessments. A final limitation is that we focused on two broad patient diagnosis categories: neurologic and orthopaedic. While these patient groups make up a significant portion of health care spending, there are numerous other diagnoses for which a similar examination should be performed. Importantly, however, many of the assessments included in this analysis can be and are used in other patient populations.

Conclusion

In this work, we found that the OMOP CDM includes a portion of functional assessments that are recommended for use in clinical practice with individuals with neurologic or orthopaedic conditions. While these findings are encouraging, many of the assessments that were mapped did not have a single unique term in the OMOP CDM. To include functional assessments in databases that allow us to understand and improve the value of health care, we must (1) ensure that the functional assessments that are already mapped into the OMOP CDM are included in the development of large databases, (2) work towards guidelines for the ETL process of functional assessments into the OMOP CDM, and (3) continue to expand the OMOP CDM such that it includes all key functional assessments.

Clinical Relevance Statement

The OMOP CDM is well-positioned to harmonize functional assessments in large-scale, real-world datasets based on its current content coverage of functional assessments. Informaticians, clinical teams, and researchers should use our results to increase the inclusion of these important measures in their data. However, it is also clear that we need to improve the OMOP CDM to cover functional assessments more completely and accurately, which will require rehabilitation professionals to actively engage with the OHDSI community.

Multiple-Choice Questions

1. Which of the following CDMs demonstrates the best flexibility and has a suite of open-source tools?

   • Informatics for Integrating Biology in the Bedside

   • Observational Medical Outcomes Partnership

   • National Patient-Centered Clinical Research Network

   • Sentinel Common Data Model

   Correct Answer: The correct answer is option b. OMOP demonstrates superior coverage of standard terminologies, flexibility, and open-source tools in comparison to other available CDMs.

2. Which of the following options represents the final step of concept mapping as completed by the two sets of authors?

   • Calculation of initial metric agreement

   • Reconciliation of differences in mapping

   • Identification of metrics for evaluation

   • Use of Usagi to link assessment items with concept ID number

   Correct Answer: The correct answer is option b. The final step of concept mapping in this paper was to review three categories: assessments that were unmapped by both reviewers, assessments that were mapped by one but not the other reviewer, and concept ID numbers that were mapped by one but not the other reviewer. After these three categories were reviewed, the authors were able to reconcile any differences identified in the mappings.

Conflict of Interest

None declared.

Protection of Human Subjects

No human subjects were involved in this project.

• References

• 1 Papanicolas I, Woskie LR, Jha AK. Health care spending in the United States and other high-income countries. JAMA 2018; 319 (10) 1024-1039
• 2 Gunja MZ, Gumas ED, Williams RD. II. . U.S. Health Care from a Global Perspective, 2022: Accelerating Spending, Worsening Outcomes. The Commonwealth Fund; 2023.
• 3 Jewell DV, Moore JD, Goldstein MS. Delivering the physical therapy value proposition: a call to action. Phys Ther 2013; 93 (01) 104-114
• 4 Porter ME. What is value in health care?. N Engl J Med 2010; 363 (26) 2477-2481
• 5 Teisberg E, Wallace S, O'Hara S. Defining and implementing value-based health care: a strategic framework. Acad Med 2020; 95 (05) 682-685
• 6 Baumhauer JF, Bozic KJ. Value-based healthcare: patient-reported outcomes in clinical decision making. Clin Orthop Relat Res 2016; 474 (06) 1375-1378
• 7 Yoo S, Yoon E, Boo D. et al. Transforming thyroid cancer diagnosis and staging information from unstructured reports to the Observational Medical Outcome Partnership Common Data Model. Appl Clin Inform 2022; 13 (03) 521-531
• 8 Weeks J, Pardee R. Learning to share health care data: a brief timeline of influential common data models and distributed health data networks in U.S. Health Care Research. EGEMS (Wash DC) 2019; 7 (01) 4
• 9 Voss EA, Makadia R, Matcho A. et al. Feasibility and utility of applications of the common data model to multiple, disparate observational health databases. J Am Med Inform Assoc 2015; 22 (03) 553-564
• 10 Garza M, Del Fiol G, Tenenbaum J, Walden A, Zozus MN. Evaluating common data models for use with a longitudinal community registry. J Biomed Inform 2016; 64: 333-341
• 11 Kent S, Burn E, Dawoud D. et al. Common problems, common data model solutions: evidence generation for health technology assessment. PharmacoEconomics 2021; 39 (03) 275-285
• 12 Hripcsak G, Duke JD, Shah NH. et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform 2015; 216: 574-578
• 13 Observational Health Data Sciences and Informatics. The Book of OHDSI. Accessed February 23, 2024 at: https://ohdsi.github.io/TheBookOfOhdsi/
• 14 Klann JG, Joss MAH, Embree K, Murphy SN. Data model harmonization for the All Of Us Research Program: transforming i2b2 data into the OMOP common data model. PLoS ONE 2019; 14 (02) e0212463
• 15 Cho S, Sin M, Tsapepas D. et al. Content coverage evaluation of the OMOP vocabulary on the transplant domain focusing on concepts relevant for kidney transplant outcomes analysis. Appl Clin Inform 2020; 11 (04) 650-658
• 16 Sathappan SMK, Jeon YS, Dang TK. et al. Transformation of electronic health records and questionnaire data to OMOP CDM: a feasibility study using SG_T2DM Dataset. Appl Clin Inform 2021; 12 (04) 757-767
• 17 Reinecke I, Zoch M, Reich C, Sedlmayr M, Bathelt F. The usage of OHDSI OMOP - a scoping review. Stud Health Technol Inform 2021; 283: 95-103
• 18 Lamer A, Depas N, Doutreligne M. et al. Transforming French electronic health records into the Observational Medical Outcome Partnership's Common Data Model: a feasibility study. Appl Clin Inform 2020; 11 (01) 13-22
• 19 Maier C, Lang L, Storf H. et al. Towards implementation of OMOP in a German University Hospital Consortium. Appl Clin Inform 2018; 9 (01) 54-61
• 20 Cai CX, Halfpenny W, Boland MV. et al. Advancing toward a common data model in ophthalmology: gap analysis of general eye examination concepts to Standard Observational Medical Outcomes Partnership (OMOP) concepts. Ophthalmol Sci 2023; 3 (04) 100391
• 21 Lamer A, Abou-Arab O, Bourgeois A. et al. Transforming anesthesia data into the Observational Medical Outcomes Partnership Common Data Model: development and usability study. J Med Internet Res 2021; 23 (10) e29259
• 22 Dieleman JL, Cao J, Chapin A. et al. US Health Care Spending by Payer and Health Condition, 1996-2016. JAMA 2020; 323 (09) 863-884
• 23 Feigin VL, Forouzanfar MH, Krishnamurthi R. et al; Global Burden of Diseases, Injuries, and Risk Factors Study 2010 (GBD 2010) and the GBD Stroke Experts Group. Global and regional burden of stroke during 1990-2010: findings from the Global Burden of Disease Study 2010. Lancet 2014; 383 (9913) 245-254
• 24 French MA, Hartman P, Hayes HA. et al. Examination of the coverage of functional assessments in the OMOP common data model. medRxiv 2024; ; 2024.2005.2003.24306841. Doi:
• 25 Potter K, Cohen ET, Allen DD. et al. Outcome measures for individuals with multiple sclerosis: recommendations from the American Physical Therapy Association Neurology Section Task Force. Phys Ther 2014; 94 (05) 593-608
• 26 Academy of Neurologic Physical Therapy; Parkinson Evidence Database to Guide Effectiveness. 2018. . Accessed May 1, 2024 at: https://www.neuropt.org/practice-resources/neurology-section-outcome-measures-recommendations/parkinson-disease
• 27 Kahn JH, Tappan R, Newman CP. et al. Outcome measure recommendations from the Spinal Cord Injury EDGE Task Force. Phys Ther 2016; 96 (11) 1832-1842
• 28 Sullivan JE, Crowner BE, Kluding PM. et al. Outcome measures for individuals with stroke: process and recommendations from the American Physical Therapy Association neurology section task force. Phys Ther 2013; 93 (10) 1383-1396
• 29 McCulloch KL, de Joya AL, Hays K. et al. Outcome measures for persons with moderate to severe traumatic brain injury: recommendations from the American Physical Therapy Association Academy of Neurologic Physical Therapy TBI EDGE Task Force. J Neurol Phys Ther 2016; 40 (04) 269-280
• 30 Academy of Neurologic Physical Therapy; Vestibular EDGE. 2018. Accessed May 1, 2024 at: https://www.neuropt.org/practice-resources/neurology-section-outcome-measures-recommendations/vestibular-disorders
• 31 Martin RL, Chimenti R, Cuddeford T. et al. Achilles pain, stiffness, and muscle power deficits: Midportion Achilles Tendinopathy Revision 2018. J Orthop Sports Phys Ther 2018; 48 (05) A1-A38
• 32 Martin RL, Davenport TE, Fraser JJ. et al. Ankle stability and movement coordination impairments: Lateral Ankle Ligament Sprains Revision 2021. J Orthop Sports Phys Ther 2021; 51 (04) CPG1-CPG80
• 33 Erickson M, Lawrence M, Jansen CWS, Coker D, Amadio P, Cleary C. Hand pain and sensory deficits: Carpal Tunnel Syndrome. J Orthop Sports Phys Ther 2019; 49 (05) CPG1-CPG85
• 34 Koc Jr TA, Bise CG, Neville C, Carreira D, Martin RL, McDonough CM. Heel pain - plantar fasciitis: Revision 2023. J Orthop Sports Phys Ther 2023; 53 (12) CPG1-CPG39
• 35 Arundale AJH, Bizzini M, Dix C. et al. Exercise-based knee and anterior cruciate ligament injury prevention. J Orthop Sports Phys Ther 2023; 53 (01) CPG1-CPG34
• 36 Logerstedt DS, Snyder-Mackler L, Ritter RC, Axe MJ, Godges JJ. Orthopaedic Section of the American Physical Therapist Association. Knee stability and movement coordination impairments: knee ligament sprain. J Orthop Sports Phys Ther 2010; 40 (04) A1-A37
• 37 Logerstedt DS, Scalzitti DA, Bennell KL. et al. Knee pain and mobility impairments: Meniscal and Articular Cartilage Lesions Revision 2018. J Orthop Sports Phys Ther 2018; 48 (02) A1-A50
• 38 Willy RW, Hoglund LT, Barton CJ. et al. Patellofemoral pain. J Orthop Sports Phys Ther 2019; 49 (09) CPG1-CPG95
• 39 Martin RL, Cibulka MT, Bolgla LA. et al. Hamstring strain injury in athletes. J Orthop Sports Phys Ther 2022; 52 (03) CPG1-CPG44
• 40 McDonough CM, Harris-Hayes M, Kristensen MT. et al. Physical therapy management of older adults with hip fracture. J Orthop Sports Phys Ther 2021; 51 (02) CPG1-CPG81
• 41 Cibulka MT, Bloom NJ, Enseki KR, Macdonald CW, Woehrle J, McDonough CM. Hip pain and mobility deficits-hip osteoarthritis: Revision 2017. J Orthop Sports Phys Ther 2017; 47 (06) A1-A37
• 42 Enseki KR, Bloom NJ, Harris-Hayes M. et al. Hip pain and movement dysfunction associated with nonarthritic hip joint pain: a revision. J Orthop Sports Phys Ther 2023; 53 (07) CPG1-CPG70
• 43 Lucado AM, Day JM, Vincent JI. et al. Lateral elbow pain and muscle function impairments. J Orthop Sports Phys Ther 2022; 52 (12) CPG1-CPG111
• 44 Delitto A, George SZ, Van Dillen L. et al; Orthopaedic Section of the American Physical Therapy Association. Low back pain. J Orthop Sports Phys Ther 2012; 42 (04) A1-A57
• 45 George SZ, Fritz JM, Silfies SP. et al. Interventions for the management of acute and chronic low back pain: Revision 2021. J Orthop Sports Phys Ther 2021; 51 (11) CPG1-CPG60
• 46 Quatman-Yates CC, Hunter-Giordano A, Shimamura KK. et al. Physical therapy evaluation and treatment after concussion/mild traumatic brain injury. J Orthop Sports Phys Ther 2020; 50 (04) CPG1-CPG73
• 47 Blanpied PR, Gross AR, Elliott JM. et al. Neck pain: Revision 2017. J Orthop Sports Phys Ther 2017; 47 (07) A1-A83
• 48 Daley D, Payne LP, Galper J. et al. Clinical guidance to optimize work participation after injury or illness: the role of physical therapists. J Orthop Sports Phys Ther 2021; 51 (08) CPG1-CPG102
• 49 Clinton S, Newell A, Downey P. et al. Pelvic girdle pain in the antepartum population: Physical Therapy Clinical Practice Guidelines Linked to the International Classification of Functioning, Disability, and Health From the Section on Women's Health and the Orthopaedic Section of the American Physical Therapy Association. J Womens Health Phys Therap 2017; 41: 102-125
• 50 Kelley MJ, Shaffer MA, Kuhn JE. et al. Shoulder pain and mobility deficits: adhesive capsulitis. J Orthop Sports Phys Ther 2013; 43 (05) A1-A31
• 51 OHDSI Usagi. USAGI. Accessed February 23, 2024 at: https://ohdsi.github.io/Usagi/
• 52 R Core Team. A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna, Austria: 2022
• 53 McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012; 22 (03) 276-282
• 54 Dettori JR, Norvell DC. Kappa and beyond: is there agreement?. Global Spine J 2020; 10 (04) 499-501
• 55 Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 2008; 61 (Pt 1): 29-48
• 56 Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990; 43 (06) 551-558
• 57 Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990; 43 (06) 543-549
• 58 Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med 2005; 37 (05) 360-363
• 59 Chen P, Lin KC, Liing RJ, Wu CY, Chen CL, Chang KC. Validity, responsiveness, and minimal clinically important difference of EQ-5D-5L in stroke patients undergoing rehabilitation. Qual Life Res 2016; 25 (06) 1585-1596
• 60 van Agt HM, Essink-Bot ML, Krabbe PF, Bonsel GJ. Test-retest reliability of health state valuations collected with the EuroQol questionnaire. Soc Sci Med 1994; 39 (11) 1537-1544
• 61 Brazier J, Jones N, Kind P. Testing the validity of the Euroqol and comparing it with the SF-36 health survey questionnaire. Qual Life Res 1993; 2 (03) 169-180
• 62 Fritz S, Lusardi M. White paper: “walking speed: the sixth vital sign”. J Geriatr Phys Ther 2009; 32 (02) 46-49
• 63 Foreman KB, Addison O, Kim HS, Dibble LE. Testing balance and fall risk in persons with Parkinson disease, an argument for ecologically valid testing. Parkinsonism Relat Disord 2011; 17 (03) 166-171
• 64 Lin JH, Hsu MJ, Hsu HW, Wu HC, Hsieh CL. Psychometric comparisons of 3 functional ambulation measures for patients with stroke. Stroke 2010; 41 (09) 2021-2025
• 65 Marchetti GF, Lin CC, Alghadir A, Whitney SL. Responsiveness and minimal detectable change of the dynamic gait index and functional gait index in persons with balance and vestibular disorders. J Neurol Phys Ther 2014; 38 (02) 119-124
• 66 Moore JL, Potter K, Blankshain K, Kaplan SL, OʼDwyer LC, Sullivan JE. A core set of outcome measures for adults with neurologic conditions undergoing rehabilitation: a clinical practice guideline. J Neurol Phys Ther 2018; 42 (03) 174-220
• 67 Beninato M, Fernandes A, Plummer LS. Minimal clinically important difference of the functional gait assessment in older adults. Phys Ther 2014; 94 (11) 1594-1603
• 68 Wrisley DM, Kumar NA. Functional gait assessment: concurrent, discriminative, and predictive validity in community-dwelling older adults. Phys Ther 2010; 90 (05) 761-773
• 69 Simonds AH, Abraham K, Spitznagle T. Clinical practice guidelines for pelvic girdle pain in the postpartum population. J Womens Pelvic Health Phys Ther 2022; 46: E1-E38
• 70 Baez SE, Hoch MC, Hoch JM. Psychological factors are associated with return to pre-injury levels of sport and physical activity after ACL reconstruction. Knee Surg Sports Traumatol Arthrosc 2020; 28 (02) 495-501
• 71 Burgess R, Mansell G, Bishop A, Lewis M, Hill J. Predictors of functional outcome in musculoskeletal healthcare: an umbrella review. Eur J Pain 2020; 24 (01) 51-70
• 72 Philofsky M, Olakpe U. Implementing and adopting a customized OMOP Common Data Model. Paper presented at: OHDSI Global Symposium. Virtual; 2021
• 73 Zhuk O, Klebanov G, Reich C. et al. Source vocabulary mapping, typical pitfalls, solutions and quality assurance. OHDSI Symposium. Virtual; 2020
• 74 GitHub. OHDSI Vocabulary-v5.0. 2024
• 75 OHDSI. OHDSI Forums.
• 76 Choi S, Joo HJ, Kim Y, Kim JH, Seok J. Conversion of automated 12-lead electrocardiogram interpretations to OMOP CDM vocabulary. Appl Clin Inform 2022; 13 (04) 880-890

Publication History

Received: 15 May 2024

Accepted: 21 August 2024

Accepted Manuscript online:
22 August 2024

Article published online:
27 November 2024