Subscribe to RSS
Download PDF
DOI: 10.1055/s-0044-1787647
Predicting Provider Workload Using Predicted Patient Risk Score and Social Determinants of Health in Primary Care Setting
Funding None.
- Abstract
- Background and Significance
- Objectives
- Methods
- Results
- Discussion
- Conclusion
- Clinical Relevance Statement
- Multiple-Choice Questions
- References
Abstract
Background Provider burnout due to workload is a significant concern in primary care settings. Workload for primary care providers encompasses both scheduled visit care and non–visit care interactions. These interactions are highly influenced by patients' health conditions or acuity, which can be measured by the Adjusted Clinical Group (ACG) score. However, new patients typically have minimal health information beyond social determinants of health (SDOH) to determine ACG score.
Objectives This study aims to assess new patient workload by first predicting the ACG score using SDOH, age, and gender and then using this information to estimate the number of appointments (scheduled visit care) and non–visit care interactions.
Methods Two years of appointment data were collected for patients who had initial appointment requests in the first year and had the ACG score, appointment, and non–visit care counts in the subsequent year. State-of-art machine learning algorithms were employed to predict ACG scores and compared with current baseline estimation. Linear regression models were then used to predict appointments and non–visit care interactions, integrating demographic data, SDOH, and predicted ACG scores.
Results The machine learning methods showed promising results in predicting ACG scores. Besides the decision tree, all other methods performed at least 9% better in accuracy than the baseline approach which had an accuracy of 78%. Incorporating SDOH and predicted ACG scores also significantly improved the prediction for both appointments and non–visit care interactions. The R 2 values increased by 95.2 and 93.8%, respectively. Furthermore, age, smoking tobacco, family history, gender, usage of injection birth control, and ACG were significant factors for determining appointments. SDOH factors such as tobacco usage, physical exercise, education level, and group activities were strongly correlated with non–visit care interactions.
Conclusion The study highlights the importance of SDOH and predicted ACG scores in predicting provider workload in primary care settings.
Keywords
social determinants of health - ACG risk score - provider workload - burnout - machine learningBackground and Significance
Provider burnout due to workload has emerged as a significant issue within the medical care system. In the United States, studies have reported alarming rates of burnout among physicians, with approximately 50% experiencing burnout at some point in their careers.[1] Burnout can have severe consequences for both providers and patients. Reports indicate that as many as 400 U.S. physicians die by suicide every year due to burnout's detrimental impact on their lives.[2] Moreover, burnout significantly diminishes providers' performance, leading to increased medical errors, posing a considerable risk to patients' safety,[3] [4] and retention and turnover in an already shrinking supply of primary care providers.[5] The mitigation of providers' burnout holds significant importance within the context of medical care.
Among the causes of provider burnout, a lack of workload control stands out as a major contributing factor. In primary care, the measurement of providers' workload predominantly centers on two key factors: the quantity of appointments within their patient panel and the time devoted to non–visit care activities. Research indicates that patients with higher levels of medical complexity are more likely to require additional non–visit care interactions.[6] [7] Therefore, to appropriately assign a new patient to a primary care physician (PCP) panel, a patient's health conditions and complexity are important factors to understand. However, for patients who are established with primary care services for the first time, health and risk factor information may not be available when the appointment request is made. Therefore, estimating a new patient's health complexity is imperative, as it facilitates the prediction of their potential appointments and non–visit care interactions.
An extensively used metric to evaluate health status is the Adjusted Clinical Group (ACG) score, serving as a reflection of patients' projected or actual utilization of health care services and it has demonstrated a strong correlation with the physician contact rate.[8] [9] ACG scores are typically computed based on patients' age, gender, and the complete set of medical diagnoses documented during a specified time frame, typically spanning 1 year. These scores have also been used as evidence for care management and intervention, and resource assignment.[10] [11] [12] However, when medical diagnosis information is not available, especially for new patients, our institution relies solely on age and gender as predictors, resulting in a reduction in accuracy.
To address this limitation, we propose leveraging social determinants of health (SDOH) information as additional predictors to enhance risk prediction for new patients. SDOH, which encompasses nonmedical factors that influence health outcomes, demonstrates strong correlations with patients' health status and health care expenditures.[13] It is increasingly acknowledged as a primary factor influencing clinical health results, and is useful in patient care transition planning.[8] [9] Existing research has consistently demonstrated the significant relationship between SDOH and patients' health conditions. For instance, studies have shown that incorporating SDOH information can greatly enhance the accuracy of predicting patients' health care utilization and health outcomes.[10] Moreover, there are benefits of including SDOH as predictors in machine learning models within the health care domain. This is evident in areas such as cardiovascular disease prediction,[11] sepsis readmission prediction,[12] and missed breast imaging appointment prediction,[14] where the integration of SDOH has proven to improve model performance. Additionally, obtaining SDOH information does not require resources from the medical team, but can be obtained directly from patient surveys or via digital conversation.[15] The extensive body of existing research strongly supports the validity of utilizing SDOH as essential predictors for patient risk and medical needs that indicate provider workload.
Achieving a balanced workload distribution among providers is crucial for optimizing patient assignments and enhancing overall quality of care. While concerns for patient satisfaction and experience may deter clinics from reassigning patients to different panels, proper assignment of new patients to provider panels holds the potential to achieve workload equilibrium. This study offers insights into optimizing patient assignments to mitigate provider burnout and cultivate a sustainable professional landscape for health care providers, ultimately addressing the pressing issue of burnout within the medical care system.
Objectives
In this study, we examined data from a family medicine department in a large teaching hospital setting, which is currently using only age and gender to estimate the ACG patient risk score for new patients and collecting the SDOH at the time of patient visit. The SDOH collection timing can be adjusted to the time of requesting an appointment. Thus, the objective of this study was twofold. We first examine the impact of integrating SDOH data with age and gender for predicting ACG scores in new patients. Second, we investigate the potential improvement in model accuracy by incorporating the predicted ACG score as a predictor for the number of appointments and non–visit care interactions.
Methods
The study focused on patients who are new to family medicine services and have not yet been seen for their initial appointment. Information on age, gender, and SDOH was collected for each patient. When predicting the number of appointments and non–visit care interactions, the pivotal elements in the workload of medical providers, assessing patient risk emerges, as a crucial consideration. To predict their ACG scores, five machine learning algorithms, namely random forest,[16] gradient boosting,[17] logistic regression,[18] support vector machine (SVM),[19] and decision tree[20] were utilized and compared for predictability. These algorithms were chosen due to their effectiveness in handling both categorical and continuous features, making them suitable for ACG score prediction. We then evaluated the models based on appropriate metrics such as accuracy and AUC values to select the best performing algorithm.
Subsequently, the predicted ACG scores were incorporated with the previously collected demographic data (i.e., age, gender, and SDOH) to predict the number of appointments and non–visit care interactions that each patient would require over the next year via the linear regression model, which is known for its strong interpretability.[21] The integration of ACG scores in this step enables a more comprehensive and accurate estimation of patients' expected health care utilization, considering both their health status and predicted service needs. [Fig. 1] demonstrates the framework of this method.
Data Description
The dataset was composed of patients' initial visit requests within a 1-year timeframe and the calculated ACG risk score, number of appointments and non–visit care interactions for the subsequent year as the outcomes for the modeling. The data spanned from 2018 to 2019 for a total visits of 33,262. The study encompasses a total of 56 variables, consisting of demographic variables—age and gender, and the remaining variables are related to SDOH. More specifically. The SDOH variables from the participating family medicine department primarily pertain to education and social activities. Education activities include only education level in our dataset. Social activities consist of 14 categories ([Table 1]).
To facilitate the analysis, categorical variables were transformed into dummy variables, resulting in a total of 182 features ([Table 2]). Dummy variables represent categorical data in a numerical format, with each category of the original variable assigned a distinct binary value, typically 0 or 1. For instance, the categorical variable “SURGICAL_YN” in our dataset, denoting the use of a surgical method of birth control, was transformed into two dummy variables: “Y” and “N.” These dummy variables assume a value of 1 if the observation belongs to the corresponding category and 0 otherwise. This conversion facilitates the inclusion of categorical variables in regression models and other analytical techniques that necessitate numerical inputs. The distributions of the number of appointments and non–visit care interactions are shown in [Fig. 2]. The ACG scores were categorized into five groups, each corresponding to a different level of risk ([Table 3]).[22] Higher ACG scores indicate a higher level of risk. The majority of patients were categorized within the 0 to 1 range. This was expected as the majority of individuals seen in primary care settings do not exhibit severe illnesses.
Abbreviation: ACG, Adjusted Clinical Group.
Abbreviation: ACG, Adjusted Clinical Group.
To address missing values, we adopted specific strategies based on variable type. For numeric variables, we filled missing values with zeros. For categorical variables with missing values, we treated them as a distinct category, enabling us to retain valuable information while accounting for the absence of data in those instances. These approaches aimed to maintain the integrity of the dataset and ensure comprehensive analysis for the predictive modeling tasks.
Model Training and Specification Details
The dataset was partitioned into training and test sets with randomized proportions of 80 and 20%, respectively. The refinement of hyperparameters for all machine learning models and the selection of features were exclusively performed within the training set. Subsequently, final assessments were executed on the independent test set. To fortify the study against potential biases and enhance robustness, the experimental procedure was iterated 10 times, employing distinct random seeds for the generation of train-test datasets.
Within the training set, a fivefold cross-validation methodology was adopted to determine optimal hyperparameters and identify significant features. The specific hyperparameters under consideration are detailed in [Table 4]. A systematic exploration was conducted to identify the top k important features, where k assumes values of 20, 50, 100, 150, and the total number of features. Implementation of all models was performed using the scikit-learn package in Python.[23]
Abbreviation: SVM, support vector machine.
Evaluation Metrics
For the prediction of ACG scores, a multi-class classification problem, we employed the widely used classification metrics of accuracy and area under the receiver operating characteristic curve (AUC). Accuracy measures the proportion of correctly classified instances among all instances in the dataset, providing an overall assessment of the classifier's performance. AUC quantifies the classifier's ability to discriminate between classes by calculating the AUC curve. However, traditional AUC is designed for binary classification tasks. To adapt to multi-class scenarios, modified versions known as AUC, One-vs-Rest (AUC_ovr) and AUC, One-vs-One (AUC_ovo) are utilized. In AUC_ovr, each class is treated as the positive class while the rest are grouped as negatives, resulting in separate evaluations for each class, effectively transforming the task into multiple binary classification problems. AUC_ovr calculates the AUC curve for each binary problem and averages these scores across all classes. Conversely, AUC_ovo entails training a binary classifier for every pair of classes, with each classifier discerning between instances of the two paired classes. AUC_ovo computes the AUC curve for each binary classifier and averages the scores from all pairwise comparisons. Hence, AUC_ovr considers each class as positive against the rest, whereas AUC_ovo evaluates classifiers based on pairwise class combinations. Both metrics offer valuable insights into the performance of classifiers in multi-class classification tasks.
For the prediction of the number of appointments and non–visit care interactions, regression techniques were employed. The evaluation metric used is R 2, also known as the coefficient of determination. R 2 quantifies the proportion of the variance in the dependent variable that is explained by the independent variables. Higher R 2 values indicate a better fit of the regression model to the data, reflecting its predictability.
Results
Prediction of Adjusted Clinical Group Score
The results from different machine learning algorithms were evaluated and compared ([Table 5]). We conducted a comparison between the machine learning models and a baseline approach. The baseline approach solely relied on age and gender information to predict ACG scores, mirroring the conventional practice within our institution before our exploration into leveraging SDOH for enhanced performance. In this baseline method, for each new patient, the ACG was estimated based on the ACG scores from the existing patients with the same gender and the closest age. The gradient boosting algorithm performed better compared with other machine learning models in terms of AUC_ovr (82.8%) and AUC_ovo (70.1%), while random forest achieved the highest accuracy (87.3%). Moreover, machine learning models such as random forest, gradient boosting, SVM, and multinomial logistic regression perform at least 9% better in accuracy than the baseline approach, demonstrating the effectiveness and superiority of incorporating SDOH and machine learning techniques in predicting ACG scores for new patients. The decision tree model, however, exhibited only a slightly better performance than the baseline.
Abbreviations: ACG, Adjusted Clinical Group; SVM, support vector machine.
The findings suggest that the machine learning models, especially random forest and SVM, offer valuable insights for predicting ACG, surpassing the performance of the baseline approach. To gain deeper insights into the classification performance of each class, we constructed receiver operating characteristic curves for individual classes using the one-versus-rest approach with the random forest model ([Fig. 3]). The majority of the AUC values consistently achieve approximately 0.8, indicating robust classification ability across classes. However, when discerning between ACG score category 2 and the rest, the corresponding AUC value is relatively lower. This outcome implies an elevated challenge in accurately distinguishing low-risk cases from others within the dataset.
Prediction of Number of Appointments and Nonvisit Care Interactions
Having achieved satisfactory prediction results for ACG scores, we proceeded to enhance our prediction models by incorporating the predicted ACG score along with age, gender, and SDOH information to predict the number of appointments and non–visit care interactions. Among the 182 features examined, 28 features exhibited significance with p-values smaller than 0.05 for predicting the number of appointments, while 49 features showed significance for predicting the number of non–visit care interactions. Only these significant features were incorporated into the final models. The integration of SDOH yielded notable improvements in model performance, as evident from the considerable increase in R 2 values ([Table 6]). Specifically, the inclusion of SDOH led to a 71.3% increase in the R 2 value for predicting the number of appointments and a 65.6% increase for the model predicting the number of non–visit care interactions. Additionally, the integration of the predicted ACG scores also made a meaningful contribution to enhancing model performance, albeit to a lesser extent. This observation can be attributed to the fact that age, gender, and SDOH information are already considered when predicting ACG scores. Nonetheless, incorporating the predicted ACG scores resulted in an additional increase of 13.9 and 17.0% for predicting appointments and non–visit care interactions, respectively.
Abbreviations: ACG, ACG, Adjusted Clinical Group; SDOH, social determinants of health.
Note: The percentage in parentheses represents the increase compared with the baseline model using only age and gender as inputs.
Overall, our findings underscore the significant impact of incorporating SDOH and predicted ACG scores in the prediction models, resulting in substantial improvements in their predictability for both the number of appointments and non–visit care interactions. These results offer valuable insights into optimizing health care resource allocation and improving patient care and reducing clinician burnout.
Feature Importance
To gain deeper insights into the most influential factors affecting the prediction tasks, we created two plots showcasing the top 10 absolute values of log(p-value) for prediction on number of appointments ([Fig. 4]) and non–visit care interactions ([Fig. 5]) with all the features as input. Larger values correspond to smaller p-values, and the use of the logarithmic function facilitates easier visualization, considering that p-values may vary significantly, spanning several orders of magnitude.
For the number of appointment prediction, age emerged as the most influential factor, followed by variables related to smoking tobacco, patient's awareness of their family history, gender, usage of injectable birth control, and predict ACG scores. The impact of age on the prediction was expected, given its widely recognized significance as a determinant of overall health conditions.[24] Additionally, tobacco use is a well-established risk factor,[25] further reinforcing its importance in the prediction. Literature also supports that certain health conditions disproportionately affect men and women,[26] contributing to the relevance of gender in the prediction. The current literature offers evidence indicating a strong association between injectable birth control and a decline in bone density, a critical factor influencing the health condition of patients.[27] Additionally, the estimated ACG score holds significant importance in predicting the number of appointments, indicating its relevance as a predictor in the prediction model.
For predicting the number of non–visit care interactions, it is noteworthy that the top 10 significant features were exclusively composed of SDOH variables. Factors such as tobacco usage, engagement in physical exercise, level of education, and participation in group activities appear strongly correlated with predictions of non–visit care interactions. Moreover, while not among the top 10 features, the ACG score categories of 1 and 2 also hold significance for providers' workload prediction in this case. Patients with severe illnesses typically have fewer appointments with primary care providers, often being directed toward specialists. Conversely, individuals with mild health concerns are more likely to seek care from primary care providers, thus influencing the number of non–visit care interactions. This correlation aligns with the commonly observed patterns of health care utilization based on illness severity. Overall, these findings offer valuable insights into the key factors influencing the provider workload prediction task, providing a basis for further understanding to optimize health care resource allocation and patient panel assignment.
Discussion
The inclusion of SDOH variables has, first, demonstrated a substantial enhancement in the accuracy of ACG score prediction, indicating the relevance of social and environmental factors in predicting patients' health status and risk. Second, the predicted ACG scores have proven to be instrumental in forecasting the number of appointments as well as exhibiting a high level of accuracy in predicting the number of non–visit care interactions. This suggests that the ACG scores, in combination with other patient-specific information, offer valuable insights into patients' expected health care resource utilization. Lastly, the research underscores the pivotal role of SDOH variables in predicting the number of appointment and non–visit care interaction predictions, with a particularly strong impact observed in the non–visit care prediction.
Despite the promising findings, this study has several limitations that should be acknowledged. The prediction models heavily rely on the accuracy and completeness of the data used. Missing values in the dataset were addressed by filling numeric variables with zeros and treating missing values in categorical variables as a new category. While these strategies were adopted to maintain data integrity, they may introduce bias and impact the model performance. Future studies should aim to incorporate more sophisticated imputation methods to handle missing data effectively.
Our organization has successfully implemented quite a few automatic triage tools in the specialty such as Cardiovascular and Pain Medicine departments. In the primary care settings, the SDOH information is already collected in a form of digital questionnaire at the beginning of each visit. Due to the time constraint, some patients are not able to complete the questionnaire before they are called into the exam room, which is the reason of many missing values. This study requires the SDOH information to be collected between when the appointment is requested and when it is scheduled to assign a PCP. The questionnaire link will be sent to patients and required to complete in a timely manner before an appointment can be granted. When the SDOH information is read into our electronic medical record, it will be sent to the server where our models will be hosted along with age and gender. It then triggers the models to run in sequence, the ACG predictive model first and then the appointments non–visit care interactions' predictive models. The prediction results are then published back to our electronic systems via the Application Programming Interface (API) and temporarily stored on a platform. Subsequently, when a scheduler selects a new patient to schedule, the scheduling system retrieves the predicted result from the platform and assigns the provider who has the least predicted workload and availability.
Conclusion
This study offers significant insights into the realm of health care analytics, underscoring the crucial role of SDOH and predicted ACG score in enhancing prediction for health care utilization. By leveraging SDOH and predicted ACG scores, our study presents a valuable tool for health care systems to balance provider workload through equitable new patient assignments and mitigate burnout effectively. This research holds the potential to guide health care systems in refining patient panel assignments, thereby elevating the quality of care provided, and fostering a more sustainable environment for providers. The findings contribute to the broader landscape of health care analytics, offering practical implications to address workload disparities and alleviate the pressing concern of provider burnout, ultimately working toward the advancement of patient-centric and resilient health care practices.
Clinical Relevance Statement
This study provided evidence on the impact of social determinants of health for predicting patient risk and medical needs to further assist practice with resource planning. The appropriate planning could significantly improve provider workload distribution and reduce burnout.
Multiple-Choice Questions
-
When considering age, gender, and SDOH, which ACG score category proves to be the most challenging to predict?
-
ACG category 1
-
ACG category 2
-
ACG category 3
-
ACG category 4
Correct Answer: The correct answer is option b.
Explanation: The results outlined in the “Prediction of ACG Score” section reveal that when distinguishing between ACG score category 2 and the remaining categories, the associated area under the curve (AUC) value is relatively lower. This suggests an increased difficulty in accurately discerning low-risk cases from others within the dataset. From a medical standpoint, this observation aligns with intuition. Severely ill or entirely healthy patients are typically straightforward to identify, whereas patients experiencing mild discomfort pose a challenge in classification.
-
-
When predicting the number of appointments, which social determinant of health below is deemed the most crucial?
-
Alcohol use
-
Tobacco use
-
Exercise
-
Education
Correct Answer: The correct answer is option b.
Explanation: The findings are detailed in the “Feature Importance” section, where it is evident that the remaining three options do not rank within the top 10 important features. Tobacco use, supported by well-established medical knowledge, emerges as a significant risk factor for patients' health.
-
Conflict of Interest
None declared.
Protection of Human and Animal Subjects
The research study was not human subject research.
-
References
- 1 Kannampallil T, Abraham J, Lou SS, Payne PRO. Conceptual considerations for using EHR-based activity logs to measure clinician burnout and its effects. J Am Med Inform Assoc 2021; 28 (05) 1032-1037
- 2 McFarland DC, Hlubocky F, Susaimanickam B, O'Hanlon R, Riba M. Addressing depression, burnout, and suicide in oncology physicians. Am Soc Clin Oncol Educ Book 2019; 39: 590-598
- 3 Dzau VJ, Kirch DG, Nasca TJ. To care is human - collectively confronting the clinician-burnout crisis. N Engl J Med 2018; 378 (04) 312-314
- 4 Shanafelt TD, Balch CM, Bechamps GJ. et al. Burnout and career satisfaction among American surgeons. Ann Surg 2009; 250 (03) 463-471
- 5 Willard-Grace R, Knox M, Huang B, Hammer H, Kivlahan C, Grumbach K. Burnout and health care workforce turnover. Ann Fam Med 2019; 17 (01) 36-41
- 6 Linzer M, Konrad TR, Douglas J. et al. Managed care, time pressure, and physician job satisfaction: results from the physician worklife study. J Gen Intern Med 2000; 15 (07) 441-450
- 7 Huang YL, Berg BP, Horn JL, Nagaraju D, Rushlow DR. Balancing clinician workload through strategic patient panel designs. Qual Manag Health Care 2023; 32 (03) 137-144
- 8 Khurshid A, Hautala M, Oliveira E. et al. Social and health information platform: piloting a standards-based, digital platform linking social determinants of health data into clinical workflows for community-wide use. Appl Clin Inform 2023; 14 (05) 883-892
- 9 Feldman SS, Davlyatov G, Hall AG. Toward understanding the value of missing social determinants of health data in care transition planning. Appl Clin Inform 2020; 11 (04) 556-563
- 10 Chen M, Tan X, Padman R. Social determinants of health in electronic health records and their impact on analysis and risk prediction: a systematic review. J Am Med Inform Assoc 2020; 27 (11) 1764-1773
- 11 Zhao Y, Wood EP, Mirin N, Cook SH, Chunara R. Social determinants in machine learning cardiovascular disease prediction models: a systematic review. Am J Prev Med 2021; 61 (04) 596-605
- 12 Amrollahi F, Shashikumar SP, Meier A, Ohno-Machado L, Nemati S, Wardi G. Inclusion of social determinants of health improves sepsis readmission prediction models. J Am Med Inform Assoc 2022; 29 (07) 1263-1270
- 13 Taylor LA, Tan AX, Coyle CE. et al. Leveraging the social determinants of health: what works?. PLoS One 2016; 11 (08) e0160217
- 14 Sotudian S, Afran A, LeBedis CA, Rives AF, Paschalidis IC, Fishman MDC. Social determinants of health and the prediction of missed breast imaging appointments. BMC Health Serv Res 2022; 22 (01) 1454
- 15 Langevin R, Berry ABL, Zhang J. et al. Implementation fidelity of chatbot screening for social needs: acceptability, feasibility, appropriateness. Appl Clin Inform 2023; 14 (02) 374-391
- 16 Breiman L. Random forests. Mach Learn 2001; 45 (01) 5-32
- 17 Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat 2001; 29 (05) 1189-1232
- 18 Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley & Sons; 2004
- 19 Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995; 20 (03) 273-297
- 20 von Winterfeldt D, Edwards W. Decision Trees. In: Decision Analysis and Behavioral Research. Cambridge: Cambridge University Press; 63-89
- 21 Montgomery DC, Peck EA, Vining GG. Introduction to Linear Regression Analysis. New York, NY: John Wiley & Sons; 2021
- 22 Department of Health Policy and Management at The Johns Hopkins University Bloomberg School of Public Health. The johns hopkins ACG® system version 12.0 user documentation. 2019
- 23 Pedregosa F, Varoquaux G, Gramfort A. et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 2011; 12: 2825-2830
- 24 Piazza JR, Charles ST, Almeida DM. Living with chronic health conditions: age differences in affective well-being. J Gerontol B Psychol Sci Soc Sci 2007; 62 (06) 313-321
- 25 Bergström J, Preber H. Tobacco use as a risk factor. J Periodontol 1994; 65 (5, Suppl): 545-550
- 26 Vlassoff C. Gender differences in determinants and consequences of health and illness. J Health Popul Nutr 2007; 25 (01) 47-61
- 27 Scholes D, LaCroix AZ, Ichikawa LE, Barlow WE, Ott SM. Injectable hormone contraception and bone density: results from a prospective study. Epidemiology 2002; 13 (05) 581-587
Address for correspondence
Publication History
Received: 20 December 2023
Accepted: 07 May 2024
Article published online:
03 July 2024
© 2024. Thieme. All rights reserved.
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
-
References
- 1 Kannampallil T, Abraham J, Lou SS, Payne PRO. Conceptual considerations for using EHR-based activity logs to measure clinician burnout and its effects. J Am Med Inform Assoc 2021; 28 (05) 1032-1037
- 2 McFarland DC, Hlubocky F, Susaimanickam B, O'Hanlon R, Riba M. Addressing depression, burnout, and suicide in oncology physicians. Am Soc Clin Oncol Educ Book 2019; 39: 590-598
- 3 Dzau VJ, Kirch DG, Nasca TJ. To care is human - collectively confronting the clinician-burnout crisis. N Engl J Med 2018; 378 (04) 312-314
- 4 Shanafelt TD, Balch CM, Bechamps GJ. et al. Burnout and career satisfaction among American surgeons. Ann Surg 2009; 250 (03) 463-471
- 5 Willard-Grace R, Knox M, Huang B, Hammer H, Kivlahan C, Grumbach K. Burnout and health care workforce turnover. Ann Fam Med 2019; 17 (01) 36-41
- 6 Linzer M, Konrad TR, Douglas J. et al. Managed care, time pressure, and physician job satisfaction: results from the physician worklife study. J Gen Intern Med 2000; 15 (07) 441-450
- 7 Huang YL, Berg BP, Horn JL, Nagaraju D, Rushlow DR. Balancing clinician workload through strategic patient panel designs. Qual Manag Health Care 2023; 32 (03) 137-144
- 8 Khurshid A, Hautala M, Oliveira E. et al. Social and health information platform: piloting a standards-based, digital platform linking social determinants of health data into clinical workflows for community-wide use. Appl Clin Inform 2023; 14 (05) 883-892
- 9 Feldman SS, Davlyatov G, Hall AG. Toward understanding the value of missing social determinants of health data in care transition planning. Appl Clin Inform 2020; 11 (04) 556-563
- 10 Chen M, Tan X, Padman R. Social determinants of health in electronic health records and their impact on analysis and risk prediction: a systematic review. J Am Med Inform Assoc 2020; 27 (11) 1764-1773
- 11 Zhao Y, Wood EP, Mirin N, Cook SH, Chunara R. Social determinants in machine learning cardiovascular disease prediction models: a systematic review. Am J Prev Med 2021; 61 (04) 596-605
- 12 Amrollahi F, Shashikumar SP, Meier A, Ohno-Machado L, Nemati S, Wardi G. Inclusion of social determinants of health improves sepsis readmission prediction models. J Am Med Inform Assoc 2022; 29 (07) 1263-1270
- 13 Taylor LA, Tan AX, Coyle CE. et al. Leveraging the social determinants of health: what works?. PLoS One 2016; 11 (08) e0160217
- 14 Sotudian S, Afran A, LeBedis CA, Rives AF, Paschalidis IC, Fishman MDC. Social determinants of health and the prediction of missed breast imaging appointments. BMC Health Serv Res 2022; 22 (01) 1454
- 15 Langevin R, Berry ABL, Zhang J. et al. Implementation fidelity of chatbot screening for social needs: acceptability, feasibility, appropriateness. Appl Clin Inform 2023; 14 (02) 374-391
- 16 Breiman L. Random forests. Mach Learn 2001; 45 (01) 5-32
- 17 Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat 2001; 29 (05) 1189-1232
- 18 Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley & Sons; 2004
- 19 Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995; 20 (03) 273-297
- 20 von Winterfeldt D, Edwards W. Decision Trees. In: Decision Analysis and Behavioral Research. Cambridge: Cambridge University Press; 63-89
- 21 Montgomery DC, Peck EA, Vining GG. Introduction to Linear Regression Analysis. New York, NY: John Wiley & Sons; 2021
- 22 Department of Health Policy and Management at The Johns Hopkins University Bloomberg School of Public Health. The johns hopkins ACG® system version 12.0 user documentation. 2019
- 23 Pedregosa F, Varoquaux G, Gramfort A. et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 2011; 12: 2825-2830
- 24 Piazza JR, Charles ST, Almeida DM. Living with chronic health conditions: age differences in affective well-being. J Gerontol B Psychol Sci Soc Sci 2007; 62 (06) 313-321
- 25 Bergström J, Preber H. Tobacco use as a risk factor. J Periodontol 1994; 65 (5, Suppl): 545-550
- 26 Vlassoff C. Gender differences in determinants and consequences of health and illness. J Health Popul Nutr 2007; 25 (01) 47-61
- 27 Scholes D, LaCroix AZ, Ichikawa LE, Barlow WE, Ott SM. Injectable hormone contraception and bone density: results from a prospective study. Epidemiology 2002; 13 (05) 581-587