Appl Clin Inform 2021; 12(03): 539-550
DOI: 10.1055/s-0041-1730324
Research Article

The My Diabetes Care Patient Portal Intervention: Usability and Pre-Post Assessment

William Martinez
1   Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, United States
Amber J. Hackstadt
2   Department of Biostatistics, Vanderbilt University Medicine Center, Nashville, Tennessee, United States
Gerald B. Hickson
3   Center for Patient and Professional Advocacy, Vanderbilt University Medical Center, Nashville, Tennessee, United States
Thomas Knoerl
4   Upstate Medical University, State University of New York, Syracuse, New York, United States
S. Trent Rosenbloom
5   Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
Kenneth A. Wallston
6   Center for Health Services Research, Vanderbilt University Medical Center, Nashville, Tennessee, United States
Tom A. Elasy
7   Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, United States
› Author Affiliations
Funding This work was supported by NIDDK/NIH K23DK106511 and P30DK092986 and NCATS/NIH UL1TR000445. The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.


Background My Diabetes Care (MDC) is a novel, multifaceted patient portal intervention designed to help patients better understand their diabetes health data and support self-management. MDC uses infographics to visualize and summarize patients' diabetes health data, incorporates motivational strategies, and provides literacy level–appropriate educational resources.

Objectives We aimed to assess the usability, acceptability, perceptions, and potential impact of MDC.

Methods We recruited 69 participants from four clinics affiliated with Vanderbilt University Medical Center. Participants were given 1 month of access to MDC and completed pre- and post-questionnaires including validated measures of usability and patient activation, and questions about user experience.

Results Sixty participants completed the study. Participants' mean age was 58, 55% were females, 68% were Caucasians, and 48% had limited health literacy (HL). Most participants (80%) visited MDC three or more times and 50% spent a total of ≥15 minutes on MDC. Participants' median System Usability Scale (SUS) score was 78.8 [Q1, Q3: 72.5, 87.5] and significantly greater than the threshold value of 68 indicative of “above average” usability (p < 0.001). The median SUS score of patients with limited HL was similar to those with adequate HL (77.5 [72.5, 85.0] vs. 82.5 [72.5, 92.5]; p = 0.41). Participants most commonly reported the literacy level–appropriate educational links and health data infographics as features that helped them better understand their diabetes health data (65%). All participants (100%) intended to continue to use MDC. Median Patient Activation Measure® scores increased postintervention (64.3 [55.6, 72.5] vs. 67.8 [60.6, 75.0]; p = 0.01).

Conclusion Participants, including those with limited HL, rated the usability of MDC above average, anticipated continued use, and identified key features that improved their understanding of diabetes health data. Patient activation improved over the study period. Our findings suggest MDC may be a beneficial addition to existing patient portals.

Author Contributions

W.M., A.J.H., S.T.R., K.A.W., and T.A.E. developed the concept and designed the study. Acquisition of data was done by W.M.. All the authors participated in the analysis and interpretation of data. W.M. drafted the manuscript. All the authors participated in critical revision of the manuscript for important intellectual content. W.M., A.J.H., G.B.H., T.K., S.T.R., and T.A.E. gave the final approval of the version to be published. W.M. was responsible for obtaining funding and supervised the study.

Protection of Human and Animal Subjects

The Vanderbilt University Institutional Review Board approved this study.

Kenneth A. Wallston passed away on October 27, 2020.

Supplementary Material

Publication History

Received: 08 November 2020

Accepted: 01 April 2021

Article published online:
30 June 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 Powers MA, Bardsley J, Cypress M. et al. Diabetes self-management education and support in type 2 diabetes: a joint position statement of the American Diabetes Association, the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics. Diabetes Educ 2017; 43 (01) 40-53
  • 2 Centers for Disease Control and Prevention. National Diabetes Statistics Report, 2020. Atlanta, GA: Centers for Disease Control and Prevention, U.S. Department of Health and Human Services; 2020
  • 3 Zimbudzi E, Lo C, Ranasinha S. et al. The association between patient activation and self-care practices: a cross-sectional study of an Australian population with comorbid diabetes and chronic kidney disease. Health Expect 2017; 20 (06) 1375-1384
  • 4 Almutairi N, Hosseinzadeh H, Gopaldasani V. The effectiveness of patient activation intervention on type 2 diabetes mellitus glycemic control and self-management behaviors: a systematic review of RCTs. Prim Care Diabetes 2020; 14 (01) 12-20
  • 5 Rask KJ, Ziemer DC, Kohler SA, Hawley JN, Arinde FJ, Barnes CS. Patient activation is associated with healthy behaviors and ease in managing diabetes in an indigent population. Diabetes Educ 2009; 35 (04) 622-630
  • 6 Dalal AK, Dykes P, Samal L. et al. Potential of an electronic health record-integrated patient portal for improving care plan concordance during acute care. Appl Clin Inform 2019; 10 (03) 358-366
  • 7 Baldwin JL, Singh H, Sittig DF, Giardina TD. Patient portals and health apps: Pitfalls, promises, and what one might learn from the other. Healthc (Amst) 2017; 5 (03) 81-85
  • 8 Tulu B, Trudel J, Strong DM, Johnson SA, Sundaresan D, Garber L. Patient portals: an underused resource for improving patient engagement. Chest 2016; 149 (01) 272-277
  • 9 Tieu L, Sarkar U, Schillinger D. et al. Barriers and facilitators to online portal use among patients and caregivers in a safety net health care system: a qualitative study. J Med Internet Res 2015; 17 (12) e275
  • 10 Lyles CR, Nelson EC, Frampton S, Dykes PC, Cemballi AG, Sarkar U. Using electronic health record portals to improve patient engagement: research priorities and best practices. Ann Intern Med 2020; 172 (11, Suppl): S123-S129
  • 11 Nystrom DT, Singh H, Baldwin J, Sittig DF, Giardina TD. Methods for patient-centered interface design of test result display in online portals. EGEMS (Wash DC) 2018; 6 (01) 15
  • 12 Martinez W, Threatt AL, Rosenbloom ST, Wallston KA, Hickson GB, Elasy TA. A patient-facing diabetes dashboard embedded in a patient web portal: design sprint and usability testing. JMIR Human Factors 2018; 5 (03) e26
  • 13 Martinez W, Wallston KA, Schlundt DG. et al. Patients' perspectives on social and goal-based comparisons regarding their diabetes health status. BMJ Open Diabetes Res Care 2018; 6 (01) e000488
  • 14 Gee PM, Greenwood DA, Paterniti DA, Ward D, Miller LMS. The eHealth Enhanced Chronic Care Model: a theory derivation approach. J Med Internet Res 2015; 17 (04) e86
  • 15 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
  • 16 Taha J, Sharit J, Czaja SJ. The impact of numeracy ability and technology skills on older adults' performance of health management tasks using a patient portal. J Appl Gerontol 2014; 33 (04) 416-436
  • 17 Sarkar U, Piette JD, Gonzales R. et al. Preferences for self-management support: findings from a survey of diabetes patients in safety-net health systems. Patient Educ Couns 2008; 70 (01) 102-110
  • 18 Tieu L, Schillinger D, Sarkar U. et al. Online patient websites for electronic health record access among vulnerable populations: portals to nowhere?. J Am Med Inform Assoc 2017; 24 (e1): e47-e54
  • 19 Boston Children's Hospital and Health Level Sevel International. HL7 FHIR SMART Application Launch Framework Implementation Guide Release 1.0.0. 2018. Accessed January 23, 2021 at:
  • 20 Kincaid JP, Fishburne Jr RP, Rogers RL, Chissom BS. Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Millington, TN: Institute for Simulation and Training, University of Central Florida; 1975
  • 21 American Diabetes Association. Introduction: standards of medical care in diabetes-2018. Diabetes Care 2018; 41 (Suppl. 01) S1-S2
  • 22 Wolff K, Chambers L, Bumol S. et al. The PRIDE (Partnership to Improve Diabetes Education) Toolkit: development and evaluation of novel literacy and culturally sensitive diabetes education materials. Diabetes Educ 2016; 42 (01) 23-33
  • 23 Learning About Diabetes Inc. Learning About Diabetes. Accessed February 10, 2018 at:
  • 24 American Diabetes Association. ADA Support Community. 2020 . Accessed April 27, 2020 at:
  • 25 Sarkar U, Schillinger D, López A, Sudore R. Validation of self-reported health literacy questions among diverse English and Spanish-speaking populations. J Gen Intern Med 2011; 26 (03) 265-271
  • 26 Norman CD, Skinner HA. eHEALS: the eHealth Literacy Scale. J Med Internet Res 2006; 8 (04) e27
  • 27 Ando H, Cousins R, Young C. Achieving saturation in thematic analysis: Development and refinement of a codebook. Compr Psychol 2014; 3: 3
  • 28 Brooke J. SUS: quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL. eds. Usability Evaluation Industry. London: Taylor & Francis; 1996: 189-194
  • 29 Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact 2008; 24 (06) 574-594
  • 30 Hibbard JH, Mahoney ER, Stockard J, Tusler M. Development and testing of a short form of the patient activation measure. Health Serv Res 2005; 40 (6, Pt 1): 1918-1930
  • 31 Wallston KA, Rothman RL, Cherrington A. Psychometric properties of the perceived diabetes self-management scale (PDSMS). J Behav Med 2007; 30 (05) 395-401
  • 32 Fitzgerald JT, Davis WK, Connell CM, Hess GE, Funnell MM, Hiss RG. Development and validation of the Diabetes Care Profile. Eval Health Prof 1996; 19 (02) 208-230
  • 33 Quandt SA, Ip EH, Kirk JK. et al. Assessment of a short diabetes knowledge instrument for older and minority adults. Diabetes Educ 2014; 40 (01) 68-76
  • 34 Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care 2000; 23 (07) 943-950
  • 35 McGuire BE, Morrison TG, Hermanns N. et al. Short-form measures of diabetes-related emotional distress: the Problem Areas in Diabetes Scale (PAID)-5 and PAID-1. Diabetologia 2010; 53 (01) 66-69
  • 36 Solomon M, Wagner SL, Goes J. Effects of a web-based intervention for adults with chronic conditions on patient activation: online randomized controlled trial. J Med Internet Res 2012; 14 (01) e32
  • 37 Nelson LA, Bethune MC, Lagotte AE, Osborn CY. The usability of diabetes MAP: a web-delivered intervention for improving medication adherence. JMIR Human Factors 2016; 3 (01) e13
  • 38 Strauss A, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage; 1990
  • 39 User Experience Basics. Accessed September 2, 2019 at:
  • 40 Giardina TD, Baldwin J, Nystrom DT, Sittig DF, Singh H. Patient perceptions of receiving test results via online portals: a mixed-methods study. J Am Med Inform Assoc 2018; 25 (04) 440-446
  • 41 Turchioe MR, Myers A, Isaac S. et al. A systematic review of patient-facing visualizations of personal health data. Appl Clin Inform 2019; 10 (04) 751-770
  • 42 Marciano L, Camerini A-L, Schulz PJ. The role of health literacy in diabetes knowledge, self-care, and glycemic control: a meta-analysis. J Gen Intern Med 2019; 34 (06) 1007-1017
  • 43 Apter AJ. Can patient portals reduce health disparities? A perspective from asthma. Ann Am Thorac Soc 2014; 11 (04) 608-612
  • 44 Price-Haywood EG, Harden-Barrios J, Ulep R, Luo Q. eHealth literacy: patient engagement in identifying strategies to encourage use of patient portals among older adults. Popul Health Manag 2017; 20 (06) 486-494
  • 45 Wildenbos GA, Maasri K, Jaspers M, Peute L. Older adults using a patient portal: registration and experiences, one year after implementation. Digit Health 2018; 4: 2055207618797883
  • 46 Son H, Nahm E-S. Older adults' experience using patient portals in communities: challenges and opportunities. Comput Inform Nurs 2019; 37 (01) 4-10
  • 47 Georgsson M, Staggers N. Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. J Am Med Inform Assoc 2016; 23 (01) 5-11
  • 48 Fu HNC, Rizvi RF, Wyman JF, Adam TJ. Usability evaluation of four top-rated commercially available diabetes apps for adults with type 2 diabetes. Comput Inform Nurs 2020; 38 (06) 274-280
  • 49 Fu HN, Adam TJ, Konstan JA, Wolfson JA, Clancy TR, Wyman JF. Influence of patient characteristics and psychological needs on diabetes mobile app usability in adults with type 1 or type 2 diabetes: crossover randomized trial. JMIR Diabetes 2019; 4 (02) e11462
  • 50 Nelson LA, Mayberry LS, Wallston K, Kripalani S, Bergner EM, Osborn CY. Development and usability of REACH: a tailored theory-based text messaging intervention for disadvantaged adults with type 2 diabetes. JMIR Human Factors 2016; 3 (02) e23
  • 51 Steitz BD, Wong JIS, Cobb JG, Carlson B, Smith G, Rosenbloom ST. Policies and procedures governing patient portal use at an academic medical center. JAMIA Open 2019; 2 (04) 479-488
  • 52 Nijland N, van Gemert-Pijnen JE, Kelders SM, Brandenburg BJ, Seydel ER. Factors influencing the use of a web-based application for supporting the self-care of patients with type 2 diabetes: a longitudinal study. J Med Internet Res 2011; 13 (03) e71
  • 53 Althouse AD. Adjust for multiple comparisons? It's not that simple. Ann Thorac Surg 2016; 101 (05) 1644-1645
  • 54 Moukheiber Z. Behind Epic Systems, A Low-Key Health IT Company Called InterSystems. Forbes; 2013 . Accessed April 18, 2019 at: