U.S. COVID-19 State Government Public Dashboards: An Expert ReviewFunding None.
Background In the United States, all 50 state governments deployed publicly viewable dashboards regarding the novel coronavirus disease 2019 (COVID-19) to track and respond to the pandemic. States dashboards, however, reflect idiosyncratic design practices based on their content, function, and visual design and platform. There has been little guidance for what state dashboards should look like or contain, leading to significant variation.
Objectives The primary objective of our study was to catalog how information, system function, and user interface were deployed across the COVID-19 state dashboards. Our secondary objective was to group and characterize the dashboards based on the information we collected using clustering analysis.
Methods For preliminary data collection, we developed a framework to first analyze two dashboards as a group and reach agreement on coding. We subsequently doubled coded the remaining 48 dashboards using the framework and reviewed the coding to reach total consensus.
Results All state dashboards included maps and graphs, most frequently line charts, bar charts, and histograms. The most represented metrics were total deaths, total cases, new cases, laboratory tests, and hospitalization. Decisions on how metrics were aggregated and stratified greatly varied across dashboards. Overall, the dashboards were very interactive with 96% having at least some functionality including tooltips, zooming, or exporting capabilities. For visual design and platform, we noted that the software was dominated by a few major organizations. Our cluster analysis yielded a six-cluster solution, and each cluster provided additional insights about how groups of states engaged in specific practices in dashboard design.
Conclusion Our study indicates that states engaged in dashboard practices that generally aligned with many of the goals set forth by the Centers for Disease Control and Prevention, Essential Public Health Services. We highlight areas where states fall short of these expectations and provide specific design recommendations to address these gaps.
Keywordsdata visualization - coronavirus infections - pandemics - geographic information systems - access to information
Protection of Human and Animal Subjects
Our study is exempted from Institutional review board.
Received: 12 August 2020
Accepted: 11 January 2021
14 April 2021 (online)
© 2021. Thieme. All rights reserved.
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
- 1 Wu DTY, Chen AT, Manning JD. et al. Evaluating visual analytics for health informatics applications: a systematic review from the American Medical Informatics Association Visual Analytics Working Group Task Force on Evaluation. J Am Med Inform Assoc 2019; 26 (04) 314-323
- 2 Streeb D, El-Assady M, Keim D, Chen M. Why visualize? Untangling a large network of arguments. IEEE Trans Vis Comput Graph 2019
- 3 Wahi MM, Dukach N. Visualizing infection surveillance data for policymaking using open source dashboarding. Appl Clin Inform 2019; 10 (03) 534-542
- 4 Hester G, Lang T, Madsen L, Tambyraja R, Zenker P. Timely data for targeted quality improvement interventions: Use of a visual analytics dashboard for bronchiolitis. Appl Clin Inform 2019; 10 (01) 168-174
- 5 Kummer BR, Willey JZ, Zelenetz MJ. et al. Neurological dashboards and consultation turnaround time at an academic medical center. Appl Clin Inform 2019; 10 (05) 849-858
- 6 Shneiderman B, Plaisant C, Cohen M, Jacobs S, Elmqvist N, Diakopoulos N. Designing the User Interface: Strategies for Effective Human-Computer Interaction. 6th ed.. Hoboken, NJ: Pearson Education; 2016
- 7 Ware C. Information visualization: Perception for design. 3rd ed.. Waltham, MA: Morgan Kaufmann; 2013
- 8 Koch T, Denike K. Crediting his critics' concerns: remaking John Snow's map of Broad Street cholera, 1854. Soc Sci Med 2009; 69 (08) 1246-1251
- 9 Jordan EO. Influenza encyclopedia. Epidemic influenza: a survey. Accessed July 22, 2020 at: http://hdl.handle.net/2027/spo.8580flu.0016.858
- 10 Cheng CK, Lau EH, Ip DK, Yeung AS, Ho LM, Cowling BJ. A profile of the online dissemination of national influenza surveillance data. BMC Public Health 2009; 9: 339
- 11 Cheng CK, Ip DK, Cowling BJ, Ho LM, Leung GM, Lau EH. Digital dashboard design using multiple data streams for disease surveillance with influenza surveillance as an example. J Med Internet Res 2011; 13 (04) e85
- 12 World Health Organization. Ebola situation report – 21. Accessed July 1, 2020 at: https://apps.who.int/ebola/en/status-outbreak/situation-reports/ebola-situation-report-21-january-2015
- 13 World Health Organization. WHO director-general's opening remarks at the media briefing on COVID-19 - 11 March 2020. Accessed June 1, 2020 at: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020
- 14 Kamel Boulos MN, Geraghty EM. Geographical tracking and mapping of coronavirus disease COVID-19/severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) epidemic and associated events around the world: how 21st century GIS technologies are supporting the global fight against outbreaks and epidemics. Int J Health Geogr 2020; 19 (01) 8
- 15 Rosenkrantz L, Schuurman N, Bell N, Amram O. The need for giscience in mapping covid-19. Health Place 2020; (e-pub ahead of print) DOI: 10.1016/j.healthplace.2020.102389.
- 16 Muhareb R, Giacaman R. Tracking COVID-19 responsibly. Lancet 2020; (e-pub ahead of print) DOI: 10.1016/S0140-6736(20)30693-0.
- 17 Grandi S, Bernasconi A. Convergence of web design and spatial, statistical, genomic and epidemiological information: the case of geo-dashboards in the covid-19 crisis. Doc Geogr 2020; 1: 463-476
- 18 Matheus R, Janssen M, Maheshwari D. Data science empowering the public: Data-driven dashboards for transparent and accountable decision-making in smart cities. Gov Inf Q 2018; 37 (03) 101284
- 19 Bonifati A, Cattaneo F, Ceri S, Fuggetta A, Paraboschi S. Designing data marts for data warehouses. ACM Trans Softw Eng Methodol 2001; 10 (04) 452-483
- 20 Center for Systems Science and Engineering (CSSE) at Johns Hopkins University. COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Accessed June 1, 2020 at: https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6
- 21 Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis 2020; 20 (05) 533-534
- 22 WorldoMeter. World/countries/United States. 2020 . Accessed June 2, 2020 at: https://www.worldometers.info/coronavirus/country/us/
- 23 Wissel BD, Van Camp PJ, Kouril M. et al. An interactive online dashboard for tracking COVID-19 in U.S. counties, cities, and states in real time. J Am Med Inform Assoc 2020; 27 (07) 1121-1125
- 24 Anderson M, Vogels EA. Americans turn to technology during covid-19 outbreak, say an outage would be a problem. Accessed October 7, 2020 at: https://www.pewresearch.org/fact-tank/2020/03/31/americans-turn-to-technology-during-covid-19-outbreak-say-an-outage-would-be-a-problem/
- 25 Rogers K. Johns Hopkins dashboard: the people behind the pandemic's most visited site. Accessed November 16, 2020 at: https://www.cnn.com/2020/07/11/health/johns-hopkins-covid-19-map-team-wellness-trnd/index.html
- 26 Limousin P, Azzabi R, Bergé L, Dubois H, Truptil S, Gall LL. How to build dashboards for collecting and sharing relevant informations to the strategic level of crisis management: an industrial use case. 2019 International Conference on Information and Communication Technologies for Disaster Management (ICT-DM); 2019 Paris, France: 1-8 DOI: 10.1109/ICT-DM47966.2019.9032970
- 27 Irtaimeh HJ, Obaidat A, Khaddam A. Strategic role of dashboard application in enhancing crisis management capabilities in organizations field study on jordanian cellular companies. Int J Management Sci Business Res 2016;5(10):
- 28 National Governors Association, Association of State and Territorial Health Officials. Roadmap to recovery: a public health guide for governors. Accessed October 7, 2020 at: https://www.astho.org/COVID-19/Roadmap-to-Recovery/
- 29 State of Nevada Department of Health and Human Services. Current status. Accessed July 1, 2020 at: https://app.powerbigov.us/view?r=eyJrIjoiMjA2ZThiOWUtM2FlNS00MGY5LWFmYjUtNmQwNTQ3Nzg5N2I2IiwidCI6ImU0YTM0MGU2LWI4OWUtNGU2OC04ZWFhLTE1NDRkMjcwMzk4MCJ9
- 30 Florida Department of Health Division of Disease Control and Health Protection. Florida's covid-19 data and surveillance dashboard. Accessed July 1, 2020 at: https://experience.arcgis.com/experience/96dd742462124fa0b38ddedb9b25e429
- 31 Everts J. The dashboard pandemic. Dialogues Hum Geogr 2020; 10 (02) 260-264
- 32 Centers for Disease Control and Prevention. 10 essential public health services. Accessed November 16, 2020 at: https://www.cdc.gov/publichealthgateway/publichealthservices/essentialhealthservices.html
- 33 Jaccard P. Distribution de la flore alpine dans le bassin des dranses et dans quelques regions voisines. Bullet Soci Vaudoise Sci Naturelles 1901; 37: 241-272
- 34 Finch H. Comparison of distance measures in cluster analysis with dichotomous data. J Data Sci 2005; 3 (01) 85-100
- 35 Ward J. Hierarchical grouping to optimize an objective function. J Am Stat Assoc 1963; 58: 236-244
- 36 Kulper FK, Fisher L. 391: A Monte Carlo comparison of six clustering procedures. Biometrics 1975; 31 (03) 777
- 37 Overall JE, Gibson JM, Novy DM. Population recovery capabilities of 35 cluster analysis methods. J Clin Psychol 1993; 49 (04) 459-470
- 38 Hands S, Everitt B. A Monte Carlo study of the recovery of cluster structure in binary data by hierarchical clustering techniques. Multivariate Behav Res 1987; 22 (02) 235-243
- 39 Caliński T, Harabasz J. A dendrite method for cluster analysis. Commun Stat Theory Methods 1973; 3 (01) 1-27
- 40 Duda RO, Hart PE, Stork DG. Pattern Classification. 2nd ed.. New York, NY: Wiley; 2000
- 41 Ettorchi-Tardy A, Levif M, Michel P. Benchmarking: a method for continuous quality improvement in health. Healthc Policy 2012; 7 (04) e101-e119
- 42 Centers for Disease Control and Prevention. Information for health departments on reporting cases of COVID-19. Accessed July 22, 2020 at: https://www.cdc.gov/coronavirus/2019-ncov/php/reporting-pui.html
- 43 Merrilees A. Inconsistencies in official Missouri COVID-19 numbers point to need for standard data reporting. Accessed July 22, 2020 at: https://www.stltoday.com/lifestyles/health-med-fit/coronavirus/inconsistencies-in-official-missouri-covid-19-numbers-point-to-need-for-standard-data-reporting/article_f022ac55-0678-51ef-8a0f-0d053c07e3cf.html
- 44 Servick K. ‘Huge hole’ in COVID-19 testing data makes it harder to study racial disparities. Accessed July 22, 2020 at: https://www.sciencemag.org/news/2020/07/huge-hole-covid-19-testing-data-makes-it-harder-study-racial-disparities
- 45 Wamsley L. Fired Florida data scientist launches a coronavirus dashboard of her own. Accessed July 22, 2020 at: https://www.npr.org/2020/06/14/876584284/fired-florida-data-scientist-launches-a-coronavirus-dashboard-of-her-own
- 46 Harrison TMS, Transparency DS. participation, and accountability practices in open government: A comparative study. Gov Inf Q 2014; 31 (04) 513-525
- 47 Villeneuve JP. Transparency of transparency: The pro-active disclosure of the rules governing access to information as a gauge of organisational cultural transformation. The case of the swiss transparency regime. Gov Inf Q 2014; 31 (04) 556-562
- 48 Nacoti M, Ciocca A, Giupponi A. et al. At the epicenter of the covid-19 pandemic and humanitarian crises in Italy: changing perspectives on preparation and mitigation. NEJM Catal 2020; 1 (02) 1-5
- 49 Wadhera RK, Wadhera P, Gaba P. et al. Variation in COVID-19 hospitalizations and deaths across New York City boroughs. JAMA 2020; 323 (21) 2192-2195
- 50 Khunti K, Singh AK, Pareek M, Hanif W. Is ethnicity linked to incidence or outcomes of covid-19?. BMJ 2020; 369: m1548
- 51 Ogen Y. Assessing nitrogen dioxide (NO2) levels as a contributing factor to coronavirus (COVID-19) fatality. Sci Total Environ 2020; 726 (138605): 138605
- 52 Allio MK. Strategic dashboards: Designing and deploying them to improve implementation. Strategy Leadersh 2012; 40 (05) 24-31
- 53 Bharosa N, Janssen M, Meijer S, Brave F. Designing and evaluating dashboards for multi-agency crisis preparation: A living lab. In: Wimmer MA, Chappelet JL, Janssen M, Scholl HJ. eds. Electronic Government. 9th IFIP WG 8.5 International Conference, EGOV 2010, Lausanne, Switzerland: 180-191
- 54 Ackoff RL. From data to wisdom. J Appl Syst Anal 1989; 16: 3-9
- 55 Goodman RA, Patrick LR, Howard RJ. Communicating information for action within the public health system. In: Teutsch SM, Churchill RE. eds. Principles and Practice of Public Health Surveillance. New York, NY: Oxford; 2000
- 56 German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN. Guidelines Working Group Centers for Disease Control and Prevention (CDC). Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep 2001; 50 (RR-13): 1-35, quiz CE1–CE7