Public Health, Population Health, and Epidemiology Informatics: Recent Research and Trends in the United States

Summary

Objectives: To survey advances in public and population health and epidemiology informatics over the past 18 months.

Methods: We conducted a review of English-language research works conducted in the domain of public and population health informatics and published in MEDLINE or Web of Science between January 2015 and June 2016 where information technology or informatics was a primary subject or main component of the study methodology. Selected articles were presented using a thematic analysis based on the 2011 American Medical Informatics Association (AMIA) Public Health Informatics Agenda tracks as a typology.

Results: Results are given within the context developed by Dixon et al., (2015) and key themes from the 2011 AMIA Public Health Informatics Agenda. Advances are presented within a socio-technical infrastructure undergirded by a trained, competent public health workforce, systems development to meet the business needs of the practice field, and research that evaluates whether those needs are adequately met. The ability to support and grow the infrastructure depends on financial sustainability.

Conclusions: The fields of public health and population health informatics continue to grow, with the most notable developments focused on surveillance, workforce development, and linking to or providing clinical services, which encompassed population health informatics advances. Very few advances addressed the need to improve communication, coordination, and consistency with the field of informatics itself, as identified in the AMIA agenda. This will likely result in the persistence of the silos of public health information systems that currently exist. Future research activities need to aim toward a holistic approach of informatics across the enterprise.

Keywords

Public health informatics - population health - surveillance - public health practice - epidemiology

Introduction

In 2000, Yasnoff and colleagues [[1]] defined public health informatics (PHI) as the systematic application of information, computer science, and technology to public health practice, research, and learning. Public health informatics as a field under the larger umbrella of public health began to mature and continues to grow. Today, the scope of public health informatics includes the conceptualization, design, development, deployment, refinement, maintenance, and evaluation of communication, surveillance, information, and learning systems relevant to public health [[2]]. Informatics enables public health practitioners to assess public and population health by accurately combining data from disparate sources including health care, labor and industry, correctional facilities, etc. The information must then be presented to decision makers in a compelling way for interventions to be funded and developed. Public health informatics is a specialized area of informatics related to providing the public health services and monitoring the health of populations, both domestic and globally. Global health informatics, which focuses on the planning and delivery of health and public health services in low-resource settings, while important for the international informatics community, is not included given the scope of this review. Population health informatics, focusing on the planning for and care of clinical populations, overlaps with public health informatics in that both are concerned with the population rather than the individual, and within public health practice, any clinical service provided is focused on meeting the needs of vulnerable populations. Currently, there are multiple definitions of population health, all of which hinge upon the nature of the denominator which defines the population of interest [[3]]. For a small ambulatory provider, the population of interest might be all of that provider’s patients who are in a disease management program, whereas, for a regional hospital, the denominator of interest might be the catchment area for the population that facility serves. Although in practice, many public health professionals generally view population health as an important subset under the purview of public health. Because much of what public health is concerned with overlaps with population health, and there are opportunities to learn from the published literature in population health, we included these articles in our review.

[Figure 1] depicts the relationship between the scope of informatics and the intersection within the fields of global health informatics, population health informatics, and public health informatics [[4]]. This article discusses public health informatics in the context of the five tracks from the 2011 American Medical Informatics Association (AMIA) Public Health Informatics Agenda and relates the literature reviewed to overarching themes emerging from the 2011 meeting.

Fig. 1 Relationship between areas of informatics based on Dixon et al. [[5]]

Methods

For this review, we conducted searches of the biomedical informatics literature published in MEDLINE and Web of Science between January 1, 2015 and June 30, 2016. We used the following MEDLINE Subject Headings (MeSH) keywords in various combinations to maximize sensitivity: “public health informatics”, “public health”, “population health”, “medical informatics”, “informatics”, “information systems”, and “health information systems”. Use of MeSH headings was preferred, although articles were also identified using keywords to check article titles and abstracts. The search strategy we used was similar to that of Dixon et al. [[4]] based on the International Medical Informatics Association (IMIA) Yearbook to select candidate “best papers” [[5]]. Search terms used in the Web of Science were similar to the MeSH headings.

We used inclusion and exclusion factors to narrow the list of articles. To be included an article had to focus on a topic related to (a) public health practice or research; or (b) population health practice and research, and (c) include informatics, information science, or computing as the primary subject of the study or a main component of the study methodology. Criteria by which we assessed each article’s relevance to public health or population health practice and research included: (1) activities conducted by, with, or involving a local, state, or federal health agency; (2) assessment and monitoring of diseases and health outcomes; (3) primary and secondary prevention of diseases; (4) social determinants of health as well as health disparities; (5) development of the public health workforce including public health informatics education and competencies; and, (6) the interface of public and clinical health care. Articles were excluded if they did not constitute original research such as letters, editorials, perspectives, opinions, whitepapers, comments, meeting or workshop summaries, and study protocols. We further excluded articles focused on populations living outside the United States to limit the scope.

In 2011 at its Spring Congress meeting, AMIA updated its 2001 agenda to further advance the field of public health informatics [[6], [7]]. Participants at the meeting self-selected into one of five tracks: (1) technical framework; (2) research and evaluation; (3) ethics; (4) education, professional training, and workforce development; and (5) sustainability. For the purposes of this paper, we combined the technical framework and research and evaluation tracks, as all publications described were focused on research by definition, and most pertained to the technical framework. Discussions in each of the five tracks resulted in 62 detailed recommendations and three cross-cutting key themes. The themes traversed all five tracks and focused on the important aspects of informatics that are still relevant today such as real-time data availability, user-centered design, information system evaluation, and informatics research.

To describe recent advances in public health informatics, articles that met these criteria were grouped and presented using a thematic analysis adapted from the 2011 AMIA Public Health Informatics (PHI) Agenda. Findings related to these articles are discussed within the context of the themes emerging from the 2011 meeting, presented in [Table 1].

Table 1

Key Themes from the 2011 AMIA PHI Agenda Meeting.

1	Enhance communication and information sharing within the PHI community
2	Improve the consistency of PHI through common public health terminologies, rigorous evaluation methodologies, and competency-based training
3	Promote effective coordination and leadership that will champion and drive the field forward

Results

The initial search results from MEDLINE and Web of Science produced 278 citations. Full abstracts for all citations were reviewed by the two authors of this paper, and inclusion and exclusion criteria, as well as relevance to public or population health practices and research were applied. This resulted in 76 citations included in the final full manuscript review. This review further narrowed the selection to 42 articles, with 30 of those falling under Socio-Technical Framework Infrastructure and Research; eight within Education and Workforce Development; one pertaining to Governance and Ethics; and, three in the area of Sustainability.

(1)Socio-Technical Infrastructure and Research

In 2015, the National Association of County and City Health Officials (NACCHO) conducted the Informatics Needs and Capacity of Local Health Departments (LHD) survey. The purpose of the survey was to develop an informatics profile for LHDs [[8]]. The profile included topics addressing physical infrastructure [[8]], public health workforce [[9]], capacity for informatics [[10]], electronic health records (EHRs), and electronic exchange of data [[11]–[13]].

The survey outlined several challenges that must be addressed to build informatics capabilities in local health departments including developing the workforce, building capacity in smaller LHDs, and developing informed and consistent leadership [[14]]. The size and governance type of LHDs influenced technical infrastructure governance and control structures as well as the use of EHR technology. Best practices for informatics are lacking in public health departments and workforce development is not common. Standardized, well-informed informatics practices are critical to respond to the needs of the community and to promote local health.

One of the important services of public health is to inform and educate the public about important public health issues. Turner and colleagues [[15], [16]] performed workflow studies and translation quality analysis with public health professionals to develop a prototype machine translation tool for improving access to multilingual health promotion materials for limited English proficiency individuals. The use of requirements gathering and the analysis of workflows explain work processes and describe how data, information, and knowledge are used. To enhance the usability of consumer health information websites and recommend specific design features to support the users’ ability to explore health information, Pang and colleagues [[17]] took a user-centered approach and recommended four design considerations to support information seeking behavior: (1) providing a dynamic information scope; (2) supporting serendipity; (3) considering trust implications; and (4) enhancing interactivity.

(1-a) Evaluation of Surveillance Practice

The intention of Immunization Information Systems (IIS) is to facilitate immunization information exchange between health care providers and public health [[18]] and this represents one of the first informatics initiatives linking public health and health care providers. To achieve the goal of monitoring health through IIS, certain conditions must exist, such as interoperable EHRs, confidentiality policies, and data of sufficient quality and timeliness [[19], [20]]. Although IIS have made significant progress over time, technological constraints and workforce demands still affect timeliness and completeness of immunization data. Groom and colleagues noted that despite the potential advances, much remains to be studied in determining the impact of immunization data on immunization rates, coverage, and surveillance practice [[19]].

The evaluation of the effectiveness, accessibility, and quality of personal and population-based health services is one of the assurance aspects of public health. Duncan et al. evaluated the stability, quality, and value of birth certificate data [[21], [22]]. Changes to the birth certificate identities over the first year of life are common; understanding the scope of the changes is important in linking birth registry data into public health information systems. The notion of integrating disparate information from multiple systems including EHRs is not a new concept in health care or public health. Successful integration strategies offer the potential for improved health outcomes from both the public health and clinical care perspectives.

(1-b) Surveillance Methods

Novel sources of surveillance data include social media and internet-accessible information such as news and Real-time Simple Syndication (RSS) feeds, email subscriptions from surveillance systems, HTML scraping, and user-submitted reports. Ahmed and colleagues [[23]] used HealthMap to examine meningitis surveillance and compare Health-Map’s real-time internet-based data streaming to traditional surveillance methods. They found that HealthMap supplemented traditional methods by providing information on the modes of disease transmission and the ability to visualize that transmission in outbreaks. Using Twitter data, Allen and colleagues [[24]] conducted influenza surveillance using real-time tweets and developed a comprehensive data mining process to accurately identify influenza-related tweets. This process combined a geographic information system (GIS) that targeted, filtered, and normalized the Twitter data, along with a machine-learning procedure to filter relevant messages from noise. The results were improved granularity of the spatial and temporal details of the tweet data. A committee formed by the Institute of Medicine defined a panel of 11 domains and 12 measures related to social and behavioral determinants of health (SBDH) to be included in EHRs [[25]]. Informatics issues identified that present challenges for the adoption of these measures included standardization, efficient collection and review, decision support, and support for research. Standardization challenges are centered on the lack of existing standards for SBDH determinants and the need for an agreement around operational measures of the domains, particularly around the source of information for each measure. The effective collection and review of the measures requires not only an agreement on the source, but the verification of the data, and the determination of currency, i.e., was present in the past, but not currently. These issues have important implications for the clinical workflow when collecting and using these data. Decision support could provide patient-specific recommendations based on the answers provided to SBDH questions. Before this is put in place, processes for how to generate decision support, how to present it to the user, and how to update decision logic must be developed. Enhanced and expanded SBDH data can provide support for primary research on the SBD measures and applied research on how using the information can improve health. This research could include identification of new SBDH and disease associations, the effect of interventions related to SBDH variables, and feedback to enhance decision support for SBDH.

Bugeja and colleagues [[26]] conducted a review of the literature on the ability of the National Coronial Information System (NCIS) to inform medico legal investigations of death. Similar to the challenges presented by the lack of SBDH data in EHRs, the authors found that the NCIS was a reliable source of detailed, good-quality information for injury surveillance, but data were not always complete due to open cases and missing information. These limitations have the potential to introduce selection and reporting biases and to preclude identification of confounders in the data.

Several important efforts related to model the development for surveillance and processing of surveillance data are noteworthy. Jafarapour and colleagues [[27]] developed a probabilistic model to identify outbreaks and quantify the determinants of outbreak detection. In this work they describe a comparison of detection methods for various surveillance scenarios that can guide the configuration of surveillance systems to better allow for outbreak detection. Poulymenopoulos and colleagues [[28]] used the multifactorial etiology of obesity to examine big data processing using semantic extract-transform-load (ETL) services to annotate big data and assign unstructured text mappings to ontology classes, such that it becomes useful data for analysis. Using a Bayesian framework to link clinical diagnoses of individuals from emergency department free-text reports with epidemiologic models of influenza disease outbreaks in that population, Cooper and colleagues [[29]] demonstrated that this approach could be used to detect and characterize outbreaks early.

Epidemiologists have traditionally thought about disease distribution primarily as a function of time, as demonstrated by the epidemiologic curve depicting incident cases and their onset dates. Using molecular strain types of tuberculosis isolates, Smith and colleagues [[30]] developed an application (DotMapper) to depict the location of cases geographically and display their epidemiologic characteristics without the need for expertise in geographical information systems (GIS). The addition of geographic analyses provided additional information that highlighted cases that likely shared epidemiologic links. The DotMapper tool was specifically developed to be user-friendly, providing an alternative to the typical specialized GIS software that has steep requirements for technical expertise.

(1-c) Interoperable Health Information Exchange Infrastructure

Recent literature points to the increased use of large datasets to address the essential service of linking individuals to needed personal health services and the assurance of the provision of health care when otherwise unavailable. These large datasets allow data providers to move data out of traditional silos into a centralized platform for public health use. Krause et al. used a centralized platform to collect and visualize health-related data and then link individuals to health care services in underserved areas. These centralized platforms, or data networks, serve as the foundation on which informaticians and researchers have the capacity to collect, store, maintain, and analyze data for decision-making [[31]]. Curtis and colleagues describe four such large data research networks, comprised of electronic data collected during clinical care from EHRs and administrative data: the Food and Drug Administration’s Mini-Sentinel program (medical product safety); the National Patient-Centered Clinical Research Network (comparative effectiveness research); the National Institutes of Health’s Health Care Systems Research Collaboratory Distributed Research Network (biomedical research); and ESPnet (public health surveillance). The paper concludes: “these four programs [are] examples of the first stage in the development of a shared national big-data resource that leverages the investments of many agencies and organizations for the benefit of multiple networks and users” [[32]]. These efforts represent innovative efforts to break down information silos across provider and public health organizations.

(1-d) mHealth and Social Media Infrastructure

Innovative uses of mHealth and social media are beginning to have a significant impact on public health practice. As internet usage continues to grow, people are using mobile technologies and social media to search for health information and publish information about their own personal health [[33]]. Wilson et al. explored the use of mobile technologies in developing individual immunization registries to assist travelers with the proof of country-specific required vaccinations [[34]]. In addition to mobile technologies, many people are using social media to seek and share health information. Disease-specific online communities are developing around social media applications and are providing opportunities for researchers to capture new, different types of surveillance data [[35]]. Social media has also shown promise in disaster risk reduction and management for emergency preparedness professionals [[36]].

(1-e) Population Health

In 2016, Kharrazi and colleagues [[3]] reported on an agenda developed through a 1-day symposium funded by the National Library of Medicine to further the field of population health informatics. The resulting agenda consisted of 18 priority recommendations around four broad goals: (1) developing a standardized collaborative framework and infrastructure, (2) advancing technical tools and methods, (3) developing a scientific evidence and knowledge base, and (4) developing an appropriate framework for policy, privacy, and sustainability. The authors reached two important conclusions: population health informatics is closely linked to public health informatics, and collaboration and integration of activities in these areas are important. One example of this collaboration could focus on developing population-based decision-support tools for use by public health, community officials, and clinicians.

EHRs can be a source of longitudinal data about populations and have some advantages compared with traditional epidemiologic methods [[37]]. For example, they contain information about large numbers of people; provide increased sample sizes for analysis of rare diseases; are less expensive and less time consuming compared to traditional methods; and can be generalizable to patient populations. The areas of environmental epidemiology and social epidemiology can particularly benefit from this source of information because patient location are linked to other spatial data. Careful consideration of issues related to EHR data is required, including study population definition, population attrition, disease/case definition, and privacy concerns. Roosan and colleagues [[38]] used Veterans’ Affairs data to study whether population-based information (big data) could be used to reduce uncertainty in medicine. Examination of the use of population health data led to the redesign of a population information display based on user preferences.

(2) Education and Workforce Development

As part of the NACCHO’s Informatics Needs and Capacity of Local Health Departments survey, Massoudi and colleagues [[9]] found that public health workers had substantial training needs in informatics that must be addressed if health departments are to function effectively in the digital age. Many training needs identified by non-informatics specialists were core public health skills such as interpreting quantitative data, designing and running reports from information systems, and using and interpreting qualitative data. Dixon and colleagues [[39]] studied the characteristics of public health workers who specialize in informatics and assessed the informatics-related aspects of the workforce in the Public Health Workforce and Needs Survey. They found that only 1.3% of the workforce specialized in informatics and that limited training existed for those seeking such opportunities. Evidence published by Shah and colleagues [[40]] suggested that health departments with a greater number of information systems were associated with greater activity level in emergency preparedness, including having developed or updated a written emergency plan; reviewed relevant legal authorities; participated in tabletop exercises or drills; participated in functional exercises or drills; participated in full-scale exercises or drills; assessed emergency preparedness competencies of staff; and provided emergency preparedness training of staff. The capacity to use information systems effectively likely predicts the ability to meet communication needs and gather and disseminate information. Brownson and colleagues [[41]] looked at the changing needs of epidemiologic research and practice relative to macro level trends in society and identified use of big data and informatics skills as necessary to meet these needs. Specific competencies identified were linking, managing, mapping, analyzing, interpreting, displaying, and communicating findings. The opportunities presented by EHR data were identified as a specific need where the capability to link multiple large, static, and streamed datasets is critical.

Several publications in the area of training for informatics are noteworthy. Yu and faculty colleagues [[42]] developed an introductory evidence-based public health informatics course aimed at public health and information and library science graduate students. To address the training needs in the management of health data, information, and knowledge, and to support the goals of a learning health system, Gray and colleagues [[43]] promoted large-scale, integrated public health platforms as an environment in which optimal health learning and development can occur. They cited three examples of public health platforms particularly well suited for training, including the Million Hearts® [[44]]. Acknowledging a need for professional certification in informatics, AMIA is developing the Advanced Health Informatics Certification (AHIC) [[45]] for professionals working in health care, public health, and personal health. Gadd and colleagues defined two phases of the certification process and four primary paths to certification relative to health professional or graduate education, health informatics education, and health informatics experience. Next steps include developing the AHIC core content, establishing the certifying entity, and launching graduate health informatics program accreditation [[46]].

(3) Governance and Ethics

Electronic health records and clinically integrated systems have the potential for allowing the reuse of personal health data for public health research. Health care providers have increasingly come to the consensus that sharing health-related data is critical for enabling population-based research. However, the reuse of data raises ethical and legal issues. Lamas et al. discussed the lack of ethical-legal frameworks for utilization of large datasets or data warehouses [[47]]. Many large datasets and data warehouses are not always designed for the reuse of data for different purposes, such as public health research or surveillance. As a result, opportunities to reuse data (even when anonymized) for the benefit of the public’s health are limited and require additional study and development.

(4) Sustainability

One key area discussed at the AMIA Spring Congress in 2011 was sustainability along with recommendations to develop public health informatics as a priority in state and local health departments [[4]]. Because of the current political and economic environment, resources are often spread across many competing demands in public health agencies [[13]]. Public health leaders need a clear understanding of the value of informatics and how it can improve public health to invest strategically. However, leadership is often removed from the daily activities of public health so showing the impact of informatics becomes difficult. The current public health workforce must become informatics-savvy [[48]] to communicate value with leaders in the field as well as external funders. Public health agencies must have an informatics-savvy workforce coordinated through effective leadership and governance, and must use a formal approach to informatics design and use of information systems that support public health business needs. Baker et al. referenced a self-assessment tool hosted by the Public Health Informatics Institute, Building an Informatics-Savvy Health Department [[49]]. Sustainability of informatics in public health will depend on the workforce as well as the continued development of informatics-based interventions and publications across the public health enterprise.

Discussion

Similar to this research, Dixon and colleagues examined the public health informatics literature from 2012 to 2014 to identify advances in public health informatics [[4]]. In order to identify trends in themes across the 2011 AMIA Public Health Informatics Agenda, we compared themes captured in this literature review with the previous effort. Several trends emerged, some reflecting growth and accomplishment in the field:

• Continued increase in the adoption of information technology and development of informatics capacity in health departments

• Growth in the use of health-related social media data for public health research

Other trends pointed to continued challenges and unmet needs:

• Increasing need for public health informatics education and training

• Continued challenges with data quality and data standardization

• Need for increased research in governance and ethics

• Increasing need for sustainability of public health informatics efforts

The increased adoption of technology and the development of informatics capacity continue to improve public health workflow and the timely collection and analysis of information. Despite these improvements, the challenges identified across both literature reviews (2012–2014 and 2015–2016) remain the same. Data standardization and data quality challenges affect interoperability and the usability of important public health information for decision makers. While public health departments have access to increasing amounts of data, it is unclear that they have the informatics capacity and capability to actually exploit that data.

The use of social media data to search for and publish health-related data continues to grow at a fast pace. These big data sets provide a wealth of information to health researchers. New ways of analyzing and visualizing these data are improving every day. Public health needs these many data sources to monitor population health, however, health departments often lack the informatics capacity to link and analyze these large datasets.

To date, there have been many improvements in public health informatics education including the addition of public health informatics as a core competency in the Master of Public Health (MPH) degree [[50]]. Informatics is thus reaching new professionals who are entering the field. Much more training and education is needed for professionals already in the field, including a certification that focuses solely on public health informatics. AMIA’s Advanced Health Informatics Certification is a positive step in this direction, however it is not solely focused on public health.

There has been little published research that is specifically focused on governance and ethics for public health informatics. The reuse of health-related data and the proliferation of large datasets and big data will require additional study in this area.

Resources in public health departments are spread thinly across the many competing demands for keeping the population healthy. In order to sustain and grow public health informatics, we need to improve the knowledge of the value of the field and build informatics-savvy administrators and staff.

This work focused on published research related to public health and population health, appearing in the literature between January 1, 2015 and June 30, 2016, and included English-only citations. Global health research publications were excluded, as were findings in population health coming from the healthcare sector, as opposed to the public health sector. These selection criteria imposed limitations on our findings. Public health informatics is practiced globally and published in languages other than English. It is likely that some publications coming from the healthcare sector, and not specifically involving or mentioning public health or population health, could have significant impacts on public health and population health informatics. It is well recognized that a publication bias towards positive results exists in the biomedical literature in general [[51]], and is also a limitation when conducting reviews of the informatics literature [[52]–[54]]. And finally, much of the research in public health informatics is very applied, being conducted as part of the organizations’ mission to serve and protect the public’s health. The incentives that academic researchers receive for publishing may not be as strong, or even present, in these practice-focused organizations.

Conclusion

Despite the strides made in recent years, many research needs still exist in the areas of public health informatics and population health informatics. The 2011 AMIA Public Health Informatics Agenda laid out 62 recommendations within five tracks. In this paper, we examined the current informatics literature and broad areas of accomplishments within these five tracks. The majority of the articles we reviewed focused on technical solutions implemented as part of surveillance to identify and solve community health problems. These articles covered a variety of topics including immunization information systems, general public health surveillance, syndromic surveillance, behavioral health, and public health data.

Other focus areas of the literature were workforce development in informatics and linking to and providing clinical care. The NACCHO’s Informatics Needs and Capacity of Local Health Departments survey was conducted to assess the informatics capacity of LHDs, and the published results offered much insight into the workforce and other informatics aspects of public health services. Other articles focused on policy development and sustainability. Although much progress has been made in using informatics to monitor and improve the population’s health status, continued study of the development of policies to support informatics is needed. Sustainability of the field depends on several factors, including the development of partnerships, policies, and informatics leaders.

Another area of significant need is the further realization of the key themes from the 2011 AMIA Spring Congress. Although the articles we reviewed addressed some specific areas of public health practice, very few addressed the AMIA key themes, namely, the need to improve communication, coordination, and consistency with the field of informatics itself [[8]]. Public health professionals conducting these studies are focused on specific programmatic outcomes, which continues to keep informatics isolated, and on the traditional public health information silos. Future research activities need to aim toward a holistic approach to implementation of informatics across the public health enterprise.

• References

• 1 Yasnoff W, O’Carroll PW, Koo D, Linkins RW, Kilborne EM. Public health informatics: improving and transforming public health in the information age. J Public Health Manag Pract 2000; 06 (06) 67-75.
• 2 Magnuson JA, Fu P. editors. Public health informatics and information systems. 2nd edition. London: Springer; 2014
• 3 Kharrazi H, Lasser EC, Yasnoff WA, Loonsk J, Advani A, Lehmann HP. et al. A proposed national research and development agenda for population health informatics: summary recommendations from a National Expert Workshop. J Am Med Inform Assoc. 2016 Mar 27.
• 4 Dixon BE, Kharrazi H, Lehmann HP. Public health and epidemiology informatics: recent research and trends in the United States. Yearb Med Inform 2015; Aug 13; 10 (01) 199-206.
• 5 Lamy JB, Seroussi B, Griffon N, Kerdelhue G, Jaulent MC, Bouaud J. Toward a formalization of the process to select IMIA Yearbook best papers. Methods Inf Med 2015; 54 (02) 135-44.
• 6 Yasnoff W, Overhage JM, Humphreys BL, LaVenture MA. A national agenda for public health informatics: summarized recommendations from the 2001 AMIA Spring Congress. J Am Med Inform Assoc 2001; 08 (06) 535-45.
• 7 Massoudi BL, Goodman KW, Gotham IJ, Holmes JH, Lang L, Miner K. et al. An informatics agenda for public health: summarized recommendations from the 2011 AMIA PHI Conference. J Am Med Inform Assoc 2012; Sep-Oct; 19 (05) 688-95.
• 8 Chester K, Massoudi BL, Shah GH. Control of the public health IT physical infrastructure: Findings From the 2015 Informatics Capacity and Needs Assessment Survey. J Public Health Manag Pract. 2016 Nov-Dec;22 Suppl 6, Public Health Informatics: S13-S17.
• 9 Massoudi BL, Chester K, Shah GH. Public health staff development needs in informatics: findings from a national survey of local health departments. J Public Health Manag Pract 2016; Nov-Dec;22 Suppl 6, Public Health Informatics: S58-S62.
• 10 Drezner K, McKeown L, Shah GH. Assessing Skills and Capacity for Informatics: Activities Most Commonly Performed by or for Local Health Departments. J Public Health Manag Pract 2016; Nov-Dec;22 Suppl 6, Public Health Informatics S51-S57.
• 11 Shah GH, Leider JP, Castrucci BC, Williams KS, Luo H. Characteristics of local health departments associated with implementation of electronic health records and other informatics systems. Public Health Rep 2016; Mar-Apr; 131 (02) 272-82.
• 12 Williams KS, Shah GH. Electronic health records and meaningful use in local health departments: updates from the 2015 NACCHO Informatics Assessment Survey. J Public Health Manag Pract 2016; Nov-Dec;22 Suppl 6, Public Health Informatics: S27-S33.
• 13 Shah GH, Vest JR, Lovelace K, McCullough JM. Local health departments’ partners and challenges in electronic exchange of health information. J Public Health Manag Pract 2016; Nov-Dec;22 Suppl 6, Public Health Informatics: S44-S50.
• 14 Gibson PJ, Shah GH, Streichert LC, Verchick L. Urgent challenges for local public health informatics. J Public Health Manag Pract 2016; Nov-Dec;22 Suppl 6, Public Health Informatics: S6-S8.
• 15 Turner AM, Brownstein MK, Cole K, Karasz H, Kirchhoff K. Modeling workflow to design machine translation applications for public health practice. J Biomed Inform 2015; Feb; 53: 136-46.
• 16 Turner AM, Dew KN, Desai L, Martin N, Kirchhoff K. Machine translation of public health materials from English to Chinese: a feasibility study. JMIR Public Health Surveill 2015; Jul-Dec; 01 (02) e17.
• 17 Pang PC, Chang S, Verspoor K, Pearce J. Designing Health Websites Based on Users’ Web-Based Information-Seeking Behaviors: A Mixed-Method Observational Study. J Med Internet Res 2016; Jun 06; 18 (06) e145.
• 18 Martin DW, Lowery NE, Brand B, Gold R, Horlick G. Immunization information systems: a decade of progress in law and policy. J Public Health Manag Pract 2015; May-Jun; 21 (03) 296-303.
• 19 Groom H, Hopkins DP, Pabst LJ, Murphy JMorgan, Patel M, Calonge N. et al. Immunization information systems to increase vaccination rates: a community guide systematic review. J Public Health Manag Pract 2015; May-Jun; 21 (03) 227-48.
• 20 Metroka AE, Papadouka V, Ternier A, Zucker JR. Effects of Health Level 7 Messaging on Data Quality in New York City’s Immunization Information System, 2014. Public Health Rep 2016; Jul-Aug; 131 (04) 583-7.
• 21 Duncan J, Eilbeck K, Narus SP, Clyde S, Thornton S, Staes C. Building an ontology for identity resolution in healthcare and public health. Online J Public Health Inform 2015; 07 (02) e219.
• 22 Duncan J, Narus SP, Clyde S, Eilbeck K, Thornton S, Staes C. Birth of identity: understanding changes to birth certificates and their value for identity resolution. J Am Med Inform Assoc 2015; Apr; 22 E1 E120-E9.
• 23 Ahmed SS, Oviedo-Orta E, Mekaru SR, Freifeld CC, Tougas G, Brownstein JS. Surveillance for Neisseria meningitidis Disease Activity and Transmission Using Information Technology. PLoS One 2015; 10 (05) e0127406.
• 24 Allen C, Tsou MH, Aslam A, Nagel A, Gawron JM. Applying GIS and Machine Learning Methods to Twitter Data for Multiscale Surveillance of Influenza. PLoS One 2016; 11 (07) e0157734.
• 25 Hripcsak G, Forrest CB, Brennan PF, Stead WW. Informatics to support the IOM social and behavioral domains and measures. J Am Med Inform Assoc 2015; Jul; 22 (04) 921-4.
• 26 Bugeja L, Ibrahim JE, Ferrah N, Murphy B, Willoughby M, Ranson D. The utility of medico-legal databases for public health research: a systematic review of peer-reviewed publications using the National Coronial Information System. Health Res Policy Syst. 2016 Apr 12; 14. 28.
• 27 Jafarpour N, Izadi M, Precup D, Buckeridge DL. Quantifying the determinants of outbreak detection performance through simulation and machine learning. J Biomed Inform 2015; Feb; 53: 180-7.
• 28 Poulymenopoulou M, Papakonstantinou D, Malamateniou F, Vassilacopoulos G. A health analytics semantic ETL service for obesity surveillance. Stud Health Technol Inform 2015; 210: 840-4.
• 29 Cooper GF, Villamarin R, Rich FCTsui, Millett N, Espino JU, Wagner MM. A method for detecting and characterizing outbreaks of infectious disease from clinical reports. J Biomed Inform 2015; Feb; 53: 15-26.
• 30 Smith CM, Hayward AC. DotMapper: an open source tool for creating interactive disease point maps. BMC Infect Dis 2016; Apr 12; 16: 145.
• 31 Krause DD. Data lakes and data visualization: an innovative approach to address the challenges of access to health care in Mississippi. Online J Public Health Inform 2015; 07 (03) e225.
• 32 Curtis L, Brown J, Platt R. Four Health Data Networks Illustrate The Potential For A Shared National Multipurpose Big-Data Network. Health Aff 2014; Jul; 33 (07) 1178-1186.
• 33 Fernandez-Luque L, Bau T. Health and social media: perfect storm of information. Healthc Inform Res 2015; Apr; 21 (02) 67-73.
• 34 Wilson K, Atkinson KM, Westeinde J. Apps for immunization: Leveraging mobile devices to place the individual at the center of care. Hum Vaccin Immunother 2015; 11 (10) 2395-9.
• 35 Fung IC, Tse ZT, Fu KW. The use of social media in public health surveillance. Western Pac Surveill Response J 2015; Apr-Jun; 06 (02) 3-6.
• 36 Yeager V, Cooper Jr. GP, Burkle Jr. FM, Subbarao I. Twitter as a Potential Disaster Risk Reduction Tool. Part IV: Competency-based Education and Training Guidelines to Promote Community Resiliency. PLoS Curr. 2015 Jun 29;7.
• 37 Casey JA, Schwartz BS, Stewart WF, Adler NE. Using electronic health records for population health research: A review of methods and applications. Annu Rev Public Health 2016; 37: 61-81.
• 38 Roosan D, Del Fiol G, Butler J, Livnat Y, Mayer J, Samore M. et al. Feasibility of population health analytics and data visualization for decision support in the infectious diseases domain: A pilot study. Appl Clin Inform 2016; 07 (02) 604-23.
• 39 Dixon BE, McFarlane TD, Dearth S, Grannis SJ, Gibson PJ. Characterizing informatics roles and needs of public health workers: results from the public health workforce interests and needs survey. J Public Health Manag Pract. 2015 Nov-Dec;21 Suppl 6: S130-40.
• 40 Shah GH, Newell B, Whitworth RE. Health departments’ engagement in emergency preparedness activities: the influence of health informatics capacity. Int J Health Policy Manag 2016; Apr 30; 05 (10) 575-82.
• 41 Brownson RC, Samet JM, Chavez GF, Davies MM, Galea S, Hiatt RA. et al. Charting a future for epidemiologic training. Ann Epidemiol 2015; Jun; 25 (06) 458-65.
• 42 Yu X, Xie Y, Pan X, Mayfield-Johnson S, Whipple J, Azadbakht E. Developing an evidence-based public health informatics course. J Med Libr Assoc 2015; Oct; 103 (04) 194-7.
• 43 Gray K. Public health platforms: an emerging informatics approach to health professional learning and development. J Public Health Res 2016; Apr 26; 05 (01) 665.
• 44 Centers for Disease Control and Prevention. Million Hearts. Atlanta, GA: Centers for Disease Control and Prevention. n.d. [cited 2016 December 11]; Available from: http://millionhearts.hhs.gov
• 45 Gadd CS, Williamson JJ, Steen EB, Andriole KP, Delaney C, Gumpper K. et al. Eligibility requirements for advanced health informatics certification. J Am Med Inform Assoc [Article] 2016; Jul; 23 (04) 851-4.
• 46 Gadd CS, Williamson JJ, Steen EB, Fridsma DB. Creating advanced health informatics certification. J Am Med Inform Assoc 2016; Jul; 23 (04) 848-50.
• 47 Lamas E, Barh A, Brown D, Jaulent MC. Ethical, Legal and Social Issues related to the health data-warehouses: re-using health data in the research and public health research. Stud Health Technol Inform 2015; 210: 719-23.
• 48 LaVenture M, Brand B, Ross DA, Baker EL. Building an Informatics-Savvy Health Department II: Operations and Tactics. J Public Health Manag Pract 2015; Jan; 21 (01) 96-9.
• 49 Public Health Informatics Institute. Building an Informatics-Savvy Health Department. Decatur, GA: Public Health Informatics Institute; 2015 Jan; Available from: http://www.phii.org/infosavvy
• 50 Association of Schools and Programs of Public Health. MPH Core Competency Model. Washington, DC: Association of Schools and Programs of Public Health. n.d. [cited 2017 March 11]; Available from: http://www.aspph.org/educate/models/mph-competency-model/.
• 51 Dwan K, Gamble C, Williamson P, Kirkham J. the Reporting Bias Group. Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review. PLoS One 2013; Jul 5 08 (07) e66844.
• 52 Eysenbach G. Tackling Publication Bias and Selective Reporting in Health Informatics Research: Register your eHealth Trials in the International eHealth Studies Registry. J Med Internet Res 2004; 06 (03) e35.
• 53 Ammenworth E, de Keizer N. A Viewpoint on Evidence-based Health Informatics, Based on a Pilot Survey on Evaluation Studies in Health Care Informatics. J Am Med Inform Assoc 2007; 14: 368-71.
• 54 Vawdrey D, Hripczak G. Publication Bias in Clinical Trials of Electronic Health Records. J Biomed Inform 2013; Feb; 46 (01) 139-41.