Appl Clin Inform 2017; 08(03): 845-853
DOI: 10.4338/ACI-2017-04-RA-0062
Research Article
Schattauer GmbH

The Emerging Role of the Chief Research Informatics Officer in Academic Health Centers

L. Nelson Sanchez-Pinto
1   University of Chicago Medical Center, Pediatrics, Chicago, Illinois, United States
,
Abu S. M. Mosa
2   University of Missouri Columbia School of Medicine, Department of Health Management and Informatics, Columbia, Missouri, United States
,
Kate Fultz-Hollis
3   Oregon Health & Science University, Department of Medical Informatics and Clinical Epidemiology, Portland, Oregon, United States
,
Umberto Tachinardi
4   UW-Madision, ICTR, Madison, Wisconsin, United States
,
William K. Barnett
5   Regenstrief Institute Inc, Indianapolis, Indiana, United States
,
Peter J. Embi*
5   Regenstrief Institute Inc, Indianapolis, Indiana, United States
› Author Affiliations
Further Information

Correspondence to:

L. Nelson Sanchez-Pinto, MD, MBI
Division of Critical Care Medicine, Ann & Robert Lurie
Children‘s Hopsital of Chicago, 225 E Chicago Ave
Chicago, IL, USA
Phone: (312) 227–4800   

Publication History

received: 12 April 2017

accepted in revised form: 28 May 2017

Publication Date:
20 December 2017 (online)

 

Summary

Background: The role of the Chief Research Informatics Officer (CRIO) is emerging in academic health centers to address the challenges clinical researchers face in the increasingly digitalized, data-intensive healthcare system. Most current CRIOs are the first officers in their institutions to hold that role. To date there is very little published information about this role and the individuals who serve it.

Objective: To increase our understanding of the CRIO role, the leaders who serve it, and the factors associated with their success in their organizations.

Methods: The Clinical Research Informatics Working Group of the American Medical Informatics Association (AMIA) conducted a national survey of CRIOs in the United States and convened an expert panel of CRIOs to discuss their experience during the 2016 AMIA Annual Symposium.

Results: CRIOs come from diverse academic backgrounds. Most have advance training and extensive experience in biomedical informatics but the majority have been CRIOs for less than three years. CRIOs identify funding, data governance, and advancing data analytics as their major challenges.

Conclusion: CRIOs play an important role in helping shape the future of clinical research, innovation, and data analytics in healthcare in their organizations. They share many of the same challenges and see the same opportunities for the future of the field. Better understanding the background and experience of current CRIOs can help define and develop the role in other organizations and enhance their influence in the field of research informatics.

Citation: Sanchez-Pinto LN, Mosa ASM, Fultz-Hollis K, Tachinardi U, Barnett WK, Embi PJ. The Emerging Role of the Chief Research Informatics Officer in Academic Health Centers. Appl Clin Inform 2017; 8: 845–853 https://doi.org/10.4338/ACI-2017-04-RA-0062


#

1. Background and Significance

The role of the Chief Research Informatics Officer (CRIO) is emerging in academic health centers to address the challenges faced by clinical researchers in our rapidly evolving, data-intensive healthcare system [[1]]. CRIOs are involved in activities such as implementing informatics tools to facilitate clinical research, designing data warehouses and workflows to improve the secondary use of electronic health record (EHR) data, developing infrastructures for advanced data analytics, bioinformatics and precision medicine research, and balancing the need for data security and privacy [[1], [2]].

Despite the rapid growth of biomedical research informatics needs across healthcare systems, the emergence of the CRIO role is very recent. There are certainly other leadership roles that are responsible for the research informatics needs across an academic health center, but only a handful of institutions have made a commitment to elevate this role to the chief officer level. To date, there is very little published information about this role and the individuals who serve it.


#

2. Objectives

The Clinical Research Informatics Working Group of the American Medical Informatics Association (AMIA) engaged in a series of activities in order to improve our understanding of the CRIO role, the background of the leaders who serve in that position, the challenges faced by them, the lessons they have learned, and their future vision of the role. This paper summarizes the two main activities performed by the Working Group: a national survey of CRIOs in the United States and a panel discussion with three expert CRIOs that took place during the 2016 AMIA Annual Symposium.


#

3. Methods

3.1. Survey

3.1.1. Design, Setting and Participants

The survey was a cross-sectional, anonymous survey. Invitations to participate in the survey were sent to the professional e-mail account of all the CRIOs and CRIO-equivalents in academic health centers in the United States who were identified through an extensive search. A CRIO-equivalent was considered as a leader at the chief officer level whose primary responsibility was to oversee the research informatics activities in their organization. The search included colleague referrals, online search engine queries, and professional networking directory searches. The identified CRIOs were invited to participate in the survey a maximum of three times.

The REDCap survey module at the University of Missouri-Columbia was used to conduct the online survey [[3], [4]]. The University of Missouri-Columbia and The University of Chicago Institutional Review Boards approved this study.


#

3.1.2. Survey Development

The survey was developed in two stages. In the first stage, a draft survey was developed by three of the co-authors (LNSP, ASMM, KFH) using two prior surveys of Chief Medical Informatics Officers as a models [[5], [6]]. In the second stage, three CRIOs were invited to review the survey and suggest changes.

Topics covered by the survey included the CRIOs background and training; experience, time distribution and salary; reporting structure and funding; and future opportunities and challenges.


#

3.1.3. Analysis

Survey responses were analyzed descriptively and simple proportions are provided. The free text answers to open-ended questions were analyzed qualitatively by three of the co-authors (LNSP, ASMM, KSH) and the responses were grouped in high-level themes for interpretation.


#
#

3.2. Expert Panel

The Clinical Research Informatics Working Group convened an expert panel of CRIOs during the 2016 AMIA Annual Symposium. Three expert CRIOs were invited to participate in the panel: Peter Embi, MD, of The Ohio State University; Umberto Tachinardi, MD, of the University of Wisconsin-Madison; and William Barnett, of the Regenstrief Institute and Indiana University School of Medicine/Indiana Clinical and Translational Science Institute. The invited panelists responded to a set of structured questions designed to address some of the key success factors related to their CRIO roles, and they also answered questions that spontaneously arose from the audience during the course of session. The structured questions addressed four topics:

  1. factors associated with the success of the role;

  2. growing the research informatics enterprise; innovation and new technology; and

  3. data sharing and collaboration.

The panelists’ utterances were captured in the form of field notes and analyzed qualitatively by three of the co-authors (LNSP, ASMM, KSH). The responses were grouped in high-level themes for interpretation. The topics discussed and themes that arose are summarized in the results.


#
#

4. Results

4.1. Survey

A total of twenty-five CRIOs were identified through the search and were invited to participate. Sixteen completed the survey (64% response rate). The actual professional titles of the CRIOs had some variations, including: Chief Research Informatics Officer, Chief Research Information Officer, Chief Clinical Research Informatics Officer, and Chief Research and Academic Information Officer.

4.1.1. Background and Training

All of the CRIO respondents reported having at least one doctoral degree (either MD, PhD, or both) as their highest level of education. Half of the CRIO respondents were physicians (two of whom also had a PhD), and the other half were non-physicians with a PhD (►[Figure 1]).

Zoom Image
Chief Research Informatics Officers (CRIOs) at al glance. *Clinical time calculated only for physicians. **Salary distribution calculated using data from the CRIOs who provided that information.

88% of the CRIOs who were also physicians had an additional Masters-level degree. Out of the eight physicians, four were pediatricians or pediatric subspecialists, three were internists and one was a family medicine practitioner. Three of them had a board certification in Clinical Informatics through the American Board of Preventive Medicine.

One of the non-physicians PhDs had additional Masters-level training outside of their primary training. The academic background of the non-physician PhDs included computer science, molecular biology and bioinformatics, microbiology, archaeology, and biomedical informatics.

94% of all respondents had some formal educational training in biomedical informatics including graduate-level courses, Masters degrees, fellowships, and PhDs.


#

4.1.2. Experience

Half of the CRIO respondents reported more than 15 years of experience in biomedical informatics, however 69% had been in the CRIO role for less than 3 years (►[Figure 1]).

88% of the respondents were the first CRIOs to hold that position at their institutions. Nine respondents had prior leadership roles in biomedical informatics before becoming CRIO, four had other leadership roles outside of informatics and three had no prior leadership experience. The titles for the leadership roles held prior to the CRIO role included: Director of Biomedical Informatics, Director of Research Informatics, Director of Bioinformatics, Director of Translational Bioinformatics, Director of Biomedical Information Technology Core, and Chairman of the Department of Biomedical Informatics.


#

4.1.3. Organizational Structure and Challenges

According to the respondents, most CRIOs report directly to their academic health center’s Chief Information Officer (38% of respondents), Vice President for Research (38%), or the Dean or Vice Dean for Research of the Medical School (31%). 40% of CRIOs reported to more than one senior leader in their organization, with the combinations Chief Information Officer/Vice Dean and Vice President for Research/Dean as the most common. CRIOs had a median of 9 people reporting directly to them (interquartile range [IQR] 5 to 24 people).

The most commonly reported challenges faced by CRIOs in their organization included: securing funding (63% of respondents), dealing with data governance issues (56%), building data analytics capabilities (50%), leveraging electronic health records for research (44%), dealing with data privacy and legal issues (44%), and securing senior leadership support (30%).


#

4.1.4. Funding of Research Informatics

All the CRIOs reported that research informatics activities was funded at least in part as a cost center for the organization, that is, a center within the organization that generates value but no revenue. 69% also reported that Clinical & Translational Science Awards (CTSA) and Cancer Center grants were used to fund their activities. 63% had a charge back model for some of the research informatics services. 50% of CRIOs reported that their activities were funded with a combination of all three sources (organizational, grants, and charge backs).


#

4.1.5. Time Distribution and Salary

Respondents reported a median of 50% of time devoted to the CRIO role (IQR 30 to 70%). Other activities include: research (30%, IQR 15 to 45%), other administrative roles (10%, IQR 5 to 15%), and teaching (5%, IQR 5 to 10%). Physician CRIOs reported a median of 10% of their time devoted to clinical practice (IQR 0 to 10%) (►[Figure 1]).

Respondents reported that within their role as CRIOs most of the time was devoted to developing and managing infrastructures, such as data warehouses, high-performance computing, or biobanks (a median of 20% of their time); managing clinical research services, such as clinical trial management systems, i2b2, or REDCap (20%); developing and managing core services, such as bioinformatics pipelines, natural language processing, and ontologies (10%); managing data requests services (10%); data governance activities, such as committees or policy development (5%); and data security activities (5%).

When asked what they would rather be doing with their time as CRIOs, the most frequent themes in their responses, in order of frequency, included: working on innovation and new technologies, consolidating and expanding the research informatics capabilities and training in their organizations, increase multi-institutional and industry collaboration, improve advance analytics capabilities, and increase their role in their organization’s strategic planning.

Thirteen respondents reported their salary. The median CRIO salary reported was between $250,000 and $299,000 of total compensation. 31% of CRIOs reported a salary greater than $350,000 (►[Figure 1]). There was a correlation between the primary medical specialty and experience of the CRIOs and their salaries.


#

4.1.6. The Future of Research Informatics and the CRIO Role

CRIOs were asked to list three challenges and three opportunities in research informatics and their role as CRIOs in the next 5 years. For the challenges, the most common themes identified were: funding and sustainability; data policies, security and governance; integration of research informatics with the clinical enterprise; keeping up with technological and analytical advances; and dealing with lack of leadership support. Among the opportunities, the most common themes included: the Precision Medicine Initiative and the Cancer Moonshot; advances in Big Data analytics and data science; and increased data sharing.


#
#

4.2. Expert Panel

During the 2016 AMIA Annual Symposium, three experts were invited to discuss their experience as CRIOs and answer questions from the audience. The topics discussed included: key factors to succeed as a CRIO in the organization, the funding of research informatics, and the role of the CRIO in managing information technology (IT) services. What follows is a summary of these discussions:

4.2.1. Achieving Success as a CRIO in the Organization

A key factor to achieving success is to think of research informatics and the CRIO’s role as a service to the organization and to provide strategic IT leadership. Developing a detailed business plan and securing a firm commitment from the senior leadership is important.

One way to ensure organizational support is to align the research informatics activities with the strategic planning of the organization, which might include goals such as increasing competitive success in federal funding for medical research, improving services to the research enterprise, or developing precision medicine and Big Data programs.

An important factor to take into account is that the CRIO role crosses many boundaries, such as responsibility for data security and privacy, IT, research, clinical informatics, etc. Keeping in mind those boundaries and the individuals involved is important. In particular, one of the CRIOs highlighted how keeping a good alignment and purposefully avoiding conflict with chief information security and privacy officers can be tremendously beneficial. The role of honest brokering between the security and confidentiality of data needs of researchers is a key responsibility of the CRIO.


#

4.2.2. Funding Research Informatics and the CRIO Role

When establishing a new CRIO position, it is important to negotiate dedicated resources up front. Resources must include some seed funding with some ongoing investment overtime that will be supplemented by grants and charge backs. This supplemental revenue can be used to grow the research informatics enterprise, hire additional personnel, and increase the service lines, but it cannot take the place of some ongoing support from the organization.

In regards to the ongoing financial support of research informatics and the CRIO role as a cost center, one factor that should be considered is the opportunity cost around the risk of research data breaches. The mechanisms developed and managed by CRIOs to minimize the risk of research data and personal health information loss or misuse can represent a huge value for the organization, and this should be taken into account in the negotiations for ongoing support.

One final advice from the expert CRIOs in regards to the negotiation of ongoing support is to use the federal grant’s request for application requirements as a way to justify research informatics expenses. An organization that cannot meet the increasing research informatics requirements for federal grants will lose competitive advantage.


#

4.2.3. Management of IT Services for Research

There does not seem to be a universal model of the CRIO’s role in managing IT services for research. While some CRIOs are in charge of managing the technical infrastructure and data pipelines involved in research data warehousing, high-performance computing, and delivery of research software services, not all CRIOs do so. For example, some CRIOs might be in charge of managing the access and use of research data, while their organization’s Chief Information Officer (CIO) is in charge of maintain the IT infrastructures that support those activities. This overlap between CIO and CRIO varies widely between institutions.

One trend that all three experts agreed on is the move towards cloud-based services for the organization’s IT needs, including for research, and concurrent reduction in the need for local IT services.


#
#
#

5. Discussion

We have presented the results of a national survey of Chief Research Informatics Officers (CRIOs) and a summary of an expert panel discussion. CRIOs come from diverse backgrounds with an even split between physician and non-physicians. Most have advance training and extensive experience in biomedical informatics, including leadership roles, but most have been CRIOs for fewer than three years. Indeed, the first CRIO appointed in the US was Dr. Embi, who was named CRIO at The Ohio State University in the October of 2010.

The reporting structures of CRIO’s also vary, but seem to reflect their dual allegiances to both the medical center (with direct reports to the Chief Information Officer or Vice President for Research) and the medical school (with direct report to the Dean or Vice Dean for Research). Most CRIOs spend at least half of their time in their CRIO role and many of them maintain a substantial amount of research time for their own projects. CRIOs identify funding, data governance and security, and building advanced data analytics capabilities as their major challenges. If they could, they would like to be doing more work in innovation and new technologies, and establishing external collaborations with other academic health centers and industry partners. CRIOs seem to remain optimistic about the future and see the Precision Medicine Initiative and advances in data science as opportunities for growth of the research informatics enterprise and their role in academic health centers.

As shown in the survey, most current CRIOs are the first officers in their institutions to hold that role. CRIOs appear to fall into two major categories. In many cases, CRIOs are the research counterparts of the Chief Medical Information Officer (CMIO), a role that emerged in healthcare organizations more than two decades ago [[7]]. A recent AMIA Task Force reported its recommendations regarding the knowledge, education, and skillset requirements of the Chief Clinical Informatics Officer (CCIO) roles, which encompass CMIOs as well as the Chief Nursing Informatics Officers (CNIOs) and their pharmacy and dental counterparts, and sometimes fall under the purview of a Chief Health Informatics Officer (CHIO).[[8]] Some of these roles are now well established and standardization of their training and knowledge requirements is starting to take place. The CRIO role remains less well defined, but it can be expected that as the number of leaders in this role grows, standardization of their training and skillset will also take place. CMIOs and other CCIOs have played a key role in establishing operational informatics programs, implementing clinical information systems, and developing clinical decision support applications, all of which has contributed to laying the foundation of the digital infrastructure in healthcare [[5], [8]]. As the CRIO workforce grows and the importance of the research informatics mission increases, it can also be expected that CRIOs will have an equally impactful role in the future of healthcare.

CRIOs who span both the academic and clinical enterprises at academic health centers often have similar roles to CMIOs and other CCIOs, only focused on the academic health center’s research missions rather than the clinical ones. Alternatively, some CRIOs focus mainly on the operational research sides of the enterprise. In those cases, CRIOs function more like counterparts to academic CIOs, emphasizing infrastructure for basic, translational, and clinical research, but with less emphasis or oversight of clinical information systems’ use to support research. In all cases, CRIOs augment the capabilities and effort of their CCIO and CIO counterparts, advancing goals in of the academic health centers research mission, especially important in this era of rapidly evolving biomedical research opportunities including Big Data, advanced analytics, the emergence of large clinical data research networks, the evolution of clinical trials, and the push for precision medicine [[9]–[15]].

This paper has several limitations. First, the survey results include responses from a limited number of CRIOs. However, because of the small size of the workforce and the fact that almost two out of every three CRIOs identified in the US completed the survey, the results can be considered representative. Also, the expert panel discussion summary presented represents the opinion of three individual CRIOs, but because of their diverse background and extensive experience, their opinions can be considered very informative. A similar paper discussing the emergence of the CMIO role more than a decade ago had information from only five participants and yet it remains the most cited paper about the CMIO role, which signals its usefulness to readers [[5]]. Finally, not all the important topics related to the CRIO role were included in the survey or discussed in the expert panel. Topics that can be included in future surveys and studies about the CRIO role include: the working budget of the CRIO, measures of success and added value to the institution, and the role of the CRIO in biobanking strategies, amongst other. We hope this paper will be helpful for aspiring and newly named CRIOs, as well as the leaders in academic health centers who are considering creating a chief officer-level position to develop and manage their organization’s research informatics activities.


#

6. Conclusion

The role of the CRIO is emerging to tackle the challenges faced by academic health centers and clinical researchers in our increasingly digitalized, data-intensive healthcare system. CRIOs share many of the same challenges and see the same opportunities for the future of research informatics. Better understanding the background and experience of current CRIOs can help define and develop the role in other organizations and enhance their ability to improve healthcare through clinical research.


#

Multiple Choice Questions

Which U.S. academic institution had the first Chief Research Informatics Officer (CRIO)?

  • University of Wisconsin

  • Ohio State University

  • Oregon Health & Science University

  • University of Chicago

On average, how many people report to CRIOs in the surveyed institutions?

  • None

  • 1 to 5

  • 5 to 10

  • More than 10


#
#

Conflict of Interest

The authors declare that they have no conflicts of interest in the research.

Acknowledgement

The authors would like to thank the AMIA Clinical Research Informatics Working Group and its leadership, especially Judy Logan and Tony Solomonides, for their support.

* For the Clinical Research Informatics Working Group of the American Medical Informatics Association.


Clinical Relevance Statement

The role of the Chief Research Informatics Officer (CRIO) is emerging in academic health centers to address the challenges clinical researchers face in the increasingly digitalized, data-intensive healthcare system. To date there has been very little published information about this role and the individuals who serve it. Better understanding the background and experience of current CRIOs can help define and develop the role in other organizations and enhance their influence in the future of research informatics.


Human Subjects Protections

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the University of Missouri-Columbia and The University of Chicago Institutional Review Boards.


  • References

  • 1 Embi PJ, Tachinardi U, Lussier Y, Starren J, Silverstein J. Integrating governance of research informatics and health care IT across an enterprise: experiences from the trenches. AMIA Summits on Translational Science Proceedings 2013; 2013: 60.
  • 2 Embi PJ, Payne PR. Clinical research informatics: challenges, opportunities and definition for an emerging domain. J Am Med Inform Assoc 2009; 16 (03) 316-27.
  • 3 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) –a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-81.
  • 4 Mosa A, Yoo I, Parker J. Online electronic data capture and research data repository system for clinical and translational research. Mo Med 2014; 112 (01) 46-52.
  • 5 Leviss J, Kremsdorf R, Mohaideen MF. The CMIO –A new leader for health systems. J Am Med Inform Assoc 2006; 13 (05) 573-8.
  • 6 Shaffer V, Lovelock J. Results of the Gartner-AMDIS Survey of Chief Medical Informatics Officers. Stamford, CT: Gartner; 2010
  • 7 Hersher B. The essential skills for the chief medical information officer. J Healthc Manag 2003; 17 (01) 11.
  • 8 Kannry J, Sengstack P, Thyvalikakath TP, Poikonen J, Middleton B, Payne T. et al. The Chief Clinical Informatics Officer (CCIO): AMIA Task Force Report on CCIO Knowledge, Education, and Skillset Requirements. Appl Clin Inform 2016; 7 (01) 143-76.
  • 9 Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med 2015; 372 (09) 793-5.
  • 10 Embi PJ, Payne PR. Future Directions for Translational Informatics. In: Embi PJ, Payne PR. eds. Translational Informatics. New York, NY: Springer; 2015: 165-78.
  • 11 Grossman C, McGinnis JM. Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Washington, DC: National Academies Press; 2011
  • 12 Hripcsak G, Duke JD, Shah NH, Reich CG, Huser V, Schuemie MJ. et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform 2015; 216: 574.
  • 13 Selby JV, Beal AC, Frank L. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. JAMA 2012; 307 (15) 1583-4.
  • 14 Smith M, Saunders R, Stuckhardt L. et al. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2013
  • 15 Wang SJ, James Hung H, O’Neill RT. Adaptive patient enrichment designs in therapeutic trials. Biom J 2009; 51 (02) 358-74.

Correspondence to:

L. Nelson Sanchez-Pinto, MD, MBI
Division of Critical Care Medicine, Ann & Robert Lurie
Children‘s Hopsital of Chicago, 225 E Chicago Ave
Chicago, IL, USA
Phone: (312) 227–4800   

  • References

  • 1 Embi PJ, Tachinardi U, Lussier Y, Starren J, Silverstein J. Integrating governance of research informatics and health care IT across an enterprise: experiences from the trenches. AMIA Summits on Translational Science Proceedings 2013; 2013: 60.
  • 2 Embi PJ, Payne PR. Clinical research informatics: challenges, opportunities and definition for an emerging domain. J Am Med Inform Assoc 2009; 16 (03) 316-27.
  • 3 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) –a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-81.
  • 4 Mosa A, Yoo I, Parker J. Online electronic data capture and research data repository system for clinical and translational research. Mo Med 2014; 112 (01) 46-52.
  • 5 Leviss J, Kremsdorf R, Mohaideen MF. The CMIO –A new leader for health systems. J Am Med Inform Assoc 2006; 13 (05) 573-8.
  • 6 Shaffer V, Lovelock J. Results of the Gartner-AMDIS Survey of Chief Medical Informatics Officers. Stamford, CT: Gartner; 2010
  • 7 Hersher B. The essential skills for the chief medical information officer. J Healthc Manag 2003; 17 (01) 11.
  • 8 Kannry J, Sengstack P, Thyvalikakath TP, Poikonen J, Middleton B, Payne T. et al. The Chief Clinical Informatics Officer (CCIO): AMIA Task Force Report on CCIO Knowledge, Education, and Skillset Requirements. Appl Clin Inform 2016; 7 (01) 143-76.
  • 9 Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med 2015; 372 (09) 793-5.
  • 10 Embi PJ, Payne PR. Future Directions for Translational Informatics. In: Embi PJ, Payne PR. eds. Translational Informatics. New York, NY: Springer; 2015: 165-78.
  • 11 Grossman C, McGinnis JM. Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Washington, DC: National Academies Press; 2011
  • 12 Hripcsak G, Duke JD, Shah NH, Reich CG, Huser V, Schuemie MJ. et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform 2015; 216: 574.
  • 13 Selby JV, Beal AC, Frank L. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. JAMA 2012; 307 (15) 1583-4.
  • 14 Smith M, Saunders R, Stuckhardt L. et al. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2013
  • 15 Wang SJ, James Hung H, O’Neill RT. Adaptive patient enrichment designs in therapeutic trials. Biom J 2009; 51 (02) 358-74.

Zoom Image
Chief Research Informatics Officers (CRIOs) at al glance. *Clinical time calculated only for physicians. **Salary distribution calculated using data from the CRIOs who provided that information.