Maturity Level of Digital Reproductive, Maternal, Newborn, and Child Health Initiatives in Jordan and Palestine

Abstract

Background While there is a rapid increase in digital health initiatives focusing on the processing of personal data for strengthening the delivery of reproductive, maternal, newborn, and child health (RMNCH) services in fragile settings, these are often unaccompanied at both the policy and operational levels with adequate legal and regulatory frameworks.

Objective The main aim was to understand the maturity level of digital personal data initiatives for RMNCH services within fragile contexts. This aim was performed by choosing digital health initiatives from each country (two in Jordan and three in Palestine) based on RMNCH.

Methods A qualitative study design was adopted. We developed a digital maturity assessment tool assessing two maturity levels: the information and communications technology digital infrastructure, and data governance and interoperability in place for the five selected RMNCH initiatives in Jordan and Palestine.

Results Overall, the digital infrastructure and technological readiness components are more advanced and show higher maturity levels compared with data governance and interoperability components in Jordan and Palestine. In Jordan, the overall Jordan stillbirths and neonatal deaths surveillance initiative maturity indicators are somehow less advanced than those of the Electronic Maternal and Child Health Handbook-Jordan (EMCH-J) application. In Palestine, the Electronic Maternal and Child Health-registry initiative maturity indicators are more advanced than both Avicenna and EMCH-Palestine initiatives.

Conclusion The findings highlighted several challenges and opportunities around the application and implementation of selected digital health initiatives in the provision of RMNCH in Jordan and Palestine. Our findings shed lights on the maturity level of these initiatives within fragile contexts. The maturity level of the five RMNCH initiatives in both countries is inadequate and requires further advancement before they can be scaled up and scaled out. Taking the World Health Organization recommendations into account when developing, implementing, and scaling digital health initiatives in low- and middle-income countries can result in successful and sustainable initiatives, thus meeting health needs and improving the quality of health care received by individuals especially those living in fragile contexts.

Keywords

Jordan - Palestine - digital health - maturity assessment - reproductive health - maternal and child health - RMNCH

Introduction

Digital technologies provide tangible opportunities to tackle challenges in the health system and, thus, offer the potential to broaden the coverage and quality of health services.[1] Digital health technologies can also address the unique unmet needs of conflict-affected settings and overcome associated challenges and help achieve sustainable development goal 3 outcomes “Health for all.”[2] Digital health solutions in reproductive, maternal, newborn, and child health (RMNCH) are of particular importance. In recent years, there have been significant applications of digital technologies to strengthen RMNCH services through the use of mobile applications, sensors, wearable devices, and others.[3] [4] While digital interventions have targeted RMNCH services, a majority of them have been applied to childcare, pregnancy, health promotion,[5] [6] scheduling of appointments,[7] access to remote care and counseling,[8] [9] data collection and reporting,[10] and supporting peer–peer support groups.[11] In low- and middle-income countries (LMICs), an additional focus has been on family planning counseling using short message service, sending reminders, and doing data collection.[12] [13] [14]

Jordan and Palestine are relatively new entrants in this domain and are currently initiating different digital health initiatives. Yet, digital technology applications in RMNCH are relatively limited in both countries. There are four major RMNCH service providers in Palestine including the Ministry of Health (MoH), United Nations Relief and Work Agency (UNRWA), nongovernmental organizations, and private providers. The MoH provides almost all the services including primary, secondary, and tertiary services and purchases unavailable services from national health providers and from neighboring counties. UNRWA covers all Palestinian refugees in the West Bank, Gaza strip, Lebanon, Jordan, and Syria. Changes in international aid policies provide direct financial threats and adversely impact the sustainability of the overall health system including the digital initiatives.

According to the Global Digital Health Index, while Jordan has a fully functional governance structure around digital health strategies, their health workforce lacks adequate training in digital health. In general, there is good access to health services in Jordan. The MoH provides free primary health care services, including maternal and child health, immunization, and school health services. Comprehensive antenatal and postnatal care services are provided through both public and private health care settings, and today nearly 79% of pregnant women have at least seven antenatal care (ANC) visits.[15]

Another aspect of digital health is the health information system (HIS), which governments have invested heavily in for the purpose of service delivery, surveillance, reporting, and monitoring.[16] Nonetheless, the notion of HIS is still uncommon in most LMICs[17]; hence, developing an integrated HIS requires a significant time investment but ultimately benefits both programs and patients.[17]

The enthusiasm for digital health has led to an overwhelming rise in short-lived implementations and a vast diversity of digital tools, with a limited understanding of their impact on health systems, human rights, data privacy and security, and patients' health.[18] [19] Given the complex goals of digital health solutions, tailored, dynamic, and agile digital health maturity frameworks and approaches that can assess progress across diverse providers and settings are highly needed.[20] The overall purpose of maturity models in the health care zone is to assess and improve the maturity of health care infrastructure, practices, as well as operations.[21] [22] These assessments allow organizations to understand their readiness level to incorporate digital solutions, which is best achieved by assessing the compatibility and competence of information systems to interface both within and across organizations.[23]

Health care providers, policy makers, and researchers need to thoroughly screen all available health care maturity models to choose appropriate models that are suitable for their needs or develop maturity new models with further specialties.[24] Moreover, understanding the notion of maturity level of digital health initiatives will help countries to support and sustain their health priorities through aligning resources, knowledge, and skills to develop, implement, and evaluate standards-based interoperable digital health programs meticulously.[21] A recent review of 14 health care maturity models highlighted the need for a holistic approach to specialized or generalized maturity models that are contextualized within the national health priorities.[21] Furthermore, there is a need to reconceptualize a collaborative and systematic maturity-based approach to guide stakeholders and policy makers in investing, developing, implementing, evaluating, and sustaining digital health initiatives for identified health priorities and customized to local contexts.[25] Having a systematic and context-appropriate maturity-based approach is particularly important when there are challenges that hinder the variety of suitable digital tools used in areas of unreliable basic and information and communication technology (ICT) infrastructure, limited service capacity, and poor access and/or affordability.[25] Therefore, the main aim was to understand the maturity level of digital personal data (DPD) initiatives for RMNCH services within fragile contexts. This aim was performed by choosing digital health initiatives from each country (two in Jordan and three in Palestine) based on RMNCH.

Methods

Study Design

A descriptive qualitative study design was employed to assess the maturity level of selected RMNCH initiatives in Jordan and Palestine.

Tool

A desk search on all tools that could help in assessing the maturity level in digital solutions in general, with a focus on health sector, was undertaken in both Jordan and Palestine collectively. The researchers who conducted the desk search took into consideration the comprehensiveness of the reviewed maturity assessment frameworks but also were mindful of the “recent” onset of digital health initiatives in both countries. Thus, choosing tools that could help in providing a clear but realistic landscape of the current maturity level in the two countries. The reviewed tools ranged from simple to more advanced and complex, reflecting the maturity level of digital health initiatives. Other maturity tools developed for nonhealth-related initiatives were included and studied.

After a thorough review of all available international maturity tools of digital initiatives,[25] [26] [27] [28] we developed a revised maturity tool because no existing tool matched all our research objectives within the context of Jordan and Palestine. The developed tool included both the maturity level checklist and levels of incorporation of human rights and gender concerns in digital infrastructure checklist. For the purpose of this study, we only analyzed the data collected by the first checklist of the maturity assessment tool assessing two maturity levels: the ICT digital infrastructure, and data governance and interoperability in place for the RMNCH initiatives in Jordan and Palestine. In detail, the ICT digital infrastructure level has one dimension: ICT infrastructure (technological readiness). The ICT infrastructure domain has five components: hardware, reliable networks and internet connectivity, operations and maintenance for computer technology, and training and education including continuous professional development. The data governance and interoperability level has four dimensions: governance, legislation, policy and compliance, standards and interoperability, and data quality and use. There are four levels of scoring system in the developed maturity assessment tool (0, 1, 2, and 3). The grading is from 0 representing no/none to 3 representing yes/fully adequate. The full adopted maturity assessment tool is available in ([Supplementary Table S1], available in the online version). There is no total score for the developed maturity assessment tool but the score for each component represents a specific narrative to guide the researchers and respondents.

Initiatives Selection in Palestine and Jordan

First, all available RMNCH digital initiatives in Jordan (11 initiatives) and Palestine (15 initiatives) were identified. Only those initiatives, which concerned digitization efforts around RMNCH services, for both individual and aggregate data, were eligible for selection for further analysis. Thus, 20 (9 in Jordan and 11 in Palestine) digital RMNCH initiatives were considered. Two RMNCH digital initiatives were selected for empirical analysis in Jordan: (1) The Electronic Maternal and Child Health (e-MCH) mobile application at UNRWA in the primary care sector; and (2) the stillbirths and Neonatal Deaths Surveillance and Auditing System “JSANDS” in the hospital sector.[29] [30] In Palestine, three RMNCH digital initiatives were selected for empirical analysis: (1) Avicenna, a hospital-based initiative, (2) MCH e-registry, a primary health care initiative, and (3) The e-MCH mobile application at UNRWA in the primary care sector.

Avicenna and the MCH e-registry are two initiatives that are public, managed mainly by the MoH and cover a large area in Palestine. The UNRWA's e-MCH initiative is a common initiative between Jordan and Palestine. It will show whether there are any context-specific differences in the implementation and application of the initiative related to the location or infrastructure between the two countries. More details about the selected five initiatives are available in ([Supplementary Table S2], available in the online version).

Participants/Stakeholders

Maturity level assessment approaches in Jordan and Palestine involved identifying a variety of key stakeholders who are involved in the selected RMNCH digital initiatives and who could answer parts or all of the aspects included in the maturity assessment tool. These stakeholders included the following categories: developers, system designers, IT or technical support persons, policy makers and high-level stakeholders, implementers, and end users engaged in the collection and use of data. For each selected initiative, we identified the key respondents from each category first by contacting the focal points of each initiative whom we identified previously, an initial step of the same research project. In addition, we used a snowballing approach to identify additional key respondents to interview.

The total number of stakeholders who participated in this assessment in Palestine was 37: 13 of them for the Avicenna, 11 for the MCH e-registry, and 13 for the e-MCH application. The total number of stakeholders who participated in the maturity assessment in Jordan was 22: 10 of them for the JSANDS and 12 for the EMCH-J initiative. Subgroups from the list of identified stakeholders for each initiative were created to address them separately for data collection on their specific perceptions/experiences/impressions reflecting the nature of their involvement with each initiative. For example, for system designers, we focused more on technical issues of software platform, design inscriptions, security, and protocols, while for the end-user group, the focus was on understanding what data they need and what they receive, what gaps they see, how do they use it, and how is it perceived to shape RMNCH delivery.

Data Collection Methods

Data collection took place in both countries in the period from June to September, 2021. A variety of data collection methods were used in Jordan and Palestine to assess the maturity level of the five selected RMNCH digital initiatives. All individual interviews, group meetings, and focus group discussions (FGDs) were audio recorded, after taking the participants' consent.

Self-Administered/Individual Interviews

In Jordan, the developed maturity assessment tool was filled out individually by four to six designers, developers, and implementers for each initiative. The tool was sent to the identified stakeholders by email after an initial phone call to invite them to participate in the study and to explain the purpose of the maturity assessment. All participating stakeholders were asked to rate/score each item within the maturity level according to a scale from 0 representing no/none to 3 representing yes/fully aware for the JSANDS and the EMCH-J separately. The completed tool was sent back by email to the principal investigator in Jordan within three to five working days of initial contact.

In Palestine, each identified stakeholder was interviewed individually to be asked questions within the developed maturity assessment tool relevant to his/her position and qualifications. The interviews were mostly conducted face-to-face at the stakeholder's office, except for the UNRWA's EMCH-P interviews which were conducted remotely using Zoom technology. Each interview lasted between 30 and 45 minutes each.

Group Meetings

A total of five group meetings were held with relevant stakeholders who filled out the maturity assessment tool; one for each selected RMNCH initiative in Jordan and Palestine. The aims of these meetings were to present the main findings from the maturity assessment interviews, to better understand the context of the maturity level of these digital solutions for all stakeholders involved, and to discuss the answers and reach a consensus in case of disagreements or variances in answers during the individual interviews or the self-administered method. The group meetings were held in the presence of a researcher from the research team from each country to moderate and facilitate the discussion during the meeting. Each group meeting lasted around 2 hours, and they were conducted through Zoom and Microsoft Teams technologies.

Focus Group Discussions

A total of 10 FGDs were conducted in Jordan and Palestine. Four FGDs were held in Jordan: 2 for JSANDS and the other 2 for the EMCH-J, whereas 6 FGDs were conducted in Palestine, 2 for each of the three selected initiatives. The overarching aim of the FGDs was to verify/validate the maturity model scores and to deeply understand the maturity levels of the selected RMNCH initiatives in terms of infrastructure and capacities, data governance and interoperability, and data sharing and data flow process.

For each initiative, one FGD was conducted with designers, developers, IT support persons, and implementers, whereas the other one was conducted with the end users. For example, users of the JSANDS were nurses, midwives, and doctors working at maternity and neonatal wards in each of the five hospitals that participated in piloting the initiative, while users of the EMCH were nurses, physicians, and midwives working in UNRWA clinics. The reason for conducting FGDs with designers, developers, IT support persons, and implementers in a separate FGD is because we needed to focus on more technical issues of software platform, design inscriptions, and security protocols for system designers and developers. The FGDs with end users were needed to focus more on what data they needed and what they received, what gaps they saw and challenges encountered, how they used data, and how it shaped RMNCH delivery.

All FGDs were held online via zoom application due to COVID-19 lockdown restrictions with seven to eight members participating in each of the 10 FGDs. All FGDs were moderated by one of the research teams and were recorded for transcription verbatim with prior oral consent from all participants at the beginning of each FGD. The aims and objectives of the FGDs were clearly explained at the beginning of the discussion. FGD guideline was developed by the researchers to address the context of maturity level of the selected digital initiatives. Digital infrastructure in place for RMNCH and data governance and interoperability were the main topics discussed in the focus groups.

Data Management and Analysis

All recorded interviews and FGDs in both Jordan and Palestine were transcribed verbatim. The research team in both countries read the transcripts and identified the maturity scores for each component of the maturity level. The team extracted the scores from the responses of the interviewees and then confirmed these scores during the group meetings. The team was also able to extract some narratives from the interviews which included some additional details that are not within the tool and found interesting to add.

Ethical Considerations

Ethical approval was obtained from the research ethics committees at Birzeit University in Palestine and Jordan University of Science and Technology in Jordan. Consent was obtained from all discussants after the disclosure of the objectives, research methodology, data sharing and utilization policy of this research, anticipated risks, burdens and benefits, and institutional affiliations of the researcher for this research.

Results

A total of five RMNCH digital initiatives were assessed. Two initiatives were applied in hospitals, one in Jordan (JSANDS) and the other one in Palestine (Avicenna). For primary health care sector, three RMNCH digital initiatives were selected including EMCH-J, EMCH-P, and MCH e-registry. Overall, the digital infrastructure and technological readiness components were more advanced and showed higher maturity levels compared with data governance and interoperability components in Jordan and Palestine. In Jordan, the overall JSANDS initiative maturity indicators are somehow less advanced than those of the EMCH-J application. In Palestine, the MCH e-registry initiative maturity indicators are more advanced than both Avicenna and EMCH-P initiatives. [Table 1] summarizes the maturity level ratings for the five RMNCH digital initiatives in the two countries. The findings presented below begin with hospital-based initiatives followed by primary health care digital initiatives.

Table 1 Maturity level ratings for selected RMNCH digital initiatives in Jordan and Palestine

Category 1: Digital infrastructure in place for RMNCH
Dimension	Components	JSANDS	Avicenna	EMCH-J	EMCH-P	MCH-e registry
ICT infrastructure (technological readiness)	Reliable power/electricity	2.5	2	3	3	2
	Hardware	3	2	2.5	2	3
	Reliable networks and Internet connectivity	2.5	2	2.5	2	2
	Operations and maintenance (for computer technology)	2	2	3	2	2
	Training and education (includes continuous professional development)	2	1	3	2.5	2
Category 2: Data governance and interoperability
Dimension	Components	JSANDS	Avicenna	EMCH-J	EMCH-P	MCH-e-registry
Governance	Drivers for Data Governance Planning	0	0	2	NA	1
	Data Governance Planning	1	0	2.5	NA	0
	Governance structure	1	0	2	NA	1
Legislation, policy, and compliance	Data ethics and human rights	2	2	2.5	3	2
Standards and interoperability	Data and exchange standards	1	1	2.5	2	2
	Personal data exchange	0	0	2.5	2	0
	Aggregate data exchange	2	0	2	2	1
Data quality and use	Data management	1	0	3	NA	2
	Data use availability strategy	2	0	3	0	0
	Data synthesis and communication	1	1	2	NA	2
	Reporting and analytics features	2	1	1	NA	2

Abbreviations: EMCH-J, Electronic Maternal and Child Health Handbook-Jordan; EMCH-P, EMCH-Palestine; JSANDS, Jordan stillbirths and neonatal deaths surveillance; MCH-e, Electronic Maternal and Child Health; NA, not available; RMNCH, reproductive, maternal, newborn, and child health.

Maturity Level of Jordan Stillbirths and Neonatal Deaths Surveillance Initiative

Category 1: Digital Infrastructure in Place for Reproductive, Maternal, Newborn, and Child Health

The ICT infrastructure domain has five components: reliable power and internet, hardware, reliable networks and internet connectivity, operations and maintenance for computer technology, and training and education including continuous professional development. All participants scored “reliable power and electricity” as 2.5. This means that power operates in a redundant mode, power failures can be bridged by redundancy or backup power, and services remain continuous throughout power outages. However, participants commented that information collected by the JSANDS initiative on electricity access, sources, and reliability profile is not used for planning and continuous improvement of JSANDS implementations and for meeting emerging needs of the health sector.

When asked about the level and reasons of reliance on electricity power in Jordan to implement JSANDS in health care settings during the FGDs, developers and implementers of the JSANDS explained that there are two aspects to electricity. The first related to the electricity within the hospitals and the second related to databases and servers of JSANDS itself. According to the developers, both electricity powers are reliable enough to help in the implementation of the JSANDS with limited challenges. They commented that the servers are located inside a large governmental university (implementing partner) which has an uninterruptible power supply (UPS) and two other alternative sources of electricity. Moreover, the FGDs revealed that all hospitals that have been involved in implementing the JSANDS have also involved in a national accreditation program that ensures 100% coverage of electricity in all wards and places in hospitals and a quick back-up plan in case of sudden electricity cut. During the FGDs, the developers assured that we can rely on electricity power even in less-advantaged places with poor infrastructure as electricity is highly stable. The users added that all MoH hospitals have generators in case of emergency situations with regular monitoring and follow-up.

Hardware of the JSANDS system was scored 3 on the maturity level, reflecting that all of participating hospitals have adequate computing hardware to support the JSANDS. Also, hardware is monitored and evaluated regularly to ensure supported functions are operational. Similarly, the developers explained during the FGDs that the servers are centralized in the university and do not need to be available in hospitals. Servers can also be centralized at the MoH with no problems. Developers also mentioned that even private hospitals in Jordan that provide RMNCH services are fully equipped with adequate infrastructure. On the contrary, the users complained that they only have the minimal number of computers and printers, and they certainly need more if the JSANDS is scaled up.

Reliable networks and internet connectivity were scored 2.5, indicating that the JSANDS has reliable network and internet (4G) and approximately all of the hospitals have access to the Internet. This score also implies moderate-to-high broadband service subscriptions, less than or equal to 20:1 contention ratio, and services available in-house. Likewise, during the FGDs, the developers assured a guaranteed, stable, and high availability internet provided for both customer level and hospital levels in Jordan. One example was provided during the FGDs concerning the successful national health automation on a national level such as the electronic medical record without any significant network failures (downtime is too little that reached one in 10,000 per year). The FGDs also revealed that the reason for the high availability throughout Jordanian hospitals is that hospitals have at least two different internet sources from different companies, which makes downtime almost impossible. However, the users said that the network and internet connectivity depend on the provider, and it is generally weak. Users also added that they only have access to the intranet not the external network. Internet connectivity is only available in the administration of each hospital.

In regard to operations and maintenance for computer technology, a score of 2 was obtained and was confirmed by participating in the FGDs. This score infers that computer operations and maintenance services are part of the JSANDS and a recovery plan is in place, and it meets best practices with existing onsite backups but no offsite backup. However, this score reflects the lack of continuous revision and adaptation of the operations and maintenance services plan for evolving the JSANDS interoperability requirements. The FGDs revealed adequate operation and maintenance services for networks and hardware. Specifically, developers mentioned that hardware and servers were already available at the hosting university and virtualized and did not need maintenance. However, there were minimal challenges in some hospitals about maintenance. Therefore, the developers and implementers suggested that a team of IT specialists and engineers from the MoH need to be established and assigned for following up with the maintenance of hardware and networks. The users also assured that the IT specialists and engineers of the JSANDS were very responsive both physically and by phone to any problem encountered with the hardware or network connectivity provided for each of the hospital that implemented the JSANDS.

As for the training and education, the JSANDS was scored 2, which entails standardized training, academic curricula, and processes for developing training and education programs to build skills and competencies nationally with clear, well-defined, and measurable learning outcomes for training courses. However, according to this maturity level, the JSANDS has insufficient infrastructure to support periodic and different modes of training such as distance training to both refresh existing skills and impart new skills. these programs lack regular revision to ensure alignment with the JSANDS needs and technology. When asked about whether the JSANDS has established training and education programs such as curriculum and learning outcomes, implementers in the FGDs clarified that JSANDS team has developed educational programs and brochures that can be further developed and refined to include educational course syllabus with clear learning outcomes. Implementers added that they used face-to-face approach to hold adequate training sessions for focal pints at hospitals. They also assured that the training manual was very simple, user-friendly and detailed that anyone can even learn by just following the instructions.

Furthermore, the developers explained the use of a systematic scientific approach for the development cycle called the Agile methodology that has several phases including business analysis and requirement gathering, design development and approval, then prototype and quality check, and finally the production stage. Kick-off meetings were initiated at the initial stages of developing the plan for JSANDS, and outcomes were utilized to have a clear and data requirement as early as possible. The implementers during the FGDs added that they were clear from the beginning about the aims of the JSANDS and the availability of research statistics at that time about neonatal mortality in Jordan but were based on paperwork and was not comprehensive and inconsistent. The users mentioned that the JSANDS team approached them at the very beginning and discussed available data and registration process as well as needed modifications to obtain reliable, comprehensive, and trustworthy data about neonatal deaths and stillbirths. They also mentioned the data they suggested to be added to the JSANDS such as the Apgar score. As per the web site, the developers said that developing the web site was a step-by-step joint effort of both implementers and developers. The only challenge was regarding the changes required of the design after piloting the JSANDS, which needed a whole modification from the source.

Category 2: Data Governance and Interoperability

This domain has four dimensions: governance, legislations, policy and compliance, standards and interoperability, and data quality and use. The first dimension includes drivers for data governance planning, data governance planning, and governance structure. Drivers for data governance planning was scored 0, reflecting no risk assessment was done. A score of 1 was given to the data governance planning. This score means that there is an established planning process which involves key eHealth stakeholders and is formally approved by the MoH. It also implies that there is a current plan that includes standards, legislation, appropriate technical and service delivery aspects, as well as ensuring there are financial and human resources to deliver them.

The governance structure was also given a score of 1, highlighting two main aspects. The first is that the governing body of the JSANDS is formally constituted and has a scope of work that includes the people responsible for data governance oversight. The second aspect is that the governing body of the JSANDS oversees interoperability directly or through a separate technical working group. However, according to the maturity level, the governing body of the JSANDS needs to be government led, consults with other ministries, as well as monitor the implementation of the JSANDS interoperability using a work plan. Most importantly, the JSANDS needs to mobilize financial, human, and political resources to accomplish its goals. The developers during the FGDs detailed several levels of governing bodies. The first is the project team, death review committee team, the technical committee, and the steering committee that approves every step of the development and design of the JSANDS. On the contrary, the users in the FGDs were not exactly sure about the governing body of this initiative.

The second dimension in the data governance and interoperability is legislation, policy, and compliance. This dimension has one component, data ethics and human rights, which was rated 2. This score suggests that the health data security and privacy laws have been implemented, and there are guidelines on how to operationalize the laws in the JSANDS. This score also reflects that the JSANDS users have been sensitized and trained on the data security and privacy laws. However, this score highlights the lack of a recognized national mechanism (e.g., committee) to review issues related to data ethics and update policies and laws as needed. The FGDs also shed light on the presence of health data regularity framework for the JSANDS. The developers and implementers of the JSANDS clarified that there were different policies specific to the JSANDS system and its web site such as the privacy policy, backup policy, and data sharing policy that details who owns the data and who and how data can be shared. All these published policies were amended as per feedback from users, who agreed that the JSANDS has a clear health data regulatory framework such as the web site and related policies. However, the users in the FGDs shared that they were able to see other hospitals' reports and data before JSANDS' modifications.

The third dimension of data governance and interoperability is the standards and interoperability. Personal data exchange, data and exchange standards, and aggregate data exchange components were rated 0, 1, and 2 respectively. A score of 0 given to personal data exchange component denotes a limited capability to support standardized patient data exchange between systems within local or specific implementations. The score also reflects that data exchange is based on peer-to-peer interfaces. A score of 1 given to data and exchange standards reflects the development of JSANDS standards regarding data sharing policy for health data management and exchange. However, these standards and policies are only localized to JSANDS with some national efforts to standardize the data sharing process. To obtain a higher level, the JSANDS needs to have a documented national policy and schema to transfer and export data in a structured way. A score of 2 given to aggregate data exchange represents that applications could automatically exchange aggregate data, using defined standards at all levels but with no robust maintenance. The developers said that the MoH owns the data and share it. They added that the JSANDS team was able to access these data with strict compliance with data sharing and privacy policies. However, in the early stages of implementation, all users had access to all data entered through all hospitals. However, this issue was amended so only users who have the mother's national ID can retrieve data with a limited number of authorized users; hence, the JSANDS system supports personal data exchange within limitations.

Another important point regarding data exchange was raised during the FGDs that the JSANDS was built and designed to allow data exchange of and integration with the department of civil status through application programming interface integration. The users, on the contrary, were not aware of data exchange and sharing policy of the JSANDS. The FGDs also raised the issue of whether the system supports exchange aggregate data and reporting processes. The developers explained that the system supports exchanging data aggregate by key performance indicators. For example, the system has a report formatted in PDF that includes all data, which then can be shared with other authorized end users and death review committee members. The JSANDS design can also accommodate for a large size of data from different hospitals and can also be scaled up as infrastructure, and the database license can be upgraded to allow huge aggregation. The same case applies in data reporting because the database administrator can tune the database to improve performance and allow for large data reported on the JSANDS.

The fourth dimension in data governance and interoperability is data quality and use. Data management, data use availability strategy, data synthesis and communication, and reporting and analytics feature components were rated 1, 2, 1, and 2, respectively. A score of 1 was given to data management suggesting that data management processes of the JSANDS are clearly documented in a nationally recognized and established mechanism to ensure the quality of data being transmitted across health system levels. A score of 2 was given for the data use availability strategy component. This score represents that the JSANDS has achieved a long way in this aspect in a way that the implementation of the data use strategy is monitored, reviewed, and overseen by an established governing body for data review. It also implies that data are shared with stakeholders in accordance with standards and within regulatory frameworks. Data synthesis and communication component was scored only 1 indicating that there is available documented guidance on the design and use of information products that highlight gender issues.

Finally, the reporting and analytics features component was given a score of 2 inferring that automated data reporting is implemented nationally from point of service on a routine basis and that features to support data discovery, integration, analysis, and visualization exist at all levels. Yet, according to the maturity model, the JSANDS still needs to utilize metrics on reporting and analysis capabilities with feedback from users for continuous improvement.

Similarly, the developers, implementers, and users during the FGDs agreed that there is a clear data backup policy with a clear plan. Specifically, participants said that the JSANDS is available on virtual machines allowing for daily, weekly, and monthly backup of data to prevent data loss. This is particularly important when data are becoming huge over the years allowing for building business intelligence and algorithm report that can be used in the automation of clinical pathways. Nonetheless, the developers highlighted the issue of the lack of a clear data use strategy for the JSANDS. They suggested to build authority matrix according to the level of data sensitivity and to be integrated with data usage policy. The real challenge according to participants is to allow for data sharing within the same hospital and across hospitals and institutions while maintaining privacy and confidentiality. The developers also explained the data flow process, in which the flow occurs within the hospital according to authority level.

Maturity Level of Avicenna Initiative

Category 1: Digital Infrastructure in Place for Reproductive, Maternal, Newborn, and Child Health

Apart from training and education, which were scored 1, all components of ICT infrastructure were rated as level 2. Electricity and power access are reliable in Palestine as each hospital has generators and an UPS system to ensure that the initiative is functioning all the time. Although almost all hospitals have adequate computing hardware to support the Avicenna, the system faces a problem with the renewal of old devices and hardware because of financial issues. Further, the FGDs revealed that hardware is not monitored and evaluated regularly to ensure that supported functions are operational, and the system does not operate with redundancy.

Avicenna does not require an internet connection. It depends on data lines that are connected with central servers. Rating operation and maintenance were complicated for the different stakeholders. The leading company (DataSel) owns the technical issues including maintenance or updates. Computer operation and maintenance service are part of the Avicenna with a disaster recovery plan in place. Unfortunately, training and education are the weakest part of the digital infrastructure component. Training of the IT staff took place when first it was implemented in Palestine and Turkey. These people were responsible for training new staff and colleagues. Similar training was provided to the end users at the beginning, and they became responsible for training their new colleagues. Further, neither there was a clear plan for the training nor were the clear learning outcomes defined. As a result, training and education programs for Avicenna are at the initial stage of development.

Category 2: Data Governance and Interoperability

The policy makers were able to rate the different parts of data governance. However, the IT staff and end users were not able to give an accurate rating. The policy makers rated drivers for data governance planning and governance structure as 0, highlighting the need to develop a plan and a governing body for all digital health initiatives since the government has endorsed digitalizing the existing programs. Currently, the IT and the information management centers at the MoH are the two bodies responsible for the data management of the current initiatives.

Data ethics was rated 2. There is a national data security regulation, some security checks built within the Avicenna initiative. However, there is no clear specific, approved data regulatory framework for Avicenna. In addition, there is no ethical committee or regulatory body to follow up on implementing the regulatory framework. Data and exchange standards were rated 1.5. The initiative follows the “health level seven—HL7” international standards for data exchange—all data exchange is controlled by the minister. Personal data and aggregate data exchange were rated 0 as there is limited capability to support standardized patients or aggregated data exchange between systems.

Stakeholders mentioned several efforts to improve data quality and use. However, data management and data use strategy were both rated 0. The data management process, although known for the different stakeholders, was not documented. Similarly, the data use strategy was absent. As for data synthesis and communications, which was rated 1, it is integrated into the HIS and health plans. Still, the general target audience and intended results of information products are not explicitly documented. Specific data synthesis and communication plans exist for health professionals using specific dashboards. Finally, reporting and analytic features were also rated 1. This means that some efficiency measures have been implemented in the reporting process by consolidating reporting tools, electronic submission of routine reports from subnational levels exists in limited settings, and analytical features are in separate statistical software applications.

Maturity Level of United Nations Relief and Work Agency Electronic Maternal and Child Health Handbook in Jordan

Category 1: Digital Infrastructure in Place for Reproductive, Maternal, Newborn, and Child Health

The maturity level rating for the reliable power was scored 3 and hardware was scored 2.5. This rating implies that the information collected on electricity access, sources, and reliability profile is used for planning and continuous improvement of implementing the EMCH-J and to meet the emerging needs of the health sector. It also means that power operates in a redundant mode, power failures can be bridged by redundancy or backup power, and services remain continuous throughout power outages. This was confirmed by developers, implementers, and users of the EMCH-J in the FGDs, who assured that they can rely on the electricity power 100% to implement this initiative. The score of 2.5 for hardware denotes that almost all of facilities have adequate computing hardware to support the EMCH-J. This score also signifies that hardware is monitored and evaluated on a regular basis to ensure supported functions. Nevertheless, the systems do not fully operate with redundancy, because the disaster recovery plan is not fully implemented yet. Likewise, developers, users, and implementers assured that all hardware including computers, servers, and printers were provided to all UNRWA clinics and health centers. They added that the EMCH-J application is centralized with one server, and it can be downloaded through Google play or App store.

The maturity level rating of reliable networks and internet connectivity was 2.5, inferring that approximately all the health facilities have access to the internet. Still, high broadband service subscriptions are not in place suggesting moderate broadband service subscriptions. In addition, services available with quality improvement are not fully implemented. The FGDs with the developers of the EMCH-J revealed that network connection is somehow slower and disconnects sometimes in places where internet connectivity is powered by 3G not 4G. Having said that, the developers and implementers mentioned that the EMCH-J initiative can also work offline and is updated automatically once the mother connects to a Wi-Fi network either at home or at the clinic. The users at the clinics mentioned that the 4G in their clinic is restricted to the use of EMCH-J application but no access is granted for other applications such as Facebook or YouTube. This configuration allows for adequate gigabytes for the use of EMCH-J.

The operations and maintenance of computer technology component was scored 3. This high maturity level means that the operations and maintenance services plan is continuously reviewed and adapted to evolving the EMCH-J interoperability requirements and follows industry-based standards. The score also suggests that regular simulations are undertaken to increase the ability of IT staff to respond to a disaster and that onsite and offsite backups exist. Likewise, developers, users, and implementers explained that they have a technical team in Gaza responsible for the support, maintenance, and new updates for everything related to the EMCH-J. Also, there is another support team in Amman that responds to and intervenes in cases of application failures or other technical issues. The developers added that the EMCH-J application has been updated with approximately four new versions since it was launched in 2017 taking into considerations mothers' (end users) feedback and suggestions. Finally, the users and implementers in the FGDs assured that both teams in Amman and Gaza communicate regularly to solve any issue as it arises.

The maturity level rating of training and education component of the EMCH-J was 3. This high score reflects the fact that training and education plans are integrated into the EMCH-J implementation plans and the results are measurable. This rating also denotes that training and education programs are conducted and reviewed periodically to refresh existing skills and impart new skills as well as to ensure alignment with the EMCH-J needs and technology with sufficient infrastructure to support different modes of trainings. The implementers and users clarified the process of its TOT training and education which included selecting focal points in each clinic to monitor the education and training as well as training senior nurses in the clinics, who then trained their colleagues. Moreover, the implementers and users mentioned that there was also a clear training manual and an educational video on how to use the application. Mothers were also provided with educational brochures and videos on how to download and use the application and its aims and benefits. Developers and implementers also stressed on the availability of regular, tailored on-job training of the EMCH-J including training on new updates.

As for the process of developing the EMCH-J mobile application, kick-off meetings were held between Japan International Corporation Agency and UNRWA headquarter to agree on roles and responsibilities, followed by focus groups to discuss how to best develop the EMCH-J and its content. Surveys on mothers' use of smartphones and their type were conducted as well during the process of development. After consensus was reached on the data to be included in the application, pilot launching of the application was conducted then rollout followed throughout clinics in Jordan. The developers added that FGDs with women who used the application were held to seek feedback and suggestions. The developers and implementers revealed a clear plan of project planning including assessment, development, and implementation. However, users at the clinics revealed uncertainty of the plan of EMCH-J development.

Domain 2: Data Governance and Interoperability

Both the drivers for data governance planning and the governance structure components were scored 2. This rating shows that formal risk assessment was done but did not inform the design, implementation, and maintenance of the EMCH-J. The data governance planning score (2.5) indicates that a budgeted plan is aligned with and integrated in the national health plan/strategy as well as implementation is monitored and there is a set schedule for periodic review and update for corrective action. Nonetheless, a continuous improvement planning process of the EMCH-J should be maintained to optimize the data governance planning.

The data ethics and human rights component was rated a score of 2.5, indicating that the health data security and privacy laws have been implemented in the EMCH-J, and there are guidelines on how to operationalize the laws in the EMCH-J. Developers and implementers mentioned that there is a governing body for the EMCH-J. They classified the EMCH-J initiative as part of the MCH programs in UNRWA, thus anything that applies to the MCH programs also applies to the EMCH-J. Specifically, the implementers assured that there is a committee consisting of the medical technical team and application development and updating team, who meet irregularly based on needs. They also indicated that there is a feature within the application where women ask questions and receive answers online. The questions are usually discussed among the committee members. The developers and implementers also revealed that there is a professional team for the EMCH-J initiative based in Gaza to verify policies based on e-health data type stored in this mobile application. The stored data do not include women's personal pictures or thoughts. If further data are needed, institutional review board approval is needed to ensure data protection and privacy. Similarly, regulations on how to deal with health data, especially sensitive ones, at UNRWA were highlighted during the FGDs as very important and imbedded within privacy and confidentiality policy.

Data exchange standards and personal exchange components were rated 2.5. This score entails that the EMCH-J electronic data transmission is the default method for moving data among systems, facilities, and information systems. However, there is no documented national policy and schema in place to transfer or export data in a structured way. Personal data exchange score highlights that minimal effort is required to realize data exchange within the EMCH-J. Also, this rating means that the EMCH-J is implemented mostly on plan with requisite interoperability quality, yet still lacks a documented schema to transfer or export data in a structured way.

Focus group participants mentioned that data exchange standards are available within the data protection and data privacy policy, and that data exchange occurs within internal systems at UNRWA and is updated daily. Implementers of the EMCH-J clarified that there is no large data stored in this mobile application as it is a read-only app which can access mother's health records and educational information such as preconception care, ANC, postnatal care, and child immunization. Women can send questions through this app and can read all medical advice uploaded there. Mothers can also access appointments, where a reminder is sent to her a day, an hour, and 5 minutes before. The users also said that even mothers can share some information on the app—without revising it—to other women or other people on different social media applications. Mothers can even share her maternal history data with her private doctor if she prefers to go there.

Aggregate data exchange component was given a score of 2, indicating that the EMCH-J could automatically exchange aggregate data, using defined standards at all levels. However, to be transformed, all data exchanges within the EMCH-J should be based on certified interoperability standards with adequate documentation.

The fourth dimension of data governance and interoperability is data quality and use. Both data management and data use availability strategy components were rated 3. This implies that standard operating procedures for health data management are periodically reviewed and revised to ensure alignment with evolving health data needs. Similarly, the data use strategy of the EMCH-J is adapted to meet emerging decision-making needs of program managers, policy makers, and providers interacting with other HISs, and it is integrated in the long-term health plan to promote a culture of data use. Likewise, the developers and implementers ensured that data management process of the EMCH-J is reviewed and therefore revised on regular basis. However, they explained that analyses of data are conducted periodically to identify patterns in usage and/or risk factors in certain clinics, thus aids in designing tailored interventions. Furthermore, the FGDs revealed that there are specific technical instructions for the EMCH-J that give certain privileges of data access to certain people according to their role and job title. All these data are then utilized by the statistician officer in the main office at UNRWA to help in writing the annual reports.

Data synthesis and communication component was rated 2. As for the data sharing and data flow inside each clinic, implementers, users, and developers identified that demographic data are collected on the pregnant women by the admission office then the midwife can access these data and write her own notes. Then the electronic file is transferred to the medical doctor who has access to all data. As for the data flow between clinics and with the main head office in Amman, it is based on the program division. Reporting and analytics features component was only scored 1. Developers and implementers commented that the reporting process of the EMCH-J is not optimal and irregular. They clarified that some reports that are extracted from the e-health system are sent quarterly from the clinic to the main office in the region.

Maturity Level of United Nations Relief and Work Agency Electronic Maternal and Child Health Handbook in Palestine

The results of the UNRWA EMCH-P digital initiative are somehow not complete. The development of the initiative and the management of data are run mainly from the Amman Head Quarter. The interviewed stakeholders, in both the group meetings and the FGDs, were mainly users and implementers and hence could not rate the maturity level of certain items. It is worth mentioning that the initiative was piloted in Jordan[31] and then scaled up to other UNRWA four regions (Jordan, Syria, Lebanon, West Bank, and Gaza).[32] The COVID-19 pandemic has had an impact on the implementation of the initiative and this justifies some of the responses mentioned below.

Category 1: Digital Infrastructure in Place for Reproductive, Maternal, Newborn, and Child Health

All UNRWA clinics have reliable power and electricity. A backup generator is available in most clinics, and there are plans to provide generators to the remaining clinics. The staff are also trained to switch into the paper system in an emergency situation and then refill the information into the electronic system when the situation is back to normal. Reliable power/electricity was rated 3, indicating that information collected on electricity access, sources, and reliability profile is used for planning and continues improvement of EMCH-P imitative implementation. It also means that power operates in a redundant mode, power failures can be bridged by redundancy or backup power, and services remain continuous throughout power outages. All clinics have adequate computing hardware to support the EMCH-P digital initiative. However, the application part depends on the availability of smartphone with women and this was not the case for all women, and hence, it was rated 2. The implementers and users during the FGDs added that hardware is not monitored and evaluated regularly. Reliable network and internet connection given the nature and services in the camp were rated 2 and were confirmed by participating in the FGDs. Implementers revealed that the linkage between the EMCH-P digital initiative and the application is interrupted because of poor internet. The operation and maintenance services are available in-house within UNRWA West Bank, and they are part of the digital initiative. Participants also mentioned that a disaster recovery plan for the EMCH-P is in place, and it meets the best practice. The operations and maintenance were rated 2. UNRWA has a strong continuous education program, and the EMCH-P has similar continuous education system. The staff have received intensive training at the beginning when the initiative started. Clear training programs for new staff and for the current staff on new updates for the initiatives are available. The stakeholders scored training and education as level 2.5, as they do not have an idea about the higher level plans for these initiatives.

Domain 2: Data Governance and Interoperability

Data governance and interoperability were difficult to answer by the different stakeholders. The interviewed stakeholders could not rate drivers for data governance planning, data governance planning, governance structure, data management, and data sharing. The other areas were rated at the clinic level rather than the whole initiative level. Data ethics and human rights were rated 3 as the UNRWA follows UN standards. As for the EMCH-J, the participants mentioned that the EMCH-P has a committee at the headquarter level for reviewing data ethics issues and for updating polices as needed. Finally, all standards and interoperability components including data and exchange standards, personal data exchange, aggregate data exchange were rated 2. These scores were almost the same as the EMCH-J standards and interoperability scores. The FGDs participants shared the same views as the EMCH-J participants.

Maturity Level of Electronic Maternal and Child Health Registry Initiative in Palestine

Category 1: Digital Infrastructure in Place for Reproductive, Maternal, Newborn, and Child Health

All stakeholders rated reliable power and electricity as level 2. The power infrastructure is available at the subnational and national levels and monitored regularly. The end users reflected their concerns when the electricity cuts down while using the program because there are no generators available in the small clinics. During FGDs, participants revealed that data and information collected about electricity including access, source, and reliability profile are neither used for planning and continuous improvement of MCH-e registry nor to meet emerging needs. They added that power does not operate in a redundant mode. Almost all facilities have adequate computing hardware to support the digital initiative, and this hardware is monitored and evaluated regularly. Hardware was rated 3 and networks and internet connectivity were scored 2. All stakeholders agreed that there are reliable networks and internet. Three types of networks are used because some clinics are located in area C, which has several political restrictions from the Israeli side.

The operations and maintenance were rated as level 2, which is part of the digital initiative where the MoH IT staff and the Palestinian National Institute of Public Health (PNIPH) staff were involved in the initiative development and maintenance. The main challenge in the maintenance part is the shortage of specialized staff. The participants highlighted that there is no operations and maintenance services plan, and there are no offsite backup systems. The operations and maintenance score of 2 also indicated that no actions (i.e., simulations) are undertaken to increase the ability of IT staff to respond to emergencies.

Training and education component was rated level 2. A training and education program for this initiative was at the initial stages of development and implementation. The training took place at several levels. The training started with the MCH supervisors and the IT staff in all clinics in each directorate. Specific training guidelines were distributed during the training. The supervisors trained current medical staff and the new team was trained by their colleagues.

Category 2: Data Governance and Interoperability

Most components of data governance and interoperability were rated between 0 and 2. In terms of planning, there is an established planning process supported by the PNIPH. The policy makers rated the drivers for data governance and planning component as level 1. Moreover, they rated data governance planning as level 0. During the FGDs, participants mentioned that there is no formal planning process which involves key eHealth stakeholders. The participants of IT and end users could not rate this level and acknowledge their limited data planning and governance information. The governance structure was rated 1 by all stakeholders. Although the MCH e-registry data are hosted within the MoH and supervised by the PNIPH, a clearly defined formal governing body for the MCH e-registry was not reported.

Data ethics was rated 2. The initiative has built-in security and privacy measures. The data use several security checks that are built into the system to ensure privacy. The IT staff and developers reported the data security law but could not relate if the initiative reflected this. The policy makers and end users referred to the Public Health Law, which covers issues related to ethics but was not sure if there is anything specific to digital initiatives.

The MCH e-registry initiative follows the “HL7” international standards, and the data exchange standards component was rated 2. The national data management and exchange standards are integrated into the national HIS. However, stakeholders reported that the system cannot implement data exchange. The IT staff said there is a vertical system of data exchange between end users and policy makers and implementers. Personal data exchange and aggregated data exchange were rated 0 and 1, respectively. Most stakeholders had limited information about this component to judge. However, data management was rated as standard operating procedures for national health data management and integrated into the national HIS.

As for data quality and use, all components were rated 2, but data use strategy was lacking and hence was rated 0. This score reflects that reporting and analytics features are limited and key health indicators are not standardized, and it occurs on an irregular basis. Also, the reporting process of the MCH e-registry is still undertaken manually.

Discussion

Our findings show that the digital infrastructure and technological readiness components are more advanced and show higher maturity levels compared with data governance and interoperability components for the five digital initiatives in the provision of RMNCH in both Jordan and Palestine. In Jordan, the overall JSANDS initiative maturity indicators are somehow less advanced than those of the EMCH-J initiative. In Palestine, the MCH e-registry initiative maturity indicators are more advanced than both Avicenna and EMCH-P initiatives.

Our findings also show that the selected initiatives were somehow mature in ICT infrastructure despite some complaints of poor internet connections in fragile areas and lack of regular hardware monitoring and evaluation. In specific, while the JSANDS and Avicenna have adequate hardware to support implementation, the two initiatives have the lack of regular hardware operation and maintenance services. This is more evident in Avicenna that has been scaled up to all governmental hospitals in Palestine. Certainly, more hardware is needed if the JSANDS initiative is planned to be scaled up on a broader aspect. Similarly, the UNRWA E-MCH in Jordan and Palestine have reliable electricity power and adequate computing hardware. However, due to the COVID-19 pandemic, this initiative has not been fully mature in Palestine as it has in Jordan, especially in governance and interoperability components. It is also plausible to consider Jordan as the pilot phase of the UNRWA initiative and the expansion to Palestine is the scale-up phase and it is still in its early stages. To overcome any potential infrastructure defaults, alternative precautionary systems need to be installed to deal with such situations.[1] This backup was witnessed in most of the digital health initiatives that we assessed in case of sudden power outrage except for the MCH e-registry. Such systems include enabling the digital health system to function without internet connection or national electricity.[1] Avicenna is an example of a digital solution that does not require the internet to operate but the rest of the studied initiatives require the internet. Similar to the case in Jordan and Palestine, the availability of high-quality hardware in some LMICs was also identified as an obstacle within the local ecosystem that may limit program scaling-up.[33] Hopefully, over the next decade, there is a great potential for rapid increases in internet access and more digital innovations, hence helping in achieving sustainable development goal 3.[33]

One issue that we need to shed light on is the root cause of inadequate monitoring, maintenance, and renewal of hardware. In Avicenna, for example, financial and political issues including interrupted funding and changes of funding are the reason for the inadequate renewal of hardware. A study conducted in Nigeria revealed that the track record of private–public partnerships (PPP) and a supportive policy environment, facilitated adoption of health technology.[34] If the public sector faces shortages in resources and experienced workers, the private sector may play a pivotal part in offering funding, human resources, and support with infrastructure.[35] For example, Avicenna's software was developed by a Turkish private company and was part of a huge 5-years project with no clear plan for financial and technical sustainability. This PPP has a significant role in scalability, sustainability, and maturity of the digital health initiatives.[35]

Another aspect of ICT infrastructure is training and education to all stakeholders involved (health care workers, implementers, and end users). Apart from Avicenna, in which the training and education was very weak, the other four initiatives had adequate standardized training and education but did not support periodic different modes of training. Palestinian initiatives were developed in conflict settings. Lack of adequate training to digital health solutions is an issue especially in LMICs.[33] A World Health Organization (WHO) report found that some health care workers struggle with using digital health technologies; hence, training and familiarity with these technologies can overcome these technical hitches.[1] This is important especially that the perceptions and experiences of health care workers toward digital health initiatives in limited-resource settings are mostly formed by their preexisting digital literacy and perceived usefulness.[1] [34] Thus, it is vital to train health care workers while ensuring that training and support are easily available through different channels such as online, individual training sessions, and through peers. Likewise, customized training on updates to the software or digital devices are needed for scaling success and sustainability.[1] To achieve scale and advance maturity level in LMICs, there is a need to motivate, support, and train end users to entirely utilize any digital health initiative with clear instructions on accountability.[33]

Besides, the maturity of the ecosystem—in terms of both the ICT and enabling environments—has a great effect on the relevance, applicability, and impact of the digital health solutions.[1] The enabling environments relevant to digital health includes leadership, workforce capacity, regulatory and policy frameworks, sociocultural issues, strategy and financial investment, governance mechanisms, and standards and interoperability.[36]

The second main category of the digital maturity assessment tool that we used is data governance and interoperability. Although the rapid increase in digital health tools worldwide can significantly address critical gaps in health systems, at the same time it amplifies the need to implement strong data governance structures and guarantees the ethical use and reuse of clients' data collected while using digital health solutions.[37] Overall, data governance and interoperability components were less “mature” than those related to ICT infrastructure across the five initiatives with some individual discrepancies. In regard to the governance component, the maturity assessment showed that the UNRWA EMCH-J is somehow more advanced than the other four initiatives. Nonetheless, while the governing body of the EMCH-J conducts irregular meetings with stakeholders, the end users at the clinics were not sure about the development plan, which may be due to the fact that the drivers of data governance planning did not inform the design, implementation, or maintenance of this initiative. Surprisingly, Avicenna scored very weak with no governing body, no clear plan or drivers for data governance and interoperability, or even specific approved data regularity framework. This is partially explained by the nature of Avicenna development; digitalization of hospital records started as early as 2000 in Palestine but at small scale with different stakeholders and then Avicenna was introduced building it on the already existing initiatives. This is different from the MCH e-registry which has a clear development and implementation plan. EMCH-P was not rated because the people interviewed were either implementers or end users, still in its early stages of development. The MCH e-registry had an established planning process but with no clear governing body although the MoH was involved, and the end users have limited information about data planning and governance. Similar to the JSANDS, digital initiatives that are specifically designed on a smaller scale are usually hard coded for a specific project or pilot study lacking the ability to accommodate a larger number of end users without recurrent dysfunction.[33]

As for the legislation, policy, and compliance, the two UNRWA initiatives in both countries (EMCH-J and EMCH-P) demonstrated adequate maturity, especially in regard to the implementation of and training users on health data security and privacy laws and development of policies assuring data ethics and human rights. On the contrary, the stakeholders of the MCH e-registry and Avicenna in Palestine were not sure whether these initiatives reflect data security. Also, there are no clear guidelines or data regulatory framework about data privacy for digital initiatives in general as well as no ethical committee or regulatory body. The UNRWA EMCH in both countries demonstrated a high level of maturity in data ethics and human rights, which were not incorporated in other initiatives.

Hence, there is a need to develop systems that meet international legal standards to ensure awareness, access, ownership, data privacy, as well as integrity and protection of patient information.[1] Having said that, it is of supreme importance to be vigilant to some unintentional risks associated with digital targeted client communication such as physical violence against women who receive targeted communications for sexual and reproductive health services.[1] Another significant factor is to ensure adequate security systems that minimize risks to data integrity such as unauthorized data change and ensure informed consent, particularly those with limited literacy.[1] Fortunately, almost all of the screened digital health initiatives restrict access and level of access to data based on job title and role to minimize such risks. However, a study conducted in Palestine found limited policies regulating data collection and access or formal protocols, such as the privacy manual for eRegistry.[38] The local eRegistry Governance Guidance Toolkit outlines responsible data supervision and protection practices and provides support in developing policies and protocols by providing a framework that ensures women and children's privacy protection.[38] Therefore, effective governance concerned with protecting women and children's privacy and confidentiality is important in maintaining trust of individuals and ensuring the sustainability of an eRegistry.[38]

As for standards and interoperability, Avicenna scored the lowest in this component, whereas the JSANDS showed better but not optimal-maturity level. The two UNRWA initiatives in both countries scored the highest in this component. For example, JSANDS and MCH e-registry have limited ability to support patient data sharing and exchange between systems as they only incorporates localized data sharing and exchange policy standards, which end users and some implementers were not aware of. These findings are not encouraging as they reflect lacking interoperability of data. It has been shown that interoperability transfers of care necessitate established digital systems.[23] This electronic exchange allows health care providers to reveal variations across health care services, organizations, and regions.[39] A qualitative study in Indonesia highlighted the absence of interoperability among 18 maternal and child HIS.[40] To support the digitalization of services, interoperability requires efforts to improve infrastructure, credible health data standards, strengthening governance and regulation, and utilization of data.[40]

The last component that we assessed was data quality and use. Again, Avicenna was the least “mature” among the five initiatives. JSANDS, on the contrary, which is still in its “infancy,” has a clear documented data management process. It also can accommodate for large data and its current data use license can be scaled up and upgraded to allow for huge aggregation. However, the JSANDS data management and data use strategy are still not mature enough and need advancement to meet emerging needs. Overall, data quality and use in primary health care were better in EMCH-J, followed by MCH e-registry, and finally the EMCH-P.

Consequently, the maturity level of the five RMNCH initiatives in both countries is inadequate and requires further advancement before they can be scaled up and scaled out. Amid all the heightened interest, several recommendations were shared by WHO reports and suggested from several studies in LMICs concerned with assessing maturity levels and scaling and sustaining digital health initiatives that became fully integrated in health care systems.[1] [33] [34] Taking the following recommendations into account when developing, implementing, and scaling digital health initiatives in LMICs can result in successful and sustainable initiatives, thus meeting health needs and improving health care quality of individuals, especially those living in fragile contexts:

• Successful scale-up of digital health innovations to address unmet needs in LMICs requires close and regular partnership among all stakeholders from the beginning through the scaling process, which must be flexible and dynamic and can accommodate for any amendments if changes occur in technology or human needs of the system in which the initiative is operating.

• Successful global partnership and collaboration in digital health care can be achieved through focusing on software interoperability, through which interoperable systems can exchange data and share information to avoid duplication and reduce the burden on clients and health care workers.

• To comprehensively plan for scale, policy and decision-makers need to take into account several factors including indolence, contradictory priorities, resistance to change, lack of adequate training, human resource incapacity ambiguous roles, and responsibilities. Human resource capacity as well as qualified technical staff in information sciences, health informatics, technology management, and statistical analysis are of particular importance.

• There is a need for an extrinsic ecosystem such as an existing suitable infrastructure to support using digital health initiatives at scale. There is also a need for joint efforts to come up with an agile and flexible approach that can easily scale and integrate digital health initiatives worldwide to be able to reach patients and health care workers in LMICs.

• There is a need for a comprehensive regulatory framework to manage and oversee data processing activities.

• Future research should establish common tools and metrics for the proper assessment of the effectiveness and cost-effectiveness of digital health solutions.

• To better understand and lessen any possible negative impacts on gender, equity, and rights, further research needs to cover a wider range of populations and contexts, including those who live in poor access areas to digital or conventional health services.

Conclusion

The findings of the digital maturity assessment tool and FGDs with stakeholders highlighted several challenges and opportunities around the application and implementation of selected digital health initiatives in the provision of RMNCH in Jordan and Palestine. Our findings shed lights on the maturity level of these initiatives within fragile contexts. The maturity level of the five RMNCH initiatives in both countries is inadequate and requires further advancement before they can be scaled up and scaled out. Amid all the heightened interest, several recommendations that we mentioned in this paper were shared by WHO reports and suggested from several studies reporting on best practices in scaling digital health in LMICs through practical experiences from real-life case studies of scaling and sustaining digital health initiatives that became fully integrated into health care systems. Taking these recommendations into account when developing, implementing, and scaling digital health initiatives in LMICs can result in successful and sustainable initiatives, thus meeting health needs and improving the health care quality of individuals, especially those living in fragile contexts.

Conflict of Interest

None declared.

Acknowledgments

We would like to thank all the stakeholders of the initiatives in Jordan and Palestine who provided more details and resources during the mapping process.

Authors' Contributions

N.M.E.A., Y.K., M.A., A.C., and S.S. designed the study and conceptualized the analysis. M.A., N.M.E.A., M.N., N.A., Y.K., A.C., L.C., and S.S. were involved in tools development. Y.K., M.A., N.M.E.A., N.A., and M.N. were involved in gaining ethical approval, data collection, data analysis and results writing. M.A., N.M.E.A., N.A., and Y.K. wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

• References

• 1 World Health Organization. WHO guideline: recommendations on digital interventions for health system strengthening. Web Supplement 2: Summary of findings and GRADE tables. Geneva: World Health Organization; 2019. Licence: CC BY-NC-SA 3.0 IGO. Accessed November 14, 2021 at: https://apps.who.int/iris/bitstream/handle/10665/324998/WHO-RHR-19.7-eng.pdf?ua=1
• 2 Asi YM, Williams C. The role of digital health in making progress toward Sustainable Development Goal (SDG) 3 in conflict-affected populations. Int J Med Inform 2018; 114: 114-120
• 3 Malvey D, Slovensky DJ. Mhealth: Transforming Healthcare. Dordrecht: Springer; 2014
• 4 Agarwal S, Labrique A. Newborn health on the line: the potential mHealth applications. JAMA 2014; 312 (03) 229-230
• 5 Jareethum R, Titapant V, Chantra T, Sommai V, Chuenwattana P, Jirawan C. Satisfaction of healthy pregnant women receiving short message service via mobile phone for prenatal support: A randomized controlled trial. J Med Assoc Thai 2008; 91 (04) 458-463
• 6 Jiang H, Li M, Wen LM. et al. Effect of short message service on infant feeding practice: findings from a community-based study in Shanghai, China. JAMA Pediatr 2014; 168 (05) 471-478
• 7 Khorshid MR, Afshari P, Abedi P. The effect of SMS messaging on the compliance with iron supplementation among pregnant women in Iran: a randomized controlled trial. J Telemed Telecare 2014; 20 (04) 201-206
• 8 Oyeyemi SO, Wynn R. Giving cell phones to pregnant women and improving services may increase primary health facility utilization: a case-control study of a Nigerian project. Reprod Health 2014; 11 (01) 8
• 9 Chuang L-L, Lin L-C, Cheng P-J, Chen C-H, Wu S-C, Chang C-L. The effectiveness of a relaxation training program for women with preterm labour on pregnancy outcomes: a controlled clinical trial. Int J Nurs Stud 2012; 49 (03) 257-264
• 10 Simonyan D, Gagnon M-P, Duchesne T, Roos-Weil A. Effects of a telehealth programme using mobile data transmission on primary healthcare utilisation among children in Bamako, Mali. J Telemed Telecare 2013; 19 (06) 302-306
• 11 Flax VL, Negerie M, Ibrahim AU, Leatherman S, Daza EJ, Bentley ME. Integrating group counseling, cell phone messaging, and participant-generated songs and dramas into a microcredit program increases Nigerian women's adherence to international breastfeeding recommendations. J Nutr 2014; 144 (07) 1120-1124
• 12 Ippoliti NB, L'Engle K. Meet us on the phone: mobile phone programs for adolescent sexual and reproductive health in low-to-middle income countries. Reprod Health 2017; 14 (01) 11
• 13 Ahlers-Schmidt CR, Chesser AK, Nguyen T. et al. Feasibility of a randomized controlled trial to evaluate Text Reminders for Immunization Compliance in Kids (TRICKs). Vaccine 2012; 30 (36) 5305-5309
• 14 Odeny TA, Bukusi EA, Cohen CR, Yuhas K, Camlin CS, McClelland RS. Texting improves testing: a randomized trial of two-way SMS to increase postpartum prevention of mother-to-child transmission retention and infant HIV testing. AIDS 2014; 28 (15) 2307-2312
• 15 Department of Statistics (DOS). ICF. Jordan population and family health survey 2017–18, key indicators report. 2018 Accessed May 26, 2022, at: http://www.dos.gov.jo/dos_home_a/main/linked-html/DHS2017_en.pdf
• 16 Chukwu E, Foday E, Konomanyi A, Wright R, Garg L, Smart F. Digital health solutions and state of interoperability: Sierra Leone's Landscape analysis. JMIR Form Res 2021; 6 (06) e29930
• 17 Nsaghurwe A, Dwivedi V, Ndesanjo W. et al. One country's journey to interoperability: Tanzania's experience developing and implementing a national health information exchange. BMC Med Inform Decis Mak 2021; 21 (01) 139
• 18 Fraser H, Bailey C, Mehl G, Sinha C. The role of evaluation in global eHealth. In: Global health delivery online [website] Cambridge, MA: Harvard University 2011. Accessed November 14, 2021, at: https://www.ghdonline.org/tech/discussion/therole-of-evaluation-in-global-ehealth
• 19 Privacy International and Partnters. Why we need to talk about digital health. In. London, UK; 2021. Accessed March 12, 2022, at: https://privacyinternational.org/long-read/4674/why-we-need-talk-about-digital-health
• 20 Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12 (06) 573-576
• 21 Carvalho JV, Rocha Á, Abreu A. Maturity models of healthcare information systems and technologies: a literature review. J Med Syst 2016; 40 (06) 131
• 22 Söylemez M, Tarhan A. (2016) The use of maturity/capability frameworks for healthcare process assessment and improvement. In: Clarke P., O'Connor R., Rout T., Dorling A. eds. Software Process Improvement and Capability Determination. SPICE 2016. Communications in Computer and Information Science, vol 609. Springer, Cham
• 23 Johnston DS. Digital maturity: are we ready to use technology in the NHS?. Future Healthc J 2017; 4 (03) 189-192
• 24 Kolukısa Tarhan A, Garousi V, Turetken O, Söylemez M, Garossi S. Maturity assessment and maturity models in health care: A multivocal literature review. Digit Health 2020; 6: 2055207620914772
• 25 Liaw S-T, Zhou R, Ansari S, Gao J. A digital health profile & maturity assessment toolkit: cocreation and testing in the Pacific Islands. J Am Med Inform Assoc 2021; 28 (03) 494-503
• 26 Govrment of South Australia. Digital strategy toolkit. digital maturity assessment tool- governance and leadership. Accessed January 22, 2021 at: https://www.dpc.sa.gov.au/__data/assets/pdf_file/0010/46567/Digital_Maturity_Assessment.pdf
• 27 Gill M, VanBoskirk S. The digital maturity model 4.0. Benchmarks: digital transformation playbook 2016. Accessed January 18, 2021, at: https://dixital.cec.es/wp-content/uploads/presentacions/presentacion06.pdf
• 28 USAIDS, MEASURE Evaluation, Health Data Collaborative. Health information systems interoperability maturity toolkit: users' guide. MEASURE Evaluation, University of North Carolina Chapel Hill, NC; 2017. Accessed January 18, 2021 at: https://www.measureevaluation.org/tools/health-information-systems-interoperability-toolkit.html
• 29 Khader YS, Shattnawi KK, Al-Sheyab N, Alyahya M, Batieha A. The usability of Jordan stillbirths and neonatal deaths surveillance (JSANDS) system: results of focus group discussions. Arch Public Health 2021; 79 (01) 29
• 30 Khader Y, Alyahya M, El-Khatib Z, Batieha A, Al-Sheyab N, Shattnawi K. The Jordan stillbirth and neonatal mortality surveillance (JSANDS) system: evaluation study. J Med Internet Res 2021; 23 (07) e29143
• 31 Kitamura A, Goto R, Nasir S, Hababeh M, Ballout G, Seita A. UNRWA's electronic MCH Handbook application for Palestine Refugees. In: Global Promotion of Maternal and Child Health Handbook 2018. Human Development Department, Japan International Cooperation Agency.
• 32 Ballout G, Al-Shorbaji N, Abu-Kishk N, Turki Y, Zeidan W, Seita A. UNRWA's innovative e-Health for 5 million Palestine refugees in the Near East. BMJ Innov 2018; 4: 128-134
• 33 Labrique AB, Wadhwani C, Williams KA. et al. Best practices in scaling digital health in low and middle income countries. Global Health 2018; 14 (01) 103
• 34 Ebenso B, Okusanya B, Okunade K. et al. What are the contextual enablers and impacts of using digital technology to extend maternal and child health services to rural areas? findings of a qualitative study from Nigeria. Front Glob Womens Health 2021; 2: 670494
• 35 International Finance Corporation. The business of health in Africa: partnering with the private sector to improve people's lives. Washington, DC: International Finance Corporation 2007. Accessed December 11, 2021, at: https://www.unido.org/sites/default/files/2016-01/IFC_HealthinAfrica_Final_0.pdf
• 36 World Health Organization, International Telecommunications Union. National e-health strategy toolkit. WHO, Geneva and International Telecommunications Union 2012. Accessed December 11, 2021 at: https://www.itu.int/dms_pub/itu-d/opb/str/D-STR-E_HEALTH.05-2012-PDF-E.pdf
• 37 Tiffin N, George A, LeFevre AE. How to use relevant data for maximal benefit with minimal risk: digital health data governance to protect vulnerable populations in low-income and middle-income countries. BMJ Glob Health 2019; 4 (02) e001395
• 38 Myhre S, Ghanem B, Salman R, Kaye J, Frøen JF. Governance guidance for an eRegistry for maternal and child health: lessons from the occupied Palestinian territory. Lancet 2018; 391 (Suppl. 01) S35
• 39 Flott K, Callahan R, Darzi A, Mayer E. A patient-centered framework for evaluating digital maturity of health services: a systematic review. J Med Internet Res 2016; 18 (04) e75
• 40 Lazuardi L, Sanjaya GY, Ali PB, Siahaan RGM, Achmad L, Wulandari H. Interoperability of health digitalization: case study on use of information technology for maternal and child health services in Indonesia. Paper presented at: 24th International Conference on Business Information Systems (BIS 2021); June 15–17 2021 Hannover, Germany

Publication History

Received: 19 March 2022

Accepted: 12 July 2022

Article published online:
15 November 2022