Keywords child health - health information systems - primary health care
Introduction
The health management information system (HMIS) has a promising role in monitoring
service delivery and provides evidence for health managers, researchers, and stakeholders
at the district, state, and national levels.[1 ]
[2 ] Numerous global initiatives, including the Sustainable Development Goals (SDG) rely
on routine HMIS to support policy-making and program implementation.[3 ] As a result of the SDG's vision, promoting neonatal and child health serves as a
global health priority, particularly in low- and middle-income countries (LMICs) with
a high burden of morbidity and mortality among newborns and children.[4 ] The importance of golden opportunity to diagnose child disorders and make appropriate
planning for treatment or rehabilitation through timely information highlights the
need for reliable HMISs.[5 ] In 2019 alone, roughly 14,000 children under the age of 5 died every day.[6 ] The majority of deaths (98%) occur in LMICs with lack of overall or cause-specific
deaths.[7 ] Several studies have drawn attention to the risk factors for death and shown that
substantial proportions of all child and perinatal deaths are attributable to the
maternal condition.[8 ] Mothers who suffer from poor nourishment, sickness, pregnancy conditions, or receive
inadequate prenatal and delivery care have babies who are at a higher risk of diseases
and death.[9 ]
[10 ]
[11 ]
[12 ] Despite increasing importance being laid on capturing accurate information from
various sources into an integrated information system to support decision-making and
public health program needs, concerns continue to be expressed over the quality and
accuracy data of HMISs.[13 ] Lack of accurate data of HMISs has led to missed opportunities to use data for quality
improvement in health care delivery and develop key interventions affecting maternal
and newborn outcomes. Several studies suggested that HMIS suffered from incompleteness
and poor quality and a tendency of over- or under-reporting in HMIS data.[14 ] These included the presence of extreme outliers, lack of consistency of the reported
data over time and between indicators (such as vaccination and antenatal care), or
irregularities in report generation, data duplication, and data inconsistencies, at
all levels of health care delivery for newborn and child health.[15 ]
[16 ]
Bringing together information from various sources into an integrated information
system could facilitate assessment and prompt appropriate services. Making comprehensive
information readily available to authorized users would facilitate entry into a community-based
services system for all children and their families.[17 ] Despite these opportunities, the existing child health care information systems
suffer from disparate systems and information silos that do not support management
decision-making, meaningful data analysis, research, and programmatic efforts.[18 ]
[19 ] The global magnitude of adverse maternal effect on child health is impossible to
determine due to the lack of a comprehensive, integrated mother and child information
system.[20 ] Therefore, identifying key determinants of child health is critical to priority
setting in policy-making. We aim to address this gap through the development and testing
requirements for an integrated child health information system (CHIS) in Iran through
the design thinking approach.
Methods
Study Setting
The presented project was launched by a collaboration of interdisciplinary stakeholders,
including health care workers, physicians, health information management (HIM) professionals,
and information technology (IT) at an academic medical university in Iran. In Iran,
maternal and child health care (MCH) services are provided through (1) the primary
health care (PHC) network, including urban health centers (UHCs), rural health centers,
and health houses, and (2) hospitals with child and maternity units. MCH services
are provided based on the health booklet and the completion of MCH forms by specialized
PHC workers, family nurse practitioners, family health practitioners, child care midwives,
and physicians.
The existing PHC network is governed at three levels of practice, including district,
provincial, and national. Data are routinely collected at health care facilities and
submitted to district offices that work under the supervision of academic medical
universities. Data are compiled from paper-based documents and electronic format titled
“SIB information system.” The SIB information system solely covers outpatient visits
for the child (child growth, child diseases, child nutrition) and mothers (antenatal,
pregnancy, and postnatal care). The labor and delivery information are collected separately
and through the hospital information system.
Study Design
We applied a design thinking methodology, which is a human-centered and iterative
approach to provide a solution-based method. By applying the five phases of empathizing,
defining, ideation, prototyping, and testing, design thinking is a comprehensive method
for information system development.[21 ]
[22 ] We identified needs (empathize), defined a problem (define), and generated ideas
for a solution (ideation). The idea was then implemented as a pilot integrated CHIS
(prototype) and evaluated (test) using Software Quality Requirements and Evaluation
Model (SQuaRE) ISO/IEC 25000. Details of each phase will be presented below.
Phase 1: Empathize
This phase involves seeking an empathetic understanding of the problem you are trying
to solve through user research.[23 ]
[24 ] The researchers with expertise in HIM and IT conducted interviews based on recommendations
from DeJonckheere and Vaughn[25 ] and also used field observations. In this phase, we started working with all health
care workers (seven persons) involved in the monitoring, handling, and analyzing MCH
data at the district level of the PHC network at the Deputy of Health in Kashan University
of Medical Sciences (DoH- KAUMS). DoH- KAUMS supervises all MCH processes conducted
at the UHC, rural health center, and health houses. These seven informants with sufficient
“information power,”[26 ] and all with more than 10 years of work experience, participated in the study as
an expert panel group to conduct design thinking phases ([Table 1 ]). They were invited to determine a suitable time and place for an interview; one
researcher interviewed each expert in their determined time and place. Interviews
began with an explanation about the study objective; then questions, such as “are
you satisfied with the existing MCH processes?” “Are you satisfied with the information
communications you have throughout MCH processes at the PHC network?” “Is there any
problem with the current MCH processes?” “Based on your point of view, what are the
main problems with the current MCH processes?” were asked; each session lasted for
almost 45 to 60 minutes. The sessions were audio-recorded, and key points were written
down. After analyzing the data obtained through the interviews to discuss the results,
two expert panel sessions were held at the DoH-KAUMS, when all members agreed to participate.
Then, field observations were performed by the researchers at the DoH-KAUMS through
inspecting the existing systems, Excel files, and forms in which data from the PHC
network were stored, to record all MCH processes, work duplication and delays, waiting
time, data collection, and report generation.
Table 1
Participant characteristics (phase 1)
Gender
Female (n = 5)
Male (n = 2)
Discipline
General practitioners (n = 1)
Childcare midwives (n = 2)
Health care workers (n = 3)
Health care management (n = 1)
Phase 2: Define
During the define phase, we analyzed the data obtained through observations and interviews
and put them to realize the problems in human-centered terms entirely.[24 ] Two researchers analyzed the qualitative data. The two researchers were HIM and
IT specialists because they could recognize the specific needs and requirements of
the end-users and turn them into computer language more appropriately. We applied
Wolcott's thematic analysis approach,[27 ] including description, analysis, and interpretation, to analyze the collected data
and transform them into insights and themes. The results were clustered into four
themes using a mind map tool ([Fig. 1 ]).
Fig. 1 Determined problems for maternal child health information system using a mind map.
Phase 3: Ideation
Generally, the ideation stage seeks potential solutions to the identified problems
using various ideation techniques.[23 ] In our study, the main solution to address users' needs was to transit to an integrated
maternal and child health information system (IMCHIS). Three 5-hour focus group discussions
(FGDs) were conducted under the supervision of trained moderators (researchers) to
identify all aspects of IMCHIS requirements and data elements. The FGD sessions were
digitally recorded and transcribed by two authors to identify common themes using
Wolcott's thematic analysis approach.[27 ]
The preliminary IMCHIS functional requirements and data elements were determined at
the end of this phase. Key functional requirements and data elements were changed
to a 5-point scale questionnaire from “strongly disagree = 1” to “strongly agree = 5”
and distributed among expert panel groups to evaluate consensus about IMCHIS requirements
and data elements. Finally, 109 data elements in seven categories of prenatal care,
labor/pregnancy, postnatal care, child disease status, child health status, child
nutrition status, and child allergy status were proposed for IMCHIS. Functional requirements
and meaningful reports, and queries were also determined in this phase.
Phase 4: Prototyping of IMCHIS
A prototype is a realistic representation of the preliminary version that allows end-users
to understand the idea's feasibility before testing through the iterative cycle.[23 ] Then, 109 data elements determined by the experts at the district level of DoH-
KAUMS in phase 3 were distributed among all specialized health care workers (12 persons)
of the departments of MCH at all urban health levels of the DoH- KAUMS ([Table 2 ]). We selected only UHCs since specialized health care workers mostly worked at UHCs.
They were more accessible than rural health centers and health houses to participate
in the different phases of the study. Data were analyzed using the Fuzzy Delphi Technique
and MATLAB software (MATLAB Simulink, Math Works Corporation, United States). A table
including all the data elements was created to investigate each data element's level
of importance, weight coefficient, and triangular fuzzy function scaled between 0
and 1. Data elements with a weight coefficient of 0.05 and higher were approved, and
those with 0.049 and lower were omitted.
Table 2
Participant characteristics (phase 4)
Characteristics
Number
Age (y)
≤29
n = 3
30–39
n = 2
40–49
n = 6
50–59
n = 1
Gender
Male
n = 2
Female
n = 10
Working experiences
(y)
<5
n = 3
5–9
n = 1
10–14
n = 3
≥15
n = 5
Job titles
Physicians
n = 3
Health care workers
n = 9
It should be noted that the data elements with the weight coefficient closer to 1
were more important based on experts' opinions. A prototype model of IMCHIS was developed
using the confirmed data elements and the discussed functional requirements in the
Structure Query Language (SQL) server environment. Health records of 200 pregnancy
cases, including maternal care status (primary care), labor, and delivery (hospital
information), were entered into the prototype model of IMCHIS by researchers to verify
the functionality of IMCHIS.
Phase 5: Testing of IMCHIS
Testing is the final stage of the five-stage model in which designers or users rigorously
test the product that was developed during the prototyping phase. In this study, we
employed ISO/IEC 25000 SQuaRE to test IMCHIS. SQuaRE is a series of next-generation
software quality evaluation standards by ISO 9000 to identify software quality characteristics,
including functionality, reliability, usability, efficiency, maintainability, and
portability.[28 ] Since this project was not a large-scale production and ran without any vendor support,
evaluation of maintainability and portability was not feasible in this study.[28 ]
[29 ] The remaining four attributes rely on users' participation from our evaluation framework
([Table 3 ]).
Table 3
ISO SQuaRE testing quality model
ISO SQuaRE characteristics
Definition
Sample questions
Reliability
Capability of the software in maintaining stability and its performance under specific
conditions.
• Can the system easily restores working and restores lost data after failure?
• Does it enable users to handle errors?
• Does it clearly and promptly inform the user when it meets an error?
Functionality
Software capability to fulfill the tasks which meet users' needs.
• Can the system perform the tasks required?
• Can it produce the expected results?
• Can it produce accurate results?
Usability
The quality of a user's experience when interacting with the software in terms of
level of effectiveness, efficiency, and satisfaction.
• Does the system support the suitability of learning?
• Is it attractive for users?
• Is it consistent and complying with characteristics of the user?
Efficiency
The relationship between the level of performance of the software and the amount of
the time and resource used.
• How quickly does the system respond?
• How good is it at handling large documents?
• How easy does the system perform work steps?
To meet ISO/IEC 14598–6 regulation,[28 ] there should be at least eight participants in each evaluation category. Forty participants
were recruited to evaluate IMCHIS ([Table 4 ]), and their answers were analyzed using MATLAB software based on the Fuzzy scale.[30 ]
[31 ]
[32 ] Two methods calculated total users' satisfaction: (1) IMCHIS quality characteristic
satisfaction rate based on triangular defuzzification value, which was multiplied
in the array, and (2) overall satisfaction rate by end-users of the IMCHIS.[30 ]
[31 ]
[32 ]
Table 4
Participant characteristics (phase 5)
Characteristics
Number
Age (y)
≤29
n = 8
30–39
n = 19
40–49
n = 7
50–59
n = 6
Gender
Male
n = 9
Female
n = 31
Working experiences
(y)
<5
n = 13
5–9
n = 9
10–14
n = 10
≥15
n = 8
Discipline
Physician
n = 8
Family nurse practitioner
n = 8
Family health practitioner
n = 8
Childcare midwives
n = 8
IT/computer specialists
n = 8
Abbreviation: IT, information technology.
Ethical Considerations
The Ethics Review Board approved this study of the Vice-Chancellorship for Research
Affairs of Kashan University of Medical Sciences (IR.KAUMS.REC.1396.7).
Results
In the process of developing the IMCHIS, one workshop, six interviews, two field observations,
and three FGDs with different stakeholders were conducted, and the outcomes of each
phase are presented as follows.
Phase 1: Empathize
The interviews lasted between 30 and 60 minutes and were transcribed verbatim; the
findings were validated in one workshop with users.
Phase 2: Define
We attempted to retell the stakeholders' needs and requirements determined in the
empathize phase using human-centered design terms during the define phase. A summary
of the problems which users face when using the existing MCH information system can
be seen in [Table 5 ].
Table 5
The outline of the problems which users faced with the existing MCH information system
End users' problems
Description
Use of information generated
Irrelevance of the information gathered: data collection focused more on reporting
MCH conditions instead of maternal and child health promotion and only partially addresses
management objectives at the health unit level or at the patient/client level.
Lack of feedback: there is weekly or monthly reports without adequate feedback, Maternal
and Child Health Workers (MCHW) rarely receives feedback on the data reported to higher
levels.
Lack of motivation: there is a lack of motivation among health workers about data
quality as a result of the absence of feedback.
Data collection
Duplication in data collection: filling endless registers and forms with shared MCH
care data elements (e.g., sex, age, maternal and child care conditions) that cause
irregularities in report generation in the form of over- or under-reporting.
Duplication in data entry: although maternal and child care data elements and forms
are interrelated, double entry of maternal and child care data happens repeatedly,
which means duplication of effort, time and increased costs, and opportunity for error.
Poor data quality: the data received often suffer from incompleteness and inaccuracy
due to standardized instructions on how to collect the data.
Lack of meaningful data analysis
“Data-driven” instead of “action-driven” Information systems: information systems
are often not helpful for management decision making and suffer from pattern discovery
exploration and analysis.
Data integration: information silo which resides within a disparate system does not support combining data in different sources and providing users with a unified wide spectrum
(data integration) .
Lack of linkage: linkage between maternal care and child care and fragmentation between
hospital services and PHC network.
Shortcomings of IT infrastructure for meaningful data analysis
Disparate system: the Information System (IS) is a disparate system without exchanging data between maternal and child information systems that causes a lack of data analysis and meaningful information.
Lack of IS capabilities: lack of powerful screen generators, report generators, query
tools, data visualization, and advanced data analysis (e.g., data mining).
Lack of interoperability: information system's data cannot be exported for use with
other software and shared with other departments.
Abbreviations: IT, information technology; MCH, child health care; PHC, primary health
care.
Phase 3: Ideation
We conducted two interactive FGDs with the experts at the district of DoH-KAUMS. The
purpose was to develop a solution to meet the following requirements:
Developing a feasible and approachable information system on a small scale and with
the minimum changes on the existing process.
Use of standardized data elements and dataset for MCH.
Making a solid integration and linkage between public health services and hospitals.
Generating meaningful reports and queries for MCH.
At the end of this phase, 109 data elements proposed for IMCHIS were distributed for
experts' consensus. Then, 74 of 109 proposed data elements were confirmed through
the Fuzzy Delphi approach. [Table 6 ] reveals that among seven proposed categories for the IMCHIS data elements, “child
disease” with weight (0.26), “child nutrition” with weight (0.20), and “prenatal care”
with weight (0.16) acquired the maximum weight coefficient.
Table 6
Weight coefficient of main requirements for IMCHIS' data elements based on experts'
agreement
Main categories for IMCHIS data elements
Weight coefficient
1
Prenatal care
0.16
2
Postpartum care
0.07
3
Labor/delivery
0.06
4
Child nutritional status
0.20
5
Child health status
0.13
6
Child allergic conditions
0.12
7
Child disease status
0.26
Abbreviation: IMCHIS, integrated maternal and child health information system.
Seventy-four confirmed data elements were applied for determining the logical design
phase of IMCHIS, including relationships between data elements, primary keys, and
foreign keys. Generating meaningful reports to investigate the effect of mothers'
conditions on adverse birth and child outcomes also considered at this phase. Users
reported the disparate design as one of the major problems with the existing MCH information
system. To address existing gaps, the mothers' national identification numbers were
applied to integrate MCH information in the IMCHIS. The flow of events to perform
a task, entity name, field, primary key, and tables for data elements' relationship
were drawn using scenarios. After confirming the scenarios, we use tools from the
visualized ideas and dialog tools such as a flowchart and VISIO software to draw entity
relationship diagrams (ERDs). The number of ideas, functional requirements, and reports
of IMCHIS were adjusted and refined throughout the ideation process.
Phase 4: Prototyping
The 74 conformed data elements were applied for developing ERDs related to MCH, and
then they were transformed into a physical design model using the SQL server environment.
At this stage, the IMCHIS user interface was also designed by the computer programmer.
Relevant IMCHIS content and functionality were validated by entering health records
of 200 pregnancy cases into IMCHIS at the prototyping phase. The proposed IMCHIS creates
new insights into maternal and child health and paves the way for future planning,
research, and evaluation based on generating simple queries and reports ([Fig. 2 ]) in the area of MCH. The followings provide some sample reports for the newly proposed
IMCHIS.
Fig. 2 IMCHIS-generated simple queries and reports. IMCHIS, integrated maternal and child
health information system.
Phase 5: Testing
According to the end-users, the highest level of IMCHIS quality was efficiency and
functionality characteristics with a weight coefficient of 0.73 and 0.48, respectively.
The “reliability” characteristic was the lowest level of the IMCHIS quality with a
weight coefficient of 0.24 ([Table 7 ]).
Table 7
End-user's evaluation for IMCHIS quality using Fuzzy assessment matrix
ISO SQuaRE quality characteristics
Strongly agree (5)
Agree (4)
No comment (3)
Disagree (2)
Strongly disagree (1)
Efficiency
0.73
0.86
0.21
0
0
Functionality
0.48
1
0.58
0.56
0
Usability
0.35
0.89
0.50
0.06
0
Reliability
0.24
0.49
0.75
0.80
0
Final result of fuzzy evaluation
0.35
0.80
0.59
0.07
0
Overall satisfaction rate
0.82
Abbreviation: IMCHIS, integrated maternal and child health information system.
Discussion
This study aimed to develop and pilot test an integrated CHIS to support child health
planning, research, and evaluation. With the aid of a design thinking methodology,
the IMCHIS innovation was developed, which is a web-based system with standardized
data elements and functional requirements to track the effect of maternal conditions
on child health, and develop MCH reports and queries based on a comprehensive information
system.
In the first phase of the design thinking approach, our results indicated that users
reported duplication in data collection, lack of feedback, lack of linkage, and data
integration as the main problems with the existing MCH. Our results are supported
by previous studies in routine MCH information systems conducted by Hinman et al[17 ] and Ouedraogo et al.[33 ]
According to the literature, duplication of data collection and data entry causes
failures in evaluating the effectiveness or efficiency of health programs. It might
cause over-reporting of MCH services ranging from 1.4 to 6%.[14 ] Thirty-eight percent of health workers claimed that their reports and registration
books might contain inconsistencies.[34 ]
Lack of feedback also was presented as one of the problematic areas which users faced
when using the existing MCH information system. The importance of implementing peer
review and feedback in completeness, timeliness, and accuracy of facility reporting
also has been highlighted in previous studies.[15 ]
[35 ]
[36 ]
[37 ] Bhattacharya et al revealed that introducing data-quality interventions, including
feedback, can improve data-quality metrics in terms of availability and timeliness
of reporting, completeness of data elements, the accuracy of facility reporting, consistency
between related data elements, and frequency of outliers reported.[38 ] Despite the increasing importance being put on the use of feedback for improvement
of data quality, in a study verifying the quality and consistency of immunization
monitoring systems in eight countries, Ronveaux et al reported that only 50% (55/108)
of districts applied standard feedback formats.[16 ] Therefore, implementing supervision, monitoring, and feedback to resolve the data
entry errors and ensure the accuracy and consistency of the reports need to be considered
in the HMIS.
In our study, the expert panel group considered the standardization of the data elements
and integration of maternal and child health information systems as a preliminary
solution in the ideation phase. The crucial role of integrating MCH information in
the continuum of care of a child can save health program resources and enhance data
quality and has been recommended in the previous studies.[13 ]
[19 ] Hinman et al argued that reducing the number of registers and developing an integrated
system with standard data elements would pave the way to obtain information readily
about a child's status concerning other programs.[39 ] Standard data elements support generating indicators to monitor the provision of
health services and facilitate vertical reporting of this information at all health
care delivery systems.[17 ]
[40 ] In our study, the standardized data elements were first identified and classified
into seven main requirements. Considering the increasing importance of integrating
the mother and CHIS, end-users mentioned “prenatal care” as one of the primary data
elements and functionalities for IMCHIS. Our results are in line with those reported
in previous studies. Evidence suggests that the original causes of many childhood
diseases and deaths stem from the neonatal and pregnancy period.[9 ]
[41 ]
[42 ]
[43 ]
[44 ]
[45 ]
[46 ]
Ultimately, design thinking is a solution-based approach to meet users' needs and
requirements.[21 ] To address this possible limitation, the IMCHIS quality was evaluated using the
SQuaRE model. The results indicated that the highest IMCHIS quality level was for
efficiency and functionality characteristics, focusing on software capability to fulfill
the task that meets users' needs. It has been reported that the managers and users
of the LMICs tend to adopt information systems that avoid fundamental changes in the
existing work structure and processes.[47 ] Our results are in agreement with previous studies that develop CHIS. Hinman et
al also argued the integrated information system, making challenges and changes, is
hard to implement. Thus, the individual needs of users should be incorporated into
the existing organizational culture and processes.[17 ]
[39 ] Active participation of stakeholders in the design, implementation, and evaluation
of information systems via a design thinking approach would address these challenges.
However, this study had two limitations: first, the studied population might not represent
the country. Although health care services in Iran are centralized, MCHs are rendered
through the same PHC network, which consists of the same organizational structure,
work processes, and information systems. Second, since CHIS failures in developing
countries are noticeable because of social, cultural, and economic difficulties, we
conducted a pilot project. Some parts of the ISO testing quality model, including
maintainability and portability, were not considered in the current study. However,
IMCHIS was validated by entering health records of 200 pregnancy cases into the prototyping
phase. Moreover, the remaining ISO characteristics were conducted through the participation
of multidisciplinary end-users from technical and clinical domains.
Conclusion
According to the end-users, the most suitable CHIS integrates mother and child health
care requirements to minimize data duplication and inconsistencies and supports management
decision-making, meaningful data analysis, research, and programmatic efforts. The
highest quality score in efficiency and functionality of the IMCHIS in the testing
phase acknowledges that it can fulfill end-users' needs for identifying key determinants
of child health. The design thinking approach serves as a user-centered and solution-based
approach for developing an integrated CHIS based on users' perspectives, workflows,
and tasks.
