Implementation Outcomes Assessment of a Digital Clinical Support Tool for Intrapartum Care in Rural Kenya: Observational Analysis

JMIR Formative Research, Volume 6, No. 6, Year 2022

Interpretation

Background: iDeliver, a digital clinical support system for maternal and neonatal care, was developed to support quality of care improvements in Kenya. Objective: Taking an implementation research approach, we evaluated the adoption and fidelity of iDeliver over time and assessed the feasibility of its use to provide routine Ministry of Health (MOH) reports. Methods: We analyzed routinely collected data from iDeliver, which was implemented at the Transmara West Sub-County Hospital from December 2018 to September 2020. To evaluate its adoption, we assessed the proportion of actual facility deliveries that was recorded in iDeliver over time. We evaluated the fidelity of iDeliver use by studying the completeness of data entry by care providers during each stage of the labor and delivery workflow and whether the use reflected iDeliver’s envisioned function. We also examined the data completeness of the maternal and neonatal indicators prioritized by the Kenya MOH. Results: A total of 1164 deliveries were registered in iDeliver, capturing 45.31% (1164/2569) of the facility’s deliveries over 22 months. This uptake of registration improved significantly over time by 6.7% (SE 2.1) on average in each quarter-year (P=.005), from 9.6% (15/157) in the fourth quarter of 2018 to 64% (235/367) in the third quarter of 2020. Across iDeliver’s workflow, the overall completion rate of all variables improved significantly by 2.9% (SE 0.4) on average in each quarter-year (P<.001), from 22.25% (257/1155) in the fourth quarter of 2018 to 49.21% (8905/18,095) in the third quarter of 2020. Data completion was highest for the discharge-labor summary stage (16,796/23,280, 72.15%) and lowest for the labor signs stage (848/5820, 14.57%). The completion rate of the key MOH indicators also improved significantly by 4.6% (SE 0.5) on average in each quarter-year (P<.001), from 27.1% (69/255) in the fourth quarter of 2018 to 83.75% (3346/3995) in the third quarter of 2020. Conclusions: iDeliver’s adoption and data completeness improved significantly over time. The assessment of iDeliver’ use fidelity suggested that some features were more easily used because providers had time to enter data; however, there was low use during active childbirth, which is when providers are necessarily engaged with the woman and newborn. These insights on the adoption and fidelity of iDeliver use prompted the team to adapt the application to reflect the users’ culture of use and further improve the implementation of iDeliver. iDeliver is a software application that allows health care providers to document relevant patient information and clinical progression throughout the continuum of maternal care in real time. Figure 1 summarizes the overall workflow. The version of iDeliver assessed in this study focused on intrapartum care; recent updates to the application also include antenatal and postnatal care components. The health care provider registers a new patient when she arrives at the labor and delivery ward and enters the key patient demographic and clinical information, which generates an acuity score for triage priority. All active registered patients can be seen on a dashboard from which users can access a patient’s digital clinical chart, navigate to any section—intake, history, vital signs, labor signs, fetal assessment, and discharge-labor summary—and enter the patient’s information at successive appointments to maintain a longitudinal health record. Digital clinical decision support algorithms and patient management guidelines for iDeliver are based on WHO’s Managing Complications in Pregnancy and Childbirth [22], Better Outcomes in Labour Difficulty Initiative [23], and Recommendations for Intrapartum Care for a Positive Childbirth Experience [24]. In addition, iDeliver includes clinical training resources, electronic medical record function, and report generation. Further information on the design, development, and implementation of iDeliver has been presented elsewhere [25]. User’s workflow through iDeliver when a patient arrives at the maternity ward for labor and delivery. ANC: antenatal care; PNC: postnatal care. iDeliver was developed in collaboration with nurses, midwives, physicians, and public health administrators in Transmara West and Transmara East Sub-Counties of Narok County, Kenya. It was first implemented in 2017 at the Transmara West Sub-County Hospital in Kilgoris, which is a level-4 tertiary facility offering comprehensive emergency obstetric and newborn care services, with an average of 1150 births annually. It has since been scaled up to 13 other sites in Kenya and Tanzania. This study focused on iDeliver implementation at the Transmara West Sub-County Hospital during the 22-month period after transition to OpenMRS (OpenMRS Inc) platform (December 2018 to September 2020). Since deployment, the application underwent significant updates. Transition from a proprietary to an open-source back end—OpenMRS—was done in November 2018. iDeliver interfaces were built as modular, encapsulated setup code built upon the OpenMRS application platform using ReactJS (Meta), a modern front-end language. As of September 2020, 5 physicians and 14 nursing officers at the Transmara West Sub-County Hospital were trained to use iDeliver, with 40% (2/5) of the physicians and 57% (8/14) of the nursing officers as current active users. User training was conducted on site every 6 months to account for any upgrades in the application and for staff rotation. Training of new staff occurred on an as-needed basis. Data were extracted and deidentified using MySQL (version 5.6.49; Oracle Corporation). Then, MySQL Workbench (version 8.0; Oracle Corporation) was used to export the data into Excel format. All statistical analyses were performed using R (version 4.0.2; R Foundation for Statistical Computing). We conducted descriptive analysis to summarize the characteristics of all mothers and newborn infants with information registered in iDeliver within the study period. Then, we assessed iDeliver’s adoption by exploring the following: (1) what proportion of services provided at the health facility are captured by iDeliver and (2) does the uptake of iDeliver use improve over time? To answer the first question and measure iDeliver’s uptake, we divided the number of deliveries registered in iDeliver by the number of deliveries recorded on paper at the Transmara West Sub-County Hospital from December 2018 to September 2020. To answer the second question, we assessed the trends in the uptake of iDeliver, by quarter-year, using simple linear regression. A P value of <.05 was considered as statistically significant. We assessed the fidelity of iDeliver use to its original purpose as a decision-making and data management tool by examining which feature or features of iDeliver are used most by users, as assessed by data completion. We used the proportion of data available across the labor and delivery workflow to identify both areas of high use and missed opportunities for use. In particular, we assessed data completion for each stage of the iDeliver’s labor and delivery workflow: (1) intake, (2) history, (3) vital signs, (4) labor signs, (5) fetus assessment, (6) quick check, and (7) discharge-labor summary to identify the aspects of the intrapartum process that were plausible for care providers to use and if the use reflected iDeliver’s envisioned function for intrapartum clinical guidance. We also assessed the data completion for each stage over time, by quarter-year, using simple linear regression. A P value of <.05 was considered as statistically significant. To assess the feasibility of using iDeliver to provide routine reports based on the priority indicators to monitor maternal and newborn health outcomes and quality of care identified by the Kenya MOH’s Reproductive and Maternal Health Services [18-21], we also examined the data completeness of those indicators from the MOH’s maternal and perinatal notification and review forms that overlapped with the data in iDeliver. These indicators were referral information (referral from community unit or health facility or referral out to community unit); mother’s HIV status; parity; fetal presentation; mode of delivery; date and time of delivery; sex of baby; condition of baby at birth; appearance, pulse, grimace, activity, and respiration score (at 1, 5, and 10 minutes); baby given tetracycline; condition of mother; and condition of baby at discharge. We also assessed the data completeness of these indicators over time, by quarter-year, using simple linear regression. A P value of <.05 was considered as statistically significant. The study was approved by the institutional review board of Johns Hopkins Bloomberg School of Public Health (protocol code {"type":"entrez-nucleotide","attrs":{"text":"I18203","term_id":"1598558","term_text":"I18203"}}I18203; December 8, 2021).

AI Digest

Study Justification:

The study aimed to evaluate the adoption and fidelity of iDeliver, a digital clinical support tool for intrapartum care, in a rural hospital in Kenya. The justification for the study was to assess the effectiveness and feasibility of using iDeliver to improve the quality of maternal and neonatal care in resource-limited settings.

Highlights:

1. Adoption of iDeliver: The study found that iDeliver captured 45.31% of the facility’s deliveries over a 22-month period. The uptake of registration improved significantly over time, indicating increasing adoption of the digital tool.
2. Fidelity of iDeliver use: The study assessed the completeness of data entry by care providers at each stage of the labor and delivery workflow. Overall, the completion rate of all variables improved significantly over time. However, there was low use of iDeliver during active childbirth, suggesting the need for further improvement in provider engagement.
3. Data completeness of key indicators: The study examined the data completeness of maternal and neonatal indicators prioritized by the Kenya Ministry of Health (MOH). The completion rate of these indicators also improved significantly over time, indicating the potential of iDeliver to provide routine reports for monitoring health outcomes and quality of care.

Recommendations:

1. Adaptation of iDeliver: The study recommended adapting the application to reflect the users’ culture of use and to further improve its implementation. This could involve enhancing features that are more easily used during active childbirth and addressing any barriers to data entry during this critical period.
2. Training and support: To ensure effective use of iDeliver, ongoing training and support should be provided to healthcare providers. Regular training sessions should be conducted to account for any upgrades in the application and staff rotation.
3. Monitoring and evaluation: Continuous monitoring and evaluation of iDeliver’s adoption, fidelity, and data completeness should be conducted to identify areas for improvement and ensure sustained use of the digital tool.

Key Role Players:

1. Healthcare providers: Nurses, midwives, and physicians play a crucial role in using iDeliver to document patient information and provide quality care.
2. Ministry of Health (MOH): The MOH is responsible for overseeing the implementation of digital tools like iDeliver and can provide guidance and support for its adoption and scale-up.
3. Software developers: The team responsible for developing and maintaining iDeliver should be involved in addressing any technical issues, adapting the application, and providing ongoing support.

Cost Items for Planning:

1. Training and capacity building: Budget should be allocated for conducting regular training sessions for healthcare providers on the use of iDeliver and any updates to the application.
2. Technical support: Funds should be allocated for technical support, including addressing any software issues, maintaining the application, and providing assistance to users.
3. Monitoring and evaluation: Budget should be set aside for monitoring and evaluating the adoption, fidelity, and data completeness of iDeliver to ensure its effectiveness and identify areas for improvement.
4. Adaptation and customization: Resources should be allocated for adapting and customizing iDeliver to reflect the users’ needs and culture of use, which may involve software development and user feedback.
5. Scale-up and expansion: If iDeliver is to be implemented in additional healthcare facilities, budget should be allocated for scaling up the application, training new users, and providing ongoing support.
Please note that the cost items mentioned here are general categories and not actual cost estimates. The specific budget requirements would depend on the context and scale of implementation.

Strength of Evidence

The strength of evidence for this abstract is 7 out of 10.
The evidence in the abstract is moderately strong, but there are areas for improvement. The study provides quantitative data on the adoption and fidelity of iDeliver over time, which is valuable. However, the abstract lacks information on the sample size and the representativeness of the study population. Additionally, there is no mention of any statistical analysis conducted to support the findings. To improve the strength of the evidence, the authors could include more details on the study methodology, such as the inclusion criteria for participants and any potential biases in data collection. They should also provide more context on the significance of the findings and how they contribute to the field of intrapartum care in rural Kenya.

Abstract

iDeliver is a software application that allows health care providers to document relevant patient information and clinical progression throughout the continuum of maternal care in real time. Figure 1 summarizes the overall workflow. The version of iDeliver assessed in this study focused on intrapartum care; recent updates to the application also include antenatal and postnatal care components. The health care provider registers a new patient when she arrives at the labor and delivery ward and enters the key patient demographic and clinical information, which generates an acuity score for triage priority. All active registered patients can be seen on a dashboard from which users can access a patient’s digital clinical chart, navigate to any section—intake, history, vital signs, labor signs, fetal assessment, and discharge-labor summary—and enter the patient’s information at successive appointments to maintain a longitudinal health record. Digital clinical decision support algorithms and patient management guidelines for iDeliver are based on WHO’s Managing Complications in Pregnancy and Childbirth [22], Better Outcomes in Labour Difficulty Initiative [23], and Recommendations for Intrapartum Care for a Positive Childbirth Experience [24]. In addition, iDeliver includes clinical training resources, electronic medical record function, and report generation. Further information on the design, development, and implementation of iDeliver has been presented elsewhere [25]. User’s workflow through iDeliver when a patient arrives at the maternity ward for labor and delivery. ANC: antenatal care; PNC: postnatal care. iDeliver was developed in collaboration with nurses, midwives, physicians, and public health administrators in Transmara West and Transmara East Sub-Counties of Narok County, Kenya. It was first implemented in 2017 at the Transmara West Sub-County Hospital in Kilgoris, which is a level-4 tertiary facility offering comprehensive emergency obstetric and newborn care services, with an average of 1150 births annually. It has since been scaled up to 13 other sites in Kenya and Tanzania. This study focused on iDeliver implementation at the Transmara West Sub-County Hospital during the 22-month period after transition to OpenMRS (OpenMRS Inc) platform (December 2018 to September 2020). Since deployment, the application underwent significant updates. Transition from a proprietary to an open-source back end—OpenMRS—was done in November 2018. iDeliver interfaces were built as modular, encapsulated setup code built upon the OpenMRS application platform using ReactJS (Meta), a modern front-end language. As of September 2020, 5 physicians and 14 nursing officers at the Transmara West Sub-County Hospital were trained to use iDeliver, with 40% (2/5) of the physicians and 57% (8/14) of the nursing officers as current active users. User training was conducted on site every 6 months to account for any upgrades in the application and for staff rotation. Training of new staff occurred on an as-needed basis. Data were extracted and deidentified using MySQL (version 5.6.49; Oracle Corporation). Then, MySQL Workbench (version 8.0; Oracle Corporation) was used to export the data into Excel format. All statistical analyses were performed using R (version 4.0.2; R Foundation for Statistical Computing). We conducted descriptive analysis to summarize the characteristics of all mothers and newborn infants with information registered in iDeliver within the study period. Then, we assessed iDeliver’s adoption by exploring the following: (1) what proportion of services provided at the health facility are captured by iDeliver and (2) does the uptake of iDeliver use improve over time? To answer the first question and measure iDeliver’s uptake, we divided the number of deliveries registered in iDeliver by the number of deliveries recorded on paper at the Transmara West Sub-County Hospital from December 2018 to September 2020. To answer the second question, we assessed the trends in the uptake of iDeliver, by quarter-year, using simple linear regression. A P value of <.05 was considered as statistically significant. We assessed the fidelity of iDeliver use to its original purpose as a decision-making and data management tool by examining which feature or features of iDeliver are used most by users, as assessed by data completion. We used the proportion of data available across the labor and delivery workflow to identify both areas of high use and missed opportunities for use. In particular, we assessed data completion for each stage of the iDeliver’s labor and delivery workflow: (1) intake, (2) history, (3) vital signs, (4) labor signs, (5) fetus assessment, (6) quick check, and (7) discharge-labor summary to identify the aspects of the intrapartum process that were plausible for care providers to use and if the use reflected iDeliver’s envisioned function for intrapartum clinical guidance. We also assessed the data completion for each stage over time, by quarter-year, using simple linear regression. A P value of <.05 was considered as statistically significant. To assess the feasibility of using iDeliver to provide routine reports based on the priority indicators to monitor maternal and newborn health outcomes and quality of care identified by the Kenya MOH’s Reproductive and Maternal Health Services [18-21], we also examined the data completeness of those indicators from the MOH’s maternal and perinatal notification and review forms that overlapped with the data in iDeliver. These indicators were referral information (referral from community unit or health facility or referral out to community unit); mother’s HIV status; parity; fetal presentation; mode of delivery; date and time of delivery; sex of baby; condition of baby at birth; appearance, pulse, grimace, activity, and respiration score (at 1, 5, and 10 minutes); baby given tetracycline; condition of mother; and condition of baby at discharge. We also assessed the data completeness of these indicators over time, by quarter-year, using simple linear regression. A P value of <.05 was considered as statistically significant. The study was approved by the institutional review board of Johns Hopkins Bloomberg School of Public Health (protocol code {"type":"entrez-nucleotide","attrs":{"text":"I18203","term_id":"1598558","term_text":"I18203"}}I18203; December 8, 2021).

Materials

Innovations

Based on the provided information, here are some potential recommendations for innovations to improve access to maternal health:

1. Mobile Application Integration: Explore the possibility of integrating iDeliver with a mobile application to enhance accessibility for healthcare providers in rural areas. This would allow them to easily access and update patient information, even in remote locations with limited internet connectivity.

2. Offline Functionality: Develop an offline mode for iDeliver to ensure that healthcare providers can continue using the application even when there is no internet connection. This would be particularly beneficial in areas with unreliable or limited internet access.

3. Training and Support: Provide comprehensive training and ongoing support to healthcare providers to ensure they are proficient in using iDeliver. This could include regular refresher training sessions, user manuals, and a dedicated helpdesk for technical support.

4. Integration with Existing Systems: Explore opportunities to integrate iDeliver with existing healthcare systems and databases to streamline data collection and reporting processes. This would help reduce duplication of efforts and improve data accuracy.

5. Community Engagement: Engage with local communities to raise awareness about the benefits of iDeliver and encourage pregnant women to seek care at facilities where the system is implemented. This could be done through community meetings, outreach programs, and collaboration with community health workers.

6. Continuous Improvement: Continuously gather feedback from healthcare providers and users of iDeliver to identify areas for improvement. This feedback can be used to refine the system’s functionality, address usability issues, and enhance user experience.

7. Scalability and Sustainability: Develop a plan for scaling up the implementation of iDeliver to more healthcare facilities in Kenya and other countries. This would involve assessing the feasibility, cost-effectiveness, and long-term sustainability of the system.

It is important to note that these recommendations are based on the information provided and may need to be further evaluated and customized to suit the specific context and needs of the healthcare system.

AI Innovations Description

The recommendation to improve access to maternal health based on the study is to further develop and enhance the iDeliver digital clinical support tool. The study found that the adoption and data completeness of iDeliver improved significantly over time. However, there were some areas of low use during active childbirth, indicating that providers may need more support and training during this critical stage.

To improve the implementation of iDeliver, the study suggests adapting the application to reflect the users’ culture of use. This could involve making the tool more user-friendly and intuitive, ensuring that it aligns with the workflow and needs of healthcare providers in the context of maternal care in rural Kenya.

Additionally, the study highlights the importance of data completeness for monitoring maternal and newborn health outcomes and quality of care. To further enhance iDeliver, it is recommended to prioritize the completeness of key indicators identified by the Kenya Ministry of Health. This could involve streamlining data entry processes, providing clear guidelines and training on data collection, and ensuring that the tool captures all necessary information for reporting and analysis.

Overall, the recommendation is to continue refining and expanding the iDeliver digital clinical support tool, taking into account the specific needs and challenges of maternal health care in rural Kenya. By improving the adoption, data completeness, and usability of iDeliver, access to quality maternal health care can be enhanced, leading to better health outcomes for mothers and newborns.

AI Innovations Methodology

Based on the provided information, here are some potential recommendations to improve access to maternal health:

1. Increase training and awareness: Provide comprehensive training to healthcare providers on the use of iDeliver and its benefits in improving maternal health outcomes. Increase awareness among healthcare providers about the importance of accurate and timely data entry to ensure the effectiveness of the digital clinical support tool.

2. Enhance user experience: Continuously improve the user interface and functionality of iDeliver to make it more user-friendly and intuitive. Gather feedback from healthcare providers and incorporate their suggestions to enhance the usability of the application.

3. Address barriers to data entry during active childbirth: Identify the challenges faced by healthcare providers in entering data during active childbirth and develop strategies to overcome these barriers. This could include providing additional support staff or streamlining the data entry process to minimize interruptions during critical moments of care.

4. Expand implementation to other healthcare facilities: Scale up the implementation of iDeliver to other healthcare facilities in Kenya and Tanzania. This will help improve access to maternal health services in a wider geographic area and reach more pregnant women in need of care.

To simulate the impact of these recommendations on improving access to maternal health, a methodology could be developed as follows:

1. Define the key indicators: Identify the key indicators that will be used to measure the impact of the recommendations. These indicators could include the proportion of facility deliveries recorded in iDeliver, data completeness rates for different stages of the labor and delivery workflow, and the completeness of maternal and neonatal indicators prioritized by the Kenya Ministry of Health.

2. Collect baseline data: Gather baseline data on the identified indicators before implementing the recommendations. This will serve as a reference point for comparison and evaluation.

3. Implement the recommendations: Roll out the recommendations, such as training healthcare providers, improving user experience, addressing barriers to data entry, and expanding implementation to other facilities.

4. Monitor and collect data: Continuously monitor the implementation of the recommendations and collect data on the identified indicators. This can be done through routine data collection within the iDeliver system or through periodic assessments and surveys.

5. Analyze the data: Analyze the collected data to assess the impact of the recommendations on improving access to maternal health. Compare the post-implementation data with the baseline data to identify any changes or improvements.

6. Evaluate the results: Evaluate the results of the analysis to determine the effectiveness of the recommendations. Assess whether the recommendations have led to increased adoption of iDeliver, improved data completeness rates, and better adherence to the Ministry of Health’s indicators.

7. Adjust and refine: Based on the evaluation results, make any necessary adjustments or refinements to the recommendations. This could involve further training, user interface enhancements, or targeted interventions to address specific challenges.

By following this methodology, it will be possible to simulate the impact of the recommendations on improving access to maternal health and make informed decisions for further improvements.

Authors & Co-Authors

Statistics:

Authors: 8

Identifiers:

DOI: 10.2196/34741

Research Areas:

Community Interventions, Health System and Policy, Infectious Diseases, Maternal Access, Maternal and Child Health, Quality of Care, Sexual and Reproductive Health, Social Determinants, Workforce

Study Countries:

Kenya, Multi-Countries, Tanzania

Study Design:

Cohort Study, Cross Sectional Study