Impact evaluation of a maternal and neonatal health training intervention in private Ugandan facilities

N/A

Global and country-specific targets for reductions in maternal and neonatal mortality in low-resource settings will not be achieved without improvements in the quality of care for optimal facility-based obstetric and newborn care. This global call includes the private sector, which is increasingly serving low-resource pregnant women. The primary aim of this study was to estimate the impact of a clinical and management-training programme delivered by a non-governmental organization [LifeNet International] that partners with clinics on adherence to global standards of clinical quality during labour and delivery in rural Uganda. The secondary aim included describing the effect of the LifeNet training on pre-discharge neonatal and maternal mortality. The LifeNet programme delivered maternal and neonatal clinical trainings over a 10-month period in 2017-18. Direct clinical observations of obstetric deliveries were conducted at baseline (n = 263 pre-intervention) and endline (n = 321 post-intervention) for six faith-based, not-for-profit primary healthcare facilities in the greater Masaka area of Uganda. Direct observation comprised the entire delivery process, from initial client assessment to discharge, and included emergency management (e.g. postpartum haemorrhage and neonatal resuscitation). Data were supplemented by daily facility-based assessments of infrastructure during the study periods. Results showed positive and clinically meaningful increases in observed handwashing, observed delayed cord clamping, partograph use documentation and observed 1- and/or 5-minute APGAR assessments (rapid scoring system for assessing clinical status of newborn), in particular, between baseline and endline. High-quality intrapartum facility-based care is critical for reducing maternal and early neonatal mortality, and this evaluation of the LifeNet intervention indicates that their clinical training programme improved the practice of quality maternal and neonatal healthcare at all six primary care clinics in Uganda, at least over a relatively short-term period. However, for several of these quality indicators, the adherence rates, although improved, were still far from 100% and could benefit from further improvement via refresher trainings and/or a closer examination of the barriers to adherence.

This quasi-experimental study was a pretest–posttest observational study design to estimate the effects of a clinical and management training intervention delivered by an international non-governmental organization (NGO) [LifeNet International] on improvements in QoC for maternal and neonatal healthcare in six LifeNet-affiliated health facilities in the greater Masaka district area, Uganda. The training intervention period was August 1, 2017 to May 29, 2018. Baseline data were collected prior to initiation of the LifeNet training programme (May 15 to July 17, 2017) and post-intervention endline data were collected immediately after the completion of the training programme (May 29 to August 12, 2018). LifeNet International, a registered as a 501(c)(3) nonprofit organization in the USA, and an international NGO in Uganda, had previously developed and implemented an integrated health training package that requires 2-years of engagement with selected partner facilities in Uganda. For the purposes of this study, a modified training intervention was designed so that for our study sites, the first 10 months were front loaded with training modules relevant to MNH. The intervention used on-site monthly staff training, addressed team-based behaviour, incorporated quality assurance activities and focused on both medical (e.g. evidence-based clinical care) and management (e.g. record keeping, essential medicine monitoring and management) knowledge and practice tools to support implementation of high-quality care. The MNH modules were evidence-based best practices that had been validated and endorsed by the international medical community to address the leading causes of mortality and morbidity for mothers and newborns in Uganda (American Academy of Pediatrics (AAP), 2016; American Congress of Obstetricians and Gynecologists (ACOG), 2015; WHO, 2011; WHO, 2012a,b; WHO, 2015b,c; WHO, 2017a). The trainings were delivered at each facility on a rotating monthly basis. The primary clinical trainer held a bachelor’s degree in public health and a diploma in nursing, while the primary management trainer had a bachelor’s degree in accounting. Trainers delivered these in-facility ∼2 hour long didactic and hands-on modules to the majority of the clinic staff at a designated time and then followed-up with an additional training for any clinicians who could not attend due to running normal operations during the training. Training materials given to staff included partographs and clinical best practice sheets that summarized the training content. The trainings were a mix of lectures, videos, demonstrations and practice (e.g. assembling a condom tamponade to control postpartum haemorrhage (PPH), neonatal resuscitation practice on a dummy, etc.). Each element of LifeNet’s MNH Package aligned with the Ugandan government’s 2016–2020 RMNCAH Sharpened Plan, which emphasizes evidence-based, high-impact health solutions (Republic of Uganda, 2016). There were 13 MNH-related training modules required by the clinical staff, and non-clinical administrative staff were required to attend at least sessions 1, 3 and 4 as noted below. Modules included (1) documentation and record keeping, (2) partograph and delivery records, (3) basic patient assessment, (4) infection prevention, (5) intravenous (IV) usage, (6) antenatal care, (7) hypertension and pre-eclampsia, (8) first trimester high-risk pregnancies, (9) second and third trimester high-risk pregnancies, (10) normal (uncomplicated) deliveries, (11) PPH, (12) first 5 minutes, APGAR (rapid scoring system for assessing clinical status of newborn), neonatal assessment and (13) neonatal resuscitation. There was significant variability between individual providers and between facilities on session attendance per module ranging from 56 to 100% attendance at individual sessions among relevant staff who should have attended. If new staff joined the clinics mid-evaluation, LifeNet tried to on-board and briefly ‘catch-up’ these new staff on missed trainings. There were no other known QoC interventions happening at the clinics during our study period to the best of our knowledge. Six rural, faith-based health facilities in the greater Masaka area, new to partnering with LifeNet International, participated in the study. Study facilities were selected based on their proximity to Masaka town (for research supervision purposes) and sufficient obstetric delivery volume (16+ deliveries per month). All facility managers were accredited by the Roman Catholic Diocese of Masaka. Maternal delivery fees averaged ∼30 000 UGX ($8.50 USD). The number of clinical employees per facility ranged from three to nine for all services and between two to six health providers for obstetric services. Clinical providers observed during the study included midwives, comprehensive nurses, clinical officers and doctors. All facilities were designated as able to serve pregnant women with uncomplicated deliveries, including women living with human immunodeficiency virus (HIV), per government guidelines that indicated they must be able to deliver basic emergency obstetric care. One facility was a level IV referral facility capable of performing surgeries including C-sections, with the remaining five facilities being of level III meaning they had no surgical capacity and thus no C-section services (WHO, 2017b). Additional details about study site selection and data collection procedures, including online access to data collection forms, have been published elsewhere (Egger et al., 2020). During both the pre- and post-intervention study periods, QoC data were collected using three methods Research assistants (RAs), all fluent in English and Luganda, were trained by Duke University, which led the external evaluation, to collect data via direct observation of clinical encounters, review of medical records and daily facility checklists. Ten RAs were deployed for baseline (pre-intervention) data collection and 10 RAs conducted endline (post-intervention) data collection. About half were hired for both timepoints and the RAs reflected a mix of clinical (licensed nurses) and non-clinical health research backgrounds. Study training for the RAs for each timepoint included a 5-day review of LifeNet’s quality improvement training programme delivered by LifeNet staff, followed by a 5-day training in study procedures and research ethics by the university-based study investigators. One to three RAs were assigned per study facility based on delivery volume. RAs were ‘on call’ to respond to all deliveries for observation at their clinics. A shift-schedule ensured that nearly all consecutive maternal deliveries were observed during the study periods. Deliveries not observed were assumed to be missed completely at random. The DCO form was informed by USAID’s (United States Agency for International Development) Maternal and Child Health Integrated Program (MCHIP) Maternal and Newborn QoC Survey for Labor and Delivery (2013). Study RAs assessed the extent to which providers adhered to best practice standards of care for all stages of maternal deliveries: initial client assessment, first stage of labour, second and third stages of labour, immediate newborn and postpartum care and medical information documentation, including emergency procedures of newborn resuscitation and PPH management. Data were recorded on paper-based DCO forms and then entered electronically into a Research Electronic Data Capture (REDCap) form. More than one RA could observe a single delivery if RAs changed shifts during a delivery. As such, the RA noted the sections of the delivery process that she observed. Several of the RAs had at least as much clinical training as the providers they observed. To maintain objectivity, the RAs were trained to intervene in the delivery only if they felt that they were critically needed, and if they believed either the life of the mother or the child was in danger. If the RA intervened in any way, this was recorded in the relevant notes section of the DCO form and the respective data were excluded from primary analyses. Data from medical records were largely from the post-intervention timepoint only and were typically extracted and directly entered into REDCap after the DCO data were entered. Part of the intervention was that clinics were trained to use a more comprehensive two-page medical record form developed by LifeNet in order to document clinical care and health outcomes in a standardized way across sites. While most clinics had access to individual partograph sheets from the Ministry of Health, the partograph was embedded in the LifeNet medical record form so patients had one in their record. During the discharge process, women were asked to provide a contact phone number to be included in their medical chart for the purposes of a brief 28-day follow-up check-in by the RAs or clinic staff. The facility checklist was also completed daily for each facility. The checklist was informed by the USAID’s MCHIP Facility Inventory Quality of Care Tool (2013a). Study RAs documented availability of resources, support systems and facility infrastructure elements necessary to provide a level of service intended to meet national or international standards. The checklist was completed on paper first and then entered into REDCap. Priority indicators of clinical quality for all deliveries were determined based on the global literature as noted above in the modules and were identified prior to data collection to align with each stage of maternal delivery: initial client assessment, first, second and third stages of labour, immediate newborn and postpartum care and medical information documentation. We also collected observational data on newborn resuscitation and PPH management as the circumstances arose. Adherence to each QoC indicator was defined as the proportion of eligible deliveries where the provider adhered to the recommended best practice and each was measured as a binary response (0/1). For each indicator, the numerator represented the number of deliveries where the provider adhered to best practices and the denominator represented all eligible deliveries. As a result, the denominator (N) differed for each indicator. Our study indicators reflect both ‘sensitive’ indicators, those that require direct observation for confirmation and often have a critical time element to reflect optimal best practice (e.g. maintaining glove sterility and timing of cord clamping) and non-sensitive or ‘crude,’ indicators that could, in theory, have been measured using standard techniques like medical chart abstraction or provider interviews. We believe that sensitive measures may be a better true measure of clinical quality and thus better illuminate potential effects on health impact, than crude measures. Therefore, we created multiple indicators per domain to highlight varying levels of sensitivity. There are a total of 17 QoC indicators relevant to all women presenting for delivery. For handwashing we have three indicators: provider washes hands at least once during labour and delivery, provider washes hands only once but it is for clean up after birth and provider washes hands at three important timepoints—right before initial vaginal examination, again during the first stage labour and then in preparation for delivery. For sterile glove use, we documented three indicators: use of any type of gloves, use of surgical gloves specifically and then use of surgical gloves without compromising their sterility prior to use. Sterile cord cutting and clamping had three indicators: use of a sterile cord clamp or sterile string to tie off the umbilical cord, use of a sterile blade or sterile scissors to cut the cord, and delayed cord clamping recorded as more than one minute after delivery. For partograph use, we have two indicators: any partograph use, either observed or documented at any time and partograph used in real-time to monitor labour. To prevent PPH, administration of a uterotonic during the second and third stages of labour was documented, and as part of best practice during initial assessment, testing the woman’s urine for the presence of protein was observed. Appropriate use and timing of the APGAR is best practice, and we had documented use at one and/or 5 minutes post-delivery via medical record documentation and/or observation. Our study was powered on the primary aim of estimating a difference in the prevalence of the QoC indicator, ‘real-time proper use of the partograph’, comparing the baseline and endline periods. With an expected 155 maternal deliveries observed in each of the two time periods, we estimated achieving greater than 80% power to detect a difference as small as 14% in the proportion of those encounters where the partograph was used properly when the baseline (i.e. ‘pre-’) proportion is estimated to be 20% (20 vs 34%, based on a Chi-squared test of independence, assuming a two-sided alpha level of 0.05). Our power was estimated to reach 90% if the difference in the two proportions was as large as or larger than 16.5%. We expected to see similar, if not higher, power on most of our other indicators. If the baseline proportion of the partograph indicator was lower than 20%, then our estimates of power were conservative. The study exceeded our target sample size requirements and achieved a priori statistical power to meet all of our primary research aims. Data were analysed using Stata 16 software (StataCorp, College Station, TX). Analysis of study data focused on the change in the prevalence (i.e. the adherence) from baseline (pre-intervention) to endline (post-intervention). Data were analysed separately at the individual (i.e. patient-provider encounter) and clinic cluster levels and results were compared. Individual-level analysis utilized a generalized estimating equation (GEE) framework to account for clustering in the response by clinic and assumed an exchangeable working correlation structure. Models assuming an independent working correlation structure were fit as a sensitivity analysis. The GEE model was fit to prevalent data with a log-link function using a modified Poisson regression approach (Zou, 2004). Models were fit in Stata using the xtgeebcv command, which allows for a finite sample bias correction due to the small number of clusters in our study (Gallis et al., 2020). We used the bias-corrected variance method proposed by Kauermann and Carroll (2001). Models included an indicator parameter for time (i.e. baseline vs endline), and exponentiation of this parameter estimated the population averaged prevalence ratio (PR). A positive value for the PR can be interpreted as an average increase in adherence over the study period. A negative value for the PR can be interpreted as an average decrease in the adherence over the study period. Cluster-level analyses were performed in MS Excel using methods for pair-matched clusters described in Hayes and Moulton (2017). Ethical approvals were obtained from [Duke University U.S. Academic Institution], The AIDS Support Organization in Uganda [not affiliated with authors], and the Uganda National Council of Science and Technology. All maternity clients (or her self-designated proxy) gave written informed consent to participate in the study and were provided with a copy of the consent form with contact information. Consent was obtained at admission for childbirth; however, our study team also confirmed consent again post-delivery. No participant rescinded their consent.

N/A

• baumgartner, joy noel
• headley, jennifer
• kirya, julius
• guenther, josh
• kaggwa, james
• kim, min kyung
• aldridge, luke r.
• weiland, stefanie
• egger, joseph r.