Effect of a maternal and newborn health system quality improvement project on the use of facilities for childbirth: a cluster-randomised study in rural Tanzania

Objectives: Reduction in maternal and newborn mortality requires that women deliver in high quality health facilities. However, many facilities provide sub-optimal quality of care, which may be a reason for less than universal facility utilisation. We assessed the impact of a quality improvement project on facility utilisation for childbirth. Methods: In this cluster-randomised experiment in four rural districts in Tanzania, 12 primary care clinics and their catchment areas received a quality improvement intervention consisting of in-service training, mentoring and supportive supervision, infrastructure support, and peer outreach, while 12 facilities and their catchment areas functioned as controls. We conducted a census of all deliveries within the catchment area and used difference-in-differences analysis to determine the intervention’s effect on facility utilisation for childbirth. We conducted a secondary analysis of utilisation among women whose prior delivery was at home. We further investigated mechanisms for increased facility utilisation. Results: The intervention led to an increase in facility births of 6.7 percentage points from a baseline of 72% (95% Confidence Interval: 0.6, 12.8). The intervention increased facility delivery among women with past home deliveries by 18.3 percentage points (95% CI: 10.1, 26.6). Antenatal quality increased in intervention facilities with providers performing an additional 0.5 actions across the full population and 0.8 actions for the home delivery subgroup. Conclusions: We attribute the increased use of facilities to better antenatal quality. This increased utilisation would lead to lower maternal mortality only in the presence of improvement in care quality. The maternal and newborn health quality improvement (MNH+) study was implemented in four rural districts of Pwani Region, Tanzania: Bagamoyo, Kibaha Rural, Kisarawe, and Mkuranga. The study is registered through ISRCTN (http://www.isrctn.com/ISRCTN17107760). Government‐managed primary care facilities (i.e., dispensaries) and their official catchment areas define the clusters for this study. Facilities were eligible for inclusion in the MNH+ study if they were supported for prevention of maternal to child transmission of HIV care by a local non‐governmental organisation (Tanzania Health Promotion Support, THPS) and had at least one skilled healthcare provider at the start of the start of the study (e.g. a nurse or clinical officer). From the eligible facilities, within each of the four districts the six facilities with the highest volume of deliveries between January and June 2011 where chosen, resulting in 24 study facilities. At baseline, three of six facilities within each district were randomly selected to receive the MNH+ intervention (12 facilities total). Random selection was conducted by drawing facility names from a hat in the presence of representatives from the research team and the regional medical office. The intervention included three components to improve facility quality: infrastructure improvement (facility upgrades and ensuring basic equipment and supplies), provider training and supervision (continuing medical education, supportive supervision and mentoring), and peer outreach to promote facility utilisation for childbirth within the official catchment communities. Implementation of the intervention began in June 2012; by July 2013 the full intervention was underway and continued until after endline data collection was completed. Women were eligible to participate if they had delivered a child 6 weeks to 1 year prior to interview, lived in the catchment area of a study facility, and were at least 15 years of age. All survey participants provided written, informed consent, or assent together with permission from a guardian in the case of minors, prior to participation. Ethics review boards in both Tanzania, Ifakara Health Institute and the National Institute for Medical Research, and in the U.S., Columbia University and Harvard University, approved this study. The baseline round of data collection was conducted from 13 February to 28 April 2012. Midline data were collected from 3 February 2014 to 31 March 2014. The endline round was conducted from 20 January to 7 April 2016. At baseline and endline the study team enumerated all households in each catchment area and invited all eligible women to participate in the study. During the midline data collection, the study team again enumerated all households in each catchment area to create a list of eligible women. From this list, a simple random sample of 60% of women stratified at the facility level was invited to participate in the study. Trained Tanzanian research assistants conducted the household survey in Swahili using hand‐held tablets. Tanzanian research staff translated the survey to Swahili and back‐translated to English by consensus. In our prospective analysis plan we determined a limited set of predictors of utilisation that would be included in statistical analyses. To select these predictors we started from Anderson’s utilisation model of predisposing characteristics and a review of recent literature 20, 25, 26. The individual‐level demographic factors included women’s religion, age, marital status, parity, educational achievement, primary occupation, and season of delivery. We constructed an indicator of relative household wealth using principal components analysis of an 18‐question asset index 27. To indicate a woman’s exposure to mass media we constructed an additive media index utilising three questions that are also measured in the DHS: frequency of exposure to radio, newspapers, and TV (range 0–9). Finally, as an indicator of potentially changing community‐level physical access to the health system, we included an indicator for whether a woman’s village had a paved road. To assess potential pathways through which the intervention could affect utilisation (analyses described below) we collected information on antenatal care (ANC) quality, perceived obstetric quality, link between facility and community, and payment for obstetric care. For ANC quality we asked women if they had received the following during antenatal care and created an index of nine items: weight measured, height measured, blood pressure measured, urine sample collected, blood sample collected, tetanus injection administered, iron supplements provided, antimalarial medications provided and counselled on pregnancy complications. We also asked them to rate their perceived quality of ANC and separately their perceived quality of obstetric care on five‐level Likert scales ranging from poor to excellent. We categorised high‐perceived quality as a rating of ‘very good’ or ‘excellent’. To assess the link between the facility and community we asked women if they had heard of a quality improvement program in their local facility. Finally, to measure their payment for care we asked women how much they paid for all services, including any informal payments, or tips. Completed surveys were formatted as CSV files and imported into Stata version 14.1 for cleaning and analysis. The primary study outcome was facility utilisation for childbirth, which was measured as the proportion of women in the facilities’ official catchment area whose most recent delivery occurred in a healthcare facility. Women who reported delivering on the way to a facility were removed from the analysis (72 women, 1.2%). We conducted descriptive statistics of the primary outcome as well as the level of facility women delivered at (i.e. dispensary, health centre, or hospital), women’s place of delivery for her delivery prior to the index child (the index child is defined as the woman’s most recent delivery), and demographic and household characteristics. To illustrate the changing patterns of utilisation we plotted the proportion of deliveries that occurred at any health facility with lowests trends for each month. The graph includes all months where the entire month fell within eligibility for participation in the study (April 2011–January 2012, April 2013–January 2014, April 2015–January 2016). To measure the effect of the MNH+ intervention on facility utilisation for childbirth we conducted a difference‐in‐differences analysis. This analysis controls for both differences between utilisation patterns between facilities at baseline and changing patterns over time that are external to the intervention, but consistent across the region. In all models a fixed effect for district was included to account for stratification of the study facilities by district during the design phase. We used generalised estimating equations with an exchangeable correlation structure and a log link to estimate risk ratios for all models 28. The robust sandwich estimator was used to account for clustering at the facility level. In order for the difference‐in‐differences coefficient to be a valid estimator of causal effect, the parallel trends assumption must be satisfied; that is the assumption that in the absence of the intervention the intervention and control facilities would have had similar increases in facility utilisation for childbirth. We tested this assumption by creating a dataset of repeat cross‐sectional data for each month where we had birth data for at least 10 women prior to start of the intervention. We conducted several additional robustness checks and sensitivity analyses, which are described in the Appendix S5. These included different evaluation models (e.g. a post‐only analysis), and different statistical methods (e.g. Fischer Permutation tests). Recognising that women who have previously delivered at home have an increased likelihood of subsequent home delivery 29, we conducted a secondary analysis in which our sample was restricted to women whose most recent delivery prior to her index child was a home birth to test whether the intervention had a differential effect on this group (Appendix S3). This difference‐in‐differences analysis of women with previous home births was not pre‐specified but added when baseline data indicated high overall utilisation relative to hypothesised levels. This allowed us to explore effects of the intervention on a group that has historically been difficult to reach with safe delivery coverage expansion efforts. We found evidence of modification of the effect of the intervention by place of a woman’s previous birth and therefore conducted a stratified analysis by women whose delivery prior to her index child was a home birth versus women whose prior delivery was a facility birth or were primiparous. We explored four pathways through which an investment in quality could lead to increased utilisation of facilities for childbirth: ANC quality, perceived obstetric quality, link between facility and community and payment for obstetric care (Figure 1). Our model is informed by prior theoretical work as well as empirical evidence demonstrating women’s stated and revealed preferences for high quality care 5, 17, 25, 30, 31. Because the implementation evaluation found that the intervention did not affect the quality of delivery care, we do not present these results here. For each pathway we explore first the effect of the intervention on the intermediary outcome, then the association between the intermediary outcome and facility utilisation for childbirth at baseline. Conceptual framework for pathways through which investment in quality could affect facility utilisation for childbirth. [Colour figure can be viewed at http://wileyonlinelibrary.com]