Impact of a health system strengthening intervention on maternal and child health outputs and outcomes in rural Rwanda 2005-2010

Introduction Although Rwanda’s health system underwent major reforms and improvements after the 1994 Genocide, the health system and population health in the southeast lagged behind other areas. In 2005, Partners In Health and the Rwandan Ministry of Health began a health system strengthening intervention in this region. We evaluate potential impacts of the intervention on maternal and child health indicators. Methods Combining results from the 2005 and 2010 Demographic and Health Surveys with those from a supplemental 2010 survey, we compared changes in health system output indicators and population health outcomes between 2005 and 2010 as reported by women living in the intervention area with those reported by the pooled population of women from all other rural areas of the country, controlling for potential confounding by economic and demographic variables. results Overall health system coverage improved similarly in the comparison groups between 2005 and 2010, with an indicator of composite coverage of child health interventions increasing from 57.9% to 75.0% in the intervention area and from 58.7% to 73.8% in the other rural areas. Under-five mortality declined by an annual rate of 12.8% in the intervention area, from 229.8 to 83.2 deaths per 1000 live births, and by 8.9% in other rural areas, from 157.7 to 75.8 deaths per 1000 live births. Improvements were most marked among the poorest households. Conclusion We observed dramatic improvements in population health outcomes including under-five mortality between 2005 and 2010 in rural Rwanda generally and in the intervention area specifically.

We evaluated the impact of the RMOH-PIH interventions by comparing the temporal trends in health outputs and outcomes between 2005 and 2010 in the intervention target region to those in the pooled population of all ORAs. First, we assessed health system outputs, focusing on a set of indicators meant to capture the coverage of maternal and child health services and then we assessed population outcomes including neonatal, infant and under-five mortality. We used Rwanda Demographic and Health Survey (RDHS) data collected from 21 338 women living in Kirehe/S. Kayonza (K/SK) and ORAs in 2005 (K/SK: 418, ORA: 8217) and 2010 (K/SK: 2073, ORA: 10 630) (table 1). RDHSs are nationally and subnationally representative two-stage cluster samples conducted roughly every 5 years by the RMOH, National Institute of Statistics-Rwanda (NISR) and ICF International. The surveys collect information from women aged 15–49 on their reproductive health histories, practices and desires; household composition; siblings’ survival; and children’s health and survival. The DHS birth history module included the date of the birth and death of each child born alive, and through the sibling module, the age and date of death for each biological sibling (see online supplement). The 2005 RDHS was underway at the onset of the RMOH-PIH collaboration. In order to expand the sample size to allow us to compare the intervention area to other areas, we coordinated with the NISR immediately following the 2010 RDHS to collect a supplemental sample of 1391 households from 54 primary sampling units (PSUs) in Kirehe/S. Kayonza using the same sampling frame, staff and questionnaires as the 2010 RDHS (figure 1).12 Most data were collected consistently across the three surveys (2005 RDHS, 2010 RDHS and Supplemental RDHS) although neither stunting and wasting was assessed in the supplemental survey. The response rates for surveyed women were 98% in 2005 and 99% in 2010. The RDHS urban/rural boundaries were adopted from the Rwandan government. Exact urban boundaries are not published, however, urban areas are described as being built up, population dense and having public services and facilities. All areas not meeting this definition are considered rural.14 Maps of RDHS strata and primary sampling units in 2005 (left) and 2010 (right) and the PIH-RMOH intervention area in southeastern Rwanda (green). PIH, Partners In Health; RDHS, Rwanda Demographic and Health Survey; RMOH, Rwandan Ministry of Health. Summary of sociodemographic characteristics in 2005 and 2010 *Based on t-test from an OLS regression with year, group and year–group interaction terms. †With respect to 2005 wealth score definition. K/SK, Kirehe/S. Kayonza; OLS, ordinary least squares; ORA, other rural area. We assessed the following child health-related health system output indicators: whether treatment was provided for recent episodes of ARI, diarrhoea or fever in children under age 5 years; whether children under age 2 years received the recommended three doses of diphtheria-polio-tetanus (DPT) or measles vaccine; whether children received vitamin A supplementation between age 6 months and 1 year; whether children were exclusively breastfed for the first 6 months of life; whether at least one, or the recommended four, antenatal care visits took place during the last pregnancy; whether the most recent birth was attended by a skilled health worker; whether the birth was delivered by caesarean section; whether women received postnatal care within 24 hours of delivery; women’s current contraceptive use; and their unmet need for contraception. The following population health outcome indicators were assessed: neonatal, infant and under-five mortality; adult mortality (men, women and combined); recent occurrence of ARI, diarrhoea or fever in children under age 5 years; and stunting and wasting in children under age 5 years.15 We calculated a composite coverage index (CCI) for both groups to monitor overall healthcare coverage across time in the intervention and comparison areas based on that proposed by Barros and Victoria (2013) but modified to exclude BCG tuberculosis vaccination coverage (which was not available) as an indicator.16 The modified CCI is a weighted average for a group that includes the prevalence of met need for contraception (FPS), skilled birth attendance (SBA), at least one ANC visit with a skilled provider (ANCS), third DTP vaccination (DPT3), measles vaccination (MSL), oral rehydration therapy (ORT) for children with diarrhoea and care seeking for pneumonia symptoms (CPNM) as follows: We integrated the data from the supplemental survey with that from the 2010 RDHS as follows. Sampling probability weights were recalculated for the combined 2010 dataset. To protect respondent confidentiality, we randomly geodisplaced PSU latitude/longitude coordinates in the supplemental survey up to 5 km within district boundaries according to DHS guidelines.17 All geographic information was linked to displaced PSU locations in a geographic information system (ArcGIS V.10, ESRI). We combined latitude/longitude coordinates and rural residence information to identify respondents living in Kirehe/S. Kayzona and ORAs. Additional PSU geographic characteristics included straight-line distance to the nearest main road in metres (downloaded from DIVA-GIS database), straight-line distance to Kigali province in kilometres (downloaded from Map Library database), elevation above sea level in metres (from RDHS) and 30-year (1971–2000) average total rainfall in millimetres during the months of January, April, July and October (downloaded from US National Oceanic and Atmospheric Administration Climate Prediction Center). We generated comparable household wealth scores for 2010 using the principle components generated from the 2005 RDHS,18 and we considered a household to be ‘poor’ if it ranked in the bottom 20% of wealth scores of the pooled 2005, 2010 and supplemental survey datasets. Three rural PSUs (representing 69 households and 70 women) were excluded from the 2005 dataset because the PSU GPS coordinate was missing, preventing the PSU from being linked to either Kirehe/S. Kayonza or ORA. We compared baseline differences in woman, household and community (PSU) characteristics between the two comparison groups using χ2 tests and t-tests and temporal changes in and between groups using ordinary least squares regression with a year, group and year-by-group interaction term. Since the intervention area and ORAs may differ by factors that would have an impact on child mortality but which would not be altered by our health system strengthening intervention, we also compared the following social and geographic characteristics of sampled communities: fraction of each PSU with improved water, fraction of PSU adults who received a primary education, distance of PSU to a main road and to Kigali, elevation and average total rainfall in specified months. Finally, we compared health system outputs and population health measures at baseline. Kirehe/S. Kayonza had been chosen as the site of the intervention, because it had the highest rate of under-five mortality in Rwanda in 2005; thus, it was not possible to identify another Rwandan rural site with comparable under-five mortality. In an attempt to identify an optimal comparison group for Kirehe/S. Kayonza, we first limited our comparison area to Eastern Province in proximity to the intervention area (see figure 1). We found under-five mortality was higher in the intervention area than the rest of Eastern Province or any other subregion, and comparisons of other indicators were mixed (see online supplement). When a subregion was not identified for comparison, we used propensity score matching with inverse probability of treatment weights to identify comparable communities from ORAs by assessing PSU characteristics that might differ between the intervention and comparison areas but which would not be expected to be altered by the intervention (distance to road, distance to Kigali, elevation and average rainfall). We generated a bias B value to capture the difference in the standard deviation between the means of the groups and an R value that is the ratio of variances in the two groups. Following Rubin, we considered the groups to be balanced if B was less than 25% and R was between 0.5 and 2.19 Kirehe/S. Kayonza and ORAs were balanced by the R value but not by the B bias value (B=340.9, R=0.54) (see online supplement). Since neither approach identified a more appropriate comparison group, we compared Kirehe/S. Kayonza to all ORAs adjusting models for sociodemographic characteristics that followed different trends in Kirehe/S. Kayonza and ORAs over time. We used ordinary least squares regression with group, year and group–year interaction terms to model changes in binary health outputs and outcomes, controlling for differences in woman’s age and household wealth. We modelled change in childhood mortality rates using the DHS synthetic life-table approach, which uses the histories of all children a mother reports to have been alive during the previous 5 years.20 Adult mortality rates were based on a 5-year synthetic cohort of respondent’s siblings’ births and deaths.21 Expected mortality rates were calculated by standardising mortality rates of ORAs to the age structure in Kirehe/S. Kayonza. We estimated mean changes between 2005 and 2010 as the absolute difference in rates, and we calculated variances of trends as the sum of year–group variances. We adjusted for clustering of observations by PSU using Taylor linearised variance estimation in regression models and jackknife repeated replications to estimate variance in all other analyses.21 We conducted regressions in Stata V.13 and mortality analyses in SAS V.9.2. Verbal consent was obtained for all respondents before interviews took place. Protocols for the Rwanda 2005 and 2010 DHSs were approved by the Rwandan government. Protocols for the 2010 supplemental survey were reviewed and approved by the Partners HealthCare Internal Review Board (protocol #: 2009 P-001941/8) and the Rwanda National Ethics Committee. The funders had no role in study design, data collection, data analysis, data interpretation, writing of this report, nor the decision to submit this paper for publication. The corresponding author (DRT) had full access to all the data in the study and had final responsibility for the decision to submit for publication.

N/A

• Dima Health