Assessing trends in the content of maternal and child care following a health system strengthening initiative in rural Madagascar: A longitudinal cohort study

Background: In order to reach the health-related Sustainable Development Goals (SDGs) by 2030, gains attained in access to primary healthcare must be matched by gains in the quality of services delivered. Despite the broad consensus around the need to address quality, studies on the impact of health system strengthening (HSS) have focused predominantly on measures of healthcare access. Here, we examine changes in the content of maternal and child care as a proxy for healthcare quality, to better evaluate the effectiveness of an HSS intervention in a rural district of Madagascar. The intervention aimed at improving system readiness at all levels of care (community health, primary health centers, district hospital) through facility renovations, staffing, equipment, and training, while removing logistical and financial barriers to medical care (e.g., ambulance network and user-fee exemptions). Methods and findings: We carried out a district-representative open longitudinal cohort study, with surveys administered to 1,522 households in the Ifanadiana district of Madagascar at the start of the HSS intervention in 2014, and again to 1,514 households in 2016. We examined changes in healthcare seeking behavior and outputs for sick-child care among children <5 years old, as well as for antenatal care and perinatal care among women aged 15-49. We used a difference- in-differences (DiD) analysis to compare trends between the intervention group (i.e., people living inside the HSS catchment area) and the non-intervention comparison group (i.e., the rest of the district). In addition, we used health facility-based surveys, monitoring service availability and readiness, to assess changes in the operational capacities of facilities supported by the intervention. The cohort study included 657 and 411 children (mean age = 2 years) reported to be ill in the 2014 and 2016 surveys, respectively (27.8% and 23.8% in the intervention group for each survey), as well as 552 and 524 women (mean age = 28 years) reported to have a live birth within the previous two years in the 2014 and 2016 surveys, respectively (31.5% and 29.6% in the intervention group for each survey). Over the two-year study period, the proportion of people who reported seeking care at health facilities experienced a relative change of +51.2% (from 41.4% in 2014 to 62.5% in 2016) and -7.1% (from 30.0% to 27.9%) in the intervention and non-intervention groups, respectively, for sick-child care (DiD p-value = 0.01); +11.4% (from 78.3% to 87.2%), and +10.3% (from 67.3% to 74.2%) for antenatal care (p-value = 0.75); and +66.2% (from 23.1% to 38.3%) and +28.9% (from 13.9% to 17.9%) for perinatal care (p-value = 0.13). Most indicators of care content, including rates of medication prescription and diagnostic test administration, appeared to increase more in the intervention compared to in the non-intervention group for the three areas of care we assessed. The reported prescription rate for oral rehydration therapy among children with diarrhea changed by +68.5% (from 29.6% to 49.9%) and -23.2% (from 17.8% to 13.7%) in the intervention and non-intervention groups, respectively (p-value = 0.05). However, trends observed in the care content varied widely by indicator and did not always match the large apparent increases observed in care seeking behavior, particularly for antenatal care, reflecting important gaps in the provision of essential health services for individuals who sought care. The main limitation of this study is that the intervention catchment was not randomly allocated, and some demographic indicators were better for this group at baseline than for the rest of the district, which could have impacted the trends observed. Conclusion: Using a district-representative longitudinal cohort to assess the content of care delivered to the population, we found a substantial increase over the two-year study period in the prescription rate for ill children and in all World Health Organization (WHO)-recommended perinatal care outputs assessed in the intervention group, with more modest changes observed in the non-intervention group. Despite improvements associated with the HSS intervention, this study highlights the need for further quality improvement in certain areas of the district's healthcare system. We show how content of care, measured through standard populationbased surveys, can be used as a component of HSS impact evaluations, enabling healthcare leaders to track progress as well as identify and address specific gaps in the provision of services that extend beyond care access. Since 2014, PIVOT and Madagascar’s MoH have collaborated to design, implement, and assess an HSS intervention in the southeastern district of Ifanadiana (Fig 1) as a model healthcare delivery system for the country [31,32]. In the first two years of the intervention, the PIVOT-MoH partnership primarily focused on a catchment area that included 4 out of the district’s 13 communes, encompassing approximately one third of the 200,000 people living in Ifanadiana. This initial intervention area contains the district’s sole hospital and four of its primary health centers. The choice of the catchment area was done according to logistical and programmatic reasons, and there was no randomization of communes involved. Guided by the World Health Organization’s (WHO) framework for functional HSS [9], the partnership targeted all three levels of care governed by the MoH in the catchment area (community health, primary health centers, district hospital) [32]. In brief, to improve readiness, PIVOT-MoH renovated, staffed, and equipped the hospital and health centers located in the catchment area since mid-2014, as well as initiated a community health program in a subset of the catchment’s remote villages by November 2015. In addition, the partnership sought to remove logistical and financial barriers to medical care by creating an ambulance network and removing fees for commonly prescribed medications for all patients [38]. PIVOT-MoH also implemented WHO’s Integrated Management of Childhood Illness (IMCI) guidelines [39] and national guidelines for the treatment for severe acute malnutrition, as well as had social workers at health facilities for the accompaniment and follow-up of vulnerable patients. Details on the intervention are available in S1 TIDIER Checklist using the Template for Intervention Description and Replication (TIDieR) [40]. Ifanadiana has an estimated population of 200,000 people, approximately one third of which live within PIVOT’s initial catchment area. The district is comprised of 13 communes (demarcated by black lines) and 195 fokontany (the smallest administrative unit, demarcated by blue lines). Each of the communes contains a primary health center (Centre de Santé de Base 2: red diamonds and green squares). Between 2014 and 2016, PIVOT and the MoH renovated, staffed, and equipped four of them (red diamonds). The households surveyed by the PIVOT-MoH longitudinal cohort study that were located in villages nearest to a PIVOT-supported health center were categorized into the intervention group (red dots); the households located in villages nearest to a nonsupported health center were categorized into the non-intervention group (green dots). Map of Madagascar in the top right corner, with the Ifanadiana district colored in red. Base maps obtained from INSTAT and GADM.org. INSTAT, National Institute of Statistics; MoH, Ministry of Health. During this period, two independent groups implemented complementary health programs in the Ifanadiana district. The World Bank–funded Emergency Support to Critical Education, Health and Nutrition Services (PAUSENS) project created a basic package of health, nutrition, and reproductive health services in the district’s 13 primary health centers (Centre de Santé de Base 2), available free of charge to all pregnant women and children under 5 years [41]. This program provided equipment and medication for pharmacies and health centers, as well as training for obstetrics and neonatal care. Second, the United States Agency for International Development (USAID) funded Mikolo project provided some training and limited supervision for 150 community health workers stationed in remote villages (approximately half located inside the PIVOT-MoH catchment area) on monitoring and counseling for basic health practices in their community [42]. The main difference in healthcare provision between the PIVOT-MoH catchment area and the rest of the Ifanadiana district was the support in infrastructure building, training and staffing, removal of user fees, and additional support to guideline implementation of clinical programs provided by PIVOT. We collaborated with Madagascar’s National Institute of Statistics (INSTAT) to create an ongoing open longitudinal cohort study of 1,600 households (the sample size required to estimate under-five mortality within a 12% margin of error) representative of the Ifanadiana district’s population—The Ifanadiana Health Outcomes and Prosperity longitudinal Evaluation (IHOPE) [34]. The households were selected through a two-stage cluster sampling scheme. Maps from the 2009 census were used to divide the district into 169 geographical clusters, after which 40 clusters from within and 40 clusters from outside the initial catchment area were chosen at random, with probabilities proportional to population size (Fig 1). Within each cluster, an enumeration was done to obtain a complete household listing, and 20 households were randomly selected prior to conducting the survey. A questionnaire adapted from the Demographics and Health Survey (DHS) [43] was administered in person to all men aged 15–59 and women aged 15–49 living in the enrolled households at baseline (April–May 2014). This survey was repeated two years after the initiation of the intervention (August–September 2016). Households that were unavailable or declined participation in the second survey were replaced with additional households from the original sampling lists; families and individuals that moved into original households were also included in the second survey (for details, see Miller and colleagues [34]). We collected data on household characteristics, socioeconomic status, and maternal and child health, among others. Overall, among the 1,600 households sampled in each survey, 1,522 provided data in 2014 and 1,514 in 2016 (95.1% and 94.6% response rate, respectively). To measure changes in service availability throughout PIVOT’s HSS intervention, we conducted facility surveys in 2014 and 2015 at the four PIVOT-support health centers in the catchment area. These were based on the WHO Service Availability and Readiness Assessment (SARA) framework [44], which was adapted to the Malagasy health system context and norms. We assessed availability of preventative and therapeutic services, health facility personnel level, supply of essential medicines, and basic functional medical equipment, among others (S1 Appendix). Evaluations of non-PIVOT supported health centers were not conducted at that time. Hospital-level norms for service availability and readiness were updated by the MoH during the study period, precluding a longitudinal follow-up and analysis of the district hospital from baseline values. The IHOPE survey was approved by the Madagascar National Ethics Committee and Harvard Medical School’s IRB. Verbal consent was obtained from adults (18–59 years old) and from parents or legal guardians for their children (under 18 years), with assent from minors (15–18 years old) to participate in the study. The SARA survey was authorized by the MoH. We carried out our analysis on de-identified data. We reported this study as per the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines [45], which are available in S1 STROBE checklist. A prospective analysis plan for measuring changes in care seeking behavior was designed as part of the IHOPE cohort study and has been published in Miller and colleagues (2018) [34]. The current analysis of content of care was done retrospectively in 2017 to provide complementary insights around healthcare quality. We focused on three areas of healthcare: sick-child care, antenatal care, and perinatal care. To benchmark appropriate care, we used as our standard WHO’s IMCI guidelines [39], as well as their guidelines for maternal and newborn health [46,47] (S1 Appendix). Based on the parents’ responses, sick-child care seeking was measured as the proportion of children under five who had diarrhea (symptom of possible gastroenteritis), persistent cough with difficulty breathing (symptoms of possible acute respiratory infection), or fever (symptom of possible malaria infection) within the previous two weeks and who were brought to care for that indication either at one of MoH’s public health facilities (PHFs)—which includes the district hospital and the district’s primary health centers—or at any of the community health worker sites (CHWs) located throughout the district. The content of sick-child care was assessed by examining a variety of treatments and diagnostic procedures that ill children could receive for a particular symptom in accordance with IMCI guidelines. For each possible treatment, we measured the proportion of symptomatic children to whom it was prescribed at a PHF or CHW, as reported by parents in the IHOPE surveys. We examined antenatal and perinatal care among women aged 15–49 who had a live birth within the previous two years. Antenatal care seeking was assessed using three indicators: the proportion of pregnant women who attended an antenatal consultation at a PHF (a) at least once in their pregnancy, (b) at least four times, and (c) at least once during their first trimester. Perinatal care seeking was measured as the proportion of women who delivered at a PHF. Content of antenatal care was assessed by examining the coverage of several screening tests pregnant women reported to have received during at least one of their antenatal consultations at a PHF. For perinatal care, we assessed the rate of recommended newborn and maternal health assessments that women reported in the IHOPE surveys following their child’s delivery at a PHF. Based on results from the SARA surveys conducted in PIVOT-supported health centers, we examined the availability of the following services: (a) preventative services, (b) therapeutic services, (c) health promotion and administration services, and (d) complementary services (e.g., tuberculosis care, malnutrition care). We also examined health centers’ readiness to provide general healthcare services based on levels of (a) personnel (b) basic amenities (e.g., power source, water source), (c) basic equipment, and (d) essential medicines. For each indicator of service availability and readiness, we measured the available proportion of components included in the indicator, as defined by MoH/WHO standards (S1 Appendix). Given that the vast majority of the district’s population accesses health centers by foot, we stratified participants of the IHOPE surveys into an intervention and a non-intervention group based on geographic proximity of their household to a PIVOT-supported or nonsupported health center (Fig 1). Using the global positioning system (GPS) coordinates of each health center and each of the 80 geographical clusters randomly sampled (centroid of all the villages in a cluster), we identified the health center that was the shortest Euclidean distance away from a given cluster. We considered participants to be in the intervention group if they lived in households located in a cluster for which the nearest facility was a PIVOT-supported health center and in the non-intervention group otherwise. Using the standard protocols for DHS surveys [48], all results from the IHOPE surveys were adjusted using sampling weights, which account for the unequal probability of household selection, depending on the population size of each cluster surveyed and nonresponse rates. We did not adjust for potential spatial autocorrelation through mixed effects models. We used a chi-squared statistic to assess differences between outcomes in the intervention and non-intervention group for each year, and a difference-in-differences (DiD) regression analysis to estimate the overall effect of PIVOT-MoH’s intervention over time (S1 Appendix). We also compared DiD estimates, unadjusted and adjusted, with household wealth. All results from the SARA surveys were averaged across the four health centers evaluated, and changes over time were reported. We analyzed all data with R software using the “survey,” “ggplot2,” “sp,” “maptools,” “rgeos,” and “foreign” packages.