Menu
Menu
Menu
Menu
Introduction Despite renewed commitment to universal health coverage and health system strengthening (HSS) to improve access to primary care, there is insufficient evidence to guide their design and implementation. To address this, we conducted an impact evaluation of an ongoing HSS initiative in rural Madagascar, combining data from a longitudinal cohort and primary health centres. Methods We carried out a district representative household survey at the start of the HSS intervention in 2014 in over 1500 households in Ifanadiana district, and conducted follow-up surveys at 2 and 4 years. At each time point, we estimated maternal, newborn and child health coverage; economic and geographical inequalities in coverage; and child mortality rates; both in the HSS intervention and control catchments. We used logistic regression models to evaluate changes associated with exposure to the HSS intervention. We also estimated changes in health centre per capita utilisation during 2013 to 2018. Results Child mortality rates decreased faster in the HSS than in the control catchment. We observed significant improvements in care seeking for children under 5 years of age (OR 1.23; 95% CI 1.05 to 1.44) and individuals of all ages (OR 1.37, 95% CI 1.19 to 1.58), but no significant differences in maternal care coverage. Economic inequalities in most coverage indicators were reduced, while geographical inequalities worsened in nearly half of the indicators. Conclusion The results demonstrate improvements in care seeking and economic inequalities linked to the early stages of a HSS intervention in rural Madagascar. Additional improvements in this context of persistent geographical inequalities will require a stronger focus on community health.
Ifanadiana is a rural health district of approximately 180 000 people located in the region of Vatovavy-Fitovinany, in southeastern Madagascar. Per Ministry of Public Health (MoPH) norms, Ifanadiana district has one reference hospital, one main primary care health centre (CSB2) for each of its 15 communes (subdivision of a district with ~15 000 people; two additional CSB2s were built in 2016 and 2018), six additional basic health centres for larger communes (CSB1) and one community health site with two community health workers (CHW) for each of its 195 fokontany (subdivision of a commune with ~1000 population). The integrated district-level health system strengthening intervention (referred to as idHSS) carried out by the MoPH-PIVOT partnership is guided by existing MoPH policies and is implemented across all three levels of care in the district (community, health centre and hospital). This intervention (summarised in table 1 and online supplemental table S1) is structured through the integration of clinical programmes, health system ‘readiness’ and information systems. The clinical programmes include child health, with a focus on malnutrition and integrated management of child illness; maternal and reproductive health; social support; and infectious disease programmes, with a focus on tuberculosis, malaria and emerging diseases. Clinical programmes are implemented throughout community health sites, primary healthcare centres and district hospital (details can be found in33). Readiness includes infrastructure and sanitation, staffing and equipment to improve the quality of care; procurement systems; an ambulance network; trainings, frequent supervision and coaching of health staff. As part of the vision for UHC to increase healthcare access and reduce financial vulnerability, user fees are removed at all levels of care via payments from PIVOT to health facilities on behalf of patients (Section S1 and online supplemental table S2), and social support is provided to vulnerable patients. The core activities in the first 3 years (2014 to 2016) covered four communes with approximately one-third of the population of Ifanadiana (referred to as ‘idHSS catchment’). In 2017, idHSS activities expanded to a fifth commune. Some activities such as medical staff recruitments and strengthened information systems spanned the whole district (online supplemental table S1). The idHSS intervention, initially modelled after an earlier experience in HSS in Rwanda,34 35 is being tailored over time in collaboration with the government to respond to coverage gaps and intervention deficiencies identified through an iterative learning process that uses health system and population-level data (online supplemental table S1). Summary of HSS interventions implemented in Ifanadiana district between 2014 and 2017, classified by building block of HSS* affected *Building blocks of HSS: (1) Service delivery, (2) Health workforce, (3) Health information systems, (4) Medicines and supplies, (5) Financing and (6) Leadership and governance. †Implemented by PAUSENS programme (World Bank) ‡Implemented by Mikolo programme (USAID) CHW, community health workers; HMIS, Health Management and Information Systems; HSS, health system strengthening; idHSS, integrated district-level health system strengthening; IMCI, integrated management of child illness; M&E, Monitoring and Evaluation; MNCH, maternal, newborn and child health; MoPH, Ministry of Public Health. bmjgh-2020-003647supp001.pdf In addition to the idHSS intervention, during this period, the population of Ifanadiana benefited from two other notable programmes that covered both the idHSS catchment area and the rest of the district (referred to as ‘control catchment’). The PAUSENS project, funded by the World Bank, provided a basic package of services free of charge in all 13 CSB2 through a voucher system for every woman attending the health centre for antenatal, delivery or postnatal care (first 6 weeks) and children under age 5 with any illness (see Appendix, Section S1).36 The project also included training, support for child vaccination in remote areas and some equipment to health centres. The Mikolo project, funded by USAID, provided support to a network of 150 CHWs in the fokontany further than 5 km from a health centre in eight communes in Ifanadiana, four of which were in the idHSS catchment and four in the control catchment. The project organised annual trainings and periodic supervision, provided some equipment, supplies and an initial stock of medicines to each CHW. For more details, see online supplemental table S1. The main difference between the idHSS catchment and the control catchment was the implementation of the idHSS intervention by the MoPH-PIVOT partnership. Patients or the public were not involved in the design, conduct, reporting or dissemination plans of our research. A longitudinal cohort study known as the Ifanadiana Health Outcomes and Prosperity longitudinal Evaluation (IHOPE), or IHOPE cohort, was initiated in 2014 to obtain demographic, health and socio-economic information from a representative sample of 1600 households in Ifanadiana district over time.32 Questionnaires were mostly adapted from the Demographic and Health Survey (DHS).37 The Madagascar National Institute of Statistics (INSTAT), which implements all major national health surveys in the country, was responsible for data collection, survey coordination, training and oversight. A two-stage sample was used, which stratified the district by the initial idHSS and control catchments. Eighty clusters, half from each stratum, were selected at random from enumeration areas mapped during the 2009 census, and households were then mapped within each cluster. Twenty households were selected at random from each cluster. The first wave of data collection was conducted between April and May of 2014. Individual face-to-face interviews were conducted with women aged 15 to 49 years and men aged 15 to 59 years (usual residents or visitors) in 1522 of the sampled households (95.1% response rate). The original 1600 households were revisited between August and September of 2016 and again between April and May of 2018; any missing or refused households were replaced with others from the same cluster using a predefined random replacement list. Overall, 1514 households were interviewed during the first follow-up survey in 2016 (94.6% response rate) and 1512 during the second in 2018 (94.5% response rate). All residents, including children, had weight and height measured (or length in the case of infants). Data collected in the questionnaires included, among others, household composition (size, genders and ages); indicators of socio-economic status (education, employment and household durable assets); illness in last 30 days for all household members and care seeking for illness; preventive behaviour (bed net ownership, access to water and sanitation); women’s reproductive history and care seeking behaviour for reproductive health; children’s health, development and care seeking for illness; adult, maternal and child mortality. French and Malagasy questionnaires used in the cohort, as well as data collection protocols, were standardised and validated for Madagascar during previous national surveys carried out by INSTAT. All adults (≥15 years) provided verbal consent for the in-person interview and anthropometric measurements. Parents or guardians provided consent for children ≤5 years of age. INSTAT provided survey data to the investigators with all individual identifiers removed and with geographical information at the cluster level. Spatial boundaries of each cluster were made available to the investigators and are stored separately; this information will not be published or shared publicly. Further details on data collection and survey design are available in.32 We obtained data for the period of January 2013 to December 2018 from the MoPH for 13 CSB2 in Ifanadiana district on the number of new individuals per month attending each health centre for outpatient consultations or maternal care. We excluded two CSBs that did not exist for this 6-year period (built in November 2016 and April 2018), as well as the six CSB1 in the district because these lack medical doctors and provide a more limited number of health services. Given that the idHSS intervention started in early 2014 (shortly after the baseline survey) and affected utilisation rates that year, we included health system utilisation data since 2013 in order to have a true baseline before the idHSS intervention. These data were available from the health centres’ monthly reports to the district (RMA), which are aggregated from the health centres’ registers every month by MoPH staff. As a component of the idHSS intervention, data quality was maintained through joint MoPH-PIVOT supervisions carried out every 3 months at a subset of health centres to compare RMA values with registry data (see38 for more details). Starting in May 2015, the MoPH changed the estimation and reporting of outpatient utilisation rates. Thus, all subsequent utilisation data were gathered directly from the registers to ensure consistent estimates throughout the 2013 to 2018 period. Information on total catchment population for each health centre was obtained from official MoPH records. Consistent with MoPH estimates, catchment population of children under 5 years of age, expected number of pregnant women and expected number of deliveries were set at 18%, 4.5% and 4% of the total catchment population, respectively. Although official population data are sometimes deemed inaccurate, we previously showed that estimating catchment populations using available data for our district from other recognised sources such as WorldPop39 did not change the results of per capita utilisation rates analyses.38 Under-five mortality at the population-level was estimated using the synthetic life-table method for DHS surveys.40 Under-five mortality was defined as the probability of death before age 60 months per 1000 children born alive. For each survey wave, we used information from the 5 years prior to the survey, which comprised a sample of 4063 children for 2014, 4037 children for 2016 and 3788 children for 2018. From these estimates, absolute and relative changes per year for each indicator and catchment area were estimated in univariate linear models, and these were used to estimate the difference over time between the two areas. In addition to these cross-sectional estimates, the difference in under-five mortality for the 2514 children followed-up over the 4 years was also assessed in each area. From this longitudinal analysis, incidence rates of death per person-year in children under 5 years of age were calculated using a Poisson regression. Coverage indicators (see list in table 2 and online supplemental table S3) were estimated using standard definitions for DHS surveys.40 Vaccination coverage was defined as the proportion of children aged 12 to 23 months who received all recommended vaccines (three doses of polio and DTP, one dose of BCG and measles). Access to treatment was estimated as the proportion of children under 5 years of age who were ill with fever, acute respiratory infection or diarrhoea in the 2 weeks prior to the survey and sought medical treatment (at a hospital, health centre or community health worker). In order to measure the effect of the idHSS intervention on maternal health service coverage, indicators were estimated for the last pregnancy during the 2 years preceding the survey. In addition, to track a summary indicator of maternal, newborn and child health (MNCH) coverage, we estimated co-coverage indices (three interventions or less; five interventions or more) for women and children under 5 years of age and a modified version of the composite coverage index (CCI)41 that included all standard maternal and child interventions except for family planning, which was not available in the 2014 survey. Model predictions of annual change in coverage associated with the idHSS intervention and in the rest of Ifanadiana district (control), 2014 to 2018 *p value <0.05; **p value <0.01; ***p value <0.001. †Information not available for 2014; trends are estimated for the 2016 to 2018 period. ARI, acute respiratory infection; idHSS, integrated district-level health system strengthening; MNCH, maternal, newborn and child health. Coverage and mortality indicators were calculated for 2014, 2016 and 2018 both in the idHSS catchment (which changed over time) and in the control catchment (the rest of the district). Yearly changes in coverage between the idHSS and control catchments were modelled from individual-level data using multivariate logistic mixed regressions that included a random intercept at the cluster level, using the following formula: Where Yij is the average coverage for yeari and clusterj; catchment reflects whether the household of the individual was part of the initial idHSS catchment to account for baseline differences in these two areas; exposure reflects the number of years of exposure of the cluster to the idHSS intervention to account for the expanding idHSS catchment; β1 is the yearly change in the control catchment, β2 is the baseline difference in utilisation between the initial intervention catchment and the control catchment and β3 is the yearly change associated with the idHSS intervention; bj and εj are the random intercept and error associated with each cluster, respectively. Results were reported as adjusted OR, and as predicted yearly change (using only the model’s fixed effects). The 95% CIs for yearly changes were estimated through parametric bootstrap (400 simulations per indicator). Data were entered into CSPro and all analyses were done using R statistical software, V.3.1.242 and R-package lme4 with the exception of population-level mortality rates and associated 95% CIs, which were calculated with SAS 9.3,43 and the observed under-five mortality incidence rate ratios, which were calculated with Stata, V.15.44 Changes of coverage in the initial idHSS catchment area (four communes) and in the control catchment were also estimated and compared using difference-in-difference analyses (online supplemental table S4) for consistency with analyses in our previous 2-year impact evaluation.31 Sampling weights that adjusted for unequal probability of selection due to stratification and non-response were calculated for household, women’s and men’s surveys. Estimates were obtained using survey commands available in R-package survey and applicable sampling weights.45 First, changes in the geographical distribution of coverage over time in Ifanadiana were assessed for each indicator. For this, average values for the 80 geographical clusters in the IHOPE cohort were estimated, each of which included 20 households and approximately 100 individuals. Then, given the spatial location of each cluster, a raster surface of the whole district was obtained to improve visualisation of results. This was done through inverse distance weighted interpolation on the empirical Bayes estimates of each cluster, using R-packages spdep and gstat. Second, trends in economic and geographical inequalities in the idHSS catchment were assessed for each coverage indicator. To estimate economic inequalities, a household wealth index was calculated through a principal components analysis of household assets following standard DHS methods.40 To estimate geographical inequalities, the Open Source Routing Machine (OSRM) engine was used to accurately estimate the shortest path distance between the villages in each cluster and the nearest health centre. For this, we had previously mapped the entire district of Ifanadiana on OpenStreetMap, resulting in over 23 000 km of footpaths and 5000 residential areas mapped (see46 for details). For each indicator, we estimated wealth-specific and geographic-specific averages (bottom two quantiles vs top three quantiles) as well as composite indicators of inequality, such as relative concentration index and slope index of inequality.47 The relative concentration index (RCI) is a measure of relative inequality based on the concentration curve, a plot of the cumulative distribution of each coverage indicator (y-axis) in the population ranked by wealth or geography (x-axis) and adjusted by survey weights.47 The RCI is defined as twice the area between the line of equality (45° diagonal line) and the indicator’s concentration curve, and was calculated using R-package decomp.48 The slope index of inequality (SII) is a measure of absolute inequality that represents the difference in coverage between the highest and the lowest values of the wealth or geography rank (normalised between 0 and 1). It was estimated at the individual level as the slope of the health outcome on the individual wealth or geography ranks in a logistic regression, adjusted by survey weights.47 49 Third, trends in self-reported barriers to seek care were estimated in the idHSS and control catchments. For this, individuals of all ages who reported being ill but not seeking care at a healthcare facility in the IHOPE cohort surveys were asked to provide the primary and secondary reason for why they did not seek care. This information was added to the surveys from 2016 onwards, but was not available for 2014. Reasons were classified as no barrier (‘not severe enough’ or no reason reported), knowledge barrier (‘did not think they could help me’ and ‘did not know that a treatment existed’), health system barrier (‘lack of confidence in health staff’ and ‘health staff often absent’), financial barrier (‘impossible to stop work’ and ‘too expensive’) and geographical barrier (‘too far away or hard to reach’). Percentages of each barrier reported were estimated out of all primary and secondary reasons. For each health centre, we calculated the annual average in per capita utilisation rates for maternal health services (antenatal care, first and fourth visit; deliveries, and postnatal care) and outpatient care for any illness (all patients and children under 5 years of age). We estimated annual changes during the 2013 to 2018 period for health centres in the idHSS and control catchments using mixed logistic regression models equivalent to the analyses of survey data described above (with a random intercept for each health centre). In order to build explicit information feedback loops between the impact evaluation and programme implementation, we developed a user-friendly online application to facilitate use of the IHOPE cohort results by local health staff. It consists of a website interface that builds on the same data sources (ie, IHOPE cohort database) and methods for the estimation of changes in coverage and inequalities as presented in this manuscript, making the visualisation of results flexible and easily accessible by programme managers and decision-makers (in French and English). We used the package Shiny50 for R statistical software. This application is developed and maintained by the PIVOT research team, and hosted at the PIVOT dashboard website (http://research.pivot-dashboard.org/).
N/A
