Has Ebola delayed progress on access to routine care and financial protection in Sierra Leone? Evidence from a difference-in-differences analysis with propensity score weighting

Introduction: Covid-19 has highlighted the need to understand the long-term impact of epidemics on health systems. There is extensive evidence that the Ebola epidemic of 2014-16 dramatically reduced coverage of key reproductive, maternal, newborn, child and adolescent health (RMNCAH) indicators during the period of acute crisis in Sierra Leone. However, less is known about the longer lasting effects, and whether patients continue to be deterred from seeking care either through fear or cost some years after the end of the epidemic Methods: We analysed nationally representative household surveys from before (2011) and after (2018) the Ebola epidemic to estimate the coverage of 11 indicators of access to RMNCAH, and affordability of care. We used a differences-in-differences analysis, exploiting the variation in epidemic intensity across chiefdoms, to identify the effect of epidemic intensity on access and affordability outcomes, with propensity score weighting to adjust for differences in underlying characteristics between chiefdoms. Results: 13537 households were included across both datasets. Epidemic intensity was associated with a significant stalling in progress (−12.2 percentage points, 95% CI: 23.2 to −1.3, p = 0.029) in the proportion of births attended by a skilled provider. Epidemic intensity did not have a significant impact on any other indicator. Conclusion: While there is evidence that chiefdoms which experienced worse Ebola outbreaks had poorer coverage of attendance of skilled providers at birth than would have otherwise been expected, more broadly the intensity of the epidemic did not impact on most indicators. This suggests the measures to restore both staffing and trust were effective in supporting the health system to recover from Ebola. We examined the impact of the 2014–2016 Ebola epidemic with a differences-in-differences analysis, in which we compared changes in care seeking and health expenditure outcomes from households surveyed in 2011 (before the start of the epidemic) and 2018 (up to 33 months after the last case of Ebola) in chiefdoms where few or no Ebola cases were reported and in chiefdoms with larger outbreaks. Sierra Leone’s 149 chiefdoms and two districts (referred to hereafter as chiefdoms) of Western Area (the location of Freetown) have been classified into seven patterns according to the size and length of outbreak experienced during the 2014–16 Ebola epidemic (Fang et al., 2016), using a weighted-average linkage-clustering method (Hamilton, 2009). To produce a binary measure of epidemic intensity, we categorised the 151 chiefdoms into two groups: no/mild epidemic (those with no cases, sporadic cases, or a single small-scale outbreak in a short period) and moderate/severe epidemic (with multiple small-scale outbreaks, a continuous low-level epidemic over a long period, or larger or more prolonged outbreaks). 40 chiefdoms were classified into the moderate/severe epidemic (or exposed) group and 111 into the no/mild epidemic (or unexposed) group. Further details of classification are given in the appendix. Data from the Sierra Leone Integrated Household Survey (SLIHS) in 2011 (Statistics Sierra Leone, 2011) and 2018 (Statistics Sierra Leone, 2018) were used to measure study outcomes and covariates before and after the Ebola epidemic. The SLIHS are cross-sectional surveys of a representative national sample of households in Sierra Leone, and were conducted in every chiefdom of the country to measure living standards and wellbeing. The sample was selected using a two-stage cluster design, sampled by enumeration areas (EAs) at the first level and households at the second level. In both years, 684 EAs were selected with probability proportional to size selection, stratified by rural or urban location and district, and ten households were randomly chosen to be surveyed in each selected EA, with a target sample size of 6840 households in both surveys. Fieldwork was conducted nationally in January–December 2011 and January–December 2018. The response rate for the 2011 SLIHS was 98.4%, and for 2018 it was 100%. Response rates by chiefdom were unavailable. Outcomes to measure use of services were chosen based on coverage indicators given in Sierra Leone’s 2017–2021 RMNCAH strategy (Ministry of Health and Sanitation, 2017), and which could be estimated from both 2011 and 2018 surveys. Cost of care outcomes were based on total healthcare expenditure (comprising of expenditure on outpatient, inpatient and antenatal care). To measure the affordability of healthcare, two outcomes used to monitor universal health coverage were chosen: catastrophic spending on health and impoverishing spending on health (WHO, 2017). Healthcare expenditures were winsorized at 99.9%. Outcomes and their measurement as implemented in this analysis are given in Table 1 . Outcome definitions. Individual and household level covariates from the SLIHS surveys were used in the analysis. Household level covariates were location type (rural or urban), sex of the household head (male or female), education level of the household head (none, primary secondary, post-secondary technical, or college/university), number of members of the household (continuous), and household consumption expenditure per adult equivalent, regionally adjusted and inflated to 2018 prices (continuous). Individual level covariates for women aged 15–49 were age (continuous), parity (continuous), religion (Muslim or other) and education (none, primary, secondary or post-secondary). Individual level covariates for children under-5 were age (continuous), maternal age (continuous), maternal parity (continuous), religion (Muslim or other) and maternal education (none, primary, secondary or post-secondary). Difference-in-differences analysis with propensity score weighting (Stuart et al., 2014) was used to estimate the effects of the Ebola epidemic, with households in the 40 chiefdoms experiencing a moderate/severe epidemic designated as the exposed group, and those living in the remaining 111 chiefdoms with mild or no epidemic serving as the comparison group. Significant differences at baseline and endline were observed in several household characteristics between exposed and comparison group households (Table 2 ). Propensity score methods were therefore used within the difference-in-differences analysis, to mitigate the concern that the groups may differ in ways that affect their trends over time and therefore violate the ‘parallel trends’ assumption. This method is tailored to repeated cross-sectional surveys and is described further below. Pre-epidemic trends for two outcomes, the proportion of births at a health facility and the proportion of births attended by a skilled provider, were examined for the five years before the baseline survey (January 2006–December 2010). No evidence of non-parallel trends was found, and further details are given in the appendix. We used a bootstrapping method to carry out post-facto calculations of the minimum detectable effect for each outcome, further details and results of which are given in the appendix. Observations with missing outcomes or covariates were excluded from all analyses, with no attempt to impute missing data. The extent of missingness is described in the appendix. Sample characteristics before and after propensity score weighting in 2011 (pre-epidemic) and 2018 (post-epidemic). Characteristics are described for the sample of under-2s included for analysis of the skilled provider at birth outcome. For characteristics of samples for other outcomes please see appendix. The population was divided into four groups by year (2011 or 2018) and exposure (no/mild epidemic or moderate/severe epidemic). A multinomial logistic regression model predicting the probability of being in each group as a function of the study covariates was run separately for each outcome, to account for the different populations included in the estimation of each outcome (household, woman of reproductive age, pregnant woman, child under five years). Each individual or household then has a propensity score, the probability of being in a given year and group, which is used to create weights such that each group is balanced in terms of its covariates. The propensity score weights were multiplied by the survey weights to account for the study sampling strategy. A multivariate linear (OLS) difference-in-differences regression model, with propensity score weighting and standard errors taking into account clustering by chiefdom, was estimated for each outcome as follows: where i indicates the individual or household, c is the chiefdom and t is the year of survey, 2011 or 2018. The variables Yict are each of the outcomes reported for the individual or household i in year t in chiefdom c. The dummy EIct indicates the epidemic intensity in chiefdom c at year t; it is equal to 1 in exposed chiefdoms in 2018 and 0 otherwise. Xit are the study covariates. Fixed effects for year and chiefdom are indicated by λt and μc respectively, and clustered standard errors by εict. A difference-in-differences model with a binary interaction term for wealth (above and below median per adult equivalent consumption, measured at the household level) and education (no vs some education of the household head for household level outcomes, no vs some maternal education for outcomes in children 0–5, and no vs some education for outcomes in women aged 15–49) was used to test for differential effects of the epidemic intensity in these subgroups. If the interaction term was significant at the p < 0.1 level, the overall result was presented alongside the result for each subgroup. Ethics approval was not sought for this study as it uses only anonymised data entirely in the public domain.