Measuring health system resilience in a highly fragile nation during protracted conflict: South Sudan 2011-15

Health systems resilience (HSR) is defined as the ability of a health system to continue providing normal services in response to a crisis, making it a critical concept for analysis of health systems in fragile and conflict-affected settings (FCAS). However, no consensus for this definition exists and even less about how to measure HSR. We examine three current HSR definitions (maintaining function, improving function and achieving health system targets) using real-time data from South Sudan to develop a data-driven understanding of resilience. We used 14 maternal, newborn and child health (MNCH) coverage indicators from household surveys in South Sudan collected at independence (2011) and following 2 years of protracted conflict (2015), to construct a resilience index (RI) for 9 of the former 10 states and nationally. We also assessed health system stress using conflict-related indicators and developed a stress index. We cross tabulated the two indices to assess the relationship of resilience and stress. For maintaining function for 80% of MNCH indicators, seven state health systems were resilient, compared with improving function for 50% of the indicators (two states were resilient). Achieving the health system national target of 50% coverage in half of the MNCH indicators displayed no resilience. MNCH coverage levels were low, with state averages ranging between 15% and 44%. Central Equatoria State displayed high resilience and high system stress. Lakes and Northern Bahr el Ghazal displayed high resilience and low stress. Jonglei and Upper Nile States had low resilience and high stress. This study is the first to investigate HSR definitions using a resilience metric and to simultaneously measure health system stress in FCAS. Improving function is the HSR definition detecting the greatest variation in the RI. HSR and health system stress are not consistently negatively associated. HSR is highly complex warranting more in-depth analyses in FCAS. The 2018 Fragile States Index ranked South Sudan as the world’s most fragile country (Fund for Peace, 2018). After emerging from Africa’s longest civil war (1956–2005), the Republic of South Sudan attained independence in 2011, and shortly thereafter, in December 2013, it experienced more armed conflict. Conflict resolution has remained ineffective for many reasons such as political patronage, ethnic domination, elite power struggles and an international emphasis on state-building which supersedes building social cohesion and integration of ethnic and interest groups (Gerenge, 2016; Kane et al., 2016). Currently, at least half of the population in South Sudan lives below the World Bank’s poverty line and nearly three-quarters (73.5%) lack formal education. South Sudan has one of the world’s highest CMR (104 per 1000 live births) and maternal mortality ratios (730–789 maternal deaths/100 000 live births; Valadez et al., 2015). These conditions are aggravated by nearly 1.97 million internally displaced people and 2.2 million refugees (UNHCR, 2018). The Ministry of Health (MOH) of South Sudan established a national monitoring and evaluation system using household surveys to track the progress of health indicators. This survey measured coverage of MNCH services in each of the country’s former 10 states and counties. We used the data from these national surveys to investigate the HSR definitions. We also obtained information on conflict events routinely collected by the United Nations Office of Coordination of Humanitarian Affairs (UNOCHA). These latter data we used to measure health system stress. With both sources of data, we examined for the first time, HSR and its relationship with conflict-related health system stress. We did this to understand the utility of the HSR when evaluating progress of the health system of South Sudan which is a topic of interest to the MOH and bilateral and international donors. The MOH implemented two national cross-sectional household surveys using stratified random sampling during 2011 and 2015 in which the sampling domains were the 10 states; their counties (the administrative unit of the states) were the strata. This effort was undertaken to measure numerous MNCH indicators. Although details of the survey including its participants, sampling protocols and results can be found elsewhere (Valadez et al., 2015; Republic of South Sudan, 2018), we briefly summarize them here. The MOH used two-stage sampling in each county. Firstly, villages were sampled in each state county with probability proportional to size. In each village, trained data collectors used segmentation sampling (Turner et al., 1996; Davis and Valadez, 2014) to randomly select households for interview. One person in the household was randomly selected using a random number table when more than one was eligible. Study participants included women of reproductive age (15–49 years), and mothers of children 0–11, 12–23 and 6–59 months, and those with children 0–59 months with diarrhoea, suspected pneumonia or malaria in the last 2 weeks. Sampling continued in each village until one person in each cohort was selected. Each sampling unit had its own independent sample, and the total sample collected in 2011 was (1475 × 7 cohorts) 10 325, and 9443 (1349 × 7 cohorts) in 2015. Data collectors were State MOH health workers associated with the monitoring and evaluation units, who were trained and supervised by technical advisors from the Liverpool School of Tropical Medicine to use the study protocols and pretested standardized questionnaires. Questions were asked in the local language or in Arabic. County level data were weighted by their population sizes and aggregated to produce state and national level coverage estimates with 95% confidence intervals. For this study, we used state and national weighted coverage estimates for 14 MNCH indicators to measure resilience outcome (Table 2), calculated using Stata-v14 (Statistical Software, College Station, TX; StataCorp LP, 2011), Excel-v2013 and R-v3.2.3. MNCH coverage indicators in South Sudan MNCH, maternal, new-born and child health; ANC, antenatal care; IPT2, intermittent prevention therapy; DPT3, diphtheria-pertussis-tetanus; BCG, Bacillus Calmette–Guerin; OPV3, oral polio vaccine; LLIN, long lasting insecticide-treated bednet; ITN, insecticide-treated bednet; U5, under-five; ORS, oral rehydration solution. We obtained the conflict dataset from UNOCHA in South Sudan who captured states-level conflict data from media and intelligence reports. We used three conflict indicators measured during 2011–15. We used total reported conflict incidents to measure exposure to conflict. We defined conflict incidents as any conflict event involving military forces, police forces, rebel forces, ethnic militia or civilian protests and including activities such as bombing, air attacks, raids, shootings and cattle raiding. The total reported conflict-related fatalities was a proxy measure of the severity of conflict which affects access to healthcare due to limited movements and reduces availability of healthcare due to destruction of health facilities. We used the total number of internally displaced persons (IDPs) arriving into the state (these data are for 2013–15 as they were not available for 2011) to measure the burden of care in the host state health system either due to sharing health resources with disbursed IDPs or by transferring health resources, such as healthcare providers, to IDP camps. HSR concerns the systemic response to crisis events; for this reason, we tested for difference in coverage with several MNCH interventions and services during 2011 and 2015 using a two-tailed two sample test for binomial proportions with a normal approximation, including a continuity correction to account for the binomial distribution (Rosner, 2015, pp. 373–386). However, for cases where the expected cell frequencies were less than five the two-tailed test would violate the assumptions for normal approximation; we, therefore, used a Yates-corrected chi-square test to prevent overestimation of P-values for small data. We tested for differences at P ≤ 0.05. We used thematic analysis to synthesize the HSR literature to arrive at three definitions of resilience: maintaining function, improving function and achieving the health system’s goal (Table 3). We queried each of the definitions with sensitivity analyses. Firstly, we defined the dominant definition of maintaining function as at least 80% of the 14 MNCH services maintained or improved their indicator values during the 2011–15 period. In sensitivity analysis, we also tested this definition for 100% and 50% of the indicators improving. Secondly, we set the definition of improving function at 50% of the MNCH services improved and also tested it for 80%, 40% and 30% of the indicators. Lastly, for the definition of achieving a health system coverage target, we defined resilience as at least half of the MNCH indicators achieving a 50% coverage target by 2015 since this was the coverage target established by the MOH in both 2011 and 2015 (Valadez et al., 2015). We also tested coverage targets of 40% and 30% in sensitivity analyses. We used significance tests for maintaining and improving function and spreadsheets to depict achievement of the health system target. Our analysis included data from only nine states as Unity State was under rebel control in 2015 preventing data collection. Resilience definitions based on resilience outcome HSR, health system resilience; MNCH, maternal, newborn and child health. To compare HSR and the amount of stress placed on the health system, we constructed a resilience index (RI) and health system stress index (SI) by adapting Briguglio’s formula for calculating economic vulnerability (Briguglio, 1995). When testing the three resilience definitions, we coded resilience as a binary yes/no outcome. However, to build the RI, we treated resilience as a continuum ranging from high to low. To generate RI, we first summed the per cent coverage difference between 2011 and 2015 for all indicators at state and national levels. This produced the total percentage difference (Total D%) which we used in the following formula to calculate a RI for each state and the national health system. Note: per system refers to a specific state (e.g. Central Equatoria) or national (South Sudan) health system being compared with the least performing state health system (min Total D% across systems) and the best performing state health system (max Total D% across systems) . The index ranged between one (most resilient) and zero (least resilient). We also conducted sub-group analyses for the RI, testing for state and national performance for maternal and child indicators separately. For health system stress, we first mapped each state’s total conflict fatalities (<2000 fatalities vs ≥2000 fatalities) and total number of IDP arrivals (<200 000 vs ≥200 000) to visualize the distribution of stress. Because stress variables (conflict incidents, fatalities and numbers of IDPs) were in different units, we developed a SI for each variable individually. For each of the three variables, we first calculated the annual number of conflict incidents, fatalities and number of IDPs per state and at the national level. We then used the annual number for each variable, e.g. IDPs, to generate a SI for IDPs per state and nationally using the following formula: Note: per system refers to a specific state (e.g. Central Equatoria) or national (South Sudan) health system being compared with the state health system with the least amount of stress value such as least annual number of IDPs (min amount of stress value across systems) and the state health system with the highest amount of stress value e.g. highest annual number of IDPs (max amount of stress value across systems) . We repeated this formula for the annual number of conflict incidents and conflict fatalities and generated three health system stress indices for each state. We then took the arithmetic mean of the three SI to generate an overall SI for each state and the national health system, with values ranging between one (highest stress—most affected by the three variables combined) and zero (least stress). We weighed all three variables equally to avoid overemphasizing-related variables (conflict incidents increase likelihoods of fatalities and IDPs; Brooks et al., 2005), and avoid predicting an outcome by weighting one variable greater than the other which is contrary to resilience theory’s assumptions of unpredictable outcomes (Holling, 1973). States receiving more IDPs or with more fatalities experience greater pressure for services either due to an increased population in need or by having fewer functional services and facilities, respectively. Finally, we cross tabulated RI and SI using Briguglio’s vulnerability and resilience framework. We also conducted sub-group analyses of maternal and child services to identify indicator domains displaying differential levels of resilience. The household surveys were reviewed and approved by the LSTM Research Ethics Committee and the Ethics Review Committee of the Ministry of Health of South Sudan. UNOCHA’s conflict data are secondary anonymized datasets, this study received ethics exemption from LSTM Research Ethics Committee.