Impact of results-based financing on effective obstetric care coverage: Evidence from a quasi-experimental study in Malawi

Background: Results-based financing (RBF) describes health system approaches addressing both service quality and use. Effective coverage is a metric measuring progress towards universal health coverage (UHC). Although considered a means towards achieving UHC in settings with weak health financing modalities, the impact of RBF on effective coverage has not been explicitly studied. Methods: Malawi introduced the Results-Based Financing For Maternal and Neonatal Health (RBF4MNH) Initiative in 2013 to improve quality of maternal and newborn health services at emergency obstetric care facilities. Using a quasi-experimental design, we examined the impact of the RBF4MNH on both crude and effective coverage of pregnant women across four districts during the two years following implementation. Results: There was no effect on crude coverage. With a larger proportion of women in intervention areas receiving more effective care over time, the overall net increase in effective coverage was 7.1%-points (p = 0.07). The strongest impact on effective coverage (31.0%-point increase, p = 0.02) occurred only at lower cut-off level (60% of maximum score) of obstetric care effectiveness. Design-specific and wider health system factors likely limited the program’s potential to produce stronger effects. Conclusion: The RBF4MNH improved effective coverage of pregnant women and seems to be a promising reform approach towards reaching UHC. Given the short study period, the full potential of the current RBF scheme has likely not yet been reached. At the time the intervention was designed, mortality of mothers in Malawi was high (maternal mortality ratio in 2013: 636 deaths/100,000 live births for Malawi vs. 210 deaths/100,000 live births globally; newborn mortality rate in 2013: 25.9 deaths/1000 live births in Malawi vs. 20.1 deaths/1000 live births globally) [18, 19]. Obstetric care is offered free of charge through public and contracted not-for-profit health facilities [20]. Yet, 75% of all pregnant women with obstetric complications do not actually receive satisfactory emergency obstetric care (EmOC) [21]. Salaries of publicly employed health workers stem directly from central government budgets, while publicly funded primary care facilities (i.e. health centres and district-level hospitals) receive a mix of financial allocations from both central and local government budgets. Malawi’s health system is further challenged by chronic health worker shortages and system-wide stock-outs of essential drugs and supplies [22]. The Malawi Ministry of Health (MoH) introduced the Results-Based Financing For Maternal And Neonatal Health (RBF4MNH) Initiative to four districts (Balaka, Dedza, Mchinji, Ntcheu) in April 2013 to enhance obstetric care provision at facilities designated to eventually fully function as EmOC centres. Together, these four districts consist of a total of 33 designated EmOC facilities serving a catchment population with an expected birth rate of about 111,450 deliveries per year [21]. District selection was driven by MoH decisions to avoid overlap with other existing or upcoming maternal health or health financing programs in the country. The RBF4MNH’s main objective is to improve both quality and access to facility-based obstetric care for women and newborns during birth and up to 48 h after delivery [23] through a combination of supply- (performance-based payments to facilities and district health management teams (DHMT)) and demand-side mechanisms (conditional cash transfers (CCT) to pregnant women within catchment areas). Implementation occurred in two phases: an early pilot phase (April 2013 to October 2014) and a later expansion phase (November 2014 onwards). During the early phase, only 18 out of the 33 EmOC facilities (four hospitals, 14 health centres) were empanelled and later expanded by an additional 10 facilities (two community hospitals, eight health centres) with on-going plans for a nation-wide scale-up. Empaneled facilities were selected based on the presence of at least four skilled birth attendants, catchment population size, and number of institutional deliveries. Upon verification, facilities receive payments for achieving a set of performance targets related to quality of clinical care as well as general service improvement performance indicators [see Additional file 1]. Of these rewards, 40% are earmarked for further investments improving structural working conditions, 60% for individual bonus payments to health workers and auxiliary staff (about 15–25% of staff’s baseline salary). DHMT receive payments related to performance indicators on effective management and support of the entire district [see Additional file 1), also divided into an investment and bonus portion. CCT portions to women were calculated to defray upfront costs related to childbirth (i.e. transportation, basic childbirth items, stay in maternal waiting home) and average opportunity costs anticipated by an average poor patient or her family in Malawi during a 48-h postpartum facility stay. The maximum payment per woman is about seven Euros. To ensure minimum standards in EmOC delivery, all RBF facilities received some basic infrastructure and/or equipment support (e.g. delivery beds, essential examination, EmOC and sterilization material, renovation or reconstruction of labour rooms, postpartum wards, maternal waiting homes, electricity and water supply) prior to intervention launch. The early implementation phase consisted of three six-month reward cycles and served as an opportunity for implementers to further fine-tune the intervention in response to unforeseen challenges. Feedback and experiences gained during these initial cycles resulted in some programmatic adjustments prior to its expansion in October 2014, including: As part of a larger impact evaluation assessing the effect of the RBF4MNH on MNH service utilization and quality [24], this study followed a non-randomized controlled pre-post-test design including 32 of the 33 facilities targeted by the RBF4MNH intervention (we excluded one facility since it could not be identified as EmOC given lack of a delivery ward at baseline). The 18 facilities empanelled during the early phase served as ‘interventions’. Of the remaining 14 control facilities, five turned into RBF facilities at the start of the expansion phase and were treated as ‘switchers’. Data were collected at three time points: baseline (April–May 2013, before official program launch), midterm (June–July 2014, approximately one year after program launch) and endline (June–July 2015, approximately two years after program launch). We used three different samples: a facility sample, a service user sample, and an obstetric case sample. During each survey round, we collected three different sets of data: a facility inventory, direct case observations, and a household-level survey. The facility sample included 32 facilities. During baseline and midterm this sample consisted of 18 intervention and 14 control facilities, and 25 interventions and nine controls during endline (after expansion in October 2014). Selection of interventions followed the RBF empanelment criteria. Controls included all EmOC facilities initially not included by the RBF. Facility inventories consisted of a structured checklist collecting information on availability of operational equipment, medicines, and supplies essential to routine and basic emergency obstetric care. The service user sample included 5509 randomly selected women living in catchment areas of sampled facilities who completed a pregnancy within the twelve months preceding each survey date. Women reporting pregnancies that resulted in foetal loss or demise before the third trimester were excluded. A two-stage cluster approach was used to sample eligible women. Structured household-level questionnaires collected information on women’s demographic characteristics, health-seeking behaviour during pregnancy, obstetric care service use at birth, birth outcome, and household-specific socio-economic. The obstetric case sample included a total of 383 labouring women who presented to the sampled facilities during the three data collection rounds. Convenience sampling was used to include only cases without obstetric complications to ensure comparability between cases. Case observations consisted of a structured checklist collecting information on birth attendants’ adherence to clinical standard guidelines during routine case management. Content was based on performance standards developed for the Malawi Performance Quality Improvement program [25]. Observations started once a labouring woman was admitted to the maternity unit and lasted up to two hours after delivery. We used effective coverage of pregnant women with facility-based basic obstetric care services as main outcome variables and defined it according to the literature [11, 12, 26] as ECFBD preg = UFBD preg ∣ NFBD preg ∗ QFBD preg representing effective coverage (EC) of pregnant women (preg) using facility-based delivery services (FBD) at a designated EmOC facility (U) providing a given level of quality (Q). Here, UFBD preg ∣ NFBD preg denotes FBD service use conditional on true need for basic EmOC (i.e. crude or contact coverage) and defined it as any woman carrying a pregnancy beyond the second trimester [27, 28]. We further defined service use UFBD preg as any pregnant women who used services at any of the designated EmOC facilities included in our sample (vs. non-facility-based care or facility-based care at a non-EmOC facilities). To determine the expected quality of care received by a woman using FBD services at a given facility, we created a composite score using a content-of-care approach measuring the extent to which obstetric care was provided in adherence with pertinent standards of care based on a combination of input and process indicators taken from the inventories and case observations. In developing the composite score, we followed a standard approach including weighting, aggregation and uncertainty analysis [29]. A detailed outline of this approach and the underlying indicator mapping is provided in the additional files [see Additional file 1 and Additional file 2]. The resulting composite score ranged from 0 (not meeting any of the measured obstetric care standards) to 1 (meeting all standards). In the entire sample, none of the studied facilities actually attained a score of 1 (measured scores were nearly normally distributed with a range from 0.22 to 0.86, median of 0.56 and mean of 0.55), dichotomous categorization of facilities into ‘effective’ (i.e. a score of 1) and ‘less than effective’ (i.e. a score less than 1) was not practical to our analysis. In addition to only measuring effective coverage as the percentage of service use adjusted by the respective quality score, we further created additional binary variables based on different cut-off values within the upper range of the measured scores to assess facilities’ relative achievements towards these sub-levels. These cut-offs were set at scores of 0.5, 0.6, 0.7, and 0.8 representing 50%, 60%, 70%, and 80% of full obstetric care effectiveness. Quality scores were then assigned to each sampled woman based on reported facility use during the previous year, assuming short-term changes in service quality or effectiveness at a given facility to be minimal. Due to missing data for some facilities, we were not able to determine a quality score for each facility during baseline and midterm and consequently could not assign quality information to 141 women, reducing the actual sample available for analysis to 5368 women. We use descriptive statistics and two-sample t-test to compare differences in key characteristics between interventions and controls for each sample. We used frequencies illustrating the distribution of facilities and users by quality score categories over time. To estimated the RBF4MNH impact on crude and effective coverage we used linear regression in a difference-in-differences comparison [30]: with Yi representing the outcome (crude or effective coverage), t the time point (t1 = midterm, t2 = endline), T the treatment group, and T*t the interaction between treatment and time point (T1*t1 interaction at midterm, T*t2 at endline). Coefficients β3 and β5 represent the effect estimates at midterm and endline, respectively. Models were further adjusted for potential confounders (denoted by βkXki): household characteristics (district location, distance to nearest EmOC, socioeconomic status) when modelling effects on both crude and effective coverage; additional facility characteristics (type, ownership) when modelling effects on effective coverage. Household socio-economic status was measured as a relative wealth index based on assets and living conditions using principal component analysis and described in detail elsewhere [31]. We also adjusted for clustering at catchment area level and for the late phase expansion with five initial control facilities switching treatment arms. Given the relatively small number of catchment area clusters we used bootstrapping to improve the accuracy of standard errors for our effect estimates. Given the limitations of our clustered study design, we were only able to detect effect sizes of 0.25 or larger at a significance level of 5%. STATA version 14.1 was used for all statistical analyses.