Evaluation of a maternal health care project in South West Shoa Zone, Ethiopia: Before-and-after comparison

Background: Despite recent achievements in health targets, Ethiopia still faces challenges in health service delivery. Between 2012 and 2015, a non-governmental organisation (NGO), Doctors with Africa CUAMM, implemented a multifaceted project aimed at improving access to maternal and child health services in three districts in Ethiopia. This paper evaluates the performance of this project, based on four maternal health indicators. Methods: A before-and-after study utilising data collected through cross-sectional surveys involving 999 women was conducted. The date of delivery was used to stratify the intervention period as follows: pre-intervention, early intervention, and late intervention. Changes during the intervention in the coverage of four antenatal care (ANC) visits, receipt of three basic components of ANC, skilled birth attendant (SBA) at delivery, and postnatal care (PNC) in seven days were assessed using logistic regression, adjusting for socio-demographic factors. Results: There was an increase in the coverage of receipt of all three ANC components and SBA at delivery between the pre-intervention period and the late intervention period. The percent of health centre deliveries increased from 7.3 % in the pre-intervention period to 35.6 % in the late intervention period. The odds of receiving all three components of ANC were twice higher in the late intervention period than in the pre-intervention period (OR 2.09; 95 % CI 1.12-3.89). The odds of SBA at delivery were five times higher in the late intervention period than in the pre-intervention period (OR 5.04; 95 % CI 2.53-10.06). There was no significant change in the coverage of four ANC visits and PNC after accounting for sociodemographic factors. Conclusions: This NGO implemented maternal health project in three districts in Ethiopia was associated with increased likelihood that a pregnant woman would receive three basic components of ANC and be assisted by a SBA at delivery. Increase in skilled birth attendance was driven by increased utilisation of health centres. More efforts are needed to bolster the coverage of ANC and PNC. The project was implemented in Wolisso, Goro and Wonchi districts of South West Shoa Zone, Oromia region in central Ethiopia. The districts are located about 115 km south-west of Addis Ababa, the capital of Ethiopia. The three districts had a combined population of about 398,000 inhabitants in 2014 and are served by one hospital (St. Luke Catholic Hospital), which also acts as a zonal referral hospital, 18 HCs and 89 health posts (HPs). The hospital is a private non-profit facility and hence had a system of user fees before the project began. In Ethiopia, maternity services are usually provided at hospitals and HCs. HCs, which are designed to serve a catchment population of 25,000 people, are expected to provide a full range of routine maternal health services plus emergency obstetric care services except blood transfusion and caesarean section, which can only be provided at hospital level [14]. HPs are run by salaried health extension workers (HEWs) who are mainly female community members with high school-level education and have been trained for one year to provide preventive, promotive and selective curative health services. HEWs increase the knowledge and skills of communities to deal with preventable diseases and to utilise health services provided at HCs and hospitals, and also provide care to women during pregnancy, childbirth and postnatal periods either in HPs or in households [14–16]. Thus, they spend about 75 % of their time conducting outreach activities and the rest at HPs. All the HCs and HPs in the study area are government owned and provide maternal health services free-of-charge as per the national policy. The project was embedded in the health system of the districts, and during its course, the following activities were conducted to improve maternal and neonatal health care: A detailed work plan guided the implementation of the project. Monitoring of the project was conducted jointly by CUAMM and local partners (zonal and district health authorities) through quarterly review meetings, quarterly activity and financial reports, planned field visits and supportive supervision. This study utilised before-and-after intervention design based on data collected through two cross-sectional surveys. The study population consisted of women of reproductive age who delivered within two years preceding each survey, in the study districts. Data were collected through household surveys conducted in February 2013 and March 2015. The surveys utilised similar methods and tools (questionnaires). The questionnaires were adapted from the UNICEF’s Multiple Cluster Indicator Survey questionnaires and JHPIEGO’s tools for monitoring birth preparedness and complication readiness [18], and were pretested and translated into Oromo language. During each survey, women who delivered within two years preceding each survey were asked questions related to care during pregnancy, delivery and after delivery of the youngest child. Data were also collected on household and socio-demographic characteristics, birth preparedness, knowledge of pregnancy related danger signs, perceptions towards maternal health care and perceived quality of care. The surveys utilised multistage sampling using a modified Expanded Program for Immunisation’s random walk method [19] to select study subjects. The first stage involved selection of villages and the second stage involved selection of eligible women in the selected village. Details of the sampling method are available elsewhere [12]. The first survey collected data from a sample of 500 women estimated assuming institutional delivery coverage of 20 %, an absolute precision of 0.05, and a Z score value of 1.96 for 95 % confidence interval and a design effect of 2. Due to limited resources, the second survey included a similar number of women. This evaluation was sufficiently powered (>95 %) to detect significant differences at 5 % alpha level between the pre-intervention period and the late intervention period for all the outcomes except for PNC as shown in the Additional file 1. Each survey had a reference period of preceding two years (Fig. 1). This implies that the reference period of the surveys was the entire duration of the project plus a period of 14 months before the start. Although the project began in April 2012, the first four months were spent on preparatory activities such as hiring of staffs and procurement of supplies, and so the actual intervention period began in August 2012. For the purpose of this evaluation, we have defined the intervention period (the exposure variable) based on the month and year that the woman delivered into three periods i.e. pre-intervention period (February 2011 to July 2012), early intervention period (August 2012 to December 2013) and late intervention period (January 2014 to March 2015). Timeline of the project and household surveys (not drawn to scale) We based this evaluation on four outcomes: 1) Attendance of at least four visits of ANC provided by a health professional or a health extension worker; 2) receipt of all three basic services during antenatal care: blood pressure measurement, blood sample taken, urine sample taken; 3) delivery assisted by a skilled birth attendant (SBA) i.e. a doctor, a nurse, a midwife, or a health officer; and 4) receipt of PNC within seven days of delivery by a health professional or a health extension worker. The surveys collected data on district, urban/rural residence, woman’s age; parity; education level; marital status; ethnicity; and religion, index child’s age in months, partner’s education, and distance to the nearest health facility with maternity services. Data were also collected on attitude towards maternal health care, perceived quality of maternal health care at nearest health facility, knowledge of pregnancy danger signs, and birth preparedness. These later four variables were considered to be intermediate outcomes. We derived wealth index through factor analysis of household assets, housing material, and access to water and sanitation services. We used the first of the factor scores to represent the wealth index [20]. We derived maternal health attitude score using factor analysis of eight Likert scale questions that explored perceptions of women towards birth preparedness; male involvement in maternal health; and barriers to institutional childbirth as described elsewhere [12, 18]. We analysed data in Stata version 12 using survey commands to account for the complex sampling design. We assessed the sociodemographic characteristics of women across the intervention periods using descriptive statistics and design based F tests. We cross tabulated the intermediate outcome variables namely: knowledge of pregnancy danger signs, attitude towards maternal health, perceived quality of care, attendance of any ANC and birth preparedness against the intervention periods and assessed linear trends across the periods. To assess the effect of the intervention on each outcome variable, we used logistic regression models to obtain odds ratios (ORs) and 95 % confidence intervals (CIs). The ORs were adjusted for woman’s age, place of residence, wealth index tertile, parity, partner’s education, woman’s education and religion. We used the pre-intervention period as the reference category in all analyses. We explored for linear effects by entering, in the models, the intervention period as a continuous variable.