Impact evaluation of a quality improvement intervention on maternal and child health outcomes in Northern Ghana: Early assessment of a national scale-up project

Objective. To evaluate the influence of the early phase of Project Fives Alive!, a national child survival improvement project, on key maternal and child health outcomes. Design. The evaluation used multivariable interrupted time series analyses to determine whether change categories tested were associated with improvements in the outcomes of interest. Participants. The evaluation used program and outcome data from interventions focused on health-care staff in 27 facilities. Setting. Northern Ghana. Intervention. The project uses a quality improvement (QI) approach whereby process failures are identified by health staff and process changes are tested in the health facilities and corresponding communities to address those failures. Main Outcome Measures. The maternal health outcomes were early antenatal care attendance and skilled delivery, and the child health outcomes were underweight infants attending child wellness clinics, facility-level neonatal mortality and facility-level infant mortality. Results. Postnatal care changes for the first 1-2 days of life (β = 0.10, P = 0.07) and the first 6-7 days of life (β = 0.10, P = 0.07) were associated with a higher rate of visits by underweight infants to child wellness clinics. There was an association between the early pregnancy identification change category with increased skilled delivery (β = 1.36 P = 0.07). In addition, a greater number of change categories tested was associated with increased skilled delivery (β = 0.05, P = 0.01). Conclusion. The QI approach of testing and implementing simple and low cost locally inspired changes has the potential to lead to improved health outcomes at scale both in Ghana and other low- and middle-income countries. © The Author 2013.

We include two outcomes focused on maternal health—early antenatal care (ANC) (percent of first-time ANC registrants who are in their first trimester of pregnancy) and skilled delivery (percent of total deliveries which are attended by a skilled birth attendant defined as a doctor, nurse or midwife). While increased access to skilled delivery has not been universally linked to improved maternal mortality [23], the promotion of skilled delivery is widely regarded as a key strategy for maternal health programs [24]. Three child health impact indicators were studied as key outcomes—underweight among infants (percent of infants attending child wellness clinics who are low weight for age), facility-level neonatal mortality defined as deaths <28 days of life (facility-level neonatal deaths/facility and community-level live births) and facility-level infant mortality defined as deaths <1 year of life (facility-level infant deaths/facility and community-level live births). These outcome indicators were obtained by Project Fives Alive! from facility health registers. Health-care workers report on a number of indicators directly into this registers. These indicators are included in routinely reported data sent by facilities to the GHS on a monthly basis. The key independent variable for the evaluation is the type of process change implemented. Change ideas fell into five categories: early pregnancy identification, the promotion of four ANC visits, encouraging skilled delivery and providing postnatal care (PNC) on Day 1–2 and PNC on Day 6–7 of life. Table 1 presents detailed information about the change categories. Description of the change categories and numbers of facilities testing each change Descriptive data on facility and program-level factors are presented in Table 2. The facility-level variables included affiliation (government or Catholic), type (hospital or health center) and number of staff. The program level factors included a dummy variable for the individual project officer assigned to work in specific facilities, number of site visits and profession of the QI team leader (midwife versus non-midwife). Facility catchment population and a measure of remoteness (distance from the facility to the district capital) were also included to capture facility-level heterogeneity. The program and facility-level information was collected from program records, census data and through discussions with project staff. Program and facility-level characteristics Data from non-intervention comparison facilities were not available. However, rather than compare pre- and post-intervention means we employ an interrupted time series approach whereby monthly facility-level data were included in a multivariable analysis. Time series analysis is used to detect whether an intervention (or change category tested) is associated with a change in an underlying trend for an outcome variable [25]. Data for the evaluation came from the period of April 2008 to December 2009, and the project did not reach full implementation until January 2009. It was, thus, possible to establish an underlying trend using 9 months of pre-intervention data. (January 2009 is the month after the activity period following the second learning session. The project team has indicated this is the time most QI teams were fully implementing change ideas.) In this analysis, each facility serves as its own control because the pre-change trend is compared with the post-change trend. The data constituted a time series of monthly cross sections, and the core equation to be estimated was as follows: In this specification Y is the outcome of interest, f and t denote facility and time period, respectively, and CT represents a change category tested. CTf takes on the value 1 for a time period after full implementation of a change and 0 before (the ‘interruption’) in facility f, and X is a vector of facility and program-level variables included in the model. β2 indicates the immediate impact and β3 indicates the longer term impact or trend. The test of statistical significance of β2 determines whether there is a one-time jump in the value of Y at full implementation of the change, while β3 determines whether there is a change in the slope of Y after full implementation of the change (the difference in slope from before the change was fully tested to after). If the coefficient of the change variable is positive then there is a one-time positive jump in the value of the outcome at the time the change was fully implemented (a difference in intercepts between the pre- and post-change lines). If the coefficient of the change variable is negative then there is a one-time negative jump. A negative coefficient for the interaction with time trend indicates that the post-change slope is flatter than the pre-change slope and suggests that the effect of the intervention on the trend is negative. A positive coefficient for the interaction with time trend indicates that the effect on the trend is positive. The actual post-change value of the outcome is given by the sum of the coefficients of the change variable (β2) and time trend (β3). In the regression models a non-linear trend is accommodated through a quadratic term in t multiplied by the change variable. Each change category was included in a separate multivariable time series regression model. Most change categories were expected to have a primary or direct influence on an outcome while a few would have a secondary or indirect influence. For example, the skilled delivery change category would be expected to have a primary influence on neonatal mortality but not a direct influence on the percent of underweight infants. There could however be some indirect or secondary influences. For example, by delivering her child in the health facility, a woman could become more aware of health services for children in that facility and bring an underweight child for care. However, because of the more distal nature of the secondary influences, only change categories expected to have a primary influence on an outcome were tested in the statistical models.

N/A

• Dima Health