Evaluation of the effectiveness of a quality improvement intervention to support integration of maternal, child and HIV care in primary health care facilities in South Africa

Background: Despite policies and guidelines recommending integration of health services in South Africa, provision of maternal and child health services remains fragmented. This study evaluated a rapid, scaleable, quality improvement (QI) intervention to improve integration of maternal and child health and HIV services at a primary health level, in KwaZulu-Natal, South Africa. Methods: A three-month intervention comprised of six QI mentoring visits, learning sessions with clinic staff to share learnings, and a self-administered checklist aimed to assist health workers monitor and implement an integrated package of health services for mothers and children. The study evaluated 27 clinics in four sub-districts using a stepped-wedge design. Each sub-district received the intervention sequentially in a randomly selected order. Five waves of data collection were conducted in all participating clinics between December 2016-February 2017. A multi-level, mixed effects logistic regression was used to account for random cluster fixed time and group effects using Stata V13.1. Results: Improvements in some growth monitoring indicators were achieved in intervention clinics compared to control clinics, including measuring the length of the baby (77% vs 63%; p = 0.001) and health workers asking mothers about the child’s feeding (74% vs 67%; p = 0.003), but the proportion of mothers who received feeding advice remained unchanged (38% vs 35%; p = 0.48). Significantly more mothers in the intervention group were asked about their baby’s health (44% vs 36%; p = 0.001), and completeness of record keeping improved (40% vs 26%; I = < 0.0001). Discussions with the mother about some maternal health services improved: Significantly more mothers in the intervention group were asked about HIV (26.5% vs 19.5%; p = 0.009) and family planning (33.5% vs 19.5%; p < 0.001), but this did not result in additional services being provided to mothers at the clinic visit. Conclusion: This robust evaluation shows significant improvements in coverage of some services, but the QI intervention was unable to achieve the substantial changes required to provide a comprehensive package of services to all mothers and children. We suggest the QI process be adapted to complex under-resourced health systems, building on the strengths of this approach, to provide workable health systems strengthening solutions for scalable implementation. Trial registration: ClinicalTrials.gov NCT04278612. Date of Registration: February 19, 2020. Retrospectively registered. The study was undertaken in one district in KZN comprising five sub-districts, with a population of approximately 669,000 and with over 147,000 households in the district. The sub-districts vary in size from the largest with a population of 237,500, to the smallest with 83,000 people [19]. Unemployment is high in this district, ranging from 34 to 57% in different sub-districts. More than half of the population in the district lives on less than US$20 per month [19]. The average proportion of households with access to potable water is 77% and access to electricity is 70% [19]. Health care services are provided by 37 primary health clinics, one community health centre, two district hospitals and one regional hospital [20]. The district has a high tuberculosis incidence of 533/100,000 in 2015/16 [15] and an antenatal HIV prevalence of 36.3% in 2015 [21]. Immunisation coverage for children under 1 year of age was 85% for 2015/16, and there was a high in-patient case fatality rates from severe acute malnutrition among children aged under 5 years [20]. At the time of the study, nutrition and growth monitoring for children aged under 5-years was a key priority of the SA National Department of Health. The QI intervention was structured as a BTS collaborative, a peer-to-peer learning model designed to improve system performance through structured data-driven improvement activities tied to a knowledge-sharing network [9]. This was implemented in 27 clinics in four sub-districts in one district in KZN over a one-year period. In PHC clinics, care is provided by a team of health workers including an operational (clinic) manager, specialist PHC nurses, registered nurses, enrolled nurses, nutritional advisors and lay counsellors. The number of nurses deployed in a clinic is determined by the size of the catchment population. There were between five and 10 clinics per sub-district (Table 1) and the intervention was conducted at the same time in all clinics in each sub-district. All participating clinics provided the same package of health services in compliance with Department of Health policies, which includes all services for mothers and children set out in the integrated package of care (Fig. 1). The fifth sub-district was used to pilot the methodology and was not included in the evaluation. The QI intervention aimed to support provision of an integrated service for mothers and children attending immunization services. We will refer to this integrated service as a ‘well mother-baby service’. Interviews conducted during each wave * Number of interveiws post QI intervention Comprehensive Package of Maternal, Child and HIV Health care. A description of integrated health care services health workers were expected to provide to mothers and babies at each visit to the clinic The aim of the intervention was to ensure that all mothers and children attending the clinic received a comprehensive package of services at every visit. Integration was defined as mother-baby pairs receiving all required health services contained in a defined package of services at every visit. Services included in the integrated package are shown in Fig. ​Fig.1.1. Growth monitoring comprises a series of activities conducted sequentially, all of which need to be completed for growth monitoring to be effective (Fig. ​(Fig.11). Implementation of the intervention in participating clinics was facilitated by a quality mentor (QM) employed by the project, who was a PHC nurse trained in QI techniques and methodology. The intervention comprised of a series of five QI mentoring visits per clinic conducted every 2 weeks in a single sub-district over a three-month period. Before the intervention commenced in the sub-district, staff who provided maternal and child health services from all clinics attended a learning session to introduce the project, and another learning session was conducted on completion of the five mentoring visits. After completion of the three-month intervention, a final mentoring visit (sixth visit) was conducted with the local clinic supervisor to hand over QI activities, which served as a close out for the intervention. Data on coverage of the key integrated package of services was monitored over the three-month period of intervention in each clinic using a self- administered checklist. A self-administered checklist was used in each clinic over the three-month period of implementation, this served a dual purpose of prompting health workers to implement the package of services as well as monitoring progress of implementation. The first mentoring visit was conducted as a short workshop attended by all members of the clinic team who provided services to mothers and children and served as an introduction to the project. During the workshop, participants drew a map of the clinic to show how, where and by whom, MCH and HIV services were provided. Clinic staff were asked to critically assess current service provision for mothers and children and identify service gaps to be addressed to improve integration of MCH services (Fig. ​(Fig.1).1). A clinic QI team was convened from among clinic staff providing MCH services, to support the provision of integrated services during the project intervention period and beyond. The clinic QI team included the operational manager, an all health workers in the well mother-baby clinic, community health workers and data support staff. Four subsequent mentoring visits were conducted over the three-month intervention period, each lasting 2–3 h. The QM undertook a series of activities to monitor integration of services, including: observing consultations with mothers and babies; conducting exit interviews (Supplementary File 1) with mothers; and reviewing children’s patient-held records (Road to Health Booklet; RTHB) to assess the number of services received on the day of the visit. Mentoring tools were used to support these activities, including a RTHC review checklist and an exit interview guide (Supplementary File 1). The QM met with the clinic QI team to provide feedback on these activities, review improvement plans and monitor progress towards providing an integrated well mother-baby service. QI activities successfully undertaken to improve integration included: streamlining patient flow for mothers and children; moving growth monitoring into the consulting room; providing all MCH services together in one consulting room; ensuring that all mother-baby pairs were seen by registered nurses, and ensuring that all equipment and supplies were readily available. In most clinics this meant that the comprehensive services were provided in a single MCH consulting room where health workers with different scopes of practice worked together. The QM also provided in-service training to staff to address knowledge gaps identified by staff. A sixth visit was conducted after completion of the mentoring process, during which all tools and activities were handed over to the clinic supervisor in the district to enable continuation of the project activities. Learning sessions were conducted each quarter as implementation was completed in one sub-district and started in the next sub-district to share and review the successes and challenges, encourage peer-to-peer learning and support health facility staff. Each learning session was attended by health workers from all clinics in two sub-districts, that is the sub- district where the intervention was wrapping up and the sub-district where it was just starting. Thus, clinic staff from each sub-district attended two learning sessions. Five learning sessions were conducted in total, one at the start of project and from then on every 3 months over the 1 year implementation period. Relevant members of the district management team attended the learning sessions to promote participation, buy-in and support for the project at district management level (Fig. 2). Description of Study Intervention. Outline of quality improvement intervention activities roll out in each sub-district Self-administered checklists were used to generate the data to drive the quality improvement process. The data required to monitor ongoing progress towards providing integrated services was not routinely available. Although relevant routine indicators (e.g. HIV testing, family planning etc.) were collected in clinics, these did not distinguish between services provided during an integrated well mother-baby visit and those provided at separate visits. Therefore, a self-administered checklist was developed which included all the elements of an integrated mother-baby service (Fig. ​(Fig.1).1). Health workers completed these checklists during the consultation to track the services mothers and babies received during well mother-baby clinic visit and identify the gaps in providing comprehensive services. Checklists also served as a reminder about services to be offered in the integrated well mother-baby package. Completed checklists were collected by the project team and data was collated and fed back to the clinic QI team at subsequent mentorship visits. Progress during the intervention period was reviewed at the learning sessions. A stepped wedge study was conducted to quantitatively evaluate the QI intervention using a phased approach. A stepped wedge design is a type of cluster randomised trial, where clusters are randomised and systematically and sequentially exposed to the intervention and evaluated over time [17]. Ethically appropriate, well-designed, stepped wedge studies can provide evidence of the effects of interventions and are considered higher quality than evidence from non-randomised studies [17, 18]. The stepped wedge design starts with a baseline assessment with no clusters exposed to the intervention, followed by each cluster sequentially randomised to cross from control group to intervention group at regular intervals, until all clusters are implementing the intervention [17]. In contrast to randomised controlled trials, this design allows the benefits of providing the intervention to include all clusters [17]. The stepped wedge design allows every cluster (in this case all clinics in a single sub-district) to provide pre and post intervention data over an extended period of time [17]. This is a strong research design for implementation research where it is not feasible or acceptable to have a control group that does not receive the intervention. The unit of randomisation was the sub-district. The order in which sub-districts received the intervention was randomly allocated from a list of four sub-districts. The intervention was implemented consecutively in this order in each sub-district over a three-month period. All clinics in one sub-district participated in the study and received the intervention at the same time (Table ​(Table11). The sample size was calculated to achieve 80% power to detect a 13% change in the proportion of mother-baby pairs who received a full package of care, from a non-informative baseline level of 50%. In order to take account of the clustering effect of mother’s attending the same clinic, a random effects multi-level mixed logistic model was used where the clinic was the random effect. The ICC (intra-class correlation) of 0.05 was included in the sample size calculation to adjust for clustering. Ten mother-baby pairs were enrolled in each of the 27 participating clinics, at each wave. Therefore, the total sample size was 270 mother-baby pairs for each wave of data collection, giving a total of 1350 participating mother-baby pairs after completion of five waves. Data are collected in five waves (Table ​(Table1)1) every three-months over the one-year intervention period. Data for the first wave was collected before starting implementation, where all clinics functioned as controls, and served as a baseline. Further waves of data collection continued as each sub-district completed the intervention, until the last wave of data collection when the intervention was complete in all sub-districts. Sequential sampling was used to recruit mother-baby pairs and all mothers attending the clinic for a well mother-baby visit with a child under the age of 24 months were approached to participate. This was continued until the sample size was achieved. An exit interview with participating mothers was conducted by trained fieldworkers, using a structured data collection tool developed specifically for this study on electronic tablets. The exit interview included questions regarding the maternal, child and women’s health and HIV services received during the visit. Children’s patient-held records (RTHBs) were reviewed to identify which services had been received that day and whether all health services for babies were up to date. Data collection continued in each clinic until the desired sample was reached. A multi-level mixed logistic regression model adjusting for clustering was used for analysis. to take account of random cluster (clinic) effects and a fixed time and group effect [22]. The unit of analysis was the mother-baby pair. Data was analysed using STATA V13.1. Each element of comprehensive package of care was designated as provided if the mother reported having received it (exit interview), or if it was recorded as being given on the patient held record (RTHB review). These data are presented separately, we did not combine findings from the exit interview and RTHB review. Overall differences in demographic characteristics between the control and intervention groups at baseline were compared using chi square or Mann –Whitney tests. The overall proportion of women with a positive result in each group (intervention and control) over the full period is reported, as well as the odds ratio and p value from the model. P value < 0.05 was considered to be statistically significant.