Maternal and neonatal implementation for equitable systems. A study design paper

Background: Evidence on effective ways of improving maternal and neonatal health outcomes is widely available. The challenge that most low-income countries grapple with is implementation at scale and sustainability. Objectives: The study aimed at improving access to quality maternal and neonatal health services in a sustainable manner by using a participatory action research approach. Methods: The study consisted of a quasi-experimental design, with a participatory action research approach to implementation in three rural districts (Pallisa, Kibuku and Kamuli) in Eastern Uganda. The intervention had two main components; namely, community empowerment for comprehensive birth preparedness, and health provider and management capacitybuilding. We collected data using both quantitative and qualitative methods using household and facility-level structured surveys, record reviews, key informant interviews and focus group discussions. We purposively selected the participants for the qualitative data collection, while for the surveys we interviewed all eligible participants in the sampled households and health facilities. Descriptive statistics were used to describe the data, while the difference in difference analysis was used to measure the effect of the intervention. Qualitative data were analysed using thematic analysis. Conclusions: This study was implemented to generate evidence on how to increase access to quality maternal and newborn health services in a sustainable manner using a multisectoral participatory approach.

The study was implemented in Pallisa, Kibuku and Kamuli Districts, which are mainly rural, where the SDS study that used vouchers for transport, ANC, delivery and PNC services was implemented previously [1]. Pallisa has two health subdistricts (HSDs), Kibuku has one HSD and Kamuli three. A health subdistrict is an administrative zone within the district. The three districts had a total population of 1,075,242. Kibuku has a population of 202,033, Pallisa 386,890, while Kamuli has 486,319 [19]. A DHMT heads the health system at the district level. The DHMT is located at the district headquarters within a department of health headed by a district health officer, with a team of about 15 administrative and management staff. At district level, the higher-level health facilities comprise a hospital and a health centre (IV), which provide inpatient and outpatient care as well as surgical services. The hospital serves the whole district population, which is about 335,536 on average while the health centre (IV) serves approximately 126,586 people [20]. The lower-level facilities consist of a health centre (III) and health centre (II). The health centre III has a general outpatient clinic, admits patients and has a functional maternity and laboratory service, while the health centre (II) provides mainly outpatient services. The health centre III serves an average population of 26,785, while the health centre II serves an average population of 5,057 (20). The lowest level of care in the district health system is the health centre I (HCI). The HCIs are voluntary service structures, which provide mainly mobilization, sensitization and outreach activities through VHTs. The VHT is ideally comprised of five volunteers per village who are responsible for providing these services at their respective villages. At village level, the average population size is estimated at 650 persons [20]. The health service infrastructure in the study area comprised a total of 104 health facilities; 33 in Pallisa, 17 in Kibuku and 54 in Kamuli. The intervention was implemented using a quasi-experimental design. We had three HDSs in which the intervention was implemented and two comparison HSDs. Pallisa, Kibuku and Buzaya (Kamuli) HSDs were taken as intervention areas, whereas the HSDs of Bugabula (Kamuli District) and Butebo (Pallisa district) were the comparison areas. The intervention package was provided only to the intervention arm. However, some aspects of the intervention such as home visits by the VHTs were at times provided by other non-government organizations (NGOs) in the comparison arm as well. The package offered by these VHTs was, however, often different from that provided by the MANIFEST study. They may also have received the radio messages that were aired over the radio, as well as some aspects of the support supervision that was done by the district team in both intervention and comparison areas as part of routine supervision of health facilities in the district. The study population were women of reproductive age (15–49 years) and men of reproductive age, health workers and district health officials, transporters and members of community savings groups, subcounty chiefs and community opinion leaders. The intervention had two phases: the preparatory phase and the implementation phase. The preparatory phase comprised of the development and adaptation of materials and tools, setting up of implementation structures, selection and training of CHWs. The materials for training CHWs and their supervisors were adapted from those used by the Ministry of Health and the UNEST study. The kits that were used by CHWs during home visits were also adapted from the UNEST study. These kits included registers, referral forms, birth plan forms, family cards and report forms. The VHTs were given a bag for carrying these materials and a T-shirt for identification purposes. Training materials about safe delivery, newborn care and resuscitation were adapted from the existing Ministry of Health (MoH) training and demonstration materials. The performance assessment tools for health facilities and health workers were also adapted from the existing standard assessment tools developed by MoH. The DHMTs in collaboration with the MakSPH research team were responsible for preparing radio spots messages and talk shows in respective local languages. The district level stakeholders played a lead role in the implementation of the project through the district-, subcounty- and parish-level committees. The committees at district and subcounty level were composed of existing management committees, to which additional members such as political and religious leaders were added. The parish-level committee, on the other hand, was composed of CHW representatives from each village. After setting up the community-level implementation committees, they were oriented about their roles and expected outputs by the DHMT members. The DHMTs across the three districts selected two VHTs out of the existing community volunteers working with the health department. In the villages that did not have volunteers, the local village leadership organized meetings, in which two volunteers were selected. The selection of VHTs was guided by the MoH guidelines for VHT selection. There were 27 subcounties with 847 villages in the three districts (514: 346 in Pallisa and 244 in Kibuku) from which 1,694 CHWs were selected. The CHWs received a four-day training by the district trainers. The district trainers were trained and supported by national VHT trainers. The training was conducted in local languages and focused on birth preparedness, maternal and newborn danger signs, referral of pregnant women and newborns, how to conduct home visits, and how to give health education talks. The CHWs also received training about family planning. The implementation phase was designed to have two main components: community empowerment for comprehensive birth preparedness and health provider and management capacity-building. This component had the following activities; home visits by community health workers, community mobilization and sensitization using community dialogue meetings and radios and linking communities with financial social networks and local transport providers. The trained CHWs conducted two home visits during pregnancy and two after delivery. They also counselled mothers on essential maternal and newborn care practices, safe delivery and birth preparedness. The CHWs identified women and children with danger signs and those identified were refered to the health facility for futher screening and care. The CHWs, with the assistance of local council leaders, mobilized community members for sensitization and dialogue meetings, while the district health team used radio spot messages and talk shows to sensitize the community. The sensitizations and dialogue meetings addressed issues concerning birth preparedness, men’s role in maternal and child health care, joining saving groups, organizing transport to the health units, PNC and family planning. The community dialogues were conducted on a quarterly basis and were facilitated by CHWs and resource persons within the community. The radio talk shows and spots, on the other hand, were conducted on a monthly basis and daily for specific time periods during the project implementation, respectively. The role of the radio sensitizations was majorly to reinforce the messages discussed in the community dialogues and VHT home visits. The CHW supervisors (health assistants and health workers) conducted directly observed supervision of CHWs in the villages once a month to reinforce their skills. In addition, the CHWs had monthly meetings with their supervisors to support them with report writing, problem-solving and reinforcing skills. The CHW leaders at parish level (one per parish) also mobilized and oversaw the work of fellow CHWs in each parish. On a quarterly basis, a combined meeting of CHWs, supervisors, and members of the DHMT to share work experiences and reinforce skills was undertaken. The CHWs submitted their reports to their parish-level representative, who in turn submitted them to the health assistant for record keeping on a monthly basis. Community development officers, VHTs and local council leaders mobilized community members at village level and sensitized them to form or join existing saving groups. These saving groups provided social protection in terms of financial benefits that cater for both domestic and health care needs of the members. Communities were also encouraged to form agreements through their savings groups with local transporters to ease transportation to health facilities. The agreement would garantee the transporter payment through the savings groups. This component had three main activities: (1) training and skills building for health workers and managers, (2) provision of basic support, and (3) recognition of good performance. The DHMT received training about planning and management of health services, including management of medical logistics, support supervision and mentoring of staff. This was done through two kinds of courses: (1) a short course in health services management that targeted mid- and lower-level health managers, and (2) a postgraduate diploma in monitoring and evaluation for the district health officers. In addition, these managers were actively involved in the implementation of the MANIFEST project as a means of reinforcing their skills, as well as building any specific management skills in order to foster quality and continuity as noted earlier. The general frontline health workers, on the other hand, received a work-based training in the management of obstetric emergencies and newborn care given by external experts. In addition, mentorship and support supervision were provided to reinforce these skills. The participating health facilities received equipment for conducting safe deliveries and neonatal resuscitation to improve the quality of existing services. The district health office selected the items and equipment that was to be purchased by the health facilities. The equipment provided to each of the districts was valued at approximately $3,000 and included delivery beds, newborn resuscitation kits, delivery kits, oxygen cylinders and vacuum extractors to aid difficult vaginal childbirth. The performance of the participating health facilities and health workers was assessed on a quarterly basis using the standard MoH assessment tools. The best two performing facilities and health workers, and the most improved facility per district in a period of six months (two quarters), were given recognition awards by the DHMT and the project. This was in order to motivate them, maintain their performance, and also inspire others. The participants for qualitative data were selected purposively depending on the research question being investigated and the saturation and maximum variation principles were employed in deciding the sample size. Focus group discussions (FGDs) were done with women and men of reproductive age. The FGDs were stratified according to the community sociodemographic characteristics (hard to reach, accessible, rich and poor communities). In each of the strata, three FGDs comprising 8–12 people were conducted at baseline and endline. Twenty-eight key informant (KI) interviews were done with health workers, health managers, members of the district health team and politicians. Table 1 provides detailed information about the number of FGDs and KIs that were conducted in the intervention and comparison arms. Sources of focus group discussions and key informant interviews conducted at each point of evaluation data collection (baseline and endline). The sample size for the household survey with women of reproductive age was based on the intervention’s assumption that, after three years (2013–2015) of implementation, skilled deliveries in the intervention area of Kibuku, Pallisa and Kamuli districts would increase from 38% to 58%, 62% to 72% and 68% to 78%, respectively [21]. The sample size was determined using a two-sided Z-test of the difference between proportions (Equation 1) with 80% statistical power, a 5% significance level, and 1.5 design effect, which resulted in a sample size of 2,293 women. Sample size determination formulae The sample size for the household survey with men of reproductive age was dependent on the assumption that their knowledge on maternal danger signs was 90%, in line with a study done in Kenya [22]. Using the Kish Leslie sampling formula at 5% level of significance, precision and non-response rate, a sample of 218 men per study arm in each of the districts was calculated. The sample size for the exit interviews was based on the assumption that the level of satisfaction and the quality of MNH services in the three districts would increase by 15% (from 50% to 65%) in the intervention area over the three-year period (2013–2015). The sample size was determined by a two-sided Z-test of the difference between proportions (Equation 1) with 80% power and a 5% significance level. From the calculation, a sample of 780 mothers was calculated for the baseline and endline survey. The evaluation data were collected using both qualitative and quantitative methods. Data were collected before project implementation (baseline), during project implementation (midterm), and at the end of the project implementation (endline). Table 2 details the different methods of data collection and sources used for each of the study objectives. Details of data collection methods and sources. The FGDs and KI interviews were done at community level. The participants were identified purposively with the help of local council officials who also set-up the venue ideal for the discussion in their community. Key informant interviews and focus group discussions were used to collect data about the perceived quality of maternal and newborn services, factors that affect delivery of MNH services and saving practices. They were also used to obtain the stakeholders’ perception about different components of the intervention. Stakeholder meetings were also used to collect information about the intervention. KI guides, FGD guides, personal memos, observation checklists, meeting reports were used to collect the qualitative data. Trained research assistants moderated the FGDs and KI interviews and documented the process. All the interviews were audio-recorded and transcribed later with the help of research assistants. On the other hand, the district health team members, assisted by the project team, facilitated the stakeholders’ meetings at district and subcounty level. The participants’ observations and key issues noted were documented in memos and meeting minutes. All the study documents were stored on a web-based database with access restricted to only project team members. To ensure the quality of data collected the following measures were considered. A public health qualitative expert trained and supervised the research assistants who conducted the qualitative interviews. All the data collection instruments including FGDs and interviews were pretested in one of the districts that was considered to have similar community characteristics as the study implementation districts. KI interviews and FGDs were audio-recorded in order to minimize loss of information. Regular review meetings were conducted to facilitate reflection on personal level observations of the study team and other stakeholders involved. The qualitative data collected before and after the implementation of the intervention were transcribed and reconciled with notes recorded during the interviews and then analyzed using thematic and content analysis [23]. Reading and rereading of the data to facilitate familiarization with the data preceded the coding of the data. The coding process was both inductive and deductive guided by the objectives of the study. Different levels of abstraction were then applied to the codes to develop further the analysis process depending on the method used for each specific objective. Where appropriate, theoretical frameworks were used to facilitate the analysis and interpretation of the findings. An active process of writing memos and reflection also facilitated the analysis processes. Reviewing of the processes by the different researchers was conducted to ensure study worthiness. For example, in the thematic analysis, the themes were reviewed to ensure that there was internal homogeneity and external heterogeneity between the themes. A two-stage sampling was applied per district for each of the study areas during the household survey. First, the villages were selected using probability proportionate to size sampling techniques, and thereafter all households were listed in order to identify eligible study participants. Women whose pregnancies were terminated before 20 weeks and women who were not residents and had not stayed in the community for at least one year were excluded from the study. Regarding the health facility exit interviews, all women who delivered from the health facilities were interviewed on discharge until the predetermined sample size was realized. Women aged less than 18 years but who were married and could provide individual consent were also included. On the other hand, those with severe illnesses at the time of the survey and those that refused to consent were excluded. Household surveys were conducted among women of reproductive age and used to collect information about knowledge of maternal and newborn danger signs, utilization of maternal health services and birth preparedness. A household survey was done among men of reproductive age to obtain information about knowledge of maternal and newborn danger signs. Client exit interviews were done to assess client satisfaction wth maternal and newborn health services. We assessed the quality and availability of services in all 43 health facilities that were providing maternal and newborn services from both intervention and comparison areas. This assessment was used to collect information on the availability of staff, essential equipment and drugs for maternal and newborn health services. In addition, we undertook monthly review of records on maternal death, newborn death, stock out of drugs, still births, health facility delivery, management of obstetric and neonatal complications, ANC and PNC attendance from all the 43 health facilities. The above quantitative data were collected using electronic health management information forms, VHT community records, structured questionnaires and observation checklists. Trained research assistants collected data during the household and health facility exit interview surveys. They were supervised by senior researchers. During data collection, data editors checked all the completed questionnaires for errors and missing information. Any error identified was verified and corrected immediately by the research assistants while in the field. The research assistants also collected the health facility assessment information. This was collected by observing the availability of equipment/drugs under the guidance of the health facility in-charge. The health facility records on health facility delivery, ANC attendance, PNC attendance, management of complications, maternal death, newborn death and stillbirths were collected from the health facility maternity register. To ensure the quality of data collected the following measures were considered: a data collection manual which stipulated what was to be done during sampling, data collection, entry, storage and management was developed and utilized. A team of medical doctors, a statistician, and public health specialists trained the research assistants. The field supervisor sampled and re-interviewed some respondents each day, in order to check for consistency and accuracy of the information being collected. All the data collection instruments were pretested in one of the districts that was considered to have similar community characteristics as the study implementation districts. A study advisory group, as well as the research partners for the Future Health System (FHS) consortium, provided regular oversight of the implementation of the study and provided advice. The entered data were cleaned and then analysis was undertaken. A number of analysis techniques were used to anwser the different objectives. These included descriptive statistics, regression and multivariate analysis techniques such as the difference in difference analysis and logistic regression. The results of this intervention are provided in the different papers that are part of this supplement. The MANIFEST study had several strengths. It comprised a multisectoral intervention package, which allowed it to address different demand- and supply-side constraints that hinder acess to MNH services. This intervention was implemented using a participatory action approach that is lauded for its potential to promote local ownership and participation of local stakeholders. The study also used both quantitative and qualitative data collection methods and this enabled us to have a quantitative assessment of the effect of the intervention with indepth explanations of how different factors influenced the intervention. Although the experimental design (quasi-experiemental) used allowed us to have a comparison area and to assess the contribution of the intervention, it did not allow us to assess the separate effects of each component of the intervention package. The second limitation is that the participatory approach used to implement the research is dependent on local stakeholders who may not have all the required skills, and this may have affected the intervention implementation efficiency. However the study team provided technical support through out the study. Another limitation was that there was a risk of contamination, because the comparison area is part of the district where the intervention is and staff sometimes get transferred from the intervention area to the comparison area. Another source of contamination is the radio which was used as a channel for communicating some messages. The listenership for the radio includes community members in the comparison area.

N/A

• Dima Health