‘We allwork together to vaccinate the child’: A formative evaluation of a community-engagement strategy aimed at closing the immunization gap in North-West Ethiopia

The role of community engagement (CE) in improving demand for immunization merits investigation. The International Rescue Committee developed a CE strategy to implement a vaccine defaulter-tracing tool and a color-coded health calendar aimed at increasing uptake of immunization services in north-west Ethiopia (‘The Fifth Child Project’). We report findings from a formative evaluation of this project. In May/June 2016 we conducted 18 participant observations of project activities, 46 semi-structured interviews and 6 focus groups with caregivers, health workers, community members/leaders. Audio-recordings and fieldnotes were transcribed, anonymized, translated and analyzed thematically using inductive and deductive coding. Additional data was collected in November 2016 to verify findings. The project was suitably integrated within the health extension program and established a practical system for defaulter-tracing. The calendar facilitated personalized interactions between health workers and caregivers and was a catalyst for health discussions within homes. At the community level, a regulation exercise of sanctions was observed, which served as a deterrent against vaccine default. Pre-existing community accountability mechanisms supported the CE, although varying levels of engagement between leaders and health workers were observed. The benefits of shared responsibility for immunization were evident; however, more transparency was required about community self-regulatory measures to ensure health-related discussions remain positive. The purpose of the qualitative study was to evaluate the FCP in terms of integration into the local health system, community co-management and the acceptability and utilisation of the FCP tools. A series of interviews, focus group discussions (FGDs) and observations were conducted between May–July 2016. As part of this research, IRC developed a ToC (Supplementary Materials), which depicted the inputs and activities required and key assumptions made for the FCP to achieve specified outputs and outcomes. The ToC provided a framework for evaluating the project and findings were used to refine the ToC and improve the implementation of the FCP. Three kebeles from BGRS were selected for the evaluation to include study sites with different population size, cultural and religious affiliation and distance from an urban centre (Table 1). Mugfude and Jematsa were mainly inhabited by indigenous Muslim populations and Amba 17 by Christian settler groups. The primary income-generating activity was agriculture, supplemented by gold mining in Mugfude and Jematsa. Composition of selected study sites. HEWs: Health Extension Workers; HDAs: Health Development Army member; HDALs: Health Development Army Leader. The sample frame within these sites included people who played a role in the FCP: caregivers, HDA(L)s, HEWs, HEW supervisors, nurses (seconded to health posts), kebele leaders (KLs), other community leaders (e.g., teachers) and WoHO. We included all HEWs/nurses and all KLs from the kebeles and key WoHOs responsible for immunization in these kebeles and used purposive sampling to select a set number of HDA(L)s, caregivers and other community leaders with maximum variation in terms of age, education level, gender and caregiver relationship to the infant across the three kebeles. While this mix of recruitment strategies was the only option due to the limited number of HEWs, nurses and KLs available in the three kebeles, the possible impact on the representativeness of the different sampling groups was taken into account by analysing and reporting data by type of participant. A local team of research assistants (‘research team’) from BGRS with English, Amharic and Rutana language skills were trained to conduct interviews, FGDs and observations in order to limit the influence of the European London School of Hygiene and Tropical Medicine (LSHTM) researchers on participants. The local research team lead (SW) was well known from the community as he had previously worked as a clinical officer in the regional hospital’s maternity wing. This facilitated rapport with the participants, including caregivers and district-level leaders, but could have also influenced some of the results. TC was present for a few initial interviews and focus groups to provide support and feedback to the newly trained research team, which could have influenced some of the participants’ responses. Interviews and FGDs took place in private settings and lasted 1–1.5 h. The interview style was semi-structured to cover pre-defined topics and shape exchanges according to interviewees’ roles, experiences and responses. FGDs were conducted with caregivers (mothers and fathers) and HDAs to facilitate discussions about immunization, the tools, and their engagement among themselves. With the interviewees’ permission, the research team collected basic socio-demographic information, compiled field notes, and audio-recorded interviews and FGDs. Participant observations, conducted to obtain more information about how the DTT and calendars are used along with the community’s engagement, focussed on two FCP activities: (1) monthly command post meetings with kebele leaders, HEWs, HDALs, and representatives of youth and women groups; and (2) HEW/HDA routine visits to caregivers’ homes. The research assistants conducted the observations in pairs and compiled field notes summarising the event, the type and number of people present, the purpose of the activity, and discussions held. The LSHTM researchers were not present during the observations to limit their influence on interactions and activities, but this also created a limitation as the local research team, including SW, was not fully trained on this methodology that would have required a different set of skills and more time and immersion in the field. The research team transcribed the audio recordings into Amharic with secretarial support. The transcripts were anonymised by allocating a numerical identification to each participant and storing these separately from the participant database. The transcripts were translated from Amharic to English by a firm in Addis Ababa, emailed securely to TC and EK and uploaded to NVivo, a qualitative data analysis software program. The approach to data analysis was thematic [14] and used the FCP’s ToC as a reference point. TC and EK developed a coding framework by drawing parent codes from the topic guides deductively and developing sub-codes inductively. They coded the first five transcripts separately, and met to compare findings and to start developing the framework. A report on data collection and findings compiled by SW, and field notes from the observations, were also used to inform the interpretation of the data and the identification of themes. TC coded the remaining transcripts, meeting several additional times with EK to refine the framework. SW also commented on the framework in bi-monthly phone calls. Codes were organized together with quotes from the transcripts to compare and contrast the data, including the recurrence of certain themes and the terminology used by participants. Categories and typologies were drawn from the codes and developed into emerging themes to further discuss the meaning of the data. An Ethiopian researcher based in the UK (BS) supported this data analysis by conducting translation checks and acting as a cultural interlocutor (although she was not originally from BGRS, she worked closely with the local research team lead (SW)). This analysis and initial findings were discussed with members of the research team, IRC FCP implementers, district and regional health offices, health extension workers and health development army members and some research participants during dissemination activities in BGRS in November 2016. During this time, SW, BS and TC conducted additional interviews with a WoHO, two KLs, two HEWs and two FGDs with caregivers (all mothers) and HDAs to verify some analyses, specifically those relating to the use of the Enat Mastawesha and maternal literacy levels and the application of community-agreed sanctions for non-immunization. This data was analyzed as described above. Potential participants received a study information letter in Amharic from the local research team and had the opportunity to ask questions before agreeing to be interviewed. Verbal consent is the preferred practice for obtaining agreement for participating in research in this region, hence participants audio-recorded a statement citing their name, date and willingness to participate. To preserve participants’ confidentiality, all transcripts and observation sheets were anonymised, and only the research team had access to the names of participants. The LSHTM Observational and Interventions Research Ethics Committee (Ref 10542) and the Regional Health Bureau of BGRS in Ethiopia approved this research (Ref 674/□□ m-o1).