The last one heard: The importance of an early-stage participatory evaluation for programme implementation

listen audio

Study Justification:
The study aimed to investigate the importance of early-stage participatory evaluation in community maternal and child health programs. The systematic involvement of project beneficiaries in these programs is often limited, especially during the formative stages. Understanding how feedback from communities can inform and improve existing programs is crucial for the success of community health workers and for improving health outcomes.
Highlights:
– The study took place in North Rukiga, Kabale District, Uganda, over a period of 4 weeks.
– A qualitative research approach was used, including focus group discussions and in-depth interviews with 76 female participants.
– Beneficiaries reported confidence in the program and the relationships with community health workers, pride in the knowledge they received, and improved spousal involvement.
– Program challenges were identified, including barriers to adopting promoted behaviors, program dependency, and issues with perceived ownership.
– It was found that community health workers were not reaching the entire population of intended beneficiaries.
Recommendations:
– Conduct early-stage participatory evaluations of beneficiaries’ perceptions in newly initiated health programs.
– Use the feedback obtained to inform and adjust programs, better meeting end-user expectations.
Key Role Players:
– Project beneficiaries
– Community health workers
– Village chairpersons
– Mobilizers (pregnant women or community nurse/midwives)
– Researchers
– UMOH officials
Cost Items for Planning Recommendations:
– Research materials (e.g., information leaflets)
– Training for interviewers
– Translation and transcription services
– Software for thematic analysis (e.g., NVivo 10)
– Ethical approval process
– Travel expenses for researchers
– Compensation for participants (if applicable)

Background: The systematic involvement of project beneficiaries in community maternal and child health programmes remains low and limited, especially during the formative stages of the project cycle. Understanding how positive and negative feedbacks obtained from communities can subsequently be used to inform and iterate existing programmes is an important step towards ensuring the success of community health workers for maternal and child health programming and, ultimately, for improving health outcomes.Methods: The study took place over a period of 4 weeks in North Rukiga, Kabale District of southwestern Uganda. Using a cross-sectional qualitative study that employed an epistemological approach of phenomenology, nine focus group discussions and eight in-depth interviews were conducted with a total of 76 female participants across six different sites. Women were identified as either users or non-users of the maternal and child health programme. Purposeful sampling was employed to recruit women from six different locations within the programme catchment area. Translated and transcribed transcripts were subjected to a bottom-up thematic analysis using NVivo 10 Software, whereby themes were arrived at inductively. Results: Predominant themes emerging from the focus groups and key informant interviews identified early trends in programme strengths. Beneficiaries reported confidence in both the programme and the relationships they had forged with community health workers, exhibited pride in the knowledge they had received, and described improved spousal involvement. Beneficiaries also identified a number of programme challenges including barriers to adopting the behaviours promoted by the programme, and highlighted issues with programme dependency and perceived ownership. It also emerged that community health workers were not reaching the entire population of intended programme beneficiaries. Conclusions: This research provides support for the importance of an early-stage participatory evaluation of beneficiaries’ perceptions of newly initiated health programmes. Our results support how evaluations conducted in the early phases of programme implementation can provide valuable, timely feedback as well as yield recommendations for programme adjustment or re-alignment, and in turn, better meet end-user expectations. Potential reasons for the observed lack of community participation in early stages of programme implementation are considered.

This study took place over a period of 4 weeks in North Rukiga, Kabale District of southwest Uganda. This area was chosen since it was approximately 2 months into a newly initiated 5-year CHW programme led by World Vision Ireland. The AIM-Health programme is currently being implemented across ten settings in five different sub-Saharan African contexts. In Uganda, the programme is run in collaboration with World Vision Uganda and the Ugandan Ministry of Health (UMOH). All research occurred in the North Rukiga county of Kabale District in southwest Uganda, which consists of primarily of subsistence farmers and comprises two sub-counties, Kashambya and Rwamucucu, totaling 13 parishes. The southwest region of Uganda has some of the worst MCH indicators for the country, with only 25% of deliveries occurring in a health facility, and 19% of women giving birth with no assistance whatsoever [11]. This region also has the lowest percentage of postnatal care (PNC) with only 18% attendance [11]. We conducted a cross-sectional qualitative study that used the epistemological approach of phenomenology through in-depth interviews (IDIs) and focus group discussions (FGDs) to understand a woman’s lived experiences with the CHW programme to date. Village chairpersons acted as mobilisers using purposive sampling to gain access and disseminate research materials to potential participants. A semi-structured interview guide was developed for both FGDs and IDIs to assist in eliciting individual responses; however, it was not followed rigidly as phenomenological research is concerned with experiences and issues of importance to the individuals and is based on personal knowledge and subjectivity [47]. Women were given a preference of attending an FGD or an individual interview. FGDs are recognised as a strong methodology when discussing potentially sensitive topics [48], and the researchers felt this method may help women be more forthcoming about both their pregnancy experiences, as well as any problems they were facing with the MCH programme. Individual key-informant interviews were also offered as a methodology since using both FGD and in-depth interviews increases comprehensiveness in collection and allows for a more reflexive analysis [49]. Participants were first asked a question not specific to the programme (i.e. “What are some of the challenges that pregnant and nursing women face in your community?”). During the interview, the researcher encouraged the participants to control the flow of the conversation. Though attempts were made to avoid direct questions, the researcher was required to ask additional probing questions when conversations stalled. Interviews were conducted in the local language (Rukiga) by a female health professional, trained in interview techniques, and who was familiar with the AIM-Health programme. A member of the research team was also present to capture the tone, mannerisms, and note the body language of the participants. Recorded discussions were subsequently transcribed, translated, and the content verified by a bilingual Rukiga and English speaker. In total, nine FGDs and eight in-depth interviews were conducted with a total of 76 participants across six different sites, three from each of the sub-counties (Table 1). Data collection location and response Participants were pregnant and/or breastfeeding women residing in World Vision’s programme area of North Rukiga and therefore potential beneficiaries of the AIM-Health project. Women were self-identified as either users or non-users of the service provided by CHWs as part of AIM-Health, had to be at least 18 years of age, and had to have completed the informed consent process. Women were considered “users” of 7–11 if they had been visited at least once by a trained CHW. Contrastingly, a “non-user” was defined as a woman who had not been visited by a CHW trained as part of the 7–11 strategy despite living in the programme target areas. Purposeful sampling using community “mobilisers” was employed to recruit only AIM-Health potential beneficiaries from six different locations within the programme catchment area. Community mobilisers included either pregnant women or community nurse/midwives working out of a local health facility. Mobilisers were given participant information leaflets to either distribute or read to potential participants in their communities 7 days prior to data collection, which included information on participant requirements, methodology, and purpose of study. Interested participants were asked to reconvene at a particular date and time if they wanted to participate in the study. Specific times were allocated for both the FGDs and IDIs, with participants choosing which time and therefore which interview method they wanted to participate in. Upon arrival at the study site, the self-identified as either pregnant and/or breastfeeding women were asked their age and were asked for their preference of type of interview type. The six different locations were identified in consultation with UMOH officials, who chose them as they felt these locations best represented the population of North Rukiga. Translated and transcribed data were subjected to a bottom-up thematic analysis using NVivo 10 Software. No thematic groups were determined prior to data collection and analysis. A “theme” was therefore characterised as a recurrent underlying concept that offers a general insight from the entire data range [50]. Themes were arrived at inductively, as no hypothesis or thematic groups were present prior to analysis [47]. The analysis process was continuous and relied on the researcher’s growing familiarity with the data [51]. The analytical approach used was constructivism whereby the researcher employs a transactional and subjective approach and uses the interaction between participant and herself as well as her own personal knowledge and experience to analyse the data [51]. All transcripts were read thoroughly, and two main thematic groups (positive experiences and negative experiences) were included in the first level of coding. These groups were subsequently coded further to form sub-groups, with a third level of analysis occurring in specific themes that were rich in data. Though the research team attempted to reduce preconceptions and individual convictions during the collection and analysis phase, phenomenology and constructivism both recognise that analysis is a cognitive process and as such, it is subject to the researcher’s own view, understanding, conceptual orientations and experiences of the world, and is therefore undoubtedly affected by what they witness [51,52]. Ethical approval for this research was obtained from the Health Policy and Management/Centre for Global Health Research Ethics Committee, Trinity College Dublin and the Higher Degrees Research and Ethics Committee, Makerere University School of Public Health, Uganda. Aligned with best practice for illiterate participants, both written and verbal informed consent was obtained from participants.

N/A

Based on the provided description, here are some potential innovations that could be used to improve access to maternal health:

1. Mobile Health (mHealth) Solutions: Implementing mobile health technologies, such as text message reminders for prenatal care appointments and educational information, can help improve access to maternal health services, especially in remote areas.

2. Community Health Worker (CHW) Training and Support: Enhancing the training and support provided to CHWs can improve their effectiveness in delivering maternal health services. This can include regular refresher trainings, mentorship programs, and access to necessary resources and supplies.

3. Community Engagement and Participation: Promoting community involvement and participation in maternal health programs can help increase awareness, acceptance, and utilization of services. This can be done through community meetings, focus groups, and involving community leaders in program planning and implementation.

4. Task-Shifting and Task-Sharing: Expanding the roles and responsibilities of healthcare workers, such as nurses and midwives, can help address the shortage of skilled professionals. This can involve training and empowering these healthcare workers to provide a wider range of maternal health services.

5. Strengthening Health Systems: Investing in the overall health system infrastructure, including facilities, equipment, and supply chains, can help ensure that maternal health services are accessible and of high quality.

6. Financial Support and Incentives: Providing financial support, such as subsidies or conditional cash transfers, can help reduce financial barriers to accessing maternal health services. Additionally, offering incentives to healthcare providers, such as performance-based financing, can improve service delivery.

These are just a few examples of innovations that can be used to improve access to maternal health. It’s important to consider the specific context and needs of the community when implementing these innovations.
AI Innovations Description
The recommendation from this study is to conduct an early-stage participatory evaluation of beneficiaries’ perceptions of newly initiated health programs. This evaluation should involve the systematic involvement of project beneficiaries, particularly during the formative stages of the project cycle. By understanding the positive and negative feedback obtained from communities, programs can be informed and iterated to better meet end-user expectations. This approach can help improve the success of community health workers for maternal and child health programming and ultimately improve health outcomes. The study suggests that evaluations conducted in the early phases of program implementation can provide valuable, timely feedback and yield recommendations for program adjustment or realignment.
AI Innovations Methodology
Based on the provided description, here are some potential recommendations for improving access to maternal health:

1. Increase community participation: Encourage active involvement of project beneficiaries in the planning, implementation, and evaluation of maternal health programs. This can be done through community engagement activities, such as community meetings, focus group discussions, and participatory workshops.

2. Strengthen relationships with community health workers (CHWs): Build strong relationships between CHWs and beneficiaries by providing continuous training and support to CHWs. This can help improve trust, communication, and the quality of care provided by CHWs.

3. Address barriers to adopting recommended behaviors: Identify and address barriers that prevent beneficiaries from adopting the behaviors promoted by the maternal health program. This can include addressing cultural beliefs, financial constraints, and access to healthcare facilities.

4. Promote spousal involvement: Encourage the involvement of spouses in maternal health programs by raising awareness about the importance of their support during pregnancy, childbirth, and postnatal care. This can be done through community education campaigns and targeted messaging.

To simulate the impact of these recommendations on improving access to maternal health, a methodology could be developed using the following steps:

1. Baseline data collection: Gather data on the current status of access to maternal health services in the target area. This can include information on the percentage of deliveries in health facilities, attendance of postnatal care, and barriers faced by women in accessing care.

2. Intervention implementation: Implement the recommended interventions in a selected sample of the target population. This can involve training CHWs, conducting community engagement activities, and promoting spousal involvement.

3. Monitoring and evaluation: Collect data on the implementation of the interventions, including the level of community participation, changes in relationships between CHWs and beneficiaries, and the extent to which barriers to adopting recommended behaviors are addressed.

4. Impact assessment: Analyze the collected data to assess the impact of the interventions on improving access to maternal health. This can involve comparing the baseline data with the post-intervention data to identify any changes or improvements.

5. Feedback and adjustment: Use the findings from the impact assessment to provide feedback and make adjustments to the interventions as needed. This can involve refining the strategies, addressing any identified challenges, and scaling up successful approaches.

By following this methodology, it is possible to simulate the impact of the recommended interventions on improving access to maternal health and make evidence-based decisions for program improvement.

Share this:
Facebook
Twitter
LinkedIn
WhatsApp
Email