Opening the ‘implementation black-box’ of the user fee exemption policy for caesarean section in Benin: A realist evaluation

listen audio

Study Justification:
The study aimed to understand why and under what circumstances the implementation of the user fee exemption policy for caesarean sections in Benin succeeded or failed. This policy was introduced in 2009 to improve access to maternal health services. The study used a realist evaluation approach, which is well-suited for studying complex problems in health policy and system research. By identifying the causal mechanisms that explain the outcomes of the policy, the study aimed to provide insights into how to improve the implementation of user fee exemption policies and similar universal health coverage reforms.
Study Highlights:
– The study used a realist evaluation approach to understand the implementation of the user fee exemption policy for caesarean sections in Benin.
– Two hospitals with contrasting outcomes were selected for the study.
– Data from 52 semi-structured interviews, a patient exit survey, a costing study of caesarean sections, and an analysis of financial flows were used for analysis.
– The study identified two main causal pathways: effective implementation in a state-owned hospital with a public-oriented management system and poor implementation in a non-state-owned hospital with managers guided by financial interests.
– Trust, perceived coercion, adherence to policy goals, perceived financial incentives, and fairness in their allocation were found to drive compliance and positive responses to incentives.
– Enforcement by hierarchical authority and bottom-up pressure were important for compliance, while persuasion depended on alignment with personal and organizational values.
Recommendations for Lay Reader and Policy Maker:
1. Improve accountability and administrative management in hospitals: Enhance the public-oriented management system in hospitals to ensure effective implementation of policies. This includes establishing channels for citizen demand for accountability and addressing administrative challenges.
2. Strengthen enforcement mechanisms: Ensure that hierarchical authorities enforce policy compliance and provide bottom-up pressure to drive implementation.
3. Align policy goals with personal and organizational values: Persuasion is more likely to be effective when there is alignment between the policy goals and the values of the actors involved. Consider the values and beliefs of stakeholders when designing and implementing policies.
4. Ensure fairness in the allocation of financial incentives: Fair allocation of financial incentives can drive compliance and positive responses to incentives. Develop mechanisms to ensure that incentives are distributed fairly and perceived as such by stakeholders.
Key Role Players:
1. Ministry of Health: Responsible for policy formulation and priority-setting process.
2. Hospital Managers: Responsible for implementing policies at the operational level.
3. Hierarchical Authorities: Responsible for enforcing policy compliance.
4. Citizens: Demand accountability and play a role in driving implementation through various channels.
Cost Items for Planning Recommendations:
1. Administrative reforms: Budget for administrative changes to enhance public-oriented management systems in hospitals.
2. Enforcement mechanisms: Allocate resources for training and capacity-building of hierarchical authorities to ensure effective enforcement of policies.
3. Stakeholder engagement: Budget for stakeholder engagement activities to align policy goals with personal and organizational values.
4. Financial incentives: Allocate funds for the fair allocation of financial incentives to drive compliance and positive responses to incentives.
Please note that the cost items provided are general categories and not actual cost estimates. The specific budget requirements will depend on the context and scale of implementation.

The strength of evidence for this abstract is 8 out of 10.
The evidence in the abstract is strong because it provides a detailed description of the study design, data collection methods, and analysis approach. The abstract also presents the initial programme theory and the findings of the study. To improve the evidence, the abstract could include more specific information about the sample size and characteristics of the study participants, as well as the limitations of the study.

To improve access to maternal health services, Benin introduced in 2009 a user fee exemption policy for caesarean sections. Similar to other low- and middle-income countries, its implementation showed mixed results. Our study aimed at understanding why and in which circumstances the implementation of this policy in hospitals succeeded or failed. We adopted the realist evaluation approach and tested the initial programme theory through a multiple embedded case study design. We selected two hospitals with contrastive outcomes. We used data from 52 semi-structured interviews, a patient exit survey, a costing study of caesarean section and an analysis of financial flows. In the analysis, we used the intervention-context-actor-mechanism-outcome configuration heuristic. We identified two main causal pathways. First, in the state-owned hospital, which has a public-oriented but administrative management system, and where citizens demand accountability through various channels, the implementation process was effective. In the non-state-owned hospital, managers were guided by organizational financial interests more than by the inherent social value of the policy, there was a perceived lack of enforcement and the implementation was poor. We found that trust, perceived coercion, adherence to policy goals, perceived financial incentives and fairness in their allocation drive compliance, persuasion, positive responses to incentives and self-efficacy at the operational level to generate the policy implementation outcomes. Compliance with the policy depended on enforcement by hierarchical authority and bottom-up pressure. Persuasion depended on the alignment of the policy with personal and organizational values. Incentives may determine the adoption if they influence the local stakeholder’s revenue are trustworthy and perceived as fairly allocated. Failure to anticipate the differential responses of implementers will prevent the proper implementation of user fee exemption policies and similar universal health coverage reforms.

We adopted the realist evaluation approach, which is a particular form of theory-driven evaluation (Stern et al., 2012), is rooted in realist philosophy of science. At its centre is the notion of generative causation (Pawson and Tilley, 1997; Wong et al., 2016) which makes it particularly adapted to study complex problems (Westhorp, 2013; Bamberger et al., 2016), including in health policy and system research (Gilson, 2012; Adams et al., 2016). The variable implementation of the user fee exemption policy for caesarean section across hospitals is best considered as a complex problem. Indeed, policies are implemented in health systems which are best understood as open systems. Policy implementation is likely to be influenced by existing and new health policies but also by other political, economic and social events. Many actors are involved, from various positions and with specific interests and ideas about the policy (Van Belle et al., 2017). All these point to the complex nature of the causal processes underlying the outcomes of the policy. Realist approach demands the researcher to make explicit the causal mechanisms that explain the observed outcomes and its variation (McLaughlin, 1987; Pawson and Tilley, 1997; Pawson, 2004; Wong et al., 2016). Realist evaluation indeed considers that it is not the policy, but the actors who are engaged in it who bring about the results, situated as they are in specific contexts. Policies work (or not) because actors make particular decisions in response to the resources or opportunities that the policy provides. Realist evaluators thus aim at identifying the underlying ‘reasoning and resources’ (Pawson and Tilley, 1997, p. 68) or generative mechanisms that explain how the outcomes were brought about and in which conditions (Pawson and Tilley, 1997). As such, realist evaluation fits policy implementation research well: it allows for exploring not only the policy formulation and priority-setting process, but also—like in our research question—how and why actors, within the organizational setting and in the broader societal context, implement policies (or not; Marchal et al., 2018). In the following paragraphs, we present the development of the initial programme theory, the study design used to test it, and the methods to collect, to analyse the data and to refine the initial programme theory. Realist evaluation starts with eliciting the initial programme theory of the intervention in question (Marchal et al., 2012). The initial programme theory can be considered as the hypothesis that posits how and why a particular intervention would lead to specific outcomes, for whom and in which conditions. Often, the initial programme theory is based on the results of literature reviews. We started with the berry-picking method to explore the field, followed by a non-systematic review of the policy implementation literature. Berry-picking approach is appropriate for identifying elements of the initial programme theory (Finfgeld-Connett and Johnson, 2013). Like realist synthesis (Pawson et al., 2005), the berry-picking method is an iterative, adaptive, creative and flexible process that can contribute to generate a theory that explains a social phenomenon by consolidating or federating fragmented pieces of the literature (Booth, 2008). The first step is a literature search, followed by the selection of a first set of articles relevant to the research question. Analysis of the selected publications leads to the reformulation of the research question, which in turn is followed by a second search and so forth until the synthesis of the results adequately answers the research questions (Booth, 2008). First, we searched for frameworks, models or theories that could inform our initial programme theory. We reviewed papers dealing with the implementation of user fee exemption policies for maternal healthcare and healthcare in general. We found that most publications describe policy implementation gaps and factors that may explain implementation failure, like inadequate inputs and poor communication (Witter and Adjei, 2007; Agyepong and Nagai, 2011). Few studies started from an explicit theory or theoretical framework on policy adoption or implementation. Few authors attempted to explain the success or failure of the policy implementation by using theories or models from political science (public administration and policy analysis). Authors like Ridde and Diarra (2009) and Walker and Gilson (2004) used the concept of street-level bureaucracy (Lipsky, 1980) to explain the implementation gap of user fee exemption policies. However, none of the papers referred to the broader policy implementation literature. The scant results of the first review led us to review the policy implementation literature. Given the limited consensus on policy implementation theories (O’Toole, 2004; DeGroff and Cargo, 2009), a systematic review was not likely to be useful. We, therefore, carried out a non-systematic review of the published and grey literature. We started the search using the Web of Science and Social Sciences Citation Index search engines with sets of keywords combining ‘policy’, ‘program’ and ‘implementation’ or ‘adoption’. We used extensive snowballing to track down the original papers and books from the bibliographic references of the initial list of papers and books. This review identified a first set of three publications. We selected the paper of Berman on macro- and micro-implementation of social policy (Berman, 1978) because it presents a comprehensive model, going from policy design to local implementation and back, and it emphasizes the multi-level nature of policy implementation. Second, we selected Elmore’s forward and backward mapping model (Elmore, 1982). This model identifies local agents as key actors in policy implementation and provides an advanced framework for successful ‘planning’ of policy implementation, from the first choices to the expected outcomes (forward mapping) and back (backward mapping). Third, we added the ambiguity-conflict model of policy implementation of Matland, because it represents a theory explaining implementation gaps that goes beyond the top-down and bottom-up divide (Matland, 1995). At this stage, we extracted from those papers, the contents related to intervention (in casu policy), context, actors, mechanisms and outcomes, using the Intervention-Context-Actor-Mechanism-Outcome (ICAMO) heuristic to identify the configurations that explain the reported outcomes at each level (Van Belle, 2014; Marchal et al., 2018). ICAMO is a modified version of the context-mechanism-outcome configuration (Pawson and Tilley, 1997) that stimulates a proper description of the actual intervention and ‘the actors through whom the intervention works’ (Mukumbang et al., 2018). This process pointed us to three ‘passages’ or implementation phases (Berman, 1978, p. 13) that provided the overall multilevel structure of our initial programme theory: Phase 1: This phase includes the actual decision-making process and the institution of a programme to implement the policy. It involves macro-level key stakeholders, including the ministers’ council, and results in making the decisions for the policy and its financing. Next, an administration or government agency is set up to develop a programme to implement the policy decision. The more ambiguous the policy intent, the more latitude such agency has in shaping the policy. Phase 2: This phase encompasses the transition from programme to local adoption at operational health service level, during which slippage between programme guidelines and actual implementation can occur. This is partly determined by the degree of compliance with decisions by higher authorities (and thus the enforcement capacity) and alignment of the policy goals with local needs. Programmes are actually or symbolically adopted by district health authorities and hospital managers. Phase 3: This phase concerns the implementation of the programme by service providers, who act as street-level bureaucrats. Berman calls this ‘micro-implementation’ and considers this ‘may be the most pivotal step because a social policy’s outcome depends on local delivery’ (Berman, 1978, p. 21). Service providers can implement the policy in four ways: (1) non-implementation, (2) co-optation (or adaptation of the programme to fit existing practices), (3) technological learning (adaptation of routine practices to accommodate the policy) and (4) mutual adaptation, whereby both policy and service delivery are adapted to ensure optimal fit. In previous studies, Berman found that only mutual adaptation leads to achievement of the intended policy outcomes: the policy is adapted to the organization, its staff, its target public and its environment, and organizational changes are made to better implement the policy. If the three papers provided an overarching architecture for a multi-level analysis of policy implementation, they did not present mechanisms. Our berry-picking approach led to the ‘carrots, sticks and sermons’ typology of policy instruments (Bemelmans-Videc and Vedung, 1998). Carrots, sticks and sermons can be considered as a parsimonious typology of mechanisms underlying policy instruments and more specifically as categories of drivers of individual or organizational commitment to the policy. Policies that include incentives (‘carrots’) may induce a positive response when actors perceive a financial or other gain, or if any financial losses due to the policy are compensated properly (net benefit or no net loss). Policies that are based on coercion and sanctions (‘sticks’) may induce perceived pressure and fear of sanctions that in turn may generate compliance. Policies based on persuasion (‘sermons’) trigger good implementation if the actors perceive an alignment between personal believes and policy goals and values. Figure 1 presents our initial programme theory. Initial programme theory. We adopted the multiple embedded case study design (Yin, 2009), which is well adapted to research on policy implementation gaps. We defined ‘the policy implementation’ as the case. We considered a hospital implementing the user fee exemption policy for caesarean section as the unit of analysis. Two units of analysis were purposively selected among the seven implementing hospitals covered by the FEMHealth research programme in Benin—a faith-based (non-state-owned) hospital and a public (state-owned) hospital—because they present contrasts in terms of ownership, location and policy implementation outcomes (Table 1). A detailed description of the sampling procedure of FEMHealth is reported elsewhere (FEMHealth, 2014). General characteristic of the two units of analysis in 2011 Source: Administrative records of the hospitals and Biaou et al. (2015). Realist evaluation is method-neutral: the data collection methods should allow to gather data needed to test the initial programme theory. We collected both quantitative and qualitative data to describe the case and its units of analysis. Table 2 presents the data sources, tools and their specific purposes. A multi-disciplinary team (a medical doctor, a health economist and a social scientist) collected the data between March 2012 and March 2013. Data collection techniques and data sources and study sample in the two study sites We transcribed verbatim the recordings of the interviews and entered the transcripts in QSR NVIVO 10, converted later to QSR NVIVO 12. In a first phase, we adopted thematic analysis principles (Miles and Huberman, 1984), whereby the initial coding was based on the key elements of the initial programme theory, which evolved during the analytical process. We used Microsoft Excel 2011 (version 14.6.7) to do a univariate descriptive analysis of the quantitative data from the exit survey and from the financial flow tracking study. Table 2 presents the analytical approach used for each type of data. In a second phase, we triangulated qualitative and quantitative data to develop a configurational analysis of each case using the ICAMO heuristic tool (Van Belle, 2014; Marchal et al., 2018). We described the intervention (in casu policy), context, actors, mechanisms and outcomes, and identified the configurations that explain the reported outcomes at each level. We mapped the causal configuration looking for causal links between the actually implemented intervention, relevant contextual elements, the mechanisms that were triggered for specific groups of actors and the observed outcomes. Mechanisms are key components in this heuristic and we defined them as the reasoning of the actors in response to the resources or opportunities and changes in the context introduced by the intervention (Pawson and Tilley, 1997). We used a retroductive mode of inference in which the researcher starts from the events observed to postulate the conditions without which those events cannot exist (Meyer and Lunnay, 2013; Robert et al., 2017; Mukumbang et al., 2018). At this stage, the results from the qualitative data analysis were combined with quantitative data from the costing study and the assessment of the policy outcomes. Once the in-case analysis was finished, we conducted a cross-case analysis to compare the configurations. This allowed us to challenge the initial programme theory, look for alternative explanations and refine the PT, using the ‘If …, then …, because … ’ statements (Van Belle, 2014). The Benin National Committee for Health Ethics (ref. 0792/MS/DC/SGM/DFRS/SRAO/SA) in 2012, and the respective research ethics committee of the authors’ institutes approved this study. We obtained a written authorization to conduct the study from the Ministry of Health of Benin but also written consent from the managers of each unit of analysis. All the interviewees signed an informed consent form before the interviews. All the materials are stored for confidentiality and the findings are reported anonymously.

Based on the provided information, here are some potential innovations that could be considered to improve access to maternal health:

1. Mobile Health (mHealth) Solutions: Develop and implement mobile health applications or platforms that provide pregnant women with access to information, reminders, and support throughout their pregnancy journey. This could include features such as appointment reminders, educational resources, and communication channels with healthcare providers.

2. Telemedicine Services: Establish telemedicine services that allow pregnant women in remote or underserved areas to consult with healthcare professionals remotely. This could involve video consultations, remote monitoring of vital signs, and electronic medical record sharing.

3. Community Health Worker Programs: Expand and strengthen community health worker programs to provide maternal health education, support, and referrals in local communities. Community health workers can play a crucial role in reaching pregnant women who may have limited access to healthcare facilities.

4. Transportation Support: Develop transportation support programs to address the challenge of accessing healthcare facilities, particularly in rural or remote areas. This could involve providing transportation vouchers, organizing community-based transportation services, or partnering with existing transportation providers.

5. Financial Assistance Programs: Implement financial assistance programs to reduce or eliminate the financial barriers associated with maternal healthcare services. This could include providing subsidies for prenatal care, delivery, and postnatal care, as well as covering transportation costs and other related expenses.

6. Quality Improvement Initiatives: Implement quality improvement initiatives in healthcare facilities to ensure that maternal health services are delivered in a timely, safe, and respectful manner. This could involve training healthcare providers, improving infrastructure and equipment, and implementing standardized protocols and guidelines.

7. Health Information Systems: Strengthen health information systems to improve data collection, analysis, and monitoring of maternal health indicators. This can help identify gaps in access to care, track progress, and inform evidence-based decision-making.

8. Public Awareness Campaigns: Launch public awareness campaigns to educate communities about the importance of maternal health and the available services. This could involve media campaigns, community events, and partnerships with local influencers and organizations.

9. Partnerships and Collaboration: Foster partnerships and collaboration between government agencies, healthcare providers, non-governmental organizations, and other stakeholders to leverage resources, expertise, and networks to improve access to maternal health services.

10. Research and Evaluation: Conduct research and evaluation studies to assess the effectiveness of different interventions and innovations in improving access to maternal health. This can help identify best practices, inform policy decisions, and guide future investments.

It is important to note that the specific context and needs of Benin should be taken into consideration when implementing these innovations.
AI Innovations Description
The recommendation to improve access to maternal health based on the study’s findings is to focus on addressing the following factors:

1. Trust: Building trust between healthcare providers and patients is crucial for successful implementation of user fee exemption policies. This can be achieved through transparent communication, accountability mechanisms, and ensuring that healthcare providers are perceived as trustworthy.

2. Perceived Coercion: Policies that rely on coercion and sanctions may lead to resistance and non-compliance. It is important to consider the perceptions of healthcare providers and ensure that they feel empowered rather than coerced to implement the policy.

3. Adherence to Policy Goals: Ensuring that healthcare providers understand and align with the goals of the policy is essential. This can be achieved through training, education, and ongoing support to help healthcare providers see the value and importance of the policy.

4. Perceived Financial Incentives: Financial incentives can play a role in motivating healthcare providers to comply with the policy. However, it is important that these incentives are perceived as fair and properly allocated to avoid any negative perceptions or unintended consequences.

5. Fairness in Allocation of Incentives: Fairness in the allocation of incentives is crucial to ensure that healthcare providers are motivated to comply with the policy. This can be achieved through transparent and equitable distribution mechanisms.

6. Enforcement: Effective enforcement of the policy by hierarchical authorities and bottom-up pressure is necessary to ensure compliance. This requires clear guidelines, monitoring mechanisms, and accountability measures.

7. Persuasion: Persuasion plays a role in policy implementation when healthcare providers perceive an alignment between their personal beliefs and the policy goals. It is important to communicate the rationale and benefits of the policy to healthcare providers to gain their support and cooperation.

8. Anticipating Differential Responses: Understanding the different responses and perspectives of implementers is crucial for successful policy implementation. Anticipating these responses and tailoring strategies accordingly can help address implementation gaps and ensure better outcomes.

By addressing these factors, policymakers and healthcare managers can enhance the implementation of user fee exemption policies and improve access to maternal health services.
AI Innovations Methodology
Based on the information provided, the study aimed to understand the implementation of a user fee exemption policy for caesarean sections in Benin and identify the factors that contributed to its success or failure. The study adopted a realist evaluation approach, which focuses on understanding the underlying causal mechanisms that explain how and why interventions lead to specific outcomes in different contexts.

To simulate the impact of recommendations on improving access to maternal health, a methodology could be developed based on the following steps:

1. Identify the specific recommendations: Based on the findings of the study and the identified causal mechanisms, specific recommendations can be formulated to improve access to maternal health services. These recommendations should address the factors that were found to be influential in the implementation of the user fee exemption policy.

2. Define the simulation model: Develop a simulation model that represents the complex system of maternal health services in Benin. The model should include key variables and relationships that affect access to maternal health, such as healthcare facilities, healthcare providers, financial resources, policy implementation mechanisms, and patient characteristics.

3. Specify the intervention scenarios: Define different scenarios that represent the implementation of the identified recommendations. Each scenario should describe how the recommendations would be implemented and the expected changes in the system.

4. Collect data: Gather data on relevant variables and parameters for the simulation model. This may include data on healthcare utilization, financial flows, policy implementation processes, and patient outcomes. Data can be collected through surveys, interviews, and analysis of existing datasets.

5. Calibrate the model: Adjust the parameters of the simulation model based on the collected data to ensure that the model accurately represents the current state of the system.

6. Simulate the scenarios: Run the simulation model using the defined intervention scenarios. This will allow for the exploration of the potential impact of the recommendations on improving access to maternal health services. The simulation can generate quantitative outputs, such as changes in the number of caesarean sections performed, reductions in out-of-pocket expenses, or improvements in maternal health outcomes.

7. Analyze the results: Evaluate the simulation results to assess the potential impact of the recommendations on improving access to maternal health services. Compare the outcomes of different scenarios to identify the most effective interventions.

8. Refine the recommendations: Based on the simulation results, refine the recommendations to optimize their potential impact on improving access to maternal health services. Consider the feasibility, cost-effectiveness, and sustainability of the recommendations in the context of Benin.

9. Communicate the findings: Share the findings of the simulation study with relevant stakeholders, including policymakers, healthcare providers, and community members. Use the results to advocate for the implementation of evidence-based recommendations to improve access to maternal health services.

By following this methodology, policymakers and researchers can gain insights into the potential impact of specific recommendations on improving access to maternal health services in Benin. This approach allows for a systematic exploration of different intervention scenarios and can inform evidence-based decision-making for maternal health policy and practice.

Yabelana ngalokhu:
Facebook
Twitter
LinkedIn
WhatsApp
Email