Menu
Menu
Menu
Menu
To improve access to maternal health services, Benin introduced in 2009 a user fee exemption policy for caesarean sections. Similar to other low- and middle-income countries, its implementation showed mixed results. Our study aimed at understanding why and in which circumstances the implementation of this policy in hospitals succeeded or failed. We adopted the realist evaluation approach and tested the initial programme theory through a multiple embedded case study design. We selected two hospitals with contrastive outcomes. We used data from 52 semi-structured interviews, a patient exit survey, a costing study of caesarean section and an analysis of financial flows. In the analysis, we used the intervention-context-actor-mechanism-outcome configuration heuristic. We identified two main causal pathways. First, in the state-owned hospital, which has a public-oriented but administrative management system, and where citizens demand accountability through various channels, the implementation process was effective. In the non-state-owned hospital, managers were guided by organizational financial interests more than by the inherent social value of the policy, there was a perceived lack of enforcement and the implementation was poor. We found that trust, perceived coercion, adherence to policy goals, perceived financial incentives and fairness in their allocation drive compliance, persuasion, positive responses to incentives and self-efficacy at the operational level to generate the policy implementation outcomes. Compliance with the policy depended on enforcement by hierarchical authority and bottom-up pressure. Persuasion depended on the alignment of the policy with personal and organizational values. Incentives may determine the adoption if they influence the local stakeholder’s revenue are trustworthy and perceived as fairly allocated. Failure to anticipate the differential responses of implementers will prevent the proper implementation of user fee exemption policies and similar universal health coverage reforms.
We adopted the realist evaluation approach, which is a particular form of theory-driven evaluation (Stern et al., 2012), is rooted in realist philosophy of science. At its centre is the notion of generative causation (Pawson and Tilley, 1997; Wong et al., 2016) which makes it particularly adapted to study complex problems (Westhorp, 2013; Bamberger et al., 2016), including in health policy and system research (Gilson, 2012; Adams et al., 2016). The variable implementation of the user fee exemption policy for caesarean section across hospitals is best considered as a complex problem. Indeed, policies are implemented in health systems which are best understood as open systems. Policy implementation is likely to be influenced by existing and new health policies but also by other political, economic and social events. Many actors are involved, from various positions and with specific interests and ideas about the policy (Van Belle et al., 2017). All these point to the complex nature of the causal processes underlying the outcomes of the policy. Realist approach demands the researcher to make explicit the causal mechanisms that explain the observed outcomes and its variation (McLaughlin, 1987; Pawson and Tilley, 1997; Pawson, 2004; Wong et al., 2016). Realist evaluation indeed considers that it is not the policy, but the actors who are engaged in it who bring about the results, situated as they are in specific contexts. Policies work (or not) because actors make particular decisions in response to the resources or opportunities that the policy provides. Realist evaluators thus aim at identifying the underlying ‘reasoning and resources’ (Pawson and Tilley, 1997, p. 68) or generative mechanisms that explain how the outcomes were brought about and in which conditions (Pawson and Tilley, 1997). As such, realist evaluation fits policy implementation research well: it allows for exploring not only the policy formulation and priority-setting process, but also—like in our research question—how and why actors, within the organizational setting and in the broader societal context, implement policies (or not; Marchal et al., 2018). In the following paragraphs, we present the development of the initial programme theory, the study design used to test it, and the methods to collect, to analyse the data and to refine the initial programme theory. Realist evaluation starts with eliciting the initial programme theory of the intervention in question (Marchal et al., 2012). The initial programme theory can be considered as the hypothesis that posits how and why a particular intervention would lead to specific outcomes, for whom and in which conditions. Often, the initial programme theory is based on the results of literature reviews. We started with the berry-picking method to explore the field, followed by a non-systematic review of the policy implementation literature. Berry-picking approach is appropriate for identifying elements of the initial programme theory (Finfgeld-Connett and Johnson, 2013). Like realist synthesis (Pawson et al., 2005), the berry-picking method is an iterative, adaptive, creative and flexible process that can contribute to generate a theory that explains a social phenomenon by consolidating or federating fragmented pieces of the literature (Booth, 2008). The first step is a literature search, followed by the selection of a first set of articles relevant to the research question. Analysis of the selected publications leads to the reformulation of the research question, which in turn is followed by a second search and so forth until the synthesis of the results adequately answers the research questions (Booth, 2008). First, we searched for frameworks, models or theories that could inform our initial programme theory. We reviewed papers dealing with the implementation of user fee exemption policies for maternal healthcare and healthcare in general. We found that most publications describe policy implementation gaps and factors that may explain implementation failure, like inadequate inputs and poor communication (Witter and Adjei, 2007; Agyepong and Nagai, 2011). Few studies started from an explicit theory or theoretical framework on policy adoption or implementation. Few authors attempted to explain the success or failure of the policy implementation by using theories or models from political science (public administration and policy analysis). Authors like Ridde and Diarra (2009) and Walker and Gilson (2004) used the concept of street-level bureaucracy (Lipsky, 1980) to explain the implementation gap of user fee exemption policies. However, none of the papers referred to the broader policy implementation literature. The scant results of the first review led us to review the policy implementation literature. Given the limited consensus on policy implementation theories (O’Toole, 2004; DeGroff and Cargo, 2009), a systematic review was not likely to be useful. We, therefore, carried out a non-systematic review of the published and grey literature. We started the search using the Web of Science and Social Sciences Citation Index search engines with sets of keywords combining ‘policy’, ‘program’ and ‘implementation’ or ‘adoption’. We used extensive snowballing to track down the original papers and books from the bibliographic references of the initial list of papers and books. This review identified a first set of three publications. We selected the paper of Berman on macro- and micro-implementation of social policy (Berman, 1978) because it presents a comprehensive model, going from policy design to local implementation and back, and it emphasizes the multi-level nature of policy implementation. Second, we selected Elmore’s forward and backward mapping model (Elmore, 1982). This model identifies local agents as key actors in policy implementation and provides an advanced framework for successful ‘planning’ of policy implementation, from the first choices to the expected outcomes (forward mapping) and back (backward mapping). Third, we added the ambiguity-conflict model of policy implementation of Matland, because it represents a theory explaining implementation gaps that goes beyond the top-down and bottom-up divide (Matland, 1995). At this stage, we extracted from those papers, the contents related to intervention (in casu policy), context, actors, mechanisms and outcomes, using the Intervention-Context-Actor-Mechanism-Outcome (ICAMO) heuristic to identify the configurations that explain the reported outcomes at each level (Van Belle, 2014; Marchal et al., 2018). ICAMO is a modified version of the context-mechanism-outcome configuration (Pawson and Tilley, 1997) that stimulates a proper description of the actual intervention and ‘the actors through whom the intervention works’ (Mukumbang et al., 2018). This process pointed us to three ‘passages’ or implementation phases (Berman, 1978, p. 13) that provided the overall multilevel structure of our initial programme theory: Phase 1: This phase includes the actual decision-making process and the institution of a programme to implement the policy. It involves macro-level key stakeholders, including the ministers’ council, and results in making the decisions for the policy and its financing. Next, an administration or government agency is set up to develop a programme to implement the policy decision. The more ambiguous the policy intent, the more latitude such agency has in shaping the policy. Phase 2: This phase encompasses the transition from programme to local adoption at operational health service level, during which slippage between programme guidelines and actual implementation can occur. This is partly determined by the degree of compliance with decisions by higher authorities (and thus the enforcement capacity) and alignment of the policy goals with local needs. Programmes are actually or symbolically adopted by district health authorities and hospital managers. Phase 3: This phase concerns the implementation of the programme by service providers, who act as street-level bureaucrats. Berman calls this ‘micro-implementation’ and considers this ‘may be the most pivotal step because a social policy’s outcome depends on local delivery’ (Berman, 1978, p. 21). Service providers can implement the policy in four ways: (1) non-implementation, (2) co-optation (or adaptation of the programme to fit existing practices), (3) technological learning (adaptation of routine practices to accommodate the policy) and (4) mutual adaptation, whereby both policy and service delivery are adapted to ensure optimal fit. In previous studies, Berman found that only mutual adaptation leads to achievement of the intended policy outcomes: the policy is adapted to the organization, its staff, its target public and its environment, and organizational changes are made to better implement the policy. If the three papers provided an overarching architecture for a multi-level analysis of policy implementation, they did not present mechanisms. Our berry-picking approach led to the ‘carrots, sticks and sermons’ typology of policy instruments (Bemelmans-Videc and Vedung, 1998). Carrots, sticks and sermons can be considered as a parsimonious typology of mechanisms underlying policy instruments and more specifically as categories of drivers of individual or organizational commitment to the policy. Policies that include incentives (‘carrots’) may induce a positive response when actors perceive a financial or other gain, or if any financial losses due to the policy are compensated properly (net benefit or no net loss). Policies that are based on coercion and sanctions (‘sticks’) may induce perceived pressure and fear of sanctions that in turn may generate compliance. Policies based on persuasion (‘sermons’) trigger good implementation if the actors perceive an alignment between personal believes and policy goals and values. Figure 1 presents our initial programme theory. Initial programme theory. We adopted the multiple embedded case study design (Yin, 2009), which is well adapted to research on policy implementation gaps. We defined ‘the policy implementation’ as the case. We considered a hospital implementing the user fee exemption policy for caesarean section as the unit of analysis. Two units of analysis were purposively selected among the seven implementing hospitals covered by the FEMHealth research programme in Benin—a faith-based (non-state-owned) hospital and a public (state-owned) hospital—because they present contrasts in terms of ownership, location and policy implementation outcomes (Table 1). A detailed description of the sampling procedure of FEMHealth is reported elsewhere (FEMHealth, 2014). General characteristic of the two units of analysis in 2011 Source: Administrative records of the hospitals and Biaou et al. (2015). Realist evaluation is method-neutral: the data collection methods should allow to gather data needed to test the initial programme theory. We collected both quantitative and qualitative data to describe the case and its units of analysis. Table 2 presents the data sources, tools and their specific purposes. A multi-disciplinary team (a medical doctor, a health economist and a social scientist) collected the data between March 2012 and March 2013. Data collection techniques and data sources and study sample in the two study sites We transcribed verbatim the recordings of the interviews and entered the transcripts in QSR NVIVO 10, converted later to QSR NVIVO 12. In a first phase, we adopted thematic analysis principles (Miles and Huberman, 1984), whereby the initial coding was based on the key elements of the initial programme theory, which evolved during the analytical process. We used Microsoft Excel 2011 (version 14.6.7) to do a univariate descriptive analysis of the quantitative data from the exit survey and from the financial flow tracking study. Table 2 presents the analytical approach used for each type of data. In a second phase, we triangulated qualitative and quantitative data to develop a configurational analysis of each case using the ICAMO heuristic tool (Van Belle, 2014; Marchal et al., 2018). We described the intervention (in casu policy), context, actors, mechanisms and outcomes, and identified the configurations that explain the reported outcomes at each level. We mapped the causal configuration looking for causal links between the actually implemented intervention, relevant contextual elements, the mechanisms that were triggered for specific groups of actors and the observed outcomes. Mechanisms are key components in this heuristic and we defined them as the reasoning of the actors in response to the resources or opportunities and changes in the context introduced by the intervention (Pawson and Tilley, 1997). We used a retroductive mode of inference in which the researcher starts from the events observed to postulate the conditions without which those events cannot exist (Meyer and Lunnay, 2013; Robert et al., 2017; Mukumbang et al., 2018). At this stage, the results from the qualitative data analysis were combined with quantitative data from the costing study and the assessment of the policy outcomes. Once the in-case analysis was finished, we conducted a cross-case analysis to compare the configurations. This allowed us to challenge the initial programme theory, look for alternative explanations and refine the PT, using the ‘If …, then …, because … ’ statements (Van Belle, 2014). The Benin National Committee for Health Ethics (ref. 0792/MS/DC/SGM/DFRS/SRAO/SA) in 2012, and the respective research ethics committee of the authors’ institutes approved this study. We obtained a written authorization to conduct the study from the Ministry of Health of Benin but also written consent from the managers of each unit of analysis. All the interviewees signed an informed consent form before the interviews. All the materials are stored for confidentiality and the findings are reported anonymously.
N/A
