Performance-based financing empowers health workers delivering prevention of vertical transmission of HIV services and decreases desire to leave in Mozambique

Background: Despite increased access to treatment and reduced incidence, vertical transmission of HIV continues to pose a risk to maternal and child health in sub-Saharan Africa. Performance-based financing (PBF) directed at healthcare providers has shown potential to improve quantity and quality of maternal and child health services. However, the ways in which these PBF initiatives lead to improved service delivery are still under investigation. Methods: Therefore, we implemented a longitudinal-controlled proof-of-concept PBF intervention at health facilities and with community-based associations focused on preventing vertical transmission of HIV (PVT) in rural Mozambique. We hypothesized that PBF would increase worker motivation and other aspects of the workplace environment in order to achieve service delivery goals. In this paper, we present two objectives from the PBF intervention with public health facilities (n = 6): first, we describe the implementation of the PBF intervention and second, we assess the impact of the PBF on health worker motivation, key factors in the workplace environment, health worker satisfaction, and thoughts of leaving. Implementation (objective 1) was evaluated through quantitative service delivery data and multiple forms of qualitative data (eg, quarterly meetings, participant observation (n = 120), exit interviews (n = 11)). The impact of PBF on intermediary constructs (objective 2) was evaluated using these qualitative data and quantitative surveys of health workers (n = 83) at intervention baseline, midline, and endline. Results: We found that implementation was challenged by administrative barriers, delayed disbursement of incentives, and poor timing of evaluation relative to incentive disbursement (objective 1). Although we did not find an impact on the motivation constructs measured, PBF increased collegial support and worker empowerment, and, in a time of transitioning implementing partners, decreased against desire to leave (objective 2). Conclusion: Areas for future research include incentivizing meaningful quality- and process-based performance indicators and evaluating how PBF affects the pathway to service delivery, including interactions between motivation and workplace environment factors. This research was conducted from July 2012-August 2014 in two districts in northern Inhambane province, Mozambique. At that time in Mozambique, the HIV prevalence among women of reproductive age was the fifth-highest globally at 16%, women dropped out at each step of the cascade of PVT services, and 12% of children born to women living with HIV/AIDS contracted the infection through vertical transmission.40,41 Furthermore, Mozambique continues to experience a severe shortage of health workers,42 which is problematic for PVT because maternal and child health nurses and midwives provide the majority of these services. During the period of formative work and intervention design (July 2012-July 2013),37 CARE International provided funding and technical support for HIV prevention and treatment for the districts as the PEPFAR-implementing partner. During intervention implementation (August 2013-July 2014), the Center for Collaboration for Health, a Mozambican NGO new to these districts, transitioned into the role of the PEPFAR-implementing partner. The intervention district was selected based on perceived readiness by CARE and interest of the District Health Authority. Three health facilities (one large health center in the district capital, one large health center peripheral to the district capital, one small health center) were selected for implementation of the PBF intervention (Table S1, see Supplementary file 1). The comparison district was matched based on geographic and administrative similarities, and three facilities in this district were roughly matched for similarities in their catchment areas and workforce. Community volunteer associations in the two districts were also engaged in the PBF intervention,43 but the results are beyond the scope of this paper. Our participatory approach to intervention design began with an 8-month formative assessment of the barriers that health workers experience in delivering PVT services and the appropriateness of PBF to address them.37 Following the formative assessment, repeated meetings with intervention health centers, district authorities, and health system implementing partners were held to discuss, debate, and design the intervention. The intervention district’s senior health administrator also visited another district in Mozambique with an on-going performance-based financing intervention to learn about challenges and facilitators to implementation. The five indicators that were chosen represented all women and women living with HIV only and spanned the PVT service cascade: number of all women (1) and women living with HIV (2) attending first antenatal care visit; number of all women (3) and women living with HIV (4) delivering at health facility; and number of children exposed to HIV attending child preventative health visits (5). These indicators were suggested by the research team in design meetings with intervention district health authority and facility leaders because they are indicators used for facility- and district-level goals and reporting. The indicators were were agreed to by facility leaders, health workers, and district administrative as they aligned with current focus. Facility-specific goals for each indicator were set by taking the monthly mean for each indicator for the previous year and then increasing it by 10%. Setting goals as a proportion of baseline measures builds on existing capacity and has been used in other results-based financing initiatives.2 This approach created challenging but realistic goals that allowed scaling for facilities with differently-sized patient populations. The total budget for facility-based incentives for the three intervention facilities was US$18 000 or $4500/quarter. The quarterly budget was approximately ¼ of each worker’s monthly salary, so that over the year of implementation the maximum total financial incentive would be equivalent to one month’s salary for all health workers. This quarterly budget was allocated to the three intervention facilities in proportion to the number of workers and their respective salaries at the time of the intervention planning phase (Table 1). For reference, a nurse’s monthly salary at the time was US$350. Abbreviation: PBF, Performance-based financing. An additional 5% of total health facility incentives (US$900) was allocated as incentives to engage the District Health Authority in supporting health facilities. These incentives would be awarded based upon the mean percent of goals achieved each quarter across the facilities. The five indicators were assessed quarterly in both the intervention and comparison districts, with the proportion of goals achieved based on the monthly mean for that quarter. Service delivery data were initially planned to be collected at the worker and patient levels. However, resource limitations and concerns about data completion precluded this approach and service delivery was reported at the facility level, with n = 6 health facilities. The research team calculated proportion of goal achieved for each quarter and then held a meeting with each facility to review quarterly performance and challenges encountered. At intervention facilities, the amount of incentives earned and priorities for spending the earned incentives were also discussed. Zero to 100% of the PBF funds were awarded based on quarterly goal achievement. Funds were to be disbursed to the District Health Authority using processes modified from CARE’s existing PEPFAR mechanisms. In order to receive the incentives, each facility was to prepare a solicitation of funds to the district and then the district administrator would generate a bank check. A justification of expenditures was required before the next quarterly transfer could occur. Each facility created a committee with the autonomy to decide how to use the incentives earned by the facility each quarter. During intervention development, committees expressed interest in splitting earned incentives evenly between individuals and the facility. Of note, committees decided that all facility workers irrespective of job description were eligible to receive personal incentives because all staff, from the lead physician to janitor, contributed to creating a positive environment that would help retain patients in the cascade of PVT services. The research team sought to streamline data collection by using the existing data collection processes; the Center for Health Collaboration (the PEPFAR-implementing partner) independently checked original paper records against monthly reports submitted by health facility (Figure 2). Service delivery data were extracted from these reports, entered into Excel 2011 (Microsoft, Seattle, WA), and imported into Stata v14 (StataCorp LP, College Station, TX) for analyses. Data Collection Timeline for the PBF Intervention Implementation and Evaluation. Abbreviation: PBF, performance-based financing. Instruments. Motivation, workplace environment factors, and thoughts of leaving were evaluated longitudinally using a survey at intervention baseline, midline, and endline (Figure 1). Constructs from our formative work were compared to two known tools to capture health worker motivation: a questionnaire for community health workers used in Haiti and Zimbabwe44-46 and a tool developed to measure motivation among hospital workers in Kenya.47 The community health worker questionnaire used by Mbuya and colleagues captured the vast majority of constructs from our formative work and was thus selected as the basis for our survey.44 We modified survey language to more clearly reflect the facility-based health cadres being interviewed and the deletion of three questions that referred to home-based care (Supplementary Material 1). The survey questions captured the constructs in our conceptual framework (Figure 1; 87 questions) as well additional constructs to monitor unintended consequences on time spent (11 questions), workload (5 questions), and other key aspects of workplace environment (eg, training, 3 questions; PVT knowledge, 15 questions). Knowledge questions were asked at baseline and endline only, and those on goals and incentives (11 questions) were asked at endline only. Questions on incentives were only asked of workers at intervention facilities. Response options were given on a 1-5 Likert-type scale (eg, never, rarely, sometimes, usually, always). The survey was adapted to be specific to each facility worker type (eg, maternal and child health nurses, other clinicians, non-clinicians), translated into Portuguese, and back-translated into English. Experts in the local language Xitswa helped standardize the best oral translations from written Portuguese. Research assistants who conducted the surveys were native or fluent speakers in Portuguese and Xitswa and had prior experience in research or as community HIV counselors. Surveys were piloted with health workers in a similar, neighboring district not involved in the intervention and then finalized. Participants. We invited all workers at the small peripheral facilities and 63% of workers at the other facilities (all maternal and child health nurses and midwives, representatives of other departments who were chosen based on departmental seniority). Only five health workers that were successfully contacted declined to participate, citing time constraints. The majority of participants filled out the surveys independently and subsequently reviewed their completed survey with a member of the research team for quality control. Others were asked the survey questions by the enumerator, Participant observation. Ethnographic observation occurred at longitudinal planning meetings, quarterly performance review meetings, regular phone contact, and unstructured observation at health facilities (Figure 2). These were recorded in detailed handwritten notes in Portuguese and English. Notes were typed and compiled into an electronic notebook daily. Exit interviews. Key stakeholders from both districts (eg, district health administrators, facility leaders, health workers) were invited to participate in semi-structured interviews at the end of the intervention. Participants were asked about their perceptions of the intervention in discrete steps, from design through incentive disbursement. Questions probed facilitators, challenges, and suggested improvements for the PBF intervention design and implementation (Supplementary Material 2). Interviews were conducted in Portuguese and took 55-75 minutes to complete. The non-parametric Wilcoxon-Mann-Whitney was used to assess differences in goal attainment status (yes, partial, no) for total proportion of goals achieved and survey questions only asked at endline. The Wilcoxon-Mann-Whitney was also used to compare demographic characteristics among survey and exit interview participants. Intermediary constructs: Quantitative analysis. We used factor analysis with an oblique rotation for correlated factors to reduce survey questions into salient constructs, preserving factor loadings >0.4. We made exceptions for factor loadings >0.3 if the item was strongly conceptually related to the construct. We checked the internal consistency of factors using Chronbach α >0.7 for all workers together and by intervention and comparison status, and made four exceptions for factors with Chronbach α >0.55 for all workers together in cases where the factors were strongly aligned with findings from our formative work (Table S2, see Supplementary file 1). Chronbach α can underestimate reliability and is thus considered the “lower bound” of reliability if multiple traits underlie scale items48 and cannot be considered an absolute confirmation of scale reliability. Composite scores for multi-item constructs were created by summing item values and dividing by the number of items. Separate mixed effects models were used to test the impact of PBF on each of motivation, workplace environment, job satisfaction, and desire to leave constructs (objective 2) at the level of the health worker. Treatment (PBF), time, and the treatment*time interaction were modeled as fixed effects, the health facility as a random effect, and the construct as the outcome. Linear contrasts were used to test differences in construct Likert-type scores between intervention and comparison facilities between baseline and endline (objective 2). Post-hoc power analysis. The sample size for the quantitative survey analysis was limited by the number of health workers at each clinic. For analysis of the psychometric variables measured on the Likert scale of 1-5, we assumed an intra-class correlation of 0.05 since workers at the same facilities may be similarly influenced by facility-level factors. Using the baseline number of health workers (intervention n = 37 and comparison n = 27), we calculated a design effect of 1.57 for intervention workers and 1.40 for comparison workers. This lead to a 60% power to detect a large effect size of 0.7. Audio recordings of meetings were used to augment handwritten notes. Audio recordings of the exit interviews were transcribed in Portuguese and translated into English. Two members of the research team independently coded the exit interviews using thematic analysis according to the principles of frequency, universality, differentiation, and emphasis.49