Implementing a social accountability approach for maternal, neonatal, and child health service performances in Ethiopia: A pre-post study design

Global Health Science and Practice, Volume 9, No. 1, Year 2021

Interprétation

Background: Social accountability approaches, such as the community scorecard (CSC), can improve the performance of health systems in low-income countries by providing a mechanism for obtaining and incorporating community input. This longitudinal study assessed the effects of CSCs implemented by primary health care units (PHCUs) on health system performance in Ethiopia. Methods: This study used a pre-post design and was conducted from October 2018 to September 2019 in 159 PHCUs in 31 districts in Amhara and Southern, Nations, Nationalities and Peoples’ regional states. The data were extracted from a routine health information management system database at baseline, midterm, and endline stages over 12 months for statistical analysis. The effects of implementing CSCs on health system performance were evaluated against selected key performance indicators (KPIs). Results: The CSC measurement results were based on input from 38,556 community representatives. The mean CSC score with standard deviation (6SD) was 60.8%612.5%, 66.3%610.8%, and 70.6%610.0% at baseline, midterm, and endline, respectively. The mean KPI score was 54.9%617.4%, 61.9%615.1%, and 67.6%614.6% at baseline, midterm, and endline, respectively. The average CSC and KPI values were positively correlated (r>0.37). Using a nonparametric Friedman’s test, we found a statistically significant difference in CSC and KPI scores at baseline, midterm, and endline (P=.001). Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction and the results showed higher CSC and KPI values from baseline to midterm and from midterm to endline (P<.017). Conclusions: The use of CSCs in Ethiopia contributed to the health system’s performance in terms of maternal and child health services. The responsiveness of health workers and utilization of basic health services by community members were found to increase significantly with CSC use. We recommend continued implementation of the CSC intervention at PHCUs. This study was conducted in South Wollo and Kembata Tembaro administrative zones of Amhara and SNNP regional states, respectively. The USAID Transform: Primary Health Care project provides technical, financial, and other resource support to 91 districts in Amhara and 84 in SNNP, as well as to districts in other regional states.20 The project supported 31 districts and 159 PHCUs to start and implement the CSC intervention for over 12 months as a social accountability tool for performance management.21 Ethiopia has a 3-tier health care delivery system. Level 1 is the district (woreda) level, composed of primary hospitals that cover 60,000–100,000 people, health centers serving 15,000–25,000 people, and their satellite health posts covering 3,000–5,000 people, connected to each other through a referral system. The primary hospitals, health centers, and health posts form PHCUs. Districts are subdivided into kebeles (villages), the lowest administrative units. Level 2 includes general hospitals covering 1–1.5 million people. Level 3 includes specialized hospitals covering 3.5–5 million people.16 In the last 3 decades, the country has expanded access to primary health care through 17,187 health posts; 7,245 private health facilities; 3,724 health centers; and 266 hospitals. In addition, more than 151,053 health professionals are serving communities.20 To institutionalize accountability and transparency as a tool for performance management in the health system in both Amhara and SNNP regional states of Ethiopia, 632 participants attended the CSC training of trainers, including 155 from district health offices and 477 from PHCUs. A 3-day classroom theoretical orientation and practical sessions were facilitated in July 2018 at the respective capital cities of the regional states. The CSC orientations were given to 4,053 client councils within the targeted 159 health centers in South Wollo and Kembata Tembaro administrative zones in August 2018.21 The client councils were informed of the 6 standards of the CSC: (1) compassionate, respectful, and caring health workforce; (2) patient waiting time; (3) availability of services, biomedical equipment, and pharmaceutical supplies; (4) health facility infrastructure; (5) ambulance service and management; and (6) clean and safe health facility, citizens' rights, service providers' duties, facilitation techniques, counting and organizing scores, observation skills, verification tools, report submission, and provision of feedback. In addition, every 3 months, the client councils were actively engaged in facility-community interface meetings, presented the results of the CSCs and feedback of community members, addressed issues raised by town hall meeting participants, and closely monitored the implementation of developed action plans. The project supported all client councils with 1-page job aids, reporting forms, and a minute book. From September 2018 to December 2019, USAID Transform: Primary Health Care provided technical, financial, and other resource support to its targeted districts. Some forms of this support were providing team-based strategic problem-solving trainings for health care providers with the formation of performance improvement and quality improvement projects, enhancing the capacity of performance management team members through offering the use of data for decision-making trainings, providing performance improvement subgrant funding for primary health care entities, and organizing and facilitating community-facility interface meetings. In addition, the project's staff and experts from zone health departments provided follow-up visits and on-site coaching for all PHCUs and district health offices on a quarterly basis. During facility visits, the coaching team facilitated the exploration of CSC measurements with feedback and gave opportunities for PHCU management staff to systematically analyze the root causes of any issues, propose prioritized solutions, and develop doable action plans. In addition, the coaches revised the concepts of social accountability, provided feedback on performance, and supported the client councils and health care providers. The coaches then submitted a copy of agreed measurement reports, identified gaps, and developed action plans with district health offices and to zone health departments. Baseline data were extracted from the period of October 10–20, 2018. At 6 and 12 months after the intervention periods, secondary data were collected during April 10–20, 2019, as midterm assessments, and during October 10–22, 2019, as endline measurements by the data collection teams.21 For this study, the investigators used a longitudinal pre-post interventional study design.22 The necessary data were collected from October 2018 to September 2019. We used quantitative methods to measure the effects of CSC implementation on health-seeking behavior, health service utilization, and responsiveness of health care providers. The study participants were identified from the Amhara and SNNP regional states in Ethiopia. Two administrative zones where the project had provided technical and other resource support for 12 months or more were purposively selected. All 31 districts (woredas) and 159 PHCUs were included in selected zone administrations. Documents and records from each primary health care entity were reviewed at the 3 points of the assessment. The required routine health management information system data on CSC and on maternal and child health services were extracted from 159 PHCUs' RHIMS database. To ensure data completeness, accuracy, consistency, and reliability, 8 data managers and 2 supervisors were trained for 3 days. The training covered the objective of the pre-post interventional study, ethical issues, quantitative data extraction methods, and piloting all tools and ethical principles. During the real data collection, all investigators actively monitored completeness and consistency of data on a daily basis. Data were extracted using structured and pretested forms. Dependent variables: The average measures, from 0% to 100% on 10 key performance indicators (KPIs) were reviewed and collated: (1) contraceptive acceptance rate, (2) syphilis screening among antenatal care (ANC) clients, (3) skilled delivery services coverage, (4) postnatal care coverage, (5) full immunization coverage, (6) under 2 years growth monitoring coverage, (7) proportion of available essential or tracer drugs, (8) proportion of clean and safe health facility standards met, (9) proportion of available laboratory and diagnostic services, and (10) patient flow and service organization (Table 1).18,21 The effects of CSC implementation on the health system's performance were measured at baseline, midterm (after 6 months), and endline (after 12 months). Description of Key Performance Indicators and Community Scorecard Scoring Standards to Assess Effects of Implementing a Social Accountability Approach on Improving Health System Performance in Maternal and Child Health Services in 2 Regional States in Ethiopiaa Independent variables: The data extraction forms dedicated to capture information on CSCs were adopted from the nationally endorsed CSC reporting tools. The forms captured the summary of the following 6 standards: (1) compassionate, respectful, and caring health workforce; (2) patient waiting time; (3) availability of services, biomedical equipment, and pharmaceutical supplies; (4) health facility infrastructure; (5) ambulance service and management; and (6) clean and safe health facility, reported as an average score from a 5-point Likert scale measurement (Supplement 1), that is, 5=very good to 1=very low. The overall summary of CSCs was reported as a percentage from 0% to 100.0% on 3 occasions17 (Table 1). The data were collated using a Microsoft Excel sheet and exported to Statistical Program for Social Science (SPSS IBM V 20) software for analysis. To ensure consistency and reliability, the data were double entered by experienced data encoders. The PHCU scores of CSCs and KPIs were reviewed for completeness and consistency. Quantitative data analysis methods were used, which included descriptive statistic frequencies, mean, median, interquartile ranges, and standard deviations. To determine the presence of a linear relationship between baseline, midterm, and endline scores, a Pearson Product-Moment Correlation technique was employed. After checking the assumption of the nonparametric test, we used the Friedman23 test, which includes (1) a single group measured on 3 or more different occasions, (2) a group that is a random sample from population, (3) use of a continuous level of dependent variables, and (4) samples that do not need to be normally distributed. Post hoc analysis with the Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied and a statistical test result with a P-value of <.017, indicating presence of a significant difference between CSCs and KPIs at baseline, midterm, and endline measurements. The ethical clearances of this study were granted by the JSI Institutional Review Board (IRB), and the Amhara Public Health Institute and SNNP Regional State Health Bureaus' IRBs. The research protocol of this pre-post interventional study was granted certification from the Amhara Public Health Institute (reference number: M/T/SH/D/03/435) and the SNNP Regional State Health Bureau Research and Ethics Review Committees (reference number NS 12/36/22). In addition, the study protocol was reviewed at the JSI Research & Training Institute, Inc. IRB, which determined that this activity was exempt from human subjects' oversight (reference number IRB no. 19-16E). To maintain the confidentiality of collected data, anonymity was maintained throughout the research process. Permission to use data from zone health departments and PHCUs were obtained through formal written requests. Informed individual written consent was taken from each study subject. Both quantitative and qualitative data were collected in aggregate forms. All PHCUs were encouraged to use the data for evidence-based decision making and to be responsive to the demands of rights holders. The summary of CSCs and KPIs were submitted to district health offices, zone health departments, and regional health bureaus at the 3 points of the assessment. Throughout the research process, the investigators maintained national and international ethical principles.

Résumé de l’IA

Study Justification:

– Social accountability approaches, such as the community scorecard (CSC), can improve the performance of health systems in low-income countries.
– This study aimed to assess the effects of CSCs implemented by primary health care units (PHCUs) on health system performance in Ethiopia.

Highlights:

– The study used a pre-post design and was conducted in 159 PHCUs in 31 districts in Amhara and Southern, Nations, Nationalities and Peoples’ regional states.
– CSC measurement results were based on input from 38,556 community representatives.
– The mean CSC score increased from 60.8% at baseline to 70.6% at endline.
– The mean key performance indicator (KPI) score increased from 54.9% at baseline to 67.6% at endline.
– The average CSC and KPI values were positively correlated.
– Statistical analysis showed a significant difference in CSC and KPI scores at baseline, midterm, and endline.

Recommendations:

– Continued implementation of the CSC intervention at PHCUs to improve maternal and child health services.
– Institutionalize accountability and transparency as tools for performance management in the health system.

Key Role Players:

– Primary health care units (PHCUs)
– District health offices
– Zone health departments
– Regional health bureaus

Cost Items for Planning Recommendations:

– Training of trainers for CSC implementation
– Technical and resource support from USAID Transform: Primary Health Care project
– Strategic problem-solving trainings for health care providers
– Capacity enhancement for performance management team members
– Performance improvement subgrant funding for primary health care entities
– Community-facility interface meetings
– Follow-up visits and on-site coaching by project staff and experts
– Data managers and supervisors for data collection and monitoring

Force de la preuve

N/A

Abstrait

This study was conducted in South Wollo and Kembata Tembaro administrative zones of Amhara and SNNP regional states, respectively. The USAID Transform: Primary Health Care project provides technical, financial, and other resource support to 91 districts in Amhara and 84 in SNNP, as well as to districts in other regional states.20 The project supported 31 districts and 159 PHCUs to start and implement the CSC intervention for over 12 months as a social accountability tool for performance management.21 Ethiopia has a 3-tier health care delivery system. Level 1 is the district (woreda) level, composed of primary hospitals that cover 60,000–100,000 people, health centers serving 15,000–25,000 people, and their satellite health posts covering 3,000–5,000 people, connected to each other through a referral system. The primary hospitals, health centers, and health posts form PHCUs. Districts are subdivided into kebeles (villages), the lowest administrative units. Level 2 includes general hospitals covering 1–1.5 million people. Level 3 includes specialized hospitals covering 3.5–5 million people.16 In the last 3 decades, the country has expanded access to primary health care through 17,187 health posts; 7,245 private health facilities; 3,724 health centers; and 266 hospitals. In addition, more than 151,053 health professionals are serving communities.20 To institutionalize accountability and transparency as a tool for performance management in the health system in both Amhara and SNNP regional states of Ethiopia, 632 participants attended the CSC training of trainers, including 155 from district health offices and 477 from PHCUs. A 3-day classroom theoretical orientation and practical sessions were facilitated in July 2018 at the respective capital cities of the regional states. The CSC orientations were given to 4,053 client councils within the targeted 159 health centers in South Wollo and Kembata Tembaro administrative zones in August 2018.21 The client councils were informed of the 6 standards of the CSC: (1) compassionate, respectful, and caring health workforce; (2) patient waiting time; (3) availability of services, biomedical equipment, and pharmaceutical supplies; (4) health facility infrastructure; (5) ambulance service and management; and (6) clean and safe health facility, citizens' rights, service providers' duties, facilitation techniques, counting and organizing scores, observation skills, verification tools, report submission, and provision of feedback. In addition, every 3 months, the client councils were actively engaged in facility-community interface meetings, presented the results of the CSCs and feedback of community members, addressed issues raised by town hall meeting participants, and closely monitored the implementation of developed action plans. The project supported all client councils with 1-page job aids, reporting forms, and a minute book. From September 2018 to December 2019, USAID Transform: Primary Health Care provided technical, financial, and other resource support to its targeted districts. Some forms of this support were providing team-based strategic problem-solving trainings for health care providers with the formation of performance improvement and quality improvement projects, enhancing the capacity of performance management team members through offering the use of data for decision-making trainings, providing performance improvement subgrant funding for primary health care entities, and organizing and facilitating community-facility interface meetings. In addition, the project's staff and experts from zone health departments provided follow-up visits and on-site coaching for all PHCUs and district health offices on a quarterly basis. During facility visits, the coaching team facilitated the exploration of CSC measurements with feedback and gave opportunities for PHCU management staff to systematically analyze the root causes of any issues, propose prioritized solutions, and develop doable action plans. In addition, the coaches revised the concepts of social accountability, provided feedback on performance, and supported the client councils and health care providers. The coaches then submitted a copy of agreed measurement reports, identified gaps, and developed action plans with district health offices and to zone health departments. Baseline data were extracted from the period of October 10–20, 2018. At 6 and 12 months after the intervention periods, secondary data were collected during April 10–20, 2019, as midterm assessments, and during October 10–22, 2019, as endline measurements by the data collection teams.21 For this study, the investigators used a longitudinal pre-post interventional study design.22 The necessary data were collected from October 2018 to September 2019. We used quantitative methods to measure the effects of CSC implementation on health-seeking behavior, health service utilization, and responsiveness of health care providers. The study participants were identified from the Amhara and SNNP regional states in Ethiopia. Two administrative zones where the project had provided technical and other resource support for 12 months or more were purposively selected. All 31 districts (woredas) and 159 PHCUs were included in selected zone administrations. Documents and records from each primary health care entity were reviewed at the 3 points of the assessment. The required routine health management information system data on CSC and on maternal and child health services were extracted from 159 PHCUs' RHIMS database. To ensure data completeness, accuracy, consistency, and reliability, 8 data managers and 2 supervisors were trained for 3 days. The training covered the objective of the pre-post interventional study, ethical issues, quantitative data extraction methods, and piloting all tools and ethical principles. During the real data collection, all investigators actively monitored completeness and consistency of data on a daily basis. Data were extracted using structured and pretested forms. Dependent variables: The average measures, from 0% to 100% on 10 key performance indicators (KPIs) were reviewed and collated: (1) contraceptive acceptance rate, (2) syphilis screening among antenatal care (ANC) clients, (3) skilled delivery services coverage, (4) postnatal care coverage, (5) full immunization coverage, (6) under 2 years growth monitoring coverage, (7) proportion of available essential or tracer drugs, (8) proportion of clean and safe health facility standards met, (9) proportion of available laboratory and diagnostic services, and (10) patient flow and service organization (Table 1).18,21 The effects of CSC implementation on the health system's performance were measured at baseline, midterm (after 6 months), and endline (after 12 months). Description of Key Performance Indicators and Community Scorecard Scoring Standards to Assess Effects of Implementing a Social Accountability Approach on Improving Health System Performance in Maternal and Child Health Services in 2 Regional States in Ethiopiaa Independent variables: The data extraction forms dedicated to capture information on CSCs were adopted from the nationally endorsed CSC reporting tools. The forms captured the summary of the following 6 standards: (1) compassionate, respectful, and caring health workforce; (2) patient waiting time; (3) availability of services, biomedical equipment, and pharmaceutical supplies; (4) health facility infrastructure; (5) ambulance service and management; and (6) clean and safe health facility, reported as an average score from a 5-point Likert scale measurement (Supplement 1), that is, 5=very good to 1=very low. The overall summary of CSCs was reported as a percentage from 0% to 100.0% on 3 occasions17 (Table 1). The data were collated using a Microsoft Excel sheet and exported to Statistical Program for Social Science (SPSS IBM V 20) software for analysis. To ensure consistency and reliability, the data were double entered by experienced data encoders. The PHCU scores of CSCs and KPIs were reviewed for completeness and consistency. Quantitative data analysis methods were used, which included descriptive statistic frequencies, mean, median, interquartile ranges, and standard deviations. To determine the presence of a linear relationship between baseline, midterm, and endline scores, a Pearson Product-Moment Correlation technique was employed. After checking the assumption of the nonparametric test, we used the Friedman23 test, which includes (1) a single group measured on 3 or more different occasions, (2) a group that is a random sample from population, (3) use of a continuous level of dependent variables, and (4) samples that do not need to be normally distributed. Post hoc analysis with the Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied and a statistical test result with a P-value of <.017, indicating presence of a significant difference between CSCs and KPIs at baseline, midterm, and endline measurements. The ethical clearances of this study were granted by the JSI Institutional Review Board (IRB), and the Amhara Public Health Institute and SNNP Regional State Health Bureaus' IRBs. The research protocol of this pre-post interventional study was granted certification from the Amhara Public Health Institute (reference number: M/T/SH/D/03/435) and the SNNP Regional State Health Bureau Research and Ethics Review Committees (reference number NS 12/36/22). In addition, the study protocol was reviewed at the JSI Research & Training Institute, Inc. IRB, which determined that this activity was exempt from human subjects' oversight (reference number IRB no. 19-16E). To maintain the confidentiality of collected data, anonymity was maintained throughout the research process. Permission to use data from zone health departments and PHCUs were obtained through formal written requests. Informed individual written consent was taken from each study subject. Both quantitative and qualitative data were collected in aggregate forms. All PHCUs were encouraged to use the data for evidence-based decision making and to be responsive to the demands of rights holders. The summary of CSCs and KPIs were submitted to district health offices, zone health departments, and regional health bureaus at the 3 points of the assessment. Throughout the research process, the investigators maintained national and international ethical principles.

Matériaux

Innovations

The study recommends implementing a social accountability approach, specifically the community scorecard (CSC), to improve access to maternal health services. The CSC intervention involves training participants, establishing client councils, conducting facility-community interface meetings, developing action plans, and providing technical and resource support. The study found that implementing CSCs in primary health care units (PHCUs) in Ethiopia contributed to the performance of the health system in terms of maternal and child health services. The study measured the effects of CSC implementation on key performance indicators (KPIs) and found a significant improvement in CSC scores and KPI scores from baseline to midterm and from midterm to endline. The study recommends the continued implementation of the CSC intervention at PHCUs to improve access to maternal health services and enhance the responsiveness of health workers. By involving the community in the accountability process, the CSC can help address gaps and improve the overall performance of the health system.

AI Innovations Description

The recommendation from the study is to implement a social accountability approach, specifically the community scorecard (CSC), to improve access to maternal health services. The study conducted in Ethiopia found that implementing CSCs in primary health care units (PHCUs) contributed to the performance of the health system in terms of maternal and child health services.

The CSC intervention involved training participants, including district health offices and PHCUs, on the standards of the CSC. Client councils were established and actively engaged in facility-community interface meetings, where they presented CSC results and feedback from community members. Action plans were developed and implemented based on the identified issues. The project also provided technical and resource support, such as team-based trainings, performance improvement projects, and follow-up visits.

The study measured the effects of CSC implementation on key performance indicators (KPIs) related to maternal and child health services. The results showed a significant improvement in CSC scores and KPI scores from baseline to midterm and from midterm to endline. The average CSC and KPI values were positively correlated.

Based on these findings, the study recommends the continued implementation of the CSC intervention at PHCUs to improve access to maternal health services. This approach can enhance the responsiveness of health workers and increase the utilization of basic health services by community members. By involving the community in the accountability process, the CSC can help address gaps and improve the overall performance of the health system.

AI Innovations Methodology

To simulate the impact of implementing the main recommendations of this abstract on improving access to maternal health, you could consider the following methodology:

1. Selection of study sites: Choose a representative sample of primary health care units (PHCUs) in Ethiopia that are similar to the ones mentioned in the study. Ensure that the selected PHCUs are located in areas with similar demographic and health characteristics.

2. Baseline data collection: Collect baseline data on key performance indicators (KPIs) related to maternal health services from the selected PHCUs. This can include indicators such as contraceptive acceptance rate, skilled delivery services coverage, postnatal care coverage, and immunization coverage. Also, collect data on the community scorecard (CSC) scores to assess the initial level of social accountability.

3. Intervention implementation: Implement the CSC intervention at the selected PHCUs following the same approach described in the study. This involves training participants, establishing client councils, conducting facility-community interface meetings, developing action plans, and providing technical and resource support.

4. Data collection during intervention: Collect data on the KPIs and CSC scores at regular intervals during the intervention period. This can be done at midterm (around 6 months) and endline (around 12 months) stages. Ensure that the data collection methods are consistent with the study’s approach.

5. Data analysis: Analyze the collected data to assess the impact of the CSC intervention on the KPIs. Compare the baseline data with the midterm and endline data to identify any changes in the KPI scores. Also, analyze the CSC scores to evaluate the level of social accountability achieved through the intervention.

6. Statistical analysis: Use appropriate statistical tests, such as Friedman’s test and Wilcoxon signed-rank tests, to determine the significance of the changes in KPI scores and CSC scores over time. Calculate correlation coefficients to assess the relationship between CSC scores and KPI scores.

7. Interpretation of results: Interpret the findings of the data analysis to understand the impact of implementing the CSC intervention on improving access to maternal health services. Assess the extent to which the intervention contributed to the performance of the health system in terms of maternal and child health services.

8. Recommendations: Based on the results, make recommendations for the continued implementation of the CSC intervention at PHCUs to improve access to maternal health services. Highlight the potential benefits of involving the community in the accountability process and address any gaps or challenges identified during the simulation.

9. Reporting and dissemination: Prepare a comprehensive report summarizing the methodology, findings, and recommendations of the simulation study. Disseminate the findings to relevant stakeholders, such as policymakers, health practitioners, and community members, to promote evidence-based decision-making and encourage the adoption of social accountability approaches in maternal health services.

By following this methodology, you can simulate the impact of implementing the recommendations from the study and gain insights into how the CSC intervention can improve access to maternal health services in a similar context.

Auteur

Dima Health

Statistics:

Citations: 2

Identifiers:

DOI: 10.9745/GHSP-D-20-00114

Research Areas:

Community Interventions, Health System and Policy, Infectious Diseases, Maternal Access, Maternal and Child Health, Quality of Care, Sexual and Reproductive Health, Social Determinants

Study Countries:

Ethiopia

Study Design:

Cohort Study, Cross Sectional Study, Grounded Theory, randomised-control-trial

Study Approach:

Mixed-methods, Qualitative, Quantitative, Systematic Review

Partagez ceci :