Adaptation and validation of social accountability measures in the context of contraceptive services in Ghana and Tanzania

listen audio

Study Justification:
– Changes in values, attitudes, and interactions of service users and healthcare providers are important in social accountability processes in reproductive health.
– There is a need to measure these changes accurately to understand the effectiveness of social accountability interventions.
– This study aims to adapt and validate measures that capture these changes in the context of contraceptive services in Ghana and Tanzania.
Study Highlights:
– The study used the CaPSAI theory of change to determine the dimensions of the measures.
– Existing survey items were adapted to develop the survey questionnaire.
– Data was collected from 752 women in Tanzania and 750 women in Ghana attending contraceptive services.
– Reliability analysis, exploratory factor analysis, and confirmatory factor analysis were conducted to assess the validity and reliability of the measures.
– The measures showed high construct validity and reliability in both countries.
– Several subscales were identified in each country, indicating the different dimensions of social accountability.
– The study suggests that the measures may be relevant in different settings and should be validated in new settings.
Recommendations:
– The adapted and validated measures should be used to assess social accountability interventions in the context of contraceptive services.
– Further validation of the measures in different settings is recommended.
– The measures can be used to monitor changes in values, attitudes, and interactions of service users and healthcare providers in reproductive health.
Key Role Players:
– National civil society organizations (CSOs) with experience in social accountability.
– Health system authorities and policymakers.
– Community members and health service actors.
– Researchers and data collectors.
Cost Items for Planning Recommendations:
– Training of data collectors.
– Survey administration and data collection.
– Data analysis and interpretation.
– Dissemination of findings.
– Monitoring and evaluation of social accountability interventions.
– Stakeholder engagement and coordination.

The strength of evidence for this abstract is 8 out of 10.
The evidence in the abstract is rated 8 because it provides a detailed description of the methodology used to develop and validate the measures in both Tanzania and Ghana. The study used reliability analysis, exploratory factor analysis, and confirmatory factor analysis to assess the validity and reliability of the measures. The abstract also mentions that the measures have high construct validity and reliability in both countries. However, to improve the evidence, it would be helpful to provide more specific information about the results of the reliability analysis, factor analysis, and confirmatory factor analysis, such as the Cronbach’s alpha values, factor loadings, and fit statistics for the CFA models. This would provide a clearer understanding of the strength of the evidence.

Background: Changes in the values, attitudes, and interactions of both service users and health care providers are central to social accountability processes in reproductive health. However, there is little consensus on how best to measure these latent changes. This paper reports on the adaptation and validation of measures that capture these changes in Tanzania and Ghana. Methods: The CaPSAI theory of change determined the dimensions of the measure, and we adapted existing items for the survey items. Trained data collectors used a survey to collect data from 752 women in Tanzania and 750 women in Ghana attending contraceptive services. We used reliability analysis, exploratory, and confirmatory factor analysis to assess the validity and reliability of these measures in each country. Results: The measure has high construct validity and reliability in both countries. We identified several subscales in both countries, 10 subscales in Tanzania, and 11 subscales in Ghana. Many of the domains and items were shared across both settings. Conclusion: The study suggests that the multi-dimensional scales have high construct validity and reliability in both countries. Though there were differences in the two country contexts and in items and scales, there was convergence in the analysis that suggests that this measure may be relevant in different settings and should be validated in new settings. Trial registration: ACTRN12619000378123.

The larger study’s theory of change about how service users’ attitudes changed during the social accountability process determined the dimensions of the measures. To develop the survey items, we adapted existing items. Trained data collectors used a survey to collect data from women attending contraceptive services in Tanzania and Ghana. We used reliability analysis, exploratory, and confirmatory factor analysis to assess the validity and reliability of these measures in each country. The development of these measures is part of a more extensive complex intervention study, Community and Provider Social Accountability Intervention (CaPSAI), undertaken in Tanzania and Ghana [36]. These countries were selected based on the following criteria: (1) existence of a national civil society organization (CSO) partner with experience in social accountability, (2) low modern contraceptive prevalence rate, (3) availability of contraceptive services at the point of contact where cost is not a barrier to access, (4) an enabling environment in terms of the potential for the health system to respond to the social accountability, and (5) the existence of formal structures linking the community with the health system (e.g. health committees). The study took place in Mbeya City and Chunya districts in Tanzania, and Abura-Asebu-Kwamankese, Komenda-Edna-Eguafo-Abirem, and Mfantsiman districts in Ghana. The sites were selected based on (1) the provision of contraceptive services; (2) availability of the following methods: a barrier method, a short and long-acting method, emergency contraception, and at least referral for permanent methods in districts; and (3) no social accountability interventions in FP/C currently underway [36] Table ​Table1Table1. Characteristics of the study settings [36] The theory of change drew on existing empirical and theoretical work on social accountability more broadly and specifically related to sexual and reproductive health (see [7, 8, 36]). This informed the dimensions in the measure. The theory of change, Fig. 1, details the inputs of the social accountability process (across the top of the diagram), how these correspond to the cumulative intermediate outcomes at three levels: (1) service users, (2) social networks and (3) service providers, which in turn, effect intended reforms in the quality of care that contribute to contraceptive choice, including increased uptake and use of modern contraceptive methods. As detailed in elsewhere, social accountability engages community members and health services actors in dialogues to identify shared challenges and develop action plans that can lead to improvements in service quality in the health system and in at the individual level, the service user or potential user knowledge and engagement with the health system, both in terms of their own health seeking behaviour and their participation in dialogues with authorities [36]. CaPSAI Theory of Change Following the development of the dimensions, we identified existing validated measures for each domain. We drew heavily on CARE’s Women VOICES tool, a validated measure the aimed to capture similar intermediate outcomes concerning maternal health in Malawi [34]. We added three domains to those used in the VOICES tool to represent the CaPSAI theory of change fully. First was the ‘Knowledge and awareness of rights’ domain that aimed to capture the service user’s perception of rights were also included [42]. For service users’ efficacy with health care providers, we used the National Health Service (NHS) measure of patient activation [28]. To capture changes in service user’s awareness of how to bring about changes and improve their local services, we created items based on theoretical work on political capabilities [22, 41]. We adapted five VOICES validated scales with acceptable reliability to the contraceptive services and local context; for example, family planning services have a charge in Ghana but not in Tanzania. A total of 14 domains were included with 75 items (see Tables 2 and ​and3).3). A five-point Likert scale was used for all the item with the exception of two sets of items had different ranges in their original format. A 6 point scale was used for Self-efficacy with health care providers (set A) ‘Women’s participation in household decision- making’ and a dichotomous scale was used for ‘Self-efficacy with health care providers (set A)’ as originally used [28, 34]. Domains identified based on Theory of Change avalidated from CARE’s Women VOICES Tool bscales tested in this study cnew scales List of items included, per domain (a) retained from [34] We ascertained content validity of the overall items through consultation with experts in social accountability and family planning, and the Principal Investigators who reviewed the questionnaire. Also, the World Health Organization (WHO) Forms Committee, which was composed of technical experts in contraception, social scientists, biostatisticians, and data managers, reviewed the instruments. English surveys were available in both countries and translated in Akan in Ghana and into Kiswahili in Tanzania. Back translation was within a normal range, and pretesting the questionnaire was satisfactory for use in the study populations. The same eligibility criteria for participants were used in both sites (see [36]). A sample of over 750 women aged 15 to 49 years accessing contraceptive services was interviewed prior to the start of the intervention in each country. Sampling was calculated using a priori sample size calculation with the ratio of 10 responses per item ratio and guidance of more than 500, which equals a very good sample for validation [4, 34]. Our sample calculation was based on 75 items of the full survey of items, including post test items. The same items were administered as part of a client exit interview survey in Tanzania and Ghana. The item related to cost of service varied between the countries as there is a nominal fee charged in Ghana. In Tanzania, the item was ‘The government ensures that family planning services are free of cost’ and in Ghana it was ‘The government ensures that family planning methods are free of cost.’ A total of 118 questions were asked of respondents upon leaving a facility, and only 58 scale items and 9 domains were included in the following analysis. Five domains and 17 items were excluded from this analysis because they were not scales or were items that applied after the intervention had been implemented. The other survey items included questions about demographics, reproductive and family planning history, relationship status, income, occupation, and religion. Client exit interviews were conducted on the day of recruitment at the facility in a private location. In Ghana, a total of 15 data collectors (5 females and 10 males) were trained in the survey over 3-day training in April 2018. In Tanzania, a total of 14 data collectors (8 females and 6 males) were trained over in the survey over 5 days in March 2018. Data collection was conducted using a tablet-based questionnaire to capture real-time data using OpenClinica and was later uploaded onto a secure server. Data collection in Tanzania started on 26 March and was completed on 25 May 2018, and all respondents choose to be interviewed in Kiswahili. In Ghana, data collection started on 9 April 2018 and was completed on 4 June 2018, and 46.4% choose to be interviewed in English while 53.6% choose to be interviewed in Akan. Having being assessed for eligibility, respondents completed the informed consent process. There were no incentives given to women and girls to participate in the study. However, study participants who agreed to participate were reimbursed for their travel cost, where it was permitted by country-specific ethical requirements. In Ghana, the research team supported the travel cost to the facility with five Ghana cedis (~ 1 US dollar) given after the interview. In Tanzania, no reimbursements were given. We assessed the item and scale reliability followed by exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) for each country. We assessed the reliability of items and scales to test the internal consistency. The EFA aimed to identify the relationships among items and then group the items as part of a factor. CFA was conducted to confirm the theory behind the grouping of items. We started with assessing reliability using the Cronbach’s alpha to determine item -to- item correlation (or homogeneity) of all 58 observed items and determined the overall alpha for each scale in each country. A Cronbach’s alpha of 0.60 was considered acceptable reliability and 0.70 or higher to be good reliability [16]. Items were removed, according to standard procedures, if the overall alpha improved substantially if an item was removed [23]. Scales with Cronbach’s alpha of ≥.60 were retained [34]. We conducted exploratory factor analysis (EFA) to determine how all 58 observed items clustered together and explore the underlying factor structure in each country. We computed the communality for each item, defined as the proportion of variance in the item attributable to common factors; and used a Kaiser-Meyer-Olkin (KMO) of Sampling Adequacy to assess the suitability of items for the factor analysis. Overall, and factors with KMO value > 0.5, for factor analysis were considered suitable for factor analysis [19, 21]. To determine the factors, we used eigenvalues in accordance with the Kaiser Criterion [20]. We examined the eigenvalues and the scree plot of eigenvalues, and factors with Eigenvalues greater than 1.0 were retained [20]. We used a rotated factor analysis using the maximum likelihood estimation (MLE) with oblique oblimin rotation to determine the factor loadings and variance. Factor loadings assess how items are weighted for each factor and the correlation between the variables and the factor. We used the proportion of variance in the item explained by the factors jointly to assess the reliability of the item in the context of all the factors. Items with factor loadings with values less than 0.40 were excluded. A minimum of three items per factor is recommended, and factors with two items or less were excluded [26]. Confirmatory Factor Analysis (CFA) was done to confirm whether the constructs identified in EFA had a good fit statistically. We applied three recommended models to test for goodness-of-fit [32]. The Standardized Root Mean Square Residual (SRMR) is a measure of the mean absolute correlation residual with a threshold of ≤0.08; the Root Mean Square Error of Approximation (RMSEA) measures the estimated discrepancy between the population and model-implied population covariance matrices per degree of freedom, and a score of ≤0.06 is acceptable, and Comparative Fit Index (CFI) measures the relative improvement in the fit of a researcher’s model over that of a baseline model, and a CFI ≥ 0.95 considered an acceptable fit [6]. The CFA structural model was is presented for each country.

The innovation described in the provided text is the adaptation and validation of social accountability measures in the context of contraceptive services in Ghana and Tanzania. This innovation aims to measure changes in the values, attitudes, and interactions of both service users and healthcare providers during the social accountability process in reproductive health. The measures were developed based on the CaPSAI theory of change and existing items were adapted for the survey. Trained data collectors used the survey to collect data from women attending contraceptive services in both countries. Reliability analysis, exploratory factor analysis, and confirmatory factor analysis were conducted to assess the validity and reliability of the measures. The study found that the measures have high construct validity and reliability in both Ghana and Tanzania. Multiple subscales were identified in each country, indicating the different dimensions of social accountability. The study suggests that these measures may be relevant in different settings and should be validated in new settings.
AI Innovations Description
The recommendation described in the provided text is to adapt and validate social accountability measures in the context of contraceptive services in Ghana and Tanzania. This recommendation aims to improve access to maternal health by measuring changes in the values, attitudes, and interactions of both service users and healthcare providers. The study conducted surveys with women attending contraceptive services in both countries and used reliability analysis, exploratory factor analysis, and confirmatory factor analysis to assess the validity and reliability of the measures. The results showed that the measures have high construct validity and reliability in both countries, with several subscales identified. The study suggests that these measures may be relevant in different settings and should be validated in new settings.
AI Innovations Methodology
The methodology described in the text is focused on the adaptation and validation of measures to capture changes in values, attitudes, and interactions of service users and healthcare providers in the context of social accountability processes in reproductive health, specifically contraceptive services in Ghana and Tanzania. The goal of this methodology is to assess the validity and reliability of these measures in each country.

The methodology includes the following steps:

1. Determining the dimensions of the measures: The CaPSAI theory of change, which outlines how service users’ attitudes change during the social accountability process, is used to determine the dimensions of the measures.

2. Adapting existing items: Existing items are adapted to develop the survey items for data collection. These items are based on validated measures from previous studies and tools such as CARE’s Women VOICES tool.

3. Data collection: Trained data collectors use a survey to collect data from women attending contraceptive services in Tanzania and Ghana. The survey includes the adapted items and scales.

4. Reliability analysis: Reliability analysis is conducted to assess the internal consistency of the items and scales. Cronbach’s alpha is used to determine item-to-item correlation and overall scale reliability. Scales with Cronbach’s alpha of 0.60 or higher are considered reliable.

5. Exploratory factor analysis (EFA): EFA is conducted to explore the underlying factor structure of the observed items. The communality of each item and the Kaiser-Meyer-Olkin (KMO) value are assessed to determine the suitability of items for factor analysis. Factors with eigenvalues greater than 1.0 are retained.

6. Confirmatory factor analysis (CFA): CFA is conducted to confirm the factor structure identified in EFA. Three models are tested for goodness-of-fit using measures such as the Standardized Root Mean Square Residual (SRMR), Root Mean Square Error of Approximation (RMSEA), and Comparative Fit Index (CFI).

7. Validation of measures: The results of the reliability analysis and factor analysis are used to assess the validity and reliability of the measures in each country. The study concludes that the measures have high construct validity and reliability in both Ghana and Tanzania.

In summary, the methodology involves adapting existing items, collecting data through surveys, conducting reliability and factor analyses, and validating the measures to assess changes in values, attitudes, and interactions in the context of social accountability processes in reproductive health.

Partagez ceci :
Facebook
Twitter
LinkedIn
WhatsApp
Email