Efficacy of a digital health tool on contraceptive ideation and use in Nigeria: Results of a cluster-randomized control trial

Global Health Science and Practice, Volume 7, No. 2, Year 2019

Interpretation

Background: Contraceptive prevalence in Nigeria remains among the lowest in the world, which substantially contributes to the country’s high maternal and child mortality. Mobile phone technology penetration has increased considerably in Nigeria, opening opportunities for programs to use this medium for reaching their intended audience with health-protective information. Methods: In 2017, the Health Communication Capacity Collaborative conducted a cluster-randomized control trial in Kaduna City to assess the efficacy of the digital health tool Smart Client on ideational and behavioral variables related to family planning. Twelve wards in the city were randomly assigned to intervention (6 wards) and control (6 wards) arms of the study. A total of 565 women aged 18-35 years were randomly selected from study wards and consented to participate in the study. At recruitment, the women completed a baseline survey. The women in the intervention group were registered to receive 1 welcome call, 13 program calls, and 3 quiz calls on their mobile phones. Each of the program calls had several segments, including introduction, drama episode, and friend-to-friend chat. The last quiz call included evaluation questions. Women in the control arm received no intervention. The efficacy of the intervention was assessed using both perprotocol and intent-to-treat differences-in-differences techniques. Results: The intervention and control arms were equivalent in terms of key sociodemographic characteristics, with the exception of religion. Attrition was a major challenge in the study. On average, participants receiving the intervention listened to 7.2 drama episodes but only 2.6 personal stories and 1.1 sample dialogues. The results of both per-protocol and intent-to-treat analyses show that the intervention was efficacious in improving relevant ideational and behavioral outcomes. For example, the intent-to-treat results show that the intervention increased women’s perceived level of confidence to discuss family planning with a provider by 27.7 percentage points and modern contraceptive prevalence by 14.8 percentage points. Conclusion: This efficacy assessment showed that using an interactive voice response-based digital tool that includes drama is a viable option for promoting positive ideation about family planning and increasing contraceptive use in Nigeria. Significant lessons learned from this efficacy trial include informing participants at the time of recruitment of what the opening segment of the calls will sound like to avoid the calls being mistaken for telemarketing calls and intensive testing prior to scale-up to avoid potential attrition due to technical issues. A cluster-randomized control trial was used for this study. Clusters (wards of residence) were randomly assigned to one of two intervention conditions: receive the digital intervention or receive nothing. The required sample size was determined based on the proportion of women who had discussed contraceptive use with their spouse in the last 12 months. Since the value of this indicator was unknown in the study population, we assumed it to be 50% in our calculations since this level provided maximum variability. We also assumed that this indicator would increase by 15 percentage points among the women in the intervention group, and the required sample size was therefore 240 women for each arm. Estimating a loss to follow-up rate of 20%, we deemed it necessary to recruit 300 women into each arm. This number would provide a 90% power to detect a difference of 15 percentage points between the intervention and the control groups in the proportion of women who had discussed family planning with their husband or partner. Clusters (wards of residence) were randomly assigned to one of two intervention conditions: receive the digital intervention or receive nothing. To recruit women into the study, we randomly selected 6 wards from each of the 2 local government areas (LGAs) in Kaduna metropolis—Kaduna North and Kaduna South. The study wards have comparable access to family planning services. Three wards from each local government area were randomly assigned to the intervention group and 3 to the control group. Trained female field agents, fluent in Hausa, went door to door in sample wards to identify eligible women, explain the purpose and method of the study, obtain informed consent, and recruit participants. Consenting participants in the intervention group were registered to receive the Smart Client calls. The pre-intervention survey for the intervention group was administered as an automated survey at the end of the first call. The post-intervention survey for this group was also an automated survey that directly followed the last call, between 3 and 11 weeks after they started the intervention depending on the frequency of the calls. The control arm did not receive the Smart Client intervention but received 2 calls on their mobile phone: one at the beginning of the study with the automated pre-intervention survey and the other 6 weeks later with the automated post-intervention survey. At the time of recruitment, after informed consent was obtained in person, each participant completed a pre-study questionnaire to provide information on her age, number of children, religion, marital status, LGA and ward of residence, address, preferred nickname to be used during the study, primary and secondary cell phone numbers, and whether she shares a phone with anyone. Data from the pre-intervention and post-intervention survey calls, as well as user analytics collected by the IVR platform, were combined with pre-study data to conduct our analyses. The study took place in North and South Kaduna LGAs of Kaduna State, Nigeria, from March 7, 2017, to June 5, 2017. The 2 LGAs are urban and make up the Kaduna metropolis. Residents included a mixture of Muslims and Christians, although the residents of Kaduna North are predominantly Muslim, while Kaduna South is predominantly Christian. Kaduna metropolis had an estimated total of 1.3 million inhabitants in 2017 and is a melting pot for various Nigerian ethnic groups. While the predominant ethnic group in the city is Hausa, the metropolis also includes large proportions of Yoruba, Igbo, Fulani, Gbaju, and other Nigerian ethnic groups. Secondary analysis performed by the lead author of survey data collected by Measurement Learning & Evaluation, in 201550 revealed that the majority (78.6%) of the women in the city had postprimary education while one-fifth had tertiary education. In the same survey, 21.0% of women of reproductive age reported using a modern contraceptive method, while 6.5% reported using a traditional method. The intended audience for the digital tool is women of reproductive age. As such, women eligible for recruitment into the study were those with the following characteristics: aged between 18 and 35 years and not currently using a nonbarrier contraceptive method (e.g., pill, intrauterine device, implant, emergency contraceptives, tubal ligation, vasectomy, Lactational Amenorrhea Method), owned a mobile phone or had access to one, resident in Kaduna City, and fluent in Hausa. The intended audience for the digital tool is women of reproductive age. The ideational and behavioral outcomes assessed in this manuscript include the following: Data from the automated surveys (e.g., pre-intervention and post-intervention) and user analytics (e.g., number of calls received, number of episodes and segments completed) were combined with the demographic information collected at the time of recruitment and analyzed using summary statistics to compare ideational and behavioral outcomes among participants in the intervention or control groups. To assess the short-term effects of the digital health tool, difference-in-differences (DID) analytic method was employed. Note that each relevant outcome is measured in both the intervention and the control groups at 2 points in time: at the beginning and at the end of the study. DID evaluates the significance of the difference in gains over time between the intervention and control groups. More formally, the DID model is as follows: where δ is the difference-in-difference estimator; Y1p is the relevant outcome at the end of the study for the intervention group; Y0p is the relevant outcome at the beginning of the study for the intervention group; Y1c is the relevant outcome at the end of the study for the control group; and Y0c is the relevant outcome at the beginning of the study for the control group. To strengthen the claim about the causal effect of the tool on assessed outcomes, the analyses controlled for relevant sociodemographic variables in the estimation of DID. Specifically, the estimation models controlled for the following variables: religion, current age, parity, education, marital status, and number of days elapsed between the pre-study interview and the end of the study interview. The findings reported in this manuscript were derived from both per-protocol and intention-to-treat DID. In the per-protocol DID, only participants that met eligibility criteria were recruited into the study and only those that completed the post-study assessment were included in the analysis. The choice to do per-protocol analysis was due to the high level of attrition and because of the heterogeneity between the women who participated in the post-study survey and their peers that were lost to follow-up. Nonetheless, in conformity with CONSORT (Consolidated Standards of Reporting Trials) recommendations51, we also performed intention-to-treat analysis with all the women recruited into the study and who participated in the baseline survey. In the intention-to-treat analysis, eligible and recruited participants are included in the analysis, irrespective of whether they completed the post-study. The outcome was not measured for women who did not participate in the endline survey. For the intention-to-treat analyses, at post-study, baseline responses to ideational and behavioral questions were attributed to the women lost to follow-up since these responses were the most recent and only outcome information that we had for them. The significance of the intention-to-treat analysis should strengthen the claim about the efficacy of the intervention. The findings reported in this article were derived from both per-protocol and intention-to-treat DID. Furthermore, for the intervention group, user analytics were analyzed to track usage patterns (e.g., number of calls, average length of time listened to segments, navigation patterns, number of questions answered in quizzes, number of episodes heard) and gather general feedback on the user experience with the tool. The study was approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board and by the National Health Research Ethics Committee in Nigeria. Study participants gave informed consent prior to their participation in the study. Every participant was made to understand that participation was entirely voluntary and that they could choose not to participate at any time. At the completion of the study and consistent with what was stated in the consent script, all intervention participants who listened to any part of the final call and control participants who listened to any part of the post-intervention survey received an incentive of a nominal amount of airtime credit equivalent to US$1.50 for their participation in the study.

AI Digest

Study Justification:

– Contraceptive prevalence in Nigeria is low, contributing to high maternal and child mortality rates.
– Mobile phone technology penetration in Nigeria is high, providing an opportunity to reach the target audience with health information.
– The study aimed to assess the efficacy of a digital health tool called Smart Client in improving family planning ideation and behavior.

Highlights:

– Cluster-randomized control trial conducted in Kaduna City, Nigeria.
– 565 women aged 18-35 years participated in the study.
– Intervention group received 1 welcome call, 13 program calls, and 3 quiz calls on their mobile phones.
– Control group received no intervention.
– Results showed that the intervention improved women’s confidence to discuss family planning with a provider by 27.7 percentage points and increased modern contraceptive prevalence by 14.8 percentage points.

Recommendations:

– Using an interactive voice response-based digital tool with drama is a viable option for promoting positive ideation about family planning and increasing contraceptive use in Nigeria.
– Inform participants at the time of recruitment about the opening segment of the calls to avoid confusion.
– Conduct intensive testing prior to scale-up to avoid technical issues and attrition.

Key Role Players:

– Researchers and study coordinators
– Health communication experts
– Mobile phone service providers
– Community health workers
– Policy makers and government officials
– NGOs and international organizations

Cost Items for Planning Recommendations:

– Research and study coordination costs
– Development and maintenance of the digital health tool
– Mobile phone service costs
– Training and support for community health workers
– Awareness and promotion campaigns
– Monitoring and evaluation costs
– Incentives for study participants (airtime credit equivalent to US$1.50)

Strength of Evidence

The strength of evidence for this abstract is 8 out of 10.
The evidence in the abstract is strong because it describes a cluster-randomized control trial with a clear methodology and sample size calculation. The study used both per-protocol and intention-to-treat analyses to assess the efficacy of the intervention. The findings show significant improvements in ideational and behavioral outcomes related to family planning. However, the abstract mentions attrition as a major challenge, which could have affected the generalizability of the results. To improve the evidence, future studies could address attrition by implementing strategies to minimize participant dropout, such as providing incentives or improving communication with participants throughout the study period.

Abstract

A cluster-randomized control trial was used for this study. Clusters (wards of residence) were randomly assigned to one of two intervention conditions: receive the digital intervention or receive nothing. The required sample size was determined based on the proportion of women who had discussed contraceptive use with their spouse in the last 12 months. Since the value of this indicator was unknown in the study population, we assumed it to be 50% in our calculations since this level provided maximum variability. We also assumed that this indicator would increase by 15 percentage points among the women in the intervention group, and the required sample size was therefore 240 women for each arm. Estimating a loss to follow-up rate of 20%, we deemed it necessary to recruit 300 women into each arm. This number would provide a 90% power to detect a difference of 15 percentage points between the intervention and the control groups in the proportion of women who had discussed family planning with their husband or partner. Clusters (wards of residence) were randomly assigned to one of two intervention conditions: receive the digital intervention or receive nothing. To recruit women into the study, we randomly selected 6 wards from each of the 2 local government areas (LGAs) in Kaduna metropolis—Kaduna North and Kaduna South. The study wards have comparable access to family planning services. Three wards from each local government area were randomly assigned to the intervention group and 3 to the control group. Trained female field agents, fluent in Hausa, went door to door in sample wards to identify eligible women, explain the purpose and method of the study, obtain informed consent, and recruit participants. Consenting participants in the intervention group were registered to receive the Smart Client calls. The pre-intervention survey for the intervention group was administered as an automated survey at the end of the first call. The post-intervention survey for this group was also an automated survey that directly followed the last call, between 3 and 11 weeks after they started the intervention depending on the frequency of the calls. The control arm did not receive the Smart Client intervention but received 2 calls on their mobile phone: one at the beginning of the study with the automated pre-intervention survey and the other 6 weeks later with the automated post-intervention survey. At the time of recruitment, after informed consent was obtained in person, each participant completed a pre-study questionnaire to provide information on her age, number of children, religion, marital status, LGA and ward of residence, address, preferred nickname to be used during the study, primary and secondary cell phone numbers, and whether she shares a phone with anyone. Data from the pre-intervention and post-intervention survey calls, as well as user analytics collected by the IVR platform, were combined with pre-study data to conduct our analyses. The study took place in North and South Kaduna LGAs of Kaduna State, Nigeria, from March 7, 2017, to June 5, 2017. The 2 LGAs are urban and make up the Kaduna metropolis. Residents included a mixture of Muslims and Christians, although the residents of Kaduna North are predominantly Muslim, while Kaduna South is predominantly Christian. Kaduna metropolis had an estimated total of 1.3 million inhabitants in 2017 and is a melting pot for various Nigerian ethnic groups. While the predominant ethnic group in the city is Hausa, the metropolis also includes large proportions of Yoruba, Igbo, Fulani, Gbaju, and other Nigerian ethnic groups. Secondary analysis performed by the lead author of survey data collected by Measurement Learning & Evaluation, in 201550 revealed that the majority (78.6%) of the women in the city had postprimary education while one-fifth had tertiary education. In the same survey, 21.0% of women of reproductive age reported using a modern contraceptive method, while 6.5% reported using a traditional method. The intended audience for the digital tool is women of reproductive age. As such, women eligible for recruitment into the study were those with the following characteristics: aged between 18 and 35 years and not currently using a nonbarrier contraceptive method (e.g., pill, intrauterine device, implant, emergency contraceptives, tubal ligation, vasectomy, Lactational Amenorrhea Method), owned a mobile phone or had access to one, resident in Kaduna City, and fluent in Hausa. The intended audience for the digital tool is women of reproductive age. The ideational and behavioral outcomes assessed in this manuscript include the following: Data from the automated surveys (e.g., pre-intervention and post-intervention) and user analytics (e.g., number of calls received, number of episodes and segments completed) were combined with the demographic information collected at the time of recruitment and analyzed using summary statistics to compare ideational and behavioral outcomes among participants in the intervention or control groups. To assess the short-term effects of the digital health tool, difference-in-differences (DID) analytic method was employed. Note that each relevant outcome is measured in both the intervention and the control groups at 2 points in time: at the beginning and at the end of the study. DID evaluates the significance of the difference in gains over time between the intervention and control groups. More formally, the DID model is as follows: where δ is the difference-in-difference estimator; Y1p is the relevant outcome at the end of the study for the intervention group; Y0p is the relevant outcome at the beginning of the study for the intervention group; Y1c is the relevant outcome at the end of the study for the control group; and Y0c is the relevant outcome at the beginning of the study for the control group. To strengthen the claim about the causal effect of the tool on assessed outcomes, the analyses controlled for relevant sociodemographic variables in the estimation of DID. Specifically, the estimation models controlled for the following variables: religion, current age, parity, education, marital status, and number of days elapsed between the pre-study interview and the end of the study interview. The findings reported in this manuscript were derived from both per-protocol and intention-to-treat DID. In the per-protocol DID, only participants that met eligibility criteria were recruited into the study and only those that completed the post-study assessment were included in the analysis. The choice to do per-protocol analysis was due to the high level of attrition and because of the heterogeneity between the women who participated in the post-study survey and their peers that were lost to follow-up. Nonetheless, in conformity with CONSORT (Consolidated Standards of Reporting Trials) recommendations51, we also performed intention-to-treat analysis with all the women recruited into the study and who participated in the baseline survey. In the intention-to-treat analysis, eligible and recruited participants are included in the analysis, irrespective of whether they completed the post-study. The outcome was not measured for women who did not participate in the endline survey. For the intention-to-treat analyses, at post-study, baseline responses to ideational and behavioral questions were attributed to the women lost to follow-up since these responses were the most recent and only outcome information that we had for them. The significance of the intention-to-treat analysis should strengthen the claim about the efficacy of the intervention. The findings reported in this article were derived from both per-protocol and intention-to-treat DID. Furthermore, for the intervention group, user analytics were analyzed to track usage patterns (e.g., number of calls, average length of time listened to segments, navigation patterns, number of questions answered in quizzes, number of episodes heard) and gather general feedback on the user experience with the tool. The study was approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board and by the National Health Research Ethics Committee in Nigeria. Study participants gave informed consent prior to their participation in the study. Every participant was made to understand that participation was entirely voluntary and that they could choose not to participate at any time. At the completion of the study and consistent with what was stated in the consent script, all intervention participants who listened to any part of the final call and control participants who listened to any part of the post-intervention survey received an incentive of a nominal amount of airtime credit equivalent to US$1.50 for their participation in the study.

Materials

N/A

Innovations

The innovation described in the study is the use of a digital health tool called Smart Client to improve access to maternal health in Nigeria. The tool utilizes mobile phone technology to deliver health-protective information to women of reproductive age. The study conducted a cluster-randomized control trial to assess the efficacy of the intervention.

Key findings from the study include:

1. The intervention was effective in improving relevant ideational and behavioral outcomes related to family planning.
2. Participants who received the intervention showed increased confidence in discussing family planning with a provider and increased modern contraceptive prevalence.
3. Attrition was a major challenge in the study, highlighting the importance of addressing technical issues and providing clear information to participants.
4. The digital tool, which included drama episodes and interactive segments, was well-received by participants.
5. The study demonstrated that using an interactive voice response-based digital tool is a viable option for promoting positive ideation about family planning and increasing contraceptive use in Nigeria.

Overall, the study highlights the potential of digital health tools to improve access to maternal health by providing information and support to women of reproductive age.

AI Innovations Description

The recommendation based on this study is to develop and implement a digital health tool, similar to the Smart Client intervention used in the cluster-randomized control trial, to improve access to maternal health in Nigeria. This digital tool should utilize mobile phone technology to reach women of reproductive age with health-protective information related to family planning and maternal health.

The digital health tool should include interactive voice response-based calls that incorporate drama episodes, friend-to-friend chats, and quizzes. These components have been shown to be effective in improving ideational and behavioral outcomes related to family planning, such as increasing women’s confidence to discuss family planning with a provider and increasing modern contraceptive prevalence.

To ensure the success of the digital health tool, it is important to address challenges identified in the study, such as attrition and technical issues. Informing participants at the time of recruitment about the content of the calls can help prevent the calls from being mistaken for telemarketing calls. Additionally, conducting intensive testing prior to scale-up can help identify and address any potential technical issues that may lead to attrition.

By implementing a digital health tool that leverages mobile phone technology, Nigeria can improve access to maternal health information and services, ultimately reducing maternal and child mortality rates.

AI Innovations Methodology

Based on the provided information, here are some potential recommendations for improving access to maternal health:

1. Implement a digital health tool similar to Smart Client: The cluster-randomized control trial showed that the Smart Client digital health tool was effective in improving ideational and behavioral outcomes related to family planning. Implementing a similar tool in other regions or countries could help improve access to maternal health by providing women with health-protective information and promoting positive ideation about family planning.

2. Expand mobile phone technology penetration: Since mobile phone technology penetration has increased considerably in Nigeria, efforts should be made to further expand access to mobile phones in underserved areas. This can be done through initiatives that provide affordable mobile phones or improve network coverage in remote areas, ensuring that more women have access to digital health tools and information.

3. Conduct awareness campaigns: To improve access to maternal health, it is important to conduct awareness campaigns that educate women about the importance of maternal health and the available services. These campaigns can be conducted through various channels, including radio, television, social media, and community outreach programs.

4. Strengthen health systems: Improving access to maternal health requires strengthening health systems, including increasing the number of skilled healthcare providers, improving infrastructure and equipment in healthcare facilities, and ensuring the availability of essential medicines and supplies. Investing in the training and deployment of midwives and other healthcare professionals can help improve access to quality maternal health services.

To simulate the impact of these recommendations on improving access to maternal health, a methodology could be developed as follows:

1. Define the indicators: Identify the key indicators that measure access to maternal health, such as the percentage of women receiving antenatal care, the percentage of women delivering with a skilled birth attendant, or the contraceptive prevalence rate.

2. Collect baseline data: Gather baseline data on the selected indicators to establish a starting point for measuring improvement. This can be done through surveys, interviews, or data from existing health information systems.

3. Implement the recommendations: Implement the recommended interventions, such as the digital health tool, mobile phone penetration initiatives, awareness campaigns, and health system strengthening activities.

4. Monitor and collect data: Continuously monitor the implementation of the interventions and collect data on the selected indicators. This can be done through surveys, routine health facility data collection, or mobile phone-based data collection systems.

5. Analyze the data: Analyze the collected data to assess the impact of the interventions on the selected indicators. This can be done using statistical methods, such as regression analysis or difference-in-differences analysis, to compare the changes in the indicators between the intervention and control groups.

6. Evaluate the results: Evaluate the results of the analysis to determine the effectiveness of the interventions in improving access to maternal health. Assess whether the recommended interventions have led to significant improvements in the selected indicators.

7. Adjust and refine: Based on the evaluation results, make any necessary adjustments or refinements to the interventions to further improve access to maternal health. This could involve scaling up successful interventions, addressing any challenges or barriers identified during the evaluation, and continuously monitoring and evaluating the impact of the interventions.

By following this methodology, it would be possible to simulate the impact of the recommended interventions on improving access to maternal health and make evidence-based decisions on how to best allocate resources and implement effective strategies.

Author

Dima Health

Statistics:

Citations: 10

Identifiers:

DOI: 10.9745/GHSP-D-19-00066

Research Areas:

Community Interventions, Genetics and Genomics, Health System and Policy, Maternal Access, Maternal and Child Health, Quality of Care, Sexual and Reproductive Health, Social Determinants, Technology and Innovations

Study Countries:

Nigeria

Study Design:

Cohort Study, Cross Sectional Study, randomised-control-trial

Study Approach:

Quantitative

Participants Gender:

Female