Are stage-based health information messages effective and good value for money in improving maternal newborn and child health outcomes in India? Protocol for an individually randomized controlled trial

Background: Evidence is limited on the effectiveness of mobile health programs which provide stage-based health information messages to pregnant and postpartum women. Kilkari is an outbound service that delivers weekly, stage-based audio messages about pregnancy, childbirth, and childcare directly to families in 13 states across India on their mobile phones. In this protocol we outline methods for measuring the effectiveness and cost-effectiveness of Kilkari. Methods: The study is an individually randomized controlled trial (iRCT) with a parallel, partially concurrent, and unblinded design. Five thousand pregnant women will be enrolled from four districts of Madhya Pradesh and randomized to an intervention or control arm. The women in the intervention arm will receive Kilkari messages while the control group will not receive any Kilkari messages as part of the study. Women in both arms will be followed from enrollment in the second and early third trimesters of pregnancy until one year after delivery. Differences in primary outcomes across study arms including early and exclusive breastfeeding and the adoption of modern contraception at 1 year postpartum will be assessed using intention to treat methodology. Surveys will be administered at baseline and endline containing modules on phone ownership, geographical and demographic characteristics, knowledge, practices, respectful maternity care, and coverage for antenatal care, delivery, and postnatal care. In-depth interviews and focus group discussions will be carried out to understand user perceptions of Kilkari, and more broadly, experiences providing phone numbers and personal health information to health care providers. Costs and consequences will be estimated from a societal perspective for the 2018-2019 analytic time horizon. Discussion: Kilkari is the largest maternal messaging program, in terms of absolute numbers, currently being implemented globally. Evaluations of similar initiatives elsewhere have been small in scale and focused on summative outcomes, presenting limited evidence on individual exposure to content. Drawing upon system-generated data, we explore linkages between successful receipt of calls, user engagement with calls, and reported outcomes. This is the first study of its kind in India and is anticipated to provide the most robust and comprehensive evidence to date on maternal messaging programs globally. Trial registration: Clinicaltrials.gov, 90075552, NCT03576157. Registered on 22 June 2018. Does exposure to mobile health information messages during pregnancy and postpartum improve infant feeding and family planning practices in MP, India? Exposure to Kilkari messages on infant feeding will correspond to a 5% increased prevalence in the reported practice of exclusive breastfeeding for infants 0–6 months of age and use of modern contraceptive methods at 1 year postpartum between intervention and control arms. The control arm is expected to have a prevalence of 58% for the exclusive breastfeeding indicator and based on the recent NFHS data for MP. Kilkari is comprised of 90 min of content delivered through 72 once weekly voice calls: 24 during pregnancy, 24 within the first 6 months postpartum, and 24 from 6 to 12 months postpartum. An estimated 16% of content is comprised of health information on family planning, 13% on infant feeding, 11% on child immunizations, 10% on pregnancy care, 8% on child malnutrition, 7% on diarrhea, and 7% on postnatal care (Table 1). Messaging duration in seconds by content area In the 13 states where Kilkari implementation is ongoing, pregnant woman or postpartum women’s data are entered into one of two health information systems registries: maternal and child health tracking system (MCTS) or reproductive and child health (RCH) system. Data, including mobile phone numbers, are collected in paper registers by frontline health workers, including auxiliary nurse midwives (ANMs) and accredited social health activists (ASHAs), and entered by data entry operators (DEOs) into the government’s electronic health records system—either MCTS or RCH system depending on the state. Data are then sent from there to the MOTECH system. Before the data are accepted by MOTECH, the system automatically runs validations to check that the mobile numbers are in the correct format, locations match location masters in the MOTECH database, and last menstrual period (LMP) and date of birth are within the Kilkari timeframe. The MOTECH system then determines the schedule for Kilkari messages based on the LMP or delivery date and the MOTECH engine provides the list of phone numbers (clients) to be called each day to the IVR system, which then calls the numbers and plays the appropriate pre-recorded message. For the purposes of this intervention, mobile numbers will be collected directly by the research team for randomization. Pregnant and postpartum women randomized to receive Kilkari messages will receive messages from their enrollment in pregnancy through to 12 months postpartum. Women enrolled to the comparison arm (status quo) will not receive messages. Figure 1 depicts a theory of change for Kilkari’s impact on health outcomes specific to immediate and exclusive breastfeeding. This theory of change builds off of frameworks proposed by Kimani-Murage et al. [8] and Rollins et al. [9] for assessing factors affecting the actualization of breastfeeding recommendations. Theory of change for assessing the impact of Kilkari The study is an individually randomized controlled trial (iRCT) with a parallel, partially concurrent, and unblinded design. Study participants will include pregnant and postpartum women from four districts (Rajgarh, Rewa, Hoshangabad, Mandsaur) of MP where Kilkari implementation has not yet occurred. Additional qualitative research specific to MCTS/ RCH will be carried out in two districts (Pali and Sikar) of Rajasthan. In both states, district selection was made by the MOHFW with inputs from the implementing partner BBC Media Action. The study setting in MP is characterized by disparities in access to education, mobile phones, and health services by gender and geographic location (rural/urban). With a population of over 75 million, MP is home to over 20% of India’s population. MP ranks as one of the worst performing states in India economically (gross domestic product per capita of $1100 USD versus $1709 nationally) and in terms of health outcomes, particularly with regard to child nutrition. In 2015, only 35% of children were breastfed within one hour of birth and 58% of children exclusively breastfed until 6 months [10]. One in four children aged under 5 years were wasted (weight-for-height), and 42% were stunted (height-for-age) [10]. While over half of pregnant women attended ANC in the first trimester, only 36% received the recommended four ANC visits [10]. Rates of institutional delivery were high with over 81% of women reportedly delivering in a health facility in 2015, and yet only 18% received a postnatal health check within 2 days following birth [10]. Health behaviors and care-seeking practices differ markedly between urban/rural areas and are underpinned by high rates of illiteracy (41% of women, 18% of men) and poor access to mobile phones among women [10]. In 2015, 19% of rural and 50% of urban women reported having access to a mobile phone [10]. Trial inclusion will be restricted to women who are 12–34 weeks of gestation at randomization, ≥ 18 years of age, speak and understand Hindi, and own or have access to a mobile phone during the morning and afternoon. Women will be excluded who are BSNL mobile subscribers (due to poor network coverage) and do not provide consent to participate in the trial. To identify eligible women, all FLHWs in the study villages will be contacted to obtain a list of the currently pregnant women in their village. If the FLHW is not available, a census of those villages will be carried out to find women greater than 18 years of age, within the fifth to seventh month (12–34 weeks of gestation at randomization) of their pregnancy. Women who complete the face to face survey and consent to receive Kilkari messages will be randomized to receive Kilkari calls (intervention) or not receive messages (comparison). Intervention area participants will start to receive Kilkari calls by no later than 8 months (34 weeks) after conception. The timing of this enrolment is intended to mirror that observed in other state deployments of Kilkari which rely on the identification of pregnant women via the MCTS/RCH database and in practice therefore enroll women in the later stages of pregnancy and/or early postpartum. Gestational age, parity, age of woman, and ownership of phone are important variables potentially associated with exposure (listening to messages) and are likely to influence outcomes. The sample will be stratified by the above variables to allow for equal numbers of these categories in the sample between intervention and control groups (balance of covariates in the sample). Randomization will be performed using the sample command in stata with the use of the above listed variables as stratifiers. The randomization procedure will be done in blocks (block randomization) after the enrollment is completed for each sub-district administrative unit called as Community Development Block. This is to ensure a similar number of intervention and control subjects for each Community Development Block. Table 2 summarizes the sample size requirements by activity, along with underpinning assumptions. To determine changes in RMNCH knowledge, practices, and careseeking over time (study aim 1), sampling will be powered to detect changes in coverage across study arms. To detect a 5% difference in reported practice of exclusive breastfeeding assuming an alpha of 0.05, 80% power, the estimated sample size for a two-sample proportions test would be 3200 women. After adjusting for a design effect (variance inflation factor) of 1.25 due to clustering at the level of the Community Development Block and a 20% loss to follow-up from enrollment to endline surveys and a potential loss of 35% of women due to poor reporting of phone numbers and/or changes to original phone numbers provided, a total of 5000 women will be enrolled to the study. Assuming 40% of women have access to a mobile phone, we anticipate that nearly 700,000 households will be listed in all four districts of MP. Sample size estimates by study aim *Sample estimates based on kappa of 0.80, alpha of 0.05, margin of error of 0.05% Among pregnant women, a module on RMC during antenatal care will be integrated into the baseline assessment amongst the enrolled women. To measure RMC during childbirth, a total of 880 women with a birth outcome in the preceding 1 to 4 months will be interviewed. This sample size was designed to accommodate inter-modal testing and is sufficient to additionally measure the prevalence of the RMC indicator of reported verbal abuse during childbirth with 80% power, alpha of 0.05, and precision of 5%. To generate validated phone survey tools for RMC, we will aim to determine the inter-modal reliability of face to face versus computer-assisted telephone interviewing (CATI)/phone surveys for both pregnant and postpartum women. Assuming a kappa of 0.80, a margin of error of 0.05, an alpha of 0.05, and the proportion of positive responses of 0.35 for rater 1 and 0.40 for rater 2, 146 participants who have completed the survey are required. Adjusting for a 20% loss to follow-up between the face to face women’s survey and the following mobile phone survey, and a 20% baseline completion percentage for the latter, 880 women with a birth outcome in the preceding 1 to 4 months will be interviewed face to face. Within 1–2 weeks of the initial interview, a random sample of those completing the face to face interview who consent to be called for the follow-up phone survey will be contacted. Assuming a 20% response rate, all 880 women will be contacted as part of the phone survey to yield the 146 completed face to face and phone survey interviews. Figure 2 provides an overview of enrolment and study design over time and the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) are presented in Fig. 3 and Additional file 1. At enrollment, women will be administered a survey containing modules on phone ownership, geographical and demographic characteristics, knowledge (danger signs during pregnancy, delivery, and postpartum; essential newborn care; nutrition; hygiene, diarrhea treatment, family planning, etc.), practices (immediate, early, and exclusive breastfeeding; complementary feeding; skin-to-skin care, etc.), RMC, and coverage for antenatal care, delivery, and postnatal care. Around 6–8 weeks after the expected date of delivery, a phone survey will be administered to a random sample of the enrolled women until a target sample size of 3500 is attained (1750 each in intervention and control arms) to assess breastfeeding and newborn care practices. An endline survey will be carried out from November 2019 to February 2020 and each woman will be followed up at 12 months postpartum based on their expected date of delivery at enrollment. The endline survey will capture data on practices during pregnancy and intrapartum and postpartum periods. Overview of enrolment and study design over time Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Checklist Knowledge attitudes and practices (KAP) surveys will be administered at baseline and endline containing modules on phone ownership, geographical and demographic characteristics, knowledge (danger signs during pregnancy, delivery, and postpartum; essential newborn care; nutrition; hygiene, diarrhea treatment, family planning, etc.), practices (immediate, early, and exclusive breastfeeding; complementary feeding; skin-to-skin care, etc.), RMC, and coverage for antenatal care, delivery, and postnatal care. Questions on exposure to and sources for health information content will additionally be asked to identify possible sources of contamination. The list of individuals randomized to receive Kilkari content will be provided to BBC Media Action for enrollment into Kilkari continuously throughout the implementation of the baseline survey. Following 11–12 months, participants from both study arms will be interviewed as part of the face to face endline survey. A phone survey will be administered to all study arm participants on a rolling basis at 1 month postpartum to measure RMNCH practices, including infant feeding practices and essential newborn care as well as RMC. Methods to generate a validated phone survey tool are described elsewhere and in brief will include a face to face survey administered to women during pregnancy and with a live birth outcome in the preceding 1–4 months postpartum at the time of the household listing. A sub-sample of respondents from the initial face to face survey will be administered a second test-retest survey by a separate but similar profile of enumerator. Findings from the face to face surveys will be used to generate phone survey tools which will be implemented amongst a random sample of pregnant and postpartum women who consent to be called for the follow-up phone survey. Analyses to assess the intermodal reliability of face to face versus phone survey responses will be used to generate a validated tool. The tool developed for postpartum women will subsequently be administered to women enrolled into the iRCT during pregnancy at 1–4 months postpartum (Fig. 2). User experiences with RMNCH services received in the public sector, including care received during pregnancy and childbirth, are anticipated to influence decision-making on whether recommended practices promoted through Kilkari messages are adhered to. Women’s experiences with health services during pregnancy (RMC) will be measured as part of the baseline face to face survey administered to women 5–7 months pregnant. In addition to user experiences with RMNCH, in-depth interviews (IDIs), focus group discussions (FGDs), and direct observations will be carried out among pregnant and postpartum women to identify barriers to and facilitators of pregnancy and birth registration as well as the enrolment of their mobile phone numbers in MCTS/ RCH, including the sharing of personal data and mobile phone numbers in MP and Rajasthan, India. Following enrollment to Kilkari, we will draw from system-generated data to determine factors underpinning the successful receipt of calls not only among iRCT participants but across the 13 states where Kilkari implementation is underway. Since it is not possible to discern who listens to calls, emphasis will be placed on the mobile phone number registered to receive Kilkari content as the unit of analysis. Predicators for exposure to Kilkari content will be captured by four stakeholders: 1) government’s data system (MCTS, RCH, webservices); 2) program’s call delivery and receipt systems; 3) mobile network operator (network coverage and quality); and 4) user device characteristics (switched on, within range of network). Table 3 summarizes key indicators proposed for assessment. Essential steps and barriers to exposure to Kilkari content in four districts of MP *Less relevant to iRCT as eligible users will be identified as part of the household listing at baseline and a list of phone numbers provided to BBC Media Action for content delivery To measure user engagement with Kilkari content (behavioral performance), we will measure the proportion of calls listened to out of those received on average among the mobile phones enrolled to Kilkari. MOTECH data will be merged with IVR, call center data, and baseline face to face survey data on user characteristics, including socioeconomic status, education, phone access, mobile literacy, and numeracy, and used to identify predictors for exposure to Kilkari content. Differentials in listening patterns for each mobile device will be used to classify Kilkari users into high and low listenership groups. In addition to quantitative data analyses, qualitative interviews will be carried out to examine user perceptions of mobile messaging and use of data for program design and decision-making. Differentials in user perceptions of content by topical area as well as over time will be assessed to facilitate understanding of possible differentials in user engagement. To measure the economic costs of Kilkari we will prospectively measure program, user, and health system costs in the four districts of MP. Program costs associated with the development, start-up, and implementation of program activities will be assessed using an ingredients approach for the 2018–2019 analytic time horizon. Costs to users will be assessed through face to face household surveys and include direct (out of pocket costs for drugs, supplies, consultation fees, transportation) and indirect (wages lost, child care) costs incurred by users for RMNCH careseeking. Finally, we will explore the feasibility of measuring incremental costs to the health system above and beyond those captured by the measurement of program and user costs. Collective consideration of program, user, and health system costs will allow for a societal perspective to be taken and thus a more comprehensive estimate of the full economic costs associated with the program across study arms of individuals exposed versus those not exposed to Kilkari. To measure the incremental effectiveness of Kilkari, data on changes in coverage for RMNCH careseeking and practices assessed through household surveys will be inputted into the Lives Saved Tool to generate a measure of lives saved associated with program activities. Lives saved will be used to yield an estimate of the incremental DALYs averted and ultimately a cost per DALY averted for Kilkari. The statistical analysis plan will ensure that the analyses are not data driven or selectively reported. The presentation of primary analyses is expected in spring 2020, after all participants have been followed up to 1 year postpartum. Unadjusted and adjusted results will be presented as appropriate effects sizes with 95% confidence intervals (CI). Demographic and population characteristics of the women at baseline will be compared between study groups. Significance tests will be performed to investigate for differences between groups at baseline. Categorical outcomes will be analyzed using contingency tables and chi-square test. Continuous normally distributed variables will be analyzed using t-tests and non-normally distributed variables using non-parametric tests. The primary outcomes, early and exclusive breastfeeding and the adoption of modern contraception at 1 year postpartum, will be analyzed by intention-to-treat methodology, with adjustment for stratifying and baseline factors, including gestational age at enrollment, ownership of mobile phone, age of the pregnant woman at enrollment, and birth order of index pregnancy. Binary logistic regression (BLR) models with generalized estimation equations will be used to account for clustering effects at the level of the Community Development Block. The secondary outcomes of interest are anticipated to include immediate breastfeeding, skilled attendance and facility delivery, postnatal care, newborn birth practices, and immunizations. Binary logistic regression will be used for binary outcomes, and multiple linear regression will be used for continuous measures. Wilcoxon rank sum test will be used for continuous measures which are not normally distributed, including duration of breastfeeding. Cox proportional hazards regression will be used for time to event analyses, including time to initiation of breastfeeding and adoption of a modern contraceptive method in the postpartum period. All regression analyses will be performed with adjustment for baseline factors. Comparisons between the treatments will be performed in pre-specified subgroups, including phone ownership, caste, age groups, etc. The subgroup analysis does not comprise the primary analysis and did not inform the sample size calculation. The interpretation of any heterogeneity in effects due to the subgroups will be based on using interaction terms between the sub-group variables and the intervention group with no adjustment for multiple testing. The exposure of women to different messages will be assessed using the system-generated data from the MOTECH database. We will analyze the effect of different messages on specific related outcomes using a per-protocol methodology. Missing data will be assessed for any systematic biases based on the dropout of women using their baseline characteristics. Sensitivity analysis will be performed to assess the effect of different levels of systematic bias on the effect sizes. The phone survey analyses will principally aim to determine the reliability of the phone survey tools. Reliability analyses will aim to determine the stability and consistency of results [11]. A Cronbach’s alpha of 0.7 or higher is proposed as the cutoff for determining sufficient evidence of reliability [11]. Additional analyses related to the internal consistency of the scale as well as the presence of floor and ceiling effects will be conducted and overall findings summarized in an overarching table on validity and reliability [11]. A full health-economic analysis will only be performed after completion of the main trial. Once costs and effects are estimated, uncertainty analyses, including one-way and probabilistic sensitivity analyses, will be conducted. The latter will be carried out in Microsoft excel using a Monte Carlo simulation with 1000 iterations per analysis. The resulting mean point estimate will be obtained by dividing mean costs by mean effects, while a 95% confidence interval for the ICER is presented based on percentiles. A cost effectiveness plane and cost effectiveness acceptability curve will be used to calculate the probability that the intervention will be cost-effective for each of a number of standard thresholds of cost-effectiveness. Cost-effectiveness will ultimately be determined according to thresholds set forth by the Commission for Macroeconomics and Health and WHO which stipulate that an intervention is “highly cost-effective” and “cost-effective” interventions at one and three times, respectively, the value of per capita gross domestic product per DALY averted. To facilitate comparison with alternative resource uses, we will additionally compare findings against those available in the literature.