The role of a decision-support smartphone application in enhancing community health volunteers’ effectiveness to improve maternal and newborn outcomes in Nairobi, Kenya: Quasi-experimental research protocol

Introduction Improving maternal and newborn survival remains major aspirations for many countries in the Global South. Slum settlements, a result of rapid urbanisation in many developing countries including Kenya, exhibit high levels of maternal and neonatal mortality. There are limited referral mechanisms for sick neonates and their mothers from the community to healthcare facilities with ability to provide adequate care. In this study, we specifically plan to develop and assess the added value of having community health volunteers (CHVs) use smartphones to identify and track mothers and children in a bid to reduce pregnancy-related complications and newborn deaths in the urban slums of Kamukunji subcounty in Nairobi, Kenya. Methods and analysis This is a quasi-experimental study. We are implementing an innovative, mHealth application known as mobile Partnership for Maternal, Newborn and Child Health (mPAMANECH) which uses dynamic mobile phone and web-portal solutions to enable CHVs make timely decisions on the best course of action in their management of mothers and newborns at community level. The application is based on existing guidelines and protocols in use by CHVs. Currently, CHVs conduct weekly home visits and make decisions from memory or using unwieldy manual tools, and thus prone to making errors. mPAMANECH has an in-built algorithm that makes it easier, faster and more likely for CHVs to make the right management decision. We are working with a network of selected CHVs and maternity centres to pilot test the tool. To measure the impact of the intervention, baseline and end-line surveys will be conducted. Data will be obtained through qualitative and quantitative methods. Ethics and dissemination Ethical approval for the study was obtained from the African Medical Research Foundation. Key messages from the results will be packaged and disseminated through meetings, conference presentations, reports, fact sheets and academic publications to facilitate uptake by policy-makers. The overall objective of the proposed work is to develop and validate a decision-support algorithm within an mHealth application in improving maternal and newborn health outcomes in urban slums in Kamukunji subcounty in Nairobi, Kenya. The project seeks to assess the added value of using a CHV decision-support module of mPAMANECH in reducing prenatal and postnatal maternal complications and newborn deaths. Specifically, the project will assess (1) the feasibility and acceptability of using a decision-support module of mPAMANECH and (2) whether decision-support platform contributes to increase in use of Maternal and Newborn Health (MNH) services. We will measure several outcomes (figure 1) related to the two specific objectives above. The outcome measures have been used in different studies assessing CHV abilities to assess and make referrals.26 27 Online supplementary files 1–3 show the quantitative and qualitative measures under study. Key outcome measures. CHV, community health volunteer; mPAMANECH, mobile Partnership for Maternal, Newborn and Child Health. Our innovation centres around enhancing the functionality and utility of the current mPAMANECH application to include a community decision-support tool to be used within a local system comprised CHVs and five maternity centres serving five slum settlements in Nairobi. The decision-support module in the mPAMANECH application will supplement the existing MNCH data capture modules and help in the identification of high-risk facing pregnant women, new mothers and newborns with complications, and to make timely and correct decisions on referral for cases that need intervention. The main beneficiaries are pregnant women (over 24 weeks), mothers in the immediate postpartum period and their newborns (up to 28 days old). The use of CHVs in delivering the mobile phone-based intervention and follow-up by the sub-County Health Management Team (sCHMT) will ensure sustained support and adherence to the intervention during pregnancy, immediate post partum and in the neonatal period. The mPAMANECH has the official 100, 513, 514 and 515 forms in use by CHVs and CHEWs, who supervise the CHVs.28 29 A dedicated team of CHEWs will ensure that the intervention is delivered as expected and non-adherence captured and documented. CHV adherence will be measured by the CHEWs who will provide additional support, correcting CHVs but not forms. They will be used to sample some of the CHVs’ work. For each woman or newborn with referral symptom, seen at household level, we will compare the CHEW’s assessment with that of the CHV in regard to the proportion of cases for whom community health volunteers (CHVs) identified and indicated correctly on their forms as requiring a referral, and the proportion for whom a written referral was provided. This is beneficial because the review meetings are at the end of the month. The sampling of the work will enable ongoing supportive supervision to improve the quality of the CHVs work. In addition, more assessment will be made for those referrals that will be seen at health facility level, comparing danger signs identified by the CHV to those seen by the clinician. The attributes of the solution include: (1) use of any android devices—the entire application is operable on phones, tablets or desktops/laptops; (2) within a linked local system—messaging is delivered internally via the portal and no extra network charges are required for messaging; (3) the application is bandwidth light and takes about Kenya Shillings ‎100 a month per user, based on local data bundle charges. The application can work offline and auto-synchronise as soon as network connectivity is re-established; (4) updates are automated and there is no need for an IT expert to have physical access to the device to execute a software patch or change; (5) the application has security features that protect users’ confidentiality and limit access to a closed but linked local system of health facilities and CHVs with varied access rights; (6) it has the ability to limit use of other phone functionality to only allow this application to run. This helps prevent abuse of the device and promote saving on bandwidth. We aim to enhance the functionality and utility of the current mPAMANECH application to include a community decision-support tool to be used within a local system comprised of selected CHVs and five maternity centres serving slum settlements in Nairobi. In the ‘Operationalisation’ section, we describe more on how the decision-support system is expected to work. From our proposed theory of change, figure 2, we anticipate that the decision-support module of mPAMANECH will assist the CHVs in the identification of high-risk pregnant women, new mothers and newborns with complications and to make timely and correct decisions on referral for cases that need intervention. With the improved knowledge and skills possessed by the CHVs, more women and neonates in need of healthcare will be identified and referred to the necessary health facilities. As a result, there will be increased use of maternal and neonatal health services and reduced maternal and neonatal complications and deaths. Theory of change. CHV, community health volunteer; DSS, decision-support system. The mobile decision-support tool/ system (mDST/S) mobile application (app) is an Android app that installs from the phone and runs as an application (figure 3). The app will be hosted at Google store and is accessible for download via the internet automatically to the user handset. The system. APHRC, African Population and Health Research Center; CHW, community health worker. Once installed, the handset and the application will be configured with the credentials of the CHV including username and password, which are linked with the operating phone number. This set-up allows the system to register and associate collected records to the respective CHV; useful for reporting and analysis purposes. The CHVs will be provided with a mobile phone running the mHealth app for data collection which is then relayed to the head office at APHRC campus. When visiting households for data collection, by initiating a new record the system automatically picks the geolocation of the household and associates it with the record. The CHV will complete the respective form and save the information at any stage on the handset. This method allows the CHV to continue collecting more information and only submit the completed form to the server over the internet. Where the handset detects that internet connectivity is not available for one reason or the other, the completed forms will be stored in the phone and only be relayed to the server when the connectivity is established. Because the application is in communication with the server in real time, any update of the data is immediately available to the CHVs. The application is preconfigured with danger signs for both mothers and newborns as defined by the WHO.30 When any of the danger signs is picked by the system, the CHV is immediately prompted to refer the patient to a facility within the network. The system is to be set up in such a way that the CHV will not be allowed to proceed and complete the forms until the referral is done. The patient’s information will be available to all the facilities within the network and he/she will be able to visit any of the networked facilities for treatment. This information is also relayed to the rest of the other facilities and a record of the visit is kept in the server. Within 24 hours if the patient has not been seen in any of the networked facilities, an SMS is relayed to the respective CHV handset. At this point, the CHV should follow up with the patient to inquire about their condition and whether they chose to honour the referral to a facility outside the network or not. When the patient visits a facility within the network, using the household number and the name, he/she will be easily identified and treated. The clinician is able to record the treatment in the patient’s information, which is recorded in the server. Both the CHV and the clinician’s information are available to a reviewer in real time from the web. The reviewer’s role would be to assess the CHV’s reason for referral and the clinician’s final diagnosis for correctness. The feedback from the reviewer will be available and shared on a monthly basis during review meetings. Every week, the CHEW, will randomly sample referrals made by each CHV to assess their correctness. This will be compared with the ‘reviewers’ summary. The key features of mDSS solution are given below and in figure 4. How the system works. CHV, community health volunteer; SMS, short message service, mDST mobile decision support tool Through this app, the mobile Decison Support Tool/System (mDST/S) data collection tools are accessible on the CHV mobile handset. It allows for completion and submission of the information to the mDSS Server. A database-driven web application that acts as a host for the collected information. It also has a web interface for data manipulation by the facility healthcare worker(s) and reviewer. The mobile Decision Support Tool/System (mDST/S) server application incorporates an application logic component that connects with the SMS and email gateways. This allows for real-time communication of certain critical indicators to the respective recipients. For example, where a patient has not shown up at the referred facility, the CHV will be alerted. This module is accessible at the facility via the web. The application allows the clinician to treat the patient and record the treatment. The facility portal is accessible by facility staff to manage referrals. The ‘Messages’ to alert the CHVs are to be integrated with WhatsApp for effective message delivery over the same mobile phone. This module is also accessible via the web and provides the reviewer a view to assess both the CHV referral report and the clinician’s diagnosis. It is based on this analysis that a decision is measured. Usage of a CHV decision-support module does not have an effect on use of MNH services. To answer the question on effect of the decision-support system, we will use a quasi-experimental design with preassessments and postassessments to determine the impact, if any, on the MNH services and selected health indicators. We will work with three community units (CUs) from which a group of 50 CHVs will serve as the intervention group. Three CUs in Embakasi subcounty, with 50 CHVs will serve as the control. The CUs in the control group will be geographically distant from the intervention site to limit contamination. The control group will be facilitated within the normal standard of practice which includes paper-based reporting. The idea is to compare access with MNH services, referral practices and MNH outcomes between those with a decision-support tool with those working with traditional paper-based tools. We will conduct a baseline survey and an end line at least 1 year after the full implementation of the intervention. The control group will be assessed at the same time as the surveys are being conducted in the intervention group. To strengthen the case for causal inference, we will also do a systematic monitoring and documentation of the intervention based on our theory of change. The monitoring and documentation will also capture any other contextual factors that may influence the same outcomes as our intervention. To answer the question on feasibility, we will conduct a mixed-method assessment of the CHVs and CHEWs. Qualitative assessments of their current work experiences with the paper-based system as well as after the introduction of the mobile-based system will be conducted. We will assess ease of use, challenges experienced and opportunities for improvement. These will be triangulated with their Health Management Infomation System (HMIS)reports (paper and electronic). The intervention site covers informal settlements in Kamukunji subcounty, which, like other slums, are characterised by poverty, poor coverage of social services and poor health outcomes. We will identify CUs and five facilities that serve the CUs. Embakasi subcounty will serve as the control site. We will identify three CUs to serve as controls. The CUs in the intervention and control groups will be purposively selected based on discussions with the subcounty community strategy coordinators for Kamukunji and Embakasi. We will select CUs with the worst health indicators for both the intervention and control groups, including those serving informal settlements, and with more likelihood to benefit from the intervention. For the population based surveys the detailed strategies are defined below: The project will target households with women of reproductive age (15–49 years) in the informal settlements of Kamukunji and Embakasi. It will also target healthcare providers in five selected health facilities as well as the subcounty Health Management teams of Kamukunji, Embakasi, Makadara and Nairobi City County. A quantitative survey will be conducted focussing mainly on maternal and newborn health and family planning services. The key outcome of interest is correct referral practices. Using proportion of neonates with at least one danger sign referred by CHVs as the indicator, since it is representative of the target population, the total sample size for each group has been estimated as 173 (346 in total). Using a difference in proportion 13.3% for neonates with danger signs referred (rising from 16.7% to 30%), at 95% CI for 80% power and sample proportions of 50% in the intervention and 40% in the control groups, respectively, we came up with a sample size of 315. Accounting for a potential non-response rate of 10% based on previous studies in similar study populations, the effective sample size is 346. Using correct decisions made by CHVs (one of the indicators to be measured), the sample size estimated as 199. 346 is therefore an appropriate sample size. To measure feasibility and acceptability, all the CHVs in the intervention site will be assessed, in addition to an audit of the functionality of the system. We will measure the percentage time for which the system is down on a monthly basis (numerator: number of times (in minutes) when the system is down; denominator: total active time in a month. Down time defined as 30 min of hanging and active time as time without hitches), proportion of CHVs effectively using the decision support system (within the system, on a quarterly basis we will be able to generate reports on decisions, correct or otherwise, made by the CHVs. These will also be compared with the control site that will only have a paper-based system of data collection) and the experiences of the CHVs and the mothers with the mobile-based system. These will be compared with the investment. To answer the question on effect of the system on use of services, the data above will be triangulated with data from the ‘mPAMANECH application which already has an integrated data collection module’, the data generated by the CHVs and participating health facilities will be retrieved, cleaned and analysed to derive estimates of the main outcome of interest—correct referral practices. In addition, these data will be triangulated with other sources such as the CHV monthly reports and health facility HMIS. A system of random numbers generated using STATA will be used to select the respondents based on a sampling frame that is going to be informed by an updated household register in the selected community units. We will use focus group discussions (FGDs) and in-depth interviews among the direct project beneficiaries and CHVs, and key informant interviews with key actors (SCHMTs, health providers, CHVs and community leaders). Participants will be purposively selected to represent the different stakeholders as well as different health service user categories (users and non-users). Data from the quantitative survey will be used to identify women who have or not used specific MNH services, and these will be approached to participate in the focus group discussions or in-depth interviews. Other respondents will be identified based on their position in the community and their role in the project. We will conduct a baseline survey and an end line at least 1 year after the full implementation of the intervention. The control group will be assessed at the same time as the surveys are being conducted in the intervention group. To strengthen the case for causal inference, we will also do a systematic monitoring and documentation of the intervention based on our theory of change. The monitoring and documentation will also capture any other contextual factors that may influence the same outcomes as our intervention. The effects and impact of the project will be determined by triangulating data and information from different sources, examining trends where possible and trying to find and support explanations for the observed findings. Our key data sources will be the following: household surveys, routine HMIS data and qualitative assessments (interviews and FGDs). We will conduct simple and multiple logistic regression analyses comparing differences in the proportions of women in reproductive age and children under five at baseline and end line for variables like antenatal coverage, vaccination coverage, skilled birth attendance, among others, (see figure 1 for key indicators). These analyses will control for any differences in the samples (if any) at the two time points as well as the contribution of contextual factors that may have occurred in the course of the implementation. mPAMANECH data: Descriptive data in terms of referrals by diagnostic decisions, among others, will be summarised using means and SD for the parametric continuous data and medians with interquartile ranges for the non-parametric data. Categorical data such as referrals by age and gender will be summarised using proportions and percentages. Qualitative data: Qualitative data will be transcribed and saved in Word format. Transcribed word files will be imported into NVivo software (QSR International) for coding and further analysis. Analysis across all transcripts will be conducted using a constant comparative method to identify themes and their repetitions and variations. The analysis will also aim to identify changes (if any) in various indicators that could be attributed to the intervention. Figure 5 shows the study’s timeline. Project timeline. CHV, community health volunteer; CHW, community health worker,sCHMT sub-County Health Management Teams