Impact of using eHealth tools to extend health services to rural areas of Nigeria: Protocol for a mixed-method, non-randomised cluster trial

listen audio

Study Justification:
– eHealth solutions using internet and related technologies can enhance healthcare delivery in resource-limited countries like Nigeria.
– Little is known about the effectiveness of eHealth tools in improving health systems and client outcomes in these countries.
– This study aims to evaluate the impact of eHealth tools on health system functions and health outcomes in rural areas of Nigeria.
Highlights:
– The study will use a mixed-method, non-randomized cluster trial design.
– It will evaluate the process, contextual influences, and impact of eHealth interventions.
– Multiple quantitative and qualitative datasets will be analyzed to understand the effects of eHealth tools on health worker knowledge, health system quality and efficiency, and client outcomes.
– The study has obtained ethics approvals and will ensure data confidentiality.
– Findings will be presented to Ministries of Health and published in peer-reviewed journals.
Recommendations:
– The study recommends the use of eHealth tools, such as video training and digitization of health data, to extend health services in rural areas of Nigeria.
– It suggests ongoing technical support and training to ensure the functionality and capacity of health workers.
– The study highlights the importance of considering contextual factors and conducting comparative analysis to understand the effectiveness of eHealth interventions.
Key Role Players:
– ‘InStrat’ will implement the eHealth interventions in collaboration with State Ministries of Health in Ondo and Kano States and the Department of Health in the Federal Capital Territory.
– University teams in Nigeria will collect and analyze data.
– The University of Leeds will provide support, training, and quality assurance for the study.
Cost Items for Planning Recommendations:
– Tablet computers loaded with data plans for primary healthcare facilities.
– Broadband global area network link-based SatCom hardware for satellite connectivity in some facilities.
– Ongoing technical support and training for health workers.
– Data collection and analysis by university teams.
– Secure online portals for data sharing and storage.
– Communication and dissemination of study findings through scientific conferences, peer-reviewed journals, and plain language summaries for patients and policy-makers.
Please note that the provided information is based on the given text and may not include all details of the study.

The strength of evidence for this abstract is 7 out of 10.
The evidence in the abstract is strong, but there are some areas for improvement. The study design is a non-randomized cluster trial, which may limit the ability to establish causality. To improve the evidence, the researchers could consider implementing a randomized controlled trial design, which would provide stronger evidence. Additionally, the abstract does not provide information on the sample size or power analysis, which are important factors in determining the strength of the evidence. Including this information would enhance the rating of the evidence.

Introduction eHealth solutions that use internet and related technologies to deliver and enhance health services and information are emerging as novel approaches to support healthcare delivery in sub-Saharan Africa. Using digital technology in this way can support cost-effectiveness of care delivery and extend the reach of services to remote locations. Despite the burgeoning literature on eHealth approaches, little is known about the effectiveness of eHealth tools for improving the quality and efficiency of health systems functions or client outcomes in resource-limited countries. eHealth tools including satellite communications are currently being implemented at scale, to extend health services to rural areas of Nigeria, in Ondo and Kano States and the Federal Capital Territory. This paper shares the protocol for a 2-year project (€EXTEND’) that aims to evaluate the impact of eHealth tools on health system functions and health outcomes. Methodology and analysis This multisite, mixed-method evaluation includes a non-randomised, cluster trial design. The study comprises three phases – baseline, midline and endline evaluations – that involve: (1) process evaluation of video training and digitisation of health data interventions; (2) evaluation of contextual influences on the implementation of interventions; and (3) impact evaluation of results of the project. A convergent mixed-method model will be adopted to allow integration of quantitative and qualitative findings to achieve study objectives. Multiple quantitative and qualitative datasets will be repeatedly analysed and triangulated to facilitate better understanding of impact of eHealth tools on health worker knowledge, quality and efficiency of health systems and client outcomes. Ethics and dissemination Ethics approvals were obtained from the University of Leeds and three States’ Ministries of Health in Nigeria. All data collected for this study will be anonymised and reports will not contain information that could identify respondents. Study findings will be presented to Ministries of Health at scientific conferences and published in peer-reviewed journals. Trial registration number ISRCTN32105372; Pre-results.

The eHealth interventions will be implemented by ‘InStrat’ from March 2017 to March 2019, in collaboration with the State Ministries of Health in Ondo and Kano States and the DOH in the FCT. A successful pilot testing of VTR and CliniPAK apps in Ondo State in 2016 led to scaling up of eHealth interventions to Kano state and the FCT in 2017. In this evaluation study, we have selected two clusters in each state corresponding to LGAs: one LGA with facilities implementing VTR and CliniPAK tools, and the other LGA with facilities not implementing any eHealth intervention. The ‘intervention’ LGAs will be assessed against non-intervention LGAs. Intervention LGAs (see table 1) were selected because they had many primary healthcare facilities situated in areas without access to regular mobile network service. Intervention and control local government areas (LGAs) selected by state SatCom, satellite communication. A total of 126 PHC facilities in intervention LGAs across the three states have, since April 2017, been incrementally supplied with tablet computers loaded with data plans to enable the VTR and CliniPAK interventions. Health workers in these PHC facilities were then trained by InStrat staff to use the tablets. See table 2 for a description of VTR and CliniPAK interventions. Moreover, 75 SatCom facilities in intervention LGAs will be supplied with a broadband global area network link-based SatCom hardware, to enable internet connectivity in the PHC facilities. The remaining 51 non-SatCom facilities in intervention LGAs are already connected via regular terrestrial mobile network operators and so do not require linking via broadband global area network link-based SatCom hardware. Beyond the training to enable staff to use the tablets, InStrat staff will provide ongoing technical support to ensure that SatCom and tablets continue to function and that FHWs capacity is maintained despite attrition. Overview of eHealth tools FHW, frontline health worker; PHC, primary healthcare. The target population for this evaluation study comprise three groups: (1) FHWs and facility heads at intervention PHC facilities; (2) pregnant women at participating PHC facilities and (3) policy-makers. The FHWs will include nurses, midwives, laboratory technicians and community health workers. The study will use a mixed-method design to evaluate the acceptability and effects of novel eHealth tools implemented across three states of Nigeria. The quantitative part of the study will use a non-randomised cluster trial design, collecting longitudinal data before and after the implementation of eHealth tools in intervention facilities to compare with longitudinal National Health Management Information System (NHMIS) data in non-intervention facilities to understand the impact of the eHealth tools on health systems functions and health outcomes. The quantitative arm will also assess the impact of eHealth tools on FHWs’ knowledge in intervention sites only. The qualitative part of the study will enable: (1) process and impact evaluations of satellite connectivity and the scaled-up VTR and CliniPAK interventions in the three states; (2) evaluation of the influence of contextual factors on the implementation of the interventions. Implementation in states from different regions of Nigeria (Ondo in west, FCT in middle belt and Kano in the north) facilitates the examination of different contextual factors that may affect implementation and project outcomes. To assess the relationship between project inputs, processes of implementation and outcomes, we will use the framework in figure 1 to conceptually explore how inputs lead to processes, how processes lead to outputs and how outputs contribute to outcomes and impact.20 Given the significance of context to attaining project results, we will examine the roles of SatCom, VTR and CliniPAK interventions in achieving project effects within a wider context, rather than ascribing changes in results and outcomes to our project alone. To achieve this, we will use insight from analysis of documents review and qualitative interviews (see the ‘Methods of data collection’ section) to assess whether/how the ‘context of implementation’ of the project affects project results. For example, though figure 1 depicts linear and simplified relationships between inputs, activities, outputs and outcomes of the project, we acknowledge that the study findings can be influenced by competing/concurrent maternal and child health interventions in either the intervention or control arm of the study (or both) that were unknown or unanticipated at the time of developing the protocol. We also acknowledge that there can be unintended positive or negative consequences of our interventions that are not currently mentioned in this protocol. Conceptual framework for EXTEND project, Nigeria. CliniPAK, Clinical Patient Administration Kit; HWs, health workers; PHC, primary healthcare; VTR, video training. Three hundred FHWs in 126 intervention sites will be selected to participate in CliniPAK and VTR interventions based on lessons from pilot testing in 2016, alongside resource and logistical feasibility considerations. This will comprise 200 FHWs in 75 SatCom facilities and 100 FHWs in 51 non-SatCom facilities (ie, three from each SatCom facility and two from each mobile network-enabled facility). As part of their orientation, the objectives of the EXTEND project will be explained to FHWs in intervention sites. To minimise possibility of coercion, FHWs at intervention sites will then be approached by a member of the research team a week after their orientation and invited to participate in the project. FHWs who agree to participate in the project will be trained to use the CliniPAK app for the daily documentation of MNCH care. For VTR intervention, participants will be provided with login instructions for completing a pretutorial survey, reviewing an electronic tutorial (see table 2), and completing a post-tutorial survey. Participants will complete a consent form prior to participation. The project plans to provide 4–6 monthly refresher of VTR modules to encourage FHW retention in the study. The evaluation will comprise three phases: baseline assessment within 3 months of start of project, midline assessment at 12 months (March 2018) and endline evaluation at 24 months (March 2019). Baseline assessment was conducted from 23 May to 30 June 2017 in intervention and control sites to ascertain the status of target key performance indicators (KPIs), before full implementation of the project, and involved assessment of three types of data: These phases will compare results with the baseline assessment. During each of these phases, a multidimensional approach will be adopted comprising: A project plan is shown in figure 2. All data collection and analysis during baseline, midline and endline assessments will be done by in-country university partners. Following baseline assessment, the three Nigerian universities produced state-level reports integrating quantitative and qualitative findings for the three states and these formed the dataset for a country baseline report. This approach to reporting will be repeated for midline and endline evaluations, respectively, to make sense of the effects and impacts of eHealth interventions. Project work plan. CliniPAK, Clinical Patient Administration Kit; FHWs, frontline health workers; KPIs, key performance indicators; MNCH, maternal, newborn and child health; PHC, primary healthcare; SatCom, satellite communication; VTR, video training. Additionally, a routine monitoring exercise, led by the University of Leeds, will run alongside the three phases of the study to provide quality assurance for the study. Interim evaluation of data quality (completeness and accuracy) will occur following monthly collation of routine NHMIS data alongside data from CliniPAK and VTR, collected by in-country teams. Monthly data will be collated into quarterly reports and submitted in aggregate form for audit by the University of Leeds monitoring team. This will support identification of discrepancies or irregularities in reporting and facilitate periodic performance reviews against KPIs. While monitoring will occur independently, findings from interim evaluations will be shared with study sponsors to inform project monitoring. To facilitate monitoring and ensure standardisation and consistency of reporting across the three states, a project logical framework (logframe) has been developed, outlining the project’s targets, indicators and means of verification of data collected by PHC facilities in the three states, to track progress towards meeting outputs, outcomes and potential impacts of the projects (see online supplementary appendix 1). In addition to the logframe, we have developed a defined set of KPIs to measure performance of against operational criteria (see online supplementary appendix 2). The KPIs will be monitored through periodic performance reviews and within baseline, midline and endline evaluation. bmjopen-2018-022174supp001.pdf bmjopen-2018-022174supp002.pdf The primary outcome for the trial is a binary facility-level indicator measuring whether the monthly NHMIS indicator ‘total number of ANC visits’ is complete (ie, available through the NHMIS) for every month of the 6-month postintervention period. The secondary outcomes are: (1) binary facility-level indicators of whether the monthly NHMIS indicators ‘total PNC visits’ and ‘percentage skilled birth attendance’ are complete or not for every month of the 6-month postintervention period; and (2) the NHMIS indicators ‘total number of ANC visits’, ‘total number of PNC visits’ and ‘percentage of skilled birth attendance’. For the non-randomised trial based on available resources, we will have 6 clusters in the intervention arm and 5 in the control arm, having a mean cluster size (number of facilities) of 25 and a cluster-size variance of 23. Based on preintervention data, for the primary outcome, we assumed an existing proportion in both arms of 0.18 and an intracluster correlation coefficient of 0.025. Using two-tailed testing at the 5% significance level, this allows us to detect an absolute reduction in the intervention arm to ≤0.01 with >80% power.21 We will analyse the primary outcome, adjusted for covariates, using a two-stage method that accounts for between-cluster variation and is appropriate for cluster trials with relatively few clusters per arm.21 First, we will use a logistic regression model of the primary outcome including our covariates of interest, but excluding the treatment effect, to compute a difference residual for each cluster. Second, we will estimate the intervention effect as the absolute difference in the primary outcome (intervention minus control), and base our inference on the associated (t-statistic based) 95% CIs and p value (two-sided, 5% level of significance). We will analyse all secondary outcomes related to NHMIS indicator data completeness using the same methods. All results will be adjusted for the baseline level of the relevant outcome, calculated as the facility-level proportion of data completeness for the monthly relevant outcome as collected over the 12 months prior to the implementation of the intervention, and for LGA. We will use controlled interrupted time-series analysis to analyse whether there have been any changes in the levels and/or trends of all NHMIS indicators following implementation of the intervention. For all these monthly indicators, we will have 12 months’ worth of preintervention data and 6 months’ worth of postintervention data for both intervention and control clusters. We will analyse all NHMIS indicators, aggregated at the LGA level, using a linear regression model including a time × treatment × period (preintervention vs postintervention) interaction to provide estimates of the changes in level and trend of outcomes before and after the intervention period. If model errors display non-negligible autocorrelation, this will be accounted for using by fitting a generalised least squares model adjusting for AR(1) errors. The models will include a random intercept for individual, potentially nested within a random intercept for facility if necessary. We will estimate the mean change in knowledge score percentage points based on the coefficient for a fixed effect of test time (post-test vs pretest). We will also control for a range of likely influential and potentially confounding covariates: age, sex, staff level (community health extension worker or nurse/midwife), facility type (basic or comprehensive) and state (FCT, Kano or Ondo). We will also explore whether any changes in knowledge scores differ between the following subgroups: (1) FHWs at SatCom vs non-SatCom sites, (2) FHWs at basic versus comprehensive facilities, (3) community health extension workers versus nurses/midwives, and (4) male versus female FHWs. We will again use linear mixed models (including the above covariates) to analyse changes in knowledge scores for each subgroup, and separate linear mixed models (including the above covariates) with an interaction between test time and the relevant subgroup indicator variable to provide estimates of any differences in change in knowledge scores between the subgroup comparisons listed. All inferences will be based on the associated (t-statistic based) 95% CI and two-sided p value (5% significance level) for the relevant coefficients. During each phase of the project, IDIs with policy-makers, facility heads, FHWs and service users will be audio recorded (subject to informed consent), transcribed and where appropriate translated into English for manual data analysis. Framework approach will be used for understanding the impact of eHealth interventions on health system functions, while allowing for emergence of new themes. The framework approach includes the stages of familiarisation with data, coding, indexing and charting, mapping and interpretation of data.22 Quantitative and qualitative findings will be integrated and triangulated to answer the research questions. Furthermore, we will conduct a comparative analysis of variations in adoption and effectiveness of eHealth innovations in the three states to ascertain the influence of contextual factors on processes of implementation and project outcomes. The two datasets will be repeatedly triangulated especially during the midline and endline evaluations to understand the impact of interventions on health systems functions and health outcomes. The project will be conducted with full respect for relevant legislations (eg, the Charter of Fundamental Rights of the European Union) and international conventions (eg, Helsinki Declaration). Data collection and analysis will take account of four key issues: The University of Leeds team will compile and analyse data collected by university teams in Nigeria and support their training, including providing information on protocols for anonymising and securely sharing study data. Data will be shared using online secure portals and will be stored with passwords and access only made available to data for those directly involved in data analysis. All transcripts from the study will be anonymised prior to sharing with the Leeds team. We will preserve the anonymity of study participants at all times. Unnecessary collection of personal data will be avoided, and respondents will have the right to review outputs and withdraw consent. Where personal data are collected (eg, age, sex, level of education), it will be coded, removed from the data for analysis and stored separately. Only designated project staff will have access to the keys linking the data with the personal information. We will routinely review the independence of the research team when undertaking monitoring work. While working closely with partners in the consortium, we will ensure that we are free of influence over the judgements relating to the evaluation. A comprehensive evaluation framework has been developed to direct data collection in the study. The project will also develop a shared online platform to facilitate data capture and reporting of variables for monitoring KPIs across project sites. The framework is impartial to any group and inclusive of all groups. The project will be implemented according to standard governance practice at the University of Leeds for the implementation of collaborative projects. This includes ensuring regular communication between partners and engagement with policy-makers and practitioners; quality assurance through regular peer-review both within and between teams; appropriate mentoring and coaching support of junior researchers. Improving MNCH knowledge and practice is a national and international priority. This initial scale-up of eHealth interventions to the FCT, Ondo and Kano states will be further expanded to other states of Nigeria and to non-health sectors (education, agriculture and civic identity management). The high demand for this study from policy-makers and funders provides an excellent opportunity to ensure uptake of high-quality evidence into policy and practice. Specific methods of communicating study findings include a combination of the following: We will ‘embed’ the research strategy development and assessment into policy and practice, working with the Federal Ministry of Health and State Ministries of Health in Ondo and Kano States and the DOH in the FCT. This embedded approach, developed by the Nuffield Centre of the University of Leeds, has been used in many countries to improve the quality and effectiveness of scaled-up programmes.23–25 We will engage decision makers throughout the process in a research-policy partnership to facilitate adoption and scale-up of eHealth tools to other states in Nigeria.26 Patients were not involved in the development or design of the study. We will work with patient advocacy groups to ensure that plain language summaries of study findings are shared to both participating service users and wider patient groups.

Based on the provided information, the following innovations can be recommended to improve access to maternal health:

1. eHealth Tools: The use of internet and related technologies, such as mobile apps and satellite communications, can extend health services to remote areas of Nigeria. These tools can support cost-effective care delivery and improve access to maternal health services.

2. Video Training: Implementing video training programs can enhance the knowledge and skills of frontline health workers in maternal health care. This can be done through interactive tutorials and online courses, which can be accessed remotely.

3. Digitization of Health Data: Digitizing health data can improve the efficiency and accuracy of data collection, storage, and analysis. This can help in monitoring and evaluating maternal health outcomes, as well as identifying areas for improvement.

4. Satellite Connectivity: Providing broadband internet connectivity to primary healthcare facilities in rural areas through satellite communication can enable access to online resources, telemedicine consultations, and real-time communication with healthcare professionals.

5. Clinical Patient Administration Kit (CliniPAK): Using mobile apps like CliniPAK can streamline the documentation and management of maternal and child health care. This can improve the quality of care, reduce errors, and enhance communication between healthcare providers.

6. Ongoing Technical Support: Ensuring ongoing technical support for eHealth tools and infrastructure is crucial to maintain their functionality and address any issues that may arise. This can be done through training programs and regular monitoring and maintenance of the technology.

7. Collaboration with Stakeholders: Collaborating with state ministries of health, policy-makers, and other relevant stakeholders is essential for the successful implementation and scale-up of eHealth interventions. This can help in aligning the interventions with existing healthcare systems and policies.

8. Dissemination of Findings: Sharing the findings of the evaluation study with ministries of health, scientific conferences, and peer-reviewed journals can contribute to evidence-based decision-making and promote the adoption of eHealth tools in maternal health care.

It is important to note that these recommendations are based on the specific context and objectives of the mentioned study. The implementation of these innovations should be tailored to the local context and needs of the target population.
AI Innovations Description
The recommendation to improve access to maternal health is to implement eHealth tools, such as video training and digitization of health data interventions, in rural areas of Nigeria. These tools utilize internet and related technologies to deliver and enhance health services and information. By using digital technology, eHealth solutions can improve the cost-effectiveness of care delivery and extend the reach of services to remote locations.

The specific eHealth interventions being implemented in this project are the VTR (Video Training) and CliniPAK apps. These interventions involve providing tablet computers loaded with data plans to primary healthcare facilities in intervention areas. Health workers in these facilities are then trained to use the tablets for various purposes, such as daily documentation of maternal, newborn, and child health (MNCH) care.

The project will evaluate the impact of these eHealth tools on health system functions and health outcomes through a mixed-method, non-randomized cluster trial design. The evaluation will include baseline, midline, and endline assessments to measure the effectiveness of the interventions. Quantitative and qualitative data will be collected and analyzed to understand the impact on health worker knowledge, quality and efficiency of health systems, and client outcomes.

Ethics approvals have been obtained for the study, and data collected will be anonymized to protect the privacy of participants. Study findings will be presented to Ministries of Health and published in peer-reviewed journals to disseminate the results and inform policy and practice.

Overall, implementing eHealth tools in rural areas of Nigeria has the potential to improve access to maternal health services and enhance the quality and efficiency of healthcare delivery.
AI Innovations Methodology
The study described in the provided text aims to evaluate the impact of eHealth tools on health system functions and health outcomes in Nigeria, specifically in the areas of maternal, newborn, and child health (MNCH). The eHealth interventions being implemented include the use of video training and digitization of health data, as well as the provision of satellite communication (SatCom) and tablet computers loaded with data plans to primary healthcare facilities.

To simulate the impact of these recommendations on improving access to maternal health, the study will use a mixed-method, non-randomized cluster trial design. The evaluation will involve three phases: baseline, midline, and endline assessments. The study will collect both quantitative and qualitative data to assess the effectiveness of the eHealth tools.

The quantitative part of the study will use a non-randomized cluster trial design, comparing intervention facilities (those implementing eHealth tools) with control facilities (those not implementing any eHealth intervention). The study will collect longitudinal data before and after the implementation of eHealth tools to compare with National Health Management Information System (NHMIS) data in non-intervention facilities. The primary outcome measure will be the completeness of monthly NHMIS indicators related to antenatal care (ANC) visits, postnatal care (PNC) visits, and skilled birth attendance. Secondary outcomes will include other NHMIS indicators related to ANC visits, PNC visits, and skilled birth attendance.

The qualitative part of the study will involve interviews with policy-makers, facility heads, frontline health workers (FHWs), and service users. These interviews will be audio recorded, transcribed, and analyzed using a framework approach. The qualitative data will help evaluate the impact of eHealth interventions on health system functions and understand the influence of contextual factors on the implementation of the interventions.

Throughout the study, ethical considerations will be followed, and data will be anonymized to protect the privacy of participants. The study findings will be presented to Ministries of Health and published in peer-reviewed journals to ensure the dissemination of high-quality evidence into policy and practice.

Overall, the methodology described in the study aims to assess the impact of eHealth tools on improving access to maternal health in Nigeria through a combination of quantitative and qualitative data collection and analysis.

Share this:
Facebook
Twitter
LinkedIn
WhatsApp
Email