Mobile Technology for Community Health in Ghana: What happens when technical functionality threatens the effectiveness of digital health programs?

Background: Despite the growing use of technology in the health sector, little evidence is available on the technological performance of mobile health programs nor on the willingness of target users to utilize these technologies as intended (behavioral performance). In this case study of the Mobile Technology for Health (MOTECH) program in Ghana, we assess the platform’s effectiveness in delivering messages, along with user response across sites in five districts from 2011 to 2014. Methods: MOTECH is comprised of “Client Data Application (CDA) which allows providers to digitize and track service delivery information for women and infants and “Mobile Midwife” (MM) which sends automated educational voice messages to the mobile phones of pregnant and postpartum women. Using a naturalist study design, we draw upon system generated data to evaluate message delivery, client engagement, and provider responsiveness to MOTECH over time and by level of facility. Results: A total of 7,370 women were enrolled in MM during pregnancy and 14,867 women were enrolled postpa1rtum. While providers were able to register and upload patient-level health information using CDA, the majority of these uploads occurred in Community-based facilities versus Health Centers. For MM, 25% or less of expected messages were received by pregnant women, despite the majority (>77%) owning a private mobile phone. While over 80% of messages received by pregnant women were listened to, postpartum rates of listening declined over time. Only 25% of pregnant women received and listened to at least 1 first trimester message. By 6-12 months postpartum, less than 6% of enrolled women were exposed to at least one message. Conclusions: Caution should be exercised in assuming that digital health programs perform as intended. Evaluations should measure the technological, behavioral, health systems, and/or community factors which may lead to breaks in the impact pathway and influence findings on effectiveness. The MOTECH platform’s technological limitations in ‘pushing’ out voice messages highlights the need for more timely use of data to mitigate delivery challenges and improve exposure to health information. Alternative message delivery channels (USSD or SMS) could improve the platform’s ability to deliver messages but may not be appropriate for illiterate users. Trial registration: Not applicable. Ghana is home to a population of almost 25 million disbursed across 216 districts in 10 administrative regions [11]. Over 40% of Ghanaians are under the age of 15 and over half live in urban areas [11]. While child mortality rates have declined by 50% since 1990, for every 1,000 live births 62 children die; 47% within the first 28 days of life [12]. Similar declines in maternal mortality have been observed over the last three decades, however, the lifetime risk of maternal death remains high at 1 in 66, and for every 100,000 live births an estimated 380 women die [12]. Observed declines in mortality have been driven by concurrent increases in the utilization of critical MNCH services, yet gaps in continuity of care during pregnancy, delivery, and postpartum persist. Despite near universal attendance of at least 1 antenatal care (ANC) session and high utilization of 4 or more ANC visits (84%), nearly 30% of deliveries are not attended by a skilled birth attendant, and nearly 20% do not receive postnatal care (PNC) [12]. Field level implementation of the MOTECH program launched in August of 2010 in Kassena-Nankana West (KNW) district in the Upper Each Region of Ghana, and replicated in 2011 in Awutu Senya District in the Central Region. In 2012, with added funding from USAID and the Bill and Melinda Gates Foundation (BMGF), MOTECH was expanded into new districts: Gomoa West in Central region, Dangme East in Greater Accra region, South Tongu in the Volta region. Administrative division of two districts occurred during implementation; effectively splitting Awutu Senya into Awutu Senya East and West and Dangme East into Ada East and West and raising the total number of program districts to seven across four regions. Figure 1 provides an overview of the MOTECH program and its two inter-related components: Client Data App (supply side) and Mobile Midwife (demand side). The Client Data App consists of simplified digital and paper registers consolidating information previously collected in over a dozen paper based registers in Health Centers and Community-based Health Planning and Services (CHPS) facilities. The Client Data App is a supply side intervention which allows facility and community based providers to record all care provided into five simplified paper registers, and digitize clinical care information pertaining to MNCH and other essential care to better track and deliver care to women and infants. Frontline health workers record patient care data onto mobile devices and upload these data to the server (Additional file 1: Figure S1). Data collected by health workers using Client Data App are uploaded into MOTECH’s central database and cross-checked against GHS guidelines on routine care needs for pregnant women, infants, and lactating mothers to trigger a system of alerts about upcoming and missed care sent to both clients in Mobile Midwife program and health workers [13]. These alerts are sent as weekly short message service (SMS) lists containing Mobile Midwife clients with their IDs and care they require. Field teams, composed of Grameen Foundation and district health data staff, routinely monitored and scored health facilities on the accuracy and completion of clinical information uploaded into the MOTECH server compared to care captured in the paper-based registers. Facilities in MOTECH districts that attained 85% completion and accuracy ratings for reporting for three consecutive months were eligible to be ‘automated.’ Automated facilities received auto-generated monthly reports containing aggregate care. This threshold for automation was determined in collaboration with GHS, and based on government standards for data quality. Mobile Midwife is a demand side component of the MOTECH program which aims to improve client knowledge and awareness of key health information during pregnancy and postpartum period, with the goal of stimulating best practices and encouraging timely and appropriate service utilization. Mobile Midwife is comprised of a maximum of 88 stage-based educational messages and care alerts for pregnant and postpartum women timed to their gestational age or age of their infants, respectively. Educational content was developed based on global and national MNCH guidelines and varied slightly between districts in order to debunk local cultural beliefs and practices. Clients received messages as automated voice recordings in local languages or SMS messages at a day and time of their choosing every week. The voice messaging service was persistent. A weekly message was resent to a registered phone number if a client listened to the call for less than 30 s, either because they missed the call, the device was powered off, there were call congestion issues during message delivery, or if the mobile network was temporarily down. In this case, the system was designed to send that message multiple times over the subsequent two hours, and once every other day until the next message is due. Mobile Midwife and Client Data App are underpinned by a larger technology infrastructure which includes platform support and server hosting. The MOTECH platform is based on a modular, extensible open-source software comprised of a core platform and several modules, which were developed in 2009 and upgraded in 2012. Specific details on system architecture and evolution over time are available elsewhere, including security aspects and available modules [13, 14]. The software development responsibilities were split between two teams: the server-side components were designed by a group from the University of Southern Maine, and the mobile phone components were designed by a young company based in Ghana [13]. Following the initial development of the overall system architecture, the platform and server hosting were maintained at the Grameen Foundation offices in Accra, Ghana. The MOTECH program was developed and implemented in collaboration with GHS, regional and District Health and Management Teams (DHMT). Table 1 summarizes program activities and inputs required to develop, startup and sustain implementation of the MOTECH program, including Mobile Midwife and Client Data App. To initiate project activities at a district level, extensive profiling was first undertaken to gather data on the health systems and telecommunications infrastructure. Content initially developed at a national level was localized to specific regional deployments and mobile phones1 were procured (Nokia 1680c-2, Nokia 2330c-2, Nokia C1-01, Nokia Asha 200, Nokia 2330c-2 s) for facility-based providers. Orientation of district-level leadership and facility-based provider training was undertaken. As part of Mobile Midwife, community health volunteers and facility level providers2 were trained to register clients to receive stage-based actionable advice as well as service delivery alerts and reminders. Training in the use of Client Data App included DHMT, regional, and national level staff as well as facility-based providers in 69 CHPS facilities and 23 Health Centers/Hospitals. Trainings were followed by a three-month practice interval which allowed providers time to practice, adjust to the five simplified paper registers, and acclimate to data entry and uploading on mobile phones. To facilitate the trouble-shooting of technical and/or programmatic issues, a customer support service was established and maintained at the Grameen Foundation offices in Accra, Ghana. This service provided support for health workers using Client Data App, and tracked overall data uploads, and flagged changes in data upload trends.3 Once facility-based providers demonstrated proficiency in using MOTECH’s Client Data App, Mobile Midwife activities were initiated. At a community level, district launch events inclusive of key political figures, religious leaders, health workers and community members were held through durbars (festivals) and other marketing events to promote the program and encourage registration. Posters and murals were additionally installed on the exterior walls of community structures to raise awareness and promote registration into Mobile Midwife. Description of MOTECH program activities and inputs *Modified from Willcox M et al. 2017 (Willcox M, et al. Is Mobile Technology for Community Health good value for money? Evidence on the cost effectiveness of mobile health in Ghana. Submitted for publication) Following start-up activities, health facilities began patient registration and enrollment into MOTECH. Providers were encouraged to enter and upload patient level information on a daily basis to facilitate the sending of alerts and reminders. In two districts (Awutu Senya East and Awutu Senya West), high performing facilities were automated and thus given the opportunity to receive monthly feedback reports from Grameen Foundation, summarizing service delivery statistics. To support facility-based activities routine district level meetings were held and supervision visits made to support facility level providers. Where possible, DHMT members sought to include program tasks in their current workflow. However, the Grameen Foundation provided additional program support including device replacement, refresher training, and routine data entry, management and feedback. Using a naturalistic study design, this case study sought to describe variation across study sites over time in the receipt of and engagement with health information messages [15]. Figure 2 presents our conceptual framework for assessing the optimal pathway between health care providers who identify and register clients; the technological platform that receives and sends health information messages as well as alerts and reminders for care; end-users who receive and access messages; service delivery and careseeking; and ultimately, improved health status. In practice, multiple breaks are likely to have occurred at each point along this pathway as a result of technological (network coverage, feasibility/usability of the device, size of the data bundle, phone functionality/access); behavioral (user’s willingness to listen to/access messages, motivation/satisfaction); health systems (workload, organizational and structural inputs); and/or community constraints (financial/physical barriers to care, social norms, power dynamics). In this analysis, we draw upon system generated data to explore the chain of events denoted by a dotted arrow linking provider registration, message delivery, and receipt. Measuring program fidelity: was the program delivered as it was intended?. The dotted line denotes the pathway assessed as part of this manuscript. Yellow boxes denote factors which are influenced by health systems and/or providers, the light green represents technological factors, and the light blue community/client level factors Technological performance was assessed by exploring the MOTECH platform’s effectiveness in ‘pushing’ out messages across the continuum of care and by content area to registered women. Technological indicators included the proportion of messages ‘pushed out’ out of the total number of messages expected to be ‘pushed’ (Additional file 1: Figure S1 and Additional file 2: Table S1). Behavioral performance was assessed by capturing trends in provider and client engagement with Client Data App and MM, respectively. To measure provider engagement with Client Data App and ascertain the broader feasibility of recording patient-level health information, we assessed the frequency and volume of data uploads over time and by level of facility. To explore client engagement, we identified the proportion of and characteristics of message recipients who listened to at least 50% the length of each message received (defined as ‘active listeners’). Data from Mobile Midwife and Client Data App were obtained from MOTECH server housed at Grameen Foundation offices in Accra, Ghana. Data elements specific to Client Data App included health facility data uploads, while data from Mobile Midwife included data on health information messages, including messages sent and accessed by pregnant women and mothers of infants. Messaging access was defined by the number of clients who ‘listened’ to stage-based health information messages successfully ‘pushed out.’ Since sharing of phones and phone numbers is common practice in Ghana, many clients were listed under the same phone numbers. For the purposes of this analysis, we extracted details of ‘listening’ for unique phone numbers to facilitate client level analysis. We further excluded registered women who did not have their own or have access to a mobile phone in their household and insteadhad to rely on a community phone to call into the system to retrieve their weekly messages. As part of the Client Data App, facility based providers are required to upload data on the careseeking practices of women registered into MM. Data on the number of uploads per month were analyzed using proportions and frequencies. We used confidence intervals at the 95% level to assess statistical differences in rates of active listening across thematic content areas. Data were analysed using Stata 13.2 for five districts: Awutu Senya West, Awutu Senya East, Gomoa West, Ada East and Ada West. The timeline for implementation varied across sites, and spanned from July 2011 to September 2014 overall.