Project design and technology trade-offs for implementing a large-scale sexual and reproductive health mHealth intervention: Lessons from Sierra Leone

Background: The Coronavirus 2019 (COVID-19) pandemic threatened decades of progress in sexual and reproductive health (SRH) and gender-based violence as attendance at health facilities plummeted and service uptake dwindled. Similarly, misinformation regarding COVID-19 was rife. The demographics in Sierra Leone are diverse in the education, economic, and rural/urban divide. Telecommunications coverage, phone ownership, and preference for information access medium also vary greatly in Sierra Leone. Aim: The aim of the intervention was to reach Sierra Leoneans at scale with information about SRH during the early stages of the COVID-19 pandemic. This paper presents the approach and insights from designing and implementing a large-scale mobile health (mHealth) messaging campaign. Method: Between April and July 2020, a cross-sectional multichannel SRH messaging campaign was designed and launched in Sierra Leone. Through a secondary analysis of project implementation documents and process evaluation of the messaging campaign report, the project design trade-offs and contextual factors for success were identified and documented. Result: A total of 1.16 million recorded calls were initiated and 35.46 million text messages (short message service, SMS) were sent to telecommunication subscribers through a two-phased campaign. In phase one, only 31% of the 1,093,606 automated calls to 290,000 subscribers were picked up, dropping significantly at 95% confidence level (p = 1) after each of the four weeks. In addition, the listening duration dropped by one-third when a message was repeated compared to the first 3 weeks. Lessons from phase one were used to design an SMS and radio campaign in the scale-up phase. Evidence from our analysis suggests that the successful scaling of mHealth interventions during a pandemic will benefit from formative research and depend on at least six factors, including the following: (1) the delivery channels’ selection strategy; (2) content development and scheduling; (3) the persona categorization of youths; (4) stakeholder collaboration strategies; (5) technology trade-offs; and (6) cost considerations. Discussion and Conclusion: The design and implementation of a large-scale messaging campaign is a complex endeavor that requires research, collaboration with other diverse stakeholders, and careful planning. Key success ingredients are the number of messages to be delivered, the format, cost considerations, and whether engagement is necessary. Lessons for similar low-and-middle-income countries are discussed.

The approach for designing and the deployment of the multichannel intervention is first presented, followed by the evaluation. The intervention components were discussed, including content adaptation, transcription, recording, channel selection, message scheduling, and targeting. The internal document and report reviews and evaluation were conducted after the intervention to extract learnings for future interventions. Sierra Leone covers a land area of 72,180 km2 with an estimated population of 8.4 million people (2, 24). Based on the projections of the latest United Nations data, 43% of the population resides in urban areas (2). Freetown, the capital city, is the main urban district with a population of 802,639, split between the western urban and western rural areas. The country is made up of 14 health districts. Sierra Leone is one of the least developed countries in the world. Access to healthcare is limited by the inequitable allocation of skilled healthcare workers, poor service quality, geographical barriers, and high out-of-pocket expenses for health (24). Moreover, health facilities are unevenly distributed, with referral hospitals concentrated in Freetown (25). In 2019, UNFPA conducted formative research to understand how young people use mobile phones and the best strategy for reaching young people using mobile technology with SRH messages. The report showed that phone ownership and phone type increased with education and income (20). Young people, out of school, generally used basic phones and often could not read or write. Educated young people in secondary school used a mix of smartphones and basic mobile phones and were able to read and write. Graduates generally had smartphones and were comfortable using social media. User personas were developed for the six groups of participants, as in Figure 1. The personas show the demographic information, behavior/personality/lifestyle, their level of phone use expertise, current source of sexual reproductive health information, their preferred information source, and how the intervention can potentially help. The following paragraphs describe the multiple channels for spreading SRH messages. From the Leeds EPaCCS program evaluation hierarchy, it was determined that the published formative research addressed the pre-implementation usability and technical aspects (23). Six user personas developed as part of the UNFPA WiTok intervention approach. UNFPA, United Nations Populations Fund. The UNFPA Sierra Leone country office constituted an internal multidisciplinary task team to facilitate the development and adaptation of SRH and GBV content for delivery to Sierra Leoneans. The team included technical members specializing in maternal health, family planning, gender, communications, Monitoring and Evaluation (M&E), the audiovisual, and mHealth focal points. Furthermore, the team partnered with other UN agencies (UN Women and UNICEF) and government ministries (Ministries of Gender and Children’s Affairs, and Health and Sanitation) to develop, review, and approve the messages. The team met regularly in person and later adopted WhatsApp group collaboration coupled with Zoom calls due to the COVID-19 measures. The team developed relevant messages through several brainstorming sessions around three thematic areas in line with the UNFPA core mandate, namely, GBV, FP, MH, and COVID-19. Select contents were agreed to and first developed into English master text transcripts through an iterative review process. The master transcripts were then interpreted and transcribed into the Krio language because Krio is widely spoken in Sierra Leone. We also back-translated to ensure the adequacy of the translation. The Krio messages were audio-recorded for radio and automated voice calls. The recorded messages had a 1-min restriction and were recorded in a jingle style. The English transcript master file was also used to adapt SMS format messages with the 160-character limit. The number of unique messages developed by the thematic area is shown in Table 1. Number of developed messages by thematic area. SMS, short message service; GBV, gender-based violence; MH, maternal health; FP, family planning; COVID-19, coronavirus 2019. The channels identified for the delivery of social media messages were WhatsApp groups and the UNFPA Sierra Leone Twitter account. Each message was shared every Monday, Wednesday, and Friday. On designated days, the first message was shared via UNFPA Twitter, and additional messages could be shared further thereafter. Ten team members provided a total of 39 WhatsApp groups, each with 200–250 members in Sierra Leone, where they recommended sharing the WhatsApp content. The messages were also shared through the UNFPA Facebook page on the same days. The content delivery strategy was to ensure alignment with other channels. Based on advice from the communications analyst, four radio stations were needed to cover the entire Freetown area. Freetown was chosen because it is urban and with the highest number of COVID-19 infections in the country. Each of the four radio stations played a schedule of GBV, FP, MH, and COVID-19 messages in the mornings and evenings using the specified schedules. Messages were designed and delivered to telecommunications subscribers using different strategies throughout the life of the campaign. Initially, messages targeted UNICEF U-Reporters subscribers. Subsequently, Africell telecommunications subscribers were targeted based on their network data subscription with a pre-recorded automated voice call or SMS. Africell telecommunications is one of the three main subscribers in Sierra Leone. The messaging campaign was later scaled to a nationwide SMS campaign to all 3.9 million subscribers on the network. This technical section details the technology decisions and trade-offs for IVR, voice calls, and SMS. The trade-offs were based on the cost, willingness of the telecommunications service providers, technical capacity, and security considerations. As part of our engagement, the National Telecommunication Authority of Sierra Leone (NATCOM) indicated that mobile network operators (MNOs) do not support hardware-based gateways. Further engagement with the MNOs showed that they support software-based gateway solutions only and that the leading software GSM gateway is an open source technology called Asterisk and its other licensed derivatives. Because of this, the research undertaken in March and April focused on Asterisk being open source and known in the industry (almost the only solution), and it was the natural option to investigate and consider it (26). The SMS channel has been used for SRH services with mixed success (27). Findings from our research from 2019 show that current leading SMS options for interactivity are limited to RapidPro (28) and Textit (29). They both have the same design, which includes an SMS flow designer and reporting interface. RapidPro is proprietary to UNICEF, and UNICEF has already configured the system in many of their UNICEF countries. The Textit application is licensed (proprietary) and available for anyone interested to acquire and use at a fee. They are both cloud-based services (i.e., hosted by a cloud service provider and not on UNFPA premises or Telecommunications premises). This takes away the need for hosting and server administration, as is the case for voice calls or IVR-based systems. The 67 FP messages developed in 2019 were designed into interactive flows and tested on the Textit SMS platform and uploaded on the U-Report SMS platform in November and December 2019. The message flow for the 67 SRH messages was configured on the UNICEF RapidPro platform. The main advantage of the RapidPro flow system is that it enables message personalization based on recipient engagement with the platform. The original plan was to advertise the intervention short code and allow users to opt in for in-depth SRH messages, as seen in the proposed flyer in Figure 2. The lack of dedicated staff with the requisite capacity to focus on the project resulted in little platform monitoring and content obsolescence. As a result, UNICEF disabled the controls after 30 days (starting in December 2019). Proposed flyer for advertising the short code. In addition, as part of a Memorandum of Understanding (MoU) between UNFPA and Africell, Africell delivered SMS content on behalf of UNFPA to registered subscribers through their own Africell SMS server platform. A decision was made to deliver the initial SMS messages using UNFPA as the ID instead of 2422, and this may change in the future. Content delivered on behalf of UNFPA by Africell was not interactive and were push-only-based systems. At the end of the campaign, Africell had delivered over 35.46 million FP, GBV, and MH messages nationwide using this approach. Africell emailed a monthly message delivery log with an aggregate bio-details distribution of recipients. Voice messages have been shown to be effective in increased service uptake (30, 31). Technically, using Asterisk will require technical knowledge to manage the physical server hosting, its administration, and regular content updates. Server hosting can be either of three options: the internet cloud service providers (e.g., Google, AWS, or MS Azure); in the UNFPA office; or at the premises of the telecommunications service provider. The MoU with Africell ensured that Africell hosts and manages the message delivery with spare capacity on their server. A dedicated server managed by Africell would mean the added responsibility of server content updates, updating flow appearance, and assigning permissions to UNFPA. The initial strategy was in three stages: first, to deliver recorded messages using the Africell existing system; second, to transition from an Africell temporary server to a UNFPA local server; and third, to evaluate the performance and throughput and choose the right server for the given scale desired. The first step of this process started as planned with Africell provisioning their server (spare capacity) with the UNFPA short code 2422 and delivering calls beginning 23 April 2020. However, the Africell system could not make the agreed 290,000 calls three times per week. The bottleneck meant that it took 1 week of daily calls at off-peak periods to complete the 290,000 calls (and). As a result, the efforts were discontinued after careful consideration of the technical trade-offs. The WHO and its partners have identified collaboration as critical for the success of mHealth interventions, particularly in low- and middle-income countries (32). The project was designed to reuse all existing systems from existing partners as much as possible. Stakeholder engagements were conducted with organizations that currently had an mHealth or SRH intervention or were planning one in the country. At the end of the conversation, two organizations were at the top of the collaboration and engagement list—UNICEF and the Directorate of Science, Technology and Innovation (DSTI) (33). At UNFPA, a conceptual strategy for engaging and working with both stakeholders was outlined, as in Figure 3. The Directorate of Science, Technology, and Innovation has just launched a multi-sectoral USSD platform (*468#) and was testing it (34). The engagement with UNFPA was to ensure integration and utilization for either messaging service enrollment or messaging service delivery to young people in Sierra Leone. Similarly, UNICEF has been operating the U-Report SMS platform for almost a decade in Sierra Leone. The aim of the engagement was to leverage the infrastructure, experience, and shared resources to bootstrap the project. Conceptual strategy for working with collaborating partners. The RapidPro only supported SMS in Sierra Leone at the time of the project, though plans were ongoing to integrate other telecommunications channels such as USSD and Voice. The UNFPA Sierra Leone then opted to leverage the SMS infrastructure of RapidPro to deliver essential messages to young people. In the spirit of One UN, UNFPA in 2019 signed an MoU with UNICEF to use the UNICEF’s RapidPro platform to send up to one million interactive SRH SMS messages at no cost to the almost 200,000 registered U-Reporters. Conversely, UNFPA will advertise the 2080 short code to reach 20,000 young people through school counselors, community learning centers, youth advisory panels, social media, and other channels. The message flow was then configured in RapidPro after UNICEF Sierra Leone provided access. The high-level flow for on-demand SRH messages is shown in Figure 4. Workflow coded into RapidPro for automated and personalized SMS messaging. SMS, short message service. The initial aim was to use the DSTI multi-sectoral platform for the registration of users on the RapidPro platform. As the platform was under development, and the COVID pandemic had limited ability to conduct traditional awareness to drive enrollment and uptake, this option was not used. In addition, according to a recent tweet by the DSTI on 29 September 2022, a milestone 254,669 service usage was recorded on the *468# platform, mainly driven by use for West African Senior School Certificate Exams (WASSCE) results (34). The GSM Association (GSMA), in their eight-country mNutrition intervention in 2018, collaborated with local telecommunications providers in the project countries (35). There are two main telecommunications service providers in Sierra Leone: Africell and Orange (formerly Airtel). Both have significant telecommunications infrastructure investment and user base. In March 2020, UNFPA reached out to Africell and Orange telecommunications requesting to collaborate on the WiTok mHealth campaign intervention. Only Africell responded, and subsequently, based on several discussions, an MoU was executed between UNFPA and Africell. Based on the MoU, Africell made available an SMS, and IVR short code (2422), for text and voice content transmission (send and receive) on the Africell network. The MoU required the delivery of SMS to 355,000 subscribers (internet users) and recorded voice messages to 290,000 subscribers (non-Internet users) in the first instance. These numbers were arrived at in discussion with Africell and leveraging the outcome of our 2019 formative research. The 290,000 subscribers represent those covered by the Freetown cell coverage who have never registered for any internet bundle. Similarly, the 355,000 represent the subscribers in Freetown who have subscribed to an internet bundle once in the 90 days before the query. Africell also offered to make their studio available for recording and provide UNFPA Asterisk server hosting for free. Africell agreed to deliver pre-recorded IVR-style messages to segments (by region or other metrics) or a percentage of their user base. Each voice message will be between 30 s and 1 min. Similarly, each SMS message will be 160 characters or less. UNFPA will pay an agreed lump sum every month for the invoicing service. Africell telecommunications will cover the monthly invoiced costs in excess of this amount. Under the agreement, three calls will be scheduled to 290,000 subscribers per week and three SMS messages to 355,000 subscribers per week. Subsequently, the MoU was extended to deliver three SMS messages per week to all 3.9 million Africell subscribers nationwide. The UNFPA Sierra Leone country office constituted an internal multidisciplinary task team to facilitate the development and adaptation of SRH and GBV content for delivery to Sierra Leoneans. The project approach was different from the traditional interview approach (36). In April 2020, an audiovisual consultant was engaged to support the interpretation and subsequent recording of translated messages from English transcripts to Krio audios for use as telecommunications voice messages and radio messages. A WiTok content team was inaugurated with technical members from MH, FP, and gender-based violence. In the task team were also the communications analyst, the M&E analyst, the audiovisual consultant, and the mHealth consultant. The group started meeting regularly and then extended to online sessions due to the COVID-19 crisis. The team used WhatsApp extensively for online collaborations in April and May 2020. The content team developed English and Krio transcripts of GBV, FP, MH, and COVID-19 messages grouped for radio, telecommunication voice, and SMS (see Appendix 1). The radio messages were recorded in the Krio language, just like the telecommunications voice messages. The content audiovisual consultant working with a team of local experts recorded the messages through an iterative process. The recorded messages had a 1-min time limit and were recorded in a jingle style. The number of messages per health thematic area per delivery channel is illustrated in Figure 5. The voice messages for telecommunication calls were adapted with multimedia content for delivery via social media (i.e., Twitter, WhatsApp, and Facebook). Number of messages by health thematic technical area and delivery channel. The strategy for content delivery was initially to send messages on GBV, FP, MH, and COVID-19 on alternate days synchronously across all channels in the first phase. However, a limitation in delivering all 290,000 calls a day on the Africell network forced a change in strategy. The strategy for the subsequent week was adjusted to delivering content per health thematic (GBV, MH, FP) area per week to mobile phone users weekly. The first health thematic area was GBV, followed by MH, followed by FP. The messages were targeted to clients based on their network data subscription status. It was untargeted based on lifestyle, gender, education, or economic status. Table 2 details the message schedule for the different health thematic areas and the content delivery channels (37). The messages sent are in Appendix 1. Message schedules. IVR, interactive voice response; SMS, short message service; GBV, gender-based violence; MH, maternal health; FP, family planning; COVID, coronavirus. The schedule for radio continued with a mix of all messages daily except Sunday. The aim was that messaging would start the week of 21 April 2020. The proposed schedule is shown in Table 3. Four radio stations for disseminating the WiTok recorded contents. SLBC, SL Broadcasting Cooperation. The second part of the methodology involved a combination of observational study and case study documentation detailing the methodology and approach leveraged in the deployment of a large-scale messaging intervention. A secondary analysis and process evaluation of project implementation documents and reports aim to understand the design trade-offs and contextual determinants of the successful scaling of a multichannel digital intervention during early stages of COVID-19 pandemic. The list of reviewed documents is shown in Table 4. Documents reviewed. UNFPA, United Nations Populations Fund; GBV, gender-based violence. According to the WHO classification of digital health intervention version 1, our messaging campaign targeted at potential or current end users of health services in Sierra Leone can be categorized as “intervention for clients” (38). According to the classification, the caregivers of clients who receive service fall within these categories. The other three areas are interventions targeting healthcare professionals, health systems managers, and data services interventions. The intervention whose process is evaluated delivered targeted and untargeted health information to clients in Sierra Leone. Multiple datasets, including quantitative data extracted from project implementation documents and reports of project evaluation, were repeatedly analyzed and triangulated to facilitate a better understanding of the determinants of the successful scaling of the interventions. A framework approach was used for the data analysis while allowing for the emergence of new themes. A framework analysis involves the stages of familiarization with data, coding, indexing, charting, mapping, and interpretation (39). Manual data analyses of project implementation documents and the reports of process evaluation were led by EC and SG. All three authors approved the analysis following reviews of extracts to facilitate immersion in the data to identify factors for scaling successful interventions and other contextual factors that shaped project results. Inductive coding was used to understand how the designed messaging interventions improved the health-seeking behavior of recipients (40) and increased the utilization of health services in urban and rural areas. Deductive coding was used identify contextual factors that influenced intervention scale-up.

N/A

• Dima Health