Menu
Menu
Menu
Menu
Background: With the growing ubiquity of smartphones and wearable devices, there is an increased potential of collecting passive sensing data in mobile health. Passive data such as physical activity, Global Positioning System (GPS), interpersonal proximity, and audio recordings can provide valuable insight into the lives of individuals. In mental health, these insights can illuminate behavioral patterns, creating exciting opportunities for mental health service providers and their clients to support pattern recognition and problem identification outside of formal sessions. In the Sensing Technologies for Maternal Depression Treatment in Low Resource Settings (StandStrong) project, our aim was to build an mHealth application to facilitate the delivery of psychological treatments by lay counselors caring for adolescent mothers with depression in Nepal. Methods: This paper describes the development of the StandStrong platform comprising the StandStrong Counselor application, and a cloud-based processing system, which can incorporate any tool that generates passive sensing data. We developed the StandStrong Counselor application that visualized passively collected GPS, proximity, and activity data. In the app, GPS data displays as heat maps, proximity data as charts showing the mother and child together or apart, and mothers’ activities as activity charts. Lay counselors can use the StandStrong application during counseling sessions to discuss mothers’ behavioral patterns and clinical progress over the course of a five-week counseling intervention. Achievement Awards based on collected data can also be automatically generated and sent to mothers. Additionally, messages can be sent from counselors to mother’s personal phones through the StandStrong platform. Discussion: The StandStrong platform has the potential to improve the quality and effectiveness of psychological services delivered by non-specialists in diverse global settings.
The StandStrong platform development occurred in the context of a study conducted in Chitwan, a southern district of Nepal, between May 2018 and October 2019. The implementation was tailored to psychological treatment for depressed adolescent and young mothers (ages 15–25 years) with infants under 12 months of age. The study protocol for implementation and evaluation of StandStrong has been published 12. Prior to the study, we developed an approach to capture passive sensing data that could be used by StandStrong platform. However, many freely available software tools, such as RADAR 13 and Passive Data Kit 14, can be used to capture passive data from smart devices. In our current study, the StandStrong platform was designed for use with android smartphones to process the data collected through a passive data collection tool, and then provide visualizations of the data for a lay counsellor delivering the psychological treatment. The platform comprises freely available components, namely, TensorFlow service (TF-Audio), Scheduler (StandStrong ELT), web services (StandStrong REST API), Viber Webhook, and the StandStrong Counsellor App. Figure 1 shows the StandStrong Architecture with server locations and data flow. Each tool’s source code and application hosted locations are summarized in the Software availability section and a full user guide is available as Extended data 15. EBM: Electronic Behavior Monitoring; DAO: Data Access Object. Rationale for selected passive sensing domains. We have included four domains that can create a picture of the daily life of the mothers. Mother-child proximity, along with audio data can capture mother-child interaction. Audio data combined with GPS is intended to eventually be a proxy for social support. Activity data will capture the physical activity of the mother as a proxy for physical health. The combined information from these passive data can be used to create a picture of the daily schedules of mothers. This can be a proxy for routinization and stability. An instability of daily schedule as determined from the passive data can be a proxy for stress. Although we are moving towards a clearer understanding of how to analyze and operationalize to maximize the benefit of mothers, the initial idea with this study was just to surface behaviors of interest. We wanted to see if it was possible to create an app and integrate passive data within the app and whether such information would inform the counselor during the session. We were mostly concerned if passive data, in general, was a feasible form of data that we could collect in a rural setting. With what we have learned and the advances in the field mean we can move beyond the exploratory, particularly in operationalizing the domains, and what integrating passive data into these behavioral patterns means for mothers. Gathering passive sending data sources. Passive sensing data can be collected from smart devices such as smartphones and smartwatches through a variety of approaches. We had collected passive sensing data such as GPS, activity, proximity, and audio. Use of additional devices such as Bluetooth beacons along with smart devices can capture proximity data. Preparing passive sensing data. For audio data, mp3 recordings are fed to the TensorFlow service to generate audio predictions such as speech, music, vehicles, insects and so on. TensorFlow is an open software framework for machine learning 16, which we trained with YouTube human and environment sound models. The Amazon Web Services (AWS) S3 bucket is used for transferring files to the inbound folder in the cloud server. The scheduler job scans the incoming files periodically and loads them into the database. The web service in Heroku cloud provides the endpoints to access the data, which the StandStrong Counselor app gets for visualization. The platform also allows sending messages to and from a counselor and a mother using the popular Viber chat app. There is a wide range of approaches that could be used for analysis of the passive sensing data, and this is a rapidly evolving field. For the purposes of this initial work, our operationalization of passive sensing results was as follows: Data visualization on the Counselor App. The StandStrong Counsellor App is an android-based mobile application that retrieves data through the web service and stores it on the device. While developing the system, one of the considerations was limited access to the internet; therefore, the app was designed using the ‘offline first’ principle. The StandStrong mobile app has the following features to provide visual representations of the passively collected data for counselor use: Proximity data ( A), Global Positioning System heat map ( B), and activity data ( C) as visualized in StandStrong app. The StandStrong platform components, web services (StandStrong REST API), StandStrong Counsellor App, and scheduler (StandStrong ELT) are available on Github for public access at https://github.com/mmhss (see Software availability). StandStrong app in counselor’s tablets. Counselors can use the StandStrong app while providing counseling services to depressed adolescent mothers. The application’s use is both preparatory (the counselor explores the mother’s behavioral patterns to inform her session) and participatory (the counselor uses the visuals of the application with the mother to help discuss, identify, and set behavioral goals). We designed StandStrong for use with Samsung Galaxy Tab A7.0, which costs around $160 USD (purchased in Sept 2018) with accessories. The StandStrong app works on most android devices. However, we recommend a screen size of seven inches, RAM 1.5 GB, and Android 6.0 or higher. Figure 2 shows the proximity chart, GPS heat map, and activity chart for the participants. The proximity chart shows the time spent by the mother alone and with her child, the GPS heat map shows the locations where the mother spends her time, and the activity chart visualizes the mother’s activities, such as standing and running, as recorded by the mobile phone throughout the day. The detailed operation of the StandStrong Counselor App is documented and provided as Extended data 15. Equipment. Three types of equipment are used in the study. Different approaches and combinations of devices are available to collect passive sensing data including phones, smartwatches, Bluetooth beacons. For StandStrong, the counselors use tablets to access the passive sensing data visualized in the StandStrong app and discuss the data with mothers in counseling sessions. Detailed specifications of the equipment can be found in Table 1. EBM, Electronic Behavior Monitoring; GPS, Global Positioning System. Below, we provide examples of input and output datasets, as well as two case studies to illuminate the platform’s architecture and pragmatic use. Use case 1 describes how StandStrong would be implemented from a project director perspective. Use case 2 describes how StandStrong is used by a lay counselor providing psychological treatment to a depressed adolescent mother. Implementation of passive sensing data collection. Our first step was to identify what types of information could be collected using passive sensing on mobile devices that would be technologically feasible and culturally acceptable in Nepal 17. Based on the feedback from users in prior studies, smartphones and Bluetooth beacons were considered the most appropriate tools for passive data collection in Nepal. In our prior study, we designed the Electronic Behavioral Monitoring (EBM) app, which can be installed on any android smartphone with Bluetooth version 4.0 to collect passive sensing data. EBM captures the four domains of data: audio recordings, physical activity, GPS location, and proximity of the phone to a passive Bluetooth beacon (attached to the infant’s clothing). To ensure affordability in a low-resource setting, we explored low cost devices that could run our applications. We chose the Samsung J2 Ace phone, which costs around $114. The EBM app collects audio data in m4a format; GPS location, activity and proximity of Bluetooth beacon are saved in CSV format on the device. During pilot testing, project staff retrieved the passive sensing data from the device once a week. Each week, around 200 MB of data are generated including seven GPS files, seven activity files, seven proximity files and 280 m4a files. Through the EBM app, GPS, activity, proximity and audio data are passively collected at 15-minute intervals on the mother’s android smartphone and the information is stored locally on the smartphone as CSV, with the exception of audio files, which are m4a format. In EBM, the mobile phone’s accelerometer sensor and Google library are required to return movement information (walking, in vehicle, cycling, running, still) to crudely track activity. Proximity data are collected each time the mobile Bluetooth searches for the Bluetooth beacon attached to the child’s clothes. An audio clip of 30 seconds is collected every 15 minutes and saved to the phone. Finally, GPS location is recorded each time the mobile phone has some activity (screen on and off, phone calls and so on). When participants are enrolled, we provide a participant code so that generated data files are prefixed with the participant code, followed by the type of the data and date. The StandStrong platform is used by the counselor to visualize these data, as well as provide automated awards to the mother if she achieves behavioral targets. Alternative passive data collections apps exist that could be used to collect the raw data for the StandStrong system. For example, the RADAR-base passive data collection app has the capability to capture data both on smartphones and smart watches 13. The app provides access to a wide variety of sensors including positioning sensors, movement sensors and social sensors supporting Bluetooth devices. Through a simple reformatting (e.g., using the StandStrong date format) collected data would be supported and usable by the StandStrong platform. Input dataset. During pilot testing, passive sensing data were collected by the EBM app, generating a separate file for each sensing data type. The naming convention for the file has several pieces of information each separated by a dash (-) and underscore (_). It starts with the mother identification code, followed by the delimiter “-” before the passive sensing data type name, followed by underscore and lastly, the date (e.g. SSXXXX-GPS_20190x0x.txt, SSXXXX-PROXIMITY_20190605.txt, SSXXXX-ACTIVITY_20190605.txt). The raw audio files captured by the EBM app are of around 30 seconds in length. On average, around 40 raw audio files (e.g. SSXXXX-EAR_201900x0x192732.m4a) are generated each day, which then are passed to the TensorFlow audio processor to generate the predictions. Tensorflow generates a CSV file (e.g. SSXXXX-AUDIO.csv) for weekly audio clips. The .csv file contains audio predictions that can be used for analysis. When using the RADAR-base passive data collection app as an alternative to the EBM app, the collected data for different sensors require transformation to make it compatible with the StandStrong platform. For example, the collected data should be exported as csv following the file naming convention used by StandStrong. Output dataset. The StandStong scheduler job loads the datasets into the “sstrong” database, which is implemented in the MySQL relational database system. The data transfer between the StandStrong components and database system is achieved by establishing a secured link. The first two database tables are configuration tables and named “project” and “mother”. The “project” table must have a setting for an inbound folder, the location where passive data files are uploaded in the system. There must be a record for the mother with a unique identification number, which is required to map the data loaded into the tables named “GPS”, “proximity”, “activity”, and “audio” for each mother. The database is the source providing daily information to the StandStrong Counselor App. The app gets the data through the web service, which is repeatedly synced to the database. During each attempt to sync, the app looks for new data added to the system. Our analytic approach has been to quantize down to the 15-minute level and then generate 24-hour mappings that include missing data so as to be able to compare behavioral rhythms day by day and across sensors which may not always be collected simultaneously. Use case 1. A project director interested in implementing StandStrong in the context of psychological treatment would begin by preparing for software installation on tablets for counselors and on mobile phones for mothers. Access to the components in the architecture is needed, namely, the EBM app, StandStrong Counselor App, servers and AWS S3 Storage. The installation guide and user manual are provided as Extended data 15. Psychosocial counselors can be trained to use the app. When procuring devices, low-cost Android smartphones like the Samsung J2 Ace phone are compatible to operate the StandStrong app. Thus, adding this passive sensing tool would be inexpensive from both technical and human resources perspectives. In terms of training, since it is passive data collection, mothers would not require training to use this technology. We will need to train the facilitators on using the technology, ensuring proper use during each visit, and integrating passive data in a psychosocial counseling session. Psychosocial counseling (HAP) is already a part of government-implemented training for psychosocial counselors in Nepal. Training of counselors in HAP in Nepal typically is delivered in approximately 5-days in the government curriculum. To integrate StandStrong, we recommend an extra day in the training. This is needed to train them on how to set up the passive sensing technology and troubleshooting. In addition, for each of the HAP sessions, the trainer should discuss with counselors what information is useful and can be incorporated into the session. Other training should include the use of awards and messaging with mothers. After training, counselors typically have weekly supervision with a HAP specialist, this will need to additionally include discussions of technological challenges and how the passive sensing information is being optimally used 18. When participants (e.g. mothers) are enrolled, they are all given a unique ID for data management in the StandStrong platform, e.g. SS-XXXX. Prior to enrollment, project staff visits the mother to sign the consent form. Two layers of consent were sought. First, we obtained a written consent from the mothers to use the devices. Second, we explained the technology to the family members and only proceeded with the study after the family members gave verbal consent. We addressed any privacy and confidentially-related concerns from the participants and their family members. In our study, it was integral to seek participant as well as family consent to ensure we addressed any privacy or confidentiality concerns of the family. We also shared a one-page description of the study with the participant and family, so she could describe the study to her family and friends, even when the study team was not present. Some of the key considerations related to privacy are, a) educating participants on how to delete the files from the phones, b) ongoing interactions with the participant and family throughout the study duration to ensure easy communication, and c) ensuring good rapport building with the participants so they feel comfortable to raise privacy concerns (if any) throughout the study period. In future studies, we will explore processing the audio files in the phone itself, so we do not have to store the audio clips on the phones. We have discussed the issues related to privacy and confidentiality in detail in a separate paper 18. Following participant and family consent, project staff visit the mother and give them a Samsung J2 Ace phone with the EBM. The EBM app is configured with the mother identification number, enabled audio recordings, access to device’s location and file system, and enabled motion, audio, proximity and phone interaction data sources ( Figure 3). The project staff ensure that GPS and mobile data are enabled for tracking GPS location. A Bluetooth beacon is attached to the infant’s clothes. RadBeacon Locate, a utility app, is also installed in the smartphone. In the case of RadBeacon Locate, the app is used to confirm that the signal from the proximity beacon is received by the phone. The RadBeacon app could be used at any time to see if the beacon is working or not. The mother’s information is then recorded in the database so that the passive data can be loaded into the system. This can be set up, either by inserting a new record directly through the SQL query or through the endpoints. General tab ( A), mother identifier ( B), and passive data sources ( C) while setting up the Electronic Behavior Monitoring app. At the end of a week of passive data collection, the project staff visit the mother’s home to download the passive data that are saved in a folder named Namaste in the mother’s phone ( Figure 4). In our current implementation, the data are collected weekly by the project staff. However, future implementation is intended to have realtime upload and processing so that mothers and counselors can immediately get feedback. The data are uploaded into a secured database with a week number identification for backup. Each of the passive data files are named with the mother’s identification number (without personally identifiable information). The project staff then upload the data text files into the AWS S3 bucket, from which they then are loaded into the server. Furthermore, the project staff confirm that the uploaded data files are successfully loaded into the database server using the utility tool every week. Folder structure ( A) and GPS file ( B) as captured by the Electronic Behavior Monitoring app. GPS, Global Positioning System. Once the data is analyzed, it is visualized in the StandStrong app. Four types of data are analyzed to be used in the counseling sessions – a) Audio data, b) Proximity, c) Activity, and d) GPS data. Using RADAR-base app would require setting up RADAR passive remote monitoring technology (pRMT) and the RADAR management portal. The portal allows adding a new participant to the study. Once a subject is created, the pRMT app can login to the management portal to provide permission for different sensors. The collected data then has to be exported to the format supported by the StandStrong Platform. More detail can be found at using pRMT is available on the RADAR-base website. Use case 2. Following Case 1, the next step is to integrate the passive data in the counselor’s StandStrong app. The four types of passive data collected using smartphones and Bluetooth beacons can be visualized in the StandStrong app installed on the counselor’s tablet. The counselor receives daily passive data updates for each mother. Before the weekly psychosocial session, the counselors synchronize the latest data in their tablet to discuss during the counseling session. Once downloaded, the data is available offline for sessions which are typically held in mothers’ homes that might not have reliable internet access. The counselor then meets the mother in her scheduled weekly psychosocial counseling session. During the counseling session, the counselor shares and discusses the mother’s data with her using the StandStrong app on her tablet. This information is integrated as a part of the psychosocial session, particularly for behavior activation, that reinforces positive behaviors among the mothers. For example, they may show the proximity chart, GPS heat map, and activity tracking while discussing mothers’ daily life and behavioral patterns in the last one week. Any weekly change in behavioral patterns between sessions can also be discussed. Finally, counselors can provide suggestions to the mothers on behavior change for better recovery based on her passive data. Besides interactions with the counselors during the psychosocial sessions, mothers can also send Viber text messages to counselors between sessions. These messages appear in the counselor’s StandStrong app on their tablet which they can respond through the app. Another reinforcing feature of StandStrong is “Awards” that are given to support certain self-care, daily routine, social interaction-related behavioral goals. This weekly progress is documented through passive data triggered behavioral change.
N/A
