The Development of an Electronic Clinical Decision and Support System to Improve the Quality of Antenatal Care in Rural Tanzania: Lessons Learned Using Intervention Mapping

Frontiers in Public Health, Volume 9, Year 2021

Interpretation

It is widely recognised that high quality antenatal care is a key element in maternal healthcare. Tanzania has a very high maternal mortality ratio of 524 maternal deaths per 100,000 live births. Most maternal deaths are due to preventable causes that can be detected during pregnancy, and antenatal care therefore plays an important role in reducing maternal morbidity and mortality. Unfortunately, quality of antenatal care in Tanzania is low: Research has shown that healthcare workers show poor adherence to antenatal care guidelines, and the majority of pregnant women miss essential services. Digital health tools might improve the performance of healthcare workers and contribute to improving the quality of antenatal care. To this end, an electronic clinical decision and support system (the Nurse Assistant App) was developed and implemented in Tanzania in 2016 to provide digital assistance during antenatal care consultations to healthcare workers. The current study systematically evaluated the development and implementation process of the Nurse Assistant App in Magu District, Tanzania, with the aim of informing future programme planners about relevant steps in the development of a digital health intervention. Desk research was combined with semi-structured interviews to appraise the development process of the digital health tool. We employed the criteria stipulated by Godin et al., which are based on the six steps of Intervention Mapping [IM; Bartholomew Eldredge et al.]. Findings indicated that five of the six steps of IM were completed during the development and implementation of the Nurse Assistant App. Tasks related to community engagement, adjustment to local context, implementation in the practical context in collaboration with local partners, and rigorous evaluation were accomplished. However, tasks related to identifying theory-based behaviour change methods were not accomplished. Based on the lessons learned during the process of developing and implementing the Nurse Assistant App, we conclude that programme developers are recommended to (1) engage the community and listen to their insights, (2), focus on clear programme goals and the desired change, (3), consult or involve a behaviour change specialist, and (4), anticipate potential problems in unexpected circumstances. This qualitative study draws upon two sources of data: documents describing the development and implementation process of the NAA, complemented by semi-structured interviews designed to verify the results of the desk research. This evaluation is guided by Intervention Mapping, a six-step protocol for theory- and evidence-based intervention development that can also be used for the post-hoc evaluation of the soundness of digital health interventions, such as an electronic clinical decision and support system (35). Intervention Mapping provides systematic guidance to programme developers that facilitates making sound decisions during the process of intervention development (34). In step 1 of Intervention Mapping, the health problem is analysed through a needs assessment and the subsequent development of a logic model of the problem to specify the causes and contributing factors related to the health problem. Step 2 focusses on what behaviour and underlying personal determinants need to change in order to reduce the health problem, visualised in a logic model of change. In step 3, an intervention is designed, based on behaviour change theory and relevant change methods. In step 4, intervention materials, activities, and protocols are designed and pre-tested and the final intervention is produced. In step 5 an implementation plan is developed to ensure adoption and full and correct use of the intervention and aim for long-term use. Step 6 focusses on programme evaluation including both an effect and process evaluation. All six steps consist of distinct tasks that need to be completed before proceeding to the next step. At the same time, Intervention Mapping allows for an iterative work process going forwards and backwards through steps and tasks. A flowchart of the different steps is shown in Figure 2. The six steps of Intervention Mapping. To assess the soundness of the development process of the NAA, we make use of the planning evaluation tool developed by Godin et al. (35), which is based on the six steps of Intervention Mapping (34). The planning tool was developed to assist professionals in performing a rigorous evaluation of an intervention and consists of 40 criteria that make up 19 tasks (see Table 1) that can be scored. The application of this tool to the intervention development process provides insight into the steps and tasks that were undertaken and completed during the development and implementation of the NAA. To score the criteria of step 1 and step 2 in the planning tool, models, and matrices were retrospectively created. This gave insight into the extent to which choices made by the project team were in line with models that would have been created based on the available empirical literature, theory and collected data. The logic model of the problem was retrospectively created mainly based on the results of the needs assessment while the logic model of the problem was retrospectively created based on stakeholder meeting reports. Previous research has been published documenting a similar method of retrospectively evaluating an intervention (35–37). Planning evaluation tool of Godin et al. (35). In 2012, the African Women Foundation created a partnership with Crop Marketing Bureau (CROMABU), a Tanzanian organisation aimed at empowering small-scale farmers by using Information and Communication Technologies (ICT) in Magu district, Tanzania and commenced conducting research related to maternal health. In 2013, the Women Centered Care Project was initiated. The project team of the Women Centered Care Project consisted of one Tanzanian head of staff, one Dutch programme manager, one Dutch research officer, one Dutch liaison programme officer, and three Tanzanian research assistants. The research activities of the project were conducted in close collaboration with four universities (three in the Netherlands and one in Tanzania). The programme manager worked closely together with the board of the African Woman Foundation in the Netherlands, which had an advisory, fund-raising, partnership, and decision-making role. The programme manager also worked on the development and maintenance of the NAA in collaboration with ICT Healthcare Technology Solutions (formerly Buro Medische Automatisering B.V.), a Dutch private company specialised in digital solutions in the domain of obstetrics (38). To assess processes and milestones in the development and implementation of the NAA, desk research was conducted using digital project archives, project documents, and content from the NAA itself. Data sources consisted of minutes of meetings, for which a template was made available at the time (i.e., minutes of 67 bi-weekly project team meetings from November 2013 until August 2016, minutes of meetings with stakeholders, including three meetings with the district reproductive and child health coordinator, minutes of 13 meetings with healthcare workers at their health facility, and minutes of 59 meetings with district officials held twice a year per subdivision of the district); reports on project activities that were approved by the board of the African Woman Foundation (i.e., workshop and training sessions offered by the project team to healthcare workers, monthly financial reports of the project, annual and quarterly project reports); field notes (i.e., supervision visits undertaken by the project team to all participating healthcare facilities, logs of the technical support provided by the research assistants of the project team to healthcare workers working with the NAA); personal timesheets of members of the project team kept for verification of working hours; and e-mail conversation between board members, the project team, and the ICT specialists. One researcher conducted the search among the data sources and carried out the document synthesis, which was verified by a second researcher. Project-team discussions were held to discuss comprehensiveness of data, access to archives, and making sense of the data. Data obtained through the desk research were used to assess the development and implementation process of the NAA by scoring the criteria specified in the planning tool. Firstly, the two researchers worked independently from each other to score each of the 40 criteria of the planning tool (+ fully accomplished; +/− partially accomplished; – not accomplished). Secondly, the two researchers jointly evaluated whether or not a task was accomplished. Tasks were considered accomplished if at least one criterion was coded as “fully accomplished” (35). Thirdly, the two researchers jointly evaluated whether each of the Intervention Mapping steps was completed. Steps were considered completed when at least half of the tasks within that step were scored as accomplished (35). For example, step 3 of the Intervention Mapping approach consists of two tasks, reflected in four criteria of the planning tool (numbered 21–24). If, for example, criterion 21 was scored as “not accomplished” and criterion 22 as “partly accomplished,” task 10 was considered to be not accomplished overall. However, if criterion 24 was scored as “accomplished” and criterion 23 as “not accomplished,” task 11 was considered accomplished. Consequently, since half of the tasks in step 3 were accomplished, the step was considered completed (see Table 1). To complement the results of the analysis, semi-structured interviews were held with two developers of the NAA selected to gain insight into the development process from distinct perspectives. The team of the Women Centered Care Project consisted of six people. Of these, only two could be contacted, as others had moved on: a Tanzanian project research assistant based in Tanzania and a Dutch programme researcher based in Magu district, Tanzania at the time of NAA development. Both these participants were involved from the beginning to the end of the development of the NAA, and had the possibility to influence the decision making process regarding the development of the NAA. It proved not possible to include software developers involved in the development of the NAA, as both had moved on. Participants were approached individually by e-mail with a request for participation and both agreed. During the semi-structured interviews, participants were encouraged to share their critique on the completed planning tool, to verify the results and check for any inconsistencies. Prior to the interview, the completed planning tool was sent to the participants so that they could prepare in advance. Interviews were conducted by phone and lasted for ~40 min. Interviews were audio-recorded with verbal informed consent of the participants. During the interview, the intervention planning tool was discussed until consensus was reached by the interviewee and the interviewer about the appropriate coding. Data obtained through the desk research and semi-structured interviews that did not correspond to criteria of the planning tool were included as additional information to assess the practical execution of the development and implementation of the NAA. Although, the analysis of the practical execution of the six steps of Intervention Mapping is not officially part of the planning tool, we believe that the examples on the practical execution of the six steps might help to inform future programme planners. Intervention Mapping was retrospectively used to structure the data and assess what steps of Intervention Mapping were executed. Specifically, required models and matrices were retrospectively created based on the data. At the time of the development of the NAA, Intervention Mapping was not applied, therefore this part of the data created a comprehensive overview of all activities conducted during the process of development and implementation of the NAA.

AI Digest

Study Justification:

– High quality antenatal care is crucial for maternal healthcare.
– Tanzania has a high maternal mortality ratio, largely due to preventable causes that can be detected during pregnancy.
– Quality of antenatal care in Tanzania is low, with poor adherence to guidelines and missed essential services.
– Digital health tools have the potential to improve healthcare worker performance and the quality of antenatal care.

Study Highlights:

– Development and implementation of an electronic clinical decision and support system (the Nurse Assistant App) in Tanzania.
– Evaluation of the development and implementation process using Intervention Mapping, a six-step protocol for intervention development.
– Five out of six steps of Intervention Mapping were completed, including community engagement, adjustment to local context, implementation in collaboration with local partners, and rigorous evaluation.
– Tasks related to identifying theory-based behavior change methods were not accomplished.
– Lessons learned include the importance of community engagement, clear program goals, involvement of behavior change specialists, and anticipating potential problems.

Study Recommendations:

– Engage the community and listen to their insights.
– Focus on clear program goals and the desired change.
– Consult or involve a behavior change specialist.
– Anticipate potential problems in unexpected circumstances.

Key Role Players:

– Tanzanian head of staff
– Dutch program manager
– Dutch research officer
– Dutch liaison program officer
– Three Tanzanian research assistants
– Board of the African Women Foundation
– ICT Healthcare Technology Solutions (formerly Buro Medische Automatisering B.V.)

Cost Items for Planning Recommendations:

– Community engagement activities
– Program development and maintenance
– Training and capacity building for healthcare workers
– Technical support for healthcare workers
– Monitoring and evaluation activities
– Collaboration with local partners
– Communication and coordination expenses
Please note that the above information is a summary of the study and its findings. For more detailed information, please refer to the publication “The Development of an Electronic Clinical Decision and Support System to Improve the Quality of Antenatal Care in Rural Tanzania: Lessons Learned Using Intervention Mapping” in Frontiers in Public Health, Volume 9, Year 2021.

Strength of Evidence

The strength of evidence for this abstract is 7 out of 10.
The evidence in the abstract is based on a qualitative study that draws upon two sources of data: documents describing the development and implementation process of the Nurse Assistant App, and semi-structured interviews. The study systematically evaluated the development and implementation process of the Nurse Assistant App in Tanzania using the criteria stipulated by Godin et al. The findings indicate that five out of six steps of Intervention Mapping were completed, but tasks related to identifying theory-based behavior change methods were not accomplished. The study provides actionable steps to improve future program development, such as engaging the community, focusing on clear program goals, consulting behavior change specialists, and anticipating potential problems. However, the evidence is limited to the specific context of the Nurse Assistant App in Tanzania and does not include quantitative data or a large sample size. To improve the strength of the evidence, future research could include a larger sample size, quantitative data on the impact of the Nurse Assistant App on antenatal care quality, and replication of the study in different settings.

Abstract

This qualitative study draws upon two sources of data: documents describing the development and implementation process of the NAA, complemented by semi-structured interviews designed to verify the results of the desk research. This evaluation is guided by Intervention Mapping, a six-step protocol for theory- and evidence-based intervention development that can also be used for the post-hoc evaluation of the soundness of digital health interventions, such as an electronic clinical decision and support system (35). Intervention Mapping provides systematic guidance to programme developers that facilitates making sound decisions during the process of intervention development (34). In step 1 of Intervention Mapping, the health problem is analysed through a needs assessment and the subsequent development of a logic model of the problem to specify the causes and contributing factors related to the health problem. Step 2 focusses on what behaviour and underlying personal determinants need to change in order to reduce the health problem, visualised in a logic model of change. In step 3, an intervention is designed, based on behaviour change theory and relevant change methods. In step 4, intervention materials, activities, and protocols are designed and pre-tested and the final intervention is produced. In step 5 an implementation plan is developed to ensure adoption and full and correct use of the intervention and aim for long-term use. Step 6 focusses on programme evaluation including both an effect and process evaluation. All six steps consist of distinct tasks that need to be completed before proceeding to the next step. At the same time, Intervention Mapping allows for an iterative work process going forwards and backwards through steps and tasks. A flowchart of the different steps is shown in Figure 2. The six steps of Intervention Mapping. To assess the soundness of the development process of the NAA, we make use of the planning evaluation tool developed by Godin et al. (35), which is based on the six steps of Intervention Mapping (34). The planning tool was developed to assist professionals in performing a rigorous evaluation of an intervention and consists of 40 criteria that make up 19 tasks (see Table 1) that can be scored. The application of this tool to the intervention development process provides insight into the steps and tasks that were undertaken and completed during the development and implementation of the NAA. To score the criteria of step 1 and step 2 in the planning tool, models, and matrices were retrospectively created. This gave insight into the extent to which choices made by the project team were in line with models that would have been created based on the available empirical literature, theory and collected data. The logic model of the problem was retrospectively created mainly based on the results of the needs assessment while the logic model of the problem was retrospectively created based on stakeholder meeting reports. Previous research has been published documenting a similar method of retrospectively evaluating an intervention (35–37). Planning evaluation tool of Godin et al. (35). In 2012, the African Women Foundation created a partnership with Crop Marketing Bureau (CROMABU), a Tanzanian organisation aimed at empowering small-scale farmers by using Information and Communication Technologies (ICT) in Magu district, Tanzania and commenced conducting research related to maternal health. In 2013, the Women Centered Care Project was initiated. The project team of the Women Centered Care Project consisted of one Tanzanian head of staff, one Dutch programme manager, one Dutch research officer, one Dutch liaison programme officer, and three Tanzanian research assistants. The research activities of the project were conducted in close collaboration with four universities (three in the Netherlands and one in Tanzania). The programme manager worked closely together with the board of the African Woman Foundation in the Netherlands, which had an advisory, fund-raising, partnership, and decision-making role. The programme manager also worked on the development and maintenance of the NAA in collaboration with ICT Healthcare Technology Solutions (formerly Buro Medische Automatisering B.V.), a Dutch private company specialised in digital solutions in the domain of obstetrics (38). To assess processes and milestones in the development and implementation of the NAA, desk research was conducted using digital project archives, project documents, and content from the NAA itself. Data sources consisted of minutes of meetings, for which a template was made available at the time (i.e., minutes of 67 bi-weekly project team meetings from November 2013 until August 2016, minutes of meetings with stakeholders, including three meetings with the district reproductive and child health coordinator, minutes of 13 meetings with healthcare workers at their health facility, and minutes of 59 meetings with district officials held twice a year per subdivision of the district); reports on project activities that were approved by the board of the African Woman Foundation (i.e., workshop and training sessions offered by the project team to healthcare workers, monthly financial reports of the project, annual and quarterly project reports); field notes (i.e., supervision visits undertaken by the project team to all participating healthcare facilities, logs of the technical support provided by the research assistants of the project team to healthcare workers working with the NAA); personal timesheets of members of the project team kept for verification of working hours; and e-mail conversation between board members, the project team, and the ICT specialists. One researcher conducted the search among the data sources and carried out the document synthesis, which was verified by a second researcher. Project-team discussions were held to discuss comprehensiveness of data, access to archives, and making sense of the data. Data obtained through the desk research were used to assess the development and implementation process of the NAA by scoring the criteria specified in the planning tool. Firstly, the two researchers worked independently from each other to score each of the 40 criteria of the planning tool (+ fully accomplished; +/− partially accomplished; – not accomplished). Secondly, the two researchers jointly evaluated whether or not a task was accomplished. Tasks were considered accomplished if at least one criterion was coded as “fully accomplished” (35). Thirdly, the two researchers jointly evaluated whether each of the Intervention Mapping steps was completed. Steps were considered completed when at least half of the tasks within that step were scored as accomplished (35). For example, step 3 of the Intervention Mapping approach consists of two tasks, reflected in four criteria of the planning tool (numbered 21–24). If, for example, criterion 21 was scored as “not accomplished” and criterion 22 as “partly accomplished,” task 10 was considered to be not accomplished overall. However, if criterion 24 was scored as “accomplished” and criterion 23 as “not accomplished,” task 11 was considered accomplished. Consequently, since half of the tasks in step 3 were accomplished, the step was considered completed (see Table 1). To complement the results of the analysis, semi-structured interviews were held with two developers of the NAA selected to gain insight into the development process from distinct perspectives. The team of the Women Centered Care Project consisted of six people. Of these, only two could be contacted, as others had moved on: a Tanzanian project research assistant based in Tanzania and a Dutch programme researcher based in Magu district, Tanzania at the time of NAA development. Both these participants were involved from the beginning to the end of the development of the NAA, and had the possibility to influence the decision making process regarding the development of the NAA. It proved not possible to include software developers involved in the development of the NAA, as both had moved on. Participants were approached individually by e-mail with a request for participation and both agreed. During the semi-structured interviews, participants were encouraged to share their critique on the completed planning tool, to verify the results and check for any inconsistencies. Prior to the interview, the completed planning tool was sent to the participants so that they could prepare in advance. Interviews were conducted by phone and lasted for ~40 min. Interviews were audio-recorded with verbal informed consent of the participants. During the interview, the intervention planning tool was discussed until consensus was reached by the interviewee and the interviewer about the appropriate coding. Data obtained through the desk research and semi-structured interviews that did not correspond to criteria of the planning tool were included as additional information to assess the practical execution of the development and implementation of the NAA. Although, the analysis of the practical execution of the six steps of Intervention Mapping is not officially part of the planning tool, we believe that the examples on the practical execution of the six steps might help to inform future programme planners. Intervention Mapping was retrospectively used to structure the data and assess what steps of Intervention Mapping were executed. Specifically, required models and matrices were retrospectively created based on the data. At the time of the development of the NAA, Intervention Mapping was not applied, therefore this part of the data created a comprehensive overview of all activities conducted during the process of development and implementation of the NAA.

Materials

N/A

Innovations

The development and implementation of the Nurse Assistant App (NAA) in Tanzania followed the six-step protocol of Intervention Mapping, a framework for theory- and evidence-based intervention development. The steps include analyzing the health problem, identifying behavior and personal determinants for change, designing the intervention based on behavior change theory, developing intervention materials and protocols, creating an implementation plan, and conducting program evaluation.

Based on the lessons learned from the development and implementation process of the NAA, several recommendations for future program planners are suggested:

1. Engage the community and listen to their insights: Involving the community in the development process helps ensure that the intervention is tailored to their needs and preferences. This can be done through community consultations, focus groups, or interviews.

2. Focus on clear program goals and the desired change: Clearly defining the goals and desired outcomes of the intervention helps guide the development process and ensure its effectiveness. This includes identifying specific behaviors that need to change and setting measurable targets.

3. Consult or involve a behavior change specialist: Engaging a behavior change specialist can provide valuable expertise in designing interventions that effectively promote behavior change. They can help identify appropriate behavior change theories and techniques to inform the intervention design.

4. Anticipate potential problems in unexpected circumstances: Considering potential challenges and developing contingency plans can help address unforeseen obstacles during the implementation of the intervention. This includes anticipating technical issues, resource constraints, or changes in the local context.

By following these recommendations, future program planners can develop effective interventions to enhance maternal healthcare and improve access to maternal health services.

AI Innovations Description

The recommendation to improve access to maternal health is the development and implementation of an electronic clinical decision and support system, known as the Nurse Assistant App (NAA). This digital health tool was designed to provide assistance to healthcare workers during antenatal care consultations in Tanzania. The NAA aims to improve the quality of antenatal care, which is crucial in reducing maternal morbidity and mortality.

The development and implementation of the NAA followed the six-step protocol of Intervention Mapping, a framework for theory- and evidence-based intervention development. The steps include analyzing the health problem, identifying behavior and personal determinants for change, designing the intervention based on behavior change theory, developing intervention materials and protocols, creating an implementation plan, and conducting program evaluation.

Lessons learned from the development and implementation process of the NAA suggest several recommendations for future program planners. These recommendations include:

1. Engage the community and listen to their insights: Involving the community in the development process helps ensure that the intervention is tailored to their needs and preferences.

2. Focus on clear program goals and the desired change: Clearly defining the goals and desired outcomes of the intervention helps guide the development process and ensure its effectiveness.

3. Consult or involve a behavior change specialist: Engaging a behavior change specialist can provide valuable expertise in designing interventions that effectively promote behavior change.

4. Anticipate potential problems in unexpected circumstances: Considering potential challenges and developing contingency plans can help address unforeseen obstacles during the implementation of the intervention.

Overall, the development and implementation of the Nurse Assistant App demonstrate the potential of digital health tools to improve access to maternal health services. By following the recommendations and lessons learned from this study, future program planners can develop effective interventions to enhance maternal healthcare.

AI Innovations Methodology

To simulate the impact of the main recommendations mentioned in the abstract on improving access to maternal health, a methodology could be developed as follows:

1. Identify the target population: Determine the specific population that will be the focus of the intervention, such as healthcare workers providing antenatal care in rural Tanzania.

2. Define the objectives: Clearly define the objectives of the intervention, which could include improving adherence to antenatal care guidelines, increasing the utilization of essential services by pregnant women, and reducing maternal morbidity and mortality.

3. Develop a simulation model: Create a simulation model that represents the current state of access to maternal health in the target population. This model should consider factors such as healthcare worker behavior, availability of resources, and utilization of antenatal care services.

4. Incorporate the recommendations: Integrate the recommendations mentioned in the abstract into the simulation model. This could involve modifying the behavior of healthcare workers to reflect community engagement, setting clear program goals, involving behavior change specialists, and anticipating potential problems.

5. Run the simulation: Execute the simulation model to observe the impact of implementing the recommendations. This could involve running multiple scenarios with different levels of implementation to assess the potential outcomes.

6. Analyze the results: Evaluate the results of the simulation to determine the impact of implementing the recommendations on improving access to maternal health. This could include measuring changes in healthcare worker behavior, utilization of antenatal care services, and maternal health outcomes.

7. Refine the model: Based on the analysis of the simulation results, refine the simulation model to incorporate any additional insights or adjustments that may be necessary.

8. Repeat the simulation: Run the simulation again with the refined model to assess the impact of the recommendations after refinement.

9. Validate the results: Validate the simulation results by comparing them with real-world data or conducting further research to confirm the findings.

By following this methodology, researchers and program planners can gain insights into the potential impact of implementing the recommendations mentioned in the abstract on improving access to maternal health. This can help inform decision-making and guide the development of effective interventions in the future.

Author

Dima Health

Statistics:

Citations: 1

Identifiers:

DOI: 10.3389/fpubh.2021.645521

Research Areas:

Community Interventions, Health System and Policy, Maternal Access, Maternal and Child Health, Quality of Care, Sexual and Reproductive Health, Social Determinants, Technology and Innovations

Study Countries:

Tanzania

Study Design:

randomised-control-trial

Study Approach:

Qualitative

Participants Gender:

Female