Menu
Menu
Menu
Menu
The World Health Organization (WHO) has collected information on policies on sexual, reproductive, maternal, newborn, child and adolescent health (SRMNCAH) over many years. Creating a global survey that works for every country context is a well-recognized challenge. A comprehensive SRMNCAH policy survey was conducted by WHO from August 2018 through May 2019. WHO regional and country offices coordinated with Ministries of Health and/or national institutions who completed the questionnaire. The survey was completed by 150 of 194 WHO Member States using an online platform that allowed for submission of national source documents. A validation of the responses for selected survey questions against content of the national source documents was conducted for 101 countries (67%) for the first time in the administration of the survey. Data validation draws attention to survey questions that may have been misunderstood or where there was a lot of missing data, but varying methods for validating survey responses against source documents and separate analysis of laws from policies and guidelines may have hindered the overall conclusions of this process. The SRMNCAH policy survey both provided a platform for countries to track their progress in adopting WHO recommendations in national SRMNCAH-related legislation, policies, guidelines and strategies and was used to create a global database and searchable document repository. The outputs of the SRMNCAH policy survey are resources whose importance will be enriched through policy dialogues and wide utilization. Lessons learned from the methodology used for this survey can help to improve future updates and inform similar efforts.
To optimize the approach used for SRMNCAH policy tracking, WHO established an SRMNCAH policy reference group to obtain external expert advice on the contents of the survey. The group’s members, via an online survey, identified priority areas within SRMNCAH to include in the survey and suggested topics that could be excluded. Concurrently, WHO researched existing global policy and legislative databases and found thirty SRMNCAH-related data sources, identifying key topics that could be eliminated from the SRMNCAH policy survey. 3 To create a final questionnaire that combined previously separate data collection efforts, key focal points in multiple WHO departments and from all six WHO regional offices, along with stakeholders in partner organizations, reviewed the draft questionnaire and provided feedback on its content. As it was important to provide a common understanding of key technical terms used throughout the survey, a glossary was included in the questionnaire. The survey tool was available in all official United Nations (UN) languages (Arabic, Chinese, English, French, Russian, Spanish) and Portuguese. 4 Various resources were consulted to determine the definitions used for all terms in the glossary. The definitions provided within the survey for guideline, 5 health policy, 6 law 7 and healthy strategy/plan 8 (Table 1) help to show the differences and relationships between these key terms. Strategic plans often build off policies in that a health policy provides a vision and outlines goals for health outcomes, while a national health strategy sets forth the process for achieving these. Guidelines similarly work in conjunction with policies and strategic plans, providing evidence-based guidance on various interventions and public health activities for key stakeholders. Additionally, health policies can describe priorities and roles of stakeholders and provide information to a population, while laws “govern behaviour” showing that policies and laws should ideally reflect each other and work in parallel. Abbreviation: SRMNCAH, sexual, reproductive, maternal, newborn, child and adolescent health. In previous rounds of the MNCAH policy survey, respondents reported on the existence of national laws, policies, guidelines, and strategies within their countries without providing the source documents from which these responses stemmed. Some of the questions had been asked in a manner that could have led to overly affirmative responses, such as whether a national guideline followed WHO recommendations rather than asking whether the national guideline contained specific interventions which would allow for an assessment of alignment with WHO guidance. For the 2018-2019 SRMNCAH policy survey, use of an online platform for data collection permitted respondents to submit source documents, allowing for validation of responses. The questionnaire was structured into several modules: Cross-cutting SRMNCAH, Maternal and newborn health, Child health, Adolescent health, Reproductive health, and Gender-based violence. An online platform, programmed using LimeSurvey software, permitted various respondents to be assigned to specific modules, allowing for concurrent data entry within the modules by several users within a single country. Two webinars were held to train regional focal points in how to use the online survey platform. A user guide and video tutorial were also provided. Regional focal points for the survey coordinated with assigned focal points in each country to collect information on designated respondents. The WHO country office, or other assigned country focal point, was responsible for coordinating with the Ministry of Health (MoH) or national agencies/institutions to complete the survey. In the majority of participating countries, the principal respondents were from the MoH. Multiple respondents may have been consulted on specific topics to assist in completion of the survey, including officials from other government agencies, WHO country offices and other UN agencies. For each module, there was slight variation in the affiliations of personnel who assisted the main MoH respondent in completing the questionnaire, however the majority of those consulted were others within the MoH or from WHO (Figure). Affiliations of Respondents Consulted by Main Respondent for the Cross-Cutting Module of the Survey. Abbreviation: MoH, Ministry of Health; WHO, World Health Organization; UNFPA, United Nations Population Fund; UNICEF, United Nations Children’s Fund. Various approaches to administering the survey were employed by WHO regional offices and country teams based on whether WHO has a country office presence. In countries with WHO country offices, SRMNCAH focal points coordinated with the MoH and partners to complete the survey. In countries without a WHO country office, WHO regional focal points coordinated survey completion directly with the MoH and/or other national institutions. In the Western Pacific Region, after the MoH focal points were identified through coordination with WHO country offices, the regional office focal point provided a brief overview of the survey and monitored progress. Amongst countries within the Region of the Americas, Ministries of Health nominated a survey coordinator to organize inputs from colleagues and to review the responses before submitting. If the MoH requested support, the regional office provided an external professional to interview all relevant units and fill out the survey online. The survey was designed with the ability to be filled out by multiple respondents based on their area of expertise, allowing for specific respondents to be mapped to defined modules to complete just those sections. The questionnaire could also be completed offline by multiple respondents with a single respondent entering the responses into the survey platform. A limited number of countries were able to hold review meetings of the completed questionnaire with key stakeholders allowing for agreed revisions before submitting the final version though some countries reviewed the survey responses after submission and requested edits which were incorporated into the database used for final analysis by WHO. Response rate to the SRMNCAH policy survey varied by region. Of all 194 WHO Member States, 150 completed the survey (77%) (Table 2). 9 Variation in response rates were due to several reasons. Abbreviations: SRMNCAH, sexual, reproductive, maternal, newborn, child and adolescent health; WHO, World Health Organization. aFive additional non-Member States, which are not reflected in the above response rate, completed the survey. These include: British Virgin Islands (Americas), French Polynesia (Western Pacific), Guam (Western Pacific), Occupied Palestinian territory (Eastern Mediterranean), Wallis and Futuna Islands (Western Pacific). Survey enumeration lasted from August 2018 through May 2019. Several regional offices thought that the timing of the beginning of data collection, in coinciding with various holiday schedules and end of year administrative processes, factored into a low response rate by the end of that year. Due to this, the period of enumeration was extended through May 2019. Most regional offices listed staffing issues within each country as a main factor in why some were unable to complete the survey. The main issue cited was limited capacity amongst Ministries of Health to direct time towards completing the survey. Specific issues included staff turnover, dismissals, retirements; changes in leadership and internal restructuring in MoH and/other national institutions; and limited personnel covering the wide range of programs of SRMNCAH. In the Western Pacific Region, staff in a few countries were sometimes diverted to public health emergencies during the time of survey enumeration. Commitment from the MoH was another key factor in countries’ completion of the survey. The Regional Office of the Americas stressed the importance of informing countries of the potential benefits of the survey results for planning, updating current policies and strategies, and assessing SRMNCAH programs. In the South-East Asia Region, where all countries completed the survey, obtaining government agreement and identifying a focal point were key factors to this full response rate. The European Regional Office cited lack of official hard copy invitation letters as a possible reason for some Ministries not participating. MoH commitment also extended to the final step of authorizing submission of the survey responses. For example, in one country, the survey was completed but the MoH did not authorize its submission, citing that more time was needed to review the survey. Despite follow up from the regional office, the survey was not submitted. Connectivity issues were also a challenge across several regions, due to low bandwidth, outdated software, and high security settings that blocked access to the survey platform. Some regional and country offices offered to enter the survey responses into the online platform in these situations. Two countries within the African and Eastern Mediterranean regions had issues in submitting their completed surveys online and were unable to submit them within the deadline. Participating countries submitted national source documents used to support their answers to the survey. For each document, country respondents were asked to provide information on the type of document, year of publication and language when uploading the document. The cataloguing process involved several activities: standardizing file names; categorizing documents by key identifying characteristics; and creating a database which linked each document to the survey question(s) for which they were uploaded to be used as a key for validating survey responses against the source documents. Standardizing file names for the online repository was completed simultaneously with the larger task of cataloguing the documents based on key characteristics. Each document was reviewed to confirm its official title, topic, type, year of publication, and language as the initial categorization. Many documents fell under multiple topics, for example covering both child and adolescent health. It was also necessary to re-categorize some documents from the initial survey categorization. Despite providing definitions of the various types, many of the documents were mis-categorized, indicating that some respondents may not have been able to clearly distinguish between a law, policy, guideline, or strategic plan. The end result of the cataloguing process was the creation of a searchable repository of national laws, policies, guidelines, and strategic plans related to SRMNCAH accessible to the public. This repository allows for analysis of the content of the documents as well as validation of the survey responses against the source documents completed in 2020. Validation of the survey responses against the source documents was completed in 2020. The work was split between two institutes, one focusing on laws and one focusing on policies, guidelines and operational guidance. Each institute applied their expertise to reviewing the preliminarily catalogued documents to ensure that they were correctly categorized and to abstracting information from the documents to verify the responses to specified questionnaire items. A set of questionnaire items were selected from the full survey tool for inclusion in the validation exercise. For validation of laws, only documents in one of the six official UN language were included in the analysis. For validation of policies, guidelines, and operational guidance, in addition to the UN languages, documents in Danish, Serbo-Croatian, and Slovenian were included. This reduced the dataset for validation from the 150 WHO Member States that completed the survey to 99 countries for validation of laws and 101 countries for validation of policies, guidelines and operational guidance documents. Within both institutes, research assistants with qualifications in the respective languages conducted the validation. Study protocols were followed and all research assistants received training prior to starting the work. Project managers conducted regular quality checks. WHO provided the institutes with a database of the survey responses and a database of the source documents, mapping which documents were uploaded for specific content sections of the questionnaire. From the selected set of questions for validation, the institutes searched the documents for the relevant content necessary to check whether the responses were congruent with the text. This included searching the documents for selected key words or phrases specific to each survey question. As a first step, the document(s) specifically used to answer each survey section were searched. When a discrepancy was discovered, details documenting it were recorded. Confirmation of the survey responses as well as instances when no information was available in the document(s) to support the survey response were also recorded, hence, the following categories were used to classify the findings: Overall, of the 101 countries included in the validation of responses against policy, guideline, and operational guidance documents, a quarter of the validated survey responses (26%) corresponded to information found in the source documents (matches). Four percent of the responses contradicted information found in the source document (mismatches), whereas 53% of the responses could not be validated either because no document on the topic was uploaded (37%), or because the relevant information was not available in the document uploaded for the question (16%). For validation of responses against laws, 99 countries were included in the analysis. Fourteen percent of the validated survey responses corresponded to information found in the documents provided (matches). An inconsistency in response was found in 13% of responses (mismatches), due to information in the documents contradicting the responses or no information being found in the documents provided. Seventy-three percent of responses were unable to be validated primarily because no law documents were uploaded under that question; but also in cases where no law documents were submitted for that question in a UN language.
N/A
