Improving child nutrition and development through community-based childcare centres in Malawi – The NEEP-IE study: Study protocol for a randomised controlled trial

Background: The Nutrition Embedded Evaluation Programme Impact Evaluation (NEEP-IE) study is a cluster randomised controlled trial designed to evaluate the impact of a childcare centre-based integrated nutritional and agricultural intervention on the diets, nutrition and development of young children in Malawi. The intervention includes activities to improve nutritious food production and training/behaviour-change communication to improve food intake, care and hygiene practices. This paper presents the rationale and study design for this randomised control trial. Methods: Sixty community-based childcare centres (CBCCs) in rural communities around Zomba district, Malawi, were randomised to either (1) a control group where children were attending CBCCs supported by Save the Children’s Early Childhood Health and Development (ECD) programme, or (2) an intervention group where nutritional and agricultural support activities were provided alongside the routine provision of the Save the Children’s ECD programme. Primary outcomes at child level include dietary intake (measured through 24-h recall), whilst secondary outcomes include child development (Malawi Development Assessment Tool (MDAT)) and nutritional status (anthropometric measurements). At household level, primary outcomes include smallholder farmer production output and crop-mix (recall of last production season). Intermediate outcomes along theorised agricultural and nutritional pathways were measured. During this trial, we will follow a mixed-methods approach and undertake child-, household-, CBCC- and market-level surveys and assessments as well as in-depth interviews and focus group discussions with project stakeholders. Discussion: Assessing the simultaneous impact of preschool meals on diets, nutrition, child development and agriculture is a complex undertaking. This study is the first to explicitly examine, from a food systems perspective, the impact of a preschool meals programme on dietary choices, alongside outcomes in the nutritional, child development and agricultural domains. The findings of this evaluation will provide evidence to support policymakers in the scale-up of national programmes. The overall programme theory for the package of nutritional and agricultural interventions is guided by the Lancet series framework on Maternal and Child Nutrition [1] and broadly summarised in Fig. 1. For a more detailed analysis of the complex pathways linking agriculture and nutrition, including the different processes, actors, effects and lags, see [2, 12–14]. The package of interventions affects health and nutrition directly by improving diets and feeding practices through the behaviour-change communication and nutritional education. This, in turn, has an indirect impact on preschooling, as improving health and nutrition has a positive impact on attention, cognition and learning. By increasing the regularity and quality of the CBCC meals the interventions will also directly influence children’s participation in the CBCC. The interventions can also affect agriculture by increasing production, sales and profits, and changing the crop production mix. Overall programme theory for the agricultural and nutritional intervention The three determinants of undernutrition in children include food, health and care practices [1]. The main channels through which the intervention has an impact on nutrition and health is through improved diets via increased consumption of nutrient-rich foods, and through improved nutrition, health and hygiene practices. The proposed intervention package can potentially have an impact on the nutrition and health of children enrolled in CBCCs and their younger siblings, as summarised in Fig. ​Fig.33 in the Appendix. This involves a combination of direct transfers to the CBCC children (e.g. transfer of nutritious food through preschool meals) and indirect channels involving the behaviour-change campaigns promoting the consumption of nutritious foods and improved nutrition practices at household level. Improved diets, when accompanied by adequate feeding, health and hygiene practices can then contribute to improved health and nutrition. In particular: Impact pathways for the intervention on child nutrition and development (Fig. 3). (Source: adapted from [28]) Substitution, or intrahousehold reallocation, may occur when households readjust consumption patterns in response to the CBCC meals, or to a change in diet, by substituting foods normally consumed at home, or with other foods with similar properties. Substitution is a complex issue involving changing household dynamics where gender plays a fundamental role. Influencing possible substitution effects will be critical in determining the potential impacts on children’s nutrition and health. Changes in individual-level dietary diversity have been found to be strongly associated with micronutrient adequacy of diets for women [15, 16] and micronutrient density adequacy of diets in children [15]. Addressing micronutrient deficiencies can improve a range of health, nutritional and developmental outcomes in infants and young children, particularly if implemented alongside behaviour change on health and nutritional practices [17]. Conceptually, this framework suggests that the emphasis of the interventions in the short term should focus on integrating nutritional education and messaging, alongside the CBCC meals, to deliver improved intake of micronutrient-rich foods, with the potential of leading to improved household diets. Restoring micronutrients and enhancing energy intake can also have an impact on attention and motivation. Energy [18] and iron intake [19] can have an impact on hyperactivity, withdrawal, nervousness, hostile behaviour and happiness. The emotional status of children may also affect attention span and have other positive spill-overs. Caregivers and peers are also likely to be affected by the increase in attention and concentration. From an agricultural perspective, the intervention focusses on increasing food production in the CBCC garden and home-gardens by increasing yields or efficiencies through input provisions or training on farming practices. The intervention can also influence the basket of products that are being produced, supporting the production of higher-value crops and/or more nutritious crops through the provision of seeds, educational campaigns, or the opening of new market channels. The selection of particular crops involves balancing the pros and cons of substitution between crops for production, sale and consumption and the long-term impacts for both incomes and nutrition (see [20] for more details). The expected impact of the intervention discussed in the analysis of the programme theory is summarised below. The intervention is expected to have a positive impact on: A limited impact is expected on: The main indicators for the evaluation are summarised in Table 2. Main outcome indicators of the intervention CBCC community-based childcare centres, IYCF Infant and Young Child Feeding, MUAC mid-upper arm circumference, WHO World Health Organization Note that in addition to outcome indicators we will also observe the programme impact on intermediate indicators, particularly for those outcomes that are more difficult to observe directly. In the agricultural domain, we will look at intermediate outcomes such as input use (labour, land, seeds and fertiliser), investments (farm capital, such as tools and machinery) and market access (marketed surplus, prices and markets). The quantity, quality and timely preparation and delivery of food in the CBCCs will also be explored. A cluster randomised trial (CRT) is being implemented in 60 rural communities with CBCCs supported by Save the Children’s ECD programme in Zomba district, Malawi. The evaluation follows a mixed-methods approach (combining quantitative and qualitative methods) with two rounds of surveys and assessments timed 1 year apart, including child, caregiver, household, CBCC and market-level data collection. The intervention is targeted to disadvantaged communities within Zomba district in Malawi. The proposed study population includes CBCCs currently supported by Save the Children with an ECD package, including parenting and CBCC quality improvement. The geographical area for intervention was targeted by Save the Children on the basis of a set of education variables that impact pupil attendance and achievement in school. Within Zomba, two traditional authorities (TAs) and two sub-TAs were selected based on need (education and health) and the presence of other NGOs to implement the ECD programme. Save the Children’s ECD programme currently reaches 228 communities, 109 in TA Chikowi, 42 in TA Mbiza, 37 in sub-TA (STA) Ntholowa and 40 in STA Ngwelero. Sixty-eight of these (27 in TA Chikowi and 41 in TA Mbiza) had benefited from the agricultural and nutritional components already and were, therefore, excluded from the evaluation. The evaluation targets all children aged 0–6 years in the 60 selected communities and their caregivers. The primary reference group for this study is children aged 3–6 years old living in the service area of a Save the Children‐supported CBCC. Secondary reference groups include their siblings and caregivers. The 60 communities were randomly assigned to one of two treatment arms: The integrated intervention package will be implemented in 30 of the 60 rural communities after the baseline survey and extended to the control communities after the end-line survey. There are several reasons why the control group in this case is not a control without intervention. The Government of Malawi is committed to scaling-up the ECD quality improvement across all CBCCs and an impact evaluation on the cost-effectiveness of different ECD quality improvement strategies is underway.2 The proposed evaluation complements the ongoing work by examining the relative impact and costs of alternative implementation models, focussing on how to enhance participation in the CBCCs and, at the same time, supporting the nutrition of children at a critical age in their development. The 60 CBCCs were randomly selected in two stages from a pool of 235 CBCCs in 47 clusters currently assisted by Save the Children in Zomba. Due to the clustering of the CBCCs around primary schools, the list of 235 CBCCs was screened to flag clusters where more than one CBCC was being assisted. Twenty-six clusters were excluded from the first stage of randomisation to minimise possible contamination. An additional 10 CBCCs were dropped as they had ongoing activities. The 20 clusters were then randomly assigned to two groups of 10 clusters, where the randomisation was stratified geographically by traditional authority areas. In the second stage of randomisation, within each cluster, three CBCCs were then selected at random for the study. As six clusters had fewer than three CBCCs available for the study, in order to select a full sample of 30 CBCCs per treatment arm, additional CBCCs were randomly selected from three clusters (Gologota, St. Pius and Machereni). Initial power calculations and resource availability had suggested the adoption of a sample with 30 clusters (communities) per treatment arm with 20 households in each cluster to identify reasonable treatment impacts of the intervention on the primary study outcomes. Data for power calculations was obtained from the 2010 Demographic and Health Survey (DHS) survey. We calculated means, standard deviations and intracluster correlation coefficients (ICCs) for rural children in Malawi. For Dietary Diversity Score (DDS), the mean and standard deviation for rural children aged 0–5 years were estimated to be 2.5 and 1.03, respectively. The ICC was 0.01. Plots of the standardised minimum detectable effect size (MDES) against the number of clusters assuming a sample of 25 children measured in each community (cluster size), consistent with 20 household interviews per community and considering that several children may end up not being tested, showed that only a marginal gain can be obtained by expanding the sample beyond 20 as power is mainly driven by the number of clusters [21] (see Fig. 2 for example simulations of MDES versus number of clusters with high and low ICCs). Diet diversity: minimum detectable effect size versus number of clusters, simulations with high and low intracluster correlation coefficients (ICCs) After preliminary design visits to the targeted communities, the sampling strategy was modified to account for the implementation approach adopted by Save the Children, involving the clustering of CBCCs around surrounding primary schools. As a result, the cluster, or unit of randomisation was the primary school cluster that included a number of different CBCCs, rather than the CBCC itself. Adjusting for ICCs at the primary school cluster level, where 60 CBCCs were clustered into two groups of 15 primary school clusters, would provide 80% power to detect a 0.24-SD difference between treatment groups at the 5% level of significance (Table ​(Table33). Sample sizes Note: Number of children estimated based on demographic data from the Demographic and Health Survey (DHS), 2010 CBBCs community-based childcare centres The sampling of households was conducted through a census within a certain catchment area for each CBCC including information on the target age groups living within each household. Households with children aged 3–5 years were then randomly selected for participation in the survey. The impact evaluation includes child-, caregiver-, household-, CBCC- and village-level data collection (Table 4). The household questionnaire collected data at the household level as well as for each relevant household member separately (main caregiver and all children in reference age groups). Survey questionnaire modules CBCC community-based childcare centre, IYCF Infant and Young Child Feeding, MDAT Malawi Development Assessment Tool The randomised design allows for the identification of causal impacts of interventions using comparisons of mean outcomes between the randomised treatment arms at end line. The analysis will follow the intention-to-treat approach as protocol and as treated, using econometric analysis for all the relevant outcomes of the intervention. Following [22], impact will be assessed using both a ‘difference-in-difference’ (DID) estimator and a single difference analysis of covariance (ANCOVA) model. Depending on the level of clustering of the outcome, we will employ multilevel regression models that account for the hierarchical nature of the data [23]. Multilevel models, also known as mixed-effects models, use both fixed effects (covariates) and random effects in at-school and household levels. The DID estimate is calculated as the average change in the outcome of interest in the treatment arm minus the change in outcome in the control group. A difficulty of DID analysis involves serial correlation [15] resulting from unobserved factors affecting the outcomes that are themselves correlated over time. Serial correlation affects estimated standard errors and can lead to erroneous acceptance or rejection of null hypotheses but not the estimation of the effect size of the intervention. It may, therefore, lead to erroneously finding or not finding a statistically significant impact of the intervention. This problem can be addressed by calculating clustered standard errors [24]. Clustered standard errors will also be employed in all cases in which correlated outcomes are observed within the same unit of analysis. The analysis of covariance (ANCOVA) estimator has been shown to provide a more efficient estimate of programme impact when autocorrelation of outcomes is low [22]. The large dataset will allow for extensive subgroup analyses, including gender, age and geographic characteristics. The impacts of preschool feeding may be quite heterogeneous and context specific [24, 25]. School meals, for instance, have been associated with marked improvements in school participation of girls in rural areas where there are large gender disparities in access to education [26]. Furthermore, smallholder farmers targeted by the programme are mostly female. Cost data will be collected retrospectively following an ingredients approach using a semistructured questionnaire. The survey will be based on a standardised costing framework capturing capital (fixed) and recurrent costs incurred at the school level. The questionnaire will also cover both cash and in-kind contributions and will be used to estimate both financial and economic costs. Financial costs capture actual expenditures in terms of programme implementation on an annual basis. Economic costs included the opportunity costs of community members, teaching staff and other stakeholders involved in the intervention provision. Opportunity costs of preschool staff and community members will be calculated using local pay scales. Capital costs will be annuitised over the useful life of all relevant school-level assets using a discount rate of 3% as per World Bank recommendations. Annuitisation enables an equivalent annual cost to be estimated and reflects the value in-use of capital items, rather than reflecting when the item was purchased. Process and output data covering the adequacy of the service delivery will be collected from monitoring visits on a quarterly basis using standardised data collection forms. Output data will be combined with the costs to provide estimates of cost-efficiency metrics, including costs per beneficiary, kilocalories, iron, and vitamin A delivered. Sensitivity analysis will be undertaken to account for uncertainties in the economic evaluation. The figures obtained in this way will then be compared to figures calculated for other interventions. The enumerators will be recruited from Chancellor College, University of Malawi and trained for the baseline survey. Each team, led by a supervisor and assisted by community leaders, will conduct household listings and sampling in each enumeration area. The data collection will be undertaken using electronic tablets. Data collection will be reviewed daily by a team supervisor and inconsistencies clarified. Dietary assessment will be undertaken by trained and supervised enumerators using the interactive 24-h recall method. Prior to the recall interview, caregivers will be briefed on the purpose and methods of interview. The interview will be conducted using visual aids to assist in estimating portion sizes of the foods consumed. The 24-h dietary assessment will be repeated on nonconsecutive days for a subset of households (approximately 20%) to obtain estimates of usual intake [27]. Anthropometry collection will include measurements of children’s height and weight. Height or length will be measured to the nearest 0.1 cm using portable fixed base stadiometers and weight will be measured to the nearest 0.1 kg using electronic scales. The height and weight measures will be assembled and placed on a level surface. In the absence of a level ground in the household, a suitable place will be identified for the measurement in the community. Training on the MDAT will be provided to all supervisors and enumerators by trained staff from the College of Medicine, Malawi. During the MDAT training all enumerators will be reviewed for consistency and reliability.