Design and participants

A quasi-experiment was conducted, nested in the 2015 Pelotas (Brazil) Birth Cohort Study. Pelotas is a city in southern Brazil, with around 340 000 inhabitants. All hospital-delivered children live-born in Pelotas between 1 January 2015 and 31 December 2015, whose mother lived in the urban area of the city, were eligible for inclusion in the study. Out of the 4333 eligible live births, 4275 were assessed at birth (response rate 98.7%). All these children and their mothers were invited to follow-up assessments at 3, 12, 24 and 48 months; the 4-year follow-up was conducted in a university research centre, while the earlier postnatal visits were conducted in the children’s homes. Further information about the 2015 Pelotas Birth Cohort is available elsewhere.26 At 4 years, when the main outcome data for the current analyses were collected, 4010 participants were assessed (mean age=3.8 ± 0.2 years), and 67 participants were identified as having died (giving a total follow-up rate of 95.3% of the original cohort).

Primary data from the cohort and secondary data from the state PIM information system were linked based on municipality (Pelotas), child’s date of birth, child’s name and mother’s name. The data collected from the PIM information system were age at admission, length of stay in the programme, reason for withdrawal from PIM, number of different visitors who accompanied the child (carried out home visits) throughout their participation in the programme, and involvement of any older sibling in the programme. PIM funding depends on the number of children registered by the municipality, so it is unlikely that children receiving the intervention were not registered.

The mothers and interviewers were blind to the hypotheses of this study. The assessors of child development were not aware of the child’s participation in PIM, or of the aim of the current study to evaluate the impact of PIM. The linkage of databases was carried out without any involvement of the assessors who collected information about the outcome and the potential confounders at cohort follow-ups.

Intervention

PIM was first implemented in Pelotas in 2003, and is managed by the Municipal Health Department with direct support from the State Health Department, which developed the programme. It aims to enhance sensitive and responsive caregiver-child interactions through engagement in age-appropriate play activities, along with provision of information for nurturing care and facilitating access to health and social services. Weekly home visits (45–60 min) are made by trained health, education and social science undergraduate students, selected by the health authority to work with children and their caregivers in this role; additional group activities are used with 3–5 years. Visitors are selected via a process including an initial 60-hour training period (selection takes place partly during this training period), and then selected visitors receive ongoing weekly training. Each visitor serves a maximum of 17 families with a maximum weekly workload of 30 hours. They receive scholarship support in this role from the Municipal Health Department (not from their university), for a maximum duration of 2 years. They are directly supervised by a monitor responsible for up to eight visitors, with whom they discuss the families’ care plan based on programme materials and conduct joint home visits using an observation guide to assess the quality of the visit and provide feedback. The monitors are visitors who, after a period of fieldwork with families, are identified by the municipal management group as having a profile for team leadership, and are hired with the same type of contract as the visitors. Home visits involve listening to the family, reviewing the activities of the last week, conducting a playful activity aimed at stimulating child development and planning activities for the next week. Information on child health and nutrition is also provided, and referrals to healthcare services or social assistance are made as necessary. Routine assessments of the child’s development are made every 3 months.27

PIM focuses on families with greater social vulnerability, although no objective eligibility tool was used. The families included were those identified by visitors and municipal staff as vulnerable during day-to-day work, or those indicated by healthcare services or even by previously assisted families. Inclusion also depended on resources available and the family’s agreement to participate. The families of children enrolled in PIM were mostly of low socioeconomic status: 34% and 29%, respectively, belonged to the poorest and second poorest quintiles of family income at birth. Additionally, 54% of PIM mothers had eight or less years of education, compared with 27% of the mothers of children who did not receive the intervention (online supplemental table 2). Nonetheless, from a population perspective, PIM was not applied to all those who would be considered to be priorities for programme eligibility: 67% of all the children in the cohort in the lowest quintile of family income at birth were not included in PIM. The relatively low coverage of the programme among those in need made it possible to apply the quasi-experimental method used in this study.

We examined the intervention defined in two different ways according to timing of enrolment. First, we examined the impact of any enrolment in PIM: all children enrolled in the programme from any age up to 4 years were considered as having participated in PIM. Second, we examined the impact of participating in PIM according to whether families were enrolled during the mother’s pregnancy (with the focal study child) or after birth.

Outcome

Child development was measured at age 4 years using the screening version of Battelle’s Developmental Inventory (BDI). This instrument consists of 96 items divided in five domains of neurodevelopment (personal-social, adaptive, motor, communication and cognitive) for children ranging in age from birth to 8 years.28 This had previously been translated to Brazilian Portuguese and was adapted to form a reduced 66-item instrument (using all items for each age level from birth to 4–5 years of age, but excluding items for older ages).29 BDI was applied by trained interviewers who were supervised by child development psychologists. The instrument was divided into 13 questions for the mothers (applied first) and 53 items or fewer that were directly applied to or observed among the children (without the mother’s presence in the room). After applying the items assessing milestones for children aged 4–5 years and 3–4 years, which are unconditionally applied to all children, the evaluation continued with application of items relevant to younger ages (first ages 2–3 years, then younger). The evaluation (each domain) terminated when the child achieved the maximum score (2) for two consecutive items. At that point, items referring to lower-difficulty (younger age) skills were automatically scored as two points.

A total development score ranging from 0 to 132 and scores for each subdomain were standardised based on their distribution in the study sample. Children with a development score <50 were excluded because they were considered to have a severe mental deficit. The total score (after exclusion of children scoring <50) was also dichotomised to define a group with low developmental score, using the cut-off point for the 10th percentile; this identified children whose developmental score did not surpass that expected of children aged 30 months.

For age 4 years, the BDI instrument presents good validity for predicting later development.30 Quality control was performed in the current study for 200 randomly selected children, through use of videos recording the application of the instrument to the child. The total score calculated, based on the coding by trained interviewers, showed strong agreement with the total score calculated from coding by senior psychologists who observed the videos, taking into account the application environment, the interviewers' approach and the children’s responses (kappa statistics indicated strong or excellent agreement for all the domains analysed).

Statistical analysis

After linking databases and identifying children in the cohort who received the PIM programme, propensity scores (PS) were calculated31 for the probability of participation in PIM. Subsequently, we matched each child who received PIM to one participant from the pool of potential controls based on their PS, to subsequently estimate the effect of PIM on child development. Altogether, 27 covariates (details of measurements are presented in online supplemental box 1) were used in order to estimate the PS using logistic regression, in which participation in PIM was the dependent variable. First, covariates weakly associated (p<0.20) with both the intervention and the outcome were used to estimate a preliminary PS and balance was examined. Any additional covariates that were imbalanced after this initial matching were then included in a new logistic regression model (along with the originally included covariates) to improve balance. Individuals with missing data for any covariate used in this logistic model were excluded from the analytical sample, given imputation methods would be difficult to operationalise in paired analysis with double adjustment (further adjustment of covariates in final estimates of the effects of PIM on study outcomes). The covariate with the highest percentage of missing data (couple’s relationship quality: 17.5%) was not included in the PS calculation, in order to reduce losses.

All potential confounders were measured from maternal reports during the perinatal assessment, except for the following: main caregiver until the child reached 3 months of age; depressive maternal symptoms and the couple’s relationship quality, which were measured at the 3-month assessment; childcare attendance, which was measured at the 2-year assessment; and neighbourhood violence, which was measured at the 4-year assessment. Having an older sibling who had received PIM was included as a covariate to reduce residual confounding, since families previously involved in the programme, but whose study child (from the 2015 birth cohort) did not receive the intervention, were considered to provide robust controls against self-selection bias.

PS distribution curves were compared for groups that received and did not receive PIM, for an initial assessment of the plausibility of adequate matching. PS matching was performed without replacement, starting with individuals in the intervention group with the highest PS value. After matching, covariate balance was assessed for all 27 covariates, considering a standardised mean difference of 0.1 as a maximally acceptable difference between groups.32

Analyses of the BDI score were based on linear regression. Analyses of the dichotomous indicator of ‘belonging to the group below the 10th percentile’ used Poisson regression for direct estimation of prevalence ratios.33 In both analyses, the matched pairs were treated as clusters, through the ‘survey’ command. This allowed straightforward additional adjustment for covariates in the outcome regression, thus enabling ‘double adjustment’ for covariates included as predictors in estimating the PS, which were all prognostically important factors. In view of the matching process, the results can be interpreted as the mean causal effect of the PIM in the intervention group (ie, the causal effect in the treated group).

We initially analysed any enrolment in PIM, to evaluate the whole programme as implemented in the population (including families enrolled at any time between pregnancy and child age 4 years). Afterwards, the intervention group was stratified according to whether families enrolled in PIM during or after pregnancy, and separate effects were estimated for these different enrolment times. For these stratified analysis, children who did not received PIM were randomly divided proportionally to the number of individuals in each intervention subgroup (PIM starting during or after pregnancy). Covariate balance between the two randomly generated non-PIM groups was examined to assess their interchangeability. For each stratum of the intervention group (enrolled during pregnancy or after birth), the steps described above for matching and analysis were conducted separately. For the stratum with initiation of PIM after birth, the same set of 27 covariates was used in the analysis. For the stratum starting during pregnancy, 15 covariates were selected for analyses. These were all potential confounders that were not on the causal pathway of the effect of PIM starting during pregnancy on ECD (online supplemental box 1). Cochran’s Q heterogeneity χ² test was used to examine modification of the effect of PIM according to the timing of intervention initiation.

To investigate the programme’s potential to reduce inequalities in child development, effect modification according to family income quintile at birth was explored in double-adjusted analysis with low BDI (below the 10th percentile) as an outcome—in situations where significant main effects were found in the primary analyses. All analyses were conducted using Stata V.15.1.

Patient and public involvement

The public was not involved in the design or conduct of our research. The municipal and state management of the PIM was involved in the planning of this evaluation. The results are being disseminated and discussed with those responsible for implementing the programme, to improve its impact.