Accuracy of women’s self-report of birth attendants

Population-based surveys rely on women’s self-report of the cadre of birth attendant to ascertain whether the attendant was skilled or not. We systematically reviewed published literature to examine how accurately women can recall the cadres of healthcare personnel assisting them during delivery. Results are summarised below, with a full description of methods and results shown in online supplementary appendix 1. Of 1743 titles screened, and 29 full texts reviewed, 4 studies (in Mexico, Kenya and Ghana) met the inclusion criteria comparing women’s self-reports against a gold standard, but only three were large enough to calculate statistical measures of validity or reliability (table 1).

The limited evidence suggests that SBA coverage at the population level can be accurately estimated for an aggregate category of ‘skilled’ providers, but that women have difficulty accurately reporting distinct cadres of healthcare professionals and, according to Blanc and colleagues’ study in Kenya, women tend to underreport attendance by less skilled providers.18 The high sensitivity but low specificity of the SBA indicator suggests that women’s self-reports in population-based surveys are close to capturing the true prevalence in high SBA-coverage settings but would tend to overestimate it in low coverage settings. The four studies were all conducted among women delivering in hospitals, with very few deliveries reported (by women or via the gold standard comparison) as being attended by unskilled attendants. Thus, the findings may not be generalisable to lower-level facility or home-based settings where delivery with non-SBA is likely to be more common. Three of the four studies interviewed women very soon after delivery. Only one study used a longer recall period (up to 15 months) similar to household surveys; it showed some deterioration in the accuracy of women’s reporting.19

Women use multiple ways to determine the cadre of their provider. Hussein and colleagues found women in Ghana most often identified the cadre of providers by the activities they performed, by the colour of the uniform or by knowing the individual previously, though these means were frequently inaccurate.20 Qualitative interviews by McCarthy and colleagues in Kenya found women differentiated cadres by the care components they gave, with broad agreement on the kinds of interventions performed by higher cadre providers (eg, caesarean sections). However, many women could not distinguish between types of skilled cadres if multiple attendants provided needed care.19

The four studies asked women who was their ‘main attendant’, rather than using the DHS/MICS question identifying all those assisting with the delivery (figure 1). Women—and researchers—may have differing interpretations of the term ‘main attendant’. Interpretations could include recalling the person who spent the most time with the woman, was the most helpful or was the most qualified, leading to potential discrepancies between women’s responses and observers’ reports. For example, in the study observing deliveries in Mexico, researchers noted the ‘main provider’ to be the person who ‘caught’ the baby.21 The tendency in the Kenya study for women to overreport attendance by doctors and higher-ranked providers compared with observers may have occurred because respondents understood their ‘main provider’ to be the person in-charge, whereas observers recorded the cadre of individual administering the most care.18

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Question and response options from sample MICS and DHS individual women's questionnaires. DHS, Demographic and Health Surveys; MICS, Multiple Indicator Cluster Surveys.

In addition to these four studies that tested recall of individual cadres, other studies have examined the accuracy of women’s recall of other questions on intrapartum care provision. In a study in Scotland, where women were allocated a midwife, obstetrician or general practitioner as a maternity care coordinator, less than half of women reported the same cadre as specified by staff.22 A study in Mozambique found women’s recall of the number of health professionals assisting during delivery was inaccurate compared with direct observations (area under the curve (AUC): 0.48, 95% CI: 0.44 to 0.53), but that women could more accurately report the presence of a companion during labour or delivery (AUC: 0.61, 95% CI: 0.55 to 0.66).23 The study also found high individual-level accuracy (AUC: 0.88, 95% CI: 0.84 to 0.91) and low population-level bias in women reporting whether they delivered in a hospital or health centre at 8–10 months recall.23

Methodological issues in SBA coverage measurement on population-based surveys

In addition to the issues arising from women’s self-report of individual cadres, the accuracy of SBA coverage measurement also relies on how these data are captured and recorded in household surveys. DHS and MICS provide standard questionnaires with multiple-choice response options which countries can customise to reflect the national cadres providing intrapartum care (figure 1). Countries indicate whether these cadres are considered ‘skilled’ for estimating SBA coverage based on the Ministry of Health’s (MOH) definition of SBA; this may change over time. There is currently a lack of transparency on what basis cadres are designated as ‘skilled’. Improving population-based measurement of SBA demands clarity and comparability in response options, careful phrasing and probing to reduce improbable responses and represent all women with births in the recall period.

The lack of clear standards for classifying a delivery attendant as ‘skilled’ on DHS and MICS has been widely acknowledged.24 Cadres with the same or similar professional designations can have different competencies in different countries,16 and cadres may change in whether they are considered skilled from one survey to the next. For example, in Ethiopia, the Health Extension Worker (HEW) cadre was not considered skilled in the 2011 DHS25 but was considered skilled in the 2016 DHS26 because HEWs had received an additional 1 year training. Recent evaluations show that HEWs with minimal training lack the competencies to be considered skilled, but those with the additional 1 year higher level training do demonstrate greater competence.27 However, women cannot be expected to differentiate HEWs with differing levels of training, and it is unclear whether even the more highly trained HEWs meet the 2018 joint statement criteria to be considered SBA.

Our estimates from 23 countries with a DHS or MICS conducted since 2015 (and data available in February 2018) show the close alignment, with a few exceptions, of the percentage of facility-based deliveries with SBA coverage (table 2, see online supplementary appendix 2 for methods). Notably, Mali and Senegal had higher facility-based deliveries than SBA coverage estimates, indicating some women giving birth in facilities reported assistance by non-SBAs, namely with a ‘matrone’, an auxiliary cadre providing delivery care in many Francophone countries who may or may not be skilled.28 In fact, the29 Mali 2015 MICS report listed SBA coverage including ‘matrone’ (60.4%) and excluding ‘matrone’ (43.7%),29 openly reflecting uncertainty as to whether the country-level categorisation meets international definitions (see online supplementary appendix 2 for a list of individual cadres included in DHS/MICS reports’ SBA coverage estimates). In order to have comparable indicators, we need to be confident that cadres designated by countries as skilled meet international standards for SBA.

DHS and MICS also often combine response categories of cadres (eg, nurse/midwife, see figure 1), possibly in response to respondents’ inability to accurately distinguish distinct cadres. These combined cadres on the questionnaire or in the datasets limit the ability to check individual cadre competencies against international SBA criteria or assess the role of specific cadres separately. This is less important if all the cadres in such combined categories are skilled; however, categorisations that combine skilled personnel with lower-level cadres who may not be considered skilled lead to inaccuracies in SBA coverage.24

The question on DHS and MICS to capture SBA coverage reads: ‘Who assisted with delivery of (NAME)?’ with an instruction for enumerators to probe for all the persons assisting (figure 1). However, the question phrasing could be interpreted in different ways within different contexts. We explored two of the ways this impacts on estimates of SBA coverage.

First, surveys attempt to capture all those assisting with delivery, but results can be difficult to interpret. Table 2 uses data from 23 countries to show the large variation in the percentage of women delivering in a facility who reported two or more cadres, from 7% in Malawi to 91% in Turkmenistan. This wide range could reflect the reality of differently resourced health systems, or the quality of data collection, including how much time the enumerators had to probe, or how the term ‘assisted’ was interpreted when translated into local languages. In countries where a team of providers might be considered to encompass all the required competencies under the 2018 joint statement, a woman responding with a single cadre when multiple cadres were supporting care, including those with more skills performing supervisory roles, would tend to underestimate SBA coverage.

Second, misreporting and poor understanding of the survey question may be responsible for seemingly implausible responses, for example, women reporting delivery with a doctor at a health post in settings where this is unlikely,7 or women indicating delivery by caesarean section with a non-SBA. In Senegal, 10% of women with caesarean sections reported delivery assistance only by non-skilled attendants.30 It is unclear if these women were interpreting the question to refer to cadres assisting before surgery, whether they were misreporting the cadre or whether they were misreporting the caesarean section. Possible solutions would be to provide an alternate phrasing of the survey question for women delivering by caesarean section to ask who performed the operation or further enumerator instructions to check obvious discrepancies.

Finally, current estimates of SBA coverage omit some births to women in need of skilled delivery care, by excluding women with stillbirths. By only including live births, SBA coverage estimates may be artificially inflated since they fail to capture the providers of care during adverse delivery outcomes. We recommend changing the denominator to be all births (live and stillbirths), while recognising that survey design changes may be needed to improve reporting.31 For instance, evidence suggests women’s ability to accurately report birth attendants deteriorates somewhat over time. We recommend recall periods be shortened to strike a balance between accurate recall and adequate sample sizes for analysis, and support the planned reduction of the 5-year DHS recall period to match the 2-year MICS recall period. The resulting reduced time in completing the questionnaire may also improve data quality.32 A denominator of all births would still enable disaggregation by live births for comparison to older survey estimates (ie, bridge coding).

Ways forward for improved measurement of SBA coverage

The global call for improving the quality of maternal and newborn care33–35 demands we acknowledge the conceptual and measurement limitations of SBA coverage as an indicator of progress. Meaningful measurement of SBA coverage under the 2018 joint statement would capture the proportion of all births that are cared for by an individual healthcare provider or team who can provide all the internationally defined components of maternal and newborn care in an enabled environment.2 The current approach of tallying the most skilled cadre(s) present, as reported by the woman, ignores the extent of assistance that was provided, whether the healthcare provider had the necessary support or supervision or whether they were working in an environment equipped with the drugs and infrastructure needed to perform routine and emergency care. The way forward must include improving documentation for classifying country cadres as ‘skilled’, capturing the enabling environment and understanding the complexity of care provision at the country level.

Understanding complexity in skilled assistance

As illustrated in the example of Kenya, SBA coverage estimates tell only part of the story, and further disaggregation of where and with whom women deliver can help to unravel what delivery care looks like in its complexity.7 The patterns arising can inform efforts to improve delivery care that are tailored to the specific country and health system contexts.

In response to the 2018 joint statement that a team of MNH professionals could cover the required competencies, we examined how women reported different combinations of attendants among facility-based deliveries (figure 2, see online supplementary appendix 2 for methods). For nearly all facility-based deliveries in Turkmenistan (91%), women reported assistance by both a doctor and a nurse/midwife. In Mali, the majority of women reported assistance by a nurse, midwife or matrone (55%). However, nearly a third of women delivering in facilities in Mali reported that a combination of both a doctor and a matrone, traditional birth attendant (TBA) or community health worker assisted, of whom only doctors are SBAs. This suggests that reliance on SBA coverage as indicated by ‘doctor’ as the highest cadre of attendant would mask these possible collaborations between the formal health system and community-based, potentially unskilled cadres, though the extent of assistance performed by the different cadres may vary considerably. Measurement efforts should consider capturing teams of professionals assisting with deliveries to enhance understanding of a country’s provision of childbirth care.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Attendants at delivery among all facility-based deliveries in recall period. TBA, traditional birth attendant; CHW, community health worker; MICS, Multiple Indicator Cluster Surveys.

Disaggregation and linking of current data sources provide additional insight, but engaging with local, contextual knowledge and refining the survey question are needed to interpret and understand what delivery assistance actually looks like. For common patterns, such as the nearly one-third of women in Mali reporting delivery with a doctor and lower-skilled cadre (figure 2b), research is needed to understand who is present and/or provides most of the care during labour and delivery, and who is responsible for intervention in the event of complications. More clarity is also needed on what we are currently capturing as attendant at birth in population-based surveys. The survey question could be rephrased to more accurately capture everyone who was present at any point during labour and delivery, the person(s) assisting at a critical moment (eg, the person who caught the baby), or the person in charge. It is worth keeping in mind, however, that women are in a unique situation during childbirth and may be unaware of or unable to recall the details needed to answer some of these questions. Women would also not be expected to know, for example, who was supervising the ward or her attendants if they were not directly involved in her care. Efforts to refine the question should, in the long term, join forces with movements promoting respectful care, which encourage providers to introduce themselves and let woman know their name and position.