‘Who assisted with the delivery of (NAME)?’ Issues in estimating skilled birth attendant coverage through population-based surveys and implications for improving global tracking

Introduction

Skilled attendance at birth is believed to prevent maternal and neonatal mortality during labour, delivery and the early postnatal period via effective care provided by trained and competent health personnel in an environment equipped with the necessary drugs and equipment.1–3 The percentage of live births attended by specified cadres of health personnel (eg, midwives) is widely used to measure skilled birth attendant (SBA) coverage and is a key proxy indicator for monitoring progress in reducing maternal mortality (see box 1). While household surveys, such as the Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), have measured delivery with medically trained personnel (doctor, nurse or midwife) since the 1980s and 1990s, SBA coverage was first tracked globally during the Millennium Development Goal (MDG) era (2000–2015), and remains an indicator on the Sustainable Development Goal (SDG) and Global Strategy for Women’s Children’s and Adolescents’ Health agenda. Yet, the discrepancy between increasing SBA coverage and lagging improvements in maternal and neonatal mortality in many low-income and middle-income countries (LMICs) has brought increasing attention to the challenge of SBA measurement and the indicator’s limitations in capturing effective, life-saving care.4–7 Amidst increasing calls to move away from measuring contact with the health system toward capturing quality of care,8 a critical review and reflection on SBA coverage measurement is needed.

In 2004, the WHO, along with the International Confederation of Midwives (ICM) and the International Federation of Gynaecology and Obstetrics (FIGO) defined the core skills and training required for skilled attendants.1 WHO considered the statement to be a policy document and has indicated it was not intended to serve as a basis for measurement.9 Shortages of midwifery-trained personnel led many LMICs to ‘upskill’ mid to low level health cadres to expand the range and numbers of health personnel they considered to be SBAs to address gaps in coverage.10–12 There are concerns that the length and quality of training may be insufficient,9 12 that large gaps exist between international standards and actual provider competencies and practices13 14 and that the lack of standardisation in cadre names, training and competencies make cross-country comparisons challenging.10 15 16

In the transition from the MDG to the SDG era, WHO, Unicef and United Nations Population Fund (UNFPA) undertook a process of revising the definition of SBA. The resulting 2018 joint statement expanded the 2004 statement to consider eight competencies essential for improving the quality of care received by women and newborns.2 The competency related to intrapartum care was identified as the one specifically relevant to SDG monitoring; it defined a competent maternal and newborn health (MNH) professional as someone who (1) provides and promotes evidence and human rights-based, quality, socioculturally sensitive and dignified care to women and newborns; (2) facilitates physiological processes during labour and delivery to ensure a clean and positive childbirth experience and (3) identifies and manages or refers women and/or newborns with complications.2 The care competencies include all signal functions of emergency obstetric and newborn care that could be provided by an integrated team of MNH professionals, and the 2018 joint statement noted the importance of the enabling environment to the provision of quality maternal and newborn healthcare.2 9

The 2018 joint statement also acknowledged that it was a first step towards improved measurement and called for household surveys and administrative data collection methods at the country level to be revised to support ‘meaningful measurement’ of SBA coverage.2 In parallel to the process of reviewing the definition of SBA, Unicef and WHO convened experts on several occasions between 2016 and 2018 to reflect on global tracking of SBA coverage. The aim was to improve measurement of the SBA indicator, and distinguishing which healthcare providers were ‘skilled’ for the purposes of the indicator and improving country-level reporting.2 9 To understand what this means in practice, we need to unpack what women can tell us and how the SBA indicator is currently generated via population-based surveys.17 Then, we can consider the implications of the WHO definitions (2004 and 2018 versions) on meaningful measurement of SBA coverage and effective tracking of progress in MNH. We acknowledge that country-level monitoring needs may differ from what is measured at the global level, but here we focus on the challenges and implications for global tracking of SBA coverage. In this paper, we aim to (1) systematically collate and synthesise published evidence on whether women can accurately recall the person(s) assisting with delivery, (2) describe methodological considerations and points of improvement around how SBA coverage is estimated from DHS and MICS and (3) suggest ways forward in conceptualising SBA coverage and measuring care received by women and newborns.

Accuracy of women’s self-report of birth attendants

Population-based surveys rely on women’s self-report of the cadre of birth attendant to ascertain whether the attendant was skilled or not. We systematically reviewed published literature to examine how accurately women can recall the cadres of healthcare personnel assisting them during delivery. Results are summarised below, with a full description of methods and results shown in online supplementary appendix 1. Of 1743 titles screened, and 29 full texts reviewed, 4 studies (in Mexico, Kenya and Ghana) met the inclusion criteria comparing women’s self-reports against a gold standard, but only three were large enough to calculate statistical measures of validity or reliability (table 1).

The limited evidence suggests that SBA coverage at the population level can be accurately estimated for an aggregate category of ‘skilled’ providers, but that women have difficulty accurately reporting distinct cadres of healthcare professionals and, according to Blanc and colleagues’ study in Kenya, women tend to underreport attendance by less skilled providers.18 The high sensitivity but low specificity of the SBA indicator suggests that women’s self-reports in population-based surveys are close to capturing the true prevalence in high SBA-coverage settings but would tend to overestimate it in low coverage settings. The four studies were all conducted among women delivering in hospitals, with very few deliveries reported (by women or via the gold standard comparison) as being attended by unskilled attendants. Thus, the findings may not be generalisable to lower-level facility or home-based settings where delivery with non-SBA is likely to be more common. Three of the four studies interviewed women very soon after delivery. Only one study used a longer recall period (up to 15 months) similar to household surveys; it showed some deterioration in the accuracy of women’s reporting.19

Women use multiple ways to determine the cadre of their provider. Hussein and colleagues found women in Ghana most often identified the cadre of providers by the activities they performed, by the colour of the uniform or by knowing the individual previously, though these means were frequently inaccurate.20 Qualitative interviews by McCarthy and colleagues in Kenya found women differentiated cadres by the care components they gave, with broad agreement on the kinds of interventions performed by higher cadre providers (eg, caesarean sections). However, many women could not distinguish between types of skilled cadres if multiple attendants provided needed care.19

The four studies asked women who was their ‘main attendant’, rather than using the DHS/MICS question identifying all those assisting with the delivery (figure 1). Women—and researchers—may have differing interpretations of the term ‘main attendant’. Interpretations could include recalling the person who spent the most time with the woman, was the most helpful or was the most qualified, leading to potential discrepancies between women’s responses and observers’ reports. For example, in the study observing deliveries in Mexico, researchers noted the ‘main provider’ to be the person who ‘caught’ the baby.21 The tendency in the Kenya study for women to overreport attendance by doctors and higher-ranked providers compared with observers may have occurred because respondents understood their ‘main provider’ to be the person in-charge, whereas observers recorded the cadre of individual administering the most care.18

Request permission

In addition to these four studies that tested recall of individual cadres, other studies have examined the accuracy of women’s recall of other questions on intrapartum care provision. In a study in Scotland, where women were allocated a midwife, obstetrician or general practitioner as a maternity care coordinator, less than half of women reported the same cadre as specified by staff.22 A study in Mozambique found women’s recall of the number of health professionals assisting during delivery was inaccurate compared with direct observations (area under the curve (AUC): 0.48, 95% CI: 0.44 to 0.53), but that women could more accurately report the presence of a companion during labour or delivery (AUC: 0.61, 95% CI: 0.55 to 0.66).23 The study also found high individual-level accuracy (AUC: 0.88, 95% CI: 0.84 to 0.91) and low population-level bias in women reporting whether they delivered in a hospital or health centre at 8–10 months recall.23

Methodological issues in SBA coverage measurement on population-based surveys

In addition to the issues arising from women’s self-report of individual cadres, the accuracy of SBA coverage measurement also relies on how these data are captured and recorded in household surveys. DHS and MICS provide standard questionnaires with multiple-choice response options which countries can customise to reflect the national cadres providing intrapartum care (figure 1). Countries indicate whether these cadres are considered ‘skilled’ for estimating SBA coverage based on the Ministry of Health’s (MOH) definition of SBA; this may change over time. There is currently a lack of transparency on what basis cadres are designated as ‘skilled’. Improving population-based measurement of SBA demands clarity and comparability in response options, careful phrasing and probing to reduce improbable responses and represent all women with births in the recall period.

The lack of clear standards for classifying a delivery attendant as ‘skilled’ on DHS and MICS has been widely acknowledged.24 Cadres with the same or similar professional designations can have different competencies in different countries,16 and cadres may change in whether they are considered skilled from one survey to the next. For example, in Ethiopia, the Health Extension Worker (HEW) cadre was not considered skilled in the 2011 DHS25 but was considered skilled in the 2016 DHS26 because HEWs had received an additional 1 year training. Recent evaluations show that HEWs with minimal training lack the competencies to be considered skilled, but those with the additional 1 year higher level training do demonstrate greater competence.27 However, women cannot be expected to differentiate HEWs with differing levels of training, and it is unclear whether even the more highly trained HEWs meet the 2018 joint statement criteria to be considered SBA.

Our estimates from 23 countries with a DHS or MICS conducted since 2015 (and data available in February 2018) show the close alignment, with a few exceptions, of the percentage of facility-based deliveries with SBA coverage (table 2, see online supplementary appendix 2 for methods). Notably, Mali and Senegal had higher facility-based deliveries than SBA coverage estimates, indicating some women giving birth in facilities reported assistance by non-SBAs, namely with a ‘matrone’, an auxiliary cadre providing delivery care in many Francophone countries who may or may not be skilled.28 In fact, the29 Mali 2015 MICS report listed SBA coverage including ‘matrone’ (60.4%) and excluding ‘matrone’ (43.7%),29 openly reflecting uncertainty as to whether the country-level categorisation meets international definitions (see online supplementary appendix 2 for a list of individual cadres included in DHS/MICS reports’ SBA coverage estimates). In order to have comparable indicators, we need to be confident that cadres designated by countries as skilled meet international standards for SBA.

DHS and MICS also often combine response categories of cadres (eg, nurse/midwife, see figure 1), possibly in response to respondents’ inability to accurately distinguish distinct cadres. These combined cadres on the questionnaire or in the datasets limit the ability to check individual cadre competencies against international SBA criteria or assess the role of specific cadres separately. This is less important if all the cadres in such combined categories are skilled; however, categorisations that combine skilled personnel with lower-level cadres who may not be considered skilled lead to inaccuracies in SBA coverage.24

The question on DHS and MICS to capture SBA coverage reads: ‘Who assisted with delivery of (NAME)?’ with an instruction for enumerators to probe for all the persons assisting (figure 1). However, the question phrasing could be interpreted in different ways within different contexts. We explored two of the ways this impacts on estimates of SBA coverage.

First, surveys attempt to capture all those assisting with delivery, but results can be difficult to interpret. Table 2 uses data from 23 countries to show the large variation in the percentage of women delivering in a facility who reported two or more cadres, from 7% in Malawi to 91% in Turkmenistan. This wide range could reflect the reality of differently resourced health systems, or the quality of data collection, including how much time the enumerators had to probe, or how the term ‘assisted’ was interpreted when translated into local languages. In countries where a team of providers might be considered to encompass all the required competencies under the 2018 joint statement, a woman responding with a single cadre when multiple cadres were supporting care, including those with more skills performing supervisory roles, would tend to underestimate SBA coverage.

Second, misreporting and poor understanding of the survey question may be responsible for seemingly implausible responses, for example, women reporting delivery with a doctor at a health post in settings where this is unlikely,7 or women indicating delivery by caesarean section with a non-SBA. In Senegal, 10% of women with caesarean sections reported delivery assistance only by non-skilled attendants.30 It is unclear if these women were interpreting the question to refer to cadres assisting before surgery, whether they were misreporting the cadre or whether they were misreporting the caesarean section. Possible solutions would be to provide an alternate phrasing of the survey question for women delivering by caesarean section to ask who performed the operation or further enumerator instructions to check obvious discrepancies.

Finally, current estimates of SBA coverage omit some births to women in need of skilled delivery care, by excluding women with stillbirths. By only including live births, SBA coverage estimates may be artificially inflated since they fail to capture the providers of care during adverse delivery outcomes. We recommend changing the denominator to be all births (live and stillbirths), while recognising that survey design changes may be needed to improve reporting.31 For instance, evidence suggests women’s ability to accurately report birth attendants deteriorates somewhat over time. We recommend recall periods be shortened to strike a balance between accurate recall and adequate sample sizes for analysis, and support the planned reduction of the 5-year DHS recall period to match the 2-year MICS recall period. The resulting reduced time in completing the questionnaire may also improve data quality.32 A denominator of all births would still enable disaggregation by live births for comparison to older survey estimates (ie, bridge coding).

Ways forward for improved measurement of SBA coverage

The global call for improving the quality of maternal and newborn care33–35 demands we acknowledge the conceptual and measurement limitations of SBA coverage as an indicator of progress. Meaningful measurement of SBA coverage under the 2018 joint statement would capture the proportion of all births that are cared for by an individual healthcare provider or team who can provide all the internationally defined components of maternal and newborn care in an enabled environment.2 The current approach of tallying the most skilled cadre(s) present, as reported by the woman, ignores the extent of assistance that was provided, whether the healthcare provider had the necessary support or supervision or whether they were working in an environment equipped with the drugs and infrastructure needed to perform routine and emergency care. The way forward must include improving documentation for classifying country cadres as ‘skilled’, capturing the enabling environment and understanding the complexity of care provision at the country level.

Understanding complexity in skilled assistance

As illustrated in the example of Kenya, SBA coverage estimates tell only part of the story, and further disaggregation of where and with whom women deliver can help to unravel what delivery care looks like in its complexity.7 The patterns arising can inform efforts to improve delivery care that are tailored to the specific country and health system contexts.

In response to the 2018 joint statement that a team of MNH professionals could cover the required competencies, we examined how women reported different combinations of attendants among facility-based deliveries (figure 2, see online supplementary appendix 2 for methods). For nearly all facility-based deliveries in Turkmenistan (91%), women reported assistance by both a doctor and a nurse/midwife. In Mali, the majority of women reported assistance by a nurse, midwife or matrone (55%). However, nearly a third of women delivering in facilities in Mali reported that a combination of both a doctor and a matrone, traditional birth attendant (TBA) or community health worker assisted, of whom only doctors are SBAs. This suggests that reliance on SBA coverage as indicated by ‘doctor’ as the highest cadre of attendant would mask these possible collaborations between the formal health system and community-based, potentially unskilled cadres, though the extent of assistance performed by the different cadres may vary considerably. Measurement efforts should consider capturing teams of professionals assisting with deliveries to enhance understanding of a country’s provision of childbirth care.

Request permission

Figure 2

Attendants at delivery among all facility-based deliveries in recall period. TBA, traditional birth attendant; CHW, community health worker; MICS, Multiple Indicator Cluster Surveys.

Disaggregation and linking of current data sources provide additional insight, but engaging with local, contextual knowledge and refining the survey question are needed to interpret and understand what delivery assistance actually looks like. For common patterns, such as the nearly one-third of women in Mali reporting delivery with a doctor and lower-skilled cadre (figure 2b), research is needed to understand who is present and/or provides most of the care during labour and delivery, and who is responsible for intervention in the event of complications. More clarity is also needed on what we are currently capturing as attendant at birth in population-based surveys. The survey question could be rephrased to more accurately capture everyone who was present at any point during labour and delivery, the person(s) assisting at a critical moment (eg, the person who caught the baby), or the person in charge. It is worth keeping in mind, however, that women are in a unique situation during childbirth and may be unaware of or unable to recall the details needed to answer some of these questions. Women would also not be expected to know, for example, who was supervising the ward or her attendants if they were not directly involved in her care. Efforts to refine the question should, in the long term, join forces with movements promoting respectful care, which encourage providers to introduce themselves and let woman know their name and position.

Conclusion

The global health community must acknowledge the limitations of SBA as an indicator and seek to improve it by providing clear metadata on who is considered to be an SBA, and including stillbirths in the denominator. It also needs to advocate for multiple indicators reported at different levels to track the complexity of inputs required to improve MNH, including by cross-tabulating where and with whom women deliver, linking to facility assessments to characterise the enabling environment, and understanding teams of multiple attendants. Global monitoring requires harmonised indicators that lose the level of detail required to improve care and service delivery at the local level. As such, greater use of current and emerging data sources, including for disaggregation and linking, is needed to understand the kind and quality of delivery care women receive, including for respectful care. The 2018 joint statement represents an aspirational approach to care but also to coverage measurement. While improvements in measurement of the SBA indicator are possible, we must also recognise what SBA coverage can and cannot tell us about ongoing efforts to ensure high-quality, life-saving care for women and newborns.