Article Text
PDF
Abstract
Objective Assess the quality of healthcare across African countries based on health providers’ clinical knowledge, their clinic attendance and drug availability, with a focus on seven conditions accounting for a large share of child and maternal mortality in sub-Saharan Africa: malaria, tuberculosis, diarrhoea, pneumonia, diabetes, neonatal asphyxia and postpartum haemorrhage.
Methods With nationally representative, cross-sectional data from ten countries in sub-Saharan Africa, collected using clinical vignettes (to assess provider knowledge), unannounced visits (to assess provider absenteeism) and visual inspections of facilities (to assess availability of drugs and equipment), we assess whether health providers are available and have sufficient knowledge and means to diagnose and treat patients suffering from common conditions amenable to primary healthcare. We draw on data from 8061 primary and secondary care facilities in Kenya, Madagascar, Mozambique, Nigeria, Niger, Senegal, Sierra Leone, Tanzania, Togo and Uganda, and 22 746 health workers including doctors, clinical officers, nurses and community health workers. Facilities were selected using a multistage cluster-sampling design to ensure data were representative of rural and urban areas, private and public facilities, and of different facility types. These data were gathered under the Service Delivery Indicators programme.
Results Across all conditions and countries, healthcare providers were able to correctly diagnose 64% (95% CI 62% to 65%) of the clinical vignette cases, and in 45% (95% CI 43% to 46%) of the cases, the treatment plan was aligned with the correct diagnosis. For diarrhoea and pneumonia, two common causes of under-5 deaths, 27% (95% CI 25% to 29%) of the providers correctly diagnosed and prescribed the appropriate treatment for both conditions. On average, 70% of health workers were present in the facilities to provide care during facility hours when those workers are scheduled to be on duty. Taken together, we estimate that the likelihood that a facility has at least one staff present with competency and key inputs required to provide child, neonatal and maternity care that meets minimum quality standards is 14%. On average, poor clinical knowledge is a greater constraint in care readiness than drug availability or health workers’ absenteeism in the 10 countries. However, we document substantial heterogeneity across countries in the extent to which drug availability and absenteeism matter quantitatively.
Conclusion Our findings highlight the need to boost the knowledge of healthcare workers to achieve greater care readiness. Training programmes have shown mixed results, so systems may need to adopt a combination of competency-based preservice and in-service training for healthcare providers (with evaluation to ensure the effectiveness of the training), and hiring practices that ensure the most prepared workers enter the systems. We conclude that in settings where clinical knowledge is poor, improving drug availability or reducing health workers’ absenteeism would only modestly increase the average care readiness that meets minimum quality standards.
- child health
- health systems
- health systems evaluation
- maternal health
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key questions
What is already known?
Country-specific studies have shown that low-income and middle-income countries suffer shortages of providers, drugs, equipment and medical supplies.
Individual country reports of Service Delivery Indicator surveys from Kenya, Niger, Senegal, Tanzania and Uganda use summary statistics to show low rates of knowledge and high rates of absenteeism among health service providers.
What are the new findings?
This study offers a single metric that quantifies the relative contribution of staff knowledge, staff availability and other facility inputs to the probability of care readiness which meets minimum quality standards.
Our work shows severe gaps in care readiness in various countries in sub-Saharan Africa, and health provider knowledge is a particularly severe constraint on the readiness of care across countries. Across the countries sampled and the evaluated medical conditions, which account for major loss of life across the continent, healthcare providers were able to correctly diagnose only 64% of cases, and the treatment plan was aligned with the correct diagnosis in 45% of cases.
What do the new findings imply?
While effective healthcare systems need to include a variety of components (knowledge, effort, equipment and essential medicines), not all components merit the same initial priority.
Our findings suggest that in the absence of provider knowledge, even improvements in other key areas such as effort, medication and equipment cannot save patients’ lives.
As a result, health systems will need to invest in better knowledge for their health providers along with other resources.
Background
There is a growing consensus that poor-quality care is a major constraint in further improving health outcomes in low-income and middle-income countries (LMICs).1–3 Based on the most recent estimates, 8.6 million people per year in LMICs die from conditions that could be treated by health systems, and a majority of these deaths are due to receipt of poor quality care.4
Measuring quality at various levels of the health system—especially where the system fails the people in need—remains rare in LMICs. Existing indicators of structural quality, such as availability of medicine, equipment and qualified health attendants, are not sufficient to adequately capture the quality of care being offered.1 The lack of quality measurement can help to explain both why there is limited research on how to effectively transform low-quality health systems into high-quality ones, and why there are few such successful reforms in LMICs. Actionable and credible data on quality, if properly disseminated, can play an important role in holding country health systems accountable for effective and safe care and can form the basis for quality improvement.5
Our paper reports on large-scale assessments of clinical knowledge, health provider absenteeism and means to provide care in sub-Saharan Africa. We use data collected through clinical vignettes, unannounced visits and visual inspections of over 20 000 providers in over 8000 facilities in 10 sub-Saharan African countries, which together represent over 40% of the region’s total population. We focus on conditions that account for a large proportion of maternal and child mortality. We include the Strengthening the Reporting of Observational Studies in Epidemiology checklist for cross-sectional, observational data analysis in online supplemental appendix section S1.6
Supplemental material
We use these data to measure dimensions we would argue are essential for providing quality care. Clinical vignettes tell us what providers know. Unannounced visits to facilities tell us if providers are available at health facilities. Visual inspections of facilities provide information as to the means that health workers have at their disposal. To put this work in context, the provider assessment falls within the framework set out by Miller, that a holistic assessment of health worker competence should assess their clinical knowledge, if they know how to apply that knowledge, if they can show how to apply that knowledge, and if they actually apply that knowledge in practice.7 Our vignette data capture a combination of clinical knowledge and their knowledge of how to apply it. Other, complementary methods (specifically, standardised patient studies) demonstrate whether health workers can show how to apply their knowledge and whether they actually apply that knowledge in practice.8–10 Nationally representative standardised patient data are unavailable in the settings and for the wide array of conditions we study, but limitations revealed in the first aspects of the health worker assessment (what they know and if they know how to apply it) and to one aspect of practice (absenteeism) potentially imply even further limitations on the latter part (whether they can and do actually apply knowledge in practice).
Methods
Data
We draw on data from the Service Delivery Indicators (SDI) programme—an ongoing initiative to collect informative and standardised measures of what providers know, what they do and what they have to work with. The SDI programme—piloted in 2010–2011—grew out of concern about quality of services in health and education, especially in sub-Saharan Africa, most clearly (and perhaps most damagingly) manifested at the facility level, in fast-expanding systems of primary health provision.11 12 To date, the SDI programme has collected data from ten countries: Kenya (2018), Madagascar (2016), Mozambique (2014), Nigeria (2013), Niger (2017), Senegal (2010), Sierra Leone (2018), Tanzania (2016), Togo (2014) and Uganda (2013), and has since expanded to countries outside the Africa region.13–17 While every context is different, these 10 countries serve as a reasonable proxy for much of sub-Saharan Africa, with an average income per capita, life expectancy and infant mortality similar to those of the continent as a whole (online supplemental appendix table S2).
Sampling
All country datasets were representative at the national level except for Nigeria which, mainly owing to security concerns, covered 12 of 37 states and was representative at the state level. The surveys used a multistage cluster-sampling design to ensure data were representative of rural and urban areas, private and public facilities and of each facility type (eg, primary care clinics vs secondary care hospitals).
The surveys collected a broad set of facility and provider specific information, using standardised instruments, with an approach that relied on clinical vignettes, direct physical verification of provider presence through unannounced visits and visual inspections of the equipment and facilities. Every interview respondent was asked for their consent to participate in the survey. Sample instruments are publicly available.18–22
In total, the sample includes information from 89 826 vignettes across 13 754 providers and absenteeism observations from 22 747 providers across 8061 facilities in 10 countries. In each country, the sample size was selected in order to provide representative data for the country and for subgroups (eg, rural vs urban areas).
Empirical methods
There are three essential conditions for health workers to provide quality care. First, providers have sufficient knowledge to diagnose and treat patients suffering from conditions that are amenable to care. Second, they are available to provide care when patients seek service. Third, providers have the means (eg, equipment and drugs) to diagnose and treat patients. We operationalise the assessment of these necessary, but not sufficient, conditions for quality primary care using three methods. Diagnostic and treatment accuracy (knowledge) were measured using clinical vignettes focused on seven common conditions associated with the main causes of child and maternal mortality in sub-Saharan Africa: malaria, tuberculosis, diarrhoea, pneumonia, diabetes, neonatal asphyxia and postpartum haemorrhage. All seven of these are major global health challenges, with three of them—diarrhoea, neonatal asphyxia and malaria—together causing more than 180 000 000 disability-adjusted life-years (online supplemental appendix table S3). Whether providers are available to provide care at regular opening hours was assessed by verifying their presence through unannounced visits at the health facilities. Availability of drugs and functionality of equipment was assessed by visual inspections of storage facilities and consulting rooms at the health facilities.
Clinical vignettes have been used extensively in the medical literature to assess clinical knowledge of health workers. Data from vignette studies are directly policy relevant as they are closely linked to the quality of medical training and quality improvement programmes. Validation studies show that clinical vignettes, for scenarios such as those assessed as part of the SDI programme, are an effective way to capture provider knowledge.23 24 Although a health worker’s clinical vignette performance is not a measure of quality of care per se, it represents the highest level of quality care which she/he can deliver given his/her current knowledge, assuming that there are no constraints in drugs, equipment and motivations. (More details on the clinical vignettes are provided in online supplemental appendix S4.) For a measure of providers’ actual performance, studies often use standardised patients, in which actors are trained to portray a given medical condition, appear incognito at facilities for recommended treatment and report on provider activity.
The clinical vignettes were adjusted to country-specific diagnostic and treatment protocols. The enumerators presented the provider with the patient’s symptoms and prompted the provider to take the patient’s history, suggest physical examination and diagnostic tests that in principle could be conducted on-site, make a diagnosis and prescribe a course of treatment and follow-up management plan.
For this study, we identified the lowest common denominators across the country-specific protocols to define comparable criteria across countries for diagnostic and treatment accuracy for each of the conditions. This methodological choice implies that knowledge is assessed at a lower standard than more stringent WHO protocols. We discuss cross-country variation in country-specific protocols in online supplemental appendix S6. Based on the responses to each vignette, we generated two binary variables: diagnostic accuracy and treatment accuracy, which are the percentage of respondents who provided an accurate diagnosis and—separately—an accurate treatment for a given condition. Both outcomes should be viewed as measuring the minimum required knowledge to prescribe correct diagnostic and treatment services. (See online supplemental table S7 for information on the coding of the two knowledge scores.) These outcomes do not represent the ideal level of knowledge; for example, they do not account for co-morbidities.
To quantify whether providers are available to provide medical service when patients seek care, we followed a standard procedure in which enumerators visited each facility twice.25 During the first visit, information on all staff working at the facility, by professional cadre, was collected. Provider availability does not, of course, fully capture provider effort, but it is a necessary condition for treating patients. (See online supplemental appendix S8 for details on the selection of staff, drugs and equipment for surveys.) Vignettes were administered to a random sample of doctors, medical assistants, nurses and nurse-midwives (ie, any staff that provide outpatient or maternity services) who were present in the facility on the day of the first, announced visit. The first visit also included a visual inspection of drugs and equipment.
A few days after the first visit, enumerators made a second, unannounced visit to the facility and assessed the presence of a randomly preselected list of providers (or all providers, if fewer than 10 providers were available). Only staff on duty that day were included in the absenteeism measure. Staff who were away from the facility due to outreach or fieldwork were counted as present. Country-level averages for each assessment were calculated using country-specific sampling weights. This gives us a mean of the outcome in question that is representative of the country population.
Estimates of care readiness
Combining data from the clinical vignettes (to measure knowledge) with data from the unannounced visit (to measure the extent to which providers are present in the facility at regular opening hours), and visual inspections (to quantify the availability of drugs and equipment), we estimate the probability that a facility is ready to provide care that meets minimum quality standards. A facility is defined as being ready to provide care that meets minimum quality standards for a given condition if there is at least one provider present in the clinic at a given day who can correctly diagnose and treat that condition, and the facility has a minimum set of drugs to treat it. (See online supplemental appendix S9 for further details of the calculation.) The probability of care readiness that does not meet minimum quality standards represents the sum of two important quality gaps: the gap between what health workers are supposed to know and what health workers actually know, and the gap between what health workers know and what health workers can do (either due to absenteeism or a lack of essential drugs). By varying the underlying parameters used to construct the measure, we can assess the relative importance of these two gaps. This research does not examine an additional important gap, between what workers can do and what they actually do in practice. We report three estimates for care readiness for each condition. Our main estimate is based solely on the survey data. We then consider a hypothetical scenario in which all health workers are present: we replace the survey-based estimate of absenteeism at the facility level for a type of worker with 0, and re-estimate care readiness. We then repeat the same procedure, but instead of replacing the estimate for absenteeism at the facility level, we assume all facilities have the essential drugs for the condition(s) in question.
We present averages and 95% confidence intervals (CIs) by first averaging all observations across facilities (or health workers) in each country (using country-specific facility weights to calculate a representative country average), and then averaging over the country means to calculate an overall average for the whole sample. (This is a suitable approach if we treat the country as the core unit of observation; that is, we want to know how countries compare and how they perform as a group.) This procedure is equivalent to calculating a weighted average and its SE across all observations in the sample, where the weight on observation i in country j is 
, where 
denotes the (facility) weight of observation i in country j and 
is the sum of the facility weights in country k.
Ethical considerations
Each of these surveys was carried out in collaboration with the Ministry of Health in the target country. Each interview respondent provided consent to participate in the survey and was made aware that they could desist from the survey at any point. The study involved no deception of participants. The current study involves secondary analysis of the SDI data.
Patient and public involvement
Neither patients nor the public were involved in the design of this study. However, each SDI survey was discussed—both at design and results stage—at length with representatives of the country where the survey was carried out. This research article presents the analysis of data across ten countries.
Role of the funding source
The funding source did not have any role in the design, data collection, data analysis, data interpretation or writing of the report.
Results
Diagnostic accuracy
Across all conditions and countries, healthcare providers correctly diagnose 64% (95% CI 62% to 65%) of the cases. The poorest diagnostic performance is for diabetes, with only 35% (95% CI 33% to 37%) of the health workers providing a correct diagnosis (figure 1A; table 1). For the three main killer diseases for children in sub-Saharan Africa, accounting for approximately one-third of all deaths among children under age 5 in the countries surveyed, average diagnostic accuracy is 57% (95% CI 54% to 59%) for pneumonia, 59% (95% CI 56% to 61%) for diarrhoea and 85% (95% CI 82% to 89%) for malaria.
Diagnostic and treatment accuracy. Dots represent country-specific means, calculated using country-specific sampling weights, vertical bars indicate mean performance across countries, and boxes delineate the interquartile range. Data are from clinical vignettes from Kenya (2018), Madagascar (2016), Mozambique (2014), Nigeria (2013), Niger (2017), Senegal (2010), Sierra Leone (2018) Tanzania (2016), Togo (2014) and Uganda (2013). The malaria vignette was not used in the Kenya (2018) survey and the postpartum haemorrhage, neonatal asphyxia and diabetes vignettes were not used in the Senegal (2010) survey.
Diagnostic accuracy by country
There are large variations in diagnostic accuracy across conditions and across countries (figure 1A, table 1). For example, only 11% (95% CI 04% to 18%) of the providers in Togo correctly diagnose neonatal asphyxia, compared with 86% (95% CI 85% to 87%) of the providers in Kenya, and only 18% (95% CI 9% to 26%) of the providers in Togo correctly diagnose diarrhoea, compared with 95% (95% CI 93% to 97%) of the providers in Sierra Leone. The within-country variation also shows different patterns (see online supplemental figure S10), varying from relatively good performance, although still low in absolute terms, across the whole distribution of providers in the country (Kenya), to countries with relatively poor performance across the whole distribution (Madagascar), and countries that span the breadth of diagnostic accuracy (Nigeria). Online supplemental figure S11 provides additional information on the clustering of diagnostic performance. Overall, 32% of the providers manage to diagnose all or almost all of the conditions (no more than one condition not correctly diagnosed), while 27% of the providers diagnose less than half of the conditions. There are again large differences across countries. Online supplemental figure S11 divides the countries into two groups based on performance. In the higher performance group, 49% of the health workers diagnose all or almost all of the conditions, compared with 11% in the lower performing group.
Accuracy in prescribing treatment
Diagnostic knowledge is associated with knowledge of correct treatment, but a correct diagnosis is not a guarantee for prescribing correct treatment. The providers prescribed the correct (minimum) treatment following a correct diagnosis in 72% of the cases. Across all conditions and countries, healthcare providers were able both to correctly diagnose and treat 45% (95% CI 43% to 46%) of the cases. For diarrhoea, malaria and neonatal asphyxia a majority of the providers managed to do so (54% (95% CI 43% to 46%); 70% (95% CI 66% to 73%) and 55% respectively). Taking diarrhoea and pneumonia together, two common causes of under-five deaths, 27% (95% CI 25% to 29%) of the providers correctly diagnosed and prescribed the appropriate treatment for both conditions. For the other four conditions, the majority of providers either fail to diagnose the condition correctly or fail to provide the correct treatment (figure 1B; table 2).
Diagnostic and treatment accuracy by country
The required treatments varied in complexity across conditions, from prescribing a single drug or treatment (artemisinin-based combination therapies—ACTs—for malaria, dehydration therapy for diarrhoea, amoxicillin or similar antibiotic for pneumonia) to prescribing a combination of drugs for tuberculosis. There is substantive heterogeneity in providers’ ability to reach a correct treatment after correctly diagnosing the condition. As a result, the relative ranking in ability to diagnose differs from the relative ranking in diagnosing and treating patients. For some conditions, the key constraint appears to be at the diagnostic stage, as with diarrhoea, where 59% of the providers diagnosed correctly but 91% of those who did so prescribed the correct treatment, and neonatal asphyxia, where 59% of the providers diagnosed correctly but 93% of those who did so prescribed the correct treatment. For other conditions, the key constraint instead appears to be prescribing the correct treatment, as with tuberculosis, where 81% of the providers correctly diagnosed the condition but only 37% of these providers prescribed the correct treatment and postpartum haemorrhage, where 71% of the providers correctly diagnosed the condition but only 45% of them prescribed the correct treatment.
Misdiagnosis or poor knowledge about correct treatment can result in inappropriate prescription of antibiotics. The vignette for diarrhoea does not call for prescription of antibiotics. However, prescription of antibiotics for diarrhoea is common among providers both when the providers diagnosed the condition correctly—32% (95% CI 29% to 35%) of the providers then prescribed antibiotics—and when they did not—36% (95% CI 32% to 40%) of the providers then prescribe antibiotics (figure 2; online supplemental table S12).
Overprescription of antibiotics. Dots represent country-specific means, calculated using country-specific sampling weights, vertical bars indicate mean performance across countries and boxes delineate the interquartile range. Data are from clinical vignettes from Kenya (2018), Madagascar (2016), Mozambique (2014), Nigeria (2013), Niger (2017), Senegal (2010), Sierra Leone (2018) Tanzania (2016), Togo (2014) and Uganda (2013). See online supplemental appendix table S11 for the values in this figure.
Provider absenteeism
For quality care to be provided, providers must have sufficient knowledge to diagnose and treat patients. But providers must also be available. The rate of provider absence is, on average, 30% (95% CI 29% to 32%), with large variations across countries (see tables 3 and 4). For example, doctors in Togo and nurses in Uganda have absence rates of 50% (95% CI 26% to 74%) and 47% (95% CI 43% to 50%), respectively, while the average absence rate of doctors in Tanzania is 16% (95% CI 12% to 21%).
Care readiness: knowing how to diagnose and treat, health worker attendance and availability of drugs
Absence rate by country
Availability of drugs and medical equipment
The availability of drugs for treatment of diarrhoea (oral rehydration salts, ORS), postpartum haemorrhage (oxytocin) and pneumonia (amoxycillin or cotrimoxazole) was collected for 9 out of 10 countries and drugs for malaria (ACT) for eight countries. Data on a minimum set of medical equipment (thermometer, stethoscope, sphygmomanometer) were collected for nine countries. On average, 42% (95% CI 40% to 45%) of the facilities were stocked with all four (or all three for Kenya) types of drugs and 70% (95% CI 69% to 73%) of the facilities had a minimum set of functioning medical equipment (see online supplemental table S13). ORS is available in 84% (95% CI 82% to 87%) of the facilities, ACT in 77% (95% CI 75% to 80%) of the facilities, antibiotics for pneumonia in 69% (95% CI 66% to 72%) of the facilities and oxytocin in 62% (95% CI 60% to 65%) of the facilities (see table 3). There is substantial variation in drug availability across countries (see online supplemental table S13 and figure S14).
Overall care readiness
Table 5 reports the probability of care readiness that meets minimum quality standards for five individual conditions and three sets of conditions. To assess how binding to care readiness each of the underlying factors that define the minimum quality standards we use here is, we report three estimates for each condition. Column 1 reports the probability of care readiness which meets minimum quality standards in terms of availability of essential drugs and the probability that providers are present in the facility and knowledgeable to diagnose and treat the condition as measured by the survey. Column 2 of table 5 reports the probability of care readiness if we assume all facilities have available the essential drugs for the condition(s) in question, but we observe provider availability as it is in the facilities. Column 3 of table 5 reports the probability of care readiness if we use the availability of drugs as it is but assume that all providers are present.
Probability of care that meets minimum quality standards (%)
Table 3 reports summary statistics on the three components used to derive the probability of care readiness, namely (1) whether there is at least one provider working in the clinic that can correctly diagnose and prescribe treatment for the condition(s); (2) health worker attendance and (3) whether or not the facility has a minimum set of drugs to treat the condition(s). The raw data, by country, on vignette performance, drug availability and provider presence are reported in tables 1, 2 and 4, and online supplemental appendix S13.
Five results from table 5 stand out. First, on average, only 14% (95% CI 12% to 15%) of facilities are ready to provide selected child, neonatal and maternity care that meets minimum quality standards, with (1) at least one provider available and able to correctly diagnose and treat diarrhoea, pneumonia, postpartum haemorrhage and neonatal asphyxia, and (2) minimum drugs as required (column 1, table 5). Second, the share of facilities with care readiness that meets minimum quality standards increases to 19% (95% CI 18% to 21%) when all facilities are assumed to have the essential drugs in stock (column 2, table 5). That is, given the (low) level of clinical knowledge, even if all facilities were fully stocked with key essential drugs, the share of facilities ready to provide care that meets minimum quality standards would increase by 5% points. Third, addressing absenteeism; that is, assuming all staff are present during working hours, increases the share of facilities ready to provide care that meets minimum quality standards only by a small margin (less than 3% points) on average (column 3, table 5). Ultimately, without provider knowledge to diagnose and treat patients, improving either the supply of medicines or the attendance of health providers will accomplish little. Fourth, there is substantial variation in the probability of care readiness that meets minimum quality standards across conditions, from 62% (95% CI 60% to 65%) for malaria to 34% (95% CI 31% to 36%) for postpartum haemorrhage (column 1, table 5). Fifth, the average outcomes reported in table 5 mask large variations both within and across countries (see online supplemental table S15). In figure 3 and online supplemental figure S16, we plot the relationships between the probability of care readiness under the assumption of full availability of equipment and drugs vs the probability of care readiness under the conditions we actually observe, and then between the probability of care readiness under the assumption of little absenteeism vs actually observed conditions in order to investigate some of this heterogeneity. For Sierra Leone and Tanzania, the share of facilities meeting minimum quality standards for child, neonatal and maternity care would increase from 26% to 41% and from 25% to 39%, respectively, if all facilities were stocked with essential drugs. On the other hand, no health workers absenteeism would have a smaller impact in both countries (increasing care readiness from 26% to 31% in Sierra Leone and from 25% to 27% in Tanzania). In Kenya, the share of facilities with care readiness meeting minimum quality standards is estimated to increase from 45% to 60% with no worker absenteeism, while ensuring 100% drugs availability will only increase care readiness in Kenya from 45% to 51%.
Comparing estimates of minimum quality care (A) Plots the relationship between the estimated probability of care readiness that meets minimum quality standards for child, neonatal, and maternity care, verus the estimated probability of care readiness that meets minimum quality standards for child, neonatal, and maternity care assuming essential drugs treating the conditions are available. (B) Plots the relationship between the estimated probability of care readiness that meets minimum quality standards for child, neonatal, and maternity care, versus the estimated probability of care readiness that meets minimum quality standards for child, neonatal, and maternity care assuming no absenteeism. Child, neonatal and maternity care includes diarrhoea, pneumonia, postpartum haemorrhage and neonatal asphyxia. Essential drugs are oral rehydration salts for diarrhoea, antibiotics for pneumonia and oxytocin for postpartum haemorrhage. Data are from clinical vignettes, unannounced visits and visual inspections from Kenya (2018), Madagascar (2016), Mozambique (2014), Nigeria (2013), Niger (2017), Sierra Leone (2018), Tanzania (2016), Togo (2014) and Uganda (2013). ISO 3-digit alphabetic codes are: KEN (Kenya), MDG (Madagascar), MOZ (Mozambique), NER (Niger), NGA (Nigeria), SLE (Sierra Leone), TZA (Tanzania), TGO (Togo), UGA (Uganda).
In much of health policy, there is an underlying assumption that staffing facilities with ‘qualified’ medical providers (and a basic set of medical equipment) is associated with quality care, hence the use of indicators such as number of doctors per 10 000 population. Figure 4 reports the share of providers accurately diagnosing and treating the two main child killer conditions (diarrhoea and pneumonia) by different cadres in all 10 countries. While doctors are more knowledgeable than nurses, a high share of doctors do not possess sufficient knowledge to diagnose and treat the main child killer condition. On average, 36% (95% CI 26% to 46%) of doctors and 15% (95% CI 11% to 20%) of nurses can accurately diagnose and treat both conditions, while 22% (95% CI 12% to 31%) of the doctors and 34% (95% CI 27% to 41%) of the nurses fail to accurately diagnose and treat any of the conditions.
Diagnostic and treatment accuracy across cadres. Share of providers among doctors, nurses, and community health workers correctly diagnosing the two main killer diseases for children (diarrhoea, pneumonia). The estimates are (unweighted) mean outcomes across countries, with the country means calculated using country-specific sampling weights. The error bars represent the 95% CI. Data are from clinical vignettes from Kenya (2018), Madagascar (2016), Mozambique (2014), Nigeria (2013), Niger (2017), Sierra Leone (2018) Tanzania (2016), Togo (2014) and Uganda (2013).
Online supplemental figure S17 illustrates how providers’ ability to diagnose and treat the two main child killer conditions relates to inputs (medical equipment (online supplemental figure S17A) and drugs (online supplemental figure S17 B,C)). Thirty-four per cent (95% CI 32% to 37%) of the facilities with functioning equipment and 29% (95% CI 24% to 35%) of the facilities without such equipment, have at least one provider employed which has the knowledge to diagnose and treat both conditions. Thus, in the majority of facilities, independent of access to functioning equipment, there is no provider who can diagnose and treat both conditions.
Fifty per cent (95% CI 44% to 57%) of the facilities with antibiotics in stock have at least one provider employed who has the knowledge to diagnose and treat pneumonia, while in 51% (95% CI 44% to 58%) of the facilities with ORS in stock, there is at least one provider that has the knowledge to diagnose and treat diarrhoea. About half of the facilities have at least one provider employed which has the knowledge to diagnose and treat pneumonia, and about one in four of the facilities have at least one provider employed which has the knowledge to diagnose and treat diarrhoea, but they do not have the drugs in stock to treat them (antibiotics and ORS, respectively).
Discussion
This analysis demonstrates the value of measuring clinical knowledge, provider availability and other structural dimensions of quality in order to provide a diagnostic measure of key challenges, most clearly (and perhaps most damagingly) manifested at the facility level, in a health system. The evidence, and more generally the SDI instrument, provides a useful way for governments to benchmark health systems performance on such measures. The SDI is complementary to other survey instruments, such as the Service Availability and Readiness Assessment and the Service Provision Assessment. The distinguishing characteristics of SDI are the clinical vignettes to assess providers’ clinical knowledge and the use of unannounced visits to quantify absenteeism.
This study uses the lowest common denominator across the country-specific protocols to define comparable care readiness criteria across countries. Moreover, there is the well-known gap between what health workers can do and what they actually do (the ‘can-do’ gap).26 For these reasons, we expect that in most surveyed countries the probability of providing minimum quality care according to country-specific guidelines is even lower than that reported in this study. Other studies in the region suggest that the same low performance we observe in vignettes also appear—often to a greater extent—in tests of what providers actually do, with anonymous standardised patients. In Kenya and South Africa, anonymous standardised patients presented with symptoms of tuberculosis: few received an appropriate diagnostic test and most received medication that was either unnecessary or inappropriate.27 In India, 21% of health workers offered potentially harmful diarrhoea treatments in vignettes and 72% offered them to standardised patients.28 Thus, even when diagnostic accuracy is high (as for malaria and tuberculosis in our study), treatment may be inaccurate.27
While we focus on country averages in this study, care readiness may vary dramatically both across regions and within countries. For example, other research on quality of care shows that health workers in South Africa were much less likely to prescribe unnecessary medications for tuberculosis than in India, and in India, correct diagnosis and treatment rates were higher in urban than in rural areas.27 29 Ultimately, policy action by governments should hinge on appropriate national and subnational analysis.
Not all misdiagnoses and mistreatments impose equal costs on patients and on healthcare systems. Some mistreatments—for example, prescription of antibiotics for viral diarrhoea—impose longer-term costs on the patient population with greater resistance to antibiotics but limited short-term adverse impacts to the patient. In this analysis, we focus on the country-specific protocols, but one could imagine an analysis that weighs the direct and indirect cost to patients of different errors in diagnosis and treatment.
Furthermore, there are times that deviations from diagnostic and treatment protocols may be guided by information rather than ignorance. In the case of prescribing antibiotics for diarrhoea, recent evidence suggests that antibiotics promote growth among young children, so medical providers could be incorporating that information into their treatment.30 That said, while a medical provider may overprescribe antibiotics for that reason in practice, there is less reason to expect that she would do so in a vignette as a response to a specific diarrhoeal condition. In contexts where providers seem to have limited skills, ensuring that guidelines adhere to the best and latest knowledge and practice, and encouraging providers to follow those guidelines, are likely to enhance the quality of care.
The complementarity between different aspects of care is an important dimension to consider when improving the quality of care. For example, despite the fact that the study finds significant levels of provider absenteeism and gaps in medications and equipment, this study suggests that not all of these dimensions merit the same priority. Without provider knowledge of symptoms and treatments, no amount of effort or medication will save a patient’s life. Once health providers have sufficient knowledge and norms or incentives are in place for them to be present in the facility, equipment and medicines become more essential. Ultimately, a healthcare system that provides quality care will have to meet all of these needs—knowledge, effort, equipment and essential medicines.
A critical question arising from this work is what actions a health system can take to improve on the effectiveness of its workforce. This research demonstrates that health worker knowledge is a major constraint. Health systems must upgrade the skills of their current and future workers, from doctors to community health workers. The Lancet Commission recommended adopting a competency-based clinical education based on ‘active learning, early clinical exposure and problem-solving learning.’ The evidence on the cost-effectiveness of different modalities through which this can be done remains weak. For example, a major training programme—implemented in 76 LMICs—had limited impact on appropriate treatment for basic child conditions.31 The combination of training with supervision or training with group problem solving delivered more promising results.32 With the increasing penetration of digital technologies, more systems are experimenting with innovations such as mobile devices for training, supervision and mentoring, easy access to decision tools to improve diagnosis and treatment, and the collection of more timely and relevant data. But the use of these technologies in LMICs is still in its infancy and evidence on its effectiveness and costs remains limited.33 Additional interventions to improve quality of care are more of regulatory and policy nature, such as licensing of providers, accreditation of facilities and oversight of training institutions.
After a decade of evidence from SDI surveys, we identify a set of potential developments that could further strengthen its use and impact in improving quality of care, such as faster and cheaper ways to collect data, attention to a broader set of medical conditions and increased effort to understand drivers of performance differences. As countries increasingly track availability of inputs through health management information systems, SDI could progressively focus more on process indicators of quality. This is in line with the recommendations by the recently published findings from the Lancet Global Health Commission on High Quality Health Systems which identified competent care as one of the key dimensions to measure to advance the quality agenda.
Acknowledgments
We are grateful to the many researchers, survey experts and enumerators who have workedon the country surveys that made this cross-country paper possible and are especially grateful for the contributions the African Economic Research Consortium. We are grateful to the World Bank, and in particular the Service Delivery Indicators Trust Fund (funded in large part by the Hewlett Foundation) for supporting the research. JS also thanks Handelsbankens forskningsstiftelse for financial support. The authors would like to thank Amina Mendez Acosta and Evelina Linnros for assistance in this analysis and are grateful for helpful comments from Tessa Bold, Ruben Conner, and Amanda Glassman.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Seye Abimbola
Twitter @DaveEvansPhD
Contributors JS and WW led the team developing the design of the Service Delivery Indicators instrument. WW supervised data collection in the 10 countries. All authors contributed in drafting and revising the manuscript and approved the final version for submission.
Funding The World Bank, Service Delivery Indicators Trust Fund (funded in large part by the Hewlett Foundation).
Disclaimer The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organisations, or those of the Executive Directors of the World Bank or the governments they represent.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement The Service Delivery Indicators data used in this paper are available at https://worldbank.github.io/SDI-Health or upon request from the Service Delivery Indicators team at sdi@worldbank.org.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.