Introduction

India accounts for a quarter of the estimated 10.4 million new tuberculosis (TB) cases worldwide annually, nearly a third of the 1.7 million annual TB deaths, and a third of the estimated 4 million “missing patients” who are either not diagnosed or are not reported to national TB programs [1]. Identifying these missing patients with TB, accurately diagnosing patients in a timely manner, and providing all patients with quality treatment is critical for reducing TB incidence and mortality rates [2].

In India, studies have demonstrated that the private health sector provides the bulk of primary care, is the first point of contact for 50% to 70% of patients with TB symptoms [3–5], and prescribes nearly twice the amount of anti-TB drugs compared to the public sector [6]. However, navigating the private health sector can be medically and economically costly: patients with TB who seek care experience a median of 33.5 days of diagnostic and treatment delays, convoluted pathways with multiple visits to an average of 3 providers before diagnosis, and a broken cascade of care even if diagnosed [4,5,7,8]. With the emergence of drug-resistant strains, these deficits may perpetuate disease transmission and hinder control efforts, particularly for high-density urban areas [9].

The Government of India now recognizes that engaging the private sector is critical to TB control, and it has included this as an explicit goal in the government’s National Strategic Plan (NSP) for TB elimination (2017–2025). The NSP articulates a commitment to massively expand private provider engagement and calls for a 6-fold increase in TB case notifications from the private sector, to 2 million patients per year by 2020 [10].

One key challenge for any effort that engages the private sector is the wide array of healthcare providers, most of whom are not engaged with India’s Revised National TB Control Programme (RNTCP) [11]. The sector comprises providers trained in biomedicine with Bachelor of Medicine, Bachelor of Surgery (MBBS) or higher degrees (such as MDs and specialized MDs), practitioners of alternative medicine and healthcare (such as those who hold degrees in Ayurveda, Yoga, Unani, Siddha, or Homeopathy [abbreviated AYUSH]), informal providers with minimal or no qualifications, and pharmacists and laboratories.

All private providers with MBBS or higher qualifications can provide TB care, and AYUSH practitioners, other non-MBBS providers, and pharmacists generally do not treat TB [12–14]. TB medications are included in Schedule H1, a restricted drug list, in India and therefore can only be dispensed on the basis of prescriptions from MBBS or higher providers. Additional evidence demonstrates that pharmacists do not dispense TB medications over the counter [13,14].

In the private sector, there is no uniform diagnostic algorithm that is practiced, and research demonstrates that providers use diverse approaches (e.g., clinical diagnosis only and clinical diagnosis with chest X-ray [CXR] confirmation, with infrequent use of microbiological tests) [12,15]. Xpert Mycobacterium tuberculosis/Rifampicin (Xpert MTB/RIF), also known as GeneXpert (Cepheid, Sunnyvale, CA), a molecular diagnostic test that detects TB as well as rifampicin-resistant TB, is available in certain private laboratories, and anecdotal evidence suggests that providers use these diagnostics selectively based on patients’ ability to pay and patient risk factors, such as suspected drug resistance. Effective private-sector engagement therefore demands a deep understanding of how private providers behave and what predicts their practice patterns [11].

Between 2014 and 2016, the Bill & Melinda Gates Foundation India Country Office, along with city and state governmental partners, funded 2 Private Provider Interface Agency (PPIA) pilot programs in Mumbai and Patna. The PPIA pilots were private-sector engagement efforts to increase case notifications and improve TB management. Our study was a part of this larger effort to improve quality of care in both cities. Using the baseline data from the quality of care surveillance, the objectives of this study are to answer the following 2 research questions: (1) what are city-representative levels of quality of TB care in urban India across private provider qualifications and cities and (2) to what extent can variation in quality of care be explained?

In this study, we build on our extensive prior work that validated the use of the standardized (or simulated) patient (SP) method for assessing TB care quality and for directly measuring levels of quality of TB care in India, China, and Kenya [12,14,16–18]. The SP method, which is considered the gold standard method to assess provider practice, has been increasingly used to capture levels of quality of care for TB and other health conditions [12,16–20]. In order to provide city-representative measures on what explains levels of quality, we report data on 2,602 SP–provider interactions across 1,203 representatively sampled health facilities in Mumbai and Patna. We additionally report results from 2 micro-experiments aimed to explain variation in our main outcome of interest: correct case management during a single encounter with a provider.

Methods

Study design

The SP survey methodology consists of 3 steps: creating a sampling frame, measuring quality, and analyzing the resulting data. We briefly describe each step here, with additional details in Supporting information (S1–S3 Text).

Provider and facility sampling.

In both Mumbai and Patna, field officers hired by the PPIAs conducted street-by-street mapping exercises in 2014 to construct a comprehensive, “universe” list of all providers and facilities across the private health sectors of both cities. Providers eligible for the SP study were restricted to those known to see adult outpatients with respiratory symptoms in the private health sector. These include most primary care providers but exclude, for instance, orthopedists, gynecologists, ophthalmologists, and pediatricians. Using the universe list, we then representatively sampled eligible providers from purposively sampled geographical areas within each city, which were identified with implementation partners for the SP study in the context of the PPIA pilot programs. Providers were selected with random sampling stratified by PPIA program enrollment status and provider qualification (Table 1). This was done to estimate baseline measures for quality of TB care in these cities and, in ongoing work, evaluate the impact of the programs (see S1 Text) (we note that a stratified analysis by PPIA versus non-PPIA is not a focus of this paper but a focus of the larger impact evaluation study that is ongoing). In Mumbai, we sampled 331 MBBS facilities and 500 non-MBBS providers, and in Patna, we sampled 471 MBBS and 120 non-MBBS providers (S7 Fig). To estimate baseline quality measures, the data were weighted according to appropriate sampling proportions in both cities (S2 Text).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Sampling and weighting descriptions.

https://doi.org/10.1371/journal.pmed.1002653.t001

SP cases were then assigned to providers to ensure that (a) providers of all types would receive a mix of multiple SP case scenarios and (b) the risk of detection would be minimized (i.e., in order to minimize the likelihood of providers being suspicious of SPs or detecting SPs as not real patients, the schedule for SP visits were conducted such that providers would not receive more than 2 SP cases within 5 days of each other and that Case 1 would visit before Cases 2–4 in order to prevent any potential priming effect from a provider seeing a more advanced or obvious TB case before a less advanced one).

Interaction completion rates were uniformly high, with only Patna MBBS, non-PPIA providers having less than 85% of initially scheduled providers successfully visited (this proportion was 71%, due to many being discovered as specialized practitioners who did not treat respiratory conditions). When interactions could not be completed at an initially scheduled provider, they were completed at an identically sampled replacement if possible (S1 Text); therefore, the total number of unique providers in the data is greater than the number initially sampled. Each provider in our sample received up to 4 SP visits at a given location.

Measuring quality using SPs.

The advantages of SP-based measures of medical care quality in general [16,25] and for TB specifically have been discussed previously [12,14]. For this study, 4 distinct SP cases were developed and agreed upon by a Technical Advisory Group (TAG) made up of clinicians, economists, anthropologists, experts in international and national TB guidelines, and other stakeholders (S1 Text), after which the cases were validated in a pilot study [12]. Our main outcome, correct management, was benchmarked against the Standards for TB Care in India (STCI) and the International Standards for TB Care (ISTC) and was agreed upon by the TAG [26,27] (Table 2). Each case was developed with a standardized opening statement and scripted presentation that would advance the provider towards a TB diagnosis and an appropriate case management action, which could include referral, laboratory testing, or treatment initiation, depending on the case scenario.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. SP case descriptions, patient presentations, and correct management definitions.

https://doi.org/10.1371/journal.pmed.1002653.t002

Cases 1, 2, and 3 represented adult pulmonary TB at various stages of diagnostic certitude. Case 1 presents as a classic case of pulmonary TB with 2 to 3 weeks of cough and fever. Case 2 is similar to Case 1; however, the SP additionally has completed a 1-week course of broad-spectrum antibiotics without any improvement and carries an abnormal CXR dated within 2 weeks of the interaction. Case 3 has visited the local government hospital and carries the results of a sputum smear microscopy acid-fast bacillus (AFB) test, which is positive for active TB.

Similar to previously published SP studies, the SPs in this study were recruited from the local community, hired, and extensively trained to present the same case to multiple providers. Interaction details were recorded by field supervisors in a structured exit questionnaire within 1 to 2 hours after each visit. For this study, a total of 24 SPs (7 females and 17 males) were recruited, trained, and hired as staff (see S1 Text for a further description).

In all scenarios in which the SP carried medical reports or images, the SPs conveyed to the provider that they did not know or understand what the reports showed—thereby varying the information available to the provider without altering the patient’s revealed beliefs or expectations. To assess the extent to which provider behavior is consistent across patients with identical presentations, 109 of the Case 1 interactions in Patna were completed at providers who had already completed a Case 1 interaction with a different SP actor.

Case 4 presents as an adult multidrug resistant (MDR) TB suspect with 4 weeks of cough and fever. The SP recalls receiving treatment from the government hospital in the past year for a similar condition and, if questioned, admits to not completing the TB treatment during the previous episode. All flags point towards TB recurrence, which should raise concern about drug resistance. Among 50 of the sampled Mumbai MBBS or higher providers, we randomly assigned SPs to present an experimental variant of Case 4, who carried the same TB-positive sputum AFB report as in Case 3. With this variant, we aimed to analyze the effect of a TB-positive diagnostic test signal by comparing quality of care provided to SPs portraying Case 4 with versus without the AFB report.

To facilitate outcome assessment for each interaction, labeled medicines and prescriptions given to the SPs were coded into 4 categories: anti-TB drugs, fluoroquinolone (FQ) antibiotics, other broad-spectrum antibiotics, or steroids. Coding was done independently by 2 doctors with expertise in TB (SS) and infectious diseases (RS). We chose to retain FQs as a distinct category of antibiotics because they can mask the presence of TB, rendering diagnosis more difficult, and can lead to delays in diagnosis [28]. Additionally, in Mumbai, providers often dispensed loose, unlabeled pills to the SPs. To properly identify these pills, we employed 2 pharmacists who worked independently to code them. On the basis of their assessments, we determined whether the variety of medicines given for each interaction included at least 1 broad-spectrum antibiotic and/or at least 1 steroid.

In addition to reporting the details of case management, each interaction was broadly classified as “correctly managed” or not, according to the STCI [29]. Taking a lenient approach, providers were not penalized for the use of unnecessary or even potentially harmful medicines, and thus the results presented are upper-bound estimates of quality, as measured by adherence to TB standards of care. The definitions of correct management are detailed in Table 2, and the levels of prescribing and dispensing medications are analyzed in the Appendix (S3 Text). S1 Fig elaborates on the “lenient” approach, illustrating the proportion of our “correct management” providers who would have been considered not correct if we had penalized other medications.

Analysis.

We report raw proportions for outcomes of interest, with population mean estimates and CIs computed using inverse probability weighting. Based on the universe of private-sector providers listed from the mapping exercise in both cities (S1 and S2 Text), these weights are calculated such that each of the 8 city–case combinations contribute equally to overall estimates. Within each city–case combination, individual interactions are weighted based on the actual proportion of providers enrolled and not enrolled in the PPIA in that city’s provider sampling list within both the MBBS and non-MBBS strata versus the realized sample.

Therefore, the percentages reported for case management behaviors represent the estimated likelihood of the outcome occurring if a provider were chosen at random from the citywide population of providers rather than the percentage of interactions in our sample in which the behavior was observed. In addition to using these weights to estimate population likelihoods, we use them to calculate weighted odds ratios (ORs) in logistic regressions comparing variation in quality of care across provider types, city settings, and SP case scenarios.

In Supporting information (S3 Text), we present an ANOVA analysis, which was not prespecified, to determine how well our primary stratification characteristics explain variations in SP management. Among the sampled providers who received repeat Case 1 visits in Patna, we assess the level of consistency that those providers displayed across identical SP Case 1 visits using a different SP actor. We then use a quality proxy—checklist of history questions—to illustrate the amount of variation in provider behavior within each city–case combination.

In addition to the ANOVA analysis, we include the outcome of whether an SP was asked to return, which was not prespecified and requested during peer review. Considering that the data for this analysis are a subset of the data generated from a larger research project, we did not encounter any other deviations from the stated prospective analysis plan within the research protocol submitted to the ethics committees (S4 Text). All analyses and programs were written in Stata 14 (Stata, College Station, TX).

Results

The results are presented in 3 sections. In the first section, we describe overall standards of care, focusing on correct case management, medicine use, and laboratory tests. In the second section, we document variation in the data by provider qualification and city. In the third section, we document variation across SP cases, focusing on the role of diagnostic certainty.

Practice quality overview

In our study, 1,288 distinct provider practices were successfully visited by SPs at 1,203 health facilities across both cities. Among these providers, the majority were male (88%), had a clinic assistant (65%), and fell into the age category of 30–50 years (71%), which were all characteristics observed by SPs during the interactions. Fig 1 illustrates our main city-level estimates of average case management outcomes among the city-representative sample, with proportions estimated with weights to represent the estimated likelihood of the outcome occurring if a provider was visited by a patient at random from the citywide population of providers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. City-representative quality of care estimates.

City-level estimates of quality of care for each of our case scenarios. These proportions represent the estimated frequency with which the action would be observed if the standardized case scenario was presented to a provider randomly selected from the sampling frame. These estimates are calculated using inverse probability weights corresponding to the sample frame as detailed in S2 Text for every city–qualification–PPIA–case combination in the data. N = 2,602. AFB, acid-fast bacilli; PPIA, Private Provider Interface Agency; TB, tuberculosis; Xpert MTB/RIF, Xpert Mycobacterium tuberculosis/Rifampicin, also known as GeneXpert.

https://doi.org/10.1371/journal.pmed.1002653.g001

Among 2,602 SP interactions, 959 were correctly managed (35%; 95% CI 32%–37%), and 536 of 2,602 interactions (29%; 95% CI 26%–31%) had any diagnosis given to the patient. Among the 959 correctly managed interactions, 260 (37%; 95% CI 32%–42%) received any diagnosis. Among the correctly managed interactions, a greater proportion of providers ordered a CXR (677 of 959 [53%; 95% CI 48%–57%]) or referred the SP for further care (194 of 959 [36%; 95% CI 31%–40%]) than ordered a microbiological test for diagnosis (318 of 959 [31%; 95% CI 27%–36%]). Among the 194 referrals, 52% were to the private and 48% to the public sector.

Microbiological testing, including drug susceptibility testing (DST), was relatively infrequent across all case scenarios. Sputum smear testing was ordered in 389 of 2,602 (18%; 95% CI 16%–20%), sputum culture in 28 of 2,602 (2%; 95% CI 1%–3%), and Xpert MTB/RIF in 108 of 2,602 interactions (2%; 95% CI 1%–3%). For Case 2, when the SP carried an abnormal CXR suggestive of TB, 45 of 385 SP interactions (7%; 95% CI 4%–10%) resulted in any DST, while 99 of 385 were ordered a new CXR (25%; 95% CI 20%–30%). Similarly, of 486 interactions in which the SP presented with recurrent TB and suspected MDR-TB (Case 4), 26 were recommended any DST (3%; 95% CI 1%–5%).

Medicines were very frequently prescribed or dispensed, and we did not count the use of additional or unnecessary medications against the provision of correct management. At least 1 medication was prescribed or dispensed in 2,239 of 2,602 interactions (79%; 95% CI 76%–82%), with an average of 3.11 medications per interaction (95% CI 2.99%–3.23%). Across all 2,602 interactions, broad-spectrum antibiotics other than FQs were given in 1,227 (39%; 95% CI 36%–42%), FQ antibiotics in 328 (15%; 95% CI 13%–17%), and steroids in 164 interactions (6%; 95% CI 5%–7%).

The use of anti-TB medications was minimal and was mostly limited to MBBS-qualified providers. Anti-TB medications were prescribed in 118 of 2,602 interactions (5%; 95% CI 4%–6%), with MBBS or higher providers accounting for 112 of those 118 instances. Among the 118 TB prescriptions, 113 included correct isoniazid, rifampicin, pyrazinamide and ethambutol (HRZE) prescriptions—1 of these additionally included the second-line drug clofazimine, and 2 also included streptomycin. Among the 5 instances that did not include HRZE, 4 were HRE, and 1 was the second-line medication cycloserine prescribed alone.

In contrast with correctly managed interactions, interactions in which correct case management was not observed did not demonstrate any kind of consistent “alternate” protocol that could theoretically be justified in a polluted urban environment that can result in non-specific respiratory symptoms like cough. Given a patient with a 2- or 3-week cough in a polluted city like Patna, providers could, for example, be observed to adopt a “wait-and-see” approach by offering palliative symptomatic care and asking the patient to return after a few days, even though this practice technically falls short of the international and national standards of TB care we used to benchmark management in this study.

We do not observe this type of concentration on a single alternative protocol in our data. For example, of the 834 Case 1 interactions that were not correctly managed (Fig 2), 183 (22%) received only a non-FQ broad-spectrum antibiotic; 294 (35%) received a non-FQ broad-spectrum antibiotic as well as an FQ, steroid, and/or cough syrup; 185 (22%) received 1 or more of those medications without a non-FQ broad-spectrum antibiotic; and 172 (21%) received something else entirely (or nothing at all).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Management of Case 1 when no correct treatment was given.

Frequency in which Case 1 was managed with possible combinations of steroids, cough syrups, broad-spectrum antibiotics, and FQs, when no correct management was given. There were N = 834 Case 1 interactions that did not meet the criteria for correct management, and 172 interactions resulted in none of these case management behaviors. FQ, fluoroquinolone.

https://doi.org/10.1371/journal.pmed.1002653.g002

Variation by qualification and setting

One potential explanation is that the wide variation in management reflects systematic practice variation by qualification and/or setting, and we turn to this hypothesis next. Fig 3 reports ORs for differences in quality of care outcomes by qualification (top panel) and by city stratified by qualification (middle and bottom panels). MBBS providers, who make up 58% of all providers in the Patna sampling list and 50% of all providers in the Mumbai list, were more likely than non-MBBS providers to correctly manage cases (weighted OR 2.80; 95% CI 2.05–3.82; p < 0.0001), ask for CXR and/or sputum tests, and initiate anti-TB treatment. Despite providing relatively higher-quality care, MBBS providers only correctly managed 709 of 1,304 interactions (54% of interactions; 95% CI 52%–57%). MBBS providers were also more likely than others to prescribe unnecessary or harmful antibiotics, including FQs, although their use of steroids was notably lower.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Quality of care differences by provider qualification and location.

Estimated ORs between various groups of providers, for the frequency in which the indicated management action is observed across all case scenarios. Panel A reports differences by MBBS qualification level, pooled across all observations. This regression includes controls for city setting and case scenario (N = 2,602). Panels B and C report similar ORs estimated across cities, stratified by MBBS qualification (N = 1,448 and 1,154, respectively). These regressions include controls for case scenario. AFB, acid-fast bacilli; MBBS, Bachelor of Medicine, Bachelor of Surgery; OR, odds ratio; TB, tuberculosis; Xpert MTB/RIF, Xpert Mycobacterium tuberculosis/Rifampicin, also known as GeneXpert.

https://doi.org/10.1371/journal.pmed.1002653.g003

Additional analysis in S6 Fig shows that the differences between AYUSH and other non-MBBS providers in Patna, where both practice, were surprisingly small. In fact, AYUSH providers were less likely than others to manage or refer cases according to STCI, although the others gave more unnecessary antibiotics. Use of nonallopathic medications was also diverse. For example, although most non-MBBS providers in our Mumbai sample are AYUSH, only 63 of 996 non-MBBS SP interactions in Mumbai (6%; 95% CI 4%–8%) involved explicitly labeled homeopathic or Ayurvedic medicines. By contrast, non-MBBS providers in Patna used labeled Ayurvedic or homeopathic medicines in 113 of 302 interactions (41%; 95% CI 33%–49%).

Differences in case management were smaller across the 2 cities. Across all cases, Mumbai non-MBBS providers were more likely than Patna non-MBBS providers to correctly manage cases (weighted OR 1.79; 95% CI 1.06–3.03; p = 0.0293), primarily by ordering CXR (weighted OR 3.62; 95% CI 1.85–7.10; p = 0.0002). Most other behaviors were not significantly different across the cities. MBBS-qualified providers were even more similar across cities in terms of correct management (weighted OR 1.15; 95% CI 0.79–1.68; p = 0.4642), with the notable exception that Mumbai MBBS providers utilized Xpert MTB/RIF testing much more frequently than MBBS providers in Patna (weighted OR 8.60; 95% CI 2.83–26.10; p = 0.0001). Both MBBS and non-MBBS providers in Mumbai were less likely to prescribe FQs than Patna providers (weighted OR 0.4; 95% CI 0.24–0.66; p = 0.0004, and 0.25; 95% CI 0.15–0.45; p < 0.0001, respectively), and the dispensing or prescribing of steroids and broad-spectrum antibiotics did not differ systematically across the 2 cities.

ANOVA decomposition (S3 Fig) shows that our primary stratification characteristics of qualification and setting typically predicted less than 25% of the observed variation in each of our primary case management outcomes with the exception of correct case management and ordering a CXR (for which 25%–50% of observed variation was predicted). By contrast, in a subsample in which 109 Patna providers received a repeat Case 1 interaction (S4 Fig), consistency levels between the 2 visits were near 75% for all behaviors. Therefore, we observe that practice is highly variable across providers, but setting and qualification strata can explain only a fraction of the wide variety in management practices.

Using essential-history checklist completion as a proxy for the distribution of individualized quality levels (S5 Fig), our results suggest that there is a substantial and unexplained idiosyncratic component to quality that varies widely even within each city and qualification level. Rather than tight clustering around the group means, in all cases, we observe a “fat tail” of both low- and high-quality MBBS and non-MBBS providers in each city. Thus, moving beyond averages to full distributions of quality yields important and nuanced additional results for the estimation of “average” outcome quality, both by city and by qualification.

When do providers perform better? Variation by SP case

The final source of variation we assess is the SP case presentation itself. In our previous work [14], we found that pharmacists were more likely to correctly manage a case if SPs carried a TB-positive diagnostic test, even as the SP made it clear that she did not know what the results of the test meant. The SP cases presented in this study reflect a similar design, with the varied presentation intended to exogenously adjust the providers’ initial degree of certainty about the patient’s true diagnosis. When the SP carried either an abnormal CXR (Case 2) or an AFB-positive sputum report (Case 3), providers were observed to be more likely to order tests consistent with a suspicion of TB compared to the Case 1 presentation.

In Case 2, in which the SP carried an abnormal CXR, a second CXR (which would ostensibly provide no new information) was ordered in 99 of 385 instances (25%; 95% CI 20%–30%). In Case 3, in which the SP carried a TB-positive sputum AFB report, 208 of 354 were recommended a CXR (56%; 95% CI 49%–63%), and 54 of 354 were ordered a new AFB smear (16%; 95% CI 11%–21%). In terms of inappropriate behaviors, variation across SP case had some impact on the use of inappropriate medications, but it did not reduce it anywhere close to 0 in any case (S1 Fig). Prescribing unnecessary medicines ranged from 284 of 385 instances in Case 2 (72%; 95% CI 67%–77%) to 1,290 of 1,377 instances in Case 1 (93%; 95% CI 91%–95%). Use of broad-spectrum antibiotics ranged from 105 of 385 Case 2 interactions (29%; 95% CI 23%–34%) to 799 of 1,377 Case 1 interactions (53%; 95% CI 50%–57%), and use of FQs ranged from 43 of 385 Case 2 interactions (12%; 95% CI 8%–16%) to 52 of 354 Case 3 interactions (19%; 95% CI 13%–25%).

Fig 4 highlights selected cross-case variation using logistic regressions. The figure reports estimated differences between Case 1 and Case 3, using only providers who received both cases. It also reports differences observed in Case 4 against our alternate version of Case 4 carrying the same TB-positive sputum AFB report at identically randomly sampled MBBS-qualified providers in Mumbai, in which the variation in diagnostic certainty is causally identified through the use of random assignment of the AFB report in an otherwise-identical case presentation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Quality of care differences between SP case scenarios.

Estimated ORs between specific case scenarios for the frequency with which the indicated management action is observed. Panel A reports estimated ORs between Case 1 and Case 3, including only those providers who received both cases (N = 759 interactions). Panel B reports estimated ORs between Mumbai MBBS providers who received the experimental version of Case 4 that carried the same sputum report against a comparable sample who received the ordinary Case 4 presentation (as described in S1 Text; N = 101 interactions). AFB, acid-fast bacillus; MBBS, Bachelor of Medicine, Bachelor of Surgery; OR, odds ratio; SP, standardized patient; TB, tuberculosis; Xpert MTB/RIF, Xpert Mycobacterium tuberculosis/Rifampicin, also known as GeneXpert.

https://doi.org/10.1371/journal.pmed.1002653.g004

Carrying a positive sputum report in Case 3 was associated with an increase in the frequency of CXRs (OR 3.13; 95% CI 1.98–4.96; p < 0.0001) and sputum AFB tests (OR 2.45; 95% CI 1.34–4.47; p = 0.0035), as well as a nonsignificant increase in Xpert MTB/RIF tests (OR 3.88; 95% CI 0.81–18.49; p = 0.0888) and referrals (OR 8.69; 95% CI 3.25–23.2; p < 0.0001) compared to the same providers’ behavior in the Case 1 interactions. In the experimental comparison in which only the diagnostic information was varied for SPs portraying Case 4 while carrying versus not carrying a TB-positive sputum report (S1 Text), we use the fact that the report was randomly assigned as indicative of causal impact. Among the 50 instances in which Case 4 SPs carried a sputum report, the provider saw the report in 45 instances (90%) and asked detailed questions about the past treatment in 38% of interactions. The results are broadly similar although nonsignificant: when the Case 4 SP carried the sputum report, correct management increased from 29% to 48% (OR 2.22; 95% CI 0.98–5.02; p = 0.0569), largely due to greater provider use of appropriate DST. Providers’ use of medication also decreased from 86% to 72%, though this was not significant (OR 0.41; 95% CI 0.15–1.12; p = 0.0824). However, quality improvements were not consistent across all dimensions: the use of first-line anti-TB medicine, which is not considered correct treatment for a suspected drug-resistant case and could contribute to greater drug resistance, increased from 2% to 18% when the report was presented (OR 10.98; 95% CI 1.33–90.24; p = 0.0258).

Discussion

TB is a persistent health challenge for India and is one of the top 5 causes of death between the age of 30 to 69 [31]. With India’s goal of eliminating TB by 2025 as stated in the NSP, the success of this plan heavily depends on whether India’s large, unregulated, and diverse private sector can be effectively engaged to identify missing patients with TB and ensure that all patients with TB receive quality TB care [32].

This study extends the evidence from our pilot study that validated the use of the SP method for assessing TB quality of care in an urban India setting [12] and from our research on TB-management practices of pharmacists assessed with SPs across 4 Indian cities [13,14]. Our validation study assessed quality of TB care from a purposive provider sample, and with this study, we were able to further apply the SP method for TB to analyze representative levels and variation of quality among MBBS and non-MBBS providers in the private health sector of 2 cities. In addition, 2 micro-experiments allow us to better understand the drivers of quality of care in this setting.

Our city-representative study shows significant deficits in the average provider’s management of TB cases in both study cities. This low quality is characterized by underuse of appropriate diagnostics and widespread use of unnecessary medications, including antibiotics and contraindicated FQs. Even though MBBS-qualified providers managed the SP cases better on average, there was still considerable variation within qualification in each setting and relatively little difference between the 2 cities on average. We further observed that TB-specific management increased with diagnostic certainty. We also present suggestive evidence that specific providers adhere consistently to an idiosyncratic protocol when faced with repeated identical cases. Because providers are making the same mistakes consistently, the SP method provides novel information to TB-control programs on the specific actions that need to be improved.

Representing the quality variation during the first year of city-wide private-sector engagement efforts, our data further underscore the need to work with the private sector to improve quality of TB care. These study results, complemented with the 4-city analysis on pharmacist behaviors, provides new multicity, qualification-specific information for TB control in India. It remains to be seen the extent to which private-sector engagement efforts—such as the PPIA pilots—will make an impact on quality and be able to sustain improvements in quality of care. Incorporating quality measures alongside program implementation is the first step, particularly as aspects of the PPIA model are being scaled up in more than 40 Indian cities, supported by the Global Fund.

We emphasize that the low observed proportions of correctly managed interactions do not fit a hypothesis in which providers followed an alternate protocol that reflects “what is good for the patient but not for society” (such as a wait-and-see approach for a patient with persistent cough). Providers treated the SPs in idiosyncratically different ways without a consistent protocol. Neither are the data consistent with the view that very high patient loads are responsible for low quality. Of the weighted 2,602 interactions, 45% had no other patients waiting, 65% had a queue of 1 or fewer, 75% had 2 or fewer, and 95% had 10 or fewer. This is similar to what has been observed in previous SP studies as well as time-and-motion studies in health clinics [33].

We consider, instead, the following 2 broad classes of explanations for this behavior: (a) providers have a difficult time diagnosing TB appropriately, and (b) private providers deviate from established standards for financial gain. There is evidence for both types of behavior, implying that quality deficits are not driven by either knowledge gaps or financial incentives alone. In favor of an explanation driven by poor diagnostic skills, we find that improving certainty about the diagnosis had a positive effect on quality of care (although the results are not statistically significant in the Case 4 comparison because of the small sample size, they are quantitatively as large as those in the Case 1 to Case 3 comparison). However, lack of diagnostic certainty was not the only indicator of poor quality we observed in our study, nor did quality improve across every dimension when the test results were provided to the provider. Increasing diagnostic certitude improved correct case management but had smaller effects on reducing inappropriate medicine use.

This suggests that financial considerations and poor diagnostic ability may both play a role in explaining the patterns we observe and mirror previous findings with pharmacists in 4 urban Indian locations [14,34]. Across settings, we find that these diagnostic practices are highly predictive of whether the provider offers correct management in a given interaction, but we cannot “force” providers to take detailed histories from their patients because a good history and physical is an essential indicator of quality in and of itself.

Our study has several strengths and contributes to the literature in several ways. First, we representatively sampled large numbers of private health providers in 2 Indian cities, and after weighting to the city universe of providers, we provide precise estimates of provider behavior at the city level. Because the analysis is representative of these 2 Indian cities, caution is warranted when generalizing to a context outside of urban India. Second, by using unannounced SPs, we captured actual provider behavior, as opposed to self-reported knowledge or practices. Given our prior work showing a big “know-do gap” (the gap between what providers know and what they do in actual practice) [12], the SP methodology better reflects reality for patients than any other existing method used to measure quality of care. Third, our study included MBBS-qualified providers, as well as AYUSH providers and those with other or no qualifications, capturing the complexity of training within the Indian private healthcare sector. Fourth, by developing 4 different SP case presentations, we studied how providers dealt with various stages of TB disease and varying levels of diagnostic certainty. Lastly, by assessing outcomes by city, provider qualification, and type of case, we assessed the most important sources of variation in quality of care. While previous studies showed suboptimal quality of care, our study was able to explore the role of provider qualifications on quality using 2 city-wide, representative samples.

Our study has limitations. First, because we do not observe how patients actually choose providers, patient sorting by qualification, geography, personal relationships, price, reputation, or other unobserved signals of quality prevents extrapolation to the likely outcomes for actual patients with TB. This remains an important area for future work. Second, our cases are designed as one-time interactions, and the SP data do not reflect follow-up visit pathways, which have been shown by other studies to be long and convoluted and had various forms in our data (S2 Fig). For instance, we cannot say, from this study, what the doctor would do after the patient has returned after completing a CXR as ordered. To the extent that doctor behavior is different when the patient comes with a CXR that the doctor herself recommended (rather than with a CXR ordered by another doctor), our approximations of provider behavior under different scenarios may be erroneous. The ability for the SP method to measure quality of care measures in follow-up visits with the same SP individual in similar settings has not been published to our knowledge, and—given the frequency of providers asking for patients to return exhibited in this study—it could be worthwhile to explore the potential for the method to assess whether the likelihood of receiving better care increases when a patient returns. With this, it would be important to understand the extent to which real patients return upon a provider’s request, as well as the necessary work needed so that SPs are not detected.

While not a limitation per se, we also highlight that our definition of correct case management follows national and international TB standards of care. We have chosen to use these definitions because they allow for comparability across studies and disciplines our analyses of the data using a clearly prespecified protocol. But our data could also raise questions about the viability and validity of these standards. For instance, it is reasonable to ask whether—in cities with air pollution, which can cause respiratory symptoms—healthcare providers should be asked to send a patient with a 3-week cough for a sputum test rather than giving them symptomatic therapy and asking them to follow up as required.

Our data are not consistent with the idea that providers were following a single alternate protocol because virtually every combination of drug classes was used for the SP cases. Nevertheless, there could be justifiable reasons why providers still choose to deviate from established standards of care. One way, then, to interpret our findings and our use of the term “correct case management” is as an ordinal ranking rather than adherence to a standard: providers who follow the standards are of higher quality, but this does not imply that those who do not follow the standards necessarily provide “incorrect” care. Instead of imposing alternate standards, we have chosen to present the full set of management practices for each SP case to allow readers to make their own more nuanced judgments. We have also made the data publicly available so that researchers can simulate correct management under alternate standards, in turn stimulating debate around whether these standards themselves require further revision.

Despite the limitations, our large-scale, 2-city quality of care study provides accurate and representative estimates of provider behavior that may inform not only quality-improvement efforts in health but also interventions to improve TB care and reduce transmission in the community.

Supporting information

Acknowledgments

We thank Puneet Dewan, Sameer Kumta, Shibu Vijayan, Sirisha Papineni, and Nita Jha for useful input on implementation, analysis, and interpretation; Rajan Singh, Purshottam, Chinar Singh, Geeta, Devender, Varun Kumar, Anand Kumar, Babloo, and Charu Nanda, who supervised and implemented the ISERDD field work; all the standardized patients for their dedication and hard work; and Caroline Vadnais for excellent administrative support.

The findings, interpretations, and conclusions expressed here are those of the authors and do not necessarily represent the views of the World Bank, its executive directors, or the governments they represent.