Article Text

Sponsorship bias in published pharmacoeconomic evaluations of national reimbursement negotiation drugs in China: a systematic review
  1. Zixuan He1,2,
  2. Xianqin Huang1,
  3. Dingyi Chen1,
  4. Guoan Wang1,
  5. Yuezhen Zhu1,
  6. Huangqianyu Li1,2,
  7. Sheng Han1,2,
  8. Luwen Shi1,2,
  9. Xiaodong Guan1,2
  1. 1International Research Center for Medicinal Administration, Peking University, Beijing, China
  2. 2Department of Pharmacy Administration and Clinical Pharmacy, School of Pharmaceutical Sciences, Peking University, Beijing, China
  1. Correspondence to Xiaodong Guan; guanxiaodong{at}pku.edu.cn

Abstract

Background China’s National Reimbursement Drug List (NRDL) has become the primary route for drug reimbursement in China. More recently, the authority has made pharmacoeconomic evaluation an integral part of the application for NRDL inclusion. The underlying financial conflict of interests (FCOI) of pharmacoeconomic evaluations, however, has the potential to influence evidence generated and thus subsequent decision-making yet remains poorly understood.

Methods We searched for studies published between January 2012 and January 2022 on the 174 drugs added to the 2017–2020 NRDLs after successful negotiation. We categorised the study’s FCOI status into no funding, industry funding, non-profit funding and multiple fundings based on authors’ disclosure and assessed the reporting quality of included studies using the Consolidated Health Economic Evaluation Reporting Standards 2022 checklist. We compiled descriptive statistics of funding types and study outcomes using t-tests and χ2 tests and conducted multivariate regression analysis.

Results We identified 378 records and our final sample included 92 pharmacoeconomic evaluations, among which 69.6% were conducted with at least one funding source. More than half (57.6%) of the evaluations reached favourable conclusions towards the intervention drug and 12.6% reached a dominant result of the intervention drug over the comparison from model simulation. The reporting quality of included studies ranged from 19 to 25 (on a scale of 28), with an average of 22.3. The statistical tests indicated that industry-funded studies were significantly more likely to conclude that the intervention therapy was economical (p<0.01) and had a significantly higher proportion of resulting target drug economically dominated the comparison drug (p<0.05).

Conclusion The study revealed that FCOI bias is common in published pharmacoeconomic evaluations conducted in Chinese settings and could significantly influence the study’s economical results and conclusions through various mechanisms. Multifaceted efforts are needed to improve transparency, comparability and reporting standardisation.

  • Health economics
  • Health policy

Data availability statement

Data are available in a public, open access repository.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

  • Published studies have discovered significant sponsorship bias in clinical trials, reviews, as well as clinical guidance.

WHAT THIS STUDY ADDS

  • This study examined the impact of financial conflict of interest on economic results, conclusions and reporting quality of pharmacoeconomic evaluations and explored possible mechanisms.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

  • Findings of this research shall serve as a reference for future policy decision-making to promote consistent and evidence-based reimbursement decisions.

Introduction

Pharmacoeconomic evaluations that estimate the value of medicines in treating certain diseases are widely used to inform price negotiation and insurance coverage policies.1 2 These evaluations primarily answer the question of whether the intervention drug is economically advantageous over comparisons.3 4 In July 2020, China’s National Healthcare Security Administration announced that pharmacoeconomic evaluation was made mandatory for drugs intending to participate in national price negotiations to be included in the Chinese National Reimbursement Drug List (NRDL). This announcement established the policy basis for the critical role of pharmacoeconomic evaluations in price negotiations and policy decision-making in drug reimbursement.5 However, published economic evaluations are often written by authors with a financial conflict of interest (FCOI) related to the pharmaceutical industry, government or other non-profit institutes. Previous research has explored clinical trials funded by the industry and found these trials to favour the intervention drug more than those funded by other sources.6–8 Such bias is also observed in other types of research, including systematic reviews and clinical guidance.9 10 Recent studies conducted by Bilcke et al and Xie and Zhou indicated that industry-funded pharmacoeconomic cost-effectiveness analyses tend to choose costs and effectiveness input values that might potentially lead to more favourable economic conclusions.11 12 Therefore, we explored the possible impact of industry sponsorship on China’s national reimbursement decision-making through funding the generation of pharmacoeconomic evidence. Results from this study could help standardise the future generation of pharmacoeconomic evidence and help optimise the rationale of policy decisions regarding healthcare reimbursement. To achieve this, we expanded the search database and included all types of pharmacoeconomic evaluations, compared study results, conclusions and reporting quality separately as the research outcomes, and further explored the mechanism of sponsorship bias.

While China’s national drug price negotiations have continuously improved the accessibility, availability and affordability of medicines among Chinese patients, enhancing equity in drug use.13 14 Huang C et al compared prices and clinical benefits of successfully negotiated anticancer drugs in 2019 with those of the comparators, and pointed out that the negotiated price may be irrational based on pharmacoeconomic evaluations.15 For pharmacoeconomic evaluations, the researchers may have larger control over input parameters, making the research results more susceptible to FCOI than clinical trials. To this end, we conducted this study to investigate the impact of FCOI on the conclusion of pharmacoeconomic evaluations and examine the effect of sponsorship on a study’s health–economic results and reporting quality. Our study further analysed and demonstrated the impacts of the choice of the comparison drug, the simulation model, simulation period, cost and effectiveness input data, willingness to pay (WTP) threshold and discount rate on the results and conclusions of the studies. We provide research implications and policy recommendations based on these comprehensive analyses, aiming to serve as a reference for future policy decisions in national healthcare reimbursement to promote consistent and evidence-based decision-making.

Methods

This systematic review was conducted according to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines.16

Searching strategy

We identified 174 generic names of drugs that were successfully added to China’s 2017–2020 NRDL after price negotiation (online supplemental table 1) and defined these 174 drugs as the study’s target drug. Searches were performed in PubMed (MEDLINE), Embase and the Cochrane Library for studies published between January 2012 and January 2022. Systematic multistring search strategies were developed using a combination of text words and index terms in line with eligibility criteria to retrieve published studies. We employed three search term categories: 174 generic names of target drugs, China and Chinese region determiner and economic evaluation determiner, with searching fields restricted to title, abstract, keywords and Medical Subject Headings term (example shown in online supplemental figure 1).17 The review protocol was registered online with PROSPERO (CRD42022347795).

Supplemental material

Inclusion and exclusion criteria

Studies that met the following criteria were included: (1) Participants: Chinese patients aged≥18; (2) Intervention: 174 target drugs listed in online supplemental table 1; (3) Comparison: No restriction; (4) Study Type: Pharmacoeconomic evaluations. We excluded publications that: (1) did not meet the search criteria, including studies that were conducted in non-Chinese populations, assessed non-local costs, not published in a peer-reviewed journal, not in English or published not within the study time frame; (2) listed the target drug was as the comparison drug, not the intervention drug; (3) were not pharmacoeconomic evaluations or did not report the economic results of the target drug; (4) were a multidrug combination therapy that made the cost and clinical benefit of the target drug undistinguishable; (5) had no FCOI disclosure or full text is not available. Two researchers independently screened the identified articles based on titles and abstracts and then full texts to assess eligibility. Uncertainties or disagreements on the inclusion of specific articles were resolved through discussions among all authors until a consensus was reached.

Terminology and definition

We categorised the study’s FCOI status into no funding, industry funding, non-profit funding and multiple funding based on authors’ disclosure or absence of disclosure. A study was categorised into industry funding if it was conducted based on full financial funding from the pharmaceutical industry or its authors received grants from the industry. A study was categorised into non-profit funding if the research or authors received grants from governmental agencies, academic institutions, research foundations or other non-profit organisations. Studies that received funding from both industry and non-profit sources are categorised as ‘multiple funding sources’. A study was categorised into no funding studies if the authors explicitly declared no conflict of interest.

Outcome measures

Our primary outcome was the type of conclusion achieved in the pharmacoeconomic analysis. We defined conclusions that indicated the treatment drug was more cost-effective than the comparison as ‘favourable conclusion’, conclusions that indicated the treatment drug was unlikely to be cost-effective as ‘unfavourable conclusion’ and conclusions implying that the treatment and comparison drug had similar cost-effectiveness under certain threshold as ‘no difference’.

In addition, we examined two secondary outcomes: the type of study results and study quality. We paid particular attention to whether the study suggested that the treatment drug could increase a patient’s quality of life while saving costs. If the answer to this question was positive, we categorised the result as ‘dominant result’ and otherwise ‘non-dominant result’. The reporting quality of the pharmacoeconomic evaluation was assessed using the latest Consolidated Health Economic Evaluation Reporting Standards (CHEERS 2022.18 We evaluated the study’s reporting quality with scores ranging from 0 (low) to 28 (high) and an increment of 1 point. Reporting quality was assessed by two researchers independently scoring and crosschecking each piece of literature, with any disagreements resolved through discussion or with the assistance of a third researcher.

To further explore the mechanism of how FCOI could possibly affect the researcher’s choice and impact study outcome, we categorised the economic evaluations with the same intervention drug and compared its health–economic result and conclusion along with its funding type. For studies of the same intervention drug reaching contradicting economic conclusions under different types of funding, we categorised them into (1) studies with precisely the same intervention and comparators; (2) studies with the same intervention but chose different comparators.

Statistical methods

We first described the overall scope of all included pharmacoeconomic studies in terms of drug WHO Anatomical Therapeutic Chemical (ATC) classification and funding types. Then we compiled descriptive statistics of included drugs, studies and drug-study pairs and tested differences using t-tests and χ2 tests, as applicable, with a two-sided p<0.05 considered statistically significant. We further conducted multivariate logistic and generalised linear model regression analysis with the control of ATC classification and study publish year to ensure the robustness. All analyses were performed with Stata V.15.

Patient and public involvement

The authors declare no patient or public involvement in this study.

Results

We identified a total of 378 published pharmacoeconomic evaluations through our electronic database searches. After removing duplicates, we screened 226 records and assessed 138 full-text articles based on inclusion and exclusion criteria. Eventually, 92 pharmacoeconomic evaluations were included in the review and underwent data extraction (figure 1), detailed information on included studies can be found in online supplemental table 2.19–110

Figure 1

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram. FCOI, financial conflict of interest.

Descriptive statistics

We present the summary of the studies’ characteristics included in this analysis in table 1. Of the 92 pharmacoeconomic evaluations included in our study, 87 (94.6%) were cost-effectiveness analyses, and 64 (69.6%) were conducted with at least one funding source. Among them, non-profit funding was most frequently cited as the funding source (43/92, 46.8%), followed by industry funding (16/92, 17.4%). By drugs’ ATC type, 61 (66.3%) pharmacoeconomic studies focused on antineoplastic and immunomodulating agents, followed by alimentary tracts and metabolism drugs (18/92, 19.6%). We also noticed a large variation in the distribution of ATC classifications across different funding types (figure 2). Industry funding is more likely to support pharmacoeconomic evaluations of alimentary tracts and metabolism drugs (ATC category ‘A’) and less likely to sponsor evaluations on antineoplastic and immunomodulating agents (ATC category ‘L’).

Table 1

Characteristics of published pharmacoeconomic evaluations of drugs successfully negotiated into China’s 2017–2020 National Reimbursement Drug List

Figure 2

Distribution of the drug Anatomical Therapeutic Chemical classification by funding types.

A total of 53 studies (57.6%, 53/92) reached favourable conclusions towards the intervention drug, and 12.6% (11/87) reached a dominant result of the intervention drug over comparison from model simulation. The reporting quality of included studies ranged from 19 to 25 (on a scale of 28), with an average of 22.3 according to CHEERS 202218 standards. Most research was published after 2016 (77/92, 83.7%%), as presented in table 1.

Statistical analyses

Statistical tests were conducted for all 92 pharmacoeconomic evaluations to assess the impact of funding types on the outcomes. As shown in table 2, studies sponsored by the industry were significantly more likely to conclude that the intervention therapy was more economical than the comparison therapy (p<0.01 for industry funding only, p<0.05 for industry-involved multiple funding). For the 87 cost-effectiveness analyses, we found that studies supported by non-profit or multiple funding sources showed a similar likelihood of achieving a dominant economic result. Industry-funded studies showed a significantly higher proportion of resulting target drugs economically dominated the comparison drug (p<0.05). The reporting quality did not differ significantly across sponsorship types. The result of multivariate regression analysis further consolidates the impact of FCOI on the conclusion of economic evaluations (p<0.01) (online supplemental table 3).

Table 2

The univariate regression analysis on impacts of financial conflict of interest on the conclusion, economic result and reporting quality of pharmacoeconomic evaluations

Comparing study outcomes by drug

By categorising the economic evaluations with the same intervention drug and comparing its outcomes, we found that a few studies have reached contradicting economic conclusions for the same study drug under different types of funding. As presented in online supplemental table 4, two pairs of studies on dapagliflozin and ceritinib using precisely the same intervention and comparator have reached contradicting conclusions under different types of funding. The other two pairs of studies on fruquintinib and apatinib with different types of funding chose different comparators for the same intervention drug under the same indication and reached contradicting conclusions. We also observed differences across studies in choosing WTP thresholds and discount rates. Detailed information can be found in online supplemental table 4.

Discussion

In this study, we discovered that studies involving industry sponsorship were significantly more likely to reach dominant economic results and concluded that intervention drugs indicate more health-economic benefits than comparison. Nonetheless, we did not identify a significant impact of funding types on the reporting quality of studies.

Our findings confirm that FCOI could affect research outcomes, which aligns with the conclusion of previous studies on the impact of FCOI on clinical trials, systematic reviews, clinical guidelines and health technology assessments.8 9 11 Further, we found that although most included evaluations of the same intervention drug reached similar economic conclusions, regardless of funding types, some evaluations have reached contradicting economic conclusions for the same study drug under different types of funding.

We found various factors that could alter the study conclusion by examining studies on the same intervention drug but reaching contradicting conclusions. First, external funding could affect the choice of comparator therapy, which is one of the most apparent factors contributing to contradicting conclusions. For example, as an active comparator tends to be more costly than a placebo, it is more likely to obtain an economical conclusion by choosing a high-cost, active-controlled drug with modest clinical benefit.25 26 61–63 Second, the simulation model is critical in pharmacoeconomic evaluation, however, the model’s design could vary greatly among researchers. Taking the model simulation of diabetes as an example, models such as the IQVIA CORE Model, the Cardiff Diabetes Model, the IHE Model and the Chinese COMT are all well-recognised models with significantly different modelling algorithms, which could lead to contradicting results.50 51 111 112 Meanwhile, how the model simulation period is set can also significantly impact the final economic outcomes and is therefore suggested as a key variable in the deterministic sensitivity analysis. Thus, the researcher’s choice of model and simulation period, which might be affected by sponsorship, could contribute to contradicting conclusions.50 51 Third, although all included studies were conducted in the Chinese setting and extracted model input data from published resources or open-access databases, the choice of input parameters was subject to researchers. For instance, the selection of health state utility values and the health utility indicators of the identical target population differed across studies. Moreover, costs, such as the price gaps between those listed in the Price Bureau of China and those listed in provincial price bureaus and that recorded by hospitals of different tiers, varied considerably.50 61 61 62 84 85 Thus, the representativeness and accuracy of the cost and utility data are open to scrutiny. Fourth, the ultimate determinant of whether an intervention is economical or cost-effective is the WTP threshold. If the intervention cost or incremental cost-effectiveness ratio (ICER) is below the WTP, the intervention under consideration is deemed cost-effective and vice versa.113 The pairs of studies presented in online supplemental table 4 allow us to observe the differing choices of WTP between studies, but we did not find a direct correlation between favourable conclusions and choice of WTP. The study that chose one times per capita gross domestic product (GDP) did not reach a more unfavourable conclusion than the study that chose three times per capita GDP. The same was true for the discount rate, where we observed that studies chose among 0–5%, but there was no direct correlation between the different choices of discount rate and a favourable conclusion. The Chinese Guideline for Pharmacoeconomic Evaluations (2020 edition) suggests setting the WTP at one to three times per capita GDP and a discount rate of 5%.114 Theoretically, a higher discount rate will result in a lower present value of the treatment and a higher WTP threshold would increase the probability of a treatment being cost-effective. However, we did not directly observe this phenomenon in the published studies, most likely because the effects of WTP and discount rate work in conjunction with other input factors on the outcome. Thus, we suggest researchers conduct scenario analysis for WTP thresholds and discount rates.

As medical costs continue to rise and put pressure on the national health insurance fund, national drug price negotiations, through which experts from the National Health Insurance Bureau negotiate drug prices with drug companies for entering NRDL, can promote accessibility and reduce the economic burden of drugs.13 14 Pharmacoeconomic evidence plays a significant role in this major policy initiative, and the current review process for negotiated drugs in China has gradually recognised the impact of subjective researcher choice on evaluation outcomes. Therefore, the National Health Security Administration has organised expert reviews on pharmacoeconomic evaluation and set strict regulations on discount rates, model adjustments and hospital payment thresholds for pharmacoeconomic modelling studies to minimise the impact of funding on researcher choice.12 115 Although the number of published pharmacoeconomic evaluations in China has increased rapidly over the past decade, yet none of these standardised, high-quality official pharmacoeconomic evaluations has been made public. With 71.3% of nationally negotiated drugs still not having relevant published economic studies as of 2022, making the results of these official pharmacoeconomic evaluation public will not only add to the current body of published evidence but will also provide a more standardised research paradigm for future studies, helping to promote higher quality pharmacoeconomic evaluations in China.

This review provides the following implications for future research. First, policies are needed to expand the evidence base for pharmacoeconomic evaluations and to strengthen the requirement for pharmacoeconomic evidence for medicine. Second, concerned institutions should develop and promote a standardised procedure for FCOI disclosure in pharmacoeconomic evaluations (eg, FCOI status of the source of input parameters). Existing disclosure forms, such as the International Committee of Medical Journal Editors disclosure form, should be required when submitting relevant studies to enhance transparency. Third, we suggest experts, while developing and updating pharmacoeconomic guidelines, make recommendations for the choice of the comparison drug and for model input data. Such guidance could assist researchers in appraising and generating rational and plausible evidence. In addition, sensitivity and scenario analyses should be required for key parameters such as the model simulation period, WTP threshold and discount rate. Besides, the comparativeness across pharmacoeconomic studies needs to be enhanced by standardising reporting using checklists such as CHEERS and establishing a comprehensive evidence base that could account for multiple economic evaluations to reduce possible FCOI bias in a single study. It should be noted that the definition of different FCOI categories varies across studies intending to serve different objectives. While some define ‘industry funding’ as studies supported purely by the industry, others may define it as studies involving industry funding or collaboration.8–11 As there is no generally accepted standard currently, the research should be aware of the inconsistent definition in comparing the results and conclusions between studies. Nevertheless, future studies should re-examine the role of negative and non-inferior outcomes in driving healthcare decisions and reinforce the importance of sensitivity analysis.

Our study has several limitations that may affect our results. First, our results are subject to the publication bias of existing studies. Studies that produce favourable results are more likely to seek publishment, and journals are inclined to publish studies with non-neutral conclusions. Second, our results are also subject to the pre-existing FCOI bias of included studies. This study only accounted for the financial aspect of conflicts of interest that were explicitly stated in the included pharmacoeconomic evaluations. However, it is common for pharmaceutical companies to fund the clinical trials of their drugs, therefore, the underlying results integrated into the input parameters of pharmacoeconomic evaluation may already suffer from sponsorship bias. Third, our analysis is based on the disclosure of FCOI in the published article. However, we cannot account for inaccurate or incomplete disclosures, especially for industry-sponsored studies that may not opt to disclose conflicts of interest. Therefore, the sponsorship bias of our study might be underestimated. Besides, as there is still no generally accepted tool for evaluating the overall quality or transparency of pharmacoeconomic evaluations, we used CHEERS to assess the study’s reporting quality. However, scores evaluated by CHEERS cannot truthfully reflect overall research quality. Future research efforts can be directly at developing a standardised tool to evaluate the overall quality of pharmacoeconomic research to allow for a more comprehensive comparison of the economics of drugs across settings. Further, we cannot ignore the possibility that funding with not-for-profit objectives has a higher inclination to support the research on controversial drugs than industry funding, which may contribute to the result that research funded by non-profit sources has a higher proportion of concluding unfavourable conclusions towards the intervention drug.

Conclusion

To conclude, this systematic review revealed that FCOI is common in published pharmacoeconomic evaluations conducted in Chinese settings and influences study findings through multiple mechanisms, as seen in that evaluations supported by the industry were more likely to draw favourable conclusions towards the intervention drug. Multifaceted efforts are needed to improve transparency, comparability, and reporting standardisation.

Data availability statement

Data are available in a public, open access repository.

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Handling editor Lei Si

  • Contributors The contributions made by the individual authors are as follows. ZH, XG, SH and LS conceptualised and designed the study. ZH, XH, DC, GW and YZ participated in the screening and data collection of the studies. ZH and XG conducted the final analyses and finalised the interpretation of the results. ZH, XG, HL and LS drafted the initial manuscript. XG is responsible for the overall content as the guarantor.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests All authors declare no competing interesets.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.