Article Text
Abstract
Background The COVID-19 pandemic required science to provide answers rapidly to combat the outbreak. Hence, the reproducibility and quality of conducting research may have been threatened, particularly regarding privacy and data protection, in varying ways around the globe. The objective was to investigate aspects of reporting informed consent and data handling as proxies for study quality conduct.
Methods A systematic scoping review was performed by searching PubMed and Embase. The search was performed on November 8th, 2020. Studies with hospitalised patients diagnosed with COVID-19 over 18 years old were eligible for inclusion. With a focus on informed consent, data were extracted on the study design, prestudy protocol registration, ethical approval, data anonymisation, data sharing and data transfer as proxies for study quality. For reasons of comparison, data regarding country income level, study location and journal impact factor were also collected.
Results 972 studies were included. 21.3% of studies reported informed consent, 42.6% reported waivers of consent, 31.4% did not report consent information and 4.7% mentioned other types of consent. Informed consent reporting was highest in clinical trials (94.6%) and lowest in retrospective cohort studies (15.0%). The reporting of consent versus no consent did not differ significantly by journal impact factor (p=0.159). 16.8% of studies reported a prestudy protocol registration or design. Ethical approval was described in 90.9% of studies. Information on anonymisation was provided in 17.0% of studies. In 257 multicentre studies, 1.2% reported on data sharing agreements, and none reported on Findable, Accessible, Interoperable and Reusable data principles. 1.2% reported on open data. Consent was most often reported in the Middle East (42.4%) and least often in North America (4.7%). Only one report originated from a low-income country.
Discussion Informed consent and aspects of data handling and sharing were under-reported in publications concerning COVID-19 and differed between countries, which strains study quality conduct when in dire need of answers.
- COVID-19
- Review
- Study design
Data availability statement
Data used can be made available on request to the corresponding author.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
The quality of COVID-19 studies has been negatively influenced by fast tracking publications and the use of non-peer-review platforms.
WHAT THIS STUDY ADDS
Informed consent and aspects of data handling for privacy, as proxies for study quality conduct, were structurally under-reported in publications concerning COVID-19.
Publications from lower-income countries were sparse, showing research equity issues between high-income and low-income countries, which could potentially create blind spots in evidence.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Transparency in reporting on informed consent and other aspects of data handling should markedly improve.
We recommend the development of a framework to advise concerns of informed consent and other ethics regulations during times of crisis and in situations with limited resources.
International and intercontinental inequalities in resources should be considered, with academic journals setting the standard to improve the reporting of study quality conduct, while taking inequalities regarding resources into account to avoid selective publication of data from high-income countries.
Introduction
The unknown nature of COVID-19 unleashed an enormous drive for research. Based on the search term ‘COVID-19’ alone, the number of 283 PubMed citations in 2019 increased to 91 634 in 2020 and 208 994 in 2021. In these publications, patient data were investigated and shared to increase the understanding of the disease to support physicians globally.1 2 However, fast-track publications and publications by non-peer-review platforms of patient data were often used and have negatively influenced study quality for COVID-19.2 3 As a result, a high risk of bias was, for example, reported in diagnostic and prognostic prediction models and other observational studies.4–6 Furthermore, inferior intervention study designs were used, and retraction of randomised controlled trial study results for COVID-19 occurred.5 7
Current reporting and research quality guidelines nevertheless aim for study quality conduct improvement, reproducibility transparency with a focus on informed consent, and guaranteeing appropriate and effective data sharing.8–12 However, it is unclear how informed consent, and aspects of data handling for privacy, as proxies for study quality conduct, were reported in publications concerning COVID-19.13
For research in general, legal rules and regulations surrounding ethics apply to guide responsible conduct in a way that contributes to research quality.14 Although legal rules might differ between regions of the world, the scientific community embraces general ethical regulations globally.13 15–17 One main facet of responsible and ethical research conduct is asking for consent. According to General Data Protection Regulation (GDPR) and Health Insurance Portability and Accountability Act (HIPAA), consent must be specific, unambiguous, freely given, and most importantly, informed, that is, the patients know what data are being processed, and the purpose of the data processing, and each patient may withdraw their consent at any time.16 However, several aspects, such as the level of comprehension or the person’s capacity to consent, varying study design requirements (ie, more obligatory reporting in intervention than observational studies), as well as health inequalities between low-income and high-income countries, for example, could influence the process of obtaining consent.18 19 Another facet of good research conduct is the application of Open Science, which aims to increase responsible (re)use of data for research. The key to achieving this is through the application of Findable, Accessible, Interoperable and Reusable (FAIR) data and applying Open Science to share open data.20 21 However, legal and ethical challenges relating to these principles can likewise be identified.22–25 Moreover, facets of informed consent and responsible data use are potentially compromised during a more urgent need for clinical answers, as was the case during the COVID-19 pandemic.18 24 26 Informed consent procedures and adherence to data-sharing principles are nevertheless a prerequisite to assure high-quality and responsible conduct of clinical studies, especially in challenging eras such as the COVID-19 pandemic.27 Although there have been publications in the past that evaluated aspects such as informed consent reporting and reporting of review board approval,27–30 it is currently unknown how these aspects were handled during the pandemic. Particularly the urging need for new information during a pandemic might have affected informed consent reporting. This evidence gap is the focus of this study.
We hypothesise that reporting on prestudy protocol registration, informed consent, data handling and sharing aspects during the COVID-19 pandemic were compromised due to the need for fast information and the shortened review procedures. Therefore, we conducted a systematic scoping review of observational studies and clinical trials during the COVID-19 pandemic to investigate the above-mentioned aspects of reporting informed consent and data handling as proxies for study quality conduct.31–38 Additionally, we compared those aspects between predefined regions of the world.
Methods
We designed a study protocol to systematically identify studies meeting the inclusion criteria. Consequently, wherever possible, this study adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting systematic reviews (online supplemental table S1).39 We could not register our review due to its design and therefore our review was not-preregistered.
Supplemental material
PubMed (NCBI) and Embase (Ovid) were systematically searched, and studies published in English that reported on SARS-CoV-2 infection and COVID-19 disease published before 8 November 2020 were identified.40 41 Search terms included the MeSH (Medical Subject Headings) terms: “Coronavirus”. The second component of the search included study designs, that is, observational studies, randomized controlled trials (RCTs), cohort studies and cross-sectional studies. These were all entered as ‘publication type’ or as a MeSH term. Furthermore, the MeSH term ‘humans’ was included. Finally, the search was designed to exclude case reports, reviews, meta-analyses and animal studies as publication types. The search in Embase included all components mentioned above but was refined with an expert search offered by the Ovid team at the time. The expert search was designed to remove publications on previous Sars-CoV-1 and Middle East respiratory syndrome-virus outbreaks and animal-related studies. The expert search also included non-indexed publications. More detailed information on the Embase and PubMed search can be found in online supplemental table S2.
After identifying and excluding duplicates, studies were screened based on title and abstract to include studies with hospitalised patients 18 years and older, and a PCR confirmed SARS-CoV-2 infection. We excluded publications in languages other than English and studies with healthcare professionals as the study subjects.
Title and abstract screening were performed in duplicate by two investigators (CWEH and NW). Reasons for study exclusion were registered, and any discrepancies between the two investigators on reasons for exclusion were resolved by discussion and mutual agreement.
After screening, detailed data on the following reported variables were extracted from full papers: prestudy protocol registration, ethical approval, informed consent type and legal data handling for privacy. For multicentre studies, we extracted reporting of data transfer and sharing agreements, whether data were FAIR and whether data were classified as open data.20
Furthermore, data on study characteristics, such as publication date and journal, sample size, study design, the country where the study was conducted, and its income level organised according to the World Banks classification of country income, were collected.42 43 More information on what data were extracted is available in table 1. These variables, while not exhaustive, were predefined and chosen as multidimensional proxies for study quality conduct. We discuss each of these aspects in table 1. Data extraction was performed independently by four investigators (CWEH, CTAV, NW and SJWMC) and was not performed in duplicate.
Patient and public involvement
The Intensive Care Unit’s patient panel of Maastricht University Medical Centre+ supports transparent reporting of health data.
Statistical analysis
Data are presented as numbers and percentages according to categories of variables under investigation, using SPSS Statistics V.28.0 (International Business Machines). Furthermore, we used R (R Foundation for Statistical Computing, V.4.1.3) to construct a world map to illustrate the sample size of studies included per country. Results for informed consent are stratified according to study design, journal impact factor and country income categories. Each country’s income was classified as high income, upper middle income, lower middle income or low income, as defined by The World Bank.44 The statistical tests were carried out for the binary variable ‘any kind of consent reported’ versus ‘no consent reported’, for reasons of power and instead of testing each separate consent kind (ie, informed consent, deferred consent, verbal consent, waiver of consent, opt-out and other type of consent vs no consent reported). The journal impact factor was categorised into three classes: 0–5, 5–10 and >10 for the purpose of illustration. The Mann-Whitney U test tested whether the continuous journal impact factor differed significantly between ‘any kind of consent reported’ versus ‘no consent reported’. χ2 test tested whether ‘any kind of consent reported’ versus ‘no consent reported’ differed significantly across study design categories. A p<0.05 was considered statistically significant.
Results
The initial search on 8 November 2020, identified 3481 publications in PubMed and 4614 publications in Embase, resulting in 6290 publications after removing duplicates. Of these, 1488 were examined in detail, and 972 were included for data extraction (figure 1). In total, these publications included international data from 618 598 individual patients (figure 2).
Only 3.8% of 972 studies were clinical trials (randomised and non-randomised clinical trials), of which 94.6% reported informed consent. Sixteen per cent were prospective cohort studies, of which 27.6% reported informed consent, 34.6% reported waiver of consent, whereas 29.5% did not report information on informed consent. 21.3% of all included studies reported informed consent, 42.6% reported waiver of consent, whereas 31.4% did not report information on informed consent (online supplemental table S3). Other forms of consent, such as deferred consent, opt-out or verbal consent, were reported in <5% of studies. Ethical approval was reported in 90.9% of studies. Eighty-three per cent of studies did not report any information on pseudoanonymisation or anonymisation of the data. Of 972 studies, 16.8% reported a prestudy protocol registration or design of the study (table 2).
The majority of studies (74.0%) described retrospective data, of which 15.0% reported informed consent, 48.0% reported waiver of consent, whereas 33.5% did not report informed consent (figure 3, online supplemental table S3). The remaining 6.2% were cross-sectional studies, of which 35.0% reported informed consent. ‘Any kind of consent reported’ versus ‘no consent reported’ was significantly different between study design categories (p=0.003). Impact factor as a continuous variable was not significantly different for ‘any kind of consent reported’ versus ‘no consent reported’ (p=0.159) (table 3).
Of the included 972 publications, 257 publications reported on multicentre data. Only 1.2% reported on data sharing or data transfer agreements, none reported on FAIR principles, and only 1.2% reported on open data (table 4). Taken together, none of the multicentre studies reported on all three.
When we organised reporting on informed consent according to predefined regions in the world (excluding regions with a minimum of 10 publications and publications with an unknown location of the study population), informed consent reporting ranged from 4.7% in North America to 42.4% in the Middle East (p<0.001) (online supplemental table S4, S5).
51.2% of studies were conducted in countries classified as high-income, 44.7% of studies were conducted in countries classified as upper-middle-income and 3.9% of studies were conducted in lower-middle-income countries (table 5). Only one of the included studies originated from a low-income country (the Democratic Republic of the Congo). Informed consent was most often mentioned in studies conducted in lower-middle-income countries. A waiver of consent was most often reported in studies from upper-middle-income countries. Verbal consent and not reporting on informed consent happened most often in studies conducted in high-income countries.
Discussion
This systematic scoping review has three main findings. First, of the 972 studies included, prestudy protocol registration was reported in only 16.8%, ethical approval in 90.1% and consent in 68.6%, with a waiver of consent being the most common. Informed consent (also when assessing all kinds of consent combined) was most often reported in clinical trials. Overall, 31.4% of studies did not report consent. Second, regarding aspects of data handling for privacy, data anonymisation was mentioned in 17.0% of publications. Other aspects, such as reporting on data sharing or transfer, or FAIR and open data, were mentioned in 1.2%, 0%, and 1.2% of multicentre publications, respectively. Taken together, none of the studies reported on all aspects, namely, prestudy protocol registration, ethical approval, informed consent, data handling for privacy, data anonymisation, data sharing and transfer, irrespective of FAIR and open data. Third, differences exist globally, suggesting that inequalities and legislation play a role in study conduct and reporting. Consent was most often reported in the Middle East (42.4%) and least often in North America (4.7%). Furthermore, only one report originated from a low-income country, suggesting that data from individuals living in these countries are not published with the same frequency as data collected from those residing in higher-income countries. This latest is consistent with current literature, which states the need to include low-income countries in research and the global pandemic response regarding preventing potential blind spots and new hotspots of a global pandemic.45 Our findings suggest that overall research conduct could be substantially improved. This is in line with previous evidence showing that study quality in COVID-19 research was compromised.46 47 Although our results can be explained by the societal need for rapid answers early in the COVID-19 pandemic, we cannot exclude that the pandemic has exposed and magnified pre-existing trends focused on the quantity of publications instead of high-quality research. Supportive of the latter hypothesis is that almost 20 years ago, it was recognised that we perhaps need fewer publications but more of superior quality.48 49
Informed consent is a legal and ethical construct used in a state-of-the-art investigation of patient data, where legal and ethical rules and regulations apply. Although legal rules vary between regions and countries, which has likely driven the differences in reporting across regions of the world, the scientific community embraces general ethical regulations globally.15–17 50 Obtaining informed consent during the COVID-19 pandemic, however, came with significant challenges. For example, study personnel was at risk of infection when contacting patients to obtain a signature for written informed consent.3 51 Also, scarce personnel resources were likely to be employed clinically instead of in research activities. Importantly, informed consent protects the patients’ autonomy, particularly regarding study risks, such as interventions under study and sharing of sensitive personal health data.18 52 Therefore, the Council for International Organizations of Medical Sciences states that obtaining informed consent should continue, even in situations of duress and other methods than written informed consent are possible.15 53 For example, verbally asking for consent with a witness present is an accepted alternative, as well as asking a legally authorised representative of the patient.51 When a representative is unavailable, asking for consent at a later point (deferred consent) could also be considered.25 This may explain why many studies did not explicitly report written informed consent during the COVID-19 pandemic. We found that a waiver of consent was most often applied for observational studies, apparently balancing individual risks versus the overall general gain of data investigation. Taken together, we feel that reporting informed consent is at least a proxy for study quality conduct, although informed consent is neither an obstacle nor a guarantee for good-quality data.54 To our knowledge, no quantitative data are available about the effect of acquiring informed consent on the quality of study results. Although the rapid growth of COVID-19 publications appeared to affect journal impact factors, we found that consent reporting did not differ according to the journal impact factor.55
Rapid data sharing during the COVID-19 pandemic has changed science. The vast need for sharing was acknowledged and can be integrated into study protocols.13 Legal rules to protect individuals’ privacy were often experienced as boundaries restraining the benefits of data sharing.13 This urges the transition to more FAIR and open data to tackle future pandemics.2 20 26 56–59 However, sharing data too rapidly, for example, relying on data analysed and reported without peer review on preprint servers, might spread misinformation.24 Nevertheless, data sharing needs to happen according to specific rules and regulations, such as GDPR in Europe and HIPAA in the USA.16 22 23 27 Data transfer or sharing agreements form a legal basis for sharing data for a predefined purpose.60 The scarce reporting on data sharing and transfer agreements hampers transparency of whether legal rules are met to protect data privacy for individuals. The aim of Open Science is transparent and accessible knowledge sharing in collaborative networks and acts of FAIR principles and open data.20 57 61 Open Science increases the reproducibility, transparency and quality of scientific results.57 62 63 On the one hand, FAIR data improve the findability and reuse of data and enhances federated data analyses, whereas storing data in public archives enables public investigation of health data.64 Indeed, it has been reported that 16.0% of clinical trials in COVID-19 indicated wanting to share their raw data and many scientific journals and national research grant providers have adopted policies requesting scientists to share their raw data by default.65 66 We, however, observe that opening data for sharing has not been widely implemented in COVID-19 research. Most likely, the complexities of ethical and legal aspects obstruct the implementation of opening data for wide sharing beyond the parties involved in data transfer and sharing agreements. This must be overcome to implement suggestions like sharing raw data by default or using trusted third parties to accommodate privacy concerns.63
The results only include data from one low-income country. In fact, this shows that the current research world makes it difficult for low-income countries to publish, and we cannot exclude that strict criteria, such as informed consent or data handling for privacy, play a role. Inadequate resources, especially high rates of COVID-19 related hospitalisations and fatalities, are conditions that may have prevented researchers in low-income countries from collecting informed consent.67 Thus, the data may not adequately reflect the systematic challenges that COVID-19 researchers in low-income countries have faced due to higher rates of severe COVID-19 cases, limiting their ability to publish research in high-impact journals. These factors contribute to ongoing equity challenges in the kind of research conducted, knowledge generated and interventions developed to address health outcomes. Future research should identify more factors limiting researchers from obtaining consent and provide a comprehensive framework for approaching crises and limited resources. Journals play a crucial role in promoting complete reporting. This study, for example, identified whether consent was simply not obtained or obtained but not reported as a general unclarity. We recommend that structural reporting on ethical aspects, such as consent, anonymisation, ethical approval and legal data handling for privacy in studies involving humans, should be done, acknowledging high-quality research. This study shows researchers, who play a crucial role, that ethics regulations are more than just rules to comply with as both relevant public questions require investigation, while individuals should be respected. Clear reporting has been shown to improve the quality of research markedly.8 11 39
This study has several strengths and limitations. First, the search was comprehensive, minimising missed publications on COVID-19 during the period of investigation. Although screening was done in duplicate, which is a strength, data extraction was limited to one person and used a standardised data extraction form. This approach was chosen to minimise the risk of missing relevant studies, while optimising accurate data extraction efficiency. Another strength was using the pandemic as a case study. However, this was also a weakness as we have no information on informed consent reporting and aspects of data handling for privacy beyond COVID-19, thus limiting the generalisability of the results. Although our review was extensive, we did not include genomic surveillance studies based on viral samples and focused on patients and their consent.45 Hence, no conclusions can be drawn from surveillance data. When looking at reporting on informed consent, it is possible that mainly resource-rich countries were selected, with informed consent serving as an additional barrier in a pandemic setting. This has led to under-reporting data from low-income countries, further increasing health inequalities. Indeed, bias due to consent has been reported.31 To tackle this issue in the future, data sharing governance initiatives in low-income parts of the world that aim for high-quality data collaboration, including cross-border consent, should be supported.50 In addition, we found that some publications did not provide sufficient information on the type of study being conducted; thus, misclassification of study types may have occurred. Another limitation is that only articles in English were included. In addition, we could not differentiate between instances where consent was not obtained at all or situations where consent was obtained but not reported, potentially leading to an underestimation of reported consent. A final limitation was that during the extraction regarding FAIR data, we searched whether the term ‘FAIR’ was mentioned, not if its separate components were presented. Hence, we could have missed studies that possibly partly complied with the FAIR principles. This might have resulted in an underestimation of our results regarding the compliance of included studies with reference to FAIR data principles.
Conclusion
Informed consent, and aspects of data handling for privacy, as proxies for study quality conduct, were structurally under-reported in publications concerning COVID-19. Furthermore, publications from lower-income countries were sparse. To move Open Science ahead to support physicians’ needs based on clinical investigation and data in general and in future pandemics specifically, transparency in reporting on informed consent and aspects of data handling should markedly improve. We recommend the development of a comprehensive framework to advise concerns of informed consent and other ethics regulations during times of crisis, such as a pandemic, and in situations with limited resources. Finally, international and intercontinental inequalities in resources likely affect the possibilities of complying with consent and data sharing. Here, academic journals should set the standards in a way to incorporate data inclusively to avoid selective reporting throughout the world, while improving the reporting of study quality conduct.
Data availability statement
Data used can be made available on request to the corresponding author.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Seye Abimbola
NW and CWEH contributed equally.
Contributors NW: performed the search. NW, CWEH, SJWMC, CTAV: study selection and data extraction. NW, CWEH: analysed the data. NW, CWEH: wrote the initial draft of the paper, revised the paper and finalised the manuscript. NW, CWEH: wrote the paper. BvB, IvdH: initiated the project, developed the idea and coordinated the writing process. NW, WECH, CTAV, SJWMC, WKAvM, JCB, LAC, NM-M, JG, CW, FB-R, LW, ICCvdH, BCTvB: wrote the paper and critically reviewed the content. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. BCTvB is responsible for the overall content as the guarantor.
Funding LAC is funded by the National Institute of Health through the NIBIB R01 grant EB01720.
Map disclaimer The inclusion of any map (including the depiction of any boundaries therein), or of any geographic or locational reference, does not imply the expression of any opinion whatsoever on the part of BMJ concerning the legal status of any country, territory, jurisdiction or area or of its authorities. Any such expression remains solely that of the relevant source and is not endorsed by BMJ. Maps are provided without any warranty of any kind, either express or implied.
Competing interests LAC is funded by the National Institute of Health through the NIBIB R01 grant EB01720.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.